Menu Home

Making Smaller Haystacks

There are always more matches than we want – and experience teaches us that the ratio is not close (1000 to 1, according to BNP Paribas’ NY branch, using an exact matching methodology). We’re searching through an awful lot of hay to find needles that may or may not be hidden in there. All that reviewing our haystacks of data is awfully costly.

So, we need a way to reduce the number of matches we review. Operations staff will tell you that it’s patently obvious, in the overwhelming majority of cases, that the data matched is not the same as the listed data. These patterns of false positive matches tend to repeat regularly, the frequency determined by the nature of the business and data.

For example, wholesale businesses tend to have a more concentrated set of matches, since there are a smaller set of counterparties (some of which are used by multiple businesses due to the generic nature of the goods and services they provide) than for retail businesses. Similarly, transactional data tends to have more repetitive name data, while static data may have more recurring matches to cities and countries. A wholesale payments business will have more concentrated data than check disbursements, due to the relative frequency of the transactions. Etc., Etc.

So, we have a number of basic strategies, which we’ll discuss over a series of posts. These are:

  1. Finding patterns in our data that we trust
  2. Accepting limited amounts of additional risk in specialized circumstances
  3. Using secondary criteria to further assess the likelihood of a match

Some of these have a number of flavors to them so we may cover a topic over multiple posts, for brevity’s sake.

Categories: Matching Technologies Technology


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: