Entity Resolution: A Powerful Grasp of the Nonobvious

Entity resolution can help ferret out fraud by identifying hidden links and relationships in your databases.

By Simson Garfinkel

August 01, 2006 — CSO —

Back when I used to sell security software for a living, one of the things that drove me crazy was the U.S. Treasury Department's Office of Foreign Assets Control and its Specially Designated Nationals List.

The OFAC SDN is a list of foreign individuals and organizations with which it is illegal for U.S. companies and individuals to do business. Say a salesman at my company got a phone call from the Army of the Republic of Ilirida. Instead of taking a credit card number and sending the army its software, we were supposed to call the OFAC hotline and file a report. That's because ARI is actually a little-known terrorist group in the Balkins.

All U.S. persons must comply with the OFAC regulations. In practice, this means that any person or organization that does business overseas is supposed to download a copy of the SDN on a regular basis and use it to cross-check new customers and business relations. While it's possible to manually search the SDN for every overseas transaction, many businesses now use software to automatically scan their transaction stream and compare it against the U.S. government's blacklist

But scanning the SDN is really just the beginning of what entity resolution systems can offer. Using systems that are now on the market, it's possible to scour your existing databases to detect potential fraud and insider abuse. In the future, these techniques might even be used for performing large-scale medical or social research.

Complying with blacklists like the SDN can be difficult because the data quality is often quite poor. While the SDN contains roughly 3,200 names of individuals, many of them appear multiple times with variant spellings. This may be because U.S. intelligence has been poor, or because a person has used several spellings in an effort to hide his identity. Some people are identified by name, city of residence, date of birth and passport number; others appear on the blacklist as just a name, with no additional identifying information. One such name is Foday Sankoh, the former leader of the rebel groups in Sierra Leone. Presumably, if someone named Foday Sankoh tried to buy a copy of my company's software, we would have to refuse the sale. On the other hand, Mr. Sankoh is listed on the SDN as being deceased (although no date is given). So perhaps we could have allowed the sale, assuming that Mr. Sankoh could prove that he was actually alive—and thus, presumably, a different Foday Sankoh than the deceased person on the government's list.

• Print