Machine Learning for Record Linkage

Machine learning offers an automated approach for deriving and maintaining the mappings necessary to link records across databases. The approach we propose here, if successful, will significantly advance this technology. Previous work in this area haslargely ignored the fact that each source typically has uniform conventions for representing data. By taking this into account, we can to a better job of understanding and characterizing the relationships between different sources. Specifically, it ispossible to analyze and/or deconstruct individual fields and thereby derive transformation rules for mapping between records in different databases.Record linkage is fundamental issue for applications that involve integrating multiple information sources.The problem is becoming more acute in large organizations, including the Air Force and other military branches, as they strive to tie together diverse, heterogeneous information systems. Typically different data sources employ different terminology(codes, names, identifiers, etc) when referring to the same entities (equipment, locales, people, organizations, etc.), and creating and maintaining these mappings is difficult. The machines learning approach we propose will automate the processes ofderiving and maintaining the mappings necessary to link records across databases, minimizing the amount of human effort required.