Privacy Assured Linkage and Analytics over Datasets from Isolated Neighborhoods (PALADIN)

The US Air Force has the need to predict proficiency, outcomes, and study potential optimizations of the impact of proposed training for USAF personnel, teams, and individual pilots.  Such assessments rely on data provided by a variety of facilities and service branches, and at diverse classification and compliance levels.  While techniques do exist for linking and aggregating data for analysis, current approaches either unnecessarily reveal sensitive data, or directly harm the utility of analyses creating additional uncertainty. For example, the current gold standard used by many federal agencies is data de-identification, which offers only limited privacy guarantees and has consistently failed to provide privacy protection when evaluated in the scientific literature.  As another example, synthetic data approaches, which substitute representative but artificial data for sensitive data, require costly preparation: each data set owner must privately learn all necessary statistical relationships in the original data, and adequately replicate them in the synthetic data set. We propose as an alternative approach Privacy Assured Linkage and Analytics over Datasets from Isolated Neighborhoods (PALADIN). PALADIN cryptographically protects sensitive data while retaining its full utility and requires no de-identification or data synthesis.  PALADIN will prototype, demonstrate, and evaluate a number of methods designed to securely access and analyze diverse training data across partitions, including capabilities to: 1) query, identify, navigate to, and link data across partitions using rich search semantics while preventing the exposure of data to unauthorized users; 2) characterize limitations or uncertainties which arise from limited access to only a given subset of partitions containing usable data; 3) recommend partition changes to minimize limitations and uncertainties, while maximizing utility when in limited partition access scenarios; and 4) provide a cryptographically mature basis for analytical methods involving Bayesian statistical methods in multi-party computation settings, providing privacy guarantees while maximizing the utility of analytics and proficiency prediction.