SPECIFYING TESTING AND EVALUATING C3I SYSTEMS THAT EMPLOY ARTIFICIAL INTELLIGENCE

THE OBJECTIVE OF THIS PROPOSED RESEARCH IS TO DEVELOP NEW METHODS OF TESTING AND EVALUATION THAT ARE APPROPRIATE FOR C3I SYSTEMS THAT EMPLOY ARTIFICIAL INTELLIGENCE (AI). THE APPROACH WILL BE TO DEVELOP A MULTIATTRIBUTE UTILITY (MAU) ANALYSIS OF PERFORMANCE THAT FULLY AND MEANINGFULLY CHARACTERIZES THE PERFORMANCE OF THE SYSTEM AND LINKS THIS CHARACTERIZATION TO OBJECTIVE, TESTABLE MEASURES. PHASE I WILL INVOLVE FOUR TASKS. IN TASK 1, WE WILL REVIEW CURRENT AND LIKELY FUTURE APPLICATIONS OF AI IN MILITARY C3I SYSTEMS, AND CHARACTERIZE THOSE APPLICATIONS IN A MANNER THAT WILL FACILITATE THE DEVELOPMENT OF TEST AND EVALUATION MEASURES, AND SPECIFY AN MAU ANALYSIS. THE MAU ANALYSIS WILL INCLUDE A HIERARCHY OF ATTRIBUTES, VALUE SCALES, AND TRADEOFF WEIGHTS. IN TASK 3, WE WILL DESIGN A TEST AND EVALUATION FOR ONE SELECTED SYSTEM. IN TASK 4, WE WILL INVESTIGATE THE FEASIBILITY OF GENERALIZING THE APPROACH AND DEVELOPING A HANDBOOK OR COMPUTERIZED DECISION AID. IN PHASE 2, WE WILL CONDUCT TESTS AND DEVELOP A HANDBOOK OR COMPUTERIZED AID.