A Verification and Validation Tool for Diagnostic Systems

Advanced diagnostic systems have the potential to improve safety, increase availability, and reduce maintenance costs in aerospace vehicle and a variety of other mechanical system. Numerous recent research efforts have produced a variety of diagnostic algorithms that show significant promise, but to date advanced diagnostic approaches have seen rather limited use in operational air vehicle systems. One of the major hurdles to transitioning such systems to fleet vehicles is the lack of adequate verification and validation (V&V) approaches. Barron Associates and MUSYN propose a Phase I research effort to develop a V&V framework for diagnostic systems that combines novel analysis approaches with experimental techniques to provide high confidence in the performance of diagnostic techniques. Performance evaluation of diagnostic systems is currently based primarily on numerical testing approaches, which may be applied to both simulation results and actual experimental data. While such testing is extremely important and should form a key component of the overall V&V strategy, it is not adequate alone. This is because it is impossible to collect sufficient test data or even sufficient Monte Carlo simulation data to exhaustively cover the space of potential test conditions. To achieve reasonable confidence in the coverage of the V&V procedures, it is necessary to intelligently select Monte Carlo or experimental test points to target the regions of the test space that are most likely to reveal problems. The team will work to develop analysis approaches that can help to identify combinations of conditions (flight conditions, uncertainties, external disturbances, vehicle configuration, etc.) that are most likely to lead to inadequate performance of diagnostic algorithms. The team will also extend the existing CAESAR software tool for control law V&V to automate V&V of diagnostic systems.