Run-time Verification, Validation, and Debugging of Cognitive Systems

We propose to design a system for run-time verification and validation (V&V), and debugging, of cognitive systems. Cognitive systems have unique characteristics that make them challenging to verify, validate, and debug: they make intelligent decisions based on complex and often incomplete data, they learn from experience, and their functionality is encoded in data. These characteristics require run-time verification and validation. We will develop a prototype tool for run-time verification and validation of a cognitive system using a best-of-breed combination of existing techniques and new innovations for specifying and checking intended behavior. In the event that run-time V&V detects a violation, the characteristics of cognitive systems also require innovative debugging tools. Since a cognitive system makes intelligent decisions based on learned knowledge, it will be extremely difficult to reproduce the circumstances that lead to a fault. This will require first-fault debugging in which a running system records sufficient information to enable back-tracking a fault to the cause. Because functionality is encoded as data, the debugger will extract a data-driven trace showing the data interactions that lead up to a fault.