Continuous Verification & Validation in Chaos Engineering

DAF's critical IT infrastructure needs to be reliable, even in the face of partial system failures and infrastructure attacks by state actors. Kessel Run (KR) is developing these capabilities using Chaos Engineering, an industry term for testing system resilience via fault injection, sometimes in a production setting. However, Chaos Engineering is a young discipline, and teams which practice it typically use custom, bespoke tooling to implement the process as they explore it. Auxon's Modality product provides an end-to-end solution to system-level validation and verification of Cyber-Physical Systems (CPS). It was designed around the principles of Chaos Engineering, but as applied to CPS rather than IT systems. We propose adapting Modality to be used for IT systems, providing a rich Chaos Engineering toolchain that is not otherwise available in this context. Modality was initially designed as a design and verification tool for complex, safety critical cyber-physical systems, which typically consist of networks of embedded devices and sensors. We did so using techniques commonly used to design and analyze reliable distributed software systems, such as telecommunications and storage platforms. The overall design and approach is thus very well suited for use with software systems, which are typically used in IT and cloud environments.