Neural Collapse for Responsible Artificial Intelligence in Directed Energy

Neural-network-based models are extremely powerful and,particularly with the rise of large language models, have become a part of everyday life.  However, it remains difficult to justify their use in mission- or safety-critical settings because they are often “black-boxes” whose decisions cannot be explained to a user.  While explainable AI approaches exist and have helped to “open” the black-box, the heterogeneity in network architectures, training approaches, etc. have made making a general statement or approach nearly impossible; what works in one place may not work in another.   Instead, we focus on networks exhibiting neural collapse (NC) which can be characterized by the properties of the penultimatelayer features [1].  The presence of NC has implications for a network’s performance, transferability, and robustness, and the long-term objectives of our proposed effort are to (i) understand if it also impacts the interpretability the network and use this to(ii) design explainable and simpler networks that remain capable of exhibiting NC.