Storage Efficient Data Mining for High-Speed Data Streams

The Sonalysts Team's goal is to provide a cross-disciplinary, intelligent, efficient stream data mining and intelligent storage system that is highly portable and configurable for the plethora of systems that need to ingest huge data streams and decidewhat, when, where, and how to store data. Leveraging current algorithms, we will apply supervised and unsupervised learning approaches to streaming data. We will investigate methods for developing rules for prioritizing data and storage resources underchanging storage constraints as well as purging, backup, and offloading technologies that allow valuable storage capacity to be re-used to store higher priority data based on clever adaptive algorithms. We will demonstrate our innovative data mining andintelligent storage technologies by applying them to Sonalysts' commercial wXstationr software. Best known for its ability to decode and display overlapping information such as international weather satellite imagery, wXstation ingests approximately 20gigabytes of streaming data per day from various sources and in various formats, in support of weather and air space analysis. This rich source of streaming data, and the applications that ingest and disseminate it, will serve as our test bed fordeveloping new and efficient data mining and intelligent data storage techniques in constrained storage environments. Viable stream mining and intelligent storage products applied to tomorrows info-sphere will have broad implications for how the U.S. AirForce and other armed services glean useful information and store priority data from large data streams. Applications of such technologies could revolutionize the armed services ability to process and store intelligence, equipment, personnel, maintenance,and other operational data. Commercial customers above and beyond those in the data mining and warehousing industries that would benefit from the technology resulting from this STTR include Sonalysts' current wXstation customers in the airline industry,and a vast number of Internet site customers that purchase weather images/objects to be displayed on their Internet sites. The security, retail marketing, news, healthcare, and insurance industries that ingest large quantities of data could also benefitfrom a rule-based, context-sensitive, intelligent summarization, backup, purging, and storage, decision-making technology.