Program Semantics-Aware Intrusion Detection and Prevention

One of the most dangerous cybersecurity threats is ``control hijacking'' attacks, which hijack the control of a victim application, and execute damaging system calls by assuming the identity of the victim process's effective user. These types of attacks are highly hazardous because commercial applications with such vulnerabilities appear to be wide spread, as shown in the rampancy of recent worms such as the SQL Slammer. System call monitoring has been touted as an effective solution to ``control hijacking'' attacks because it could prevent remote attackers from inflicting damage upon a victim system even if they can successfully compromise applications running on the system. However, the Achilles' heel of the system call monitoring approach is how to construct accurate security policy that could minimize false positives and negatives. Although various approaches have been tried to solve this problem, none of them is satisfactory. This project proposes a Program semantics-Aware Intrusion Detection system called PAID, which derives a security policy from an application's source code, and checks the application's system calls against the resulting policy at run time. Not only can PAID derive these policies completely automatically, but also the resulting security policy is tailored to individual applications and thus is highly accurate. Host-based intrusion detection based on system call monitoring is a well-known technique to provide the last line of defense against remote attacks. However, existing commercial or research host-based intrusion detection systems are plagued with the lack of a systematic approach to derive an accurate security policy for a given system. As a consequence, the security policy actually deployed in real systems tend to err on the conservative side and eventually generate too many false positives to render the system ineffective. The proposed PAID system effectively solves this problem by developing a tool that can automatically derive accurate security policies corresponding to the system call patterns of individual applications, thus reducing both false positives and negatives to the minimum. PAID therefore represents a giant step in closing the gap between current host-based intrusion detection systems and the actual needs of real-world IT systems.