Enforcing C++ type integrity with fast dynamic casting, member function protections and an exploration of C++ beneath the surface

The C++ type system provides a programmer with modular class features and inheritance capabilities. Upholding the integrity of all class types, known as type-safety, is paramount in preventing type vulnerabilities and exploitation. However, type confusion vulnerabilities are all too common in C++ programs. The lack of low-level type-awareness creates an environment where advanced exploits, like counterfeit object-orientated programming (COOP), can flourish. Although type confusion and COOP exist in different research fields, they both take advantage of inadequate enforcement of type-safety. Most type confusion defence research has focused on type inclusion testing, with varying degrees of coverage and performance overheads. COOP defences, on the other hand, have predominantly featured control flow integrity (CFI) defence measures, which until very recently, were thought to be sound. We investigate both of these topics and challenge prevailing wisdom, arguing that: 1. optimised dynamic casting is better suited to preventing type confusion and 2. enforcing type integrity may be the only defence against COOP.

Type confusion vulnerabilities are often the result of substituting dynamic casting with an inappropriate static casting method. Dynamic casting is often avoided due to memory consumption and run-time overheads, with some developers turning off run-time type information (RTTI) altogether. However, without RTTI, developers lose not only secure casting but virtual inheritance as well. We argue that improving the performance of dynamic casting can make it a viable option for preventing type confusion vulnerabilities. In this thesis, we present MemCast, a memoising wrapper for the dynamic cast operator that increases its speed to that of a dynamic dispatch.

A new variant of the COOP exploit (COOPLUS) has identified a weakness in almost all modern, C++-semantic-aware CFI defences. The weakness is that they allow derived class functions to be invoked using corrupted base class instances, specifically where an attacker replaces the object's virtual pointer with one from a derived type object. A CFI defence overestimates the set of target functions at a dispatch site to cover all possible control-flow paths of a polymorphic object. Thus COOPLUS takes advantage of the lack of type integrity between related types at dispatch sites. In this thesis, we argue that CFI is an unsuitable defence against COOPLUS, and type integrity must be applied. Hence we propose a type integrity defence called Member Function Integrity (MFI) that brings type awareness to member functions and prevents any member function from operating on an invalid object type.

To understand the low-level techniques deployed in MemCast and our MFI defence policy, one has to appreciate the memory layout of the objects themselves and the conventions used by member functions that operate on them. However, in our research, we did not find adequate introductory literature specific to modern compilers. For this reason, we supplied our own self-contained introduction to low-level object-orientation.

This thesis has three contributions: a primer on C++ object layouts, an optimised dynamic casting technique that reduces the casting cost to that of a dynamic dispatch, and a new defence policy proposal (MFI) to mitigate all known COOP exploits.