Heap sprays are a new buffer overflow attack (BOA) form that can significantly increase the successful chance of a BOA even though the attacked process is protected by a lot of state-of-the-art anti-BOA mechanisms, such as ASLR, non-executable stack/DEP, signature-based IDSes, and type-safe languages. In this paper, we propose a glibc-and-ASLR-based solution to heap sprays Heap Spray Protector (HSP). HSP controls the number and location of int 80 instructions in a process and hides the whereabouts of the only legal int 80 instruction; hence, HSP makes it difficult for attackers to issue a system call, let alone a heap spray attack. Moreover HSP can help ASLR defend against memory information leaking attacks. Furthermore, because HSP only modifies the glibc library and the kernel, it does not need to modify any source code or executable file. Finally, HSP allows attackers to execute as much code as possible before an attack can really create some damage; therefore, it enables the attacked hosts to collect more information about attackers which may be useful to block future attacks. Experimental results show HSP implemented on a Linux platform can effectively defend a system against heap sprays with less than 4.56% performance overhead. (C) 2010 Elsevier Inc. All rights reserved.
