Buffer Overflow: Signed Comparison

The program uses a signed comparison to check a value that is later treated as unsigned. This could lead the program to write outside the bounds of allocated memory, which could corrupt data, crash the program, or lead to the execution of malicious code.

Buffer overflow is probably the best known form of software security vulnerability. Most software developers know what a buffer overflow vulnerability is, but buffer overflow attacks against both legacy and newly-developed applications are still quite common. Part of the problem is due to the wide variety of ways buffer overflows can occur, and part is due to the error-prone techniques often used to prevent them.

In a classic buffer overflow exploit, the attacker sends data to a program, which it stores in an undersized stack buffer. The result is that information on the call stack is overwritten, including the function's return pointer. The data sets the value of the return pointer so that when the function returns, it transfers control to malicious code contained in the attacker's data.

Although this type of stack buffer overflow is still common on some platforms and in some development communities, there are a variety of other types of buffer overflow, including heap buffer overflows and off-by-one errors among others. There are a number of excellent books that provide detailed information on how buffer overflow attacks work, including Building Secure Software [1], Writing Secure Code [2], and The Shellcoder's Handbook [3].

At the code level, buffer overflow vulnerabilities usually involve the violation of a programmer's assumptions. Many memory manipulation functions in C and C++ do not perform bounds checking and can easily exceed the allocated bounds of the buffers they operate upon. Even bounded functions, such as strncpy(), can cause vulnerabilities when used incorrectly. The combination of memory manipulation and mistaken assumptions about the size or makeup of a piece of data is the root cause of most buffer overflows.

Example 1: The following code attempts to prevent an off-by-one buffer overflow by checking that the untrusted value read from getInputLength() is less than the size of the destination buffer output. However, because the comparison between len and MAX is signed, if len is negative, it will be become a very large positive number when it is converted to an unsigned argument to memcpy().

void TypeConvert() {
   char input[MAX];
   char output[MAX];

   fillBuffer(input);
   int len = getInputLength();

   if (len <= MAX) {
      memcpy(output, input, len);
   }
   ...
}