Common Attack Pattern Enumeration and Classification

Common Attack Pattern Enumeration and Classification

80

High

The application's UTF-8 decoder accepts and interprets illegal UTF-8 characters or non-shortest format of UTF-8 encoding.

Input filtering and validating is not done properly leaving the door open to harmful characters for the target host.

High

• Injection
• Protocol Manipulation
• API Abuse

Low - an attacker can inject different representation of a filtered character in UTF-8 format. Medium - an attacker may craft subtle encoding of input data by using the knowledge that she has gathered about the target host.

Attacker may try to inject dangerous characters using UTF-8 different representation using (example of invalid UTF-8 characters). The attacker hopes that the targeted system does poor input filtering for all the different possible representations of the malicious characters. Malicious inputs can be sent through an HTML form or directly encoded in the URL.

The attacker can use scripts or automated tools to probe for poor input filtering.

A web page that contains overly long UTF-8 codes constitute a protocol anomaly, and could be an indication that an attacker is attempting to exploit a vulnerability on the target host.

A attacker can use a fuzzer in order to probe for a UTF-8 encoding vulnerability. The fuzzer should generate suspicious network activity noticeable by an intrusion detection system.

An IDS filtering network traffic may be able to detect illegal UTF-8 characters.

According to OWASP, sometimes cross-site scripting attackers attempt to hide their attacks in Unicode encoding.

The Unicode Consortium recognized multiple representations to be a problem and has revised the Unicode Standard to make multiple representations of the same code point with UTF-8 illegal. The UTF-8 Corrigendum lists the newly restricted UTF-8 range (See references). Many current applications may not have been revised to follow this rule. Verify that your application conform to the latest UTF-8 encoding specification. Pay extra attention to the filtering of illegal characters.

The exact response required from an UTF-8 decoder on invalid input is not uniformly defined by the standards. In general, there are several ways a UTF-8 decoder might behave in the event of an invalid byte sequence: 1. Insert a replacement character (e.g. '?', '�'). 2. Ignore the bytes. 3. Interpret the bytes according to a different character encoding (often the ISO-8859-1 character map). 4. Not notice and decode as if the bytes were some similar bit of UTF-8. 5. Stop decoding and report an error (possibly giving the caller the option to continue). It is possible for a decoder to behave in different ways for different types of invalid input. RFC 3629 only requires that UTF-8 decoders must not decode "overlong sequences" (where a character is encoded in more bytes than needed but still adheres to the forms above). The Unicode Standard requires a Unicode-compliant decoder to "…treat any ill-formed code unit sequence as an error condition. This guarantees that it will neither interpret nor emit an ill-formed code unit sequence." Overlong forms are one of the most troublesome types of UTF-8 data. The current RFC says they must not be decoded but older specifications for UTF-8 only gave a warning and many simpler decoders will happily decode them. Overlong forms have been used to bypass security validations in high profile products including Microsoft's IIS web server. Therefore, great care must be taken to avoid security issues if validation is performed before conversion from UTF-8, and it is generally much simpler to handle overlong forms before any input validation is done. To maintain security in the case of invalid input, there are two options. The first is to decode the UTF-8 before doing any input validation checks. The second is to use a decoder that, in the event of invalid input, returns either an error or text that the application considers to be harmless. Another possibility is to avoid conversion out of UTF-8 altogether but this relies on any other software that the data is passed to safely handling the invalid data. Another consideration is error recovery. To guarantee correct recovery after corrupt or lost bytes, decoders must be able to recognize the difference between lead and trail bytes, rather than just assuming that bytes will be of the type allowed in their position.

For security reasons, a UTF-8 decoder must not accept UTF-8 sequences that are longer than necessary to encode a character. If you use a parser to decode the UTF-8 encoding, make sure that parser filter the invalid UTF-8 characters (invalid forms or overlong forms).

Look for overlong UTF-8 sequences starting with malicious pattern. You can also use a UTF-8 decoder stress test to test your UTF-8 parser (See Markus Kuhn's UTF-8 and Unicode FAQ in reference section)

Assume all input is malicious. Create a white list that defines all valid input to the software system based on the requirements specifications. Input that does not match against the white list should not be permitted to enter into the system. Test your decoding process against malicious input.

• Privilege Escalation
• Run Arbitrary Code
• Data Modification
• Denial of Service

Bruce Schneier was one of the first to raise the security issues with Unicode in the July 15, 2000 issue of Crypto-Gram newsletter. He pointed out that with the Unicode character set, it is possible that there could be multiple representations of a single character. In a security context it is primordial to determine the meaning of a character. According to the RFC 3629, a particularly subtle form of this attack can be carried out against  a parser which performs security-critical validity checks against the  UTF-8 encoded form of its input, but interprets certain illegal octet sequences as characters. For example, a parser might prohibit the  NUL character when encoded as the single-octet sequence 00, but  erroneously allow the illegal two-octet sequence C0 80 and interpret  it as a NUL character.  Another way that Unicode can cause problems is that the application or operation system can assign the same interpretation to different code points. Thus, even though the Unicode specification dictates that the code points should be treated differently, the application actually treats them the same.

Buffer overflow

Another security issue occurs when encoding to UTF-8: the ISO/IEC 10646 description of UTF-8 allows encoding character numbers up to  U+7FFFFFFF, yielding sequences of up to 6 bytes.  There is therefore a risk of buffer overflow if the range of character numbers is not explicitly limited to U+10FFFF or if buffer sizing doesn't take into  account the possibility of 5- and 6-byte sequences. Five and six bytes long UTF-8 characters are now illegal.

The injection vector is an illegal sequences of bytes matching an UTF-8 characters or a "non-shortest form" in UTF-8 encoding format.

The interpretation of malicious characters can cause unexpected responses from the target host.

The request or command interpreter is responsible for interpreting the request sent by the client.

The malicious characters can defeat the data filtering mechanism and have many different outcomes such as path manipulation, remote code execution, etc.

• Reluctance to Trust

• RFC 3629 - http://www.faqs.org/rfcs/rfc3629.html

Penetration

G. Hoglund and G. McGraw. Exploiting Software: How to Break Code. Addison-Wesley, February 2004.

CWE – Input Validation

David Wheeler - http://www.dwheeler.com/secure-programs/Secure-Programs-HOWTO/character-encoding.html

Michael Howard and David LeBlanc - Writing Secure Code, chap12, Microsoft Press

Bruce Schneier - Crypto-Gram Newsletter, July 15, 2000 - http://www.schneier.com/crypto-gram-0007.html

WikiPedia page about UTF-8, http://en.wikipedia.org/wiki/UTF-8

RFC 3629 - http://www.faqs.org/rfcs/rfc3629.html

IDS Evasion with Unicode, by Eric Hacker, Jan. 3, 2001 - http://www.securityfocus.com/infocus/1232

Corrigendum #1: UTF-8 Shortest Form - http://www.unicode.org/versions/corrigendum1.html

UTF-8 and Unicode FAQ for Unix/Linux, by Markus Kuhn - http://www.cl.cam.ac.uk/~mgk25/unicode.html

UTF-8 decoder capability and stress test, by Markus Kuhn - http://www.cl.cam.ac.uk/%7Emgk25/ucs/examples/UTF-8-test.txt