An attacker intentionally introduces leading characters that enable getting the input past the filters. The API that is being targeted, ignores the leading "ghost" characters, and therefore processes the attackers' input. This occurs when the targeted API will accept input data in several syntactic forms and interpret it in the equivalent semantic way, while the filter does not take into account the full spectrum of the syntactic forms acceptable to the targeted API.
Some APIs will strip certain leading characters from a string of parameters. Perhaps these characters are considered redundant, and for this reason they are removed. Another possibility is the parser logic at the beginning of analysis is specialized in some way that causes some characters to be removed. The attacker can specify multiple types of alternative encodings at the beginning of a string as a set of probes.
One commonly used possibility involves adding ghost characters--extra characters that don't affect the validity of the request at the API layer. If the attacker has access to the API libraries being targeted, certain attack ideas can be tested directly in advance. Once alternative ghost encodings emerge through testing, the attacker can move from lab-based API testing to testing real-world service implementations.
Attack Execution Flow
Determine if the source code is available and if so, examine the filter logic.
If the source code is not available, write a small program that loops through various possible inputs to given API call and tries a variety of alternate (but equivalent) encodings of strings with leading ghost characters. Knowledge of frameworks and libraries used and what filters they apply will help to make this search more structured.
Observe the effects. See if the probes are getting past the filters. Identify a string that is semantically equivalent to that which an attacker wants to pass to the targeted API, but syntactically structured in a way as to get past the input filter. That encoding will contain certain ghost characters that will help it get past the filters. These ghost characters will be ignored by the targeted API.
Once the "winning" alternate encoding using (typically leading) ghost characters is identified, an attacker can launch the attacks against the targeted API (e.g. directory traversal attack, arbitrary shell command execution, corruption of files)
The targeted API must ignore the leading ghost characters that are used to get past the filters for the semantics to be the same.
Typical Likelihood of Exploit
Methods of Attack
Alternate Encoding with Ghost Characters in FTP and Web Servers
Some web and FTP servers fail to detect prohibited upward directory traversals if the user-supplied pathname contains extra characters such as an extra leading dot. For example, a program that will disallow access to the pathname "../test.txt" may erroneously allow access to that file if the pathname is specified as ".../test.txt". This attack succeeds because 1) the input validation logic fails to detect the triple-dot as a directory traversal attempt (since it isn't dot-dot), 2) some part of the input processing decided to strip off the "extra" dot, leaving the dot-dot behind.
Using the file system API as the target, the following strings are all equivalent to many programs:
As you can see, there are many ways to make a semantically equivalent request. All these strings ultimately result in a request for the file ../test.txt.
Attacker Skills or Knowledge Required
Skill or Knowledge Level: Medium
Solutions and Mitigations
Perform white list rather than black list input validation.
Canonicalize all data prior to validation.
Take an iterative approach to input validation (defense in depth).
Gain privileges / assume identity
Modify application data
Web Form, URL, Network Socket, File
The payload is the parameter that an attacker is supplying to the targeted API that will allow the attacker to elevate privilege and subvert the authorization service.
The targeted API is the activation zone. These attacks often target the file system or the shell to execute commands.
Payload Activation Impact
Failure in authorization service may lead to compromises in data confidentiality and integrity.