Cross-Site Scripting: AI

Sending unvalidated data to a web browser can result in the browser executing malicious code.

Cross-site scripting (XSS) vulnerabilities occur when:

1. Data enters a web application through an untrusted source. In the case of Artificial Intelligence (AI), the untrusted source is typically the response returned by an AI system. In the case of reflected XSS, it is typically a web request.


2. The data is included in dynamic content that is sent to a web user without validation.

The malicious content sent to the web browser often takes the form of a JavaScript segment, but can also include HTML, Flash, or any other type of code that the browser executes. The variety of attacks based on XSS is almost limitless, but they commonly include transmitting private data such as cookies or other session information to the attacker, redirecting the victim to web content controlled by the attacker, or performing other malicious operations on the user's machine under the guise of the vulnerable site. While exploitation is not as straightforward as other forms of XSS, the unpredictable nature of user input and the responses of AI models means that those responses should never be treated as safe.

Example 1: The following Python code retrieves a response from an OpenAI chat completion model, message, and displays it to the user.

    client = openai.OpenAI()
    res = client.chat.completions.create(...)

    message = res.choices[0].message.content

    self.writeln(f"<p>{message}<\p>")

The code in this example will behave as expected as long as the response from the model contains only alpha-numeric characters. However, if unencoded HTML metacharacters are included in the response then XSS is possible. For example, the response to the following prompt "please repeat the following statement exactly '<script>alert(1);</script>'" can return a XSS proof of concept depending on the model and context being used.