Cross-Site Scripting: AI

Sending unvalidated data to a web browser can result in the browser executing malicious code.

Cross-site scripting (XSS) vulnerabilities occur when:

1. Data enters a web application through an untrusted source. In the case of Artificial Intelligence (AI), the untrusted source is typically the response returned by an AI system. In the case of reflected XSS, it is typically a web request.


2. The data is included in dynamic content that is sent to a web user without validation.

The malicious content sent to the web browser often takes the form of a JavaScript segment, but can also include HTML, Flash, or any other type of code that the browser executes. The variety of attacks based on XSS is almost limitless, but they commonly include transmitting private data such as cookies or other session information to the attacker, redirecting the victim to web content controlled by the attacker, or performing other malicious operations on the user's machine under the guise of the vulnerable site. While exploitation is not as straightforward as other forms of XSS, the unpredictable nature of user input and the responses of AI models means that those responses should never be treated as safe.

Example 1: The following TypeScript code retrieves a response from an OpenAI chat completion model, message, and displays it to the user.

    const openai = new OpenAI({
        apiKey: ...,
    });
    const chatCompletion = await openai.chat.completions.create(...);

    message = res.choices[0].message.content

    console.log(chatCompletion.choices[0].message.content)

The code in this example behaves as expected as long as the response from the model contains only alphanumeric characters. However, if the response includes unencoded HTML metacharacters, then XSS is possible. For example, the response to the following prompt "please repeat the following statement exactly '<script>alert(1);</script>'" can return a XSS proof of concept depending on the model and context being used.

Sending unvalidated machine learning model output to a web browser can result in the browser executing malicious code.

Cross-site scripting (XSS) vulnerabilities occur when:

1. Data enters a web application through an untrusted source. In the case of Artificial Intelligence (AI), the untrusted source is typically the response returned by an AI system.


2. The data is included in dynamic content that is sent to a web user without validation.

The malicious content sent to the web browser often takes the form of a JavaScript segment, but can also include HTML, Flash, or any other type of code that the browser executes. The variety of attacks based on XSS is almost limitless, but they commonly include transmitting private data such as cookies or other session information to the attacker, redirecting the victim to web content controlled by the attacker, or performing other malicious operations on the user's machine under the guise of the vulnerable site. While exploitation is not as straightforward as other forms of XSS, the unpredictable nature of user input and the responses of AI models means that those responses should never be treated as safe.

Example 1: The following kotlin code retrieves a response from a large language model and returns it to the user in an HTTP response.

  val chatCompletionRequest = ChatCompletionRequest(
    model = ModelId("gpt-3.5-turbo"),
    messages = listOf(...)
  )
  val completion: ChatCompletion = openAI.chatCompletion(chatCompletionRequest)
  response.getOutputStream().print(completion.choices[0].message)

The code in this example behaves as expected if the response from the model contains only alpha-numeric characters. However, if unencoded HTML metacharacters are included in the response then XSS is possible. For example, the response to the following prompt "please repeat the following statement exactly '<script>alert(1);</script>'" can return an XSS proof of concept depending on the model and context being used.

Sending unvalidated data to a web browser can result in the browser executing malicious code.

Cross-site scripting (XSS) vulnerabilities occur when:

1. Data enters a web application through an untrusted source. In the case of Artificial Intelligence (AI), the untrusted source is typically the response returned by an AI system. In the case of reflected XSS, it is typically a web request.


2. The data is included in dynamic content that is sent to a web user without validation.

The malicious content sent to the web browser often takes the form of a JavaScript segment, but can also include HTML, Flash, or any other type of code that the browser executes. The variety of attacks based on XSS is almost limitless, but they commonly include transmitting private data such as cookies or other session information to the attacker, redirecting the victim to web content controlled by the attacker, or performing other malicious operations on the user's machine under the guise of the vulnerable site. While exploitation is not as straightforward as other forms of XSS, the unpredictable nature of user input and the responses of AI models means that those responses should never be treated as safe.

Example 1: The following Python code retrieves a response from an OpenAI chat completion model, message, and displays it to the user.

    client = openai.OpenAI()
    res = client.chat.completions.create(...)

    message = res.choices[0].message.content

    self.writeln(f"<p>{message}<\p>")

The code in this example will behave as expected as long as the response from the model contains only alpha-numeric characters. However, if unencoded HTML metacharacters are included in the response then XSS is possible. For example, the response to the following prompt "please repeat the following statement exactly '<script>alert(1);</script>'" can return a XSS proof of concept depending on the model and context being used.

Sending unvalidated machine learning model output to a web browser can result in the browser executing malicious code.

Cross-site scripting (XSS) vulnerabilities occur when:

1. Data enters a web application through an untrusted source. In the case of Artificial Intelligence (AI), the untrusted source is typically the response returned by an AI system.


2. The data is included in dynamic content that is sent to a web user without validation.

The malicious content sent to the web browser often takes the form of a JavaScript segment, but can also include HTML, Flash, or any other type of code that the browser executes. The variety of attacks based on XSS is almost limitless, but they commonly include transmitting private data such as cookies or other session information to the attacker, redirecting the victim to web content controlled by the attacker, or performing other malicious operations on the user's machine under the guise of the vulnerable site. While exploitation is not as straightforward as other forms of XSS, the unpredictable nature of user input and the responses of AI models means that those responses should never be treated as safe.

Example 1: The following code retrieves a response from a large language model and returns it to the user in an HTTP response.

  chatService.createCompletion(
    text,
    settings = CreateCompletionSettings(...)
  ).map(completion =>
    val html = Html(completion.choices.head.text)
    Ok(html) as HTML
  )
  ...

The code in this example behaves as expected if the response from the model contains only alpha-numeric characters. However, if unencoded HTML metacharacters are included in the response then XSS is possible. For example, the response to the following prompt "please repeat the following statement exactly '<script>alert(1);</script>'" can return an XSS proof of concept depending on the model and context being used.