OpenAI Misconfiguration: Unspecified Token Limit

Failure to set proper limits on the number of completion tokens can affect the stability and availability of the system, which can result in denial-of-service attacks or resource depletion.

When LLM APIs are not properly configured, a Denial of Wallet (DoW) attack can occur if attackers exploit the cost-per-use model of cloud-based AI services by initiating a high volume of requests. This leads to unsustainable financial burdens on the provider. This can result in financial ruin for the provider, as the cost of processing excessive requests becomes unmanageable.

Example 1: The following code shows how setting the max_completion_tokens parameter to None can increase the chances of producing excessive data. This can overwhelm the resources of the LLM server and potentially lead to a Denial of Wallet (DoW) attack.

import os
from openai import OpenAI

client = OpenAI(api_key='OPENAI_API_KEY')

def generate_safe_response(prompt):
    completion = client.chat.completions.create(
        model="gpt-3.5-turbo",
        timeout=10,
        max_completion_tokens=None,
        messages=[
            {"role": "user", "content": prompt}
        ]
    )
    return completion.choices[0].message

prompt = "Describe the latest advancements in machine learning."
response = generate_safe_response(prompt)
print(response)

In Example 1, the max_completion_tokens parameter limits the length of the output, which helps to ensure that the model does not generate excessively long responses that could cause issues downstream. By controlling the output size, you reduce the risk of generating unintended, potentially harmful content that could lead to security vulnerabilities.