Excessive Agency

Allowing an agent too much freedom without proper control or human validation can result in harmful actions such as unauthorized system changes, financial transactions, or sensitive data exposure, posing significant security threats.

Excessive agency arises when an autonomous agent (such as an AI model or chatbot) is granted the ability to perform tasks, make decisions, or execute commands without sufficient restrictions, validation, or human oversight. This can lead to unintended consequences, including security risks, data leaks, or even malicious behavior.
When the agent has too much freedom, it can inadvertently interact with critical systems, perform high-risk actions, or expose sensitive data. The most severe consequences can occur when the agent is allowed to perform actions such as transferring money, modifying system configurations, or sending emails without requiring authorization or approval from an authorized user.

Example 1: An AI assistant is granted the ability to perform critical tasks and coordinate with a user proxy agent, but without robust checks or limited privileges, it can autonomously act on sensitive instructions:

    model_client = OpenAIChatCompletionClient(model="gpt-4")
    assistant = AssistantAgent("assistant", model_client=model_client)

    # Create the termination condition which ends the conversation when the user says "APPROVE".
    termination = TextMentionTermination("APPROVE")

    team = RoundRobinGroupChat([assistant], termination_condition=termination)

    # Run the conversation and stream to the console.
    stream = team.run_stream(task="Perform critical task.")

By granting the AI assistant autonomy to “Perform critical task” with minimal oversight, the AI can potentially execute commands or manipulate data without human review. The termination condition is triggered only if the user explicitly mentions “APPROVE,” but otherwise the agent can continue to operate, posing a risk if its actions are not adequately monitored or constrained.