AI agents can perform tasks by interacting with tools, external services, and their environment. This thesis investigates how to safeguard such systems through a runtime enforcement layer that checks proposed actions before execution. Based on explicitly defined safety policies, the layer permits, blocks, or redirects actions while allowing the agent to complete legitimate tasks.

The student will develop and evaluate a prototype for a selected application, such as an assistant operating on files and external services. Example policies include preventing unauthorized file deletion, restricting disclosure of sensitive information, and requiring approval before consequential actions.

The central research question is: How can runtime monitoring and enforcement prevent specified undesirable agent actions while preserving useful task execution?

Addressing this question requires more than checking individual tool calls. Whether an action is permitted may depend on its arguments, previous actions, user authorization, and the current environment state. Furthermore, blocking an action may prevent immediate harm but leave the agent unable to complete its task.

The thesis will investigate how to specify such policies, monitor relevant execution context, and intervene before prohibited effects occur. It will distinguish formally enforceable properties from safeguards whose effectiveness can only be evaluated empirically. Any prevention guarantee must state its assumptions, including that all relevant actions pass through the enforcement layer and that sufficient information is available to evaluate the policy.

• Committed to producing research suitable for scientific publication.
• Good coding skills, including effective use of AI coding assistants.
• Hardworking and willing to turn ambiguous problems into concrete research tasks.
• A background in formal methods is a plus.