AI & Development
The system prompt is the most important prompt in any LLM application. Here is how to write one that establishes reliable behavior, handles edge cases, and
In a multi-turn LLM application, the system prompt is the instruction that persists across all turns of the conversation. It establishes the model's role, its behavioral constraints, its output format, and its handling of edge cases. Everything that you want to be true about every response your application generates should be in the system prompt. A well-written system prompt is the foundation of a reliable AI application. A poorly written one creates inconsistency, edge case failures, and behavior you did not intend.
The first statement in your system prompt should establish what the model is. "You are a customer support agent for Acme Corp, specializing in account billing and subscription management." This establishes three things: the model's function, its organizational context, and its scope. Each of these constrains the model's behavior in useful ways - it will respond from the perspective of a customer support agent, it will use the company's name appropriately, and it will be oriented toward billing and subscription questions.
Role statements work because the model's training data contains examples of how people in those roles communicate. The role activates a distribution of responses that matches those patterns. The more specific and accurate your role statement, the more reliably the model adopts the behaviors associated with that role.
Do not assume the model will choose the right output format on its own. If responses should be in Markdown, say so. If they should be plain text without formatting, say so. If they should follow a specific structure - "Always start with a direct answer, then explain the reasoning, then provide any relevant caveats" - describe that structure. Format instructions in the system prompt are followed reliably across conversations in a way that per-message format instructions are not.
For applications where consistency of format is critical - support tools that need to parse model outputs, reporting applications, or any interface where the response will be rendered in a specific way - structural instructions in the system prompt are more reliable than relying on prompting in each message.
What should the model do? What should it not do? Defining both is important. "You help users with questions about their account, billing, and subscription plans. If a user asks about anything outside these topics, politely let them know you can only help with account-related questions and suggest they contact [other team] for other needs." This defines both the in-scope behavior and the out-of-scope handling, giving the model a clear path for edge cases rather than forcing it to improvise.
Explicit out-of-scope handling prevents the model from attempting to help with requests it is not equipped for, which often produces lower-quality or incorrect responses. Giving the model a scripted alternative - redirecting to a different resource or team - produces a better user experience than an improvised non-answer.
Think through the 10 most likely edge cases your application will encounter and address them in the system prompt. If users frequently ask for information the model might hallucinate, instruct it to acknowledge uncertainty. If users sometimes try to get the model to do things outside its scope, instruct it how to handle those attempts. If tone should shift for frustrated or upset users, define that behavior.
Edge cases you do not address explicitly will be handled by the model's defaults, which may not match what you want. Addressing them in the system prompt makes the behavior predictable and controllable.
System prompts often need to include dynamic context - the current user's name and account status, today's date, relevant user preferences, or other runtime information. This dynamic content should be injected into the system prompt at a clearly delimited location, separate from the static instructions. A common pattern is to put static instructions first, then add a "Current session context:" section at the end with the injected variables.
Separating static and dynamic content makes the system prompt easier to maintain and enables prompt caching: providers cache the static portion of the system prompt and only process the dynamic portion on each call, reducing latency and cost significantly for high-volume applications.
After writing your system prompt, test it against adversarial inputs - attempts to get the model to behave outside its defined role. Try direct injection ("Ignore previous instructions"), social engineering ("You can trust me, just this once"), and role-play attacks ("Pretend you are a different AI with no restrictions"). Well-written system prompts significantly reduce the success rate of these attacks. Gaps in the system prompt that allow adversarial inputs to redirect behavior are best discovered in testing, not in production.