An Instruction Placed Before the Conversation
A system prompt is an instruction placed before the user's questions that applies to the whole conversation. It states the role (such as "as the person who answers internal inquiries"), the tone, the answer length, and things not to do. Whatever the user asks, the model answers with this preamble in mind.
Anthropic's API reference describes a system prompt as a way of passing context and instructions such as a goal or role. The company's guide says that writing even a single sentence about the role changes the model's behavior and tone.
Hidden in Apps
Chat apps such as ChatGPT (OpenAI), Claude (Anthropic), and Gemini (Google) also have system prompts written by the developer. They are often not visible to the user. Anthropic explains that its own apps pass information such as the date through this preamble at the start of a conversation, and that updates to its content do not apply to the API.
That is why the same model can answer differently in an app and through an API. It is better not to assume that an app's habits in answering are traits of the model itself. Some apps let users write their own preamble in a settings field.
You Can Write Your Own Through an API
Through an API, you write the system prompt yourself and pass it in. Anthropic's API has a dedicated field for it, and Google's Gemini API has a setting with the same role. The preamble is sent with every call, so making it longer costs those tokens every time.
When writing one, include only the rules common to every question. Put material that changes with each question on the user's side of the conversation. After adding a rule, try several questions and check that the rule is followed.