AI Explained
Prompt versioning: ship prompts like code
Track every prompt change as a version, ship it by moving a production label, roll back in seconds, and compare versions in traces and evals.
On this page
Someone tweaked the system prompt on Friday. On Monday, answers got worse. Which change did it? If your prompt lives in an untracked string, nobody knows.
Prompts are code
A prompt changes your app's behavior as much as any function does. Prompt versioning treats every change as a new version, never an edit in place.
- MLflow's Prompt Registry makes versions immutable, "providing strong guarantees for reproducibility," with Git-style commit messages.
- In Langfuse, saving a prompt with the same name adds a new version.
Labels decide what's live
Labels (Langfuse) or aliases (MLflow) point at versions:
| Label | Points to |
|---|---|
staging |
v3, being tested |
production |
v2, what users get |
latest |
v3, the newest |
Your app fetches the production label, not a hard-coded string:
prompt = langfuse.get_prompt("support-bot", label="production")
Langfuse "never silently falls back to production or latest," so a typo in a label won't quietly serve the wrong prompt.
Ship and roll back by moving a label
Shipping a new prompt means moving production to the new version, with no code deploy. Rolling back means moving it back: seconds, not a hotfix. Labels also make A/B tests easy, sending part of the traffic to a second version.
Caching and pinning
SDKs cache prompts on the client, so fetching by label doesn't cost a network call every time. MLflow caches pinned versions forever (they never change) and alias lookups for 60 seconds by default.
For tests and reproducibility, pin an exact version or commit instead of a label. In LangSmith you pull a prompt by name plus a commit hash or tag, like joke-generator:12344e88.
Version what the prompt depends on
A prompt only means something with its model. Version the model name and settings (temperature, max tokens) together with the template; MLflow supports storing both.
Measure before you promote
- Record which prompt version each trace used.
- Compare quality, cost and errors by version.
- Run your eval set on the new version before moving the
productionlabel.
No platform? Keep prompts as files in Git, reviewed in pull requests, with release tags. You get most of the benefit for free.
Version it, label it, measure it.