{"post":{"slug":"ai-model-registry-2026","title":"AI Model Registry: Knowing What Is Actually Deployed","excerpt":"A model, a prompt, an adapter, and a configuration together determine behaviour. Tracking them as one versioned unit is the difference between reproducible and mysterious.","category":"ai","date":"January 13, 2026","author":"ByteLedger Team","content":"<span class=\"dropcap\">S</span>omeone asks why the assistant answered differently last Tuesday. Answering requires knowing which model version was serving, which prompt version it used, what effort and sampling settings applied, and whether an adapter was loaded.\n\nIn most deployments, at least one of those is unknown. The model was referenced by an alias that has since moved, the prompt was edited without a version, or the configuration lives in an environment variable nobody recorded.\n\n## What changed in 2026\n\n- **Model turnover made pinning essential.** Frequent releases and deprecations meant unpinned references changed underneath running products.\n- **The composite unit got recognised.** Teams stopped versioning models alone and started versioning the whole behaviour-determining bundle.\n- **Rollback speed became a requirement.** As AI features became load-bearing, reverting in seconds rather than through a rebuild became necessary.\n- **Registries stayed optional for small deployments.** The recognition that configuration in a repository covers most cases limited unnecessary tool adoption.\n\n## What actually determines behaviour\n\n| Component | Changes behaviour | Commonly versioned |\n| --- | --- | --- |\n| Model identifier and version | Substantially | Sometimes |\n| System prompt | Substantially | Rarely |\n| Tool definitions | Substantially | Almost never |\n| Effort or reasoning settings | Substantially | Rarely |\n| Sampling parameters | Moderately | Rarely |\n| Adapter, if any | Substantially | Sometimes |\n| Retrieval index version | Substantially | Rarely |\n\nLook at the middle column against the right one. The things that determine what your product does are largely not tracked, which is why \"why did it behave differently\" is so often unanswerable.\n\nTool definitions deserve particular mention: they shape agent behaviour as much as the prompt does, and they are almost never treated as versioned artefacts — see [tool schema design](/blog/tool-schema-design-2026).\n\n## Never deploy against a moving alias\n\nProvider aliases that point at \"the latest version\" are convenient and a genuine hazard. Your product's behaviour changes when the provider updates the alias, with no deploy on your side, no change in your repository, and no record.\n\nPin to a specific version. Upgrade deliberately, after running your evaluation against the new version — which is exactly what [LLM regression testing](/blog/llm-regression-testing-2026) is for.\n\nThe same applies to prompts. Production referencing \"the current prompt\" means an edit is an unreviewed deploy, per [prompt registries](/blog/prompt-registry-explained-2026).\n\n## Attach evaluation results to the version\n\nA version with no scores is a version you cannot reason about. Attaching evaluation results at the point of creation turns \"is the new one better\" from an argument into a lookup, and it gives you a baseline to compare against when something regresses months later.\n\nThe results worth recording: your evaluation suite scores, cost per representative request, latency percentiles, and refusal rate. Those four cover the dimensions that actually differ between candidate configurations.\n\n## Rollback must be fast\n\nThe purpose of versioning is reverting. If reverting requires a code change, a build, and a deploy, it will take long enough that a bad configuration stays live through the incident.\n\nMake the active version a runtime configuration value. Changing it should take effect within seconds without a rebuild. That converts a bad model or prompt from an outage into a brief blip.\n\nThe caveat is the one from [prompt registries](/blog/prompt-registry-explained-2026): runtime configuration makes changes easy, which means it needs a gate. Promotion to production should be a deliberate, recorded action rather than an edit.\n\n## Common mistakes\n\n- **Versioning the model but not the prompt.** Incomplete; behaviour still unexplainable.\n- **Deploying against a moving alias.** Silent behaviour changes.\n- **Rollback requiring a rebuild.** Too slow to use during an incident.\n- **No evaluation results attached.** Versions cannot be compared.\n- **Untracked tool definitions.** A major behaviour determinant, invisible.\n- **Adopting a heavyweight registry for one model.** Configuration in the repo is enough.\n- **No record of which version served a given request.** Makes incident analysis guesswork.\n\n## FAQ\n\n**Do I need a dedicated registry tool?**\nNot for a small deployment — a versioned configuration file in your repository covers model, prompt, and settings adequately. Tools earn their place with many models, many prompts, non-engineer editing, or experimentation.\n\n**Should the retrieval index be versioned too?**\nIdeally yes. A changed index changes outputs as surely as a changed prompt, and re-indexing is a behaviour change that usually goes unrecorded — see [embedding model migration](/blog/embedding-model-migration-2026).\n\n**How do I record which version served a request?**\nEmit it in your traces alongside the request. That single field makes incident analysis tractable, and it costs nothing — see [LLM observability](/blog/llm-observability-2026).\n\n**How long should I keep old versions?**\nLong enough to roll back to and to compare against — a few generations is usually sufficient. Keep the evaluation results indefinitely; they are small.\n\n## Where to go next\n\nFor the prompt half of the versioned unit, read [prompt registries](/blog/prompt-registry-explained-2026). For gating upgrades, [LLM regression testing](/blog/llm-regression-testing-2026), and for comparing candidates before promotion, [A/B testing LLM models](/blog/ab-testing-llm-models-2026).","hue":217,"metaTitle":"AI Model Registry: Knowing What Is Actually Deployed","metaDescription":"Why model, prompt, and configuration must be versioned together, the danger of moving aliases, and when a dedicated registry is worth adopting.","keywords":["ai model registry","model versioning","deployment tracking ml","prompt and model pinning","model rollback","reproducibility ai","llmops","ai operations 2026"],"tldr":["**The deployed unit is model plus prompt plus configuration** — versioning one without the others is incomplete.","**Pin versions; never deploy against a moving alias** — an alias that updates changes your product without a deploy.","**Record evaluation results with the version** — a version without scores cannot be compared.","**Rollback must be a configuration change** — if reverting needs a rebuild, you will not do it fast enough.","**Skip:** a dedicated registry tool for one model and one prompt. Configuration in your repo is sufficient."],"readTime":"4 min read","wordCount":826},"related":[{"slug":"prompt-registry-explained-2026","title":"Prompt Registries: Version Control for the Riskiest File","excerpt":"The prompt drives more behaviour than any function in your codebase and often lives in a string literal nobody reviews. A registry fixes the parts that matter.","category":"ai","date":"August 14, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":200},{"slug":"what-is-an-ai-rollback-strategy-2026","title":"What is an AI rollback strategy in 2026: a practical guide","excerpt":"A rollback strategy is how a team reverts to a known-good model or config when a new AI deployment misbehaves. Here is what a real one looks like in 2026.","category":"ai","date":"July 21, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":200},{"slug":"ai-model-versioning-2026","title":"AI Model Versioning in 2026: Practices That Prevent Regressions","excerpt":"How teams version AI models in 2026 without breaking production — pinning, shadow testing, rollback plans, and why silent provider updates are the real risk.","category":"ai","date":"July 4, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":305}],"sameCategory":[{"slug":"agent-cost-attribution-2026","title":"Agent Cost Attribution: Finding Where the Tokens Went","excerpt":"A single monthly API bill tells you that agents are expensive. It does not tell you which feature, which customer, or which retry loop is responsible. That gap is fixable.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":220},{"slug":"human-in-the-loop-agents-2026","title":"Human-in-the-Loop Agents: Where to Put the Approval Gate","excerpt":"Approve everything and the agent is a slow autocomplete. Approve nothing and you have handed irreversible actions to a system that can be talked into things.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":197},{"slug":"reasoning-effort-controls-2026","title":"Reasoning Effort Controls: Tuning How Hard a Model Thinks","excerpt":"Fixed thinking budgets gave way to effort levels, and the default is rarely the right setting. Here is how the dial behaves and where each level actually belongs.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":210},{"slug":"retrieval-metrics-explained-2026","title":"Retrieval Metrics: Measuring the Half of RAG That Fails","excerpt":"When a RAG system answers badly, the retrieval usually failed before the model ever spoke. Measuring generation quality alone hides that entirely.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":215}],"newer":{"slug":"agent-tool-error-handling-2026","title":"Agent Tool Errors: Writing Failures a Model Can Recover From","excerpt":"Your tool returned \"Error: invalid request\". The agent has no idea what to change, so it retries the same call. Error messages are prompt surface, and most are written for humans reading logs.","category":"ai","date":"January 13, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":208},"older":{"slug":"amt-credit-explained-2026","title":"The AMT Credit: Getting Back What You Paid on ISOs","excerpt":"Exercising incentive stock options can trigger alternative minimum tax on gains you never received in cash. The credit recovers it later, slowly, and only if you know to claim it.","category":"finance","date":"January 13, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":272}}