Securing an application that calls an LLM is not the same problem as securing a traditional web app, because the attack surface includes the model's own susceptibility to manipulation through natural language. The core shift: an LLM cannot reliably distinguish trusted instructions from untrusted data embedded in its context. Anything the model reads — a retrieved document, a web page, a tool's output, a user's message — can carry text that looks like an instruction, and the model may follow it. Effective LLM security accepts that premise and builds controls around the model rather than trusting it to defend itself.
What changed in 2026
- Indirect prompt injection overtook direct jailbreaking as the primary concern. Attacks increasingly arrive through retrieved content and tool outputs rather than the user's own message, since agentic systems now read far more untrusted text than they did a couple of years ago.
- Agentic tool use widened the blast radius. As agents gained the ability to send emails, execute code, and make purchases, a successful injection stopped being "the model said something wrong" and became "the model did something wrong."
- Dedicated LLM security tooling matured. Prompt injection scanners and guardrail frameworks are now a common defense-in-depth layer, though none claim to fully solve the problem.
- Regulatory pressure pushed logging and auditability. Compliance requirements phasing in through 2026 make prompt and output logging, plus human oversight for higher-risk uses, closer to a requirement than a best practice.
- The OWASP LLM Top 10 became the shared reference taxonomy most security teams now use to talk about these risks consistently — see OWASP's LLM Top 10 explained for the full list.
Core practices that actually reduce risk
- Isolate tool execution. Anything the model can call — code execution, file access, HTTP requests — should run in a sandboxed, ephemeral environment with no more network or filesystem access than the specific task requires.
- Scope credentials per tool, not per application. A "search the web" tool and a "send email" tool should never share an API key with broader permissions than either task needs; a prompt injection that hijacks one tool call should not be able to pivot to another capability.
- Validate structured output before acting on it. If the model returns JSON meant to drive an action — a database update, an API call — validate it against a strict schema and reject anything that does not conform. Do not
eval or blindly trust generated code or commands.
- Enforce retrieval access control at the data layer, not the prompt layer. RAG pipelines must filter retrieved documents by the requesting user's actual permissions before they ever reach the model's context — a permission check phrased as a prompt instruction is not a permission check.
- Add a human confirmation step for high-consequence actions. Sending money, deleting data, or messaging external parties should require explicit confirmation, especially for agents that chain multiple tool calls autonomously.
- Log prompts, retrieved context, and outputs. When something goes wrong, you need to reconstruct what the model actually saw, not just what your application intended to send it.
- Cap cost and rate per user or session. Expensive prompts — long context, repeated tool calls — are a denial-of-wallet vector; enforce token and request budgets the same way you would rate-limit any public API.
Common mistakes
Trusting the system prompt as a security boundary. "Never reveal your instructions" in a system prompt is a behavioral nudge, not an access control — treat it as UX guidance, not a defense.
Giving agents standing credentials instead of scoped, short-lived ones. A compromised or manipulated agent session should not be able to do more damage than the specific task in front of it justifies.
Rendering model output directly as HTML or executing it as code. Model output needs the same output-encoding discipline as any other untrusted string — escape it for the context it lands in, whether that is HTML, SQL, or a shell command.
Skipping access control in RAG because "the model wouldn't just tell someone." It will, given the right prompt. Filter documents by permission before retrieval, not by hoping the model declines.
FAQ
Is prompt injection the same as jailbreaking?
No. Jailbreaking targets the model's own safety training via the user's direct message; prompt injection smuggles instructions through data the model reads, often content the end user never wrote or saw, like a retrieved webpage.
Can guardrail tools fully prevent prompt injection?
No current tool fully prevents it. Guardrails and injection scanners reduce risk as one layer; the controls that actually limit damage are scoped permissions, sandboxing, and human confirmation on consequential actions.
Do I need to worry about this if I only use a hosted API like OpenAI or Anthropic?
Yes. Model-provider safety training reduces some risks but does not address application-level issues like tool permission scope, RAG access control, or output handling — those remain your responsibility regardless of which model you call.
What is the single highest-leverage control?
Least-privilege scoping of whatever actions the model can trigger. Even a fully successful prompt injection is contained if the credentials and tools available to the model can only do a narrow, low-consequence task.
Where to go next