Threat modeling a RAG chatbot for a bank

A walk through the OWASP LLM Top 10 threat model for a retrieval-only, unauthenticated bank chatbot. Scope the system right and only four threats actually bite.

This post works through a threat model for a RAG chatbot built for a retail bank’s public website, using the OWASP LLM Top 10 as the triage framework. The system is retrieval-only and unauthenticated: no tool calls, no customer data, public corpus only. Scope it that tightly and the threat list gets short fast. That scoping is a deliberate design choice, not a shortcut.

What you’ll walk away with: an OWASP LLM Top 10 applicability triage for this specific system, a data-flow diagram with the two trust boundaries that matter, the four threats that merit real design effort, and the five controls that address them.

The reference system

The system is constructed for this exercise. It represents a plausible retail bank deployment, but doesn’t correspond to any real product.

A user loads the bank’s public website and opens a chat widget. The widget sends a query to a backend orchestration layer, which embeds the query and retrieves relevant chunks from a vector database. Retrieved chunks plus the original query go to a managed LLM API (GPT-4o or Claude; treated here as a trust boundary). The LLM returns a grounded response. The widget renders it in the browser.

The ingestion pipeline runs separately: a crawler or document feed pulls from the bank’s public content sources (product pages, FAQs, regulatory disclosures), chunks and embeds that content, and writes to the vector database. This pipeline is offline from the query path.

The reference system. Dashed boxes are the two trust boundaries: the internet edge and the ingestion pipeline.

Everything else depends on these two trust boundaries. The internet edge separates untrusted user traffic from the backend. Most implementations treat content crossing the ingestion pipeline as trusted. It isn’t. Content flows from the public web into the vector database, which feeds directly into LLM prompts. That pipeline is where most of the threat surface lives.

Structural constraints that shape the threat model:

No tool calls or actions. The LLM produces text. Nothing else.
No customer data. The vector database contains public content only. No authentication, no account lookups, no PII in the corpus.
Unauthenticated public endpoint. Anyone can query it.
Managed LLM provider. Internal security of the provider is a trust boundary, not in scope.

OWASP LLM Top 10 applicability triage

OWASP’s LLM Top 10 (2025 edition) gives ten risk categories for LLM applications. Not all ten apply to every deployment. Triage against the architecture above before working the threats in depth.

ID	Name	Applicability	Reasoning
LLM01:2025	Prompt Injection	HIGH	Indirect injection via poisoned corpus is the dominant vector. OWASP explicitly states RAG “does not fully mitigate prompt injection vulnerabilities.”
LLM02:2025	Sensitive Information Disclosure	LOW	Corpus is public. No customer data in the retrieval path. Risk reduces to incidental disclosure in the corpus itself.
LLM03:2025	Supply Chain	LOW-MEDIUM	Managed LLM provider risk exists but is a trust boundary; out of scope for this post. Corpus content sources are the relevant supply-chain surface.
LLM04:2025	Data and Model Poisoning	MEDIUM	Corpus poisoning overlaps with LLM01 and LLM08. Applicable through the ingestion pipeline.
LLM05:2025	Improper Output Handling	HIGH	Unsanitized LLM output rendered in the widget creates XSS and CSRF surface in the user’s browser session.
LLM06:2025	Excessive Agency	NOT APPLICABLE	No tool calls, no actions. The LLM is pure Q&A. This threat class is structurally eliminated by the architecture.
LLM07:2025	System Prompt Leakage	MEDIUM	Jailbreaks can extract system-prompt contents. Low direct harm here (no secrets in the prompt), but reveals backend instructions to an attacker.
LLM08:2025	Vector and Embedding Weaknesses	HIGH	Corpus poisoning via hidden text and unverified sources. The ingestion pipeline is the primary attack surface for this item.
LLM09:2025	Misinformation	HIGH	Hallucinated authoritative financial answers carry documented compliance and reputational liability. The CFPB has stated chatbots must comply with all applicable federal consumer financial laws.
LLM10:2025	Unbounded Consumption	HIGH	Unauthenticated public endpoint plus pay-per-use API equals direct denial-of-wallet risk. OWASP names this as a sub-category explicitly.

LLM06 and LLM02 drop out directly from the architecture. The four HIGH items are the ones worth working in detail.

The four threats that bite

Indirect prompt injection via the corpus

This is the threat to design around. OWASP LLM01:2025 calls out indirect injection in a RAG context explicitly: content retrieved from a data store can carry attacker-controlled instructions that alter the model’s behavior. The model can’t reliably distinguish between “content to summarize” and “instructions to follow.”

Greshake et al. (arXiv:2302.12173) framed the class: LLM-integrated applications “blur the line between data and instructions,” letting adversaries attack “by strategically injecting prompts into data likely to be retrieved.” OWASP states the injected content “does not need to be human-visible/readable, as long as the content is parsed by the model.”

That second point is what makes the ingestion pipeline the dominant trust boundary. The Hidden-in-Plain-Text benchmark (ACM WWW 2026, arXiv:2601.10923) tested web-native invisible carriers, measuring the attack success rate (ASR) of each: hidden HTML spans (34.0%), off-screen CSS (30.1%), alt-text attributes (27.8%), zero-width Unicode characters (23.2%), and ARIA attributes (9.6%), for a macro-average of 24.9% on undefended systems. Those numbers were measured on open-weight models (Llama-3, Mistral, Qwen2.5). Whether commercial managed APIs with built-in safety filtering suppress them meaningfully is unresolved. Treat 24.9% as a floor for undefended systems, not as a production estimate.

PoisonedRAG (USENIX Security 2025) demonstrated targeted corpus injection under a black-box attacker model, meaning the attacker doesn’t need access to retriever internals, only write access to some corpus content. Success rates above 90% in standard black-box conditions, dropping to around 77% against more sophisticated RAG variants that validate retrieved content (arXiv:2402.07867). Corpus write access may be easier to obtain than it looks: a bank’s public corpus typically pulls from web content, PDFs, and third-party document feeds, any of which could be a point of attacker influence. The countermeasures are architectural: a trusted-source allowlist for the ingestion crawler, ingestion-time sanitization, and isolating the ingestion pipeline from the query path. Indirect injection is a real threat, but getting the ingestion architecture right is what bounds both its likelihood and its blast radius.

MITRE ATLAS, MITRE’s catalog of adversary techniques against AI systems (the ML counterpart to ATT&CK), tags this as AML.T0051 (LLM Prompt Injection) and AML.T0051.001 (Indirect Prompt Injection).

One more finding: backdoor attacks against the dense retriever itself (arXiv:2410.14479, preprint) are stealthy in a specific way. The backdoored retriever is built to behave normally, or better, on benign queries, so retrieval precision metrics improve whether the data is benign or poisoned; standard RAG quality monitoring won’t catch the attack. Corpus security monitoring needs separate provenance controls, not just retrieval quality metrics.

OWASP LLM05:2025 (Improper Output Handling) is about what happens to LLM output before it reaches the user. If the widget renders unsanitized model output as HTML or interprets Markdown links without checking them, a corpus-injected instruction that causes the model to emit a script tag or a data-exfiltration image link can execute in the user’s browser session.

The attack chain: adversarial content in the corpus feeds a retrieval instruction to the model, the model emits a malicious script tag or data-exfiltration link, the widget renders it without sanitization, and the browser executes it on the bank’s domain, in the same browser session where the user may also be logged into online banking.

OWASP explicitly lists this: “JavaScript or Markdown is generated by the LLM and returned to a user. The code is then interpreted by the browser, resulting in cross-site scripting (XSS).” CSRF uses the same render path: rendered markup can trigger state-changing requests that ride the user’s existing cookies. CVE-2025-24981 (Nuxt MDC library XSS via markdown rendering) demonstrates that this is a production exploitability class, not a theoretical one.

This is a web application vulnerability enabled by LLM output, not a pure LLM problem. The right control lives in the output layer.

Hallucinated authoritative financial answers

RAG meaningfully reduces hallucination, though the residual rate is non-zero. On a bank’s public chatbot that residual has a compliance dimension.

The Consumer Financial Protection Bureau’s (CFPB) June 2023 issue spotlight stated directly: “Like the processes they replace, chatbots must comply with all applicable federal consumer financial laws, and entities may be liable for violating those laws when they fail to do so.” The relevant exposure runs through UDAAP (unfair, deceptive, or abusive acts or practices) and the other consumer-finance statutes the CFPB enforces. Note: the CFPB withdrew some guidance in May 2025, which may affect that document’s formal status. The underlying statutes remain in force independently.

The liability question is not theoretical. Air Canada’s chatbot misstated bereavement fare policy, and a 2024 ruling found that Air Canada couldn’t disclaim liability for its chatbot’s output. The NYC government chatbot (documented by The Markup, March 2024) advised businesses to take actions that violated city law. Neither case involved a RAG architecture specifically, but both illustrate what happens when a public-facing chatbot asserts something authoritatively and turns out to be wrong.

OWASP LLM09:2025 tags this as Misinformation.

Denial-of-wallet

The architecture has an unauthenticated public endpoint talking to a managed LLM API billed per inference. OWASP LLM10:2025 names this directly under Unbounded Consumption: “By initiating a high volume of operations, attackers exploit the cost-per-use model of cloud-based AI services, leading to unsustainable financial burdens.”

This one is operationally serious in a way that’s easy to underestimate. A sustained flood of queries against an unprotected endpoint doesn’t need to be sophisticated. To sketch the scale: a bot sending 10 requests per second against a managed API billed per input/output token can rack up roughly thousands of dollars per hour before any on-call alert fires, especially if the attacker crafts long queries to maximize token consumption per call. The cost accumulates at the API provider even if the backend is handling the volume fine.

Secondary effects in the same category: jailbreaks (MITRE ATLAS AML.T0054) can produce brand-damaging outputs at scale, and the chat interface can be used to scrape the bank’s public corpus if the rate limiting doesn’t account for high-precision sequential queries, queries that walk the corpus topic by topic.

With injection bounded by the ingestion architecture, the remaining contest for design attention is between output handling (LLM05) and hallucinated authority (LLM09). Fix output handling first. A successful injection chain that executes code in the user’s browser session is an active exploit on the bank’s domain. Hallucinated authority (LLM09) is serious from a compliance and reputational standpoint, but the harm is slower and the mitigation path (source attribution, disclaimers, escalation) is cleaner. Denial-of-wallet is mitigated quickly with rate limiting and cost controls, so it doesn’t compete for the same design attention. Fix the active exploit surface before tuning the accuracy controls.

Five controls that address them

These address the four threats above. This is architecture guidance, not a full hardening checklist.

Data isolation and least privilege. The retrieval service needs no read path to customer data stores. Enforce that in the network topology and IAM configuration, not in policy. Policy can be overridden; a network boundary that doesn’t exist can’t be misconfigured away. This is what eliminates LLM06 and keeps LLM02 low. The reason the threat list for this system is short is that the system was scoped to make isolation tractable. If isolation breaks down, the entire triage table shifts.

Corpus provenance and ingestion-time sanitization. HTML and Markdown sanitization on all ingested content, Unicode normalization to catch zero-width and invisible characters, an allowlist of trusted source domains for the crawler, and anomaly detection on retrieved content before it reaches the prompt. Standard RAG retrieval quality metrics are not a security signal; as noted above, backdoor attacks improve those metrics. Separate provenance controls are needed.

Output encoding and Content Security Policy. LLM output is untrusted HTML until your code says otherwise. Sanitize it before rendering in the widget, sanitize Markdown links, and enforce a strict Content-Security-Policy header that blocks inline script execution. That combination closes the XSS and CSRF surface from LLM05.

Rate limiting and cost controls. Per-IP and per-session rate limits on the public endpoint, token budget caps on LLM API calls, and cost-alerting at thresholds that let the operator respond before a bill becomes extreme. For an unauthenticated endpoint, these controls are the primary defense against denial-of-wallet.

Response accuracy controls with an escalation path. Source attribution in responses (which chunks was the answer drawn from), a visible disclaimer on financial information, monitoring for queries about specific regulatory or account-action topics, and a defined escalation path to human review for high-stakes queries. Keeping the corpus current matters: stale or wrong policy content is exactly what produced the Air Canada and NYC chatbot failures.

What the frameworks do and don’t give you

The OWASP LLM Top 10 and MITRE ATLAS are useful here for the same reason the Automotive Threat Matrix is useful for a TCU (a telematics control unit, the cellular gateway in a car): they give you a structured list of what to consider so you don’t skip a category. AML.T0051.001 tells you that indirect prompt injection via a retrieval corpus is a named, characterized adversary technique. LLM05:2025 tells you that output handling is a distinct threat category from the model itself. That’s genuine value.

What they don’t give you is a priority order, a likelihood estimate, or system-specific scoping. Neither the triage table nor the LLM06 elimination is in OWASP. The frameworks enumerate threats. The analyst decides which ones apply to a given design and which to fix first.

NIST AI 600-1 and the NIST AI RMF add governance framing but don’t add new technical threat categories for a deployment like this one.

The gap that matters most here is the commercial-API question. All the ASR figures for corpus injection (the 24.9% macro-average, the PoisonedRAG black-box results) were measured on open-weight models without production safety filters. Whether GPT-4o or Claude’s safety systems suppress injection success rates at commercial scale is genuinely unresolved. If you’re deploying on a managed API, that’s a question worth asking your provider, and it’s one the published literature can’t yet answer.