In this article
  1. Before you start
  2. Why interviewers ask this
  3. LLM in banking compliance architecture from first principles
  4. Worked example
  5. Trade-offs and edge cases
  6. How to answer it in an interview
  7. Common mistakes
  8. Follow-up questions
  9. Sources

Before you start

This page is one part of the generative AI system design guide.

Why interviewers ask this

A bank cannot ship a clever demo. It must show a regulator, an auditor and its own risk team why the system is safe. This question separates candidates who describe a pipeline from candidates who describe the evidence the pipeline leaves behind.

A weak answer names a model, a vector store and a "human in the loop." A strong answer asks what the model does, because the use decides how much control it needs. It then places each control outside the model, since a prompt is not a control. A Staff-level answer also knows which rules apply and which do not. It says what the bank's own policy must cover where the public guidance is silent, and it states the limits of its own knowledge instead of citing a superseded rule.

LLM in banking compliance architecture from first principles

Effective challenge
Effective challenge is, in the wording of the Federal Reserve's Supervision and Regulation (SR) letter SR 11-7 (2011), critical analysis of a model by objective, informed parties who can identify its limitations and produce appropriate changes. It is the reason validators must be independent of the people who built or use the model.
Materiality tier
A materiality tier is a rank a bank gives each model use by how much harm a wrong output could do. The tier sets how often the use is reviewed, how much testing it needs and whether a person must approve its output before release.
Decision record
A decision record is the stored evidence of one model-assisted decision: the inputs, the sources retrieved, the model and prompt versions, the output, the checks that ran and the person who approved it. It lets someone reconstruct the decision later.

In plain words

Picture a bank branch. The teller is quick and helpful. But the vault door, the ledger and the manager's countersignature are the controls, and the teller does not control them.

An LLM is the teller. It is fluent and fast, and it is sometimes wrong. The architecture is the vault, the ledger and the countersignature around it.

Model risk management is the practice of finding and limiting harm from models that are wrong or misused. The analogy breaks in one place: a teller cannot be talked into new orders by a note slipped across the counter, and an LLM can. So the controls must never depend on the model behaving well.

Step 1: Find which rules apply and which do not

The Federal Reserve's supervisory letter SR 11-7 (4 April 2011) defines a model as a quantitative method that turns input data into quantitative estimates. Free-text LLM output fits that definition awkwardly. The World of AI reports that on 17 April 2026 the Federal Reserve, the Office of the Comptroller of the Currency (OCC) and the Federal Deposit Insurance Corporation (FDIC) issued SR 26-2, which supersedes and replaces SR 11-7.

It adds that the new guidance is not enforceable, aims mainly at banks above 30 billion dollars in assets, and "leaves generative and agentic AI out of scope". Domino says generative and agentic AI "are placed outside the scope of this guidance" and that institutions apply their existing risk management practices to govern them. Both are secondary summaries; check the Federal Reserve letter itself before you quote it.

The consequence is clear: the bank needs its own governance layer for LLMs. It can reuse the old disciplines, which are an inventory, independent review, monitoring and documentation.

Which European Union rules still apply?

The European Union (EU) AI Act classifies a system by its use. Annex III point 5(b) covers AI that evaluates the creditworthiness of natural persons or sets their credit score, and excludes AI used to detect financial fraud. Its high-risk duties include risk management, data governance, documentation, logging, transparency, human oversight, and "accuracy, robustness and cybersecurity", which is the Act's own title for Article 15.

vdf.ai and ComplyLayer report that these obligations apply from 2 December 2027 under the Digital Omnibus. Check the current text before you quote a date.

The General Data Protection Regulation (GDPR) also matters. ComplyLayer reports that the EU Court of Justice, in case C-634/21, held that a credit score counts as an automated decision when a lender relies on it heavily. Rules differ by use, so the first control is knowing the use.

Step 2: Put every use case in a materiality tier

SR 11-7 states that model risk rises with complexity, uncertainty about inputs, breadth of use and potential impact. It asks banks to keep an inventory of models in use, in development and recently retired. For an LLM, the inventory entry names the use, not only the model: the same model can sit behind a low-stakes FAQ bot and a credit-decision explainer.

Give each use a materiality tier. Ask three questions. Does it touch a credit decision or a customer's rights? Does it feed a regulated workflow, such as an anti-money-laundering (AML) review? Does it handle personal data?

Domino reports that SR 26-2 replaces the de facto annual review with a risk-based cadence tied to materiality. Whatever the guidance says, the logic is sound: a high tier gets shorter review cycles, tougher tests and a human gate. The worked example below applies these rules to three use cases.

Step 3: Wrap the model in a gateway and permission checks

Every model call passes through a gateway that checks permissions before retrieval. The gateway authenticates the caller, redacts what the task does not need and logs the call. Superwise summarizes the Open Worldwide Application Security Project (OWASP) advice as: send the model only the fields a task needs, check authorization before retrieval and keep secrets out of system prompts.

Prompt injection is an attack in which text the model reads, from a user or from a retrieved file, changes what the model does. Prompt injection and sensitive information disclosure rank first and second in the 2025 and 2026 lists. Excessive agency is a model holding more tools or permissions than its task needs. It rose to third in 2026.

So give tools minimum permissions, run them as the user, treat output as untrusted and require approval for high-impact actions. The pages on permission-aware RAG and RAG in a regulated bank cover the retrieval side.

Step 4: Keep a decision record, not a chat log

A decision record holds the inputs, output, model version and reviewer. Logging is not saving the chat. Article 12 of the AI Act requires high-risk systems to allow automatic recording of events, relevant to spotting risk situations, post-market monitoring and monitoring operation.

A useful record for one LLM-assisted decision stores references, not copies of everything. Store the case identifier, the retrieved document identifiers and versions, the prompt template version, the model version, the output, the guardrail results, the reviewer and the final decision. Keep raw personal data out of the log where an identifier will do.

Without these fields an auditor cannot tell whether a bad output came from the model, a stale document, a changed prompt or a reviewer. Treat the log itself as sensitive data, with its own access rules and retention period set by the bank's policy.

Step 5: Build real human oversight and independent review

Oversight must be effective, not a rubber stamp. Article 14 says high-risk systems must be designed so that natural persons can effectively oversee them while in use, with measures commensurate with the risks, autonomy and context. A reviewer who approves every draft in seconds does not meet that test.

Design the queue so people can think. Route low-confidence or high-tier drafts to a person and sample the rest for quality. Give reviewers the sources, the authority to reject and a way to report patterns.

Effective challenge depends on independence: SR 11-7 says validation involves a degree of independence from development and use, and that it covers conceptual soundness, ongoing monitoring and outcomes analysis. The stage-by-stage RAG evaluation guide shows what to measure.

Five controls make an LLM defensible inside a bankFlow of five controls in order: find which rules apply, set a materiality tier, route calls through a gateway, keep a decision record, and add effective human oversight. Every control sits outside the model.Find theruleswhat appliesSet the tierby materialityGatewaypermissionsfirstDecisionrecordevidence trailHumanoversightcan rejectDiagram: Interview Clozo, CC BY 4.0 ·interview.clozo.ai/blog/llm-in-banking-compliance-architecture.html

Rules decide the tier, the tier decides the controls, and every control sits outside the model.CC BY 4.0 — source: LLM in banking compliance architecture: how to design it
Step-by-step description
  1. Know which rules apply and which do not.
  2. Rank the use by materiality.
  3. Check permissions before retrieval.
  4. Store inputs, versions and reviewer.
  5. Review that can actually reject.
  • Rules decide the tier, and the tier decides the controls.
  • Every model call passes through a gateway that checks permissions before retrieval.
  • A decision record holds the inputs, output, model version and reviewer.
  • Oversight must be effective, not a rubber stamp.

Rules decide the tier, and the tier decides the controls. The gateway, the decision record and the oversight queue are the controls that make the answer defensible.

Knowing it is half the job. Saying it is the other half.

Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.

Start Free Trial →

Worked example

Take three uses of one model, with rules as illustrative assumptions: a use is tier 1 if it touches a credit decision, tier 2 if it feeds a regulated workflow or handles personal data, and tier 3 otherwise.2 The credit-decision explainer is tier 1, the alert summarizer is tier 2 and the customer FAQ bot is tier 3. Illustrative review cadences are every 3, 6 and 12 months, against the at-least-annual periodic review that SR 11-7 required; SR 11-7 is superseded, so that is the historical baseline.

Higher materiality earns tighter review and a human gateTable of three illustrative bank use cases: a credit decision explainer is tier 1 with a three-month review and approval before release, an alert summarizer is tier 2, and a customer FAQ bot is tier 3.Use caseTierReviewcadenceHumanstepCredit decisionexplainerTier 13 monthsApprovalbeforereleaseAlert summarizerTier 26 monthsAnalystreviewsdraftsCustomer FAQbotTier 312 monthsSampledtranscriptsDiagram: Interview Clozo, CC BY 4.0 ·interview.clozo.ai/blog/llm-in-banking-compliance-architecture.html
The higher the materiality, the shorter the review cadence and the firmer the human gate.CC BY 4.0 — source: LLM in banking compliance architecture: how to design it
Text description of the diagram
  • The credit-decision explainer is tier 1, the alert summarizer tier 2 and the customer FAQ bot tier 3.
  • Illustrative review cadences are every 3, 6 and 12 months.

Now size the human queue for the alert summarizer. Assume 1,200 alerts a day, 20% flagged low-confidence and routed to an analyst, and a 5% quality sample of the rest.2 That is 240 routed alerts and 48 sampled out of the other 960, so 288 human reviews a day. At an assumed 6 minutes per routed review and 2 minutes per sampled one, the work is 25.6 analyst hours a day.2 The review-every-alert baseline assumes 6 minutes per alert for all 1,200 alerts.

In this illustrative case, routing low-confidence drafts and sampling the rest needs about a fifth of the analyst hours of reviewing every alert.
QuantityValue (illustrative)
Alerts per day1,200
Routed to an analyst (20% low confidence)240
Sampled from the other 960 (5%)48
Human reviews per day288
Analyst hours per day, route and sample25.6
Analyst hours per day, review every alert120
Analysts on an 8-hour day, route and sample3.2
Analysts on an 8-hour day, review every alert15
Workload share against reviewing everything21.3%

Reviewing every alert takes 120 analyst hours a day; routing and sampling takes 25.6.2 The saving only holds if the confidence signal is honest, so the sample of unflagged drafts is how you check it.

Routing and sampling cuts review work to about a fifthBar chart of illustrative analyst hours per day for an alert summarizer: 120 hours if every alert is reviewed, 25.6 hours if low-confidence drafts are routed and the rest sampled.Analyst hours per day (hours)04080120Review allRoute, sampleReview policy12025.6Diagram: Interview Clozo, CC BY 4.0 ·interview.clozo.ai/blog/llm-in-banking-compliance-architecture.html
Routing low-confidence drafts and sampling the rest cuts the illustrative review workload to about a fifth.CC BY 4.0 — source: LLM in banking compliance architecture: how to design it
Text description of the diagram
  • Reviewing every alert takes 120 analyst hours a day; routing and sampling takes 25.6.
  • Routing and sampling needs about a fifth of the analyst hours of reviewing every alert.

Trade-offs and edge cases

More review lowers risk and costs analyst time. Less review is cheaper but turns oversight into a rubber stamp. The sampling rate is the knob, and it should rise when monitoring shows drift.

A new model version changes behavior without any change to your code. Treat a version change as a change to the inventory entry: re-run the evaluation set, record the version in every decision record and re-tier if the use changed.

A hosted model brings a third party into the chain. The interagency guidance on third-party relationships (Federal Reserve SR 23-4, June 2023) covers due diligence, contract negotiation and ongoing monitoring across the life of a relationship, which fits a hosted model vendor. Contract terms such as data retention and use belong there; whether a vendor may train on your data is our own application of that guidance, not a point it names.

Text shown to a customer, such as the reasons for a declined credit application, needs legal review. The Consumer Financial Protection Bureau (CFPB) Circular 2022-03 (May 2022) says creditors using complex algorithms must still give specific principal reasons for adverse action under the Equal Credit Opportunity Act (ECOA) and Regulation B.

The National Institute of Standards and Technology (NIST) publishes NIST AI 600-1, a voluntary profile of the AI Risk Management Framework for generative AI. It lists 12 risks, including confabulation, data privacy and information security, and maps them to the framework's govern, map, measure and manage functions. Use it as a checklist of vocabulary, not as law.

Re-check every date and threshold against the primary texts before you rely on it.

How to answer it in an interview

Interviewer asks

How would you architect an LLM for compliance work in a bank?

A strong answer

I would start by asking what the model does, because the use sets the tier. A summarizer that drafts text for an analyst is lower tier than anything touching a credit decision. Then I put a gateway in front of the model that checks the user's permissions before retrieval, redacts personal data and logs every call. Output is untrusted until validated, and high-impact actions need human approval. I keep a decision record with versions, sources and reviewer, and route low-confidence drafts to people. Published summaries say SR 26-2, the 2026 replacement guidance, leaves generative AI out of scope, so I would confirm that and let the bank's own policy carry the controls.

The spoken structure: ask about the use, name the tier, put controls outside the model, show the evidence trail, then say what the guidance leaves to the bank.

Common mistakes

  • "SR 11-7 governs our LLM." (wrong) It was replaced in April 2026, and summaries of its successor, SR 26-2, report that it leaves generative AI out of scope. (right) "I use its disciplines of inventory, independent review and monitoring under the bank's own AI policy."
  • "A human in the loop makes it compliant." (wrong) Article 14 asks for effective oversight. (right) "Reviewers get the sources, time and authority to reject, and I sample to prove it works."
  • "The system prompt forbids that, so it is safe." (wrong) Prompt injection can override it. (right) "I enforce permissions in the gateway and the tools, outside the model."
  • "Accuracy testing is enough for credit use." (wrong) A credit-tier use also needs bias and fair-lending testing. (right) "I test outcomes across customer groups and keep the results with the validation file."
  • "We log the chat transcript." (wrong) A transcript cannot show which source or version produced the output. (right) "I keep a decision record with versions, sources, checks and the reviewer."

Follow-up questions

Does SR 26-2 apply to LLMs?

No, not directly: The World of AI says it places generative and agentic AI outside its scope, and Domino says it tells banks to apply existing risk practices. Fall back on the bank's own policy and the NIST profile.

Is an LLM that only drafts text out of the EU AI Act's scope?

No, the AI Act classification turns on the use, so a draft that supports a creditworthiness evaluation can still be high-risk. The fraud exclusion is specific to Annex III point 5(b), and counsel makes the final call.

What should the audit log of an LLM in a bank contain?

The audit log should contain references to the case, retrieved sources and versions, prompt and model versions, output, guardrail results, reviewer and final decision. Article 12 asks for events relevant to risk and monitoring, not a saved chat.

How do you stop prompt injection in a bank?

You cannot rely on the prompt. Authorize before retrieval, send the model only the fields it needs, treat output as untrusted and run tools with least privilege, as the LLM gateway questions describe.

What changes if the assistant can take actions?

The tier rises, because the system acts without a person reading each output. Tools get minimum permissions, high-impact actions need approval and every action enters the decision record; the agentic RAG page covers the pattern.

Do you even need retrieval for this?

Not always: if the answer is fixed policy text, a rules engine or a template can beat a model in cost and audit effort. The do-you-even-need-RAG guide gives the decision.

Key takeaways
  • Start with the use: the rules that apply and the materiality tier depend on it.
  • SR 11-7 is superseded, and summaries of SR 26-2 report that it leaves generative AI out of scope, so the bank's own governance carries LLM controls.
  • Put permissions, redaction and approval in the gateway and tools, never in the prompt.
  • Keep a decision record of versions, sources, checks and reviewers, not a chat log.
  • Make oversight effective by routing, sampling and giving reviewers authority.

Sources

  1. SR 11-7: Guidance on Model Risk Management — Board of Governors of the Federal Reserve System, accessed 2026-10-26, 4 Apr 2011
  2. Interagency Guidance on Third-Party Relationships: Risk Management (SR 23-4) — Board of Governors of the Federal Reserve System, accessed 2026-10-26, 7 Jun 2023
  3. CFPB Circular 2022-03: Adverse action notification requirements in connection with credit decisions based on complex algorithms — Consumer Financial Protection Bureau, accessed 2026-10-26, 26 May 2022
  4. SR 11-7 and SR 26-2 compliance for AI — The World of AI (secondary), accessed 2026-10-26, 17 Oct 2026
  5. SR 26-2 — Domino Data Lab (secondary), accessed 2026-10-26, 17 Apr 2026
  6. EU AI Act, Article 14: Human oversight — Future of Life Institute, annotated text of Regulation (EU) 2024/1689, accessed 2026-10-26, 12 Jul 2024
  7. EU AI Act, Article 12: Record-keeping — artificial-intelligence-act.com, text of Regulation (EU) 2024/1689, accessed 2026-10-26, 12 Jul 2024
  8. EU AI Act for banks and insurers — vdf.ai (secondary), accessed 2026-10-26, 2 Oct 2026
  9. AI credit scoring under the EU AI Act — ComplyLayer (secondary), accessed 2026-10-26, 1 Sep 2026
  10. Artificial Intelligence Risk Management Framework: Generative AI Profile (NIST AI 600-1) — NIST, accessed 2026-10-26, 26 Jul 2024
  11. OWASP Top 10 for LLM Applications: mitigations — Superwise (secondary), accessed 2026-10-26, 26 Oct 2026
  12. OWASP LLM Top 10 for 2026 — HackerDNA (secondary), accessed 2026-10-26, 1 Oct 2026