How Kepler Built Verifiable AI for Financial Services — Vinoo Ganesh

Actionable Insights

• Separate “model decides” from “code computes.” For any financial workflow, force the LLM to output a plan, formula, source reference, or tool call—not the final numeric answer. Implement a deterministic computation layer in Python/SQL/Excel formulas that pulls verified inputs and returns the number. First experiment: ask the model to identify net margin inputs, then run net_income / net_sales in deterministic code. Evaluate by replayability, exact match against filings, and whether an auditor can reproduce the answer.
• Build an atomic provenance ledger for every extracted number. Kepler’s pattern is: model chooses a source address; a deterministic system writes/validates the value; mismatches are stripped before reaching the analyst. Start with SEC 10-K/10-Q XBRL fields, accession number, filing date, table/line item, unit, period, and transformation ID. Caution: citations alone are not enough; a citation says where a claim may have come from, while provenance plus validation proves the value was read and transformed correctly.
• Create scope-deterministic tools for ratios and adjustments. Ratios such as gross margin, EBITDA adjustments, and enterprise value differ by firm. Put the firm’s approved definitions in versioned code or config (metrics/gross_margin.yaml, adjustments/ebitda_policy.md, tests). The LLM can select the applicable policy, but the deterministic layer must compute and log every input. Evaluation criteria: same inputs always yield the same result, policy version is recorded, and analyst overrides are visible.
• Use model-risk-management controls from day one. Financial institutions already have model risk expectations: governance, validation, documentation, monitoring, and controls. Map each AI workflow to NIST AI RMF / NIST GenAI Profile concepts and banking model-risk guidance such as SR 11-7 / OCC 2011-12. Minimum checklist: source whitelist, transformation tests, human signoff thresholds, audit log, exception queue, and periodic validation.
• Prefer “AI analyst assistant” over “AI portfolio manager.” In Q&A, Vinoo says customers want repeatable painful analyst tasks automated, not autonomous PM replacement. Start with earnings-call extraction, first-pass model building, 8-K/10-K comparison, and memo evidence tables. Success means analysts recover time while retaining judgment; failure means unreviewed generated work starts driving decisions.

Core thesis

Kepler’s thesis is that financial AI cannot become trusted work product through citations, evals, or better prompting alone. LLMs should reason, plan, and select sources, while deterministic systems perform retrieval, arithmetic, provenance tracking, reconciliation, and auditability.

Big ideas / key insights

• AI changed the bottleneck from writing to reading/verifying. Models can produce work quickly, but reviewing correctness becomes the expensive step.
• Citations are after-the-fact audit, not verification. A source link does not prove the extracted number or transformation is correct.
• Verification is path-dependent. It lives in the chain of sources, transformations, policies, and firm-specific rules.
• Finance needs subjective policy plus objective replayability. Two desks can disagree with the same data, but each should be able to prove its process.
• Token maxing will be replaced by right-tool-for-job architecture. Use CPU/database/code for deterministic tasks; reserve LLMs for interpretation and planning.

Best timestamped moments

• 1:33 — The talk reframes the finance-AI problem from “produce more” to “trust what was produced.”
• 5:39–6:40 — Vinoo distinguishes citations from deterministic verification with the 10-K revenue-number example.
• 8:12–8:44 — Strong claim: you cannot use AI alone for verifiable financial work product; it needs a deterministic substrate.
• 9:14–11:45 — Kepler’s three-part architecture: atomic provenance, scope determinism, derivation chains.
• 12:16–13:18 — The model decides what to compute, but never does the computation.
• 20:57–21:57 — Q&A clarifies customer demand: AI analyst workflows, not AI portfolio managers.

Practical workflow

1. Inventory target work products: DCF, comp table, investment memo, earnings-call summary, covenant review.
2. Mark every output field as source-extracted, derived, judgment-based, or narrative.
3. For source-extracted fields, store accession/source/page/table/line/unit/period.
4. For derived fields, implement deterministic functions with versioned formulas and tests.
5. For judgment/narrative fields, require citations plus analyst review.
6. Log derivation chains and allow replay from raw source to final memo.
7. Strip or quarantine any value that fails deterministic validation.

Comment insights

The only extracted comment is an event promotion from @aiDotEngineer for an AI Engineer New York finance event. It signals track relevance but adds no technical agreement, disagreement, or practitioner caveat. No substantive comment-derived workflow evidence was available.

Deep research

• Support: Kepler/Anthropic case-study basis exists. The talk references Anthropic’s customer profile on Kepler. The public Claude page title is “How Kepler built verifiable AI for financial services with Claude,” and search/result snippets describe Kepler combining frontier language models with deterministic code for data retrieval, calculation, and citation.
• Support: regulatory and risk frameworks emphasize governance, validation, and traceability. NIST’s AI RMF Generative AI Profile (NIST AI 600-1, published July 26, 2024; updated April 8, 2026) is a cross-sector profile for managing GenAI risks. SR 11-7/OCC model-risk guidance is widely used in banking for model governance, validation, and documentation. These support the claim that regulated finance needs more than fluent outputs.
• Support: SEC concern about AI claims and accountability. SEC leadership has warned against “AI washing” and emphasized truthful disclosure and controls around AI use. This supports caution around unverifiable AI systems in finance.
• Contradiction/caution: deterministic substrate does not solve judgment risk. A verified number can still be used in a bad model, with a contested adjustment, stale policy, survivorship bias, or non-compliant workflow. Verification improves data integrity; it does not guarantee investment truth.
• Contradiction/caution: extraction accuracy claims need independent validation. The slide shows 94% first-pass line-item match and frontier-model baselines around 38–46%, but this appears to be Kepler/Anthropic-reported rather than independently audited in the artifacts available here.

Verdicts on major claims

• “Evals are not verifiability.” — Agree, high confidence. Evals estimate behavior under test conditions; they do not prove any particular output is correct. Practical takeaway: keep evals, but add per-output validation.
• “Citations get only halfway there.” — Agree, high confidence. Citations help review, but deterministic extraction and transformation checks are needed for numeric work product.
• “LLMs should not compute financial numbers.” — Agree, high confidence for production finance. Use LLMs for planning and interpretation; use deterministic code/databases for arithmetic, parsing, and persistence.
• “This generalizes beyond finance to legal/science.” — Mixed, medium confidence. Provenance and deterministic extraction generalize well; domain-specific truth standards and judgment layers differ significantly.
• “AI portfolio manager is not what customers want; AI analyst is.” — Agree, medium confidence. The Q&A supports this from Kepler’s customer conversations; broader market evidence may vary.

Screen-level insights

• 0:33 — “About me” slide lists Palantir, Verast, Citadel, and advisor/builder background, establishing credibility in verifiable, high-stakes data systems.
• 1:03 — “Where this talk comes from” slide cites Anthropic’s Kepler case study and shows scale claims such as 26M SEC filings, 14,000+ companies, 27 markets, and built in under 3 months.
• 2:05 — “Look at this track” slide highlights that most talks focus on producing more, not trusting outputs; it visually frames the talk’s contrast.
• 4:37 — “Why verification matters in finance” slide states that a confident number can be simply wrong and must trace to source and checks.
• 11:15 — “What atomic provenance buys you” slide shows a model-written value, provenance ledger, deterministic check, and ship/strip outcomes. This is the clearest architecture diagram.
• 12:47 — “Scoped determinism: the model never computes” slide separates model planning from deterministic code computing net margin.

My read / why it matters

This is the strongest architecture talk in the batch. The practical lesson is not finance-specific: use LLMs where ambiguity is useful and deterministic systems where correctness is mandatory. The best AI systems will increasingly look like orchestration layers around typed tools, provenance stores, replay logs, and human judgment gates.

Verification notes

Checked transcript, comment sample, key-frame analysis, the Claude/Anthropic Kepler page title/snippets, NIST AI RMF GenAI Profile publication page, and known banking model-risk guidance. Source/evidence audit labeled Kepler performance numbers as vendor/case-study evidence. Transcript/comment/frame fidelity audit matched provenance and scoped-determinism slides to nearby transcript. Hallucination/overclaim audit separated numeric verification from investment correctness. Actionable Insights audit confirmed the top section is executable, includes tools/patterns, checks, evaluation criteria, and cautions. Residual uncertainty: the full Anthropic article body was not extractable through the fetcher, and comment evidence was minimal.