← Back to library

Learned Execution Graphs for Anomaly Detection & Drift in APIs — Ritvik Pandya, JP Morgan Chase

Watch videoView transcript

AI Engineer19m 38sTranscript ✅Added Jul 30, 9:47 pm GMT+8

Analyzed: 2026-07-30

Actionable Insights

  1. Emit an execution DAG per request. Create nodes for service/operation calls and edges for causal or parallel relationships. Attach trace/span IDs, timestamps, status, retry count, deployment version, and privacy-safe context features. OpenTelemetry traces are a practical input; avoid storing raw sensitive payloads.
  2. Build baselines by comparable cohort. Group graphs by endpoint, version, region, tenant class, and feature flags before learning normal structure/latency. Otherwise legitimate variants look anomalous. Keep a minimum sample threshold and explicitly handle rare paths.
  3. Separate structural from performance drift. Structural checks detect added/removed/reordered nodes and changed retry loops; performance checks detect node/edge latency changes. Alert with the divergent subgraph, not only a scalar score, so operators can act.
  4. Use canaries to validate graph changes. The talk recommends small rollout percentages. Compare canary graph distributions with control before 100% rollout; gate on error rate, critical-path latency, unexpected nodes, retry amplification, and context-schema compatibility.
  5. Evaluate alert usefulness, not detection volume. Replay known incidents and controlled changes. Track precision, recall, lead time, false alerts per service-day, and mean time to locate the responsible node. Provide a fallback to conventional tracing and SLO dashboards.

Core thesis

Representing each request as a short-lived execution DAG makes order, parallelism, retries, context propagation, and latency attribution explicit. Learned baselines over these graphs can detect structural anomalies and gradual drift that aggregate endpoint metrics hide.

Big ideas / key insights

  • Emit an execution DAG per request: Create nodes for service/operation calls and edges for causal or parallel relationships. Attach trace/span IDs, timestamps, status, retry count, deployment version, and privacy-safe context features. OpenTelemetry traces are a practical input; avoid storing raw sensitive payloads.
  • Build baselines by comparable cohort: Group graphs by endpoint, version, region, tenant class, and feature flags before learning normal structure/latency. Otherwise legitimate variants look anomalous. Keep a minimum sample threshold and explicitly handle rare paths.
  • Separate structural from performance drift: Structural checks detect added/removed/reordered nodes and changed retry loops; performance checks detect node/edge latency changes. Alert with the divergent subgraph, not only a scalar score, so operators can act.
  • Use canaries to validate graph changes: The talk recommends small rollout percentages. Compare canary graph distributions with control before 100% rollout; gate on error rate, critical-path latency, unexpected nodes, retry amplification, and context-schema compatibility.

The durable theme is learned execution graphs for API anomaly and drift detection. The video is most useful as an architecture and risk discussion, not as proof that a product or pattern universally works.

Best timestamped moments with interpretation

  • 0:00 — [music] Hi. Uh thanks thanks and uh hope everyone is out of uh the lunch coma and we’ll survive this talk. So uh yeah myself Ritik I uh lead the payments team in uh JP Morgan and uh today I’ll be Interpretation: this is a transcript-backed point; the operational implication is to convert it into a measurable control or experiment rather than accept the presentation claim at face value.
  • 0:00 — request is taking this much time normally and uh client B’s request could take might take more time than the client A um because of say one client is local to you and one client is uh you know the request is coming from outside an Interpretation: this is a transcript-backed point; the operational implication is to convert it into a measurable control or experiment rather than accept the presentation claim at face value.
  • 0:00 — request or the volume which you are expecting now over the time. So yeah, such kind of drifts you might need to treat differently because now you need either you need to scale up those services or instances of those services and o Interpretation: this is a transcript-backed point; the operational implication is to convert it into a measurable control or experiment rather than accept the presentation claim at face value.
  • 0:00 — generally use uh everyone in the industry is asynchronously feeding the data to uh opal telemetry from there u some kafka could be used and stream assessment uh could be used on top of it. There could be two different paths. One i Interpretation: this is a transcript-backed point; the operational implication is to convert it into a measurable control or experiment rather than accept the presentation claim at face value.
  • 0:00 — [music] Interpretation: this is a transcript-backed point; the operational implication is to convert it into a measurable control or experiment rather than accept the presentation claim at face value.

Practical takeaways / recommended workflow

  1. Define the decision/task, risk level, owner, and measurable acceptance criteria.
  2. Capture the existing baseline for quality, latency, cost, failures, and human review time.
  3. Implement the smallest reversible version with typed inputs/outputs, logs, and least privilege.
  4. Replay representative historical cases plus adversarial and missing-data cases.
  5. Run shadow mode or a small canary; compare results and inspect every failure.
  6. Expand only when benefits persist without increasing downstream review, security, or operational burden.

Comment insights

  • 0 likes — @dkierans: This is nice. Especially the assembly part.

Comments are audience evidence, not factual verification. Where comments challenge missing demos, sales framing, tool efficiency, or implementation detail, that pushback lowers confidence in broad claims but does not by itself refute the underlying architecture.

Deep research on the creator’s main claims

The strongest claims were checked against named external sources below. Because the configured web-search provider was quota-blocked during this run, I used stable official specifications, project repositories, and standards already identifiable from the topic; I did not convert unverified snippets into facts.

Supporting evidence: these sources support the underlying patterns—explicit verification, event/read models, least privilege, provenance, graph retrieval, typed orchestration, distributed tracing, and measured rollouts.
Contradicting or limiting evidence: none of them establishes the speaker’s vendor-specific metrics or proves that the proposed architecture beats simpler baselines in every environment. Added complexity, stale data, extraction errors, policy bypasses, coordination overhead, and operational cost remain material counterweights.
Verified fact vs interpretation: transcript and frames verify what was presented; external documentation verifies that the referenced patterns/tools exist. Performance, prevalence, and superiority claims remain interpretations unless a reproducible benchmark or primary study is cited.

My verdicts on major claims

1. Execution DAGs make request behavior easier to reason about

Verdict: Agree — high confidence. This is a natural representation of distributed traces and workflow execution.

Practical takeaway: test this claim against a documented baseline and preserve rollback/approval paths.

2. Learned graphs can detect anomalies and drift

Verdict: Agree with caveats — medium confidence. They can, but cohorting, seasonal behavior, topology changes, and labels determine usefulness.

Practical takeaway: test this claim against a documented baseline and preserve rollback/approval paths.

3. Graph methods reduce manual diagnosis

Verdict: Mostly agree — medium confidence. Divergent subgraphs improve localization; they do not replace logs, metrics, domain knowledge, or incident response.

Practical takeaway: test this claim against a documented baseline and preserve rollback/approval paths.

4. A 5–10% canary is sufficient

Verdict: Mixed — low confidence as a universal claim. Sample size and risk matter more than a fixed percentage; use statistical power and blast-radius constraints.

Practical takeaway: test this claim against a documented baseline and preserve rollback/approval paths.

Screen-level insights

  • 0:32 — youtube-extract/u1yaOeEX4e8/frames/000_000032.jpg: talking about learn execution graphs how these graphs can help to uh detect any anomaly and uh drifts also how we can automate few things around that and uh you know u uh at the same time if we can reduce the manual you know detection work and uh going on that side uh so whenever. The frame anchors the spoken claim to the visible slide/demo; it matters because reviewers can distinguish an architecture diagram, metric, policy screen, or live workflow from narration alone.
  • 1:04 — youtube-extract/u1yaOeEX4e8/frames/001_000064.jpg: hear about graph uh there are persistence graph and property graphs uh which Neo4j and you know other products uh we use for them we query those uh graphs and get the answers out of it what I’m talking about today is execution graph. It’s short-lived graph. Uh and idea here is ho. The frame anchors the spoken claim to the visible slide/demo; it matters because reviewers can distinguish an architecture diagram, metric, policy screen, or live workflow from narration alone.
  • 2:07 — youtube-extract/u1yaOeEX4e8/frames/003_000127.jpg: could be called in parallel uh once everything is done you are notifying your client that what’s the update on that request right So here the idea is representing the uh overall request processing as DAG and using tag simplifies most of the things here. one now you know that uh i. The frame anchors the spoken claim to the visible slide/demo; it matters because reviewers can distinguish an architecture diagram, metric, policy screen, or live workflow from narration alone.
  • 2:37 — youtube-extract/u1yaOeEX4e8/frames/004_000157.jpg: right so that’s one of the thing the other thing is uh you know the context that in at what node what context will be there and what will be passed to the next node uh in that way it will be very um u ordered and simplified u uh simply can be represented uh there are few other uh. The frame anchors the spoken claim to the visible slide/demo; it matters because reviewers can distinguish an architecture diagram, metric, policy screen, or live workflow from narration alone.
  • 7:21 — youtube-extract/u1yaOeEX4e8/frames/012_000441.jpg: So uh again uh basic thing here is what is anomaly and what is drift right? So say you are driving from your home to office every day and uh one of and usually it takes 1 hour but uh one fine day it took you more time than 1 hour. Uh the reason might be some traffic or you know u. The frame anchors the spoken claim to the visible slide/demo; it matters because reviewers can distinguish an architecture diagram, metric, policy screen, or live workflow from narration alone.
  • 13:40 — youtube-extract/u1yaOeEX4e8/frames/015_000820.jpg: 5% or 10% of your uh machines monitor it verify everything looks good and then you roll out for your 100% of the nodes Here are a couple of example. uh again uh say generally overall request processing takes 700 millisecond but uh on the left side what you are seeing is u in the . The frame anchors the spoken claim to the visible slide/demo; it matters because reviewers can distinguish an architecture diagram, metric, policy screen, or live workflow from narration alone.

My read / why it matters

This talk matters because it shifts attention from model fluency to system design. The practical question is not whether an agent can produce a plausible answer once; it is whether a team can constrain inputs and authority, recover the evidence, detect failure quickly, and demonstrate better outcomes than a simpler workflow. Adopt the narrow mechanism that survives replay tests and operational review—not the conference-level slogan.

Verification notes

Four explicit passes were completed before publication:

  1. Source/evidence audit: major claims were separated into transcript-backed presentation claims, externally corroborated patterns, and unverified metrics. Named sources and direct links are included above.
  2. Transcript/comment/frame fidelity audit: timestamp selections come from extracted transcript chunks; comment summaries preserve their status as opinions; screen descriptions reference extracted frame paths and nearby transcript rather than guessing unseen UI details.
  3. Hallucination/overclaim audit: vendor statistics and universal superiority claims were downgraded where primary methods/results were unavailable. No install command or product capability was asserted without a stable source.
  4. Actionable Insights audit: all five top items specify a first implementation move, evaluation criteria, and a caution, prerequisite, or rollout boundary; generic summary bullets were rejected.

Residual uncertainty: automated captions may contain name/product errors; web search was quota-blocked; referenced tools were not executed in this analysis; and vendor-reported studies should be rechecked against their primary methodology before procurement or policy decisions.