Week 19 / Case Studies

The incident is already live.

Your conversational assistant has shipped. At 10:07, latency doubles, tool calls begin timing out, and the support queue lights up. You have one architecture review before the next deploy. Where do you look first?

The clue is not “the model is slow.” Production AI is a system: context assembly, policy, tools, queues, fallbacks, and telemetry all shape the experience.

Live telemetry

2.4×

p95 response latency

user→context→model→tool?

Your job: reason from the architecture, not from a single component.

2.4×latency spike

38%queue pressure

7tool timeouts

02 / THE SYSTEM LENS

Stop drawing “the model.” Draw the request trace.

A modern AI product is a coordinated runtime. The visible answer is the last event in a chain of context, policy, tools, and measurement.

request→context→policy→model→tools→response

State

Conversation history, user profile, retrieved evidence, and task state.

Control

Permissions, budgets, safety checks, and human escalation.

Feedback

Latency, traces, evals, cost, and user outcomes.

requestcontextpolicytoolsresponse

Trace the failure

Click the span where you would instrument first for a tool timeout.

Choose a span. Experts instrument the boundary where the symptom occurs, then inspect upstream causes.

1 / 12 tutorial pages

03 / CONVERSATIONAL AI

Conversation is a state-management problem with a voice.

Leading assistants separate the short-lived turn from durable memory. They stream early, retrieve selectively, guard tool use, and preserve a handoff path when confidence drops.

Peel the stack

Turn state keeps the current exchange small enough to stream quickly: latest user intent, tool results, and a response plan.

Layered conversational AI architecture diagram

The product surface is a conversation; the runtime is layered state.

Stream

Send a useful partial answer while slower retrieval or tools finish.

Ground

Attach evidence to claims instead of trusting a long context window blindly.

Escalate

Treat handoff as a designed state, not an exception thrown at the end.

The architecture behind “chat”

04 / CODING AGENTS

A coding agent is a controlled loop, not autocomplete.

The reliable pattern is plan → inspect → edit → run → observe → revise. The sandbox and repository view are as architectural as the model.

plan→edit→test→observe

Sequence a safe patch

Click the next step in the loop. A production agent should not jump from request straight to code mutation.

Start with repository inspection. The agent needs a grounded view before it proposes a patch.

Design crux: give the agent narrow tools, observable state, and a reversible workspace. “More autonomy” without containment is just more blast radius.

Loop mechanics

KNOWLEDGE CHECK 01

Spot the architectural mistake

A coding agent edits production files before it has inspected the repository or run tests. What failed?

Formative · not scored

05 / DEEP RESEARCH

Research quality comes from orchestration, not one giant prompt.

Deep research products decompose a question, run source-finding in parallel, track evidence, resolve conflicts, and synthesize with citations. The architecture makes uncertainty visible.

question→decompose→parallel search→evidence graph→synthesis

Move the research clock

35% · source discovery

Multiple search workers are still expanding the evidence set.

Watch the question fan out, then converge through evidence.

Why citations are architectural: they require provenance to survive every transformation—from source fetch to extraction to claim selection to final prose.

Parallelism + provenance

06 / BROWSER & COMPUTER USE

Seeing is not acting. Computer-use agents need a safety membrane.

Browser computer-use agent with accessibility overlay

Observation can combine pixels, DOM structure, accessibility data, and action history.

A browser agent repeats observe → plan → act, but every action crosses a permission boundary. Safe systems preview risky actions, constrain destinations, and log the exact state that produced a click.

The pause before action is a feature, not latency to hide.

Choose the observation channel

For a button whose visual position moves, accessibility structure gives the most stable target.

Observe → plan → act

07 / MULTIMODAL AI

Multimodality is an interface problem before it is a model problem.

Production applications normalize images, audio, documents, and text into representations the reasoning core can use. They also preserve modality-specific uncertainty and tool outputs.

cameramicrophonedocumenttext→adapters→shared state→reasoner

Route the signal

Which adapter should own each transformation?

Pick a signal to see why modality-specific preprocessing remains valuable.

Multimodal adapters merging into shared representation

The shared representation is a contract between specialized front doors and a general reasoning core.

Adapters are architecture

KNOWLEDGE CHECK 02

Match the design decision

A multimodal assistant receives a scanned invoice. Which pipeline is the strongest default?

Formative · not scored

08 / ORCHESTRATION PATTERNS

Pick the control topology that matches the uncertainty.

There is no universal “agent architecture.” A supervisor gives global control; handoffs keep specialists focused; a graph makes transitions explicit and testable.

Three topologies, three control surfaces.

Make the routing call

You need a regulated support assistant with a fixed sequence, audit checkpoints, and no free-form delegation.

Choose a topology. The right answer is the one whose transitions you can explain and test.

Rule of thumb: orchestration is a reliability boundary. Every hidden loop is a hidden cost, latency, and failure mode.

Control topology

09 / SCALABILITY

Scale the queues around the model, not just the model.

Scalable AI infrastructure with queues and workers

The model worker is one stage inside a larger throughput system.

A responsive AI product separates interactive work from long jobs, caches safe repeats, batches compatible requests, and isolates tools. Backpressure is a product decision: it determines what the user sees when capacity is scarce.

Tune the workload mix

50% interactive · 50% async

Balanced mode: stream short answers while queueing expensive jobs.

Cache

Reuse safe, stable work; never cache user-specific secrets.

Batch

Trade a little latency for much higher accelerator utilization.

Isolate

Keep browser, code, and file tools from sharing uncontrolled state.

Latency is a topology

10 / RELIABILITY

Reliable AI is engineered around the model’s uncertainty.

Retries need idempotency. Fallbacks need compatible contracts. Evals need traces, not just final text. Human review needs a clear trigger. Reliability is the set of paths taken when the happy path stops being true.

Ambiguous side effect

timeout→inspect status→retry safely

Guardrail

Idempotency turns uncertainty into a recoverable state.

Spot the unsafe retry

A payment tool times out after the request may have succeeded. What should the agent do?

AI reliability loop with retries and human review

A fallback is useful only when its contract and observability are explicit.

Failure paths are first-class

KNOWLEDGE CHECK 03

Reliability under ambiguity

Why is “just retry the tool call” unsafe for a payment action?

Formative · not scored

11 / ARCHITECTURAL TRADE-OFFS

Every “best” architecture spends a budget somewhere.

Latency, quality, cost, and control are coupled—not independent toggles.

The mature question is not “which stack is smartest?” It is “which trade-off is acceptable for this user, action, and failure mode?”

Set the product dial

65 · quality/control leaning

Use deeper reasoning, stronger evidence, and approval gates; accept more latency.

FAST

Short context, streaming, cache, smaller model.

DEEP

Decomposition, parallel research, verification, citations.

CONTROLLED

Narrow tools, explicit graph, approvals, audit trail.

Choose the budget consciously

12 / CAPSTONE BLUEPRINT

Design a research-backed coding assistant.

A user asks: “Find the relevant API change, explain the evidence, patch my repo, and prove the tests pass.” Assemble the architecture that makes that promise credible.

Research-backed coding assistant architecture blueprint

One product promise, many cooperating boundaries.

Select the non-negotiables

0 / 4 selected

A credible system needs all four for this high-stakes promise.

Synthesis: conversational UX, coding loops, deep research, browser use, multimodality, orchestration, scale, and reliability are not separate tricks. They are composable boundaries around uncertainty.

Case study synthesis

KNOWLEDGE CHECK 04

Architect’s final call

Which design best combines the case-study patterns for a research-backed code change?

Formative · not scored

GATED ASSESSMENT

Five architecture calls. 80% to pass.

Each question asks you to apply a production pattern to a concrete system. Choose the answer that makes the boundary, trade-off, or failure path explicit.

05

questions

80%

pass threshold

01

best answer each

Review the core ideas.
Connect concepts to practice.
Prepare for assessment.

ASSESSMENT Q1 / 5

A conversational assistant must answer quickly but can spend longer on a cited research task. Which architecture supports both?

Q1 of 5

ASSESSMENT Q2 / 5

A coding agent is asked to refactor a payment service. Which control is most important before it edits?

Q2 of 5

ASSESSMENT Q3 / 5

A browser agent must submit a form that triggers an irreversible action. What is the strongest design?

Q3 of 5

ASSESSMENT Q4 / 5

A tool call times out after a side effect may have occurred. Which reliability pattern applies?

Q4 of 5

ASSESSMENT Q5 / 5

Which statement best captures the architectural trade-off lesson?

Q5 of 5

RESULTS / ARCHITECTURE REVIEW