The fast loop belongs to the agent: observe, decide, act, verify, repeat. The slower loop belongs to the human: set intent, define guardrails, evaluate outcomes, and change the system when the pattern of results says it should change.

When people say "the agent did the work," what they usually mean is that the agent handled the short-cycle execution path. The human still shaped the mission, constrained the operating envelope, and decided whether the results were acceptable. That is not removal. That is promotion.

The old mental model was simple: human thinks, human acts, software assists.

The emerging model is different: human defines the frame, agent runs inside it, and software records enough state for the system to be inspectable and correctable.

That changes the role of human judgment. It matters less in the moment-to-moment mechanics of execution and more in the structure around execution.

The new human job is not to click faster than the machine. It is to decide:

• What outcome actually matters

• What constraints cannot be violated

• What tradeoffs are acceptable

• What signals mean the system is drifting

• When the frame itself needs to change

This is a more strategic role, but it is also a more demanding one. A human in the strategic loop cannot hide behind activity. The system exposes whether the goals, thresholds, and review criteria were well chosen.

That is why the future of agents is not just about autonomy. It is about governance with shorter feedback cycles.

An agent can iterate through a task dozens or hundreds of times faster than a person. It can retry, branch, compare, and recover without waiting for a human to press the next button. But that speed is only useful if someone has defined the objective and the stopping conditions well enough for the fast loop to stay productive.

Without that frame, autonomy degrades into expensive motion. The agent looks busy. Logs fill up. Tools fire. Tokens burn. But the system is not converging on anything worth keeping.

The practical implication is that good human operators will start to look less like operators and more like system designers.

They will spend more time on:

• Intent definition

• Policy and permissions

• Success metrics

• Review checkpoints

• Exception routing

• Post-run diagnosis

And less time on:

• Manual execution

• Repetitive coordination

• Low-level tool invocation

• Short-horizon status checking

This is not a hand-wavy organizational shift. It maps cleanly to the engineering structure of an agent system.

At the execution layer, the agent runs a tight control loop. It reads the current state, chooses an action, calls a tool, checks the result, updates the state, and continues.

Above that sits a control plane shaped by humans: goals, policies, task boundaries, escalation rules, budgets, and evaluation criteria.

Above that sits an even slower loop: review the outcomes across many runs, identify patterns of failure or waste, and redesign the frame.

In other words, humans move from the transaction layer to the architecture layer.

That does not make humans less important. It makes their mistakes more consequential and their judgment more leveraged.

A weak worker can be corrected one task at a time. A weak frame gets multiplied across every autonomous run.

That is the real shift agents introduce. They amplify not only execution, but also whatever upstream judgment shaped the execution.

The organizations that benefit most from agents will not be the ones that simply automate the most steps. They will be the ones that become good at designing loops.

They will know how to:

• Give agents clear intent without over-specifying every action

• Set hard guardrails without freezing the system

• Review outputs at the right altitude

• Turn repeated failures into better policies

• Decide when a human must step back into the fast loop

This also explains why "human in the loop" is now too vague a phrase. The real question is: which loop?

Are humans approving every action in the fast path? Are they only reviewing samples after execution? Are they setting policy weekly and auditing exceptions daily? Are they redesigning the workflow monthly based on aggregate outcomes?

Those are very different systems, and they produce very different economics.

The companies that win with agents will treat loop design as a core capability. They will explicitly decide what runs at machine speed, what gets escalated, what gets logged, what gets measured, and what gets changed when the results stop matching intent.

That is the future of agents.

Not humans removed from the loop.

Humans promoted to a slower, higher-leverage one.

The Problem With Conventional Agent Frameworks

Most agent frameworks are LLM-first: the conversation loop is the core, tools are attached to it, rules layer on top, and logging is bolted on at the end for observability. State is persisted as retrievable "memory."

Figure 1. Conventional LLM-centric stacks treat logging as a secondary concern. In a log-centric design, the append-only event log becomes the source of truth, the working graph is derived from it, and the LLM is just one behavior among many.

Consequences of this design:

• State is opaque: working memory lives in conversation context, not a queryable store.

• Replaying a run is hard: there is no source of truth; logs are side-effects, not the primary record.

• Forking is expensive: you must re-run from scratch because there is no checkpoint mechanism.

• Coordination between agents is explicit: agents pass messages directly instead of reacting to a shared world state.

• Observability is retroactive: tracing gets added after bugs appear.

The Core Inversion: Log-First Architecture

The core move is to invert the stack. Make the append-only event log the source of truth. Everything else — the working graph, the agent's beliefs, and task state — becomes a deterministic projection of that log.

The architecture shift is simple but profound: instead of treating the LLM loop as the runtime and storage as a helper, the event log becomes the runtime backbone. The live graph is derived from that log, and behaviors operate against the graph rather than owning the whole system.

The log is not observability. The log IS the agent.

A Concrete Runtime Shape

To make the idea concrete, think in terms of a runtime with three layers: authoritative events, projected graph state, and reactive behaviors.

Three-Layer Model

Figure 2. A three-layer runtime separates source-of-truth events, projected state, and reactive behaviors. Relation behaviors are first-class, and every behavior writes back to the shared event log.

Relation Behaviors — Edges as First-Class Citizens

Traditional agent graphs put all logic in nodes. A stronger design puts semantic logic on edges:

• A task can depend_on another task

• A task or belief can contradict an existing belief

• Evidence can support or weaken a claim

• Blocking, dependency, and contradiction logic live in the relationship itself

When the contradicts relation fires, a behavior automatically triggers — no explicit coordination code needed.

What the Log Enables: Four Superpowers

Deterministic Replay

Figure 3. Once execution history is stored as events, replay becomes deterministic and branch exploration becomes cheap: reuse the shared prefix, pay only for divergent work, and compare outcomes directly.

Any run can be reconstructed exactly from its log. This means:

• Full audit trail from goal to individual model call

• Debugging without re-running expensive LLM calls

• Compliance/reproducibility for regulated domains

Efficient Forking

Fork at any historical event. The shared prefix replays from cache — no redundant LLM calls for the common history. Branch, diff, compare outputs.

Complete Causal Lineage

Every event knows what caused it:

Goal: "Analyze market"
  └─ spawned Task: "Research competitors"
       └─ LLM call: "List top 5 competitors"
            └─ Tool call: web_search("competitor analysis 2026")
                 └─ Evidence node: search result
                      └─ Behavior: "contradicts" edge fired
                           └─ LLM call: "Revise belief about market stability"

No more "why did the agent do that?" — the log has full provenance.

Implicit Coordination (No A2A Protocol Needed)

Agents coordinate by reacting to the graph, not by calling each other. No explicit A2A protocol. No DAG. No workflow engine.

Deep Dive — Runtime Design Details

The Event Schema

Every event in the log carries:

• `id`:id:** unique identifier.

• `type`:type:** event category such as task.created, llm.response, or tool.called.

• `payload`:payload:** event-specific data.

• `actor`:actor:** which behavior emitted the event.

• `causality`:causality:** the triggering event or events.

• `timestamp`:timestamp:** wall-clock time of emission.

The causality chain is what makes full lineage possible — every event knows its parent.

Handling LLM Non-Determinism

The hardest problem in event-sourced agents is that LLMs are not pure functions. The practical answer is content-addressed caching keyed on a normalized hash of the request such as system message, model parameters, and output schema. On replay, a matching hash serves the cached response from the log instead of making a fresh API call.

Two replay modes:

• Permissive — serves cached responses, allows new calls for edited prompts (default)

• Strict — validates byte-for-byte reproducibility; flags the first offending event if anything diverges

Forking Economics

The cost savings from cached prefix replay are concrete:

200-step run, fork at step 150:
  ├─ Steps 0–149  →  replay from cache  →  $0, ~0ms per step
  └─ Steps 150–200 →  execute normally  →  full cost

Without forking: pay 200 steps × N variants
With forking:    pay 50 steps × N variants + 150 steps once

Fork lineage is verified by literal shared event IDs — not a copy, the same events. The structural diff then shows exactly which graph objects, relations, and patches differ between branches.

The Determinism Contract

Behaviors must not:

• Call random() directly (use event-record entropy instead)

• Read wall-clock time directly (use event timestamp)

• Generate fresh UUIDs (obtain from event records)

• Perform I/O outside framework primitives

• Depend on mutable global state

The runtime does not statically enforce this. Violations surface at replay as a divergence error pinned to the first mismatched event. For LLM calls, the contract applies to replay (via cache), not initial execution.

What a Real Run Looks Like

In a realistic agent workflow, the event stream gets large quickly. A run can look like this:

671 events
 93 objects (3 companies, 24 questions, 25 claims, 3 memos)
 76 relations
103 model calls
 48 tool calls
  0 lines of orchestration code

Coordination was entirely reactive — no explicit scheduling, no DAG, no workflow engine.

Lineage is the deliverable — every claim links back to its behavior, triggering event, and specific model request. The causal chain from goal to output is reconstructable from the log alone.

Comparison to Memory-Layer Approaches

Memory-layer systems treat memory as a derived layer atop the agent. A log-centric design inverts that relationship: the log is primary, and memory in the form of a graph is a projection of it. That means you can reconstruct any past memory state, not just the current one.

Known Limitations

This architecture still has real limitations:

• Reactive cascade risk: behaviors can trigger loops, so runs need budgets such as event caps, cost limits, and recursion depth.

• Log scaling: replay cost is linear, and the system does not yet have mature compaction or checkpointing for million-event runs.

• Schema evolution: graph schema changes require migration tooling.

• Side-effect replay: external writes and emails happen only on first run; only responses replay cleanly.

• No empirical benchmarks: the contribution is architectural rather than a head-to-head performance comparison.

Where Akka Fits — Filling the Production Gap

If you want to take this architecture into production, Akka is one of the clearest reference points for the surrounding infrastructure. It gives you battle-tested event sourcing, clustering, sharding, projections, and recovery semantics without asking you to invent those mechanisms from scratch.

Conceptual Mapping

The easiest way to read the mapping is this: the agent runtime contributes the agent-native abstractions, while Akka contributes the production-grade event-sourcing primitives. The concepts line up surprisingly cleanly.

What Akka Brings That A Minimal Agent Runtime Doesn't Have Yet

• Log scaling: Akka Persistence snapshots let you snapshot state every N events and replay only the tail.

• Schema evolution: Akka Persistence event adapters can transform older event formats on read.

• Distributed multi-agent coordination: Akka Cluster Sharding routes behaviors to stable nodes and keeps state alive across failures.

• External side-effect deduplication: Akka's at-least-once delivery model pairs with idempotent receivers so tools do not get re-executed incorrectly on recovery.

• Backpressure in reactive cascades: Akka Streams gives you typed, backpressured pipelines so behaviors cannot flood one another.

• Observability and metrics: Akka Telemetry provides message-rate, mailbox, and journal-latency visibility out of the box.

How a Production Log-Centric Agent Would Look With Akka

At production scale, you would expect persistent behavior actors writing to an event journal, projections building graph views from that journal, snapshots reducing replay cost, and a dedicated layer wrapping LLM calls as event-producing behaviors.

The Shared Philosophical DNA

Both this style of agent runtime and Akka are grounded in the same insight that the distributed systems community learned in the 2010s:

Mutable shared state is the enemy. Immutable events are the foundation.

Akka's EventSourcedBehavior (in Akka Typed) is nearly a direct implementation of a behavior attached to a projected log:

// Akka EventSourcedBehavior ≈ log-projected behavior
EventSourcedBehavior[Command, Event, State](
  persistenceId = PersistenceId("agent", agentId),
  emptyState   = AgentState.empty,
  commandHandler = (state, cmd) => Effect.persist(toEvent(cmd)),
  eventHandler   = (state, evt) => state.applyEvent(evt)   // deterministic projection
)

The difference is scope. Akka is a general distributed computing toolkit. An LLM-native runtime adds content-addressed LLM caching, a causality-aware event schema, fork-and-diff workflows, and graph projections typed around agent concepts such as beliefs, tasks, evidence, and relations.

When to Use Each

The practical split is straightforward:

• Use a purpose-built log-centric agent runtime when you want the architecture directly, fast iteration, and agent-native concepts out of the box.

• Use Akka when you already live in the JVM world and need battle-tested event sourcing, clustering, sharding, snapshots, and durable distributed runtime behavior.

• Use a hybrid approach when you want the causal and replay model of a log-centric agent runtime but need the surrounding production infrastructure that mature event-sourcing stacks already solved.

Log-Centric vs Conventional — Side-by-Side

The core difference is not UI, prompt style, or framework ergonomics. It is where truth lives and what you can do once that truth is durable.

• Source of truth: conventional systems rely on LLM conversation context; log-centric systems rely on an append-only event log.

• State: conventional state is opaque and in-context; log-centric state is explicit and queryable through the graph.

• Replay: conventional systems re-run from scratch; log-centric systems replay deterministically from the log.

• Forking: conventional branching is expensive; log-centric branching is cheap because the shared prefix is cached.

• Agent coordination: conventional systems depend on explicit messages or A2A protocols; log-centric systems coordinate implicitly through graph reactivity.

• Observability: conventional observability is bolted on; in log-centric systems the record is intrinsic.

• LLM role: conventional systems make the LLM the loop; log-centric systems treat it as one behavior among many.

• Long-running agents: conventional systems hit context-window limits; log-centric systems persist durable graph state over time.

Where This Fits in the Agent Landscape

This model sits between classic orchestration frameworks and full distributed-systems thinking. It borrows from event sourcing, stream processing, Redux-style projection, and actor systems, then applies those ideas to LLM agents in a way most agent stacks still do not.

Resources

• arXiv paper

• Active Graph site

• Active Graph GitHub

• Active Graph docs

• Tweet — paper announcement

• Tweet — better visualization

• Active Graph open source tweet

• Akka Persistence docs

• Akka Projection docs

Most teams I talk to treat evals like a test suite. You build the agent, write some tests, watch it pass them, ship it. Done.

That model is wrong — and it's why so many agents plateau after the first sprint.

Evals are not a scoreboard. They're a training signal. Your agent will hill-climb on whatever you give it to optimize for. If your evals don't encode the real behavior you want — not the proxy, not the offline approximation, the actual thing your users need — you'll ship an agent that passes your tests and fails your users.

Here's what I've learned building agents in production, and what the field is converging on in 2026.

1. Evals are the substrate

"Evals are the substrate that determines what your agent does in production. They're training data for agents because we literally fit our agent to pass Evals via hill-climbing algorithms and human edits to pass failure modes. Once you get your agent into users hands, the eval generation loop compounds."

The word "substrate" is doing real work here. Not tests. Not benchmarks. Substrate — the medium in which something grows.

Your agent grows to fit its evals. Every prompt edit, every tool tweak, every model swap — you measure the result against your evals and keep what passes. That's hill-climbing. It's happening whether you've named it or not.

The question isn't whether you're hill-climbing. It's whether you're climbing the right hill.

I've built systems where the eval was a holdout dataset — historical labels, frozen in time. We got very good at predicting the past. The metric looked great. Production results were flat. The hill we climbed wasn't the hill that mattered.

The gap between "passes offline evals" and "works in production" is exactly the gap between the frozen labels you have and the live behavior you actually care about. A user who asks a follow-up because the first answer was wrong — that's signal. A document that fails to extract correctly — signal. A recommendation that gets ignored — signal. None of those were in my eval set at first.

The moment you wire those production signals back into your evals, the hill changes shape. And the agent starts improving at what you actually care about.

2. Hill-climbing is the mechanism

"This harness optimization process is becoming much more agent driven with humans reviewing and curating evals/rewards to hill climb on. Evals are a moat and thus data to produce evals is a moat. Especially true for vertical agent companies. This is because agents can fit to most Eval sets today."

Here's the operational picture: you have an agent, you have a harness — the runtime, tools, prompts, skill definitions. You run the agent against your eval set. Some pass, some fail.

For failures, you do one of three things:

1. Edit the prompt

2. Change a tool definition

3. Add the failure as a new eval case

Then you run again. That loop — run → fail → edit → run — is hill-climbing. The harness is the surface you're optimizing over.

What I've found is that curation is the hardest part. It's easy to write a hundred evals. It's hard to write evals that are:

• Representative of what users actually encounter (not what you imagine they encounter)

• Sensitive enough to distinguish genuinely good responses from superficially good ones

• Robust enough not to be gamed by surface changes

In document processing systems I've worked on, the best evals came from the failure queue — messages that failed in production, got stuck, or produced wrong output. Those aren't hypothetical edge cases. They're real ones. Each is a new data point about exactly where the agent breaks.

That failure queue is your eval generator, if you treat it as one.

3. The compound loop

"new goal unlocked: grind so everyone forgets what a harness is and they just tell us what their agent needs to do, and we just chef it all together for them (harness + evals + envs + infra + observability + self-improvement + good-vibes)"

The vision here is compounding. Not just "evals improve the agent" — evals improve the agent → the improved agent reaches users → users surface new failure modes → those become new evals → the agent improves again.

Once this loop is running, it's hard to stop. Early on you're writing evals manually, dogfooding your own product to find where it breaks. Later, production data does most of the discovery for you. The best teams I've seen have an almost-automated pipeline from "something went wrong in prod" to "there's a new eval case in the backlog."

The key enablers:

Observability: you have to be able to see failures. Structured logging with correlation IDs, run IDs, case IDs. If you can't query "show me all the cases where the agent's output was flagged," you can't close the loop.

Categorization: not all failures are equal. Some are model failures. Some are tool failures. Some are eval failures (the eval expected the wrong answer). Knowing which is which tells you where to intervene.

Low friction to add evals: if adding a new eval case takes an hour, you won't do it for every production failure. If it takes a minute, you will. This is infrastructure, not process.

The "chef it all together" framing is the right end state: a team that describes what the agent should do, and has infrastructure that measures whether it does that thing, surfaces the gaps, and closes them. The eval harness is the product, not the afterthought.

4. Gall's Law: earn your complexity

"Gall's law: 'a complex system that works is invariably found to have evolved from a simple system that worked.' Also very relevant to agent systems. Most teams are trying to jump straight to autonomous complexity before they have evals, observability, or feedback loops in place. 2026 is the year of evals."

The agent teams I've seen struggle have almost always skipped the observable-simple-system phase. They built an orchestrator with five subagents, a memory layer, and tool use on day one. When it breaks — and it breaks — there's no way to tell which part failed. The eval is "the user seemed unhappy." That's not a training signal; it's a symptom.

My rule: before you add complexity, make sure you can see what's happening. Traffic interception, request logging, structured output validation, human review queues — pick your poison, but have something. A simple agent with good observability will outrun a complex agent with none. You can improve what you can measure. You can only guess at what you can't.

The practical implication: your first eval should be manual. A human reviewing the agent's outputs and labeling them good or bad is a valid eval harness. It's slow, it doesn't scale, but it forces you to articulate what "good" looks like. That articulation is the hard part. Everything else is tooling.

Complexity earns its place when the simple system is working and you know exactly which constraint it's hitting. Not before.

5. The frozen benchmark trap

"A benchmark is a frozen view of a solvable world. An RL environment is the same world made interactive. A harness is the runtime that lets agents act in it. A recipe is a compressed solution trajectory. A data generator is the world sampled at scale."

This mental map is the clearest framing I've seen of where evals need to go.

Most agent teams are at level 1: frozen benchmark. A test set, a score, a ship decision. The problem is that this test set was assembled from a distribution of inputs that existed at a point in time, labeled by people who had to make assumptions about what the right answer was. It doesn't capture the full distribution of what the agent will actually encounter. It captures what someone had time to label.

The unlock is level 2: interactive environment. Instead of "predict the right answer for these labeled inputs," it's "act in this environment and we'll observe what happens." The eval stops being a static check and starts being a simulation of real usage. Failures are now observable behaviors, not output mismatches.

From there, the natural progression:

• Harness: you build the runtime around the environment. Tools, skills, APIs, prompts — the harness is the surface the agent operates on, and the surface you optimize over.

• Recipe: accumulated knowledge compressed into structure. A wireframe template, a skill definition, a system prompt that encodes what good behavior looks like. Recipes are what you get when a domain expert and an agent engineer work together long enough.

• Data generator: at scale, the world generates evals for you. Production traffic, user interactions, edge cases you didn't anticipate — they surface continuously and become the raw material for the next round of eval curation.

Most teams in 2026 are at level 1. The best teams are building toward level 5. The distance between them is compounding every month.

6. Evals as measurement infrastructure

"Evals, benchmarks, and RL environments. Measurement infrastructure that decides whether agents are safe to ship."

"Measurement infrastructure" — not tests, not QA, infrastructure — is the right framing.

Infrastructure implies:

• Built once, used many times

• Maintained and improved as the system evolves

• A prerequisite for everything that depends on it

• Skimping on it creates debt that compounds

When I look back at agent systems I've built that stayed healthy versus ones that rotted quickly, the difference is almost always measurement infrastructure. Systems with eval harnesses improved iteratively. Systems without froze at their initial performance level because there was no way to know if a change was an improvement or a regression.

The "moat" framing is real. If your eval set encodes all the good behavior your agent needs to exhibit — edge cases, difficult inputs, real failure modes from production — that set is genuinely hard to replicate. It took real production data and real failure events to build. A competitor starting from scratch doesn't have that signal. They have benchmarks. You have signal.

This is why measurement infrastructure compounds in value over time, not just in the short run.

7. The human layer: who actually does this work

"Companies need help figuring out which models will work best for their workflows, they need extensive evals setup often, they need change management support for workflows, they need to get their data setup for the agents, and constant tuning of the agentic system for their process."

The eval harness doesn't build itself. Someone has to:

• Watch production for failures

• Translate failures into eval cases

• Run the hill-climbing loop

• Know when the metric is being gamed versus genuinely improving

• Change the eval when the eval is wrong

That's a human job. And it's one of the most technically demanding jobs in AI right now — precisely because it sits at the intersection of domain knowledge, engineering, and judgment about what actually matters.

The best AI teams I've worked alongside treat eval curation as a primary engineering discipline. Not something that happens after the "real work" is done. The eval harness is the real work. It's the system that makes the agent improvable. Everything else — the model choice, the infrastructure, the UI — is replaceable. The evals, and the process for generating them, aren't.

The practical implication: whoever is closest to production failures should have a direct line to your eval set. Whether that's a data scientist triaging a failure queue, a product manager reviewing user feedback, or an engineer watching structured logs — the signal is there. The question is whether you've built the pipeline to capture it, and whether the person who sees the failure can turn it into a new eval case in under five minutes.

If not, that's the first thing to fix.

Close the loop

The teams winning on agents in 2026 aren't the ones with the best models. Models are increasingly commoditized. They're the teams with the best eval infrastructure — the ones who've turned production failures into training signal, moved from frozen benchmarks to interactive environments, and made "add an eval" a reflex rather than a project.

Hill-climbing never stops. The question is whether you're climbing the right hill, and whether your instrumentation is sensitive enough to tell.

Start with one eval. Make it manual. Make it honest. Wire it to something real — an actual failure mode, an actual user scenario your agent encountered yesterday. Then close the loop.

The compounding starts when you do.

Over the past year, I've shipped several agents into production at the insurance company where I work. These aren't chatbots. They're systems that autonomously process documents, query internal databases, draft correspondence, escalate edge cases, and hand off to humans — across hours, sometimes days. Here's what I learned.

Why "Long-Running" Is a Different Problem

Most agent tutorials show you something that completes in under 30 seconds. Prompt in, tool calls, response out. Clean.

Real enterprise workflows don't look like that. My agents need to:

• Wait for a human to upload a document before continuing

• Pause mid-task because a downstream API is throttled

• Resume after a weekend without losing context

• Survive infrastructure restarts, deployments, and the occasional Kubernetes eviction

The moment your agent spans more than one process lifetime, you've entered a different design space. You need persistent state, explicit checkpointing, and failure recovery — not just a retry decorator.

Architecture: The Three Layers

After a lot of iteration, I landed on a three-layer architecture.

Layer 1: The Orchestrator

This is the durable brain. It knows what the agent is supposed to accomplish, what steps have been completed, and what comes next. It lives in the database, not in memory. Every time the agent advances a step, the orchestrator writes a checkpoint before doing anything else.

I model each agent run as a state machine. States are explicit: WAITING_FOR_DOCUMENT, EXTRACTING_DATA, AWAITING_HUMAN_REVIEW, COMPLETE, FAILED. Transitions are logged. You can reconstruct exactly where any agent run is — and was — at any point.

Layer 2: The Worker

Workers are stateless. They pick up a task from a queue, execute one step of the state machine, write results back to the orchestrator, and die. If a worker crashes mid-step, the orchestrator detects it via heartbeat timeout and reassigns the task. No task is lost because no task lives only in the worker's memory.

This is the biggest mindset shift from building "chatbots": treat every agent step as a unit of work that can fail and be retried independently.

Layer 3: The Human-in-the-Loop Interface

Some steps require a human. I don't fight this — I model it explicitly. When the agent reaches a step that needs human input, it writes a task to a work queue and suspends itself. A human completes the task (via a simple UI). The agent is notified and resumes.

This means agents can be "paused" for days waiting on a human without holding any compute. The orchestrator just has a row in the database marked WAITING_FOR_HUMAN.

Checkpointing: The Thing That Actually Saves You

I checkpoint aggressively. Before every tool call. Before every LLM call. After every meaningful piece of work.

Here's the pattern I follow:

1. Write intent before acting. Before calling an external API, write "I am about to call X with these parameters" to the checkpoint store. If the process crashes, you know what was about to happen.

1. Write result after completing. After the call succeeds, update the checkpoint to "X returned Y." Now recovery is: did we complete this step? If yes, skip it. If no, re-execute.

1. Make tool calls idempotent. This isn't always possible, but where it is, do it. Generate a UUID per intent, pass it as an idempotency key to external APIs, and you can safely retry without duplicating effects.

The checkpoint store is just a Postgres table. Nothing exotic. Each row has: run_id, step_name, status, input_hash, output, created_at. The agent loads its checkpoint on startup and skips any steps already marked complete.

Observability: What I Wish I'd Built Earlier

When an agent run fails at step 7 of 12, three days into its execution, you need to be able to answer: What did it do? What did it see? Why did it make that decision?

I now instrument every agent run with:

• Structured event logs: Every LLM call, every tool invocation, every state transition. Logged as structured JSON with run_id, step, timestamp, input, output, latency_ms.

• Token usage tracking: LLM costs compound fast when you have hundreds of parallel agent runs. I log input and output tokens per call and alert when a run exceeds 2× the expected token budget.

• A trace viewer: I built a simple internal UI that lets me pull up any agent run by ID and see its full timeline. This was a weekend project that has saved me countless hours of log-diving.

• Anomaly alerts: If a run has been in WAITING_FOR_HUMAN for more than 48 hours, we alert. If a run exceeds its expected duration by 3×, we alert. These are just cron jobs querying the orchestrator table.

The key insight: treat your agent runs like distributed transactions. The tooling exists for observing distributed systems — adapt it.

Handling Failures Gracefully

Agents fail. LLMs hallucinate. APIs time out. Documents are malformed. I've accepted this and designed for it.

Retry with backoff on transient failures. Network errors, rate limits, and temporary API outages are transient. I retry these with exponential backoff, up to a maximum. After max retries, the step fails and the run enters FAILED_RETRIABLE state.

Escalate on semantic failures. If the LLM returns something structurally valid but semantically wrong (extracted a date that doesn't exist, classified a document into the wrong category), I don't silently swallow it. I escalate to human review. The agent writes a note explaining what it saw and why it's uncertain, and a human resolves it.

Dead-letter runs for catastrophic failures. If a run hits an unrecoverable error, it goes to a dead-letter queue. A human triages it. We track dead-letter rate as a KPI. If it spikes, something is systematically wrong and we investigate immediately.

Never let a run disappear silently. This sounds obvious but it's easy to miss. An agent that crashes without updating its status looks identical to an agent that's still running. I use heartbeats: every worker pings the orchestrator every 30 seconds. If a heartbeat is missed for 2 minutes, the run is marked as SUSPECTED_DEAD and reassigned.

Lessons From Production

Lesson 1: The context window is not your memory. Early on, I passed the entire history of a run into every LLM call. Costs exploded. Latency ballooned. Now I summarize completed steps into a compact "run summary" and only pass recent context in full. The agent doesn't need to re-read everything it's done — just know what it's already accomplished.

Lesson 2: Humans are part of the system. I used to think of human-in-the-loop as a failure mode — something to minimize. Now I treat it as a first-class system component. Some decisions should go to humans. The agent's job is to make those handoffs clean and well-documented, not to eliminate them.

Lesson 3: Idempotency is load-bearing. Every time I skipped making a tool idempotent because "it's unlikely to be retried," it got retried, and something bad happened. It's always worth the 10 extra minutes.

Lesson 4: Schema-validate every LLM output. Every LLM call in my agents returns structured output validated against a schema at runtime. If the schema check fails, the step fails — loudly, with a detailed error — rather than propagating garbage downstream.

Lesson 5: Separate the LLM from the decision. The LLM suggests. The orchestrator decides. I don't let LLM output directly trigger irreversible actions. There's always a thin validation and approval layer between "the model said X" and "the system did X."

The Payoff

Running agents in this environment is genuinely hard. The failure modes are subtle, the stakes are real (insurance decisions affect people's lives), and the organizational context is conservative by design.

But it's also deeply rewarding. The agents I've shipped process hundreds of cases a week that would otherwise queue for days. They catch document inconsistencies humans miss at 2am. They make the humans who do review cases faster and more confident because the boring legwork is already done.

The patterns here — durable orchestration, aggressive checkpointing, explicit human handoff, observable runs — aren't specific to insurance. They apply anywhere agents need to survive contact with the real world.

Build for failure from day one. Your future self will thank you.

The arc of browser plug-ins — from power-user toy, to security liability, to the quiet backbone of the agentic web.

There's a moment in late 2024 that I keep coming back to. Anthropic ships Computer Use — a way for Claude to take screenshots and control a mouse cursor. The cleanest path to doing this in a browser isn't a new protocol or a custom runtime. It's a Chrome extension.

Something that had been declared dead a hundred times.

Act I: The Golden Age (2008–2014)

Firefox invented the modern browser extension. Not just as a feature — as a philosophy. A browser was an OS-in-waiting, and anyone could extend it. Greasemonkey let you rewrite any webpage with a few lines of JavaScript. AdBlock Plus blocked every ad on the internet. Firebug built the first real devtools before browsers had them.

When Chrome launched in 2008, Google took the model and made it faster, more sandboxed, and distributable via the Web Store. By 2013, there were tens of thousands of extensions. The category had its own celebrity developers, acquisition stories, and venture bets.

This era's defining trait: extensions as consumer products. You installed them for yourself. Password managers, tab managers, grammar checkers, ad blockers. The browser was a platform and extensions were apps.

Act II: The Long Decline (2015–2021)

Then came the rot.

It arrived slowly — a few extensions caught data harvesting in 2017, then a few more. Between 2024 and 2026, coordinated malware campaigns would go on to affect over 8.8 million users across Chrome, Edge, and Firefox. The Web Store turned out to be a decent distribution channel for malware. Extensions requesting <all_urls> permission got acquired for five figures, had their update payload quietly replaced, and started exfiltrating browsing history to ad brokers.

Google's response was Manifest V3, announced in 2018, which became one of the most contentious spec debates in browser history. Replacing persistent background pages with short-lived service workers. Replacing the webRequest blocking API that ad blockers depended on with a declarative declarativeNetRequest API capped on rules.

The ad blocker developers were furious. uBlock Origin's developer wrote long posts explaining exactly why the new model was insufficient. The EFF weighed in. Users signed petitions.

The vibe around extensions shifted entirely. Security-conscious companies started blocking extensions in managed Chrome deployments. The narrative calcified: extensions are a supply-chain risk, a performance tax, a privacy gamble. Between 2018 and 2021, if you worked in developer tools or security tooling, extensions were something you used reluctantly — not a platform you built for.

Act III: The Resurgence (2022–Present)

Four things happened nearly simultaneously, and together they changed the calculus entirely.

1. Computer Use Agents

In October 2024, Anthropic launched Computer Use — a capability that lets Claude take screenshots and control a computer. OpenAI followed with Operator in January 2025, powered by a model they called the Computer-Using Agent (CUA).

The performance numbers were striking. The open-source Browser Use framework hit 89% on WebVoyager (a standardized web task benchmark), compared to 87% for OpenAI Operator and 56% for Anthropic's initial Computer Use release. These numbers moved fast — by mid-2025, Anthropic's computer use tooling was shipping production-ready headers across Claude Opus 4.7 and Sonnet 4.6.

In August 2025, Anthropic launched Claude for Chrome — a Chrome extension that gives Claude a persistent sidecar in the browser, with permission to take actions on the user's behalf. Rolling out to 1,000 subscribers on the Max plan as a research preview. Google launched Gemini integrations with Chrome. Perplexity launched its own AI browser, Comet. OpenAI merged Operator directly into ChatGPT as "agent mode" in July 2025.

The browser extension is the agent's body. It has the right permissions — <all_urls>, content script injection, synthetic event dispatch — and it's already trusted by the browser's security model in a way an external process is not.

Extensions didn't become popular again because they got better. They became necessary because the use case required them.

2. Remote CDP and Browser Harness Tooling

The Chrome DevTools Protocol — the WebSocket API that Chrome exposes when launched with --remote-debugging-port — has existed since 2011. For years it was mostly used by Puppeteer and later Playwright for test automation.

What changed was the architecture around it. A new pattern emerged: a lightweight daemon process holds the CDP connection to an already-running Chrome, and scripts send one-shot JSON commands over a Unix socket. No Playwright overhead, no Node.js runtime, no process spawning — just a direct line to the user's real browser, with their cookies and authenticated sessions.

browser-harness <<'PY'
new_tab("https://example.com")
wait_for_load()
print(page_info())
PY

This made browser automation feel like a UNIX tool. First navigation is new_tab() not goto() — because you're attaching to the user's live browser, not a clean test instance.

Cloud providers took the same insight and productized it at scale. Browserbase, founded in 2024, raised $67.5M total — including a $40M Series B led by Notable Capital in June 2025, valuing them at $300M. By 2025 they were running 50 million browser sessions per year, with customers including Airtable, Instacart, Notion, Stripe, Perplexity, and Vercel. Forbes named them to their Next Billion-Dollar Startups list.

The pattern is the same everywhere: get a cdpUrl, resolve the WebSocket endpoint via /json/version, drive the remote Chrome instance. The browser becomes stateless infrastructure you rent by the session.

3. WebGPU-Based In-Browser LLMs

This one is weirder and arguably bigger in the long run.

WebGPU shipped in Chrome 113 in May 2023. Unlike WebGL, it's a proper compute API — shaders, compute pipelines, buffer access. For ML inference, this means matrix multiplications at speeds that benchmark at roughly 80% of native Metal or CUDA throughput. On an Apple M3 Max, WebLLM runs Llama 3.1 8B at 4-bit quantization at ~41 tokens per second — about 80% of native MLC-LLM performance. Phi 3.5 Mini hits 71 tokens per second. On discrete NVIDIA hardware, WebGPU is 10–15× faster than WASM for token generation.

WebLLM, MediaPipe LLM Inference, and Transformers.js proved that Gemma 2B, Phi-3-mini, and Qwen-1.5B could run entirely in the browser — no server, no API key, no round-trip latency. The practical sweet spot is 1B–3B parameter models at 4-bit quantization for reliable cross-device performance.

Where do extensions re-enter? An extension can load a model once into the background service worker and expose it via message passing to every tab. This is meaningfully better than each page re-downloading and re-initializing a 2GB model file. The extension becomes a local model host.

The privacy angle is also genuinely new. An extension running Gemma via WebGPU processes data that never leaves the machine — legally and technically. No server to subpoena. No request to intercept. For health data, legal documents, personal finance: this is a capability class that didn't exist three years ago.

4. Manifest V3 Actually Landed (And It's Fine)

Here's the part that surprises people: after all the drama, Manifest V3 turned out to be largely fine for the new use cases.

Chrome 139, released July 2025, completed the MV3 transition by fully removing MV2 extensions. By August 2025, 73.4% of actively maintained extensions had migrated. More telling: 90% of new extension uploads are already in MV3. The developer community absorbed the change.

For agent workloads specifically, the MV3 constraints are non-issues or improvements. Event-driven service workers rather than persistent background pages? Better for agent tasks that should only run when triggered. The controversial declarativeNetRequest cap? Irrelevant to screenshot capture, DOM injection, or synthetic input dispatch.

The ad blocker developers had a real grievance — for ad blockers. For computer use agents, browser harnesses, and in-browser LLMs, MV3 is not a blocker. The new use cases were inadvertently designed around the new constraints.

What This Actually Means

The first wave of browser extensions was consumer software. The second wave is infrastructure.

A consumer extension is something you choose to install for yourself. Infrastructure is something an application depends on — something that has to be there, permissioned correctly, composing with other systems.

The AI extensions market reflects this shift. There were 238 AI-powered Chrome extensions with meaningful user bases in 2025. By early 2026, that number was 442 — an 85.7% year-over-year increase, with 115.5 million combined downloads. The market was valued at roughly $2.3 billion in 2025, with projections ranging from $8.2 billion to $17.5 billion by the early 2030s.

The technical primitives that make this possible — CDP, content scripts, native messaging, WebGPU — aren't new. They've been there for years. What changed is that we found use cases that need exactly those capabilities and nothing simpler.

Where It Goes

Extension-as-agent-runtime becomes a standard pattern. Just as every serious product eventually shipped a mobile app, every AI agent product involving the web will ship an extension. Not because it's trendy, but because the alternative — screen capture plus pixel-coordinate clicking — is brittle, slow, and has no access to the DOM or authenticated session state.

In-browser model inference becomes a tier, not a novelty. As models shrink and WebGPU matures, the question won't be "can you run a model in the browser" but "which tier do you want — local, edge, or cloud?" Extensions that manage this routing will be the abstraction layer.

CDP browser farms consolidate around a few infrastructure players. Browserbase's $300M valuation after 16 months of existence is a signal. The pattern is too clean not to become commodity infrastructure. Three or four players will own this the way AWS owns EC2.

The security concerns didn't go away. The same properties that make extensions powerful for agents — <all_urls> permission, access to every page, background persistence — are the properties that made them dangerous before. Security researchers have already flagged that agentic browser extensions are prompt-injection surfaces: a malicious page can try to hijack the agent's actions by embedding instructions in its content. Simon Willison has argued that the entire concept of an agentic browser extension "is fatally flawed and cannot be built safely." That's probably too strong — but it's not wrong that the security model is unsolved.

Extensions had a decade as consumer software, a half-decade as a liability, and are now becoming infrastructure for the agentic web. The category didn't change. The demand did.

The browser turns out to be the last-mile interface for AI agents — not because it was designed for that, but because it's where humans already are, where the data already lives, and where the sessions are already authenticated. The extension is the seam between the model and that world.

They were never really dead. They were just waiting for something to actually need them.

Building in this space — browser agents, CDP tooling, WebGPU inference? I'd love to hear what you're working on.

What I've learned building LLM agents that actually run in production

There's a gap between the demo-tier agent tutorials you'll find online and what it actually takes to ship an LLM agent inside a large organization. The demos show you a loop. The enterprise asks: what happens when the LLM is slow? What happens when your tool returns garbage? What happens when the user asks a follow-up question and the agent has forgotten everything?

Over the past year I've been building agentic AI systems in a regulated financial services environment. The patterns below aren't academic — each one solved a real failure mode I ran into. I've stripped out all the proprietary details, but the engineering is real.

The Enterprise Constraint Set

Before diving into patterns, it helps to understand what's different in enterprise:

• Latency budgets are tight. Users in ops or analytics workflows won't wait 8 seconds for a response.

• Cost per query matters at scale. An extra LLM call per message × 10,000 queries/month adds up fast.

• Auditability is non-negotiable. In regulated industries, you need to be able to explain why the agent used a particular tool or returned a particular answer.

• Data lives behind access controls. You can't just "give the agent everything" — tool access has to be scoped to what the user is allowed to see.

These constraints shape every pattern below.

Pattern 1: Hybrid Intent Classification (Model Router)

Problem: You need to know what the user is asking before you can pick the right tools. But calling an LLM for every classification is expensive and slow.

Solution: Two-stage classification — regex heuristics first, LLM fallback only when heuristics fail.

def classify_intent(query: str) -> dict:
    # Stage 1: fast heuristics (< 1ms)
    result = _classify_by_heuristics(query)
    if result is not None:
        return result

    # Stage 2: LLM fallback (~400–600ms)
    return _classify_by_llm(query, llm)

def _classify_by_heuristics(query: str) -> Optional[str]:
    query_lower = query.lower()
    if CLINIC_PATTERNS.search(query_lower):
        return {"intent": "clinic", "entity": _extract_clinic_name(query)}
    if POSTAL_PATTERNS.search(query_lower):
        return {"intent": "postal", "entity": _extract_postal_code(query)}
    if DOCTOR_MENTION.search(query):
        return {"intent": "name", "entity": _extract_person_name(query)}
    return None  # signal: fall through to LLM

In practice: ~80% of queries are handled by heuristics. The LLM only fires for ambiguous cases. This keeps median latency under 100ms while preserving accuracy for edge cases.

The LLM classifier returns structured JSON:

prompt = """You are an intent classifier. Return ONLY valid JSON:
{"intents": [{"type": "<name|clinic|postal|unknown>", "object": "<value_or_null>"}]}

Query: {query}"""

Forcing JSON output makes the downstream routing deterministic. No parsing surprises.

Pattern 2: Dynamic Tool Selection

Problem: Giving the agent access to every tool at all times is wasteful and confusing. An agent with 8 tools available will sometimes use the wrong one.

Solution: Only enable tools that are relevant to the detected intent.

# Base set: always enabled
enabled_tools = [tools["analyze_data"]]

# Entity-specific tools: only when we have a named entity
has_entity = any(
    i["type"] in ["name", "clinic", "postal"]
    for i in classified_intents
)

if has_entity:
    enabled_tools.extend([
        tools["search_registry"],
        tools["get_records"],
        tools["aggregate_monthly"],
        tools["analyze_activity"],
        tools["get_summary_stats"],
    ])

agent = Agent(instructions=SYSTEM_PROMPT, tools=enabled_tools, model=model)

Why it matters: Fewer tools = smaller decision space = faster tool selection + fewer wrong-tool errors. It also gives you implicit routing — the agent physically cannot call a tool you haven't enabled.

Pattern 3: Multi-Algorithm Entity Search (Search Agent)

Problem: Users don't spell consistently. "Dr. Johnson" vs "Johnson, Sarah" vs "Dr. Sara Jonhson" are all the same person.

Solution: Three-layer matching — exact, fuzzy (Levenshtein), n-gram (Jaccard). Apply all three; keep results that pass any one threshold.

def _fuzz_score(x: str, y: str) -> float:
    """Levenshtein-based partial ratio."""
    return 1.0 if x == y else fuzz.partial_ratio(x, y) / 100.0

def _ngram_sim(x: str, y: str) -> float:
    """Jaccard similarity on token sets."""
    set_x, set_y = set(x.split()), set(y.split())
    inter = set_x & set_y
    union = set_x | set_y
    return len(inter) / len(union) if union else 0.0

# Apply both scores, pass if either exceeds threshold
df["fuzz"] = df["NAME"].apply(lambda n: _fuzz_score(n.upper(), query_upper))
df["ngram"] = df["NAME"].apply(lambda n: _ngram_sim(n.upper(), query_upper))

matches = df[
    (df["NAME"].str.upper() == query_upper) |   # exact
    (df["fuzz"] >= 0.6) |                        # fuzzy
    (df["ngram"] >= 0.5)                         # n-gram
]

Why both fuzzy and n-gram?

• Fuzzy catches typos: "Jonhson" → "Johnson"

• N-gram catches reorderings: "Sarah Johnson" → "Johnson Sarah"

• Combined: you get recall without sacrificing precision

Pattern 4: Layered Fallback

Problem: Production systems fail. The LLM times out. The database is slow. The SDK isn't installed in some environment.

Solution: Multiple independent fallback layers, each one degrading gracefully to the next.

# Layer 1: SDK availability
try:
    from agents import Agent, Runner, function_tool
    SDK_AVAILABLE = True
except ImportError:
    SDK_AVAILABLE = False

if not SDK_AVAILABLE:
    return "Agent SDK not available. Please install openai-agents."

# Layer 2: Classification fallback (heuristic → LLM → default)
intent = _classify_by_heuristics(query)
if intent is None:
    try:
        intent = _classify_by_llm(query, llm)
    except RateLimitError:
        intent = _classify_by_llm(query, llm)  # single retry
    except Exception:
        intent = {"type": "unknown", "object": None}  # safe default

# Layer 3: Tool execution fallback
try:
    result = run_tool(params)
except Exception as e:
    return json.dumps({"error": str(e), "success": False})

Key principle: Every layer has exactly one job. The SDK check doesn't know about rate limits. The rate limit handler doesn't know about tool failures. Each layer handles exactly one class of failure and passes everything else down.

Pattern 5: Session Memory and Context Reuse (Feedback Loop)

Problem: Users ask follow-up questions. "What about that second provider?" doesn't make sense without memory of the previous turn.

Solution: SQLite-backed session memory + explicit context reuse for ambiguous follow-ups.

# On session start
session_id = user_session.get("id")
memory = SQLiteSession(session_id=session_id, db_path=":memory:")
user_session.set("agent_memory", memory)

# On each message
result = await Runner.run(
    agent,
    input=query,
    session=memory  # agent sees full conversation history
)

# Context reuse for unknown intents (follow-up questions)
if any(i["type"] == "unknown" for i in intents):
    previous_tools = user_session.get("previous_enabled_tools")
    if previous_tools:
        # "What about that other provider?" → reuse last search context
        enabled_tools = previous_tools

The context reuse heuristic is simple: if the intent is "unknown" (no entities detected), the user is probably asking about something already established in the conversation. Reuse the tool set from the previous turn instead of starting fresh.

Pattern 6: In-Memory Data Caching

Problem: Loading a large DataFrame from a data lake on every query adds 2+ seconds of latency.

Solution: Lazy-load into a module-level cache. First call pays the load cost; every subsequent call uses the in-memory copy.

_registry_df: Optional[pd.DataFrame] = None
_records_df: Optional[pd.DataFrame] = None

def _load_registry() -> pd.DataFrame:
    global _registry_df
    if _registry_df is None:
        dt = DeltaTable(REGISTRY_PATH)
        _registry_df = dt.to_pandas()
    return _registry_df

Result: First query: ~2s. Subsequent queries: ~50ms. 40× improvement for a few lines of code.

Watch out for: Memory pressure if DataFrames are very large, and stale data if the underlying source updates. For most analytics use cases, session-lifetime caching is a reasonable trade-off.

Pattern 7: Selective Result Limiting

Problem: Returning 50,000 rows to the LLM context window is expensive, slow, and usually useless.

Solution: Return summaries and limited result sets by default. Only go wide when the user explicitly asks.

@function_tool
def get_records(entity_ids: list[str]) -> str:
    """Get records for the given entity IDs."""
    result = fetch_records(entity_ids, limit=100)  # never return everything
    return json.dumps(result, default=str)

@function_tool
def get_summary_stats(entity_ids: list[str]) -> str:
    """Get summary statistics: first/last date, total amounts, counts."""
    # Returns 5 numbers instead of 5000 rows
    result = compute_summaries(entity_ids)
    return json.dumps(result, default=str)

Design your tool set so the agent reaches for summaries first and raw records only when it genuinely needs them. This is a prompt engineering + tool design problem together — your tool descriptions need to clearly signal when to use which.

By the Numbers

Two patterns that pay for themselves immediately — the classification split and the caching improvement:

The caching win is almost embarrassingly simple. The classification split is worth thinking about carefully upfront — the 80% heuristic coverage number will vary by domain, but even 50% coverage halves your classification cost.

How These Patterns Combine

These patterns aren't independent — they're designed to compose:

Pattern 1 (Model Router)
    → classifies intent
    → feeds Pattern 2 (Dynamic Tool Selection)
        → scopes agent's decision space
        → Pattern 3 (Search Agent) handles entity lookup
        → Pattern 7 (Result Limiting) keeps context lean
            → Pattern 5 (Feedback Loop) handles follow-ups
                → Pattern 4 (Fallback) catches everything that breaks
                    → Pattern 6 (Caching) keeps it fast

The compounding effect: classification is fast (Pattern 1), so tool selection is accurate (Pattern 2), so the agent stays focused, so responses are faster and cheaper, so you can afford the LLM fallback for genuinely hard cases.

What I'd Apply First

If you're building an enterprise agent today and you can only pick three:

1. Hybrid intent classification — the cost/latency savings pay for everything else.

2. Dynamic tool selection — directly reduces agent confusion in multi-tool systems.

3. Layered fallback — production systems fail; don't let one failure mode take down the whole chain.

The memory and caching patterns matter a lot once you're at scale, but the first three are table stakes for any agent you're running in front of real users.

These patterns aren't specific to any one framework — I've used them with OpenAI Agents SDK, LangChain, and raw API calls. The underlying logic transfers.

If you're building something similar or have patterns that worked for you, I'd love to hear about it in the comments.

Forward Deployed Engineers are the hottest job in tech right now.

OpenAI just launched its own FDE agency. Anthropic is hiring them at scale. Palantir built its entire go-to-market on them. Stripe created a new role — the "Forward Deployed AI Accelerator" — embedding AI-native engineers directly inside their marketing teams, assigned to cohorts of 20 people each, building custom tools alongside them until every person is self-sufficient. Box's CEO wrote publicly that building AI agents for internal functions is "a highly technical job, very much akin to a forward deployed engineer."

The external FDE market is exploding because someone has to do the hard part of deploying AI inside real organizations — understanding business processes end to end, wiring up the right models, setting up evals, managing workflow change, and tuning agentic systems continuously until they actually work.

Here's the thing: your company almost certainly won't bring one of these people in for your internal data team. The going rate is reportedly $10K/day for the big-name providers. And even if they did — it wouldn't be enough.

The CEO of Box said it clearly: "External FDEs, in my opinion, will not make your company an AI-first company. You can have the sleekest multi-agent orchestrations and still have the majority of your employee base hating AI, avoiding AI, and distrusting leadership decisions on it."

Stripe understood this when they designed their accelerator role. They built it specifically because "most employees won't upskill themselves. They'll need someone who is embedded within their teams to build alongside them."

That someone can be you — from the inside, with years of organizational context, at a fraction of the friction.

That's the Internal FDE. And if you're a data scientist or AI engineer inside a large enterprise, this is the most important role you're not being asked to play.

Why Your Work Disappears Without This Role

Let me describe something that happens constantly in enterprise data teams.

A team of smart people spends six months building a genuinely excellent model. It performs well on holdout. The business case is solid. The model goes into production.

And then... it gets used by three analysts, never seen by the decision-makers it was built for, and quietly deprecated when a new VP comes in with different priorities.

The code was fine. The data science was fine. The problem was that nobody translated it.

Nobody walked the floors. Nobody told the story in language that connected to the business team's quarterly priorities. Nobody made the downstream teams feel like partners instead of an upstream data source. Nobody built the internal credibility that makes a model a business asset instead of a technical artifact.

This is the FDE gap.

Deploying agents — or any complex AI capability — is fundamentally harder than deploying software. Software generally works the same way every time. With agents, you're deploying the equivalent of work output inside the enterprise. The business expects tasks solved nearly end-to-end. That means someone has to deeply understand the business process, handle model selection, set up evals, manage workflow change, get the data right, and tune the system continuously. It's not a side project. It's a mission-critical engineering role.

In the external FDE model, a vendor does this for you — at enormous cost, with no lasting institutional knowledge. In the internal model, it's you. With years of organizational context they could never have.

What an Internal FDE Actually Does

You are the connective tissue between deep technical work and business reality. That means four things, specifically.

You are a translator. When you talk to the underwriting team, you don't lead with AUC scores. You lead with "here's what this means for your loss ratio." When you're presenting to the executive leadership team, you don't lead with model architecture. You lead with the business decision this changes. Translation is not dumbing down — it's respecting your audience enough to meet them in their world.

You are a scout. Before anyone on your team builds anything, you've already had the conversations. You know what the business unit is worried about this quarter. You know where the downstream teams are frustrated. You know which operations lead would be an incredible champion if someone just brought them a relevant prototype. You don't wait for a project brief to discover business needs. You accumulate that intelligence continuously.

You are a credibility builder. Technical credibility comes from doing good work. Business credibility comes from showing up consistently, being honest when something doesn't work, and making people's lives easier before you ask them for anything. The Internal FDE is known before any project starts. When a new initiative needs a data partner, your team's name comes up because you've been in those rooms.

You are an amplifier. Great work that nobody knows about is just expensive hobby time. The Internal FDE makes sure that when the team delivers something valuable, the right people understand what it does, how it was built, and what's possible next. Not with self-promotion, but with clear, consistent communication that connects outputs to outcomes.

Your Unfair Advantage

External FDEs bring technical depth and pattern recognition from deploying similar systems across many companies. That's genuinely valuable.

But they lack something you already have: years of accumulated context about how this specific organization thinks, decides, and resists change.

There's a reason the best specialist consultants are almost always people who spent a decade inside the industry before going independent. If you tear your ACL, you want to go to an ACL surgeon — someone who has seen this specific failure mode hundreds of times in this specific context, not a generalist learning on your dime.

You are the ACL surgeon for your company's data problems. You know the internal politics. You know why the last initiative failed. You know which VP will champion something and which one will quietly kill it. You know which downstream team has the best data hygiene and which one will fight every integration request.

External FDEs have to earn that context from scratch, and they're on the clock while they do it. You already have it. The question is whether you're using it.

Lead With Beliefs, Not Features

The most common mistake in internal technical communication: leading with the work instead of the so-what.

"We've built a gradient boosted model with 87 features and a cross-validated AUC of 0.84" is accurate. It also doesn't start a conversation with anyone who isn't already thinking about the same technical problem you are.

Compare it to a belief statement: "I believe our models will only drive value if the business team owns the output, not the AI team." That tells your stakeholders immediately where you stand and why. It gives them something to react to, agree with, or push back on. It starts a real conversation.

This is what the best FDEs do with their clients — they don't open with product specs, they open with a diagnosis of the problem and a clear point of view on what matters. The same applies internally.

Before any major presentation or communication, ask yourself: what's the belief I want this audience to leave with? Work backwards from that.

Building Your Internal Audience

The first people who engage with your work set the tone for everyone else.

If the first reaction to your model readout is a VP saying "this is exactly what I've been asking for," everyone else in the room recalibrates. If the first reaction is "I don't understand what this means," you're spending the rest of the meeting recovering.

This means you need to do pre-work. Not to game the room, but to make sure the first engagers are informed engagers. Before any major internal presentation:

Talk to one or two stakeholders in advance. Not to get their approval, but to understand their current mental model and make sure your framing lands. Ask them what question they'd most want this work to answer. Then answer that question first.

Find your internal champion before you need one. In every business unit that interacts with your team, there's usually someone who "gets it" — who thinks quantitatively, who's frustrated with the status quo, who would love to have a data partner. That person is your ally. Invest in the relationship before you have a project that needs them.

Make your first engagement easy. Leave stakeholders with something clear to do or respond to. Not "let us know if you have questions" — but "we're going to pilot this with two underwriters in Q3 — can you connect us with the right people on your team by end of next week?"

The Communication Types That Build Credibility

Different types of internal communication build different kinds of trust. The most effective Internal FDEs cycle through all four.

Thought leadership: Share a perspective on where AI and data are headed in your specific domain — pricing, risk, claims, whatever your function is. You have a unique vantage point. Nobody else at your company sits at the intersection of the data and the business problem you're closest to. That vantage point is valuable, but only if you share it. Write a short internal piece. Send it to three people you respect. If it resonates, expand it.

Personal stories: The most underused communication type inside enterprise data teams. "Here's what I thought when we started this project, here's what we found, here's what we were wrong about" is more compelling than any polished methodology deck. It's also more credible. Anyone can dress up a success. The willingness to narrate the failure modes, the pivots, the things that surprised you — that's what makes people trust your judgment on the next project.

How-to guides: Technical teams build institutional knowledge that lives in people's heads and disappears when they leave. The Internal FDE turns that knowledge into artifacts. A one-pager on how to use your team's outputs. A short guide on how to read a lift chart for a non-data audience. Documentation that enables the business team to get value from your work without needing you in the room every time. This is leverage.

Predictions: "Here's what I think is going to matter in our space in the next twelve months" is a powerful thing to say internally, especially if you're right more often than not. It positions you as forward-looking, gives leadership something to react to, and often turns into a real project when someone says "actually, we've been thinking about that too."

The Practical Mechanics

Here's what actually building an internal FDE practice looks like, week to week.

Have one non-project coffee per week. One conversation with someone outside your immediate team that isn't about a current project. Just: what are you working on, what's keeping you up, what do you wish was different? This is how you build your scout intelligence. Over a year, these conversations compound into a map of the organization that nobody else has.

Write one short internal thing per month. It doesn't have to be long. A one-page take on something interesting you've learned. A short summary of a paper that has implications for your work. A retrospective on a project that didn't go as expected. Share it with a small list of people who would genuinely find it interesting. Do not mass-blast to the whole data org. Quality of audience matters more than quantity.

Create one reusable artifact per quarter. The Internal FDE leaves infrastructure behind. What would help the business teams work better with data, even when you're not in the room? A template. A guide. A decision framework. A cheat sheet. Something that gives you leverage without requiring your continuous presence.

Show up before you're needed. The worst time to build relationships with a business unit is when you need something from them on a deadline. Show up at their all-hands when you're invited. Ask to sit in on a team meeting just to understand their workflow better. This is not political maneuvering. It's the basics of being a good colleague in a large organization where teams default to siloing.

The Objection: "I'm a Data Scientist, Not a Communicator"

I've heard this from genuinely excellent technical people, and I want to push back on it.

Communication isn't a personality trait. It's a skill, and like any skill, it improves with practice and feedback. The data scientists who are most effective in enterprise settings aren't necessarily the most naturally outgoing people. They're the people who've built the habit of translating their work into business language, and who do it consistently.

There's also a structural argument here. In a large company, the value of technical work is not determined by the quality of the technical work. It's determined by whether decision-makers trust the team that produced it, understand what it does, and see a path to acting on it. That trust is built through human interaction, not through model cards.

You can be the best data scientist in the building, and if nobody above the technical level knows what you're working on or why it matters, you will have less impact than someone half as technically skilled who communicates well. That's not cynical — it's just how organizations work.

The Internal FDE role is not about becoming a politician or a salesperson. It's about deciding that you care enough about the impact of your work to do the last mile that makes it real.

What You're Really Building

The frustrating thing about doing excellent technical work in a large organization is that excellence is necessary but not sufficient. A model that lives in a notebook, or gets used by three people, or gets deprecated when priorities shift — that model didn't matter, regardless of how technically sound it was.

The Internal FDE closes the gap between good work and real impact. It's the part of the job that nobody assigns you, that doesn't show up in your performance criteria, and that most technical people quietly resent having to do.

But it's also the part that determines whether your work changes anything.

Companies are now paying $10K a day to bring in external people to do this. They're creating new job titles for it. They're embedding engineers inside teams specifically to solve the change management and translation problem that technical teams consistently fail to solve on their own.

You can be that person from the inside. With more context, more continuity, and a fraction of the friction.

That's worth building.

If you're building an AI or data practice inside a large enterprise and want to think through what the Internal FDE role looks like on your specific team, I'd love to hear what you're running into. Drop a reply or send a message.

Neither works if you actually want to own your model's behaviour.

This post is a system design walkthrough of an LLM Studio — the software infrastructure that lets a small team run the full model-improvement loop end to end. Think of it like a HelloInterview deep dive, but the system being designed is your own fine-tuning platform.

What Problem Are We Actually Solving?

The core loop of model ownership looks like this: you have a task, you have data, you want a model that gets better at that task over time. The challenge isn't the ML math — libraries handle that. The challenge is the software infrastructure around it.

You need to:

• Ingest and version training data (including synthetically generated data)

• Run fine-tuning jobs without blocking everything else

• Serve the resulting models without redeploying when new checkpoints land

• Evaluate models against each other without writing one-off scripts

• Let non-ML engineers trigger experiments through a UI

A notebook can do one of these. A studio does all of them, reliably, repeatedly.

High-Level Architecture

The studio is six loosely coupled services. Each has a single responsibility, its own database schema, and communicates over HTTP.

The two most important separations:

Finetuning and inference have different lifecycles. Training is GPU-hungry, bursty, and failure-tolerant — if a job crashes, retry it. Inference is always-on and latency-sensitive — if it crashes, users notice. Coupling them in one process means a bad training job can kill your serving layer.

The data service is its own thing. Synthetic data generation is slow (minutes per batch), LLM-heavy, and produces intermediate artefacts that need versioning. Baking it into the API would turn every data request into a blocking call.

Deep Dive: The Finetuning Service

This is the most complex service. Its job is to take a submitted training request and reliably produce a fine-tuned model checkpoint.

Job Scheduling

The service uses APScheduler with a background scheduler running three recurring jobs:

scheduler.add_job(finetuning_scheduler.run_fine_tuning_jobs, "interval", seconds=N)
scheduler.add_job(finetuning_scheduler.check_running_jobs,    "interval", seconds=N)
scheduler.add_job(finetuning_scheduler.check_jobs_waiting_for_checkpointing, "interval", seconds=N)

Why a polling scheduler instead of a task queue? Because fine-tuning jobs are long-running (minutes to hours) and stateful. Celery is designed for short tasks. APScheduler lets the service own the full job lifecycle — queuing, running, checkpointing, failure — without fighting a task framework's assumptions.

The Training Runners

Two runners live inside the finetuning service: one for SFT (Supervised Fine-Tuning) and one for GRPO (the reinforcement learning variant). They're separate processes, not just functions — the runner is launched as a subprocess with explicit hyperparameter flags. This means:

• A runner crash doesn't take down the service

• You can run multiple jobs in parallel on separate GPUs

• Runner code can be updated without restarting the scheduler

Checkpoints get written to a shared filesystem path and the service marks the job as CHECKPOINTING before promoting the adapter to the model registry.

Deep Dive: The Inference Service

The inference service exposes an OpenAI-compatible chat completions endpoint. The model underneath can be Ollama (local dev) or vLLM (production), but the contract to callers doesn't change.

POST /inference/v1/chat/completions
{ "model": "my-finetuned-adapter-v3", "messages": [...] }

Why Ollama Locally, vLLM in Prod?

Ollama is trivially easy to set up on a laptop — ollama pull llama3 and you're serving. But Ollama's throughput is single-request, which is fine for development and the model playground, but not for evaluation runs that fire 200 completions in parallel.

vLLM uses continuous batching — incoming requests share KV cache and GPU compute. For the GRPO training loop, which needs to generate 8 completions per prompt across thousands of prompts, vLLM is 5–10x faster. The service switches backends by environment variable; no code changes needed.

For burst production workloads, the same service can be pointed at a Modal-hosted endpoint — GPU spins up on demand, billing stops when idle.

Deep Dive: The Data Service (`svc`)

The data service handles everything that touches training data: ingestion, synthetic generation, augmentation, curation, and evaluation. It's the most Celery-heavy service in the stack.

The Synthetic Data Pipeline

The pipeline is async by design. A generation request comes in via the API, gets dispatched to a Celery worker, and the frontend polls for progress. The worker:

1. Ingests source documents (PDFs, plain text, YouTube transcripts via the API)

2. Chunks the content into token-bounded windows

3. Calls the LLM gateway to generate pairs (QA, CoT, or summaries)

4. Streams completed pairs back to the dataset in real-time

5. Marks the task complete and updates the dataset snapshot

The three generation modes matter architecturally, not just academically:

• QA pairs are cheap and fast to generate, good for instruction-following baselines

• Chain of Thought pairs cost ~3x more tokens but produce dramatically better structured-output models — the model learns to reason before answering

• Summaries are useful when your inference task involves compression, not extraction

Evaluation

The evaluation service can run multiple models against a snapshot in parallel and score them using either an LLM-as-judge rubric or a bring-your-own-evaluator endpoint. Results stream back per-datapoint, so you see the leaderboard filling in live rather than waiting for a batch job to complete.

Deep Dive: The Gateway

Every LLM call in the entire system — synthetic data generation, evaluation, GRPO reward scoring, the model playground — goes through a single gateway layer. This is not optional architecture astronautics. It pays for itself immediately.

What you get for free:

• Cost visibility — which service is spending what, on which model

• Routing — fall back from GPT-4o to Sonnet if one provider is down

• Caching — identical prompts during eval runs don't get billed twice

• Rate limit management — the gateway absorbs burst traffic so individual services don't need to implement retry logic

The gateway is a thin sidecar — one Docker container, no business logic. Portkey, LiteLLM, or a self-hosted proxy all work. The key is that it's a hard architectural boundary: no service calls an LLM provider directly.

Deep Dive: The API and Frontend

The api service is the single front door. It handles:

• Auth — API key verification for all service-to-service calls

• Job dispatch — accepts training requests and forwards them to the finetuning service

• Model registry — tracks which checkpoints exist and their metadata

• Dataset management — versions snapshots, stores split configurations

The frontend is SvelteKit. The key UI surfaces are: dataset view (upload, inspect, generate), experiment configurator (pick base model + hyperparameters), job monitor (live training logs), and the model playground (side-by-side completion comparison).

Local vs Production

Running locally and running in production are architecturally identical — same six services, same HTTP contracts. The only differences are the backing implementations:

The local stack boots with a single docker compose up. Each service gets its own container, Redis runs as a sidecar, and Ollama handles inference. Switching to production means swapping environment variables — no code changes.

Key Design Decisions

Why not a monolith? A monolith works until you need to scale the finetuning service to GPU machines while keeping the API on cheap CPU containers. Service separation lets each component scale independently and fail independently.

Why polling for job state instead of webhooks? Training jobs run for minutes to hours. Webhook delivery has a real failure surface over that timeframe. The frontend polls /api/jobs/:id every few seconds — simple, robust, no webhook infrastructure needed.

Why Celery for data tasks but APScheduler for training jobs? Celery is optimised for many short tasks with a shared worker pool. Training jobs are one long task per GPU — APScheduler's interval-based polling and explicit state machine fits better.

Why OpenAI-compatible inference endpoints? Every evaluation library, every agent framework, every LLM client already speaks this protocol. Wrapping your fine-tuned model in a compatible endpoint means zero integration cost downstream.

What Breaks at Scale

A few things that look fine in development but become problems in production:

Shared filesystem for checkpoints. Locally, all services can read the same path. In production, you need an object store (S3, Azure Blob) as the checkpoint backend, with the finetuning and inference services both mounting it. This is a one-line config change if you design for it upfront.

Celery worker saturation. A single LLM call for synthetic data generation can take 10–30 seconds. If you have 10 concurrent generation tasks each firing 100 LLM calls, your worker pool fills up fast. Separate worker queues for fast tasks (evaluation) and slow tasks (generation) early.

No rate limiting on the evaluation service. Running evals against all models in your registry against a large dataset can fire thousands of LLM calls in minutes. The gateway handles rate limits, but the evaluation service should have a concurrency cap per eval job.

SGLang vs vLLM: Which One Goes in the Inference Service?

The studio's inference service is deliberately abstracted behind an environment variable. Locally it points at Ollama. In production, you pick an engine. And in 2026, that means choosing between vLLM and SGLang — two projects that started as competitors but have now specialised into different niches.

The 2024 narrative of "SGLang is 3× faster" no longer holds. vLLM's V1 rewrite (January 2025) pulled the two within 3–5% on dense 70B models at typical concurrency. But diverge from that workload profile and the gap opens sharply in one direction or the other.

The Core Architectural Difference

The engines diverge fundamentally on how they store KV cache.

vLLM uses PagedAttention — fixed-size physical blocks, hash-based per-block sharing, similar to OS memory paging. Predictable, well-understood, maps cleanly to most hardware.

SGLang uses RadixAttention — the entire live KV cache is a compressed prefix trie. New requests walk the tree from root; matched nodes mean their KV state is already on-GPU. When multiple requests share a system prompt, a retrieved document, or a conversation history, RadixAttention shares that prefix automatically across tenants with no configuration. The cache-aware router biases routing toward whichever worker holds the warmest matching prefix.

This isn't a marginal implementation detail. On agent loops, RAG pipelines, and multi-turn chat — anywhere prefixes repeat across requests — RadixAttention is the correct data structure and PagedAttention is the wrong one. The papers report up to 6.4× speedup on those workloads.

Where Each Wins

The MoE picture is stark. On GPT-OSS-120B with a tuned configuration, vLLM hit 6,095 tok/s versus SGLang's 1,814 — a 3.4× lead. DeepSeek V3/R1 runs in the other direction: SGLang's 96-H100 production deployment matches DeepSeek's own infrastructure throughput at roughly $0.20 per million output tokens, about one-fifth the cost of the public API. The DeepSeek GitHub repo officially recommends SGLang.

vLLM's durable advantage is breadth. It runs on NVIDIA, AMD, Google TPU, AWS Inferentia, Intel Gaudi, Apple Silicon, and IBM Z mainframes. SGLang covers most of those but TPU support lags. If you might multi-cloud, vLLM is the safer choice. It also covers ~200 model architectures — encoder-decoder, Mamba, Whisper, T5 — that SGLang doesn't support at all.

The Production Decision

The default for a generalist team is start with vLLM. Then add SGLang as a second inference backend behind the gateway for the specific workloads where it wins: prefix-heavy pipelines, DeepSeek/MoE serving, structured output at scale, or RL training rollouts.

Both engines expose identical /v1/chat/completions semantics. The application-layer switching cost is genuinely near zero. Run both, route by workload type, measure on real traffic before committing to one.

The deeper shift is that in 2026 you're not picking an inference engine — you're picking a stack. Dynamo or llm-d on top, vLLM or SGLang underneath, with the gateway routing between them. The orchestration layer is increasingly engine-agnostic. What remains different — RadixAttention's tree, vLLM's hardware breadth, SGLang's DeepSeek depth — is exactly what makes the choice still matter.

The architecture is deliberately boring. Six services, HTTP, Postgres, Redis, one LLM gateway. The interesting engineering is in the details: the job state machine, the streaming data pipeline, the dual-backend inference layer. Those are where the real decisions live.

If you're building something like this or have questions about any of the design choices, I'd love to hear from you.

Here's the current list, with what each one does and why I built it.

LinkNotes

What it is: A private, password-protected link vault with a CRUD API. Each link has a title, URL, tags, source, and a notes field.

Why I built it: Most bookmark managers optimize for quantity. Pocket, Raindrop, browser bookmarks — they're all great at storing links and terrible at storing why you saved them. Three months later you open a bookmark and have no idea what it was for.

LinkNotes forces a notes field. "Why does this matter?" is a required part of saving. The source field tracks where it came from. Tags let me filter by project or topic. And because it exposes a simple API, I can add links from scripts, automations, or other tools without touching the UI.

Built on Cloudflare Workers + D1. The whole thing is a single Worker — no external services, no database bills.

Tree Chat

What it is: A chat interface where conversations branch into trees. Use /branch [prompt] at any point to fork from the current message and explore a different direction, preserving the original thread.

Why I built it: Standard chat UIs have a fundamental problem: they're linear. If you're exploring a problem with an LLM — comparing two architectural approaches, exploring different framings of a question, trying a prompt variation — you have to either scroll back and lose your place, or start a new conversation and lose the context.

A tree solves this. Each branch inherits the context up to the fork point. You can explore three different answers to the same question simultaneously, navigate back to any node, and keep the whole exploration in one place.

Built on Cloudflare Pages with a D1 database for conversation persistence. Each node stores its parent ID and the full message content, and the tree is reconstructed on load.

TabReplay

What it is: Upload a file of URLs, and TabReplay lets you review each one with a live preview and triage them — keep, read now, or discard.

Why I built it: Tab hoarding is a real problem. I'll open 40 tabs in a research session and then not close Chrome for a week because I'm afraid of losing something. The real answer isn't better tab management — it's committing to a decision about each tab before closing.

TabReplay makes that decision-making fast. Export your tabs as a list of URLs (most browsers support this), drop the file in, and review each one with the actual page loaded alongside it. It uses Playwright under the hood to visit and render each URL, so you see what's actually on the page, not just the title.

HAR Vision

What it is: Drop a .har file — your browser's network log — and get instant visual analysis, timeline view, and request filtering. Runs entirely in your browser; nothing leaves your machine.

Why I built it: HAR files are invaluable for debugging, reverse engineering APIs, or understanding why a page is slow. But the raw JSON is unreadable, and the only decent viewer (Chrome DevTools) requires you to reproduce the session live. If you captured a HAR from a user's session or a CI test run, you're stuck parsing JSON by hand.

HAR Vision loads the file locally and gives you a filterable timeline — filter by domain, method, status code, or response time. Useful for finding the slow request in a 300-request page load, or spotting which API endpoints a web app calls.

ClipShot

What it is: Drop a video file, scrub through a Premiere Pro-style thumbnail timeline, and capture frames at native resolution. Export individually or as a ZIP.

Why I built it: I record a lot of screen recordings for documentation and demos. Extracting a good frame usually meant: open the video in QuickTime, scrub manually, take a screenshot, crop. ClipShot turns this into: drop the video, click the frame, done.

The whole pipeline runs in the browser using the HTML5 <video> element and Canvas API — no upload, no server, no waiting for processing. You capture exactly what you see in the timeline at native resolution, then export individual frames or bulk-download as a ZIP.

GridSplitter

What it is: Upload a sprite sheet, contact sheet, or photo grid — GridSplitter auto-detects the rows and columns using computer vision and splits it into individual files for download.

Why I built it: I was building a small game prototype and needed to extract individual frames from sprite sheets I found online. Every existing tool either required you to manually specify the grid dimensions or was a desktop app. GridSplitter uses edge detection to find the grid structure automatically — you don't count rows or columns, you just upload and download.

The CV pipeline runs in the browser using a WASM port of OpenCV. Processing happens entirely client-side, so there's no file size limit and no privacy concern with uploading assets.

Healpix

What it is: A browser-based photo healing tool. Paint over blemishes, dust spots, watermarks, or unwanted objects with a brush — the inpainting engine fills them in seamlessly.

Why I built it: The Photoshop healing brush is one of those tools that feels like magic the first time you use it. But Photoshop is $55/month and overkill if that's all you need. Healpix does one thing — heal — using the same underlying inpainting algorithms (Telea and Navier-Stokes), in the browser, with no install.

Paint the region you want removed, release the mouse, and the engine fills it in. Adjustable brush size, full undo/redo history, and export when done.

Watermark Remover

What it is: Upload a PDF, set of images, or MP4 video — configure the watermark region — and it strips the watermark using CV inpainting. Processes PDF pages in batch and video frame-by-frame.

Why I built it: Most watermark remover tools online are either broken, behind a paywall, or cap file sizes aggressively. This one uses the same Telea and Navier-Stokes inpainting algorithms that Healpix uses, but targets a fixed region across many frames — which is exactly how most watermarks work (static, bottom-right corner).

The video processing pipeline extracts frames at 1fps, inpaints each frame, then re-encodes the video with ffmpeg. Slow, but it works on arbitrarily long videos.

Swap Fitter

What it is: Two-step image compositing. Upload a base image, draw a selection over the region you want to replace, then upload a second image to fill that region. Runs client-side.

Why I built it: I kept needing to answer "what would this look like if..." questions — swapping a UI component in a screenshot, replacing a label on a product photo, mocking up a design variant. Opening Figma or Photoshop for a two-minute task is too much friction. Swap Fitter is just the compositing step, nothing else.

Step 1: upload base image, draw selection. Step 2: upload replacement, which gets scaled to fit the selection. Export. Done.

AutoCarousel

What it is: Upload images in order, set the slide duration (how long each image stays on screen), and export as an MP4 video file.

Why I built it: Social media carousels — especially on LinkedIn and Instagram — get more reach than static images. But converting a set of slides or screenshots into a video carousel usually involves either a design tool with video export (slow) or a social media scheduler (locked in). AutoCarousel just turns images into a video, no account required.

Upload your images in order, drag to reorder if needed, set 3–5 seconds per slide, and export. That's it.

Video Looper

What it is: Drop a short video clip and an audio track — the tool loops the video to exactly match the audio's duration and exports a single combined file.

Why I built it: A common content format is a looping visual over a music track — ambience videos, demo backgrounds, lo-fi aesthetic clips. The problem: your video might be 4 seconds and your audio 3 minutes. Manually looping in a video editor means dragging the same clip 45 times.

Video Looper calculates exactly how many loops are needed, handles the partial last loop cleanly, and muxes the audio track in. Upload both files, click create, done.

The pattern I've noticed across all of these: the tools I actually reach for are the ones that do exactly one thing and require zero configuration. No accounts, no settings screens, no onboarding. You understand them in ten seconds or you close the tab.

That's the design constraint I try to build to. Most of these took a few hours each. The hard part was figuring out what to leave out.

Claude Code has a Telegram plugin for exactly this. You run claude --channels plugin:telegram@claude-plugins-official, pair your account, and suddenly you can message Claude from anywhere.

The problem: it kept dying.

Crash it once, it's gone. Reboot your Mac, it's gone. And since it runs as a foreground process, the moment you close the terminal, it's gone.

So I built a watchdog.

The constraint that changes everything

Claude Code is a TUI app — it needs a real terminal (a TTY) to run. You can't just start it as a background daemon directly under launchd. It fails silently.

The fix: tmux. Run Claude inside a tmux session, and launchd manages the watchdog that watches that session.

launchd → watchdog.sh → tmux session → claude --channels

This is the core architecture. Everything else is consequence.

Three layers of self-healing

I didn't want to ever think about this again, so I designed for complete autonomy:

Layer 1 — Process: watchdog.sh loops every 60 seconds and runs three checks:

1. Is the tmux session alive?

2. Is the Claude PID alive inside it?

3. Does the Telegram Bot API respond to a getMe call?

If any check fails, it kills the stale session and relaunches.

Layer 2 — Watchdog: The watchdog itself runs under launchd with KeepAlive: true. If watchdog.sh crashes or exits for any reason, launchd restarts it within ~10 seconds.

Layer 3 — Machine: The launchd agent has RunAtLoad: true. Reboot your Mac, log in, and within 60 seconds Claude is back on Telegram. No manual steps.

The only failures that still require human intervention: OAuth token expiry and bot token revocation. Everything else recovers automatically in 10–120 seconds.

Two bugs I hit that weren't obvious

Bug 1: Don't use `getUpdates` as your health check.getUpdates as your health check.**

My first instinct for the Telegram health check was to call getUpdates — if it returns updates, the bot is alive. Seems reasonable.

Wrong. Telegram only allows one long-poll consumer per bot token. The Claude plugin already holds that connection. When my watchdog called getUpdates, Telegram rejected it with a conflict error — and worse, it kicked the plugin's own connection, causing exactly the failure I was trying to detect.

The fix: getMe instead. It's a read-only endpoint that doesn't compete with the plugin's polling loop.

Bug 2: `pgrep` is dangerous when you're running multiple bots.pgrep is dangerous when you're running multiple bots.**

The original version cleaned up stale processes with:

pkill -f "claude.*channels.*telegram"

This killed every Claude channels process on the machine — including a completely separate bot (cc-sdas) that had nothing to do with this watchdog.

The fix: track only your own PID. The launcher writes its process ID to a file. The watchdog reads that file and only ever kills that specific PID. No pattern matching, no collateral damage.

Exponential backoff

If something is fundamentally broken — say, OAuth expired — you don't want the watchdog hammering restarts every 60 seconds forever. So failures back off:

• 1st failure → restart, wait 30s

• 2nd failure → restart, wait 60s

• 3rd+ failure → restart, wait 120s (capped)

• On recovery → reset to normal interval

This prevents restart storms while still recovering reasonably fast when the underlying issue is fixed.

What I'd do differently

One thing I didn't solve: Claude shows a "Do you trust this project?" prompt on first launch in a new tmux session, because the working directory defaults to /. It waits for Enter — silently blocking startup. The workaround is tmux send-keys -t claude-telegram Enter, but the real fix is launching from ~ or pre-trusting the directory.

Also worth noting: this whole setup assumes your Mac stays on and logged in. The battery-dying scenario is genuinely unrecoverable without physical intervention. If you want true 24/7 availability, you'd need a VPS instead of a local Mac. For my use case (it's my own machine), this is good enough.

The repo

Everything is at github.com/ucalyptus/claude-tmux-watchdog — the launcher, watchdog, health check script, and launchd plist. Should work on any Mac running Claude Code with the Telegram plugin.

If you're running Claude Code and want it reachable from your phone without babysitting it, this is the setup.

A world where you had months to spare. A world where hiring a lawyer for a ₹50,000 dispute made economic sense. A world where physically showing up to a courthouse was the only way to be heard.

That world is gone. But the system remains.

The math doesn't work

Here's the absurdity: in India, the average civil case takes 3–4 years to resolve. The average cost of litigation — factoring in lawyer fees, court fees, and lost time — often exceeds the disputed amount itself.

Which means most people don't pursue it. They eat the loss, swallow the injustice, and move on.

This isn't a gap in the system. It's the system working exactly as designed — for people with resources, not for everyone.

What if disputes just... resolved?

That's the question behind Sulajh (Hindi: सुलझ — to untangle, to resolve).

Sulajh is an online dispute resolution platform. You file a claim, the other party responds, a neutral reviews both sides with AI assistance, and you reach a settlement — all online, often in days.

No courtrooms. No travel. No months of waiting.

The process has four steps:

1. File a Claim — describe the dispute, upload your evidence

2. Respondent is Notified — the other party gets a chance to respond

3. Mediation & Review — a case manager and a neutral review both sides; AI flags key points and identifies common ground

4. Resolution — a fair settlement or a binding decision

That's it.

What makes AI useful here

Dispute resolution is fundamentally a reasoning problem. You have two versions of events, a set of documents, and a question: what's fair?

AI doesn't decide that. But it does something valuable: it reads everything. It doesn't miss the clause buried on page 7. It doesn't favor the party who talks more confidently. It doesn't get tired.

The AI in Sulajh acts as a research layer for the neutral — surfacing relevant precedents, summarizing documents, flagging inconsistencies, and identifying where the parties actually agree (which is usually more than either side admits).

The human neutral still makes the call. But they make it with better information, faster.

Who is this for?

• A freelancer chasing an unpaid invoice

• A tenant disputing a wrongful deduction from their security deposit

• A small business with a supplier who delivered defective goods

• A consumer whose warranty claim was rejected without reason

These are disputes that never reach court — not because they're not valid, but because the economics don't make sense. Sulajh makes them resolvable.

The broader picture

Online dispute resolution isn't new. eBay was quietly resolving 60 million disputes a year through automated mediation long before anyone was calling it ODR. PayPal, Airbnb, Uber — every platform-scale company built informal dispute systems because the alternative was chaos.

What's new is making this available outside of platforms. For disputes that happen in the real world, between real people, without a platform intermediary to absorb the coordination cost.

That's the hard part. And that's what Sulajh is trying to solve.

The legal system will not reform itself fast enough. But technology can route around it — not by replacing justice, but by making it accessible.

Sulajh is early. But the direction is right.

If you're dealing with a dispute — commercial, consumer, or otherwise — try Sulajh. Or just reply to this email. I'm curious what kinds of disputes feel most underserved to you. Sulajh. Or just reply to this email. I'm curious what kinds of disputes feel most underserved to you.