Surviving Eviction: Durable Tool Approvals in a Stateless Agent World

When an AI agent reaches for a dangerous tool — rm -rf, a database DROP, a production deploy — it should ask the user first. That part is straightforward. The hard part is what happens next.

Most systems solve this by parking the agent in memory: an asyncio.Event waits for the user’s response, holding the agent’s entire state resident in RAM until someone clicks “Allow” or “Deny.” The agent can’t be evicted. The connection must stay alive. If the process restarts, the approval is lost.

Codumentor takes a different approach: the agent is allowed to go. Its task returns, the agent becomes eligible for LRU eviction, the process can even restart — and when the user responds hours later, execution resumes from the exact tool call without re-asking the LLM.

This article covers the system we built to make that work, including three subtle bugs that shaped the final design.

The Problem: Pinning vs. Evicting

Consider the lifecycle of a permission request in a typical agent system:

1. Agent calls a dangerous tool
2. System raises a “permission needed” signal
3. Agent waits in memory for the user response
4. User responds; agent continues

Step 3 is the problem. The agent is pinned — it occupies memory, holds connections, and cannot be reclaimed. In a system serving hundreds of concurrent conversations, every pending approval consumes resources indefinitely. Worse, if the process restarts — deployment, crash, OOM kill — every pending approval is lost and the agent must restart from scratch.

The alternative — returning from the agent task and resuming later — sounds simple but introduces serious correctness challenges:

• The agent’s in-memory state (transcript, scratchpad, context variables) is gone
• The assistant message with tool calls must already be persisted — otherwise there’s nothing to resume from
• Completed tool results from earlier in the same turn must survive
• The resume path must skip the LLM call and go straight to executing the awaiting tool

Each of these is a correctness invariant that, if violated, causes silent data corruption or infinite loops.

Four Design Objectives

Our design doc establishes four non-negotiable requirements:

1. Approvals survive eviction and restart — the pending request lives in SQLite, not memory
2. The agent run returns — no parking, no asyncio.Event; the task exits cleanly so the agent becomes LRU-evictable
3. Resume re-enters cleanly — a fresh agent instance picks up where the old one left off, using the recorded decision
4. UI indicates the state — an orange dot shows turn_status === "awaiting_approval" so users know the agent is waiting

Three Load-Bearing Invariants

The entire system rests on three verified invariants. If any of these break, the durable approval mechanism fails silently.

Invariant 1: The assistant message with tool calls is persisted to disk before any tool runs.

In streaming.py, the sequence is strict:

# streaming.py — persist FIRST, execute SECOND
# 1. Add to in-memory messages
slp.append_assistant_message_with_tools(...)

# 2. Persist to storage
self.conversation_manager.store_assistant_message_with_tool_calls(...)

# 3. Now safe to execute — assistant message is on disk
async for output in self.tool_runtime.execute_streaming(tool_calls, ...):
    yield output

If tools ran before persistence, an eviction mid-execution would leave no record of which tools were called — the resume path wouldn’t know what to execute.

Invariant 2: request_permission has a single caller.

The permission manager’s request_permission() method is called exclusively from agent_runner.py, injected as an input_callback. This means we know exactly where the approval signal originates and can reason about the call stack.

Invariant 3: Transcript and plugin events survive eviction; in-memory state does not.

Messages are in SQLite and survive. Plugin events are in SQLite and survive. But the agent instance, its scratchpad, and any ApprovalPending objects in memory — all gone after eviction. The durable store must carry everything needed to resume.

The Approval Flow: Suspend

Let’s trace what happens when an agent calls a tool that requires permission.

Step 1: Permission Check

Inside tool_runtime.execute_streaming(), each tool call is checked for permissions:

# tool_runtime.py — permission check before execution
if perm_request is not None:
    _perm_cv_token = current_tool_call_id_var.set(tool_call.id)
    try:
        allowed = await self.permission_provider.request(
            tool_name=tool_call.name,
            tool_call_id=tool_call.id,
            request=perm_request,
        )
    finally:
        current_tool_call_id_var.reset(_perm_cv_token)

Note the current_tool_call_id_var — a contextvars.ContextVar carrying the active tool_call_id across async boundaries. This ensures the permission provider can key durable rows correctly even when the call chain crosses multiple async await points.

Step 2: Durable Persistence & Signal

The permission provider chains to permission_manager.request_permission(). The decision flow:

async def request_permission(self, conversation_id, turn_id,
                              tool_call_id, tool_name,
                              permission_type, details,
                              emit_callback) -> PermissionDecision:
    """
    Decision flow:
    1. Auto-approve-all (still honours deny rules).
    2. Per-conversation always-allow (still honours deny rules).
    3. Rule-based deny / allow.
    4. Durable resume: if a row for this tool call already has
       a decision, return it synchronously.
    5. Otherwise persist a new pending row, emit permission.request,
       and raise ApprovalPending so the agent task unwinds.
    """

    # Steps 1-3: policy checks (omitted for brevity)

    # Step 4: check for existing decision (resume path)
    existing = self._approval_store.get_pending_approval_by_tool_call(
        conversation_id, tool_call_id,
    )
    if existing is not None and existing.decision is not None:
        return PermissionDecision(existing.decision)

    # Step 5: first call — persist and raise
    request_id = f"perm_{conversation_id}_{tool_call_id}"
    self._approval_store.create_pending_approval(
        request_id=request_id,
        conversation_id=conversation_id,
        turn_id=turn_id,
        tool_call_id=tool_call_id,
        tool_name=tool_name,
        permission_type=permission_type,
        details=details,
    )
    await emit_callback("permission.request", { ... })
    raise ApprovalPending(
        request_id=request_id,
        conversation_id=conversation_id,
        turn_id=turn_id,
        tool_call_id=tool_call_id,
        tool_name=tool_name,
        permission_type=permission_type,
        details=details,
    )

The request_id format perm_{conversation_id}_{tool_call_id} is deliberate — an earlier version used perm_{tool_call_id} alone, which caused cross-conversation collisions when mock fixtures (and occasionally real LLMs) reused the same tool_call_id across conversations. The INSERT OR IGNORE would silently skip the second conversation’s row, and the resume path would read the stale row’s resolved decision.

Step 3: Exception Unwinds the Stack

The ApprovalPending exception propagates upward. Here’s where Bug #1 and Bug #2 matter:

Bug #1: execute_tool swallows all exceptions.

The tool registry has a broad exception handler in agent/tools.py:

# agent/tools.py — the broad catch that almost swallowed ApprovalPending
try:
    result = await tool.execute(args)
except ApprovalPending:
    raise  # Must come before the broad except!
except Exception as exc:
    # Converts ALL exceptions to error ToolResults
    return ToolResult(error=str(exc), tool_call_id=tool_call_id)

Without the targeted except ApprovalPending: raise before the broad except Exception, the approval signal is converted to an error ToolResult — the agent never suspends, and the UI never shows a permission request. This pattern appears in three places across tools.py, and the code carries an explicit comment: “Must bubble above the broad except Exception below, or the…”

Bug #2: reconcile_orphan_tool_results poisons the transcript.

In streaming.py, a broad except BaseException handler cleans up orphaned tool calls:

# streaming.py — critical exception ordering
try:
    async for output in self.tool_runtime.execute_streaming(tool_calls, ...):
        yield output
except ApprovalPending:
    # MUST come before except BaseException — otherwise
    # reconcile_orphan_tool_results writes a synthetic "canceled"
    # tool message for the awaiting tool_call_id, committing a
    # result before the user decides and poisoning the resume path.
    raise
except BaseException:
    # Handles AbortTurnSignal, CancelledError, etc.
    slp.reconcile_orphan_tool_results(self, tool_calls, session_id, user_id)
    raise

If ApprovalPending isn’t caught first, reconcile_orphan_tool_results fabricates a synthetic “canceled” tool message for the awaiting tool_call_id. This commits a result to the transcript before the user has decided — when the user responds later, the system sees a completed tool call and skips execution. The approval is effectively denied without anyone knowing.

Step 4: Sibling Durability

Bug #3: Multiple tool calls per turn.

An assistant turn often produces multiple tool calls. They execute serially in tool_runtime.py. If tool-1 succeeds, tool-2 raises ApprovalPending, and the agent is evicted before resuming — tool-1’s result is lost because it only existed in an in-memory list.

The fix: persist each tool result immediately, not in a batch at the end:

def _persist_single_result(self, result: ToolResult,
                            session_id: str, user_id: str) -> None:
    """Persist one tool result immediately. Keeps completed siblings
    durable when a later tool in the same batch raises ApprovalPending."""
    if result.tool_call_id in self._persisted_tool_call_ids:
        return  # Idempotent

    message = Message(
        role="tool", content=result.content,
        tool_call_id=result.tool_call_id,
    )
    self.storage.add_message(message, session_id, user_id)
    self._persisted_tool_call_ids.add(result.tool_call_id)

After each successful tool execution — and even error results — the result is persisted:

tool_results.append(result)
self._persist_single_result(result, session_id, user_id)

This pattern appears at five call sites across execute_streaming and related flows. It makes the end-of-batch store_results call idempotent — it only writes results not already persisted individually.

Step 5: Agent Task Returns

At the top level of streaming.py:run_loop, the ApprovalPending is caught one more time:

except ApprovalPending:
    # Suspended turn — let the caller translate this into
    # a turn.awaiting_approval SSE event. Do not log as error,
    # do not emit onError, do not yield a user-facing error string.
    raise

The agent runner catches it, persists plugin KV snapshots, emits a turn.awaiting_approval SSE event, and returns from the agent task. The agent is no longer pinned — it’s eligible for LRU eviction. The process can restart. The approval sits in SQLite, waiting.

The SQLite Store

Pending approvals live in a dedicated table:

CREATE TABLE IF NOT EXISTS ui_pending_approvals (
    request_id        TEXT PRIMARY KEY,
    conversation_id   TEXT NOT NULL,
    turn_id           TEXT NOT NULL,
    tool_call_id      TEXT NOT NULL,
    tool_name         TEXT NOT NULL,
    permission_type   TEXT NOT NULL,
    details_json      TEXT NOT NULL,
    created_at        TEXT NOT NULL,
    resolved_at       TEXT,
    decision          TEXT,
    kv_snapshot_json  TEXT,
    UNIQUE(conversation_id, tool_call_id)
);

CREATE INDEX idx_pending_approvals_conv_unresolved
    ON ui_pending_approvals(conversation_id) WHERE resolved_at IS NULL;

The partial index on unresolved approvals enables fast lookup of all pending requests for a conversation. The UNIQUE(conversation_id, tool_call_id) constraint prevents duplicate rows. The kv_snapshot_json column stores per-session context variable snapshots so plugin state survives the suspend/resume cycle — added via online migration for backward compatibility.

All operations are wrapped in a threading lock and use SQLite transactions for atomicity:

def create_pending_approval(self, request_id, conversation_id,
                             turn_id, tool_call_id, tool_name,
                             permission_type, details: dict) -> None:
    with self._lock, self._connect() as conn:
        conn.execute(
            """INSERT OR IGNORE INTO ui_pending_approvals(
                request_id, conversation_id, turn_id, tool_call_id,
                tool_name, permission_type, details_json, created_at
            ) VALUES (?, ?, ?, ?, ?, ?, ?, ?)""",
            (request_id, conversation_id, turn_id, tool_call_id,
             tool_name, permission_type, json.dumps(details), utc_now_iso()),
        )
        conn.commit()

The User Responds

When the user clicks “Allow” or “Deny” in the UI:

async def resolve_permission(self, request_id: str,
                              decision: PermissionDecision,
                              expected_conversation_id: Optional[str] = None
                              ) -> bool:
    # Validate ownership
    if expected_conversation_id:
        # Ensures the decision is applied to the right conversation
        ...

    # Handle ALWAYS_ALLOW — cache for future requests
    if decision == PermissionDecision.ALWAYS_ALLOW:
        self._always_allow[expected_conversation_id].add(tool_name)
        decision = PermissionDecision.ALLOW

    # Stamp the row atomically
    return self._approval_store.resolve_pending_approval(
        request_id, decision.value, expected_conversation_id
    )

The resolve_pending_approval method performs an atomic UPDATE setting decision and resolved_at. A resume task is then scheduled.

The Resume Path

Resuming is where the design earns its keep. A fresh agent instance — possibly on a restarted process — picks up exactly where the old one stopped.

Entry Point: run_loop_from_pending_tool_calls

Instead of starting a new turn (which would call the LLM), we enter through a specialized resume path:

async def run_loop_from_pending_tool_calls(
    self,
    pending_tool_calls: List[ToolCall],
    ...
) -> AsyncGenerator[str, None]:
    """Resume an interrupted turn by executing the pending tool calls
    without re-asking the LLM for the assistant message.

    The assistant message carrying pending_tool_calls is already in storage.
    We execute the awaiting tool — whose permission provider now sees the
    recorded decision and returns synchronously — persist results, then
    hand control to the normal run_loop for follow-up iterations.

    Callers must pass pending_tool_calls filtered to ids that have no
    persisted tool result yet; completed siblings must not be re-run.
    """

This is the key architectural decision: skip the LLM, go straight to tool execution. The assistant message is already in the transcript (Invariant 1). We just need to execute the remaining tools.

The Magic: Stored Decision Returns Synchronously

When the awaiting tool’s permission check runs again, request_permission hits Step 4:

existing = self._approval_store.get_pending_approval_by_tool_call(
    conversation_id, tool_call_id,
)
if existing is not None and existing.decision is not None:
    # Resume call — return the recorded decision synchronously
    logger.info(
        f"Permission request resumed for tool_call_id={tool_call_id} "
        f"(decision={existing.decision})"
    )
    return PermissionDecision(existing.decision)

No ApprovalPending raised. No SSE event emitted. The tool executes normally with the user’s decision. After all pending tools complete, control passes to the normal run_loop for the next LLM iteration:

# After pending tools complete, continue with normal loop
async for chunk in self.run_loop(
    messages, session_id, user_id, run_id=run_id, ctx=ctx,
):
    yield chunk

KV Snapshot Restoration

Plugin state stored in ctx["kv"] would be lost when the agent is rebuilt. The KV snapshot mechanism preserves it:

On suspend, each catch site captures the context:

def capture_kv_into_approval(exc: "ApprovalPending", ctx: Any) -> None:
    """Stash ctx['kv'] onto an in-flight ApprovalPending.
    Uses ctx['session_id'] as the key so resumed agents look up their own kv."""
    session_id = ctx["session_id"]
    exc.kv_snapshots[session_id] = serializable_kv_snapshot(ctx["kv"])

serializable_kv_snapshot() walks the dict and drops non-JSON-serializable entries — locks, file handles, and the like are silently discarded. The snapshot is persisted to kv_snapshot_json in the approval row.

On resume, the snapshot is restored:

# Restore main agent's kv directly
if approval_record.kv_snapshot:
    ctx["kv"] = approval_record.kv_snapshot.get(session_id, {})

Subagent KV snapshots ride in ctx["_resume_kv_snapshots"] and are popped by subagent_runner on the way down, ensuring nested agents recover their own state.

The Complete Flow

sequenceDiagram
    participant UI as User Interface
    participant PM as Permission Manager
    participant TS as SQLite Store
    participant TR as Tool Runtime
    participant SS as Streaming Loop
    participant AR as Agent Runner
    participant LLM as LLM

    Note over AR,TS: Phase 1: Suspend
    AR->>LLM: Generate response with tool calls
    LLM-->>AR: Assistant message + tool calls
    AR->>SS: run_loop(tool_calls)
    SS->>SS: Persist assistant message to storage
    SS->>TR: execute_streaming(tool_calls)

    loop Each tool call
        TR->>PM: request_permission()
        alt Auto-approve / rule match
            PM-->>TR: ALLOW (sync)
            TR->>TR: Execute tool
            TR->>TS: _persist_single_result()
        else No policy match
            PM->>TS: create_pending_approval()
            PM->>UI: Emit permission.request SSE
            PM-->>TR: raise ApprovalPending
            TR->>TR: Persist completed siblings
            TR-->>SS: ApprovalPending propagates
            SS-->>AR: ApprovalPending propagates
            AR->>TS: _persist_kv_snapshot()
            AR->>UI: Emit turn.awaiting_approval SSE
            AR-->>AR: Return (agent evictable)
        end
    end

    Note over UI,TS: Phase 2: User Responds
    UI->>PM: resolve_permission(ALLOW)
    PM->>TS: resolve_pending_approval(decision=ALLOW)
    TS-->>PM: Updated

    Note over AR,TS: Phase 3: Resume
    AR->>SS: run_loop_from_pending_tool_calls()
    SS->>TR: execute_streaming(pending_tool_calls)
    TR->>PM: request_permission()
    PM->>TS: get_pending_approval_by_tool_call()
    TS-->>PM: Row with decision=ALLOW
    PM-->>TR: ALLOW (sync, no raise)
    TR->>TR: Execute tool
    TR->>TS: Persist result
    TR-->>SS: Complete
    SS->>SS: run_loop() for next LLM iteration
    SS->>LLM: Continue conversation

Why This Matters

The durable approval system isn’t just about resource management. It’s about correctness under failure:

• Process restarts: Deployments, crashes, OOM kills — approvals survive because they’re in SQLite, not memory
• LRU eviction: The system can serve thousands of conversations without pinning agents in memory
• Graceful degradation: If the LLM call fails mid-turn, completed tool results are already persisted
• No silent denials: The transcript never contains fabricated “canceled” results for pending approvals

The three bugs we encountered — swallowed exceptions, poisoned transcripts, lost sibling results — are the kinds of issues that surface only in production under load. They’re not academic concerns; they’re the difference between a system that works and one that silently misbehaves.

Trade-offs

No design is free. The durable approval system has costs:

• Complexity: The exception ordering requirements, per-iteration persistence, and KV snapshot mechanism add significant complexity compared to a simple asyncio.Event
• SQLite contention: The approval store uses a threading lock — under extreme concurrency, this could become a bottleneck (though in practice, permission requests are rare enough that this hasn’t been observed)
• Resume latency: Building a fresh agent instance on resume takes longer than resuming a parked task — though the alternative (pinning agents forever) is worse at scale
• KV snapshot limitations: Non-JSON-serializable plugin state is silently dropped — plugins must keep their ctx["kv"] entries serializable

These trade-offs are acceptable for the benefits: correctness, evictability, and restart resilience.

Takeaways

Durable approvals in a stateless agent world require treating the approval as a first-class persisted entity — not an in-memory signal. The key principles:

1. Persist before execute — the assistant message with tool calls must be on disk before any tool runs
2. Catch specifically, before broadly — ApprovalPending must be caught before Exception or BaseException at every level
3. Persist incrementally — each successful tool result is written immediately, not batched at the end
4. Skip the LLM on resume — re-enter at tool execution, not at the top of the agent loop
5. Snapshot context — plugin state in ctx["kv"] must survive the agent rebuild

The system is elegant in its simplicity: a SQLite row, an exception that propagates, and a resume path that trusts the stored decision. But the elegance depends on getting the exception ordering, persistence timing, and context restoration exactly right.

Three subtle bugs taught us that.

---

This article is part of an ongoing series exploring Codumentor’s architecture.