LLM Agent Tool-Call Retry: The Loop That Survives Production

The pattern has five pieces. Each one closes one of the failure modes above. They compose into a single loop that you can run against either Anthropic's tool-use API or OpenAI's function-calling API with the same structure on top.

Errors flow back as conversation context

Anthropic documents this explicitly — when a tool throws, return the error message in the tool_result content with is_error: true, and Claude incorporates the error into its next response. OpenAI's pattern is the same shape with a different surface: you return a string in the role: tool message describing what went wrong, and the model decides what to do next. In both cases the agent does not crash. The error becomes a sentence in the conversation, and the model has the full context it needs to either retry with different parameters, switch to a different tool, or surface a clean explanation to the user.

The temptation when you first hit a tool error is to catch it in the loop and crash. Resist. The model is genuinely good at recovering from errors that are described well. The Anthropic docs are explicit about this: write instructive error messages, not generic ones. "Failed" tells the model nothing. "Rate limit exceeded. Retry after 60 seconds." tells the model to wait or try a different approach. The quality of error recovery is downstream of the quality of the error messages, and that is your code, not the model's.

Transient vs permanent error split inside the tool

Transient errors — HTTP 429, HTTP 5xx, connection reset, socket timeout — are infrastructure noise. The tool itself should retry them with exponential backoff before the model ever sees them. Permanent errors — HTTP 4xx other than 429, validation failures, not-found, forbidden — are decisions for the model. Send those back immediately with is_error: true. The split matters because every tool call you bounce back to the model costs another turn, another context window read, and another round of token spend. Absorbing transient noise inside the tool is the difference between a clean conversation history and one polluted with five back-and-forths about the same connection reset.

type ToolResult =
  | { ok: true; content: string }
  | { ok: false; isError: true; content: string };

async function executeToolWithRetry(
  fn: () => Promise<string>,
  opts: { maxAttempts?: number; baseMs?: number } = {},
): Promise<ToolResult> {
  const maxAttempts = opts.maxAttempts ?? 3;
  const baseMs = opts.baseMs ?? 250;

  let lastErr: unknown;
  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
    try {
      return { ok: true, content: await fn() };
    } catch (err) {
      lastErr = err;
      if (!isTransient(err) || attempt === maxAttempts) {
        return {
          ok: false,
          isError: true,
          content: explainError(err), // instructive, not generic
        };
      }
      await sleep(baseMs * 2 ** (attempt - 1) + jitter());
    }
  }
  // Unreachable, but TypeScript wants it.
  return { ok: false, isError: true, content: explainError(lastErr) };
}

function isTransient(err: unknown): boolean {
  if (!(err instanceof HttpError)) return isNetworkError(err);
  return err.status === 429 || (err.status >= 500 && err.status < 600);
}

Idempotency keyed on tool_use_id

The tool, not the loop, owns this. Generate an idempotency key at the start of each tool invocation and pass it to any downstream API that accepts one — Stripe, payment processors, most modern REST APIs. For your own internal tools, persist a "this tool_use_id has already run" row in the same transaction as the side effect, and short-circuit on a duplicate. The tool_use.id from Anthropic's response and the tool_call.id from OpenAI's response are both unique per invocation and both ideal keys for this. The point is that if the loop retries — for any reason, including a process restart in the middle of an agent run — the second execution is a no-op that returns the original result.

async function chargeCustomer(
  toolUseId: string,
  customerId: string,
  amountCents: number,
): Promise<string> {
  return await db.transaction(async (tx) => {
    const existing = await tx.toolSideEffects.findById(toolUseId);
    if (existing) return existing.result; // already ran — return original

    const charge = await stripe.charges.create(
      { customer: customerId, amount: amountCents, currency: "usd" },
      { idempotencyKey: toolUseId },
    );

    const result = JSON.stringify({ charge_id: charge.id, status: charge.status });
    await tx.toolSideEffects.insert({
      tool_use_id: toolUseId,
      result,
      created_at: new Date(),
    });
    return result;
  });
}

Maximum tool-call depth per user prompt

Anthropic's loop exit condition is documented — the loop continues while stop_reason is tool_use and exits on end_turn, max_tokens, stop_sequence, or refusal. That is the model's exit, not yours. You also need a hard count of how many tool calls the loop has executed in this user prompt, and you need to bail when it crosses a ceiling. Twenty-five is a reasonable interactive ceiling. One hundred is reasonable for a long-running background agent. Either way, bail with a structured error that the caller can surface as "the agent took too long to complete this task" instead of letting the process drift into an unbounded loop.

Conversation persisted after every turn

If the host process dies — OOM, container restart, deploy rollover — the user's next request should resume from the last persisted state, not start over. The unit of persistence is the messages array: the system prompt, every user message, every assistant message (including its tool_use blocks), and every user message containing tool_result blocks. Persist after each turn, key it by a conversation ID, and on resume reconstruct the loop from the last saved state. The idempotency layer in the previous piece is what makes resume safe — any tool calls that already executed get short-circuited on replay.

async function runAgentLoop(
  conversationId: string,
  userMessage: string,
  opts: { maxToolCalls?: number } = {},
): Promise<string> {
  const maxToolCalls = opts.maxToolCalls ?? 25;
  let toolCallCount = 0;

  let messages = await db.conversations.loadMessages(conversationId);
  messages.push({ role: "user", content: userMessage });
  await db.conversations.saveMessages(conversationId, messages);

  while (true) {
    const response = await anthropic.messages.create({
      model: "claude-opus-4-7",
      max_tokens: 4096,
      tools: TOOL_DEFINITIONS,
      messages,
    });

    messages.push({ role: "assistant", content: response.content });
    await db.conversations.saveMessages(conversationId, messages);

    if (response.stop_reason !== "tool_use") {
      return extractFinalText(response.content);
    }

    const toolUses = response.content.filter(
      (block): block is ToolUseBlock => block.type === "tool_use",
    );
    toolCallCount += toolUses.length;
    if (toolCallCount > maxToolCalls) {
      throw new AgentBudgetExceededError(conversationId, toolCallCount);
    }

    // Parallel execution: run all tool_use blocks from this turn together.
    const toolResults = await Promise.all(
      toolUses.map(async (block) => {
        const result = await dispatchTool(block.id, block.name, block.input);
        return {
          type: "tool_result" as const,
          tool_use_id: block.id,
          content: result.content,
          is_error: result.ok ? undefined : true,
        };
      }),
    );

    // tool_result blocks must come FIRST in the content array (Anthropic spec).
    messages.push({ role: "user", content: toolResults });
    await db.conversations.saveMessages(conversationId, messages);
  }
}

The OpenAI shape is the same loop with a different surface — tool_calls instead of tool_use blocks, role: "tool" messages instead of tool_result content blocks, tool_call_id instead of tool_use_id. The structural rules are the same: every tool_call from the assistant turn needs a matching tool message in the next request, and parallel tool calls return all their results before the loop continues.

async function runOpenAiAgentLoop(
  conversationId: string,
  userMessage: string,
  opts: { maxToolCalls?: number } = {},
): Promise<string> {
  const maxToolCalls = opts.maxToolCalls ?? 25;
  let toolCallCount = 0;

  let messages = await db.conversations.loadOpenAiMessages(conversationId);
  messages.push({ role: "user", content: userMessage });
  await db.conversations.saveOpenAiMessages(conversationId, messages);

  while (true) {
    const response = await openai.chat.completions.create({
      model: "gpt-4.1",
      messages,
      tools: OPENAI_TOOL_DEFINITIONS,
    });

    const assistantMessage = response.choices[0].message;
    messages.push(assistantMessage);
    await db.conversations.saveOpenAiMessages(conversationId, messages);

    const toolCalls = assistantMessage.tool_calls ?? [];
    if (toolCalls.length === 0) {
      return assistantMessage.content ?? "";
    }

    toolCallCount += toolCalls.length;
    if (toolCallCount > maxToolCalls) {
      throw new AgentBudgetExceededError(conversationId, toolCallCount);
    }

    // Every tool_call.id needs a matching role: tool message in the same batch.
    const toolMessages = await Promise.all(
      toolCalls.map(async (call) => {
        const result = await dispatchTool(
          call.id,
          call.function.name,
          JSON.parse(call.function.arguments),
        );
        return {
          role: "tool" as const,
          tool_call_id: call.id,
          content: result.content,
        };
      }),
    );
    messages.push(...toolMessages);
    await db.conversations.saveOpenAiMessages(conversationId, messages);
  }
}