Pause and Resume Execution | Agent Tool Protocol
Skip to main content

Pause and Resume Execution

One of the most powerful features of Agent Tool Protocol is the ability to pause code execution to request services from the client (like LLM calls or human approval), then resume with the results.

Why Pause/Resume?​

Traditional Approach (Doesn't Work)​

// ❌ This doesn't work - can't call LLM from inside sandbox
const result = await client.execute(`
  const llmResponse = await callAnthropicAPI(prompt);  // Error: No network access!
  return llmResponse;
`);

ATP Approach (Works!)​

// ✅ This works - execution pauses and client provides LLM
const result = await client.execute(`
  const llmResponse = await llm.call({ prompt: "What is 2+2?" });
  return llmResponse;
`);

// If execution pauses, handle callback
if (result.status === 'paused') {
  const llmResult = await myLLM.call(result.needsCallback.payload.prompt);
  const finalResult = await client.resume(result.executionId, llmResult);
}

How It Works​

┌─────────────┐         ┌─────────────┐         ┌──────────────┐
│   Client    │         │   Server    │         │   Sandbox    │
└──────┬──────┘         └──────┬──────┘         └──────┬───────┘
       │                       │                        │
       │──── execute(code) ────>│                       │
       │                       │                        │
       │                       │──── start ────────────>│
       │                       │                        │
       │                       │        llm.call() {    │
       │                       │          pauseForCallback()
       │                       │<─ PAUSED (needsCallback)│
       │<── status: paused ────│                        │
       │                       │                        │
       │ (Client calls LLM)    │                        │
       │                       │                        │
       │── resume(result) ─────>│                       │
       │                       │── resume with result ─>│
       │                       │                   }    │
       │                       │<── COMPLETED ──────────│
       │<── final result ──────│                        │

Basic Usage​

Client Setup​

First, provide the services that code might need:

import { AgentToolProtocolClient } from '@mondaydotcomorg/atp-client';
import Anthropic from '@anthropic-ai/sdk';

const client = new AgentToolProtocolClient({
  baseUrl: 'http://localhost:3333',
});

const anthropic = new Anthropic({
  apiKey: process.env.ANTHROPIC_API_KEY,
});

// Provide LLM service
client.provideLLM({
  async call(prompt, options) {
    const response = await anthropic.messages.create({
      model: 'claude-3-5-sonnet-20241022',
      max_tokens: 4096,
      messages: [{ role: 'user', content: prompt }],
    });
    return response.content[0].text;
  },
});

await client.init();
await client.connect();

Execute with Pause Handling​

async function executeWithPauseHandling(code: string) {
  let result = await client.execute(code);
  
  // Handle pauses in a loop
  while (result.status === 'paused' && result.needsCallback) {
    const callback = result.needsCallback;
    
    // Handle different callback types
    let callbackResult;
    
    if (callback.type === 'llm') {
      callbackResult = await handleLLMCallback(callback);
    } else if (callback.type === 'approval') {
      callbackResult = await handleApprovalCallback(callback);
    } else if (callback.type === 'embedding') {
      callbackResult = await handleEmbeddingCallback(callback);
    }
    
    // Resume execution
    result = await client.resume(result.executionId, callbackResult);
  }
  
  return result;
}

// Use it
const result = await executeWithPauseHandling(`
  const answer = await llm.call({ prompt: "What is 2+2?" });
  return answer;
`);

console.log(result.result); // "4" or similar

LLM Calls​

Basic LLM Call​

const result = await executeWithPauseHandling(`
  const response = await llm.call({
    prompt: "Explain quantum computing in one sentence",
    temperature: 0.7,
    model: "claude-3-5-sonnet-20241022",
  });
  
  return response;
`);

Structured Extraction​

const result = await executeWithPauseHandling(`
  const userData = await llm.extract({
    prompt: "Extract user info: John Doe, age 30, lives in NYC",
    schema: {
      type: "object",
      properties: {
        name: { type: "string" },
        age: { type: "number" },
        city: { type: "string" },
      },
      required: ["name", "age", "city"],
    },
  });
  
  return userData;
`);

console.log(result.result);
// { name: "John Doe", age: 30, city: "NYC" }

Text Classification​

const result = await executeWithPauseHandling(`
  const category = await llm.classify({
    text: "I love this product! It's amazing!",
    categories: ["positive", "negative", "neutral"],
  });
  
  return category;
`);

console.log(result.result); // "positive"

Approval Flow​

Basic Approval​

// Provide approval handler
client.provideApproval({
  async request(message, context) {
    // Show approval dialog to user
    const approved = await showApprovalDialog(message);
    
    return {
      approved,
      response: approved ? "Approved" : "Rejected",
      timestamp: Date.now(),
    };
  },
});

// Code that requires approval
const result = await executeWithPauseHandling(`
  const approved = await approval.request(
    "About to delete all user data. Confirm?"
  );
  
  if (!approved.approved) {
    throw new Error("Operation cancelled by user");
  }
  
  // Proceed with deletion
  await users.deleteAllUsers();
  
  return { deleted: true };
`);

Approval with Context​

const result = await executeWithPauseHandling(`
  const user = await users.getUser({ userId: "123" });
  
  const approved = await approval.request(
    "Delete user account?",
    {
      userName: user.name,
      userId: user.id,
      dataToDelete: ["profile", "orders", "payments"],
    }
  );
  
  if (approved.approved) {
    await users.deleteUser({ userId: user.id });
    return { deleted: true };
  }
  
  return { deleted: false, reason: "User declined" };
`);

Batch Callbacks​

Multiple LLM Calls in Parallel​

When code makes multiple async calls in parallel, ATP batches them:

const result = await client.execute(`
  // These three calls will be batched!
  const [response1, response2, response3] = await Promise.all([
    llm.call({ prompt: "What is 2+2?" }),
    llm.call({ prompt: "What is 3+3?" }),
    llm.call({ prompt: "What is 4+4?" }),
  ]);
  
  return { response1, response2, response3 };
`);

// Check if batched callbacks are needed
if (result.needsCallbacks) {
  console.log(`Processing ${result.needsCallbacks.length} callbacks in parallel`);
  
  // Process all callbacks in parallel
  const callbackResults = await Promise.all(
    result.needsCallbacks.map(async (callback) => {
      const llmResult = await anthropic.messages.create({
        model: 'claude-3-5-sonnet-20241022',
        max_tokens: 1024,
        messages: [{ role: 'user', content: callback.payload.prompt }],
      });
      
      return {
        id: callback.id,  // Important: match the callback ID
        result: llmResult.content[0].text,
      };
    })
  );
  
  // Resume with batch results
  const finalResult = await client.resumeBatch(
    result.executionId,
    callbackResults
  );
  
  console.log(finalResult.result);
  // {
  //   response1: "4",
  //   response2: "6",
  //   response3: "8"
  // }
}

Smart Batching Configuration​

Configure batching behavior on the server:

import { createServer } from '@mondaydotcomorg/atp-server';

const server = createServer({
  execution: {
    smartBatching: {
      enabled: true,
      maxBatchSize: 10,        // Max callbacks per batch
      batchWindowMs: 100,      // Wait up to 100ms to collect callbacks
    },
  },
});

Caching and Replay​

Automatic Caching​

Callback results are automatically cached for replay:

// First execution
const result1 = await client.execute(`
  const response = await llm.call({ prompt: "What is 2+2?" });
  return response;
`);
// Pauses, calls LLM, gets "4"

// Second execution (with cache)
const result2 = await client.execute(`
  const response = await llm.call({ prompt: "What is 2+2?" });
  return response;
`);
// Uses cached result, no pause!

Cache Control​

import { cache } from '@mondaydotcomorg/atp-runtime';

const result = await executeWithPauseHandling(`
  // Clear cache for this key
  await cache.delete("llm:call:what-is-2+2");
  
  // This will pause even if cached before
  const response = await llm.call({ prompt: "What is 2+2?" });
  
  return response;
`);

Replay Mode​

For debugging, you can replay executions:

import { setReplayMode } from '@mondaydotcomorg/atp-runtime';

const result = await client.execute(`
  // Enable replay mode
  setReplayMode(true);
  
  // These will use cached results
  const r1 = await llm.call({ prompt: "What is 2+2?" });
  const r2 = await llm.call({ prompt: "What is 3+3?" });
  
  return { r1, r2 };
`);

Advanced Patterns​

Retry with Pause​

async function executeWithRetry(code: string, maxRetries = 3) {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      let result = await client.execute(code);
      
      // Handle pauses
      while (result.status === 'paused' && result.needsCallback) {
        const callback = result.needsCallback;
        const callbackResult = await handleCallback(callback);
        result = await client.resume(result.executionId, callbackResult);
      }
      
      if (result.status === 'completed') {
        return result;
      }
      
      if (!result.error?.retryable) {
        throw new Error(result.error?.message);
      }
      
      // Wait before retry
      await sleep(1000 * (attempt + 1));
    } catch (error) {
      if (attempt === maxRetries - 1) throw error;
    }
  }
}

Conditional Approval​

const result = await executeWithPauseHandling(`
  const amount = await payments.getTransactionAmount({ txId: "123" });
  
  // Only require approval for large amounts
  if (amount > 1000) {
    const approved = await approval.request(
      \`Approve large payment of $\${amount}?\`,
      { amount, threshold: 1000 }
    );
    
    if (!approved.approved) {
      return { cancelled: true };
    }
  }
  
  await payments.processTransaction({ txId: "123" });
  return { processed: true, amount };
`);

Progress Tracking During Pauses​

const result = await client.execute(`
  progress.report("Starting processing", 0);
  
  for (let i = 0; i < 10; i++) {
    progress.report(\`Processing item \${i + 1}/10\`, (i + 1) / 10);
    
    // Each iteration might pause
    const result = await llm.call({ 
      prompt: \`Process item \${i}\` 
    });
    
    await processItem(i, result);
  }
  
  progress.report("Completed", 1);
  return { processed: 10 };
`);

Error Handling​

Handling Pause Errors​

async function safeExecuteWithPause(code: string) {
  try {
    let result = await client.execute(code);
    
    while (result.status === 'paused' && result.needsCallback) {
      try {
        const callback = result.needsCallback;
        const callbackResult = await handleCallback(callback);
        result = await client.resume(result.executionId, callbackResult);
      } catch (error) {
        // Resume with error
        result = await client.resume(result.executionId, {
          error: error.message,
        });
      }
    }
    
    return result;
  } catch (error) {
    console.error('Execution error:', error);
    throw error;
  }
}

Timeout During Pause​

async function executeWithPauseTimeout(code: string, pauseTimeout = 60000) {
  let result = await client.execute(code);
  
  while (result.status === 'paused' && result.needsCallback) {
    const timeoutPromise = new Promise((_, reject) =>
      setTimeout(() => reject(new Error('Pause timeout')), pauseTimeout)
    );
    
    const callbackPromise = handleCallback(result.needsCallback);
    
    try {
      const callbackResult = await Promise.race([
        callbackPromise,
        timeoutPromise,
      ]);
      
      result = await client.resume(result.executionId, callbackResult);
    } catch (error) {
      // Cancel execution on timeout
      await client.cancel(result.executionId);
      throw error;
    }
  }
  
  return result;
}

Best Practices​

1. Always Handle Pauses​

// ❌ Bad: Doesn't handle pauses
const result = await client.execute(code);
console.log(result.result); // Might be undefined if paused!

// ✅ Good: Handles pauses
async function execute(code: string) {
  let result = await client.execute(code);
  
  while (result.status === 'paused' && result.needsCallback) {
    const callbackResult = await handleCallback(result.needsCallback);
    result = await client.resume(result.executionId, callbackResult);
  }
  
  return result;
}

2. Use Batch Callbacks When Possible​

// ❌ Slow: Sequential pauses
const result = await client.execute(`
  const r1 = await llm.call({ prompt: "Q1" });
  const r2 = await llm.call({ prompt: "Q2" });
  const r3 = await llm.call({ prompt: "Q3" });
`);
// 3 separate pause/resume cycles!

// ✅ Fast: Batched pauses
const result = await client.execute(`
  const [r1, r2, r3] = await Promise.all([
    llm.call({ prompt: "Q1" }),
    llm.call({ prompt: "Q2" }),
    llm.call({ prompt: "Q3" }),
  ]);
`);
// 1 pause/resume cycle with batch!

3. Implement Timeout Protection​

// Always set reasonable timeouts
const result = await executeWithPauseTimeout(code, 60000); // 1 minute max

4. Cache Expensive Operations​

const result = await client.execute(`
  // Check cache first
  const cached = await cache.get("expensive-operation");
  if (cached) {
    return cached;
  }
  
  // Expensive LLM call
  const result = await llm.call({
    prompt: "Very long prompt...",
  });
  
  // Cache for 1 hour
  await cache.set("expensive-operation", result, 3600);
  
  return result;
`);

5. Provide Context in Approvals​

// ❌ Bad: No context
await approval.request("Approve?");

// ✅ Good: Rich context
await approval.request(
  "Delete user account?",
  {
    userName: user.name,
    accountAge: user.createdAt,
    dataSize: user.dataSize,
    warningLevel: "high",
  }
);

Next Steps​

Common Issues​

Issue: Execution Never Resumes​

Problem: Forgot to handle pauses

// ❌ Wrong
const result = await client.execute(code);
console.log(result.result); // undefined!

Solution: Always check for pauses

// ✅ Correct
let result = await client.execute(code);
if (result.status === 'paused') {
  const callback = await handleCallback(result.needsCallback);
  result = await client.resume(result.executionId, callback);
}

Issue: Slow Execution​

Problem: Not using batch callbacks

Solution: Use Promise.all() for parallel operations

Issue: Cache Not Working​

Problem: Cache keys are different

Solution: Ensure consistent cache keys or use sequence numbers

// Use sequence numbers for automatic cache key generation
const r1 = await llm.call({ prompt: "Q1" }); // Sequence 0
const r2 = await llm.call({ prompt: "Q1" }); // Sequence 1 (different!)

// Or use explicit cache
await cache.set("my-key", result);
Agent Tool Protocol | ATP - Code Execution for AI Agents