Architecture Overview

Understanding the architecture of Agent Tool Protocol will help you build robust and efficient AI agent systems.

High-Level Architecture

Core Components

1. Client Layer

The client layer is responsible for:

Code Execution: Sending TypeScript/JavaScript code to the server
API Discovery: Searching and exploring available APIs
Service Providers: Providing LLM, approval, and embedding services
Client Tools: Registering tools that execute locally
Session Management: Handling authentication and token rotation

Key Classes:

AgentToolProtocolClient: Main client interface
ClientSession: Session and authentication management
APIOperations: API discovery and exploration
ExecutionOperations: Code execution and pause/resume
ServiceProviders: LLM, approval, and embedding providers

2. Server Layer

The server layer manages:

Sandbox Execution: Running code in isolated V8 sandboxes
API Aggregation: Combining multiple API sources
Security Enforcement: Applying security policies and provenance tracking
Search & Discovery: Semantic search and API exploration
State Management: Managing paused executions and caching

Key Classes:

AgentToolProtocolServer: Main server interface
SandboxExecutor: Isolated code execution engine
APIAggregator: API source management
SearchEngine: Semantic search implementation
ExecutionStateManager: Pause/resume state management

3. Runtime Layer

The runtime is injected into sandboxes and provides:

LLM API: Client-side LLM calls that pause execution
Cache API: Persistent caching across executions
Approval API: Human-in-the-loop approval requests
Progress API: Progress reporting during execution
Embedding API: Vector embeddings for semantic operations

Available in Sandbox:

// These are available in executed code
import { llm, cache, approval, progress } from '@mondaydotcomorg/atp-runtime';

// LLM calls (pauses execution)
const result = await llm.call({ prompt: "..." });

// Caching
await cache.set("key", value, 3600);
const cached = await cache.get("key");

// Approval requests
const { approved } = await approval.request("Delete this file?");

// Progress reporting
progress.report("Processing item 5/10", 0.5);

4. Protocol Layer

Defines core types and interfaces:

Type Definitions: TypeScript interfaces for all protocol types
JSON Schemas: Validation schemas for requests/responses
Authentication: Auth provider interfaces
OAuth: OAuth flow support

5. Provenance Layer

Tracks data lineage and enforces security:

Provenance Tracking: Where data comes from and where it goes
Security Policies: Data exfiltration prevention, access control
Policy Engine: Evaluates policies on data operations
Code Instrumentation: Automatic tracking in executed code

Data Flow

Execution Flow

API Call Flow

Pause/Resume Flow

Sandbox Security

Isolation Model

Each execution runs in a separate V8 isolate with:

TypeScript/JavaScript execution
Access to injected APIs
Limited memory and CPU
Timeout enforcement

Memory Management

// Configurable memory limits
const server = createServer({
  execution: {
    memory: 128 * MB,  // Per execution
    timeout: 30000,    // 30 seconds
  },
});

The sandbox monitors:

Heap usage
Stack depth
Object creation
String allocations

Code Transformation

Before execution, code is transformed to:

Inject Runtime: Add runtime SDK imports
Track Provenance: Instrument data operations
Apply Security: Enforce security policies
Optimize: Remove unnecessary code

API Aggregation

Multiple Sources

The server can aggregate APIs from:

OpenAPI/Swagger:

const petstoreAPI = await loadOpenAPI({
  url: 'https://petstore3.swagger.io/api/v3/openapi.json',
  name: 'petstore',
});

Custom TypeScript:

server.registerAPI('users', {
  getUser: {
    description: '...',
    inputSchema: { ... },
    handler: async (params) => { ... },
  },
});

MCP Servers:

import { MCPConnector } from '@mondaydotcomorg/atp-mcp-adapter';

const mcpAPI = await MCPConnector.connect({
  url: 'http://localhost:8080/mcp',
  name: 'filesystem',
});

server.addAPIGroup(mcpAPI);

API Namespacing

APIs are organized in namespaces:

// Sandbox code can call:
await users.getUser({ userId: '123' });
await products.getProduct({ productId: '456' });
await petstore.getPetById({ petId: 789 });

Provenance Tracking

Data Lineage

Track where data comes from:

// Code with provenance
const user = await users.getUser({ userId: '123' });
// user now has provenance: { source: 'users.getUser', ... }

const name = user.name;
// name inherits provenance from user

await llm.call({ prompt: `Hello ${name}` });
// Provenance tracker can detect data flow to LLM

Security Policies

Enforce policies on data flow:

import { preventDataExfiltration } from '@mondaydotcomorg/atp-server';

const server = createServer({
  execution: {
    provenanceMode: 'track',
    securityPolicies: [
      preventDataExfiltration({
        sources: ['users.getUser', 'payments.getTransaction'],
        recipients: ['external-api'],
      }),
    ],
  },
});

Caching Architecture

Multi-Level Caching

Cache Keys

// LLM call caching
const cacheKey = hash({
  operation: 'llm.call',
  sequenceNumber: 1,
  prompt: "What is 2+2?",
});

// API result caching
const cacheKey = hash({
  api: 'users.getUser',
  params: { userId: '123' },
});

Communication Protocol

JSON-RPC 2.0

All communication uses JSON-RPC 2.0:

Request:

{
  "jsonrpc": "2.0",
  "id": "123",
  "method": "execute",
  "params": {
    "code": "return await users.getUser({ userId: '123' });",
    "config": {
      "timeout": 30000,
      "maxMemory": 134217728
    }
  }
}

Response:

{
  "jsonrpc": "2.0",
  "id": "123",
  "result": {
    "executionId": "exec_456",
    "status": "completed",
    "result": { "id": "123", "name": "John" },
    "stats": {
      "duration": 145,
      "memoryUsed": 1234567,
      "llmCallsCount": 0
    }
  }
}

Available Methods

init: Initialize client session
connect: Connect and discover APIs
execute: Execute code
resume: Resume paused execution
resumeBatch: Resume with batch callbacks
search: Search for APIs
explore: Explore API structure
definitions: Get type definitions

Scalability

Horizontal Scaling

Multiple server instances can run behind a load balancer:

Requirements for scaling:

Redis: For shared caching and session state
Stateless Execution: Each request is independent
Load Balancing: Distribute requests evenly

Performance Considerations

Server:

Pool V8 isolates for reuse
Implement connection pooling for APIs
Cache API schemas and type definitions
Use Redis for distributed caching

Client:

Reuse client instances
Batch callbacks when possible
Implement request queuing
Cache type definitions locally

Next Steps

Execution Model: Understand how code execution works
Sandbox Security: Learn about security features
Provenance Tracking: Implement data lineage
Client-Server Communication: Understand the protocol

Key Takeaways

Layered Architecture: Client, Server, Runtime, and Protocol layers
Secure Execution: Isolated V8 sandboxes with resource limits
Flexible APIs: Support for OpenAPI, custom APIs, and MCP
Provenance Tracking: Built-in data lineage for security
Pause/Resume: Handle async operations like LLM calls
Scalable: Horizontal scaling with Redis
Type-Safe: Full TypeScript support throughout

High-Level Architecture​

Core Components​

1. Client Layer​

2. Server Layer​

3. Runtime Layer​

4. Protocol Layer​

5. Provenance Layer​

Data Flow​

Execution Flow​

API Call Flow​

Pause/Resume Flow​

Sandbox Security​

Isolation Model​

Memory Management​

Code Transformation​

API Aggregation​

Multiple Sources​

API Namespacing​

Provenance Tracking​

Data Lineage​

Security Policies​

Caching Architecture​

Multi-Level Caching​

Cache Keys​

Communication Protocol​

JSON-RPC 2.0​

Available Methods​

Scalability​

Horizontal Scaling​

Performance Considerations​

Next Steps​

Key Takeaways​