TestAgent

API reference for TestAgent

The TestAgent class runs prompts with MCP tools enabled. It handles the multi-step prompt loop and returns rich result objects.

Import

import { TestAgent } from "@mcpjam/sdk";

Constructor

new TestAgent(options: TestAgentOptions)

Parameters

options · TestAgentOptions · required

Configuration for the test agent.

TestAgentOptions

Property	Type	Required	Default	Description
tools	Record<string, Tool>	Yes	-	MCP tools from manager.getTools()
model	string	Yes	-	Model identifier in provider/model format
apiKey	string	Yes	-	API key for the LLM provider
systemPrompt	string	No	undefined	System prompt for the LLM
temperature	number	No	undefined	Sampling temperature (0-2). If undefined, uses model default.
maxSteps	number	No	10	Maximum agentic loop iterations
customProviders	Record<string, CustomProvider>	No	undefined	Custom LLM provider definitions
mcpClientManager	MCPClientManager	No	undefined	Same connected manager used for connectToServer / getToolsForAiSdk. Required to populate getWidgetSnapshots() on each PromptResult (see note below).

Example

const agent = new TestAgent({
  tools: await manager.getTools(),
  model: "anthropic/claude-sonnet-4-20250514",
  apiKey: process.env.ANTHROPIC_API_KEY,
  systemPrompt: "You are a helpful assistant.",
  temperature: 0.3,
  maxSteps: 5,
});

MCP Apps, traces, and mcpClientManager: MCP App widgets are backed by HTML served from an MCP resource (ui.resourceUri), not by the tool’s JSON result alone. After each tool call, TestAgent uses the manager’s readResource to fetch that HTML and attach widgetSnapshots to the PromptResult. When you upload results to MCPJam, those snapshots are stored so the Evals trace viewer can replay the widget offline. If you omit mcpClientManager, prompts still run and tools still execute — but widgetSnapshots stay empty and uploaded traces will only contain messages and spans (no embedded app replay). Passing manager is also what enables replay credential capture for authenticated HTTP servers when reporting to MCPJam.

---

Methods

prompt()

Sends a prompt to the LLM and returns the result.

prompt(
  message: string,
  options?: PromptOptions
): Promise<PromptResult>

Use getWidgetSnapshots() on the returned result (or toEvalResult() when reporting) to access MCP App HTML captured during the run when mcpClientManager was set.

Parameters

Parameter	Type	Required	Description
message	string	Yes	The user prompt
options	PromptOptions	No	Additional options

PromptOptions

Property	Type	Description
context	PromptResult | PromptResult[]	Previous result(s) for multi-turn conversations
abortSignal	AbortSignal	Cancels the prompt runtime when aborted
stopWhen	StopCondition<ToolSet> | Array<StopCondition<ToolSet>>	Additional conditions for the multi-step prompt loop. Tools still execute normally. TestAgent always applies stepCountIs(maxSteps) as a safety guard.
timeout	number | { totalMs?: number; stepMs?: number; chunkMs?: number }	Bounds prompt runtime. number and totalMs cap the full prompt, stepMs caps each generation step, and chunkMs is accepted for parity but is mainly relevant to streaming APIs.
timeoutMs	number	Shortcut for a total prompt timeout in milliseconds
stopAfterToolCall	string | string[]	Short-circuits the named tool(s) with a stub result and stops after the step where they were called. Tool names and args are still captured in the PromptResult.

Returns

Promise<PromptResult> - The result object with response and metadata.

Example

import { hasToolCall } from "@mcpjam/sdk";

// Simple prompt
const result = await agent.prompt("Add 2 and 3");

// Multi-turn conversation
const r1 = await agent.prompt("Create a task called 'Test'");
const r2 = await agent.prompt("Mark it complete", { context: r1 });

// Multiple context items
const r3 = await agent.prompt("Show summary", { context: [r1, r2] });

// Stop the loop after the step where a tool is called
const r4 = await agent.prompt("Search for tasks", {
  stopWhen: hasToolCall("search_tasks"),
});
console.log(r4.hasToolCall("search_tasks"));

// Bound prompt runtime
const r5 = await agent.prompt("Run a long workflow", {
  timeout: { totalMs: 10_000, stepMs: 2_500 },
});

if (r5.hasError()) {
  console.error(r5.getError());
}

// Exit early after selecting a tool, without waiting for the real MCP call
const r6 = await agent.prompt("Search for tasks", {
  stopAfterToolCall: "search_tasks",
  timeoutMs: 5_000,
});
console.log(r6.getToolArguments("search_tasks"));

prompt() never throws exceptions. Errors are captured in the PromptResult. Check result.hasError() to detect failures.

stopAfterToolCall is intended for evals that only care about tool selection and arguments. If the model emits multiple tool calls in the same step, non-target sibling tools may still execute before the loop stops.

timeout bounds prompt runtime. The runtime creates an internal abort signal, so tools can stop early if their implementation respects the provided abortSignal. If a tool ignores that signal, its underlying work may continue briefly after the prompt returns an error result.

---

Model String Format

Models are specified as provider/model:

// Anthropic
"anthropic/claude-sonnet-4-20250514";
"anthropic/claude-3-haiku-20240307";

// OpenAI
"openai/gpt-4o";
"openai/gpt-4o-mini";

// Google
"google/gemini-1.5-pro";
"google/gemini-1.5-flash";

// Azure
"azure/gpt-4o";

// Mistral
"mistral/mistral-large-latest";

// DeepSeek
"deepseek/deepseek-chat";

// Ollama (local)
"ollama/llama3";

// OpenRouter
"openrouter/anthropic/claude-3-opus";

// xAI
"xai/grok-beta";

---

Custom Providers

Add custom OpenAI or Anthropic-compatible endpoints.

CustomProvider Type

Property	Type	Required	Description
name	string	Yes	Provider identifier
protocol	"openai-compatible" | "anthropic-compatible"	Yes	API protocol
baseUrl	string	Yes	API endpoint URL
modelIds	string[]	Yes	Available model IDs
useChatCompletions	boolean	No	Use /chat/completions endpoint
apiKeyEnvVar	string	No	Custom env var for API key

Example

const agent = new TestAgent({
  tools,
  model: "my-litellm/gpt-4",
  apiKey: process.env.LITELLM_API_KEY,
  customProviders: {
    "my-litellm": {
      name: "my-litellm",
      protocol: "openai-compatible",
      baseUrl: "http://localhost:8000",
      modelIds: ["gpt-4", "gpt-3.5-turbo", "claude-3-sonnet"],
      useChatCompletions: true,
    },
  },
});

---

Configuration Properties

tools

The MCP tools available to the agent. Obtained from MCPClientManager.getTools().

const tools = await manager.getTools();
const agent = new TestAgent({ tools, ... });

model

The LLM model identifier. Format: provider/model-id.

model: "anthropic/claude-sonnet-4-20250514";

apiKey

The API key for the LLM provider.

apiKey: process.env.ANTHROPIC_API_KEY;

systemPrompt

Optional system prompt to guide the LLM’s behavior.

systemPrompt: "You are a task management assistant. Be concise.";

temperature

Controls response randomness. Range: 0.0 (deterministic) to 1.0 (creative).

temperature: 0.1; // More deterministic, better for testing
temperature: 0.9; // More creative

maxSteps

Maximum iterations of the agentic loop (prompt → tool → result → continue).

maxSteps: 5; // Stop after 5 tool calls

Setting maxSteps too low may prevent complex tasks from completing. Setting it too high may allow runaway loops.

---

Control Multi-Step Loops with stopWhen

Use stopWhen to control whether the agent starts another step after the current step completes.

import { hasToolCall } from "@mcpjam/sdk";

// Stop after the step where "search_tasks" is called
const result = await agent.prompt("Find my open tasks", {
  stopWhen: hasToolCall("search_tasks"),
});

expect(result.hasToolCall("search_tasks")).toBe(true);

// Stop after any of multiple conditions
const result2 = await agent.prompt("Do something", {
  stopWhen: [hasToolCall("tool_a"), hasToolCall("tool_b")],
});

stopWhen does not skip tool execution. It controls whether the prompt loop continues after the current step completes, and TestAgent also applies stepCountIs(maxSteps) as a safety guard.

---

Bound Prompt Runtime with timeout

Use timeout when you want to bound how long TestAgent.prompt() can run:

const result = await agent.prompt("Run a long workflow", {
  timeout: 10_000,
});

const result2 = await agent.prompt("Run a long workflow", {
  timeout: { totalMs: 10_000, stepMs: 2_500, chunkMs: 1_000 },
});

chunkMs is accepted for parity, but it is mainly useful for streaming APIs. For TestAgent.prompt(), number, totalMs, and stepMs are the main settings to focus on.

---

Complete Example

import { MCPClientManager, TestAgent } from "@mcpjam/sdk";

async function main() {
  // Setup
  const manager = new MCPClientManager({
    everything: {
      command: "npx",
      args: ["-y", "@modelcontextprotocol/server-everything"],
    },
  });
  await manager.connectToServer("everything");

  // Create agent (pass manager when you need widget snapshots or MCPJam replay configs)
  const agent = new TestAgent({
    tools: await manager.getTools(),
    model: "anthropic/claude-sonnet-4-20250514",
    apiKey: process.env.ANTHROPIC_API_KEY,
    systemPrompt: "You are a helpful assistant.",
    temperature: 0.3,
    maxSteps: 10,
    mcpClientManager: manager,
  });

  // Single prompt
  const r1 = await agent.prompt("What is 15 + 27?");
  console.log(r1.getText());
  console.log("Tools:", r1.toolsCalled());

  // Multi-turn
  const r2 = await agent.prompt("Now multiply that by 2", { context: r1 });
  console.log(r2.getText());

  // Error handling
  if (r2.hasError()) {
    console.error("Error:", r2.getError());
  }

  // Cleanup
  await manager.disconnectServer("everything");
}

---

Related

• Testing with LLMs - Conceptual guide
• PromptResult Reference - Result object API
• LLM Providers Reference - All providers