How do you validate and sanitize tool output before it is passed back to the LLM?

Question

Accepted Answer

LangChain provides built-in middleware and custom hooks to intercept and sanitize tool results before they are sent back to the LLM. This is the most structured approach for compliance and safety tasks. import { createAgent, piiRedactionMiddleware } from "langchain";

const ToolOutputSchema = z.object({
  data: z.array(z.any()),
  count: z.number().int(),
  status: z.enum(["success", "partial", "error"]),
});

parser = PydanticOutputParser(pydantic_object=OutputSchema)
retry_parser = RetryOutputParser.from_llm(parser=parser, llm=ChatOpenAI())

def process_tool_output(raw_output, prompt_value):
    try:
        return parser.parse(raw_output)
    except OutputParserException:
        # Use the LLM to fix the output
        return retry_parser.parse_with_prompt(raw_output, prompt_value) Tools can return excessively long outputs that exceed the LLM's context window. To prevent crashes and reduce costs, you should either truncate the output or implement summarizing middleware that compresses the tool result before passing it back. import { ContextEditingMiddleware, ClearToolUsesEdit } from "langchain";

const agent = createAgent({
  model: "openai:gpt-4o",
  tools: [searchTool],
  middleware: [
    ContextEditingMiddleware({
      edits: [new ClearToolUsesEdit()],
    }),
  ],
}); Tool outputs must be sanitized to prevent injection attacks. The official LangChain documentation warns against the use of `eval()` on untrusted output, which can lead to Remote Code Execution (RCE) vulnerabilities. Always use safe parsers and validate the structure of the output instead of directly executing it. // ❌ INSECURE: Using eval() on tool output
const calculate = tool(async ({ expression }) => String(eval(expression)), {
  name: "calculate",
  schema: z.object({ expression: z.string() }),
});

// ✅ SECURE: Using math.js or validating via regex
import { evaluate } from 'mathjs';

class HighRiskItem(BaseModel):
    key: str = Field(description="A key in the JSON object")
    value: str = Field(description="A value in the JSON object")

class PentestResultInput(BaseModel):
    pentest_result: List[HighRiskItem] = Field(description="A list of JSON objects describing high risk items")

@tool("ZAP_pentest_summary", args_schema=Pentest

How do you validate and sanitize tool output before it is passed back to the LLM?

Validate and sanitize tool output by applying deterministic sanitization (e.g., PII redaction), structured output validation using schemas and retry mechanisms, and safety filtering through middleware to prevent harmful or malformed responses from reaching the LLM.

How do you validate and sanitize tool output before it is passed back to the LLM?

Validate and sanitize tool output by applying deterministic sanitization (e.g., PII redaction), structured output validation using schemas and retry mechanisms, and safety filtering through middleware to prevent harmful or malformed responses from reaching the LLM.

Scenario Questions

Follow-up Questions