How do you unit test and mock tools in isolation without invoking the LLM?

0-2 years experience

1. 1Suppose you have a LangChain chain that calls an OpenAI LLM via the OpenAIChat model. How would you write a unit test that verifies the chain's logic without actually sending a request to the OpenAI API?
2. 2If you need to mock a custom tool that fetches data from a REST endpoint inside a LangChain tool, what steps would you take to replace the real HTTP call in your test?
3. 3What happens if you forget to reset the mock between test cases? How would that affect your test suite?

2-5 years experience

1. 1You added a new LangChain tool that wraps a third‑party sentiment analysis service. During integration testing the chain hangs. Walk me through how you would debug the mock setup and why the real service might still be getting called.
2. 2Explain the trade‑offs between using LangChain’s built‑in FakeLLM versus a library like unittest.mock to isolate LLM calls in a feature that streams partial responses.
3. 3Your CI pipeline started failing because the mock for the LLM returns a different shape than the real API. How would you adjust your tests to catch such schema mismatches early?

5-8 years experience

1. 1Design a testing strategy for a production LangChain application that composes multiple tools, some of which have side effects such as writing to a database. How would you ensure each component is unit‑tested in isolation while still being able to run end‑to‑end integration tests efficiently?
2. 2At scale, mocking LLM calls can hide latency and token‑usage bugs. How would you augment your unit tests to also capture performance characteristics without invoking the real model?
3. 3Your team wants to migrate from OpenAI’s GPT‑3.5 to a self‑hosted LLM. What changes would you make to the existing mock infrastructure to support both providers during the transition?

8+ years experience

1. 1Across several product teams you notice inconsistent mocking approaches for LangChain tools, leading to flaky tests. Propose an organization‑wide framework or library that standardizes mocking LLMs and external tools, and discuss how you’d handle versioning and backward compatibility.
2. 2When legacy LangChain pipelines are being refactored, how would you evaluate the risk of removing existing mocks and replacing them with contract tests? What metrics would you track to ensure reliability during the migration?
3. 3Consider a multi‑region deployment where some LangChain tools call region‑specific services. How would you design a mock layer that can simulate regional failures and latency variations for unit and chaos testing?