We build LLM systems in the LangChain ecosystem every week, so read this as field notes rather than a takedown: some of the most effective work we do is removing LangChain from parts of a stack — and some of the worst refactors we have seen removed it from places it was earning its keep. The framework question deserves the same treatment as any dependency decision: what is it buying you, at what cost, in this specific system?
What the framework is actually for
LangChain's value concentrates in three places: integrations (a uniform interface over providers, vector stores, and document loaders), orchestration for genuinely stateful workflows (LangGraph), and observability (LangSmith tracing, which — usefully — works fine without the rest of the framework). Early in a project, the integrations alone justify it: you can swap embedding models and vector stores while you are still finding out what the system should be.
The costs concentrate elsewhere: layers of abstraction between your code and the API call, a fast-moving surface area that makes upgrades non-trivial, and defaults (prompts, retries, parsing) that do things you did not decide. Whether the trade is good depends on which side of it your system actually lives on.
The signals it is costing more than it saves
Debugging happens inside the framework, not your code. When every incident investigation descends through runnable wrappers and callback layers to find the prompt that was actually sent, the abstraction is charging you at exactly the moments you can least afford it. A tracing tool mitigates this; needing the tracer to answer "what did we send?" is itself the signal.
Your pipeline is secretly simple. A large share of production LLM features are: build a prompt from a template and some retrieved text, call a model, validate the structured output, retry on failure. That is under a hundred lines against a provider SDK, fully typed, obvious to any engineer on the team. If your chain composition expresses a linear flow, the composition machinery is pure overhead.
You fight the defaults. Hidden prompt formatting, automatic retries interacting with your own retry layer, output parsers that half-repair JSON — if a meaningful fraction of your commits configure the framework out of behaviors, the framework is not saving you decisions; it is making decisions you then reverse.
Upgrades are events. If version bumps get scheduled, that is a tax with no corresponding feature.
The flexibility you are paying for is unused. The uniform interface is valuable while you are still choosing providers. Two years in, on one model family and one vector store, an abstraction over choices you already made is indirection with no remaining option value.
What to keep
Framework exit is rarely all-or-nothing, and the ecosystem's parts decouple better than people expect:
- LangSmith / tracing works standalone. Keep your observability regardless of what happens to the orchestration layer.
- LangGraph is the piece that most often survives our reviews: for durable, multi-step, human-in-the-loop workflows, its checkpointing model is real engineering you would otherwise rebuild badly. Running LangGraph for the two stateful workflows while the plain RAG endpoints use direct SDK calls is a perfectly coherent architecture — the pieces do not obligate each other.
- Loaders and integrations can stay in offline indexing code even when the request path goes direct. The request path is where indirection hurts; the batch job does not care.
How to exit without breaking things
The refactor is only safe if behavior is pinned first, and this is where an eval suite stops being hygiene and becomes the enabling tool.
- Pin current behavior. Run your eval suite against the existing system and freeze the scores. If you do not have a suite, build it before the refactor — fifty labeled production traces is enough to start, and without it you are refactoring blind.
- Capture the real prompts. From traces, extract what the framework actually sends — including the formatting and system-message behavior you inherited without deciding. Make it explicit text in your repo. Expect at least one surprise here.
- Strangler-pattern by endpoint. Stand up the direct-SDK path behind a flag, route a slice of traffic, compare evals and cost/latency numbers, expand. Endpoint by endpoint, not big-bang.
- Diff more than correctness. Token counts, retry behavior, latency distribution, error taxonomy — the framework was doing invisible work in these areas, and the refactor must own it explicitly: your own retries, your own timeout budget, your own output validation.
- Delete as you go. The half-migrated state where both stacks live forever is the worst outcome; it doubles the surface area you were trying to shrink.
The decision, stated plainly
Keep the framework where it does stateful heavy lifting you would otherwise rebuild — that is LangGraph, most of the time. Keep the tracing everywhere. Take the request path direct when it is simple enough to own, and let your evals prove the swap changed nothing but the dependency graph. Loyalty belongs to the system, not the framework — in either direction.