Building AI features without LangChain — the raw-SDK path most teams should take first
You don't need a framework to ship an AI feature. Here's what the raw provider SDK actually gives you, the small amount of glue you write yourself, and when a framework finally earns its place.
- AI Engineering
- Architecture
- Developer Experience
When a founder asks us to "add AI" to their product, the request underneath is almost always modest: summarise this, classify that, answer questions over these docs, call a couple of internal tools on the user's behalf. The scope is a feature, not a platform. And yet the first instinct — theirs and, honestly, a lot of engineers' — is to reach for a framework before writing a single line against the model.
We think that order is backwards. Most AI features are a prompt, a tool loop, some retries, and good logging. All four of those fit in roughly a hundred lines against the raw provider SDK — openai, @anthropic-ai/sdk, google-genai — and the loop you'd be handing to a framework is the exact part of the system you most need to see and control. This isn't anti-framework dogma. Frameworks earn their keep; we'll get to when. But for the first version of most features, the raw SDK is the faster, clearer, more debuggable path, and starting there teaches you what you'd actually be delegating later.
What a raw agent loop actually is
Strip away the vocabulary and an agent is a while loop over a growing list of messages. That's it. The shape is the same across every provider:
- You hold a message list — a system prompt, the user's input, and everything that happens after.
- You call the model with that list plus a set of tool definitions (name, description, JSON schema for arguments).
- The model replies with either a final answer or one or more tool calls.
- If it asked for tools, you execute them, append the results to the message list as tool-result messages, and go back to step 2.
- When the model stops calling tools and returns prose, you're done.
In TypeScript that's a function with a loop, a switch over tool names, and an array you push onto. There is no hidden state, no callback graph, no lifecycle to learn. When something misbehaves, you log the message list and read it top to bottom — the entire reasoning trace is right there, in a data structure you own. Compare that to debugging a stuck agent inside a framework, where "what did the model actually see?" is a question you answer by reading someone else's source.
That legibility is the whole argument. The loop is the most important code path in an AI feature, and it's small enough to keep in your head.
The glue you write yourself
The loop alone is a demo. Production is the glue around it — and this is the part people assume a framework saves them from. It mostly doesn't; it just hides it.
Retries and backoff. Model APIs return 429s and 529s under load, and occasionally just fail. You want exponential backoff with jitter on the transient ones, and a hard cap so a bad call can't retry forever. This is a small wrapper around your client.messages.create call — ten or fifteen lines, and you get to decide exactly which errors are retryable.
Timeouts and a loop budget. Two different limits. A per-request timeout so a single model call can't hang your handler, and a maximum number of loop iterations so a model that keeps calling tools can't spin indefinitely. Frameworks that hide the loop also make this cap harder to reason about; when you own the while, it's if (turns++ > MAX) break.
Structured-output validation. When you need JSON back — a classification, an extraction, fields to write to a database — don't trust the model's word that it matched your schema. Parse it, validate against Zod or your schema library, and on failure feed the validation error back to the model as another turn. Providers now support strict/JSON-schema-constrained output, which dramatically cuts failures, but validation at your boundary is still the thing that keeps bad data out of your system.
Observability. Log every turn: the messages in, the tool calls out, token counts, latency, which model, and the final stop reason. This is not optional and it's not something to bolt on later. When a user reports "the AI did something weird," the trace is the difference between a five-minute fix and a shrug. You can push these to whatever you already use — a table, structured logs, an OpenTelemetry span per turn.
None of this is exotic. Added up, it's the small stack that turns the loop into a feature:
You write this once, in a shared internal module, and every AI feature in the codebase reuses it. That module is your framework — one you can read in an afternoon, that carries no version churn but your own, and that never surprises you at 2am.
What you don't get for free — and whether you need it
Being honest about the trade means naming what the raw SDK genuinely doesn't hand you.
Prompt and chat abstractions. Templating, message formatting helpers, few-shot example management. Real, but shallow — a few small functions of your own, or a light library used as a toolbox, cover it.
Retrievers and document loaders. If you're doing RAG, framework retriever/splitter/parser components are legitimately useful and worth pulling in as libraries. Using a retriever from LangChain doesn't mean handing LangChain your agent loop — those are separable decisions, and keeping them separate is the point.
Persistence and checkpointing. Pausing an agent mid-run and resuming hours or days later, surviving a process restart. The raw loop is in-memory and request-scoped; if you need durable, resumable state, you either build it deliberately or adopt something that already has it. Most features don't need this — but when you do, it's a real gap, not a nice-to-have.
Graph orchestration. Conditional routing between named steps, fan-out to parallel branches and join, human-in-the-loop approvals. A while loop with a switch models a single loop beautifully and a branching state machine badly. The moment you're writing your own conditional router between steps, the raw SDK is telling you you've outgrown it.
The pattern in that list: the things you don't get for free are exactly the things simple features don't need. When your feature does need them, that's the signal — not the starting assumption.
When a framework earns its keep
Frameworks are not the enemy; the wrong default is. A framework earns its place when the shape of the problem genuinely changes.
Reach for a framework — LangGraph is the one we'd actually reach for — when you have:
- A real graph, not a loop. Named steps with conditional routing, parallel branches that fan out and join. If you can draw it as a straight line with one loop, you don't have a graph. If the whiteboard has branches and merges, you do.
- Durable, resumable state. Human-in-the-loop pauses, workflows that span hours or days, runs that must survive a restart. Checkpointing is hard to get right and a framework that already solved it is worth the dependency.
- Many teams sharing primitives. When five teams are each building agents, a shared framework with common state handling, tracing, and eval hooks stops everyone reinventing the same glue five slightly different ways. The consistency is worth more than the last ounce of control.
The honest test: adopt the framework when not having it means you're rebuilding what it offers. Adopting it earlier — "we'll need it eventually" — buys you a dependency, an abstraction layer, and a version-upgrade treadmill before you've earned any of its benefits.
Mistakes we've watched teams make
Reaching for the framework before the graph exists. The most common one. Pick the tool for the shape you have, not the shape you imagine. Draw the agent on paper first — most first versions are a straight line.
Skipping observability because the framework "handles it." Whatever you build on, you need to be able to answer "what did the model see and do on this exact request." If you can't, you're debugging blind. Own your traces regardless of the stack underneath.
Trusting model output structure without validating. JSON that parses is not JSON that's correct. Validate at the boundary and feed failures back as a turn — every time, not just when it's convenient.
Letting a framework leak into your domain. Whether raw or framework, wrap it behind a thin adapter. Your business logic should not import the AI library directly. The day you swap models, providers, or the framework itself — and you will — a clean seam turns a rewrite into a config change.
Treating retries and timeouts as an afterthought. These are not polish; they're the difference between a feature that degrades gracefully under load and one that falls over. Build them into the shared module from day one.
TL;DR
Most AI features are a prompt plus a tool loop plus retries plus logging — and against the raw provider SDK that's about a hundred lines you can read in one sitting. The loop is a while over a message list: call the model, run any tools it asks for, append the results, repeat until it stops. The glue around it — backoff, timeouts, structured-output validation, per-turn traces — is small, it's yours, and it becomes a shared internal module that every feature reuses.
You don't get persistence, graph orchestration, or a large component library for free — but simple features don't need them, and pulling in a retriever as a library isn't the same as handing over your loop. Reach for a framework when the shape actually changes: a branching stateful graph, durable resumable runs, or many teams sharing primitives. Until then, start raw. You'll ship faster, debug quicker, and understand exactly what you'd be delegating when the day to delegate arrives.
If you're weighing framework vs. raw SDK for an AI feature, let's talk.