Writing
Two kinds of writing: topics taken apart from first principles, and build logs from real projects. Newest first — filter by what you're after.
Two kinds of writing: topics taken apart from first principles, and build logs from real projects. Newest first — filter by what you're after.
A first-principles guide to converting risk, regulatory, and ethical requirements into controls, evidence, ownership, and operational decisions
Interpretability is often described as “looking inside” a model. That phrase is dangerously vague.
How production failures become reliable training and evaluation data
A first-principles guide to data, memory, parallelism, networking, recovery, evaluation, and release
A first-principles guide to reasoning systems, verification, and adaptive test-time compute
A text-only technical-support assistant begins with a simple pipeline
A first-principles guide to inference, memory, batching, caching, quantization, parallelism, scaling, reliability, and cost
A production language model fails in many ways, but only a small subset of those failures justify changing its weights.
An AI system becomes difficult to control long before it becomes an “agent.” The trouble begins as soon as a useful request needs more than one model call, database query, external API call, or human decision.
An ordinary web application accepts untrusted input, but its code decides what that input means. An LLM application adds a probabilistic interpreter between input and action. That interpreter can confuse data with instructions, invent facts, choose the wrong tool, reproduce private context, and behave differently when wording changes.
An LLM application can look excellent and still be unreliable.
A systematic mental model for embeddings, sparse and dense search, chunking, hybrid retrieval, reranking, provenance, evaluation, and failure analysis.
An LLM application rarely fails because the model received no information. More often, it fails because the application supplied the wrong information, buried the right information, removed an important qualification, or gave the model several incompatible versions of the truth.
Prompt engineering is often presented as a collection of phrases: “think step by step,” “act as an expert,” or “be concise.” That framing is misleading. A language model is a probabilistic program whose behaviour is influenced—not fully controlled—by its input. A prompt is therefore an **input contract**: it describes the task, supplies the information needed to perform it, constrains acceptable behaviour, and defines the shape of the result.
A raw LLM call looks almost like an ordinary API call
A transformer can look like a pile of matrices, arrows, and unfamiliar names. It is easier to understand if we do not start with the finished diagram.
Traditional software begins with rules written by a programmer. Machine learning begins when those rules become too numerous, ambiguous, or unstable to write manually.
How to Know Whether an LLM Improvement Is Real
An ordinary HTTP handler is expected to do something small: validate a request, read or write a database, and return a response. If the process dies, the client can often try again.
An AI application may look correct while its model-provider client is quietly accumulating failure modes: invalid payloads reach the network, every request opens a new connection, a slow stream occupies a worker forever, cancellation is swallowed, and retries multiply cost.
An agent may run for minutes or hours. A worker can crash after sending an email but before recording success. A model provider can return `429`, time out after accepting a request, or change behavior behind the same model name. Two workers can receive the same job. A deployment can replace a pod halfway through a run. One tenant can consume the provider quota needed by everyone else.
Multi-agent systems are appealing for the same reason human teams are appealing: divide the work, assign specialists, run tasks in parallel and review one another.
For a small task such as “find the version in `package.json` and report it,” this is enough. The agent reads one file, extracts one value and stops. Adding a planning system would only create more latency, cost and failure modes.
The implementation examples use the stable Python SDK v1 line. Pin the SDK because its v2 API is still pre-release at the time of writing.
Agent frameworks do not make language models more intelligent. They organize the software around the model.
An AI agent can interpret ambiguous requests, inspect information and propose useful actions. It can also misunderstand a customer, follow a malicious instruction hidden in an order note, select the wrong account or repeat an expensive operation.
How to define success, inspect trajectories, build trustworthy graders, run controlled experiments, and turn failures into architecture changes.
“Agent” has become the default label for almost every application that calls an LLM, uses tools, or performs multiple steps.
Information may exist in state, memory or storage without belonging in the current model context.
An agent is often described as “an LLM that can use tools.” That description is incomplete.
Traditional APIs are designed for deterministic callers. A developer reads the documentation, chooses an endpoint, constructs a valid request, handles the response and writes application logic around known failure modes.
Calling an LLM is easy. Building a dependable application around one is not.
An LLM is a function that predicts the next token — everything it appears to "know" or "do" is a downstream consequence of that one objective.
Why a RAG system is never 'done', what actually goes stale, and what it really means to evaluate one.
The retrieval stack — query transformation → retrieval mode → reranking — exists because the user's raw query is the weakest possible proxy for what they actually need, and a single fast similarity score is too crude to fix on its own.
Retrieval works fast and cheap because real embeddings don't fill the high-dimensional space they live in — they sit on a thin, structured surface, and every optimization below is a way of exploiting that.
To retrieve text by *meaning* instead of *keywords*, we cut documents into chunks and turn each chunk into a vector whose position was deliberately trained so that "close in space" means "close in meaning.
One line: Build retrieval as hybrid (dense + lexical) → fuse → rerank → generate, prove every choice on a 50–200 pair labeled set, and treat the latency budget as the constraint that vetoes half the "advanced" tricks.
One-line summary: RAG doesn't make a language model know more — it slips the right text into the prompt at query time so the model can answer from a source instead of from memory.