“Agent” has become the default label for almost every application that calls an LLM, uses tools, or performs multiple steps.

But most useful AI applications do not need an autonomous agent. They need a reliable software workflow containing a few carefully chosen model decisions.

The central design question is not:

Where can we use an LLM?

It is:

Which decisions cannot be specified reliably in advance?

Every probabilistic decision introduces variability, cost, latency, and another failure mode. Model autonomy should therefore be earned by the problem—not added because it makes the architecture sound more advanced.

Begin with the next action

Suppose a customer asks:

Where is order ORD-4821?

The required steps are already known:

Authenticate the customer.
Validate the order ID.
Fetch the order.
Verify that it belongs to the customer.
Read its shipment status.
Format a response.

A model may help phrase the final response naturally, but it does not need to decide which database to query, whether authentication matters, or what action should happen next.

Now consider another request:

My headphones stopped working after I used them in light rain. The product page implied they were water-resistant, but support previously told me something different. Can I get a refund?

The system may need to:

Interpret the complaint.
Identify relevant claims.
Inspect the product description.
Retrieve the conversation history.
Compare evidence from several sources.
Determine what information is missing.
Decide which investigation step to perform next.

The investigation path is not fully known in advance. This is where limited agentic behavior may be useful.

That gives us the first principle:

If the next action is known, encode it in software. If the next action depends on information discovered during execution, consider a model decision.

Even then, the model should receive only the freedom required to resolve that uncertainty.

Deterministic code versus model decisions

A deterministic function follows rules defined by the program:

same input + same state → same transition

Examples include:

Validating an order ID
Checking whether an account exists
Comparing a refund amount with a limit
Looking up shipment status
Verifying whether an order belongs to a customer
Enforcing a return window
Requiring approval above ₹5,000

A model decision interprets information whose meaning cannot be captured economically by simple rules:

input + model + context → probabilistic output

Examples include:

Extracting a complaint from an unstructured message
Comparing conflicting statements
Identifying missing evidence
Summarizing a long support history
Deciding which investigation question would reduce uncertainty

This is not a competition between “old software” and “AI.” They solve different kinds of problems.

Use deterministic code when the rule is known. Use a model when the system must interpret ambiguity. Use an agent only when interpretation must influence which action happens next.

A useful test is:

Could an engineer write the correct transition as an if, lookup, function call, or state-machine rule?

If yes, a model usually should not own that transition.

A workflow is not an agent

A workflow executes a predefined graph.

A → B → C

or

A → if condition X, run B; otherwise run C

The nodes and possible transitions are known before execution begins. Individual nodes may still call an LLM.

An agent receives a goal and repeatedly chooses what to do:

observe → choose action → execute → observe → repeat

The exact path is discovered during execution.

Consider a system that performs:

Retrieve an FAQ.
Ask an LLM to write an answer using that FAQ.
Check whether the answer cites the retrieved policy.

This is an AI workflow, not necessarily an agent. The steps are fixed.

A system becomes agentic when the model can decide things such as:

Which tool to call
Which evidence to retrieve
Whether another action is needed
Whether to revise its plan
When the task is complete

Autonomy is therefore not determined by the number of model calls. It is determined by who controls the transitions.

Common workflow patterns

1. Prompt chaining

Prompt chaining divides a task into a fixed sequence of model operations.

For example:

extract complaint → retrieve policy → draft response → verify response

This is useful when each stage produces an output that can be validated before entering the next stage.

Why not ask one model to do everything?

Because a single prompt mixes several responsibilities. If the final answer is wrong, we may not know whether the system misunderstood the complaint, retrieved the wrong policy, or ignored the policy.

Prompt chaining improves observability and testability, but every additional model call adds cost and latency. A chain is justified only when separating stages gives us useful validation boundaries.

It is still a workflow because the sequence is known in advance.

2. Routing

Routing selects one path from a predefined set:

FAQ → retrieval flow
order status → order-status flow
account issue → account flow
refund request → refund flow
unknown → human triage

The router may be deterministic or model-based.

A deterministic router can use:

A category explicitly selected in the interface
The endpoint that received the request
Structured fields
Exact identifiers such as order numbers
Versioned keyword and priority rules

A model-based router is justified when users submit highly variable natural language that cannot be reliably constrained or classified using known rules.

Even then, the model should return a small validated schema:

{
  "route": "refund",
  "confidence": 0.87
}

The model chooses among allowed branches. It does not invent a new workflow.

3. Parallelization

Independent tasks can run simultaneously.

A refund investigation might fetch:

Order details
Payment information
Shipment history
Previous support conversations

If none depends on the output of another, running them sequentially adds unnecessary latency.

Parallelization does not create autonomy. The workflow still decides which tasks run; it merely runs known tasks concurrently.

The final latency is approximately the slowest parallel branch rather than the sum of all branch latencies.

4. Map-reduce

Map-reduce applies the same operation to many items and then combines the results.

For example, when analyzing 200 support messages:

Split the messages into chunks.
Summarize each chunk independently.
Combine the summaries.
Produce the final case history.

The map stage processes bounded units. The reduce stage combines them.

This is useful when the input exceeds a model’s practical context budget or when processing can be parallelized. The decomposition is deterministic even if the map and reduce operations use models.

The main risk is information loss: a fact omitted during mapping cannot be recovered by the reducer. Important claims should therefore retain references to their source messages.

5. Orchestrator-worker

An orchestrator inspects a task, creates subtasks, delegates them to workers, and combines their results.

For example, a research system may decide that a complaint requires:

Product-policy research
Warranty-history analysis
Conversation analysis

This pattern is appropriate when the required subtasks cannot be identified reliably before inspecting the problem.

It is more agentic than ordinary parallelization because the orchestrator decides what work needs to exist.

That flexibility has a cost:

It may generate unnecessary subtasks.
It may miss an important subtask.
Workers may duplicate effort.
Cost and latency become harder to predict.
Testing every possible plan becomes difficult.

Do not use an orchestrator-worker pattern when the workers are always the same. In that case, ordinary parallelization is simpler and more reliable.

6. Evaluator-optimizer

An evaluator-optimizer loop generates an output, evaluates it against explicit criteria, and revises it:

draft → evaluate → revise → evaluate

For example, a support reply may be checked for:

Policy consistency
Unsupported promises
Missing citations
Required disclosures
Appropriate tone

This loop is useful when quality criteria are clear but producing a compliant answer in one attempt is difficult.

It should have:

A fixed rubric
Structured evaluator output
A maximum number of revisions
A definition of acceptable quality
A fallback when the threshold is never reached

This is narrower than a general agent loop. The model is optimizing one artifact, not choosing arbitrary actions across the system.

7. Conditional branching

Conditional branching uses explicit rules:

if order_not_found:
    request_correct_order_id
elif order_owner != customer:
    reject_access
elif order_status == "delivered":
    show_delivery_details
else:
    show_tracking_status

When the condition is based on known business data, code should own it.

Asking a model whether one timestamp is inside a 30-day return window would add uncertainty to a problem already solved by arithmetic.

8. State machines

A state machine defines valid states and transitions.

A refund case might move through:

RECEIVED
→ INVESTIGATING
→ POLICY_VALIDATED
→ AWAITING_APPROVAL
→ APPROVED
→ EXECUTED

It may also move to:

NEEDS_INFORMATION
REJECTED
ESCALATED
FAILED

The state machine prevents illegal transitions. For example:

A refund cannot be executed before policy validation.
Approval cannot be skipped.
A rejected case cannot silently become executed.
Retrying payment execution must not create a second refund.

The model may recommend a transition, but application code validates whether that transition is permitted.

9. Agentic branches inside deterministic workflows

A system does not need to be entirely deterministic or entirely agentic.

The safest useful architecture is often:

A deterministic workflow containing one bounded agentic branch.

For example:

Deterministic classification
Deterministic authentication
Deterministic account and order retrieval
Agentic refund investigation
Deterministic policy validation
Mandatory human approval
Deterministic refund execution

The agent receives a restricted set of read-only tools. It may gather evidence and recommend an outcome, but it cannot approve or execute a refund.

This preserves flexibility where the investigation is genuinely uncertain while keeping business-critical transitions under explicit control.

Fallbacks, escalation and human review

Failures are part of the workflow, not exceptions to architecture.

Fallbacks

A fallback handles an expected failure using a less capable but safer path.

Examples:

If semantic FAQ retrieval fails, try keyword search.
If answer generation fails, show the retrieved FAQ excerpts.
If an account service times out, create a retryable case instead of claiming the account does not exist.
If structured model output is invalid, retry once with the validation error.
If the second attempt fails, escalate.

Fallbacks should preserve truth. A degraded answer is acceptable; a fabricated answer is not.

Escalation

Escalation transfers the case to a more capable authority when the automated system cannot proceed safely.

Useful escalation triggers include:

Conflicting evidence
Missing required information
Low-confidence classification
Repeated tool failures
Suspected fraud
Policy exceptions
Customer threats or legal claims
Investigation-budget exhaustion

“Ask the model again” is not an escalation strategy. Repeated calls to the same model with the same evidence often reproduce the same uncertainty.

Human review

Human review is appropriate when a decision is:

Financially consequential
Irreversible
Legally sensitive
Based on conflicting evidence
Outside normal policy
Difficult to evaluate automatically

The reviewer should receive:

The proposed action
Relevant evidence
Policy-validation results
Model confidence and uncertainties
The complete trace
Clear approve, reject, and request-information actions

Humans should review decisions, not reconstruct hidden model reasoning from scratch.

Case study: a customer-support system

Let us apply these principles to a support platform.

Request classification

The system first tries to classify requests deterministically.

The interface may allow customers to select a category, while the backend validates the selection against structured evidence:

An order ID suggests an order-related flow.
“Cancel order” maps to cancellation.
“Where is” plus an order ID maps to order status.
A refund form submission maps to refund.
Known FAQ topics map to FAQ retrieval.
Ambiguous requests go to general triage.

Why deterministic?

Because the available business flows are known. Classification controls which services and permissions become available, so reproducibility matters.

A model router may later be added for requests that repeatedly fall into ambiguous triage, but only after evaluation shows that deterministic classification is insufficient.

FAQ retrieval

FAQ retrieval is a deterministic workflow:

Normalize the query.
Search approved FAQ documents.
Filter by product, region, and policy version.
Return the best matching passages.
Generate an answer constrained to those passages.
Validate that important claims cite retrieved evidence.

The model may phrase the answer, but it cannot create policy.

If retrieval confidence is too low, the system asks the customer to clarify or escalates instead of answering from general model knowledge.

Account lookup

Account lookup is ordinary software:

Authenticate the customer.
Read the customer ID from the trusted session.
Query the account service.
Remove fields the support flow does not need.
Return a typed result.

There is no interpretive problem here. Giving an agent arbitrary account-search access would increase privacy and security risks without improving the result.

Order-status flow

Order status is also deterministic:

Extract and validate the order ID.
Confirm ownership.
Fetch order and shipment data.
Map provider-specific statuses to internal statuses.
Select an approved response template.
Optionally ask a model to rewrite it naturally without changing facts.

The model must not decide whether ownership verification can be skipped.

Refund-investigation agent

Refund investigation is the bounded agentic branch.

Its goal is:

Gather sufficient evidence to recommend whether this refund request should proceed to policy validation.

Its allowed tools are read-only:

get_order
get_payment
get_delivery_events
get_product_policy
get_support_history
request_customer_information

The agent may decide which evidence to inspect and whether another investigation step is necessary.

It cannot:

Modify the order
Approve a refund
Execute a refund
Change policy
Contact external parties
Continue beyond its step or cost budget

It must return structured output:

{
  "recommendation": "proceed_to_policy_validation",
  "evidence": [
    {
      "claim": "The item was reported damaged on delivery",
      "source": "support_message_183"
    }
  ],
  "missing_information": [],
  "uncertainties": [
    "The shipping photograph is inconclusive"
  ]
}

Why is this branch agentic?

Because the next useful investigation step depends on evidence discovered during previous steps. Different refund cases may require different information.

Why is it bounded?

Because investigation requires interpretation, but policy enforcement and financial execution do not.

Policy validation

Application code validates the recommendation against versioned policy rules:

purchase age <= return window
AND product category is refundable
AND refund amount <= eligible amount
AND no completed refund exists
AND required evidence is present

The agent may explain evidence, but code evaluates computable rules.

If policy contains vague language such as “reasonable evidence of damage,” a model may produce a narrow evidence assessment. The final policy result should still record that a probabilistic judgment was involved.

Human escalation and approval

A case is escalated when:

Policy requires an exception
Evidence conflicts
The amount exceeds the automatic threshold
Fraud signals are present
The agent reports unresolved uncertainty
Required services repeatedly fail

Even when the case passes policy validation, a human must approve the refund before execution.

Approval records:

Reviewer identity
Evidence reviewed
Policy version
Approved amount
Timestamp
Optional explanation

Refund execution

Execution is deterministic and idempotent:

if approval is valid
and case is not already refunded
and approved amount matches request:
    execute refund using idempotency key

The payment result is persisted before the customer receives a success message.

An agent should never decide whether a payment API timeout means “try again with a new transaction.” That can produce duplicate refunds. Retry behavior belongs in explicitly tested payment code.

Complete architecture

Customer request
    ↓
Deterministic classification
    ├── FAQ → retrieval → grounded response
    ├── Account → authenticated account lookup
    ├── Order status → ownership check → shipment lookup
    ├── Refund → bounded investigation agent
    │               ↓
    │         deterministic policy validation
    │               ↓
    │         human approval
    │               ↓
    │         idempotent refund execution
    └── Unknown → clarification or human triage

The important boundary is not between “AI components” and “non-AI components.” It is between decisions that require interpretation and decisions whose correct rules are already known.

Observability

A normal request log is not enough for a system containing probabilistic decisions.

Each case should produce a trace containing:

Request and case ID
Current workflow state
Classification result and rule
Every state transition
Retrieved document identifiers and versions
Model and prompt versions
Tool calls and sanitized results
Structured model outputs
Validation failures
Retry and fallback decisions
Token usage
Cost
Step latency
Human decisions
Final business outcome

Do not rely on hidden reasoning text as the trace. Record observable inputs, actions, evidence, outputs, and transitions.

A useful trace should answer:

What happened?
Why was this branch selected?
Which evidence influenced the result?
Which component failed?
Was the final action authorized?
Can the case be reproduced?

Evaluating reliability, cost and latency

A workflow should be evaluated as a system, not only as individual prompts.

Reliability

If three required steps independently succeed 99%, 98%, and 97% of the time, the approximate end-to-end success rate is:

0.99 × 0.98 × 0.97 ≈ 94.1%

The independence assumption is imperfect, but the calculation exposes an important fact: adding steps can reduce total reliability even when every step looks individually strong.

Measure:

Correct routing
Retrieval recall
Grounded-answer accuracy
Policy-validation correctness
Invalid transition rate
Escalation precision and recall
Duplicate-action rate
End-to-end resolution rate

Cost

For a fixed workflow:

expected cost ≈ sum of known model and tool-call costs

For an agent:

expected cost ≈ average steps × average cost per step

But the average is not enough. Also measure p95 cost and enforce a maximum budget because difficult cases may trigger many more steps.

Latency

Sequential stages add latency:

total latency ≈ L1 + L2 + L3

Parallel stages are closer to:

total latency ≈ max(L1, L2, L3) + coordination overhead

Agent loops add variable sequential latency because every decision may depend on the previous observation.

Testability

Deterministic workflows allow assertions such as:

GIVEN an authenticated customer
AND an order belonging to another account
WHEN order status is requested
THEN access is rejected

Agentic behavior requires broader evaluation:

Did it choose an appropriate tool?
Did it gather sufficient evidence?
Did it stop at the correct point?
Did it stay within its permissions?
Did it escalate when uncertain?
Did it avoid unsupported claims?

The larger the model’s action space, the larger the evaluation space.

Evaluation cases for the support system

The test suite should include at least these cases:

Case	Expected behavior
Known FAQ with strong retrieval match	Return a grounded answer with citations
FAQ with no reliable match	Ask for clarification or escalate
Valid order owned by customer	Return current shipment status
Valid order owned by another customer	Deny access without revealing order details
Malformed order ID	Request a valid identifier
Account service timeout	Create a retryable failure; do not claim account absence
Simple refund inside policy	Investigate, validate policy and request approval
Refund outside return window	Reject or escalate according to explicit policy
Conflicting product and support statements	Agent gathers both sources and escalates uncertainty
Missing damage evidence	Request information or escalate
Refund above approval threshold	Require authorized reviewer
Duplicate refund request	Detect previous execution and do not pay again
Payment timeout after submission	Check idempotent transaction status before retrying
Agent exceeds step budget	Stop and escalate with gathered evidence
Invalid model output	Retry once with schema feedback, then escalate
Prompt-injection text in customer message	Treat it as untrusted data and preserve tool restrictions

Evaluation should verify not only whether the final answer was correct, but whether the system reached it through an allowed path.

Choosing the minimum required autonomy

Before adding a model decision, ask:

Is the correct rule already known?
Can normal code express it clearly?
Is the input ambiguous enough to require semantic interpretation?
Does interpretation affect only content, or does it affect the next action?
Can the model choose from predefined branches instead of arbitrary actions?
Can its tools be read-only?
Can the result be validated before anything consequential happens?
What is the fallback when confidence is low?
What is the maximum step, time, token, and cost budget?
How will we evaluate whether this autonomy improves the system?

This produces an autonomy ladder:

deterministic function
→ fixed workflow
→ workflow with model-generated content
→ model routing among fixed branches
→ bounded evaluator loop
→ bounded agentic branch
→ general autonomous agent

Start at the bottom of complexity, not the top of autonomy. Move upward only when the previous level cannot solve the task adequately.

Final principle

Autonomy is valuable when a system must explore an environment whose required action sequence cannot be specified beforehand.

It is wasteful when it replaces rules we already understand.

A reliable AI application therefore does not ask the model to run the entire business process. It gives the model a narrow role at the points where interpretation is necessary, surrounds those decisions with deterministic software, validates the results, and reserves consequential actions for explicit policy and human authority.

The goal is not to build the most autonomous system.

The goal is to build the least autonomous system that can still solve the problem.