AI Agents and Tool Calling · 45 topics
What is an AI agent?
An AI agent is an AI system that can
Understand a goal
Decide what actions + tools are needed
Observe the results
Decide the next action
Continue until the goal is completed
A normal LLM mostly generates an answer.
An agent can take actions to achieve a goal.
Agent vs Chatbot vs Workflow
A chatbot is a conversational program that talks and answers questions. It doesn't take action.
An AI agent is an AI system that can
Understand a goal
Decide what actions + tools are needed
Observe the results
Decide the next action
Continue till the goal is completed
A workflow follows a predefined sequence of steps used to accomplish a task.
START
↓
Get resume
↓
Extract text
↓
Calculate skills
↓
Generate report
↓
ENDWorkflow | Agent | Chatbot |
Predefined path | Dynamic path | Mainly responds |
Developer controls flow | LLM can influence flow | Usually conversation-focused |
More predictable | More flexible | May not use tools |
Easier to test | Harder to test | Usually simpler flow |
Usually cheaper | Can consume more tokens | User drives each step |
Types of workflow
Sequential /Linear Chaining:
The task is divided into multiple steps.
The output of one LLM call becomes the input to the next.
Use when: A task has a clear sequence of steps.
Conditional Workflow:
A conditional workflow is a Workflow that executes specific paths or actions based on specific rules or conditions
Parallelization:
Multiple tasks execute simultaneously, and their outputs are combined.
Use when: Tasks are independent and can run concurrently.
Orchestrator – Worker:
An orchestrator breaks a complex task into subtasks, assigns them to workers and combines their results.
Use when: The subtasks cannot be fully predefined in advance.
Evaluator - Optimizer:
One LLM generates an output, while another evaluates it.
The output is refined until it meets the required quality or stopping condition.
Use when: Quality improves through iterative feedback.
Agent Loop (Tool-Calling Loop):
The LLM repeatedly decides which action to take, calls a tool, observes the result, and decides what to do next.
Use when: The next action depends on the result of the previous action.
Unlike a fixed workflow, the agent loop can dynamically choose its next step.
Human-in-the-Loop Workflow:
The workflow pauses until a person approves, rejects, or provides missing information.
What are the components of an agent?
The model (reasoning and decisions), instructions (system prompt defining role, constraints and behavior), and tools (functions, APIs, retrieval, code execution) through which it acts.
State and memory (conversation, task progress, long-term facts), and the orchestration loop that runs model calls, executes tools and manages context.
Guardrails and control: permissions, validation, budgets, human approval, logging and tracing.
Describe one turn of the agent loop.
Build the context: instructions, conversation history, retrieved memory and the list of available tools.
Call the model, which returns either a final answer or one or more tool calls.
Check permissions and policy on each requested call: allow, deny, or ask a human.
Execute the tools (with validation, timeouts and retries) and append the results to the context with the matching call IDs.
Repeat until the model answers, a stop condition fires, or a limit is reached.
How would you write a minimal tool-calling agent loop?
Keep it small: a message list, a tool registry, a step cap, error handling that returns errors to the model as tool results, and correlation IDs.
Provider-agnostic sketch:
import json
def run_agent(client, messages, tool_specs, registry, max_steps=8):
for _ in range(max_steps): # hard stop condition
msg = client.chat(messages=messages, tools=tool_specs)
messages.append(msg)
if not msg.tool_calls: # model produced a final answer
return msg.content
for call in msg.tool_calls:
try:
args = json.loads(call.arguments)
result = registry[call.name](**args) # validate args and permissions here
except Exception as e:
result = f"ERROR: {e}" # let the model see and recover
messages.append({"role": "tool", "tool_call_id": call.id, "content": str(result)})
return "Stopped: step limit reached"What is ReAct, and how does it work?
ReAct interleaves reasoning and acting: the model writes a thought, chooses an action (a tool call), receives an observation, and repeats, so each step is informed by real results.
Example: to answer an order-status question, it thinks "I need the order record", calls the order lookup tool, observes the shipping date, then answers.
It is simple and flexible, and works well for short sequential tasks, but it can wander, loop or lose the goal on long tasks, and each step costs a model call.
ReAct or plan-and-execute: how do you choose?
ReAct decides one step at a time and adapts immediately; best for short, uncertain tasks with a few steps.
Plan-and-execute writes a plan first, then runs the steps, replanning only on failure; it is cheaper, easier to audit (the plan can be shown to a user for approval), allows independent steps to run in parallel, and lets a cheaper model execute steps.
It is weaker when early results change the plan constantly. A common hierarchy is plan-and-execute at the orchestrator level and ReAct inside each specialist.
What are reflection and self-correction in agents?
The agent critiques its own output or trajectory, using a second pass, a separate critic prompt or model, or tool-based checks, then revises (as in Reflexion or evaluator-optimizer loops).
It helps when quality can be checked objectively (tests pass, schema valid, sources support the claim) and drops off when the critic shares the generator's blind spots.
Bound it with a maximum number of revisions, and prefer external verification (run tests, check citations) over asking the model whether it is correct.
What common workflow patterns exist for LLM applications?
Prompt chaining: break a task into sequential steps, with checks between them.
Routing: classify the input and send it to a specialized prompt, model or tool.
Parallelization: run independent subtasks at once, or run the same task several times and vote.
Orchestrator-workers: a central model decomposes the task dynamically and delegates to workers.
Evaluator-optimizer: one model generates, another critiques, and the loop repeats. Use these before reaching for a fully autonomous agent.
When should you use an agent instead of a workflow?
Use the simplest architecture that meets the requirement: a single prompt first, then a workflow, and an agent only when the steps, tools or number of iterations cannot be known in advance.
Good agent fits: open-ended research, coding tasks, support cases requiring investigation across systems, and tasks where each result changes the next action.
Poor fits: well-defined pipelines, latency-critical paths, and tasks with irreversible high-stakes actions unless you add strong controls.
How do reasoning models change agent design?
Reasoning models spend extra tokens thinking before answering or calling tools, which improves planning, multi-step logic and tool selection, often reducing the need for elaborate prompting scaffolds.
Costs: higher latency and token usage, so use them for planning, hard decisions and verification, and cheaper models for routine steps (routing and summarization).
Keep reasoning text and tool results consistent across turns as the provider requires, and still enforce step, cost and permission limits.
What stop conditions should an agent have?
A final answer produced by the model, a maximum number of steps or tool calls, a token or cost budget, a wall-clock timeout, and detection of repeated identical calls or no progress.
On hitting a limit, return what was done, explain what is missing, and escalate to a human or a fallback rather than looping silently.
Make completion explicit when possible (a "finish" tool or structured final answer) so success criteria are checkable.
How do agents plan, and what is the difference between chain-of-thought and tree-of-thought?
Planning turns a vague goal into steps: the model drafts a task list, orders dependencies, assigns tools, and updates the plan as results arrive.
Chain-of-thought is a single linear line of reasoning; tree-of-thought explores several branches of reasoning, scores or prunes them, and continues the best (more accurate on hard problems, much more expensive).
In production, most agents use lightweight planning (an explicit task list or plan step) with replanning on failure.
What are the common failure modes of agents?
Infinite or wasteful loops, picking the wrong tool, hallucinated tool names or arguments, and stopping before the task is done.
Error compounding across steps, context overflow in long runs, forgetting constraints, and being hijacked by instructions inside tool outputs.
Mitigate with schema validation, clear error messages, step and budget limits, verification steps, trimmed context, least-privilege tools, and trajectory-level evaluation.
How does function calling work end to end, and what does the model actually see?
You send the model a list of tool definitions (name, description, JSON schema for parameters) along with the conversation; the model sees only that text, never your code.
If it decides a tool is needed, it returns a structured request: a tool name, JSON arguments, and a call ID, instead of final prose.
Your application parses the request, validates it, checks permissions, executes the function, and sends back a tool-result message tagged with the same call ID; the model then continues, possibly calling more tools.
The model never executes anything; whatever it can "do" is limited to what your code implements and allows.
What is the difference between tool use and function calling?
Tool use is the broad pattern of a model selecting and invoking external capabilities (search, code execution, databases, other agents, computer control).
Function calling is one implementation: the model emits structured arguments for functions you define.
Other implementations include built-in provider tools (web search, code interpreter), MCP servers, and models emitting code that a sandbox runs.
How do you design a good tool schema and description?
Write the description for the model: what the tool does, when to use it, when not to, and what it returns; vague descriptions cause wrong tool choices.
Use clear names (verbs and domain terms), precise parameter types, enums for closed sets, required versus optional fields, and short examples of valid values.
Prefer a small number of well-scoped, high-level tools over many low-level ones; return concise, relevant results with identifiers the model can reuse in later calls.
Test the tools with real prompts and iterate on descriptions based on observed mistakes.
Design a robust tool-calling system for an AI agent.
Validation: check every call against the schema, business rules, user permissions, ownership and limits before executing; never pass model output straight into a query or shell.
Reliability: timeouts, bounded retries with backoff for transient errors, circuit breakers, idempotency keys for side effects, and clear error results the model can act on.
Scale and operations: a tool registry with versions, rate limits and quotas, concurrency control, structured logs and traces with call IDs, and metrics per tool (success rate, latency, cost).
Safety: least-privilege credentials per tool, approval gates for sensitive actions, and sandboxing for code execution.
Which errors should a tool return to the agent, and how should they be shaped?
Return errors as tool results the model can read, not crashes or raw stack traces: say what failed, why, and what to try (valid ranges, missing fields, next step).
Separate protocol-level problems (unknown tool name, malformed request) from tool execution failures (rate limited, not found, no permission).
Distinguish retryable errors (timeouts, rate limits) from permanent ones (bad arguments, forbidden), cap retries, and classify before acting: retry, replan, stop or escalate.
How do parallel tool calls and forced tool choice work?
Parallel tool calls: the model can request several independent calls in one turn (for example fetch three records); run them concurrently, return all results with their call IDs, and keep dependent calls sequential.
Tool choice controls: automatic (the model decides), required (it must call some tool), a specific named tool, or none; forcing a tool is useful for guaranteed structured extraction or routing steps.
Always match results to calls by ID rather than position, and decide how to handle partial failures in a parallel batch.
When do you use JSON mode, structured outputs or function calling?
JSON mode guarantees syntactically valid JSON but not that it follows your schema.
Structured outputs (schema-constrained decoding) guarantee the response matches a given JSON schema; best when you only need data back in a fixed format.
Function calling is for actions: the model chooses among tools and supplies arguments, and your code does something with them.
In all cases validate with a schema library such as Pydantic, handle refusals, and retry with the validation error when a response does not parse.
How many tools can an agent handle, and how do you scale tool selection?
Accuracy of tool choice drops as the tool list grows, and every definition consumes context tokens, so dozens of overlapping tools cause confusion and cost.
Techniques: group tools by domain and route to a specialist agent or toolset, retrieve only the relevant tool definitions per request (tool retrieval by embedding the descriptions), load tools dynamically as the task unfolds, and merge near-duplicate tools.
Use distinct names and descriptions, namespaces when combining tool servers, and measure tool-selection accuracy in evaluations.
How do you make tools with side effects safe to retry?
Make them idempotent with an idempotency key derived from the task and the call, so the same request executed twice produces one effect; this matters when a response is lost after the action succeeded (for example a refund issued but the reply timed out).
The protection must live in the system executing the business operation, not just in the prompt: unique constraints, request deduplication, and status checks before acting.
For irreversible or high-value actions, add human approval, limits, and the ability to undo or compensate.
How do you handle long-running or slow tools?
Return a job handle immediately and let the agent poll for status (or receive a callback) instead of blocking, and give progress information the agent can reason about.
Set timeouts, support cancellation, and persist job state so the agent can resume after a restart.
For anything beyond seconds, consider running it as a background task with a notification, and avoid keeping a model call open while waiting.
How do you handle very large tool outputs?
Large results bloat the context, increase cost, hide the relevant parts, and are an injection surface; truncate, paginate, summarize, or return references (IDs, file paths) the agent can fetch from selectively.
Offer parameters for filtering and limiting fields on the tool itself, so the agent asks for what it needs.
Keep full outputs in storage or logs for debugging, and send the model a compact, relevant view.
What is the Model Context Protocol (MCP), and what are its main concepts?
MCP is an open standard that lets AI applications connect to external tools and data through a common interface, so one server implementation works with any compatible client.
Architecture: a host application (the AI app), MCP clients inside it (one per server connection), and MCP servers that expose capabilities.
Server primitives: tools (functions the model can call), resources (data the application can read, such as files or records), and prompts (reusable templates users can invoke); clients can also offer features such as sampling (the server asks the host's model for a completion) and roots.
It reduces integration work and vendor lock-in (an M-by-N integration problem becomes M plus N).
What transports and authentication does MCP use?
Local servers typically run as subprocesses communicating over standard input and output (stdio); remote servers use HTTP-based transports (streamable HTTP).
Remote servers should use standard authorization (OAuth-style flows) so the server acts with the user's own permissions and tokens are scoped and revocable.
Treat each server as an integration with its own trust level: authenticate, restrict scopes, and avoid sharing broad credentials.
When would you use MCP instead of native function calling, and how does it relate to A2A?
Native function calling is simplest for a few tools inside one application you control.
Use MCP when you want reusable, shareable integrations across applications, a growing ecosystem of servers, or to let users plug in their own tools and data.
MCP connects an agent to tools and data (vertical); agent-to-agent protocols such as A2A let separate agents discover and delegate to one another (horizontal). They are complementary, and many systems use both.
What security risks come with MCP and third-party tools?
Tool poisoning: malicious or compromised servers can put instructions into tool descriptions or results that steer the model.
Excessive permissions, confused-deputy problems (a server using its broad credentials for a low-privilege user), data exfiltration through tools, and unvetted servers.
Mitigate by allowlisting trusted servers, pinning versions, reviewing tool descriptions, granting least privilege, requiring approval for sensitive actions, isolating credentials from the model, and logging all calls.
What makes a good MCP server or tool API for agents?
Few, high-level tools that match user tasks (rather than a one-to-one wrapper over every REST endpoint), with clear names, descriptions and schemas.
Token-efficient responses (concise fields, pagination, filters), helpful error messages with next steps, and stable behavior across versions.
Idempotent writes, explicit permissions and scopes, rate limits, and good observability so failures can be diagnosed.
How do code execution and sandbox tools work, and how do you secure them?
A code interpreter tool lets the model write and run code (data analysis, calculations, file processing) and read the output, which is far more reliable than mental arithmetic.
Run it in an isolated sandbox: restricted network and filesystem, resource and time limits, no access to secrets, and ephemeral environments.
Treat generated code as untrusted, log what ran, and require approval before anything that touches production systems or sends data out.
What types of memory does an agent need, and how do you implement them?
Working (short-term) memory: the current conversation and task state, held in the context window.
Episodic memory: records of past interactions and outcomes, stored externally and retrieved when similar situations arise.
Semantic memory: durable facts and user preferences, stored in a database or vector store and retrieved by relevance.
Procedural memory: learned instructions, playbooks and successful patterns, often kept as updated prompts or skill files.
Different kinds of state should live in separate stores; stuffing all history into every prompt is poor architecture. Let users view, correct and delete what is remembered.
How do you manage context in long-running agents?
Summarize or compact older turns once the context approaches its limit, keeping decisions, open questions and key identifiers.
Offload to external state: a scratchpad or notes file the agent writes to and reads back, with a task list recording the plan and progress.
Delegate to sub-agents with fresh, narrow context and have them return concise results, and retrieve only relevant memories and tool definitions instead of loading everything.
Trim bulky tool outputs, since long contexts raise cost, slow responses and degrade accuracy (context rot).
How do you decide between a single agent and a multi-agent system?
Start with one agent and good tools; add agents only when you have a clear reason: distinct roles with different instructions, tools or permissions, work that exceeds one context window, or independent subtasks that can run in parallel.
Costs of multi-agent systems: more tokens and latency, coordination overhead, harder debugging, and errors that compound across agents.
Rule of thumb: if you cannot name what each agent owns and how they hand work to each other, one agent is probably enough.
What are the main multi-agent patterns?
Orchestrator-worker (supervisor): a lead agent plans, delegates to specialist workers, and synthesizes their results; the most common pattern.
Handoffs: control transfers between agents (for example from triage to billing) and the receiving agent owns the conversation.
Hierarchical teams: supervisors managing sub-supervisors for large problems.
Pipelines (each agent does one stage), parallel specialists with a merger, and debate or critic patterns where agents challenge each other's outputs.
What is the difference between using agents as tools and using handoffs?
Agents as tools: a manager agent calls specialist agents like functions and keeps ownership of the final answer, combining outputs and enforcing shared guardrails in one place.
Handoffs: the specialist takes over the interaction directly; best when routing is part of the workflow and the specialist should talk to the user.
Choose agents-as-tools when you need synthesis or central control, and handoffs when specialization and a clean conversational transfer matter.
How do multiple agents coordinate, and how do you resolve conflicts?
Define clear ownership of tasks and data, and communicate through typed messages or a shared state object instead of free-form chat, so each agent knows what it receives and must return.
Use a single orchestrator or arbiter for decisions, ordered priorities, voting or a verifier agent when outputs conflict, and avoid two agents writing to the same resource without locking.
Add message queues or event-driven coordination for concurrency, idempotent steps, and a trace that links every agent call under one request ID.
What happens when one agent in a multi-agent system fails?
Detect it with timeouts and validation of outputs, then retry (bounded), fall back to a simpler path or alternate agent, or continue with partial results and flag what is missing.
The orchestrator should be able to replan: skip the failed step, change approach, or escalate to a human.
Contain blast radius with circuit breakers, budgets per agent, isolated state, and checkpoints so work already completed is not lost or repeated.
What are the core concepts of LangGraph, and how does it handle persistence?
A graph of nodes (functions or agents) connected by edges, including conditional edges for branching, operating on a shared typed state that each node reads and updates.
Checkpointing saves the state after each step to a backing store, which enables resuming after failures, replaying and debugging, and long-running workflows.
Interrupts pause execution at a node pending human input or an external event, then resume from the checkpoint, which is how human-in-the-loop approvals are implemented.
How do LangGraph, CrewAI, AutoGen and the vendor agent SDKs compare?
LangGraph: graph-based, explicit state and control flow, persistence and human-in-the-loop; a good choice for production reliability and complex branching.
CrewAI: role-based crews with tasks and delegation; quick to prototype collaborative multi-agent setups, with less fine-grained control.
AutoGen: conversation-driven multi-agent framework for agents talking to each other.
Vendor SDKs (such as the OpenAI Agents SDK or the Claude Agent SDK) give lightweight loops, tool integration, handoffs and guardrails tuned to their models; consider portability and lock-in.
When should you use a framework, and when a hand-written loop?
A hand-written loop is a few dozen lines, easy to understand and debug, and avoids abstraction overhead; start there to learn what is really happening.
Frameworks help with persistence, streaming, tracing, retries, handoffs and multi-agent orchestration once the system grows, and speed up teams that share conventions.
Risks: heavy abstractions hide prompts and behavior, frameworks change quickly, and debugging through layers is harder; keep business logic and tools independent of the framework.
How do you make agents durable so they survive crashes and restarts?
Persist state after every meaningful step (messages, plan, tool results, pending actions) in a database, and design the loop so it can resume from the last checkpoint.
Make steps idempotent (stable call IDs, idempotency keys) so replaying after a crash does not repeat side effects; long-running agents can run on a workflow engine (such as Temporal) or a job queue with retries.
Store enough to reconstruct the context and verify progress, and support cancellation and timeouts.
How do you design human-in-the-loop controls?
Tier actions by risk: read-only operations run freely, reversible changes may proceed with logging, and irreversible or high-value actions (payments, deletions, external messages) require explicit approval.
Implement as pause-and-resume: the agent stops at a checkpoint, presents the proposed action with its reasoning and arguments, and continues after approval, edit or rejection.
Avoid approval fatigue by grouping related actions, remembering scoped approvals, and escalating only genuinely risky or uncertain steps; log every decision for audit.
What is the A2A protocol, and what is an Agent Card?
Agent-to-agent (A2A) protocols let independent agents, possibly built by different teams or vendors, discover each other and delegate tasks over a standard interface.
An Agent Card is a machine-readable description of an agent (its capabilities, endpoint, supported inputs and authentication) that others use to decide whether and how to call it.
It complements MCP: MCP gives an agent tools and data, while agent-to-agent protocols let agents work with one another.