Generative AI and LLMs · 53 topics
What is a large language model, and how does it generate text?
An LLM is a transformer neural network trained on huge amounts of text to predict the next token given the tokens so far.
Generation is autoregressive: the model outputs a probability distribution over the vocabulary, one token is chosen (greedy or sampled), appended to the input, and the process repeats until an end-of-sequence token or a length limit.
Capabilities such as summarization, translation and reasoning emerge from this objective at scale, then are shaped by instruction tuning and preference alignment.
LLMs predict plausible text, not verified truth, which is the root of hallucinations.
What is tokenization, and how does byte-pair encoding (BPE) work?
Tokenization splits text into subword units (tokens) that map to integer IDs; models work on tokens, not characters or words.
BPE starts from characters (or bytes) and repeatedly merges the most frequent adjacent pair into a new token until the vocabulary reaches a target size, so common words become single tokens and rare words split into pieces.
Rule of thumb: 1,000 English words is roughly 750 tokens; code, numbers and non-English text often use more tokens per word.
Tokenization affects cost, context usage and quality across languages. Alternatives include WordPiece, SentencePiece and byte-level tokenizers.
Why do LLMs struggle with counting letters or exact arithmetic?
The model sees tokens, not individual characters, so "strawberry" may be two or three chunks and the letter count is not directly visible.
Arithmetic is learned statistically from text rather than computed by an algorithm, so long multiplication or precise numeric tasks are error-prone.
Fixes: let the model call a calculator or code interpreter as a tool, ask for step-by-step reasoning, and verify outputs programmatically.
What are temperature, top-k and top-p sampling, and how do you choose them?
Temperature rescales the logits before softmax: low values (0 to 0.3) make output focused and near-deterministic (greedy at 0), high values (0.7 to 1.0) make it more varied and creative.
Top-k keeps only the k most likely tokens, then samples from them.
Top-p (nucleus) keeps the smallest set of tokens whose cumulative probability exceeds p, adapting to how confident the model is.
Use low temperature for extraction, code and factual answers, higher for brainstorming and creative writing; tune one control at a time.
What are greedy decoding, beam search and sampling?
Greedy decoding picks the single most likely token at each step: fast and deterministic but can be repetitive and miss better overall sequences.
Beam search keeps the top several partial sequences and picks the best overall; common in translation, but it tends to produce bland, repetitive text in open-ended generation.
Sampling (with temperature, top-k or top-p) adds randomness for diverse, natural text. Repetition penalties and stop sequences give further control.
What is a context window, and what happens when you exceed it?
The context window is the maximum number of tokens (prompt plus output) the model can process at once.
Exceeding it either errors or forces truncation, so older content is silently dropped.
Even inside the window, models can miss information buried in the middle of a long prompt ("lost in the middle").
Manage it with conversation summarization, sliding windows, retrieval instead of stuffing everything in, and pruning history by relevance; remember attention cost and latency grow with context length.
What is the difference between pretraining, fine-tuning and in-context learning?
Pretraining: next-token prediction on a massive corpus; the expensive stage that gives general language and world knowledge.
Fine-tuning: further training on a smaller task or instruction dataset, which changes the model's weights.
In-context learning: the model adapts to a task from instructions and examples placed in the prompt, with no weight updates.
Start with prompting, add retrieval for knowledge, and fine-tune only when you need consistent style or behavior, lower cost, or higher quality than prompting reaches.
What causes hallucinations in LLMs, and how do you reduce them?
Causes: the model optimizes for plausible continuations rather than truth; gaps or errors in training data; outdated knowledge; ambiguous prompts; and pressure to answer rather than say "I don't know".
Reduce: ground answers in retrieved sources (RAG) and require citations; lower temperature; instruct the model to admit uncertainty; use structured outputs and tools for facts and math.
Verify: check claims against sources, run self-consistency or a separate verifier pass, and measure the unsupported-claim rate on a test set.
Design the product so users can see sources and the system can refuse or escalate.
What are embeddings, and how are they used in LLM applications?
Embeddings are dense vectors that capture meaning, so semantically similar texts land close together in vector space.
Uses: semantic search and RAG retrieval, clustering, deduplication, classification, recommendations, and anomaly detection.
Compare vectors with cosine similarity or dot product; pick an embedding model that fits your language, domain and length limits, and re-embed when you change models.
What are scaling laws, and what does it mean to overtrain a model?
Scaling laws show that loss falls predictably as model size, data and compute grow.
The Chinchilla result suggested a compute-optimal balance of roughly 20 training tokens per parameter, so many earlier models were too large and under-trained.
Recent models are deliberately trained on far more tokens than that: it costs more at training time but gives a smaller, cheaper-to-serve model with the same quality, which matters when inference dominates cost.
What is a mixture-of-experts (MoE) model?
An MoE layer contains many expert feed-forward networks, and a router sends each token to only a few of them.
Total parameters are large, but only a fraction is active per token, so quality improves at a lower compute cost per token than a dense model of the same size.
Trade-offs: all experts must sit in memory, routing needs load balancing, and training and serving are more complex.
What are the trade-offs between open-weight and closed (API) models?
Closed API models: usually the strongest quality, no infrastructure to run, fast to start; but per-token cost, data-privacy concerns, rate limits, vendor lock-in, and less control.
Open-weight models: full control over deployment, fine-tuning and data residency, predictable cost at scale; but you own the serving, scaling, safety and updates, and quality may trail the frontier.
A common pattern is to prototype on an API, then move specific high-volume or sensitive workloads to a tuned open model.
How do diffusion models and multimodal models work at a high level?
Diffusion models learn to remove noise step by step; generation starts from random noise and denoises iteratively, guided by a text embedding for text-to-image.
Multimodal LLMs encode images, audio or video into embeddings (for example with a vision encoder) and feed them into the language model alongside text tokens, so one model can reason across modalities.
Typical challenges: aligning modalities, higher compute and token costs for images and video, and hallucinations about visual details.
What prompt engineering techniques work best in production?
Be explicit: state the role, task, audience, constraints, and the exact output format.
Give examples (few-shot) that cover edge cases, and show the desired output structure.
Separate instructions from data using delimiters or tags, so untrusted text is clearly marked as content.
Break complex tasks into steps or chained prompts, and give the model permission to say "I don't know".
Version prompts, test them against an evaluation set, and track regressions whenever the prompt or model changes.
What is the difference between zero-shot, few-shot and chain-of-thought prompting?
Zero-shot: instructions only, no examples. Few-shot: a handful of input and output examples in the prompt to show format and behavior.
Chain-of-thought (CoT): ask the model to reason step by step before answering, which improves multi-step math, logic and planning.
CoT adds tokens, cost and latency, and the shown reasoning is not always a faithful explanation. Dedicated reasoning models do this internally.
Your chain-of-thought prompting is not improving accuracy. What do you check?
Whether the task actually needs multi-step reasoning; for simple lookups or classification, CoT may add noise.
Whether the model is large or capable enough, and whether the reasoning steps are guided (a clear plan or rubric) rather than vague.
Whether errors come from missing knowledge or bad retrieval rather than reasoning; then fix the context, not the prompt.
Try few-shot examples with worked reasoning, self-consistency (sample several reasoning paths and vote), tool use for calculations, and a verification step.
How do you get reliable structured output from an LLM?
JSON mode or schema-constrained output forces valid JSON; function (tool) calling makes the model return a function name with structured arguments that match a schema.
Validate every response with a schema library (such as Pydantic), and retry with the validation error fed back to the model if parsing fails.
Keep schemas simple, include field descriptions and enums, and set low temperature.
Use constrained decoding when you need a guarantee of valid structure.
How do you optimize prompts for cost and latency?
Trim the system prompt and remove redundant examples; measure tokens, not characters.
Use prompt caching for long static prefixes, cache repeated responses, and route simple tasks to smaller models.
Cap output length, ask for concise formats, stream responses for perceived speed, and run independent calls in parallel.
Retrieve fewer but better chunks instead of stuffing the context window.
Explain the architecture of a RAG system and its key components.
Indexing: load documents, clean and chunk them, create embeddings, and store vectors with metadata in a vector database (plus an optional keyword index).
Retrieval: embed the query, run similarity (or hybrid) search with metadata filters, and optionally rerank.
Generation: build a prompt from the question and top chunks, instruct the model to answer only from the context and cite sources, then return the answer.
Around it: evaluation, monitoring, access control, caching and refresh of the index as documents change.
When would you choose RAG, fine-tuning or long-context prompting?
RAG: knowledge that is large, private, or changes often; needs citations and access control; cheapest way to add facts.
Fine-tuning: change behavior (style, format, domain reasoning, tool-use patterns) or distill into a smaller, faster model; poor at injecting fast-changing facts.
Long context: small or one-off document sets where simplicity matters; costs more per request, adds latency, and accuracy can drop for details in the middle.
They combine: fine-tune for behavior, RAG for knowledge, long context for a few key documents. Start with the cheapest option and measure.
How do you choose an embedding model and a vector database?
Embedding model: match language and domain, check dimension (storage and speed), max input length, and benchmark on your own labeled queries rather than only public leaderboards.
Vector database: consider scale, latency, filtering support, hybrid search, operational burden, and cost (Pinecone, Weaviate, Qdrant, pgvector, FAISS, Chroma).
Approximate nearest-neighbor indexes: HNSW (graph, fast and accurate, memory-heavy) and IVF (clusters, with optional quantization for memory savings); they trade a little recall for large speed gains.
Plan for re-indexing when you change models and for deleting or updating documents.
What are hybrid search and reranking, and why do they help?
Vector search captures meaning but can miss exact terms (names, IDs, error codes); keyword search (BM25) catches those.
Hybrid search runs both and merges rankings, for example with reciprocal rank fusion.
A reranker (cross-encoder) rescoring the top 20 to 50 candidates reads the query and each chunk together, boosting precision at the cost of extra latency.
Together they usually give a large retrieval-quality gain for modest complexity.
What advanced RAG techniques do you know?
Query transformation: rewrite follow-ups into standalone queries, generate multiple query variants, or HyDE (embed a hypothetical answer to retrieve similar passages).
Better chunking: structure-aware splits, parent-child (retrieve small chunks, return larger context), and metadata enrichment.
Corrective and agentic RAG: grade retrieved results, re-query or search the web when they are weak, and let an agent decide when and what to retrieve.
Graph RAG for relationship-heavy data, and compressing or reordering retrieved context before generation.
How do you evaluate a RAG system?
Evaluate retrieval and generation separately so you know which stage fails.
Retrieval: recall@k, precision@k, MRR or NDCG against labeled relevant chunks; context precision and context recall.
Generation: faithfulness (is the answer supported by the retrieved context), answer relevance, and correctness against reference answers; tools like RAGAS automate these with LLM judges.
Build a golden set from real questions, add LLM-as-judge with a clear rubric calibrated against human review, and track regressions on every change; add online signals such as thumbs up/down and escalations.
Your AI system works in English but fails in other languages. How do you fix it?
Use a multilingual embedding model so queries and documents in different languages land in the same vector space.
Check tokenization: some languages use many more tokens per word, hurting cost and context budget.
Evaluate per language with native-speaker labeled test sets; add few-shot examples or fine-tune on target-language data if quality lags.
Consider translating queries or documents as a stopgap, and make sure prompts and guardrails also work in each language.
How do you handle messy documents such as PDFs with tables, scans and images in RAG?
Use layout-aware parsing and OCR for scans, keep tables intact (convert to Markdown or structured text rather than splitting rows), and attach captions and headings as metadata.
For charts and figures, use a vision model to generate descriptions that are indexed alongside the text.
Validate parsing quality with spot checks and automatic checks (empty pages, garbled text), because bad parsing silently ruins retrieval.
When should you fine-tune an LLM instead of prompting it, and what is the difference between full fine-tuning and parameter-efficient fine-tuning?
Fine-tune when prompting and retrieval are not enough: you need consistent style or output format, domain-specific behavior, lower latency and cost through a smaller model, or you have lots of good examples.
Do not fine-tune to add fast-changing facts; use retrieval for that.
Full fine-tuning updates all weights: highest capacity but heavy GPU memory, large checkpoints, and a higher risk of forgetting.
Parameter-efficient methods (LoRA, adapters, prefix tuning) update a small number of parameters, with far lower cost and swappable task-specific adapters.
How does LoRA work, and what is QLoRA?
LoRA freezes the pretrained weights and learns a low-rank update to selected weight matrices (often attention projections): the update is the product of two small matrices with rank r (commonly 8 to 64), scaled by an alpha factor.
Only these small matrices are trained, so memory and compute drop sharply; after training they can be merged into the base weights or kept as separate adapters.
QLoRA keeps the frozen base model quantized to 4-bit while training the LoRA adapters in higher precision, letting large models be fine-tuned on a single GPU.
Higher rank gives more capacity but more parameters and overfitting risk.
How much data do you need for supervised fine-tuning, and what matters most?
Quality and diversity matter more than quantity: a few hundred to a few thousand clean, representative examples often suffice for style and format changes; harder skills need more.
Examples must match the real prompt format, cover edge cases, and have consistent, correct outputs; noisy labels directly teach the model mistakes.
Hold out a validation set, watch for duplicates and leakage with the evaluation data, and compare against a strong prompting baseline.
What is RLHF, and what are its stages?
Stage 1, supervised fine-tuning (SFT): train on high-quality instruction and response examples so the model follows instructions.
Stage 2, reward model: humans rank multiple model responses for the same prompt; a model is trained to predict those preferences as a score.
Stage 3, reinforcement learning (PPO): optimize the language model to maximize the reward score, with a KL penalty that keeps it close to the SFT model.
Risks: reward hacking (exploiting flaws in the reward model), instability, and the cost of collecting good preference data.
What is DPO, and how does it differ from RLHF with PPO?
Direct Preference Optimization trains directly on preference pairs (prompt, chosen response, rejected response) with a simple classification-style loss that raises the likelihood of the chosen response relative to the rejected one, measured against a frozen reference model; a beta parameter controls how far the model may drift.
It needs no separate reward model and no reinforcement-learning loop, so it is simpler, more stable and cheaper than PPO.
Training data: collect or generate two responses per prompt, label which is better (human or strong-model judge), and filter noisy pairs.
PPO-style methods can still be stronger when you have a good reward signal, such as verifiable rewards for math and code.
Your fine-tuned model got worse at other tasks. What happened, and how do you prevent it?
This is catastrophic forgetting: updating weights toward the new data erases some of what the model learned before.
Reduce it with LoRA or other parameter-efficient methods, a lower learning rate and fewer epochs, mixing in some general instruction data (replay), and regularization toward the base model.
Always evaluate on a broad regression suite (general capabilities, safety, your core tasks), not only on the new task, and keep the base model as a fallback.
How do you avoid overfitting during fine-tuning?
Use a held-out validation set and stop early when validation loss rises, even if training loss keeps falling.
Use fewer epochs (often 1 to 3), a small learning rate, LoRA with moderate rank, weight decay, and dropout.
Increase data diversity, remove near-duplicates, and test on prompts phrased differently from the training set.
What is model distillation?
A smaller student model is trained to imitate a larger teacher, using the teacher's outputs (text or probability distributions) as training targets.
It gives much of the teacher's quality at a fraction of the latency and cost, and is widely used to create small task-specific models.
Check the licensing terms for using a model's outputs as training data, and evaluate the student on your real tasks.
What is perplexity, and what are its limits?
Perplexity is the exponential of the average negative log-likelihood per token; intuitively, how many equally likely next tokens the model is choosing between, so lower is better.
It is useful for comparing language models on the same data and tokenizer and for monitoring pretraining.
It does not measure helpfulness, factual accuracy, safety or instruction following, and it is not comparable across different tokenizers.
How do you evaluate an LLM on open-ended generation tasks?
Combine methods: exact-match or programmatic checks where answers are verifiable, task-specific metrics, LLM-as-judge with an explicit rubric for quality dimensions, and human review of a sample.
Use public benchmarks (MMLU, HumanEval, GSM8K and others) only as rough signals: they can be contaminated by training data and may not match your use case.
Build a custom evaluation set from real user queries, include failure cases and adversarial prompts, and track regressions across model, prompt and retrieval versions.
Pair offline evaluation with online signals such as user ratings, task completion and escalation rates.
What are the biases of LLM-as-judge evaluation, and how do you mitigate them?
Known biases: position bias (preferring the first or last answer), verbosity bias (favoring longer answers), self-preference (a model favoring its own style), and sensitivity to rubric wording.
Mitigate: randomize or swap answer order and average, use clear scoring rubrics with examples, ask for reasoning before the score, and use a strong judge model that differs from the one being tested.
Calibrate against human labels on a sample, report agreement, and keep humans in the loop for high-stakes decisions.
How would you build an evaluation set for an LLM product from scratch?
Collect real or realistic inputs (logs, support tickets, domain experts' questions) covering common cases, edge cases, ambiguous inputs and adversarial prompts.
Define success criteria per case: a reference answer, a checklist or rubric, required sources, or a programmatic assertion.
Start small (50 to 200 high-quality cases), label carefully with domain experts, version the dataset, and grow it with failures found in production.
Run it automatically on every prompt, model or retrieval change and block releases that regress key metrics.
What is an AI agent, and how is it different from a chatbot or a fixed workflow?
A chatbot responds to messages; a fixed workflow follows steps you coded in advance. An agent uses an LLM to decide its own next steps: it plans, calls tools, observes results, and repeats until the goal is met.
The core pattern is a loop (such as ReAct: reason, act, observe) with memory, tools, and stop conditions.
Prefer a simple workflow when the steps are known; use an agent only when the path genuinely varies, since agents cost more, are slower and harder to test.
How does tool (function) calling work under the hood?
You describe each tool with a name, description and JSON schema for its arguments; the model sees only these descriptions.
When it decides a tool is needed, it returns a structured request (function name plus arguments) instead of final text. Your code validates the arguments, checks permissions, runs the tool, and sends the result back as a tool message.
The model then continues, possibly calling more tools, until it writes the final answer.
The model never executes anything itself, so validation, authorization and error handling are the application's job.
A tool call fails. What should the agent see?
A clear, actionable error returned as the tool result (what failed and how to fix it, such as a missing argument or a rate limit), not a crashed loop and not a raw stack trace.
Distinguish retryable errors (timeouts, rate limits: retry with backoff) from permanent ones (bad arguments, no permission) so the agent corrects or stops.
Cap retries, and for long-running actions return a handle that the agent polls instead of calling again.
What is the Model Context Protocol (MCP), and when would you use it instead of native function calling?
MCP is a standard protocol for exposing tools, resources and prompts from servers to any compatible client, so one integration works across many applications.
Use it when you want reusable, shareable integrations (databases, SaaS apps, file systems) or to let users plug in their own tools; native function calling is simpler for a few tools inside one application.
Treat MCP servers as an attack surface: authenticate, apply least privilege, and be careful with untrusted servers and tool descriptions.
When would you use multiple agents instead of one?
Start with a single agent and tools. Split into multiple agents when roles need different instructions, tools or permissions, when contexts would overflow one prompt, or when parallel specialists speed up the work.
Common patterns: an orchestrator calling specialist agents as tools (it keeps ownership of the final answer and shared guardrails) or handoffs where control passes to a specialist.
Costs: more tokens, harder debugging and compounding errors, so add tracing under one request ID and evaluate end-to-end success.
How do you give an agent memory?
Short-term memory is the conversation context; trim or summarize it as it grows.
Long-term memory stores facts, preferences and past outcomes outside the prompt (database or vector store) and retrieves relevant items when needed.
Keep task state (plan, completed steps, decisions) in a durable store so the agent can resume after a failure, and let users view, correct and delete what is remembered.
How do you stop an agent from looping forever or running up costs?
Set hard limits on steps, tokens, tool calls, wall-clock time and spend per task.
Detect repeated identical tool calls or no progress, and escalate to a fallback or a human.
Give clear stop conditions in the prompt and require the agent to state what is still missing before continuing.
Log every step so loops are visible and budgets can trigger alerts.
What is prompt injection, and how do you defend an LLM application against it?
Prompt injection is when untrusted text (a user message, web page, document or tool result) contains instructions that override the developer's intent; indirect injection hides them in retrieved content.
No prompt wording fully prevents it, so use layered controls: treat all external content as data, separate instructions from content, validate and sanitize inputs, and screen with an injection classifier.
Limit blast radius: least-privilege tools, no secrets in the prompt, human approval for sensitive actions, and application-side permission checks.
Validate outputs, log decisions, and red-team regularly.
What is jailbreaking, and how do you mitigate it?
Jailbreaking is crafting inputs (role-play, hypothetical framing, encoding tricks, multi-turn escalation) that get the model to ignore its safety rules.
Mitigate with model-level alignment, input and output moderation, system-prompt hardening, rate limits and abuse detection, and continuous red-teaming with a growing library of attacks.
Assume some attempts will succeed and design downstream controls so a successful jailbreak cannot cause serious harm.
Users are leaking your system prompt. How do you respond?
Assume the system prompt can eventually be extracted, so never place secrets, credentials or critical business logic that must stay private in it.
Move sensitive logic and data access into server-side code and tools with proper authorization; keep the prompt to behavior guidance.
Add output filters for verbatim prompt text, canary strings to detect leaks, and instructions not to reveal the prompt, as a weak extra layer rather than the main defense.
What are guardrails, and how do you implement them?
Guardrails are checks around the model: input filters (toxicity, PII, injection attempts, off-topic requests), output filters (unsafe content, policy violations, ungrounded claims, schema validity), and action-level policies for tools.
Implement with rules and regexes for simple cases, classifier or LLM-based moderation for nuance, and a policy engine that validates tool calls against the user's permissions.
Run cheap checks first to save latency, decide whether to block, rewrite or escalate, and measure false positives so the product stays usable.
How do you control cost and latency of an LLM application in production?
Route requests to the smallest capable model, and use a larger one only when needed.
Cache: prompt caching for long static prefixes, exact-match and semantic caching for repeated queries.
Reduce tokens: shorter prompts, fewer retrieved chunks, capped outputs; batch non-urgent work and run independent calls in parallel.
Stream tokens for perceived speed, set timeouts with fallbacks, and monitor cost per request and per feature with budgets and alerts.
What should you log and monitor for an LLM application (LLMOps)?
Per request: prompt and model version, retrieved chunk IDs, tool calls and results, latency (including time to first token), token counts, cost, safety decisions, and user feedback; hash or redact sensitive text.
Track quality continuously: sampled human review, LLM-judge scores, groundedness, refusal and error rates, and drift in the topics users ask about.
Pin model versions, and run your evaluation suite before adopting a new model or prompt, because behavior changes between versions.
How do you test an AI agent before shipping it?
Unit-test each tool (inputs, errors, permissions) without the model.
Integration-test full trajectories with mocked tools: did the agent pick the right tools in a sensible order and finish within budget?
Run prompt-regression suites with LLM-as-judge plus assertions, and a red-team set for injection and misuse.
Roll out gradually (shadow mode, canary, human approval for risky actions) and watch task success, cost and escalation metrics.
How do you address bias, privacy and compliance in generative AI applications?
Bias: evaluate outputs across demographic groups and languages, use diverse data, and add review for high-impact decisions.
Privacy: minimize and redact personal data before sending it to models, set data-retention and training-use terms with providers, and enforce access control on retrieved documents.
Compliance and transparency: log decisions for audit, disclose AI use to users, offer human escalation, and follow regional regulations and your industry's rules.