Per request: prompt and model version, retrieved chunk IDs, tool calls and results, latency (including time to first token), token counts, cost, safety decisions, and user feedback; hash or redact sensitive text.
Track quality continuously: sampled human review, LLM-judge scores, groundedness, refusal and error rates, and drift in the topics users ask about.
Pin model versions, and run your evaluation suite before adopting a new model or prompt, because behavior changes between versions.
Loading…