Trim the system prompt and remove redundant examples; measure tokens, not characters.
Use prompt caching for long static prefixes, cache repeated responses, and route simple tasks to smaller models.
Cap output length, ask for concise formats, stream responses for perceived speed, and run independent calls in parallel.
Retrieve fewer but better chunks instead of stuffing the context window.
Loading…