Write an async batch processor that runs an LLM call over 50,000 documents with a concurrency limit, retries with jitter, and error isolation
Interview insights are lockedSee how often this is asked and which companies asked it with Buncho Pro.You have async call_llm(doc) -> str and 50,000 documents (id, text). Write process_all(docs, call_llm, concurrency=50) that:
- runs at most
concurrencycalls at once without creating 50,000 tasks in memory, - retries transient errors (429, 5xx, timeouts) with exponential backoff and jitter, but does not retry permanent errors,
- isolates failures: one bad document never stops the run, and every failure is recorded with its error,
- writes results as they finish and can resume after a crash without redoing finished documents.