Design an LLM inference platform (vLLM-as-a-service)
Interview insights are lockedSee how often this is asked and which companies asked it with Buncho Pro.Design a multi-tenant platform that serves open-weight LLMs through an API with high GPU utilisation, low latency, rate limits and autoscaling.