Mid-size fintech company · Bengaluru (hybrid) · 2026-11-13
Four rounds covering Python and SQL screening, a deep RAG discussion, an LLM evaluation system design round and a hiring manager chat. The RAG and evaluation rounds were the hardest.
Shared by Malik · 1.5 years of experience
Online (HackerRank style) · 90 min · 4 questions
Two coding questions, one SQL question and 15 ML and LLM MCQs. Cut-off was roughly two coding questions plus 70% on MCQs.
Q1.Given a list of document chunks with embeddings, return the top-k most similar chunks to a query vector without using any vector library (cosine similarity in plain Python or NumPy).
Q2.Merge overlapping intervals. Then extend it: merge overlapping text spans that come from the same document when building a citation list.
Q3.Write a SQL query to find the top 3 users by number of LLM requests per day, along with their average latency, from a table requests(user_id, created_at, latency_ms).
Q4.MCQs: what does temperature do to the softmax over logits, which metric is best for imbalanced classification, and what is the purpose of KV caching.
Video call · 60 min · 4 questions
Started with a walkthrough of my RAG project, then the interviewer kept pushing on failure modes: why retrieval was bad, how I knew, and what I changed.
Q1.Walk me through your RAG pipeline end to end. Why did you pick that chunk size and overlap, and how did you validate the choice?
Q2.Dense retrieval returns plausible but wrong chunks for queries with exact product codes. How do you fix it?
Q3.What is the difference between a bi-encoder and a cross-encoder? Where does a reranker sit and what latency does it add?
Q4.How would you reduce hallucinations when the retrieved context does not contain the answer?
Video call with a shared whiteboard · 75 min · 4 questions
One open-ended design question with several follow-ups on metrics, cost and online monitoring.
Q1.Design an evaluation pipeline for a customer-support RAG assistant. What do you measure offline, and what do you monitor in production?
Q2.How would you use an LLM as a judge, and how do you check that the judge itself is reliable (position bias, verbosity bias, agreement with humans)?
Q3.Faithfulness, answer relevance and context precision: define each and say which one you would alert on first.
Q4.Your eval set has 200 questions and a prompt change improves the score by 2 points. How do you decide if it is real?
Video call · 40 min · 2 questions
Mostly project ownership and behavioural questions, then a short discussion on notice period and expectations.
Q1.Tell me about a time a model or pipeline you built failed in production or in a demo. What did you do?
Q2.Why GenAI engineering and not classical ML? Where do you want to be in two years?