Top 50 LLM Interview Questions (Hoang)
A self-published PDF by Hao Hoang (May 2025, promoted via LinkedIn): 50 large-language-
model interview questions, each with a model answer. Arrived as a Google Drive file over Telegram;
the PDF is held in raw/ and the Drive link recorded for origin. Routed here 2026-08-05.
What it is
A flat list — 50 questions, one to two paragraphs of answer each — covering the concepts an LLM / AI-engineering interview reaches for. Grouped by what they actually test:
- Inference & decoding — tokenization, the attention mechanism, multi-head attention, softmax and dot-product scoring, context window, positional encodings, beam search, temperature, top-k / top-p sampling.
- Training & adaptation — LoRA vs QLoRA, PEFT, catastrophic forgetting, model distillation, overfitting, masked language modelling, next-sentence prediction, embeddings.
- The maths behind it — cross-entropy loss, gradients through embeddings, the Jacobian, eigenvalues/eigenvectors, KL divergence, the ReLU derivative, the chain rule, vanishing gradients.
- Architectures & systems — encoders vs decoders, autoregressive vs masked models, MoE, RAG, chain-of-thought, zero-/few-shot, foundation-model types, GPT-4 vs GPT-3, Gemini multimodal, knowledge-graph integration, deployment challenges, bias mitigation.
Why it files here, and not in the technical spokes
Every question is about another spoke’s subject — sampling and the KV/context story live in
../llm-inference-wiki, LoRA/QLoRA/PEFT/distillation in ../machine-learning-wiki, the GPT-4/Gemini/
foundation-model market in ../llm-providers-wiki, RAG and chain-of-thought in ../research-wiki.
The document makes no new claim about any of them; it is an assessment artifact, a distilled exam.
That is exactly the boundary this spoke draws: a source that makes a claim about a discipline routes
to the spoke owning the claim; a source shaped by how the discipline is tested routes here. So it
belongs to interview-preparation, with the technical substance cross-linked rather than restated.
The purest instance yet of the spoke’s founding thesis
The founding thesis is that what gets taught is downstream of what gets assessed. The three founding sources are curricula shaped by an exam; this is the exam itself, with no curriculum around it — 50 questions and their answers, nothing to build, no path to follow. It is the assessment regime named directly, and it is the spoke’s first source on the AI/LLM-engineering interview specifically, where the founding three cover CS fundamentals (coding-interview-university) and system design (awesome-system-design-resources).
It also tells you what the LLM interview currently rewards: recall of mechanism. Not “build a tokenizer” but “define tokenization”; not “tune a model” but “state the difference between LoRA and QLoRA.” The genre is the flashcard, not the capstone — the opposite end of the pedagogy axis from ai-engineering-from-scratch‘s build-it-by-hand, and worth recording as such.
Caveats
- T3, self-published and promotional. One author, no editorial review, and the framing is lead-generation (“Share this knowledge with your network,” “Follow me on LinkedIn”). Held with the weakness recorded, not refused.
- Introductory and occasionally loose. Answers are correct-ish at survey depth; some examples are invented (the “artificial → art / ific / ial” tokenization is illustrative, not a real BPE split) and some content dates fast (a GPT-4-vs-GPT-3 question in a field that has moved well past both).
- No provenance for its own claims. It cites nothing; it is one practitioner’s compression of common knowledge, which is the normal shape of this genre.
Related
interview-preparation · hao-hoang · ai-engineering-from-scratch · meta-curriculum · synthesis