AI Engineering & Prompt Engineering Interview Questions

Chain-of-thought (CoT) prompting works by making the model externalize its reasoning tokens before committing to an answer, which gives it "scratch-pad" space to decompose problems that require multiple inference steps — math, logic puzzles, multi-hop questions (b). It shows the largest gains on tasks where a wrong intermediate step would otherwise cascade silently into a wrong final answer. On pure fact recall (a) the answer either lives in the model's weights or it doesn't; CoT adds tokens without changing what is remembered. Creative naming (c) does not have intermediate logical steps to unroll. Speech transcription (d) is not a text-in/text-out reasoning task at all.

The system prompt is a privileged, persistent context that the developer controls: it sets the model's role, tone, hard constraints, tool definitions, and any background context that should frame every user message in the session (b). The user turn carries what the end user typed — the per-request input. The model sees both; the system prompt is not hidden (a). They are semantically distinct: system content is treated as developer intent and carries higher implicit trust in most architectures (c is wrong). Ordering is fixed — system precedes user in the token stream; it does not override by being processed last (d).

Multiple studies on large-language-model in-context learning have found a recency (or end-of-context) bias: examples positioned closest to the query tend to have an outsized influence on the output's label or style (b). This is why the order of few-shot examples matters and why practitioners often put the most representative or desired-format example last. Primacy bias (a) describes an effect in multi-choice settings where the first listed option gets over-selected — a different phenomenon, not specific to few-shot example ordering. The "middle" of a long context is often where models attend least (sometimes called the lost-in-the-middle effect), making (c) the opposite of the documented finding. Positional invariance (d) is empirically false for current models.

temperature scales the logits before sampling; at 0 the model becomes (near) greedy — it picks the highest-probability token at each step — which is what you want for deterministic, reproducible extraction. Note the honest caveat: even at temperature 0 you can still see small variation in practice from non-deterministic floating-point reduction order on GPUs, batching, and model updates, so treat it as low variance, not a hard guarantee. Raising temperature (a) does the opposite — it flattens the distribution and increases randomness. Raising top_p (c) widens the nucleus of candidate tokens, again adding variability rather than removing it. frequency_penalty (d) only discourages token repetition; it changes what is generated but does nothing for determinism.

Cosine similarity is the cosine of the angle between two vectors: it is the dot product divided by the product of the magnitudes, so it depends only on direction, not length. That is why it works well for embeddings — two passages about the same topic point the same way regardless of how long each text is. Euclidean distance (a) is a different metric that is sensitive to magnitude (though on length-normalized vectors the two ranking orders coincide). "Shared dimensions" (c) is not a real similarity measure for dense embeddings, whose dimensions are not independently interpretable. Token overlap (d) describes lexical/keyword matching (e.g. BM25), which is exactly what dense embeddings are meant to go beyond.

RAG is the right default when knowledge is large, changes frequently, and answers need attribution: you index the documents, retrieve the relevant chunks per query, and the model answers from them — so updates are just re-indexing, and you can cite the chunks you retrieved. Nightly fine-tuning (a) is slow, expensive, hard to attribute (the model can't reliably cite which training example produced an answer), and it bakes facts into weights where they go stale between runs. Stuffing the whole KB into the system prompt (c) blows the context window and cost, and degrades quality as irrelevant text dilutes the signal. A one-time LoRA on a snapshot (d) is immediately stale for daily-changing data and still can't cite sources. Fine-tuning earns its place for behavior/format/tone, not volatile facts.

Provider structured-output features (e.g. JSON Schema-constrained decoding) restrict the token sampler at each step to tokens that keep the output valid against the schema, so the result is guaranteed-parseable and schema-conformant by construction. A free-text instruction (a) is best-effort: the model can still emit prose, trailing commas, or extra keys. Temperature 0 (c) only reduces variability — a deterministic output can be deterministically malformed. Retry-on-error (d) is a reasonable fallback and improves reliability, but it adds latency and cost and still has no hard guarantee on any single attempt; constrained decoding removes the failure mode at the source.

Function/tool calling is a protocol, not remote execution: given your tool definitions (name, description, JSON-Schema parameters), the model decides whether to call one and emits a structured tool-call with arguments. Your code executes the tool, then sends the result back as a tool/function message so the model can use it in its next turn. The provider does not run your code (a) — it has no access to your database or APIs. The model returns structured arguments, not executable code to eval (c); eval-ing model output would be a serious injection risk. And it is a concrete, parseable call, not a vague wish (d).

Effective defenses are layered and architectural. Least privilege plus human-in-the-loop for dangerous actions (a) limits blast radius even when an injection succeeds. Delimiting untrusted text and labeling it as data (b) helps the model resist hijacking — it is necessary but not sufficient on its own. Validating/sandboxing what tools receive and do (d) stops a hijacked call from causing real damage. Raising temperature (c) does nothing for safety; it just adds randomness and can make the system less reliable. "The system prompt always wins" (e) is false — there is no hard precedence boundary in the token stream, and crafted injections routinely override prepended instructions, which is exactly why you cannot rely on prompt ordering alone.

Retrieval quality is about getting the right chunks into context. A reranker (a) reorders the initial candidate set with a more expensive, more accurate cross-encoder so the best passages float to the top. Chunking strategy (b) directly affects whether a chunk contains a complete, embeddable idea rather than a fragment. Hybrid retrieval (c) catches cases where exact terms/IDs matter that dense vectors miss, and vice versa. A better-matched embedding model (e) improves the similarity signal at the source. Raising max_tokens (d) only changes how long the answer may be — it does nothing about which documents were retrieved, so it cannot fix retrieving the wrong material.

Streaming (a) doesn't change total compute but dramatically improves perceived speed by showing the first tokens immediately. Model routing / cascading (b) sends the bulk of easy traffic to a cheaper model, cutting average cost and latency while preserving quality on the hard tail. Caching (c) avoids paying for work you've already done. Trimming context and bounding output length (e) reduces both input and output tokens, which is where the bill and the time go. Indiscriminately padding every prompt with more few-shot examples (d) does the opposite — it inflates input tokens (cost and latency) on every call, and beyond a point adds no accuracy, so it is a regression, not an optimization.

Hallucination drops when the model is anchored to real evidence: retrieval-grounding with an instruction to answer only from context (a) constrains it to supported claims. An explicit "I don't know" escape hatch (b) gives the model a correct option other than fabricating, which it otherwise tends to avoid. Citation + verification (c) turns grounding into something you can check, catching claims that aren't actually supported. Raising temperature (d) increases randomness and makes confident fabrication more likely, not less. A bare instruction to "never hallucinate" (e) is wishful — the model has no reliable internal signal of its own factuality, so an unenforceable command doesn't change behavior in a measurable way; grounding and verification do.

MCP is an open, client-server protocol that gives a uniform way to expose tools, resources, and prompts to LLM hosts (a), and its central payoff is write-once/reuse-everywhere interoperability across compatible clients (b). It does not remove the security burden: a server runs with whatever access you give it and the data it returns enters the model's context, so an untrusted or compromised server is a genuine injection/exfiltration risk (c) — least privilege and review still apply. MCP does not replace function calling (d); it is the transport/standard through which tool definitions and calls flow — the model still decides which tool to invoke. And it offers no guarantee against malicious server output misleading the model (e); content returned over MCP is untrusted data like any other.

The context window bounds the total tokens the model attends to: system prompt + retrieved context + conversation history + the tokens it generates. If you stuff the input close to the limit, you starve the output and risk truncated completions. Budget explicitly — reserve headroom for max_tokens of output, and remember that long outputs cost both latency and window space.

Prompt caching keys on a prefix of the request, so a long, unchanging prefix (system instructions, few-shot examples, a fixed knowledge block) can be cached once and reused — cache reads are billed at a steep discount and skip recomputation, lowering latency. The catch: ordering matters. Anything that varies must come after the stable prefix, otherwise it breaks the cacheable prefix and you pay full price every call. Structure prompts stable-first, variable-last to maximize hits.