Topic overview
You do not need a math degree to build with LLMs, but four ideas show up constantly: vectors (how meaning is stored), matrices (how models transform it), probability (how outputs are chosen), and gradients (how models learn). Knowing these at an intuitive level is enough to reason about embeddings, RAG retrieval, attention, sampling, and fine-tuning.
Core concepts
Vectors and embeddings: a vector is a list of numbers. An embedding maps text, images, or any item to a vector so that similar items sit close together. Everything in retrieval and semantic search is built on comparing these vectors.
Dot product and cosine similarity: the dot product multiplies two vectors element by element and sums the result. Cosine similarity divides that by the vectors' lengths, so it measures the angle between them and ignores magnitude: 1 means same direction, 0 unrelated, −1 opposite. It is the default way to rank documents in a vector database.
Distance vs similarity: Euclidean (L2) distance measures straight-line gap; cosine measures direction. For normalized embeddings, ranking by dot product, cosine, or L2 gives the same order, which is why many systems normalize once and use the fastest metric.
Matrices: a matrix applies a linear transformation to vectors. A neural-network layer is essentially output = W · input + b. Attention is built from matrix multiplications of Query, Key, and Value, which is why GPUs, which are very good at matrix math, are central to AI.
Probability and softmax: a language model outputs a score for every possible next token. Softmax turns those scores into probabilities that sum to 1. Sampling settings then shape the choice: temperature sharpens (low) or flattens (high) the distribution, while top-k and top-p limit which tokens can be picked.
Averages and spread, in practice: mean and variance appear in evaluation. A single test score can be noisy, so compare models on enough examples and look at variation, not just one number.
Derivatives and gradients: a derivative says how much an output changes when an input changes slightly; a gradient collects those for many inputs. Training measures how wrong the model is (the loss), computes the gradient of the loss with respect to every weight, and nudges the weights in the direction that reduces it. That repeated step is gradient descent.
Learning rate: the size of each nudge. Too large and training overshoots or diverges; too small and it crawls. It is the first thing to check when fine-tuning behaves strangely.
Why it matters
These ideas explain behaviour you will debug in real systems: why a retriever returns near-duplicates, why changing temperature makes answers erratic, why embedding model choice changes search quality, and why a fine-tune with the wrong learning rate ruins a model. Intuition here is more useful than formulas.
Interview relevance
AI Engineer interviews rarely ask for derivations. Expect short conceptual questions: what is cosine similarity and when would you prefer it over L2, what does temperature do, why is softmax used, what is a gradient, and why might training diverge. Be ready to explain each in two or three plain sentences and tie it to a concrete use such as RAG retrieval or sampling.