LearnTopic guide

ML Basics

The core machine learning ideas AI Engineers are expected to explain: how models learn, how to measure them honestly, and how they fail.

Topic overview

You do not need the full classical ML curriculum to work as an AI Engineer, but interviewers still ask warm-up questions about how models are trained and judged. The same ideas also underpin how you evaluate LLM systems: honest test data, the right metric, and awareness of failure modes.

Core concepts

Supervised vs unsupervised learning: supervised learning learns from labelled examples (classification and regression). Unsupervised learning finds structure without labels, such as clustering or dimensionality reduction. Self-supervised learning, used to pretrain language models, creates its own labels from the data.

Train, validation, and test sets: train on one split, tune choices such as hyperparameters on a validation split, and report final performance once on a test set the model never influenced. Reusing the test set to make decisions quietly inflates results.

Overfitting and underfitting: an overfit model memorizes the training data and does poorly on new data. An underfit model is too simple to capture the pattern. A large gap between training and validation scores points to overfitting.

Bias and variance: high bias means the model is too simple and misses patterns; high variance means it is too sensitive to the particular training sample. Reducing one often raises the other, which is the bias-variance tradeoff.

Reducing overfitting: get more or better data, simplify the model, add regularization (L1 or L2, dropout), use early stopping, and use cross-validation to get a more reliable estimate.

Cross-validation: split the data into folds, train on all but one, evaluate on the held-out fold, and average the results. It gives a steadier estimate when data is limited.

Classification metrics: accuracy is misleading when classes are imbalanced. Precision asks how many predicted positives were right; recall asks how many real positives were found; F1 balances the two. Choose by the cost of errors: favour recall when missing a case is costly, precision when false alarms are costly.

ROC-AUC and thresholds: many models output a score, and the decision threshold sets the precision and recall tradeoff. ROC-AUC summarizes ranking quality across all thresholds.

Regression metrics: mean absolute error is easy to interpret; mean squared error penalizes large errors more heavily.

Class imbalance: when one class is rare, use better metrics, resample or reweight classes, and set the threshold deliberately instead of trusting accuracy.

Data leakage: information from outside the training setting leaks into training, such as using future data to predict the past, or fitting a scaler on all data before splitting. Scores look excellent and then collapse in production.

Classical models vs deep learning: linear and logistic regression, decision trees, random forests, and gradient boosting are strong, cheap, and interpretable on tabular data. Deep learning and LLMs shine on text, images, and audio. Choosing the simplest tool that works is a good interview answer.

Why it matters

The same discipline carries over to LLM work: you still need a clean test set, a metric that matches the business cost of errors, and a check for leakage. Teams that skip this ship systems that look good in a demo and fail on real data.

Interview relevance

Expect quick conceptual questions: what is overfitting and how do you prevent it, what is the difference between precision and recall and when does each matter, why is accuracy a poor metric on imbalanced data, what is cross-validation, and what is data leakage. Follow-ups often ask you to apply the idea to a concrete case such as fraud detection or a classifier for support tickets.