AI Full-Stack quick reference

AI ⊃ ML ⊃ deep learning ⊃ generative AI

The nesting of AI's core terms, plus foundation and discriminative models.

supervised · unsupervised · semi-supervised · self-supervised · reinforcement

Five learning paradigms, distinguished by where the training signal comes from.

train → fit · validation → tune · test → score once · inference → predict on new data

The four data roles across a model’s life, and where each one gets used.

feature = input · label/target = answer · parameter = learned · hyperparameter = chosen

The four terms that describe one training example and one model configuration.

low train error + high val error = overfit · high error on both = underfit

How to read a train/validation error gap, plus the two failure modes that fake a good score: leakage and distribution shift.

number → regression · label → classification · groups → clustering · order → ranking

Six ML task types, named by what the output looks like.

online = one request, real time · batch = many rows, scheduled

The latency-vs-throughput split between serving modes, and its data-side counterpart.

A @ B = matrix multiply · ||v|| = length · A.T = transpose

The linear algebra objects and operations every ML library shape refers to.

Bayes: P(A|B) = P(B|A)·P(A) / P(B)

The probability vocabulary behind classifier confidence scores, naive Bayes, and calibration.

mean vs median vs p95 · confidence interval = range, not a point

The statistics vocabulary behind reporting a model metric honestly.

∇f = gradient · chain rule multiplies derivatives through layers

The calculus vocabulary behind gradient descent and backpropagation.

param -= learning_rate * gradient · Adam = momentum + adaptive rate

The gradient descent update rule and how SGD, momentum and Adam extend it.

yield = stream · with = guaranteed cleanup · @decorator = wrap · type hints = catch early

The Python features that show up constantly across ML codebases.

vectorized op > Python loop · basic slice = view · fancy index = copy

Why NumPy is fast, and the operations that silently break that speed (or that free-view guarantee).

merge = join · groupby = per-category aggregate · isna/fillna/dropna = missing values

The pandas operations that turn raw tabular files into feature-ready data.

notebook = explore · environment = pin dependencies · reproducibility = code + deps + data + seed

What notebooks and environments are each good for, and the four ingredients of a reproducible run.

many iterations + numeric body = vectorize · profile before optimizing

When a Python loop is actually the bottleneck, and what to reach for instead.

collect → label → clean → normalize → dedup → verify

The stages raw data passes through before it is ready to become features.

structured = schema already exists · unstructured = a model must extract meaning first

What separates rows-and-columns data from text/image/audio/video, and what bridges the gap.

stratify by class · group/time-split when rows are dependent

How to split a dataset so each split stays representative and independent.

imbalance → precision/recall, not accuracy · outlier = signal or error, investigate before fixing

Four data quality problems and why each needs its own fix, not one generic technique.

fit preprocessing on train only · ask: would this exist at real prediction time?

The three kinds of leakage and the single question that catches most of them.

snapshot the dataset, name it, never mutate it in place

Why a live, changing dataset makes model training unreproducible without an immutable snapshot.

provenance = origin · lineage = the full transformation chain since

The two questions "where did this come from" and "what happened to it since," and why both matter.

numerical, categorical, text, time-series, interaction — each needs its own prep step

The five feature types and what has to happen to each before a model can use it.

normalize → [0,1] · standardize → mean 0, std 1 · fit on train only

The two common scaling methods, when each is used, and the leakage rule that governs fitting them.

one-hot = safe, wide · ordinal = compact, order matters · target = powerful, leakage-prone · embedding = learned

Four categorical encoding methods and the trade-off — width, order, leakage — each one makes.

selection = keep a subset, interpretable · reduction = combine into fewer features

The two ways to shrink a feature set, and the interpretability trade-off between them.

dominant feature → check for leakage · Pipeline enforces fit-on-train-only

Reading feature importance as a diagnostic, and using a pipeline object to make leakage structurally harder.

tabular + domain knowledge → manual features · raw/unstructured → representation learning

When to hand-design features versus let a model learn them, and how transfer learning bridges the two.

linear = interpretable, assumes linearity · trees/forests/boosting = nonlinear, no scaling · k-NN = no training, slow at scale

The six core supervised models, compared on nonlinearity, scaling and interpretability.

SVM = widest margin boundary · Naive Bayes = Bayes + independence assumption

Two classic classifiers, and why the "naive" independence assumption barely hurts in practice.

k-means = choose k, round clusters · hierarchical = tree, cut anywhere · density-based = no k, handles outliers

Three clustering approaches and what each requires you to know in advance.

PCA: fewer components, ordered by captured variance, not individually interpretable

How PCA compresses correlated features, and the interpretability it trades away for compactness.

collaborative = cross-user patterns · content-based = item similarity · blend both in production

How ranking differs from classification/regression, and the two core recommendation signals.

interpretability · training cost · prediction cost · data needed — weigh all four before accuracy

The trade-off framework for choosing a model, beyond raw accuracy alone.

MAE = equal weight · MSE/RMSE = punishes large errors · R² = vs. mean baseline

Four ways to score a regression model, and when large errors should count more than small ones.

precision = correct-when-flagged · recall = caught-of-real · PR-AUC > ROC-AUC under imbalance

Which classification metric answers which question, and why accuracy and ROC-AUC mislead under imbalance.

confusion matrix = TP/FP/TN/FN · threshold trades recall against precision

The raw counts behind every classification metric, and why the decision threshold is a business choice.

calibrated = confidence matches reality · separate from ranking quality (AUC)

Why a model can rank well but still report untrustworthy probabilities, and how to fix that.

MRR = first hit · MAP = all hits ranked high · NDCG = graded relevance, position-discounted

Three ranking metrics, and which shape of task each one fits.

model metric = fit to the proxy · business metric = did it actually help

Why a model can improve on its own metric while the thing the business cares about gets worse.

offline = fast, historical, filters candidates · online = real traffic, decides

Why offline evaluation and online evaluation are sequential steps, not interchangeable alternatives.

random split + p-value + confidence interval, decided in advance

What makes an A/B test conclusion trustworthy, and the early-stopping trap that undermines it.

data → preprocess → train → validate → evaluate → package → deploy → monitor

The eight stages a model passes through from raw data to a watched, live system.

fit once on train, reuse the fitted object everywhere — not the code

Why a pipeline object prevents both leakage and training/serving skew, structurally.

one shared feature implementation, called by both training and serving

Why training/serving skew is silent and dangerous, and the architectural fix that prevents it.

model version + data version + hyperparameters + metrics, logged automatically

The three versioning/tracking pieces that together make a result reproducible and explainable.

seed every library + same data version + same software versions = reproducible

What a random seed actually guarantees, and the other ingredients reproducibility still needs.

neuron = weighted sum + nonlinearity · layer = neurons in parallel · network = layers in sequence

How individual neurons compose into layers and networks, and why the nonlinearity is load-bearing.

forward → loss → backward (chain rule) → optimizer.step()

The four-step training cycle repeated every batch, and what each step computes.

ReLU/GELU = hidden layers · sigmoid = one probability · softmax = a distribution

Five common activation functions and where each one belongs in a network.

regression → MSE/MAE · classification → cross-entropy · generative → task-specific

Which loss function matches which task, and why cross-entropy pushes toward calibrated confidence.

epoch = dataset / batch_size steps · gradient accumulation = larger effective batch, less memory

How batch size, steps and epochs relate arithmetically, and what gradient accumulation trades off.

initialization = stable start · batch norm = stable throughout · dropout/L2 = fight overfitting

Four techniques for stable, well-generalizing training, and which problem each one targets.

vanishing = gradient → 0, early layers stall · exploding = gradient → ∞, loss spikes/NaN

Why repeated multiplication through many layers can shrink or explode gradients, and the standard fixes.

tensor = device-aware, gradient-tracked array · Module = architecture · DataLoader = batching

The five core PyTorch building blocks every training script is written in terms of.

model.train()/eval() + torch.no_grad() in validation + save the best checkpoint

The full training/validation loop assembly, and the three bugs most commonly missing from it.

mixed precision = ~half the memory, often faster · gradient accumulation = fallback when it still does not fit

Why GPU memory is usually the real limit on training scale, and the two standard levers for it.

save state_dict, not the whole model · re-create architecture in code before loading

Why PyTorch recommends saving weights separately from architecture, and what a resume checkpoint needs.

data parallelism = same model, split data, average gradients · model parallelism = split the model itself

The two core distributed training strategies, and which bottleneck — speed or memory — each one solves.

read the exact shapes/devices/dtypes in the error, trace backward to the divergence

The three most common PyTorch runtime errors, and the debugging approach that fits all three.