🤖 Stuff Every AI Expert Should Know: Training Guide

The AI umbrella and a short history

Artificial intelligence is the broad goal of making machines perform tasks that usually need human intelligence. Machine learning is one part of AI where systems learn from data instead of following only handwritten rules. Deep learning is a subfield of machine learning built around neural networks, which are inspired by biological neurons.

Alan Turing's 1950 paper described a game where an interrogator chats with a person and a machine and tries to tell which is which. He called it the Imitation Game and predicted that by about 2000, judges would identify the machine correctly no more than about 70 percent of the time after five minutes of questions.

The 1956 Dartmouth summer research project is called the birth of AI as a field. John McCarthy organized it, and early researchers believed every feature of intelligence could be described precisely enough for a machine to simulate it.

Early AI was mostly logic-based symbolic AI, a top-down approach that stored knowledge as symbols and logical rules. Later, statistical and neural network methods became common tools in the AI toolbox.

The 1973 Lighthill Report, commissioned by the UK Science Research Council, was deeply skeptical of basic AI research and caused a sharp drop in UK AI research funding.

How machines learn

Supervised learning trains a model on examples paired with correct labels. A spam filter is a classic example: each training email comes labeled spam or not spam.

Unsupervised learning has no answer key. It looks for structure in unlabeled data, such as groups of similar customers. Clustering is the classic unsupervised task.

Reinforcement learning has an agent take actions in an environment and get a numerical reward back. The agent learns a policy that earns the most total reward over time, like points or wins in a game.

Key vocabulary for predictions

Features are the inputs a model looks at. The label is the answer it is learning to predict. In a model that predicts house prices from size and bedrooms, the house price is the label.

Classification picks a category, like cat or dog. Regression predicts a numeric value, like tomorrow's temperature. Despite its name, logistic regression is used for classification because it outputs a probability that is turned into a class.

Training is when the model learns its weights from data. Inference comes after training: the finished model is applied to new examples to produce predictions. Generating a chatbot reply runs inference; it does not retrain the model's weights.

Fitting, overfitting and underfitting

When you train a model, split data into training, validation, and test sets. Train on the training set, compare versions on the validation set, and use the test set only for the final double-check. Repeated reuse of validation and test sets can make a model quietly fit them.

Overfitting means the model matches the training data so closely that it fails on new data. Overly complex models and unrepresentative training data are two common causes.

Underfitting means the model is too simple or undertrained, so it misses the real patterns and scores badly on both training and new data. Fixes include a more complex model, more useful features, or training longer.

Bias comes from a model's built-in assumptions, like forcing a straight line. Variance measures how much the fitted model changes across different training sets. A big gap between low training error and high test error points to high variance; training and test error both high points to high bias.

Regularization and learning

L2 regularization adds a penalty for big weights, pulling them toward zero but not all the way. L1 regularization penalizes absolute weights and can push useless features exactly to zero, acting as simple feature selection.

Early stopping ends training when validation loss starts to increase, even if training loss is still falling. This is a simple form of regularization.

Gradient descent finds the slope of the loss and takes a small step downhill. For linear regression, the loss surface is convex, so gradient descent reaches the global minimum.

The learning rate sets step size. Too small and training crawls; too high and the model overshoots and never converges. Learning rate is a hyperparameter you set before training, such as 0.01.

An epoch is one full pass over the training set. With 1,000 examples and batch size 50, one epoch takes 20 iterations. With batch size 1, stochastic gradient descent, one epoch takes as many iterations as there are examples.

Loss and classification metrics

Mean squared error, MSE, squares each difference between prediction and label, so large errors dominate. Mean absolute error, MAE, averages absolute differences and is less sensitive to outliers.

Accuracy counts all correct calls, but it misleads on rare events. If only 1 percent of cases are positive, always guessing negative scores 99 percent accuracy. Avoid accuracy for imbalanced datasets.

Precision asks: when the model said positive, how often was it right? It is true positives divided by all positive predictions. If a spam filter flags 200 emails and 150 really are spam, precision is 0.75.

Recall asks: of all the real positives, how many did the model catch? It is true positives divided by true positives plus false negatives. Raising a classifier's threshold usually raises precision but lowers recall.

The F1 score combines precision and recall using the harmonic mean. It drags the score down if either one is low, so a model cannot hide a bad recall behind great precision.

A confusion matrix is a table showing counts of right and wrong predictions per class. For yes/no tasks, its four cells are true positives, false positives, true negatives, and false negatives. The area under the ROC curve, AUC, is 0.5 for random guessing and 1.0 for a perfect ranker.

Classic ML models and good judgment

A decision tree is a readable chain of if/then conditions, called a white box model. It can be easily explained, but trees grown too deep overfit easily.

A random forest trains many trees on bootstrap samples, drawn with replacement, and a random subset of features. Averaging many different trees lets their errors partly cancel out. This technique is called bagging.

k-nearest neighbors, k-NN, does not build a model. It stores training data, finds the k closest points, and gives the new point the majority class of those neighbors. It is called non-generalizing because it simply remembers all training data.

k-means clustering needs the number of clusters, k, set before it starts. It places k centroids, assigns each point to the nearest one, moves each centroid to the middle of its points, and repeats. It minimizes inertia.

Prefer a simple rule over machine learning when a simpler solution works. ML creates training, evaluation, deployment, and maintenance costs, so a strong heuristic is also a useful benchmark.

A predictive correlation does not prove causation. Two variables may change together without either causing the other; a confounder, selection process, or coincidence can create the pattern.

From neuron to deep network

A basic artificial neuron multiplies each input by its own weight, sums the products, and adds a bias before an activation function acts. Changing a bias can shift the neuron's pre-activation even when inputs stay fixed.

The input layer receives features and the output layer produces predictions. Any intervening computational layers are called hidden layers.

Nonlinear activation functions are placed between layers because a stack of linear operations would still collapse to one linear operation. Nonlinearities let deeper networks model far more complicated functions.

Two common activations are ReLU, which returns max(0, x), and logistic sigmoid, which maps real inputs into the open interval 0 to 1. ReLU has slope 1 on its positive side and 0 on its negative side; logistic sigmoid equals exactly 0.5 at input zero.

The deep in deep learning refers to the number of layers in the network. IBM describes a network with more than three layers, counting input and output, as deep learning.

Training and the 2012 breakthrough

Backpropagation makes gradient descent practical for multilayer neural networks. It applies the chain rule from the loss backward through the computation graph, and modern autodifferentiation libraries usually perform the backward calculations automatically.

Vanishing gradients happen when deep backpropagation multiplies many small derivative terms. The near-zero gradient gives early layers almost no useful update signal, so those layers barely train.

AlexNet, the 2012 ImageNet breakthrough network, had about 60 million parameters and five convolutional layers. An efficient GPU implementation helped make its training faster. It learned from about 1.3 million ImageNet training images sorted into 1,000 classes.

AlexNet's authors used dropout regularization to reduce overfitting in its fully connected layers. Dropout removes a random selection of units at each training step so the network cannot lean on any one of them.

ImageNet's millions of labeled images are organized by the WordNet hierarchy. Each concept is a synonym set, or synset, and may be described by multiple words. ImageNet indexes web images for WordNet synsets and does not own the image copyrights.

Convolutional and recurrent architectures

In a CNN, a neuron's receptive field is its local input region. A convolutional neuron connects locally across image width and height, and a filter normally spans the full channel depth, such as all three RGB channels.

A convolutional layer shares one learned filter across image locations instead of learning a separate detector at every position. This greatly reduces the parameter count and can detect the same edge in different parts of an image.

A recurrent neural network, RNN, carries information through a sequence using the previous hidden state. At each time step, it combines the current input with its earlier state, and the same transition function with the same parameters is reused at every step.

Backpropagation through time treats a recurrent network as a network unrolled across time. Each time step becomes a copy in the computation graph, and gradients pass backward through the linked copies.

Ordinary RNN time steps are difficult to parallelize within one sequence because steps depend sequentially. A length n sequence needs O(n) sequential operations, while self-attention connects positions with O(1) sequential operations per layer, though total work can grow quadratically.

Generative and modern vision models

A generative adversarial network, GAN, has two competing models: a generator that produces samples and a discriminator that estimates whether a sample came from real training data or from the generator. The generator is trained to increase the discriminator's mistakes.

A denoising diffusion model has two basic processes: noising, which corrupts data toward a simple noise distribution, and learned denoising, which reverses that corruption. Generation begins from noise and follows the learned reverse process toward a sample.

OpenAI's CLIP was trained to predict which caption goes with which image on 400 million internet images. Because it links pictures to free text, you can classify images just by writing the category names. CLIP matched ResNet-50's ImageNet accuracy zero-shot without using any ImageNet training examples.

A Vision Transformer, ViT, cuts an image into a grid of small patches, turns each into a vector, and feeds them in order like tokens to a Transformer. This works well without any convolutions.

Embeddings and the Transformer idea

A learned embedding converts a discrete token ID into a numeric vector. Training can arrange those vectors so useful relationships are reflected geometrically. The original Transformer used token embeddings with the same dimension as its model states.

Word embeddings can solve analogies with vector math. In word2vec, vector(King) minus vector(Man) plus vector(Woman) is closest to the vector for Queen.

Cosine similarity compares the direction of two embeddings and ignores their magnitudes. It divides the dot product by both vector lengths. For unit-length vectors, cosine similarity and dot product are equal.

The 2017 paper Attention Is All You Need introduced the Transformer architecture. It first demonstrated the architecture on machine translation and English constituency parsing.

The original Transformer dispensed with recurrence and convolution. It relied on attention plus feedforward layers, which made training computations across positions much more parallelizable.

How attention works

Self-attention is a form of attention where the same sequence provides all three roles: queries, keys, and values. This lets each token in the sequence gather context from every other token that is not masked. In a Transformer encoder, self-attention can connect each position to all positions in the layer below.

A query is compared with keys to determine relevance weights. Those weights mix the corresponding value vectors into the attention output. Keys decide how much to attend, while values carry the information that gets combined.

Scaled dot-product attention divides query-key dot products by sqrt(d_k). This keeps softmax inputs in a range with more useful gradients, because large dot products can push softmax toward regions with extremely small gradients.

Multiple attention heads each have their own learned projections of queries, keys, and values. Parallel heads can focus on different positions and kinds of relationships, capturing different subspaces. The original base Transformer used eight attention heads with 64-dimensional keys and values.

Position, encoder and decoder

The original architecture has no recurrence or convolution to carry position implicitly, so it adds positional information to token embeddings. The positional and token embeddings share a dimension so they can be added.

The original Transformer constructed fixed positional encodings using sines and cosines. Different embedding dimensions receive sinusoidal signals with different wavelengths. Even dimensions use sine and odd dimensions use cosine.

A Transformer encoder transforms every input position into a contextual representation. A decoder can then use that encoded sequence while generating outputs.

An autoregressive Transformer decoder masks later positions to block future information during next-token prediction. The paper implements illegal attention links by setting their pre-softmax values to negative infinity.

In Transformer cross-attention, the decoder supplies queries, and the encoder output supplies keys and values. This lets every decoder position attend over all encoded input positions.

Bidirectional Transformer variant

BERT's pretraining uses context from both left and right in all layers. This suits understanding tasks like question answering, but it cannot produce free-flowing text generation because it is not left-to-right. BERT stands for Bidirectional Encoder Representations from Transformers.

What LLM training needs

Parameters are the trained weights and biases retained in a model. Tokens are the atomic units the language model reads, predicts, or generates. A model can process many different token sequences using the same fixed set of parameters.

The first phase of LLM training is usually unsupervised learning on raw text. The text itself supplies the answer because the next token is already there. GPT-3's models were all trained for a total of 300 billion tokens.

Neural network layers batch many dot products into matrix multiplications. GPUs and TPUs provide specialized parallel hardware that can accelerate those regular operations efficiently.

Google's first published datacenter TPU centered on a 65,536-unit 8-bit MAC array. The chip's stated peak throughput was 92 tera-operations per second.

Scaling laws for neural language models observed that test loss changed predictably with parameter count, dataset size, and training compute, following power-law trends. The paper says some trends span more than seven orders of magnitude.

The Chinchilla compute-optimal rule says model size and training token count should scale equally. A model twice as large should train on roughly twice as many tokens.

Efficient and stable training

Mixed-precision training combines lower and higher precision. Many tensors can use compact formats such as FP16 to reduce memory and arithmetic cost, while sensitive updates can retain FP32 precision. The original mixed-precision paper reported nearly halving memory use.

Mixed-precision training often keeps an FP32 master copy of weights. Accumulating optimizer updates into FP32 prevents small changes from disappearing through rounding. The FP32 master weights can be rounded back to FP16 for forward and backward passes.

Dropout randomly removes units and their connections during training. Each training step samples a thinned network, which discourages reliance on particular cooperating units.

At inference time, standard dropout uses the full network with appropriate scaling. This approximates averaging the predictions of the many thinned subnetworks seen during training.

Layer normalization computes a mean and variance from the summed inputs across a layer's hidden units for one example. It can work even with a batch size of one.

Layer normalization uses the same computation at training and inference time. It derives statistics from the current example rather than stored batch averages, so there is no separate train-time and test-time formula.

Reusing and aligning models

Transfer learning reuses representations from a previously trained base model. Freezing a layer means its weights are not updated during the new training run.

Fine-tuning updates some or all pretrained weights on new data, usually with a low learning rate. A common workflow first trains a new head with the base frozen, then unfreezes selected upper layers.

Instruction tuning fine-tunes a model on many tasks phrased as instructions. Google's 2021 FLAN paper fine-tuned a 137 billion parameter model on over 60 tasks and beat zero-shot 175B GPT-3 on 20 of the 25 tasks tested.

InstructGPT used reinforcement learning from human feedback, RLHF. Human labelers ranked several model outputs to the same prompt. Those rankings trained a reward model, and reinforcement learning tuned the LM to score well on it. People preferred the 1.3B InstructGPT model to raw 175B GPT-3.

Direct Preference Optimization, DPO, replaces RLHF's reinforcement learning with a simple classification loss. The 2023 paper showed the same goal can be reached without sampling during fine-tuning.

Constitutional AI uses only a written list of rules or principles as human oversight. The model critiques and revises its own answers against those principles, then a model picks the better answer. That AI feedback trains a preference model, called RLAIF.

Making models smaller and cheaper

Knowledge distillation uses a big teacher model with a raised softmax temperature to produce soft targets. The student learns which wrong answers are close, such as a 2 that looks a bit like a 3.

DistilBERT made BERT 40 percent smaller and kept about 97 percent of its language understanding. Its training combined language modeling, distillation, and cosine distance losses.

Quantizing 16-bit weights to 8-bit integers cuts inference memory about in half. The 2022 LLM.int8() method did this for models up to 175 billion parameters without losing performance, keeping a few outlier features in 16-bit.

LoRA freezes pretrained weights and trains small add-on matrices in each layer. For GPT-3 175B, LoRA cut trainable parameters by 10,000 times and GPU memory by 3 times.

QLoRA freezes the base model in 4-bit form and trains only small LoRA adapters on top. This allowed fine-tuning a 65 billion parameter model on a single 48 GB GPU while keeping 16-bit task quality. QLoRA introduced a 4-bit data type called NormalFloat, or NF4.

How LLMs produce text

A large language model is trained to predict the next token from earlier tokens. It is autoregressive: each new token is predicted from the tokens before it and then added to the text, so a whole answer is built one token at a time.

A token is an atomic unit, not always a whole word. A common word may remain one token while a rare word becomes several. Byte pair encoding handles rare or unknown words by splitting them into smaller subword units, and it started as a data compression algorithm.

A context window is a token budget for a single request: your prompt, any documents, and the conversation so far. Anything outside it is not seen by the model for that answer. GPT-3 had a context window of 2048 tokens. A 2023 study found key information is most likely to be missed when buried in the middle of a long prompt, producing a U shaped curve.

Temperature reshapes the probabilities of the next token. High values make output more random; low values make the top choice win almost every time. Nucleus sampling, top-p, keeps the smallest group of tokens whose probabilities add up to p and samples among them, dropping the unlikely tail.

If a model always picks the most likely words, the text tends to be bland and strangely repetitive. Human writing is not always the most probable next word, so some randomness reads more naturally.

A hallucination is confident, plausible-sounding but false output. NIST calls this confabulation, a natural result of how generative models approximate the statistics of their training data.

Prompting techniques

Few-shot prompting puts a few examples in the prompt. The model's weights stay exactly the same, with no gradient updates. A prompt with no examples is zero-shot; with exactly one, it is one-shot.

A system prompt sits above the conversation and gives the model a role and ground rules. Even a single sentence of role can make a difference.

Chain-of-thought prompting has the model write intermediate reasoning steps before the answer. With just eight chain-of-thought examples, a 540 billion parameter model reached state of the art accuracy on GSM8K math problems in 2022. Self-consistency samples many reasoning paths and takes the majority answer, which improved GSM8K accuracy by 17.9 percent over plain chain-of-thought.

Toolformer taught a language model to call tools for arithmetic and factual lookup. With function calling, the model returns a structured request naming a tool and its arguments, and the developer's application runs the tool and sends the result back.

ReAct has a language model alternate reasoning steps with actions, such as a Wikipedia lookup. Fresh evidence from each action cuts down on hallucination.

In Anthropic's definition, an AI agent is an LLM using tools in a loop and deciding its own next steps based on environmental feedback. In a workflow, programmers fix the order of steps in code.

Retrieval, hallucination, and evaluation

Retrieval-augmented generation, RAG, combines parametric memory stored in weights with non-parametric memory from retrieved documents. The retriever fetches relevant passages and the generator writes its answer using them, which makes it easier to show sources and update knowledge.

To search millions of embeddings fast, vector indexes like HNSW usually trade away a guarantee of the exact nearest match. HNSW builds layered graphs of nearby points and hops through them, finding very close matches quickly but not always the exact closest one.

A 2025 paper by OpenAI researchers says hallucinations persist partly because most evaluations reward guessing over admitting uncertainty. Benchmarks graded right or wrong push models to bluff, like a student with no penalty for wrong answers.

Benchmark contamination happens when test questions slip into the training data. The GPT-3 paper admits a filtering bug left some test overlaps in, and the model was too costly to retrain. A fresh held-out test called GSM1k showed some model families dropped by up to 8 percent.

MMLU is a multiple-choice benchmark across 57 subjects, from math to law. When it came out in 2020, most models scored near random chance and the largest GPT-3 was only about 20 points better.

Running and serving LLMs

LLM APIs usually charge per request based on input tokens plus output tokens. Long prompts and long answers both cost more and add latency.

The max_tokens limit is a hard stop, not a request to be brief. If a reply hits the limit, it is cut off, maybe mid-sentence. Ask for short replies in the prompt instead.

Time to first token, TTFT, measures the wait from sending the prompt to the first token of the reply. With streaming, a short TTFT feels fast even if the full answer takes longer.

Speculative decoding speeds up a big model by drafting tokens with a smaller one. The big model checks several guesses at once and keeps only what it would have produced anyway, so the output stays exactly the same.

When serving an LLM, the KV cache stores keys and values for old tokens and grows with each request's length, which limits batch size. vLLM's PagedAttention manages it like virtual memory pages and reported 2 to 4 times higher throughput at the same latency.

Bias, fairness, and oversight

An unrepresentative training sample can create systematic errors. A model learns patterns from the examples it receives, and more data does not fix sampling bias if the added data has the same gaps.

Systemic AI bias can exist beyond the dataset, in organizational practices. Biased goals, workflows, review rules, or deployment choices can cause harm even when a dataset appears balanced.

Every mathematical fairness condition usually cannot be satisfied at once. Except in special cases, a classifier cannot simultaneously satisfy every desirable group fairness constraint, so the choice must fit the real context.

Explainability and privacy

NIST distinguishes explainability from interpretability. Explainability represents mechanisms behind operation or a decision, while interpretability connects an output to meaning in the user's situation.

One AI explanation format should not serve every audience. A developer, auditor, doctor, and affected person may ask different questions. NIST treats meaningfulness as one of four principles for explainable AI.

A membership inference attack tries to learn whether a specific record trained the model. The attacker queries the model and studies its outputs without seeing internal parameters, called black-box access.

Overfitting can increase training-data privacy risk because an overfit model is unusually tailored to its training examples. That behavioral gap can give a membership attacker a stronger signal about which records were used.

Attacks and harms

Specification gaming happens when an AI system is given a misspecified objective. An optimizer follows the objective it is given, so a poor proxy for human intent can lead to a high-scoring but unwanted shortcut.

Reward tampering targets either the reward function or the reward inputs. An agent might alter the rule that assigns reward or manipulate the evidence fed into that rule, raising its score without accomplishing the intended task.

AI red teaming deliberately probes for harmful or unwanted outcomes in controlled exercises. NIST describes expert, general-public, and combined red-team participation.

A direct prompt injection exploits the concatenation of untrusted input with instructions. Malicious user text can compete with application instructions and induce unintended behavior.

An indirect prompt injection hides in retrieved documents or web pages. The attacker plants instructions in content such as a document, email, or webpage, and a retrieval system or agent later ingests it and may follow it.

A jailbreak is a direct prompting attack meant to get around restrictions on what a model will output. Benchmarks such as JailbreakBench and HarmBench measure how easily models can be jailbroken.

One prompt filter cannot fully prevent every injection attack. Filtering, detection, and instruction separation can reduce risk, but systems should also constrain permissions and expose untrusted data through narrow interfaces, a defense in depth approach.

Content provenance and trust

A valid C2PA manifest does not prove that media is true. Content Credentials can bind signed provenance information to an asset, so a valid signature can show who asserted what, but a signer can still make a false claim.

Missing C2PA metadata does not prove that content is fake. C2PA is opt-in, so many legitimate creators never add Content Credentials. Trust decisions should combine provenance with source checking and other evidence.

AI governance and documentation

A model card travels with a trained model and communicates intended use and performance conditions. The original proposal recommends reporting results for relevant intersectional groups. A dataset datasheet documents how and why the dataset was made, like datasheets supplied with electronic components.

Document a dataset's intended use and context because mismatch can hurt real-world performance. Models inherit the conditions represented in training and evaluation data, and a technically accurate benchmark can still be misleading for a different population.

NIST's AI Risk Management Framework has four functions: GOVERN, MAP, MEASURE, and MANAGE. GOVERN spans and enables the other functions by building risk culture, accountability, policies, and responsibilities.

MAP frames risks in their use context. It documents purpose, users, deployment setting, impacts, and risk tolerance. Without that context, the same model behavior can be misjudged as safe or harmful.

MEASURE evaluates risks with evidence, applying quantitative, qualitative, or mixed methods to mapped risks. Testing should occur before deployment and regularly during operation.

The MANAGE function takes the risks that were mapped and measured and decides what to do about them. Teams can respond by reducing the risk, shifting it, avoiding it, accepting it, watching it, planning recovery, or switching the system off. The NIST AI RMF Core is a flexible set of activities, not a single forced sequence; organizations adjust it to their own context and often create profiles for specific uses.

AI law and policy

An open-weight AI model means anyone can download its trained weights to run or fine-tune on their own hardware. That does not guarantee training data or code are public. NTIA's 2024 report said evidence was not sufficient to decide that restricting open-weight models is warranted, and urged monitoring risks.

Under current US guidance, purely AI-generated material is not copyrightable by itself. Protection may cover human-authored expression or creative arrangement in a larger work, and copyrightability depends on human control over expressive elements.

Using AI assistance does not automatically block copyright protection. A tool can assist a human without replacing human creativity. A work can include both copyrightable human material and uncopyrightable AI material. Fair use for AI training is not automatic; some generative training uses qualify and some do not.

The EU AI Act defines four levels of risk, including high risk, transparency risk, and minimal or no risk. Obligations become stricter as the regulated risk category rises.

High-risk AI systems in the EU face risk management, data quality, logging, documentation, human oversight, robustness, cybersecurity, and accuracy requirements. High risk is regulated, not automatically prohibited.

The transparency-risk tier mainly requires disclosure in specified situations, such as when people interact directly with an AI. Deployers must also disclose deepfakes and inform people when emotion recognition or biometric categorisation tools are used. Machine-readable marks enable automated detection but do not prove truth. Minimal-risk systems face no special AI Act rules.

Production monitoring and operations

Production data drift means the live data distribution changes after deployment. A model trained on the old distribution may lose accuracy even though its code and weights stay unchanged. A drift alert is a reason to investigate, not automatic proof that the model is wrong.

To reproduce an ML result, version the full training lineage. A model artifact alone does not reveal how it was made. Reproducibility requires pipeline inputs, environment, and metadata linking them to the resulting model via a model registry and an ML metadata store.

An online A/B test compares a control and a variant under one experiment. Traffic checks and confidence intervals help distinguish an effect from setup bias or random variation. A statistically significant lift can still be a bad launch if safety or latency worsens.

Equal AI training compute can produce different carbon emissions because infrastructure and location matter. The same computation can run on different processors and electricity grids, so compute alone is not carbon. Sparse models can use far less energy than dense models.

Inference can consume more total energy than one training run because repeated inference multiplies small costs. Training may be a large one-time event, while every prediction consumes resources, so at high deployment volume those repeated costs can dominate.