What are Parameters in simple terms?
In simple terms, parameters are the millions of tiny adjustable dials inside an AI model. Training is the process of slowly turning each dial to the right setting, and together those settings are everything the model has learned.
Parameters explained
Parameters are the internal numerical values a model adjusts during training — the learned settings that store what the model has learned and determine the output it produces from a given input.
Inside a trained AI model sit a vast number of internal values that get tuned as it learns — these are its parameters. Think of them as adjustable dials. Before training, the dials are set randomly and the model is useless. Training is, quite literally, the process of nudging every dial a little at a time so that the model's outputs get closer to what they should be, repeated over and over across mountains of examples. When training finishes, the dials are frozen at their final settings, and those settings are the model's knowledge. There's nothing else stored — no database of facts, no lookup table. Everything the model "knows" lives in the particular combination of all those numerical values. Two models with identical designs but different parameter settings will behave completely differently, because the settings are the whole point.
This is why you'll constantly hear models described by their parameter count — "a 7-billion-parameter model," "a 70-billion-parameter model." The number is a rough measure of a model's size and capacity: more parameters mean more dials, and therefore more room to capture complex patterns. As a loose rule, bigger has tended to mean more capable, which is part of what drove the race toward ever-larger models. But it's only a loose rule, and treating parameter count as a pure quality score is a beginner's mistake. A smaller model trained on better data, or refined more skillfully afterward, can outperform a larger one. Parameter count tells you how big the engine is, not how well the car drives.
One useful distinction clears up a frequent muddle. Parameters are the values the model learns by itself during training — nobody sets them by hand; there are far too many. Hyperparameters, despite the similar name, are the handful of settings a human does choose before training begins, like how fast the model should learn or how big it should be. Parameters are the outcome of training; hyperparameters are the choices that shape how training runs. In the most common neural-network models, the bulk of the parameters take the specific form of weights (and related values called biases) — so if you've read about a model's "weights," those are its parameters by another, more specific name.
Real-world example of Parameters
Many model names announce their parameter count outright: Mistral 7B, Llama 3 70B, the B standing for billions. It is tempting to read those numbers as a quality ranking, and a well-known 2022 result shows why that is a mistake. DeepMind trained a model called Chinchilla with 70 billion parameters on about four times as much text as its earlier Gopher model, which had 280 billion, while spending the same total computing budget on each. Chinchilla beat Gopher across a wide range of tests, and also beat the 175-billion-parameter GPT-3. A quarter of the parameters gave better results, because for a fixed computing budget, more training text paid off more than a bigger model. A smaller count also makes a model cheaper to run on every request, which is why developers often pick the smallest model that does the job well.
Frequently asked questions about Parameters
What is the difference between parameters and hyperparameters?
The names are similar but the concepts are opposite in one key way: who sets them. Parameters are learned automatically by the model during training — there are far too many for a human to touch, and they store what the model has learned. Hyperparameters are the few high-level settings a person chooses before training starts, such as how fast the model learns or how large it is. In short, hyperparameters are the dials a human sets to control the training; parameters are the dials the training itself sets inside the model.
How do a model's parameters work?
They start as random numbers and are gradually adjusted throughout training. For each example the model sees, it makes a prediction, the prediction is compared to the correct answer, and every parameter is nudged slightly in the direction that would have made the answer a little better. Repeated across enormous amounts of data, this slowly settles the parameters into values that produce good outputs. After training they're fixed, and from then on the model produces its answers purely by running inputs through those frozen values.
What does a model's parameter count tell you?
Roughly how big and how capacious the model is — more parameters mean more capacity to capture complex patterns, which historically has tended to track with capability. It's useful as a quick size label and a hint at the computing power needed to run the model. But it's a loose signal, not a quality score: a smaller model with better training data or more skillful refinement can beat a larger one. Use parameter count to gauge scale, not to rank how good a model actually is.