What is Parameters?
Also called weights, model size.
Parameters are the numeric values inside a model that are adjusted during training and then held fixed when the model is used. They are where everything the model learned is stored. Model size is usually quoted as a parameter count, such as eight billion or several hundred billion, and that number strongly influences memory requirements and running cost.
Before training begins, parameters hold random values. Each training step compares the model's output to the desired output and shifts every parameter slightly to reduce the gap. After enough steps, the collection of numbers encodes grammar, factual associations, and reasoning patterns in a distributed way, meaning no single parameter corresponds to a single fact that you could locate, inspect, and edit by hand.
Parameter count sets hard practical limits. Weights must be held in memory throughout inference, so a model's size determines what hardware can run it and how many concurrent requests fit on one machine. This is also why quantization matters so much in deployment: storing each parameter in fewer bits shrinks the memory footprint proportionally, usually at a modest accuracy cost.
Bigger is not automatically better, and assuming otherwise is the most frequent mistake. A smaller model trained on more and cleaner data, or refined with stronger post-training, routinely outperforms a larger but older one. Mixture-of-experts designs complicate the comparison further, since they report a large total count while activating only a small fraction of it per token.
Providers of closed models often do not publish parameter counts at all, so widely circulated numbers should be treated as estimates or as simply unavailable. Parameters are also distinct from hyperparameters, which are settings chosen by the people running training, and from sampling settings such as temperature, which are chosen at request time and change nothing inside the model.
Key points
- Learned numbers set during training and frozen during normal use.
- Parameter count drives memory footprint and hardware requirements.
- Knowledge is distributed; no single parameter holds a single fact.
- More parameters does not reliably mean better results.
- Many providers never publish parameter counts for their models.
In practice
Think of a mixing desk with billions of sliders. At the start, every slider sits in a random position and the output is noise. Training plays example after example and nudges each slider a hair toward the setting that makes the output better. Once training ends, the sliders are taped in place, and every answer the model gives comes from that one frozen configuration.