What is Neural Network?
Also called artificial neural network, deep learning model.
A neural network is a mathematical model made of layers of simple units, each computing a weighted sum of its inputs followed by a nonlinear function. Learning consists of adjusting those weights so that the network's outputs match examples in training data. Networks with many stacked layers are called deep, which is where the term deep learning comes from.
Training uses two repeated steps. A forward pass runs an input through the layers to produce a prediction, and a loss function scores how wrong that prediction was. Backpropagation then computes how much each weight contributed to the error, and an optimizer nudges every weight slightly in the direction that reduces it. Millions of such cycles turn random initial numbers into useful structure.
The units are only loosely inspired by biological neurons, and the analogy is routinely taken too far. A unit performs arithmetic, not chemistry, and the learning rule has no established biological counterpart. What actually explains the results is scale, high-quality data, and gradient-based optimization. The metaphor is useful for building intuition and misleading as an explanation of why these systems work.
Architecture describes how the layers are wired together, and the choice matters enormously for what a network can learn. Convolutional networks suit images, recurrent networks were built for sequences, and transformers now dominate language. All of them share the same learning machinery, so advances in optimization, hardware, and data pipelines tend to benefit every family at once.
Everything visible in a modern language model is a consequence of this substrate. Parameters are the weights, training is the weight-adjusting loop, and inference is a single forward pass with those weights frozen. Quantization is simply the practice of storing the same weights at lower numeric precision to save memory and speed up that forward pass.
Key points
- Layers of weighted sums plus nonlinear functions, trained on examples.
- Backpropagation assigns error blame to each weight; optimizers adjust them.
- Deep simply means many layers stacked on top of each other.
- The brain analogy is loose and explains none of the results.
In practice
Picture a network that decides whether an email is spam. Each word contributes a number, the first layer combines those numbers into coarse signals such as 'contains urgent money language', and later layers combine those signals into a final score. Nobody wrote those intermediate signals by hand. They emerged because the weights were adjusted thousands of times against labeled examples.