Neural network
A model inspired by the brain, made of layers of connected 'neurons' that learn from data. The foundation of modern AI — deep learning uses many-layered neural networks. Used for image recognition, language (LLMs), speech, and more. Trained by adjusting weights via backpropagation.
- Image/speech recognition.
- Language models (LLMs).
- Recommendation systems.
- Any pattern-recognition task with enough data.
- Simple rules-based tasks.
- Small datasets (use traditional ML).
- When interpretability is required.
Definition
A neural network has input, hidden, and output layers of neurons. Each neuron computes a weighted sum + activation function. Training adjusts weights via backpropagation (gradient descent) to minimize prediction error. Deep networks have many hidden layers.
Why it exists
To learn complex patterns from data that traditional algorithms can't — recognizing images, understanding language, playing games — by automatically discovering features from examples.
How it works
Data flows forward through layers (each neuron: weighted sum + activation). The output is a prediction. Loss measures error. Backpropagation computes gradients; the optimizer updates weights to reduce loss. Repeat over many epochs. GPUs parallelize the math.
Scaling limits
Needs lots of data and compute (GPUs). Large models (billions of parameters) are expensive to train and run. Overfitting (memorizing training data). Interpretability is hard.
Cost shape
Training: expensive (GPUs, data, time). Inference: cheaper but scales with usage. Cloud GPUs (AWS, GCP) or local.
Alternatives
- Decision trees/Random forest — Interpretable, simpler, less powerful for complex patterns.
- Linear regression — Simple, interpretable, limited to linear relationships.
Failure modes
- Overfitting (memorizing, not generalizing).
- Requires lots of data and compute.
- Black box — hard to interpret.
- Bias from training data.