From a Single Neuron to a Multi-Layer Perceptron
~13 min read
A neuron just weighs its inputs, adds them up, and squashes the result through an activation function. Stack many neurons into layers and you have a Multi-Layer Perceptron — the basic shape of almost every neural network.
From a Single Neuron to a Multi-Layer Perceptron is a Pro topic
Sign in, then upgrade to Pro or Power to unlock this topic and the full AI Engineering curriculum.
Key points
- •A neuron computes a weighted sum of its inputs plus a bias, then applies a nonlinear activation function
- •Without a nonlinear activation, stacking layers is mathematically pointless — it collapses back to one big linear sum
- •ReLU (max(0, z)) is the most common activation today for its simplicity and training benefits; sigmoid/tanh squash into bounded ranges
- •A Multi-Layer Perceptron stacks neurons into layers (input -> hidden layer(s) -> output), each layer built on the previous layer's outputs
- •More hidden layers let the network combine simple patterns into progressively more complex ones — the intuition behind 'deep' learning