Loss Functions: MSE, Cross-Entropy, and Why We Minimize Loss
~12 min read
A loss function is a single number measuring how wrong the network's output was. Training is nothing more than adjusting weights to make that number smaller — MSE for numeric predictions, cross-entropy for classification.
Loss Functions: MSE, Cross-Entropy, and Why We Minimize Loss is a Pro topic
Sign in, then upgrade to Pro or Power to unlock this topic and the full AI Engineering curriculum.
Key points
- •A loss function turns 'how wrong was the prediction' into one number; training is entirely about adjusting weights to make that number smaller
- •MSE (Mean Squared Error) suits numeric predictions: it squares the error so big mistakes are punished disproportionately more than small ones
- •Cross-entropy loss suits classification: it's minimized only when the model puts full confidence on the correct class, and punishes confident wrong answers hard
- •The specific loss function must match the prediction type — MSE for continuous numbers, cross-entropy for categories/classes
- •Loss functions are smooth and differentiable (unlike raw accuracy), which is what lets backpropagation compute a useful direction to adjust weights