Before we stack thousands of them into a network, let's understand the single unit everything is built from — and why it's really just a weighted sum with a twist.
The biological inspiration
A biological neuron receives signals through its dendrites, and if the combined signal is strong enough, it "fires" down its axon to the next neuron. The idea we borrow is simple: combine many inputs, then decide whether to activate.
Think of a hiring decision. Each factor — experience, interview score, referral — carries a different weight. You add them up, and if the total clears some bar, you say yes. An artificial neuron does exactly this, with numbers.
The artificial neuron
Given inputs \(x_1, x_2, \dots, x_n\), a neuron computes a weighted sum plus a bias, then passes it through an activation function \(f\):
$$ y = f\!\left(\sum_{i=1}^{n} w_i x_i + b\right) $$
The weights \(w_i\) say how much each input matters, and the bias \(b\) shifts the threshold. Learning a neuron just means finding good values for \(w\) and \(b\).
Why we need a non-linearity
If \(f\) were just the identity, stacking neurons would collapse into a single linear function — no matter how deep. A non-linear activation (ReLU, sigmoid, tanh) lets a network bend and fold space, so it can represent curves, not just straight lines.
Build one in code
A single neuron with a ReLU activation, in a few lines:
import numpy as np
def neuron(x, w, b):
z = np.dot(w, x) + b # weighted sum + bias
return max(0.0, z) # ReLU activation
x = np.array([0.5, -1.2, 3.0])
w = np.array([0.8, 0.1, -0.5])
b = 0.2
print(neuron(x, w, b)) # -> 0.0 (ReLU clips the negative sum)
Takeaway
Weights and bias make the neuron flexible; the non-linearity makes a stack of them powerful. Next chapter: connecting neurons into layers.
