SYNTH · a plain-language walkthrough

Should I go out tonight?

That question is a neural network. Not a metaphor for one — the arithmetic is identical to the code in SYNTH, my from-scratch neural network. Four lessons, no jargon until the end of each one.

1_first_neuron.py 2_neuronLayers.py 3_layers.py

Lesson 1 — one decision

You add up what you care about, and see if it clears zero.

Three facts about tonight. Each one pulls you toward going out or away from it, and not by the same amount. Drag them.

Friends already going6

Pulls you out. This one matters most to you: ×0.9

How tired you are4

Pulls you home: ×−0.7

How hard it's raining2

Also pulls you home, but you mind it less: ×−0.4

0.0  

The +1.0 at the end is you on a normal day, before anything happens. A cheerful person carries a bigger one and needs less convincing.

This is 1_first_neuron.py, line for line. The inputs [1, 2, 3] are the three facts, [0.2, 0.8, -0.5] is how much each matters, and bias = 2 is the mood you start the day in.
The book's words: the facts are inputs, the "how much it matters" numbers are weights, the starting mood is the bias, and this whole unit is one neuron. Multiply-everything-and-add is a dot product — that's all np.dot does.

Lesson 2 — a row of them

Now ask three friends the same three facts.

Same facts, but each friend cares about different things. Nobody added new information — they just weigh it differently, so they disagree. Keep dragging the sliders above and watch all three react.

The fourth friend never hears the facts at all

She only hears what the first three said, and weighs them. She trusts Priya, half-ignores Rahul, and actively disagrees with Sam.

That's 2_neuronLayers.py. Its weights1 has three rows because there are three friends; each row is four numbers long because there are four facts. weights2 is the fourth friend listening to opinions instead of facts.
The book's words: one row of friends is a layer. A friend who hears other friends instead of raw facts is a hidden layer. Stacking rows of them is what makes it deep. The .T in the code is just turning the table sideways so the rows and columns line up.

Lesson 3 — the missing piece

A hundred friends are exactly as smart as one — until you let them shut up.

Here's the real test. Grey dots are what actually happened. The network has to draw a line through them. Add more friends, and flip the one rule that changes everything.

How many friends8
what actually happened the network's best attempt where a friend starts speaking

What "staying silent" means

Each friend has a point where they start caring. Below it they say nothing at all — not "no", not a negative number, just zero. Above it, the more you feed them, the louder they get.

That silence is the whole trick. It lets one friend handle only rainy days and another handle only weekends, each keeping quiet outside their own case. Take the silence away and every friend has an opinion about everything, always, in a straight line — and a straight line plus a straight line is still a straight line. That's why fourteen of them just drew you one boring stroke.

self.output = np.maximum(0, inputs)   # negatives become 0, positives pass through
This is the line missing from 3_layers.py — Activation_ReLU.forward is still empty there. It's one line.
The book's words: the shut-up rule is an activation function, and this particular one is ReLU. "A straight line stays a straight line" is the linearity problem; the kink is what makes the network non-linear.

Lesson 4 — the half SYNTH doesn't have

Nobody handed you those numbers. Being wrong is how they get found.

Every "how much it matters" number so far was typed in by hand, or picked by np.random.randn. Real networks start with garbage and fix themselves. Press the button and watch garbage become a curve.

Rounds0
How wrong—

The loop is three steps, forever:

  1. Guess.Run the facts through. Get an answer. This is the only part SYNTH can do today.
  2. Measure the miss.One number for how far the guess sat from the truth. That's the number falling on screen.
  3. Nudge every number a little.For each "how much it matters", ask: would raising this have made the miss smaller? If yes, raise it a hair. Do that for all of them, then guess again.
The book's words: step 2 is the loss, step 3 is gradient descent, and the trick for working out every nudge at once — without testing them one by one — is backpropagation. "A hair" is the learning rate.

What to write next

Four files and SYNTH learns something.

  1. Give it something curved to chew on
    Three hand-typed rows of numbers can't show you anything. Generate a spiral of dots in three colours and try to separate them.
    4_data.py
  2. Turn the last layer's numbers into confidence
    Right now layer 2 spits out things like -0.084. Meaningless. Squash them so they read as "70% sure it's a cat".
    5_softmax.py
  3. Score how wrong it was
    One number. Confident and right scores near zero; confident and wrong scores huge.
    6_loss.py
  4. Nudge everything and repeat
    The backward pass. Longest file by far, and the moment SYNTH stops being a calculator.
    7_backprop.py