Build a Neural Network From Scratch: Training, Fine-Tuning, and RL (Part 2 of 2)

Two pre-run Colab notebooks accompany the session, and both run end to end in about half an hour on the free tier — copy them to your Drive and experiment. Notebook 1 covers Word2Vec embeddings and a sentiment classifier; Notebook 2 is a neural-network image classifier in PyTorch.

Why look under the hood

Two weeks ago, in Part 1, the series covered what machine learning even is; this session builds on that with the models actually enabling what we call AI: neural networks. The power of the whole field comes from taking unstructured natural language and representing it with math, which is what lets statistical techniques apply at all. The goal isn’t to become an ML expert in an hour — it’s an appreciation of how the pieces depend on each other and an intuition for the numbers behind the scenes, because that intuition demystifies the tools and lets you build more reliable systems. To do software engineering well you should know what your tools do; a superstitious engineer is a poor one.

Embeddings: words as points in semantic space

An embedding (or vector) is just a fixed-length list of numbers — 512 floats, say — that represents where a word sits in semantic space. Word2Vec is the family of models Aaron uses to build them (there are sentence-to-vec and other “to-vec” variants). One of the simpler algorithms is the skip-gram: for each word, look at its neighbors within a window of plus or minus two, and treat co-occurrence as relatedness. Do that across a huge, diverse, balanced corpus and you get a statistical measure of how likely two words are to appear near each other — which should sound familiar, because predicting the next most likely token is exactly what language models do. Production embeddings like OpenAI’s aren’t just continuous skip-grams, but the results can be treated the same way; this is the baseline for building intuition.

Clean, split, tokenize

The data is IMDb reviews with a sentiment label (0 or 1). A frontier LLM handles messy text for you, but for these techniques you clean first: lowercase everything, and optionally strip stop words like “the” that carry grammar but not meaning. Then split into train and test — 70/30 here, balanced by label so both sets have equal representation of likes and dislikes — because training data trains the algorithm and held-out test data tells you how well it did. Tokenizing then turns each word into a number, and it’s important not to confuse this with the embedding: in the simplest scheme you just enumerate the vocabulary, alphabetically or by frequency. Those numbers mean nothing yet. There’s no rule that color words get 100–200; they’re just handles so the algorithm has numeric input. The embedding is what makes “red” and “ruby” land near each other. Finally the reviews are exploded into sentences — about 188,000 of them — as the unit of observation.

Where the work is (and isn't)

Training the Word2Vec model is importing a Python class, running it, and waiting. Unless you’re doing research, you don’t write algorithms from scratch — you take the cutting-edge thing someone built and use it. The human work is in the data: collecting, curating, cleaning, labeling, checking. This dataset came pre-collected and pre-labeled, which is easier than the real world. And a good model never makes up for bad data; if anything a good model fits bad data more faithfully and learns the bad stuff.

Hyperparameters: the knobs on the outside

Parameters are the numbers inside the model that the algorithm learns — the ten billion inscrutable weights people brag about. Hyperparameters are the far fewer knobs and levers on the outside that let the person training the model configure the process and the model’s characteristics: here, vector size 100 (each word becomes 100 numbers), window (the skip-gram neighborhood), a seed for reproducibility, and a minimum count of five occurrences for a word to count. Is 100 the best vector size? Aaron honestly doesn’t know from this notebook — it’s a reasonable choice that produces a working model. The only way to know the best hyperparameters for a given data-problem-model combination is a hyperparameter search: train similar models over and over with different settings and compare. Much of an ML engineer’s job is exactly that.

Similarity, cosine, and trigonometry flashbacks

Once trained, the model returns the most similar words in the corpus: “enjoyable” is closest to “entertaining” at 88.83%, then “watchable,” “uplifting,” “rewarding,” “well made,” “addictive,” “engrossing” — movie-review language, as you’d expect. Humans can’t think in 100 dimensions, so picture two: every vector is a point, and similarity is how close two points are. A common metric is cosine similarity — take the angle between the two vectors from the origin, take its cosine, and you get a 0-to-1 score: identical direction is 1, opposite is 0. It works just as well in 100 dimensions as in 2, and it handles sparse, high-dimensional data, which is why it’s the default. Dot products and other similarity metrics exist too.

What embeddings enable — and what they encode

Semantic space already powers useful things — recommender systems, a thesaurus if you wanted one — and because the vectors carry meaning, they’re a preprocessing step for downstream models: embed the words in a review, pair with the label, and train a model to predict sentiment from meaning, even without a neural network. (Spotify-style recommenders use an ensemble — collaborative filtering, the “people who liked this also liked” signal — but embeddings can be one recommender in the mix.) You can also combine similarity and dissimilarity by averaging distances and flipping a sign: similar to “corny” but dissimilar from “dumb” gives you corny-but-not-dumb. Similar to doctor and woman, dissimilar from man, puts “nurse” third; doctor and man minus woman gives “Pope.” These are just correlations in how language gets used, but they show sexism and other sociological artifacts are encoded in the corpus. Aaron’s point: people who build tools that others use with the defaults have a disproportionate impact on society. He drives his car with the defaults and trusts the mechanics to be ethical; in technology, we are the mechanics.

Training a sentiment classifier

For now, think of a neural network as y = mx + b with a whole bunch of m’s and x’s and sophisticated linear algebra combining them. Even that is one neuron; a network is many neurons per layer and many layers. Go all the way down to a single weight and it’s essentially a slope multiplied through your data. Transformers and today’s popular architectures aren’t popular because they’re theoretically the truest representation — recurrent or symbolic models might be better in theory — but because they’re easy to compute on and scale horizontally, which is why a free Colab can train on hundreds of thousands of sentences. Training a small model gives you an appreciation for how the big ones are trained. Fit to the embeddings, the classifier predicts positive or negative sentiment on the held-out test set almost 85% of the time — the kind of model major corporations are running right now on scraped social posts and feedback forms.

Reading the confusion matrix: which error is worse?

Accuracy hides two kinds of mistakes. A false positive says they liked the movie when they didn’t; a false negative says they didn’t when they did. This model’s errors are roughly balanced, which is fine for sentiment. But swap in image embeddings of clinical scans predicting a medical condition, and the asymmetry matters: a false positive means a scary result and a higher-fidelity follow-up test; a false negative means the patient goes home untreated. Medicine — practitioners and models alike — errs toward false positives. You can set a threshold and optimize which way to lean, because there’s an inherent trade-off: a model that always says “yes” has zero false negatives without any machine learning at all, and also zero true negatives and no information. Neither extreme is a real model, but they’re the baselines that illustrate the trade-off.

Anatomy of a neural network: MNIST

The second notebook opens with a visualization of a digit classifier on MNIST: a 28×28 grayscale image is 784 numbers from 0 (black) to 1 (white). Two layers you must understand: the input layer, whose size is fixed by the data — exactly 784 neurons, not 783 or 785 — and the output layer, fixed by the output, here 10 neurons for digits 0–9. The layers in between, the “deep” part, aren’t constrained by input or output shape at all; there are best practices (you’ll see a lot of powers of two) but the count of layers and neurons is a design choice, and explaining what any individual number in there means is still open research. It’s fully connected — every neuron passes its result to every neuron in the next layer — hence the compute. Digit recognition is a subset of OCR; the US Postal Service uses it to route mail, and because it’s ML it has a confidence measure, so high-confidence labels route automatically and low-confidence ones go to a human.

How training works: loss, backpropagation, learning rate

A brand-new network’s weights are randomly initialized — it’s just guessing — and that turns out to be a good starting point because it converges with an appropriate loss function and good labeled data. Feed it a 3 and the output lights up the 8 and 7 about as much as the 3. What you want is all the probability in the 3 neuron. The numerical difference between the label vector and the actual output is the loss (or cost); squared error is one way to compute it, squared so positive and negative misses add up in the same direction, because you only care how far off you are, not which way. Backpropagation pushes that loss back through the network and yields gradients — vectors saying how much to change each parameter to make the output look more like the label. Apply them, then repeat: forward propagation for inference, backward propagation for the update. The learning rate is the hyperparameter that scales each step, and it matters because too high and training never converges — you keep overshooting the optimum. On a golf green you use a putter, not a driver.

Building it in PyTorch

PyTorch is Facebook’s open-source library for this; tensors are matrices of arbitrary dimension, and they hold all the weights across all the neurons and layers — the billions of numbers in a big network. The notebook uses a teaching dataset of ten image classes (not the overused digits): airplane, frog, deer, car, cat, ship, and so on. You’ve likely labeled data like this every time you solved a “which of these has a traffic light” CAPTCHA, and training small models that could run in a car is real work. Color images mean three channels per pixel, so the input layer takes RGB. The middle layers are, again, the hard-to-explain magic: the working theory is that all those parameters are doing feature engineering — pooling and summarizing the input into a space where the output layer can easily say “that’s a dog.” The pieces you need to know: cross-entropy loss (how different the predicted class probabilities are from the true class), stochastic gradient descent (the propagation technique), an optimizer, and multiple epochs — passing over all the data, updating weights, then passing over it again. Aaron’s advice for the primitives he didn’t walk through: work through each line with an LLM and the PyTorch docs, and sometimes just read the documentation yourself.

Reading the loss curve and checking against baseline

Loss starts high, drops fast, then flattens — the classic shape. Flat means the model is as good as it’s going to get with this configuration; more training doesn’t move the weights. This curve was still edging down at the end, so longer training or different hyperparameters might improve it. On the test images the model predicts cat, ship, ship, ship — the last was actually an airplane that looks like the Loch Ness monster, an unusual image, and the classifier got it wrong. Overall accuracy is 55%. For ten balanced classes random guessing is 10%, so 55% is a good bit better; the same 55% on the binary sentiment task, where baseline is 50%, would be pretty bad. Always read accuracy against baseline. The notebook ends with exercises — see if you can improve the model.

What's happening at Gauntlet

The latest cohort started this week; the next one is planned for early January, with an exact date to be announced. Gauntlet also just launched Gauntlet Scouts, a referral program you don’t need to be a Gauntlet student to join: refer someone who’d be a fit, and if they apply, get in, and go through the program, you get paid. Terms and details are on the program page.