learn.david-monis.com
This is where I write up lectures on topics I enjoy. Each one starts from the very basics, is full of visuals, and lets you try everything important right in your browser.
A course is a series of lectures that build on each other. Your progress is stored only in your browser.
From a single artificial neuron all the way to GPT-3 with 175 billion parameters. The perceptron, backpropagation, tokens, embeddings, attention and the transformer. Built on four original papers and the 3Blue1Brown videos.
More lectures and courses will be added as I finish them.
I recommend going in order. Each lecture relies on the previous ones.
First half · How a network learns
1The neuron, the 1958 perceptron, the limits of one layer, hidden layers and a network that reads digits.
40 minCompleted 2What a network's error is, what a derivative is, and how to walk downhill to a good solution. With a mini exam.
40 minCompleted 3The algorithm from the Rumelhart, Hinton and Williams paper (1986), step by step on a tiny network.
40 minCompletedSecond half · What is inside an LLM
4The GPT tokenizer, the space of meanings, positions, encoder and decoder, and the way back from numbers to a word.
40 minCompleted 5Query, key, value, attention heads and the whole block from the paper Attention Is All You Need.
45 minCompleted 6The perceptron layers between attention blocks, superposition, and two thirds of all parameters.
35 minCompleted 7Explore the whole network with 175 billion parameters, how information flows through it and how it is trained.
45 minCompleted