Harry Potter GPT
A language model built layer by layer from scratch and trained to chat like a Harry Potter fan, taken through the same three stages real chatbots go through: pretraining, fine-tuning, and preference alignment.
Harry Potter GPT started less as "let's build a Harry Potter chatbot" and more as a question I kept getting stuck on: what actually happens, mechanically, between typing a question and getting an answer out of something like ChatGPT? The Harry Potter books were really just the excuse, training data that's fun to sample from while learning how a modern language model actually gets built, stage by stage, starting from a network that knows nothing and ending at something aligned to a preference.
The original idea behind picking Harry Potter specifically was wanting an AI that talks about the books the way a genuinely into-it fan would: not roleplaying as a specific character, but an outside voice that knows the story well enough to ask questions back, notice details you missed, and keep a conversation going instead of just answering and going quiet.
So rather than downloading a pretrained chatbot and prompting it to act like a fan, the whole thing is built from the ground up, following Andrej Karpathy's lectures on how GPT models actually work and forking his nanoGPT repo as the base to build on. It started as a GPT-2 (124M parameter) model trained on all seven Harry Potter books so it would pick up the world, the characters, and the writing style. That gave it the vocabulary of the Potter universe, but not yet the ability to hold a conversation.
From there it went through supervised fine-tuning, showing it thousands of question-and-answer style exchanges so it would learn to respond in a chat format instead of just continuing a story. Then came the last and most interesting stage: preference alignment. I generated pairs of responses to the same question, one that felt like a real fan reply and one that felt flat or robotic, and trained the model to prefer the good one using DPO (Direct Preference Optimization), a simpler cousin of the RLHF process behind tools like ChatGPT.
I'll be upfront about the results. The model's answers are nowhere near as sharp as ChatGPT or Claude, and it sometimes drifts into nonsense mid-sentence, especially the further past the Harry Potter prompt it wanders. That was never really the goal though. The goal was to understand, end to end, what actually happens at each stage: how raw text becomes tokens, how a transformer predicts the next one, how a base model gets fine-tuned into something that follows a format, and how it gets nudged toward answers people actually prefer. Every one of those stages is something I built and trained myself, not something called through an API.
How it works
Stage 1, continued pretraining: forked Andrej Karpathy's nanoGPT (reusing train.py, model.py, sample.py and configurator.py largely as-is) and trained a from-scratch GPT-2 (124M) on the text of all seven Harry Potter books (~5M tokens), on a Kaggle T4 x2, so it absorbed the world, characters, and voice of the series before it knew how to hold any kind of conversation.
Stage 2, supervised fine-tuning (SFT): trained the same model on 4,321 lines of question-and-answer pairs written in a fan-discussion format (`<|user|> question\n<|assistant|> answer<|endoftext|>`), with the loss masked so it only learns from the assistant's tokens, not the question repeated back. This is what taught it to respond conversationally instead of just predicting the next sentence of a story. Loss went from ~3.47 right after switching to the chat format down to ~1.5–1.8 once it had both the HP world and the chat format down.
Stage 3, HuggingFace conversion: converted the raw nanoGPT checkpoint into a HuggingFace-format model so it could be trained with HuggingFace's TRL library. The fiddly part here was weight transposition: nanoGPT's linear layers and HuggingFace's GPT-2 `Conv1D` layers store their weight matrices transposed relative to each other, so a naive copy silently produces a broken model.
Stage 4, preference alignment with DPO: wrote a prompt template describing the fan-voice style rules (references specific scenes, ends with a follow-up question, 2–4 sentences) and used an LLM to generate 347 (prompt, chosen, rejected) triples, a strong fan-like answer paired against a deliberately flat, generic, or factually-off one for the same question. TRL's `DPOTrainer` then trained the SFT model to prefer the chosen response over the rejected one directly, no separate reward model needed, in 3 epochs / 60 steps, which took about a minute on a single RTX 3050 (6GB). That's dramatically cheaper than the pretraining or SFT stages, since DPO is only nudging an existing distribution rather than teaching language from scratch.
What the model actually sounded like at each stage, all taken from the same final run answering the identical questions, not cherry-picked from separate attempts. Right after pretraining, with no chat format to speak of, asked about Harry Potter it just continues a scene: "'Fugitive —' 'That was Lucius Malfoy!' said Harry as they passed in front of a huge black dungeon. 'You know what Dumbledore said to Snape that night?'" Fluent, on-universe, but no notion of question and answer at all.
After SFT, the chat structure and habit of ending on a follow-up question are both there, but the facts are shaky and it visibly confuses characters. Asked who Harry Potter is: "Umbridge is the head of the Ministry's 'secret' school and her role is to train kids to be good at magic and not to be arrogant... I missed that she is the one who 'saved' Harry from the Horcrux while she was teaching her children. Do you think she was right to act so kindly toward Harry during that time?" It answers about Umbridge instead of Harry.
After DPO, that same question gets: "I've never met him, but he's definitely the one I want to hang out with. I didn't realize how much he changes my life as a kid until a book later. Do you think he ever looked back at how you felt about him at that point?" Vaguer on plot specifics, but a genuinely more natural, conversational reply that still ends on the same engagement-seeking follow-up. Asked "Is Dumbledore a good person?", SFT rambles into a fact it half-remembers ("I didn't realize until a reread that his deathbed is the only one in the castle that does not contain a Horcrux"), while DPO stays in voice and in character: "He is genuinely kind, especially when it comes to people like Harry. It's interesting that he is a total jerk to people he disagrees with like Quirrell and Hermione, but at the same time, he is also the man who was willing to protect Harry. Do you think he would have been a better teacher if he'd grown up with Ron instead of just watching him with a stack of cards?" And on a genuinely advanced fan-theory prompt, "I have a theory that Dumbledore is actually Death from the Three Brothers tale", DPO answers: "It's not even real, the way he's described is basically his 'wand-army' and his inability to keep his cool in the Deathly Hallows... Do you think the 'end of the world' storyline is the best one for him?" Still confidently wrong about plot details, but unmistakably in voice.
That trade-off, less overconfident wrongness traded for a more convincing voice, is exactly what DPO is supposed to optimize for here. It was trained on which tone to prefer, not on which facts are true.
A handful of Andrej Karpathy's Zero to Hero videos did most of the heavy lifting conceptually. "Let's build GPT: from scratch, in code, spelled out" is where the actual transformer architecture in model.py comes from. "Let's build the GPT Tokenizer" is what the BPE tokenization here is based on. "Let's reproduce GPT-2 (124M)" matches the exact model size and training regime used in this project. And "State of GPT" is the talk that laid out the pretrain → SFT → RLHF pipeline this whole project follows stage by stage.
Where it stands: the full four-stage pipeline above (pretrain → SFT → HF conversion → DPO) is done and working end to end on GPT-2 (124M), with checkpoints for every stage. A bigger version, swapping in a larger open base model with LoRA fine-tuning plus long-term conversation memory and a retrieval layer for book lore (shown in the architecture diagrams below), was the original stretch plan, but I decided to stay focused on doing the GPT-2 pipeline properly rather than scaling up.
One more piece of reference material worth naming: alongside nanoGPT, I also went through Karpathy's build-nanogpt (his "Let's reproduce GPT-2" repo, which trains a GPT-2 completely from random initialization on the FineWeb dataset) to see the from-scratch side of the picture. This project's own model wasn't trained from scratch, though: it started from pretrained GPT-2 weights and was fine-tuned from there, which is the more honest way to describe it.
Technical Breakdown
The pipeline was designed before it was built: the diagrams below are the actual planning artifacts I worked from, not after-the-fact illustrations. The dashed "checkpoint" lines mark exactly how far the real, working code got.
The full companion architecture (as designed)
The end-to-end vision: a base model taken through fine-tuning and DPO alignment, then wrapped with long-term user memory and a retrieval layer over the books, serving a chat interface. The GPT-2 pipeline that actually got built and trained covers the top half of this diagram (base model through DPO alignment); the memory and retrieval layers at the bottom were part of the original stretch goal, scoped out in favor of finishing the GPT-2 pipeline properly.
Three-phase build plan, step by step
Phase 1 (steps 1–3) and Phase 2 (steps 4–6) are the parts that are actually done: nanoGPT fine-tuned on the HP books, then DPO-aligned via TRL, exactly as described above. The "checkpoint: working GPT-2 companion" line marks where the real, tested code stops. Phase 3 (steps 7–11, re-running the same recipe on a much larger model) was the original stretch goal but was deliberately dropped in favor of doing the GPT-2 version properly.
The actual dev loop
How the code itself got written: fork nanoGPT, write and sanity-check data prep / config changes locally on a laptop with no GPU needed, verify on a tiny slice of data (a few forward passes, loss should visibly drop within 10 steps), then push to GitHub and pull it into Kaggle/Colab only once it's known to work, so GPU time isn't spent debugging.