GPT-2 Game of Thrones Q&A, a case study by Karam Abu Mokh

Ask it something

The model runs on a public Hugging Face Space. Ask about Westeros: it searches the books, reads the best passages, then answers, and the passages are listed under the answer.

Open the full demo

My role: Solo, end to end
When: July to August 2025, retrained and given a book search October 2026
Built with: Python
PyTorch
tiktoken
OpenAI API
Google Colab
Kaggle
NumPy
BM25
bge-small embeddings
Reciprocal rank fusion
MiniLM cross-encoder
ONNX Runtime
FastAPI
Docker
Hugging Face Spaces
pytest
Code and demo: Live demo Fine-tuning notebook Service code

Skip to the results

1 of 4 · inside the model

One sentence through GPT-2 medium

24 blocks · 16 heads · 1,024 numbers per word · 1,024-token window

The sentence "The words of House Stark are" is cut into 6 tokens with the ids 464, 2,456, 286, 2,097, 20,956 and 389. Each id picks a row of 1,024 numbers, a position row is added, and the 6 × 1,024 matrix passes through 24 blocks; in every block, attention (16 heads of 64) and an MLP (1,024 to 4,096 and back) add to the 6 × 1,024 matrix. The last token's vector times the LM head gives 50,257 scores; one word is chosen, turned into a token id and appended, and the loop repeats.

2 of 4 · the book search

How a question is answered

search first, then the grounded model

1The question goes in

What are the words of House Stark?

2Two searches run at the same time

BM25matches the words

top 50

bge-small embeddingsmatches the meaning

top 50

3Reciprocal rank fusion, k = 60

score = 1 / (60 + rank in BM25) + 1 / (60 + rank in bge-small)

The two lists become one; the best 20 go on.

4MiniLM cross-encoder re-reads the top 10

It reads the question and each passage together, reorders them, and keeps the best three.

5Up to 3 passages fit inside 1,024 tokens

up to 3 passages, 600 tokens at mostquestion blockanswer, up to 120

They go in front of the question, inside the template the model was trained on.

6The grounded model answers

passages + questionGPT-2 medium, trained again on 19,289 questions with their passage

7The sources are shown

Book passages it read firstbook and chaptera snippet of 25 words at most

3 of 4 · the data

From five books to a trained model

cleaned, cut, questioned, mixed, trained

1Five books

A Game of Thrones
A Clash of Kings
A Storm of Swords
A Feast for Crows
A Dance with Dragons

2Cleaned and preprocessed

plain book text

3Cut into passages of 128 tokens

26,112passages, each with its book and chapter

4A second AI model writes questions and answers

61,650pairs written by gpt-4o-mini
34,001training pairs after cleaning1,233held out for testing, 2% of the 61,650

5The fine-tuning set is built

book text

question-answer pairs

Mixed in the same training steps. A second set pairs 19,289 questions with their passage.

6Training

Fine-tunebooks and question-answer pairs
Grounding roundfree Kaggle GPU, 68 minutes; loss with passages 1.65 → 1.39

4 of 4 · the result

What the book search changed

same questions, same settings, same computer

no searchsearch + retrained (live)

Fact questions right, of 30

New fact questions right, of 42

Four old mistakes fixed, of 4

Answers with an invented name

36%

12%

Lower is better.

GPT-2, in one line

GPT-2 is the language model OpenAI released in 2019: a much smaller, older relative of the models behind ChatGPT. Its weights are public, so you can train it further yourself.

What I built

The whole path: books to text, text to pieces, practice questions written by a second AI model, the GPT-2 network classes, written in PyTorch while following a build-an-LLM course and loaded with OpenAI's released weights, fine-tuning in Colab and on Kaggle, a search over the five books (26,112 short passages, found by their words and by their meaning, then re-ranked), a second round of training so the model uses what the search finds, a FastAPI service on a free Hugging Face Space and a question box, the one on this page.

How it works

The five books are read into plain text, cleaned of copyright pages and contents lists, and cut into pieces.
A second AI model, gpt-4o-mini, writes one fact question and one full-sentence answer per piece.
GPT-2 medium, in network code I wrote myself, is fine-tuned on book passages and question-answer pairs mixed in the same training steps.
The books are cut again into 26,112 passages of about 128 tokens, each tagged with its book and chapter, and indexed twice: by the words in it and by its meaning. The index is private, because the books are copyrighted.
When you ask, the service searches both indexes, a small ranking model re-reads the ten best passages, and the best three go in front of your question.
Pasting passages in front of the model alone did not help: it ignored them. So I trained it a second time, on free Kaggle GPU time, on 19,289 questions that come with their passage.
A small FastAPI service on a free Hugging Face Space answers in about five to twenty seconds. The answer comes back with its sources: book, chapter and a snippet of at most 25 words. The first question after a quiet spell wakes it, which takes about a minute.

5books, all of A Song of Ice and Fire so far

34,001cleaned question-answer pairs, from 61,650 first written

26,112book passages in the search, each with its book and chapter

MediumGPT-2 size, from OpenAI's released weights

Retraining changed the voice

Measure	2025 model	First retrain	Second retrain
Loss on 1,233 test questions	2.734	2.146	2.119
Loss on one book passage	3.072	2.659	2.615
Ends its own answer (10 questions, 3 tries)	0 of 30	30 of 30	30 of 30

Held-out test from one part of A Feast for Crows, Oct 2026. The two 2026 runs never trained on it; the 2025 model had read the passage, but not the questions. Lower loss is better. The second retrain mixed the book text and the question-answer pairs in the same training steps.

The book search changed the facts

Measure	No search	Search, same model	Search, retrained (live)
Fact questions right (of 30)	12	10	17
New fact questions right (of 42)	2	8	13
Four old mistakes fixed (of 4)	1	1	4
Answers with an invented name	36%	28%	12%

Same questions, same settings, same computer. A fact counts as right when 2 of 3 tries had the accepted words and none of the rejected ones; the 42 new questions were never used to train or tune. An invented name is a person named in the answer who is in neither the question nor the passages (196 held-out questions). The search puts the right passage in its top 5 for 78 percent of those questions. Asked live on the Space, one try per question, the 30 facts gave 19 right in Oct 2026.

The same network, two more ways

I re-wired the same GPT-2 network into a spam classifier for text messages: 48.75% accuracy before training, 84.00% after, on 300 messages it had never seen. My build notebooks go from the tokenizer to fine-tuning, one step at a time.

Try the spam classifier

Paste a text message, or tap an example. The model answers with one word.