From Perceptron to Transformer:
The substrate of modern LLMs

1. Artificial Neural Networks: from biological neurons to math functions

1.1 The smallest unit of a neural network

An artificial neural network is inspired by the way biological neurons work. A single neuron is, in essence, a math function:

output = activation(w · x + b)

Here x is the input vector, w is the weight, b is the bias, and activation is a non-linear function (commonly Sigmoid, ReLU, or GELU). Stack thousands of these "neurons" into layers and wire them together and you have the foundation of modern deep learning.

1.2 The three elements of training

• Forward propagation: data flows from the input layer to the output layer, producing a prediction.
• Loss function: measures the gap between the prediction and the ground truth. Classification uses cross-entropy; regression uses mean squared error.
• Backpropagation + gradient descent: starting from the output, compute each parameter's "contribution" (gradient) to the loss, then nudge the weight in the opposite direction to make the next prediction more accurate.

💡 Key intuition: training a neural network is essentially "searching a billion-dimensional parameter space for a weight configuration that minimizes the loss". This is why LLMs need enormous data and compute — the parameter space is huge.

2. Transformer: the cornerstone of modern LLMs

In 2017, Google published Attention Is All You Need, introducing the Transformer architecture and changing the trajectory of AI. Its core innovation: throw out recurrence entirely and rely purely on the attention mechanism to process sequences.

2.1 Why Transformer swept NLP

2.2 Self-attention

The core idea: for every word in a sequence, "look at" every other word and decide how much each one matters to it.

Take this example sentence — "Xiaoming went to Beijing on business yesterday, and he came back today":

1. Each word produces three vectors: Query (Q), Key (K), Value (V).
2. For "he", compute the dot product of its Q with every other word's K to get a relevance score.
3. Apply Softmax to get attention weights (e.g., 0.8 for "Xiaoming", 0.1 for "Beijing").
4. Use those weights to take a weighted sum of every word's V, producing a new, context-rich representation of "he".

In math form:

Attention(Q, K, V) = softmax(Q · K^T / √d_k) · V

where d_k is the dimensionality of the Key vector; dividing by its square root prevents the dot product from blowing up the Softmax gradient.

2.3 Multi-head attention

Multi-head attention is self-attention on steroids. It runs multiple self-attention operations in parallel across different "representation subspaces" and concatenates the results. This lets the model capture different kinds of relationships at once:

• One head may focus on syntactic relations (subject-verb-object).
• Another on semantic relations (synonyms, antonyms).
• A third on long-range coreference (which noun a pronoun refers to).

2.4 Positional encoding

Transformer processes every word in parallel, so it loses "order" information by default. "Dog bites man" and "Man bites dog" have identical tokens — only the order differs. Positional encoding adds a unique "position vector" to the embedding of each word so the model knows where it sits in the sequence.

The original Transformer used fixed sinusoidal/cosine positional encodings. Later models (LLaMA, Qwen) use RoPE (Rotary Position Embeddings), which handles much longer sequences.

3. The full Transformer architecture

A standard Transformer block has two halves:

input → [LayerNorm → Multi-Head Attention → Residual]
     → [LayerNorm → Feed-Forward Network (FFN) → Residual]
     → output

• Multi-Head Attention: finds "relationships" — lets every word see every other word.
• Feed-Forward Network (FFN): does the "processing" — applies a non-linear transform to each word individually to extract higher-level features.
• Residual connection: adds the input directly to the output, avoiding vanishing gradients and enabling deep networks.
• LayerNorm: stabilizes training and prevents numerical blow-ups.

The GPT family (ChatGPT, DeepSeek, Qwen) are simply dozens to hundreds of such blocks stacked into a Decoder-only Transformer.

4. Pre-training, SFT, RLHF: the three-step training recipe

4.1 Pre-training

Train on a vast corpus of internet text (trillions of tokens) so the model learns to "predict the next token". This is the longest and most expensive step — DeepSeek-V3 used 14.8T tokens of training data and cost roughly $5.57M (GPU rental).

4.2 Supervised fine-tuning (SFT)

Train on a small set of high-quality human-curated "instruction–response" pairs (tens of thousands) so the model learns to "understand the request and respond accordingly". Volume is far smaller than pre-training, but quality requirements are extreme.

4.3 Reinforcement learning from human feedback (RLHF)

Humans rank the model's different responses; train a reward model on those rankings, then use RL (typically PPO or DPO) to nudge the model toward higher-ranked answers.

5. Why do we need GPU clusters?

Training a 100B-parameter model — a single forward + backward pass requires:

• Input token embeddings (params × sequence length)
• Attention matrices (sequence length² × num heads)
• Gradients (params)
• Optimizer state (momentum, variance — 3-4× params)

None of this fits in a single GPU (80GB), so we need multiple parallel strategies:

6. From LLMs to Agents: the leap in capability

Once LLMs got strong enough, people started adding "extras": the model can not only "talk" but also "do". This is the starting point of the AI Agent:

• ReAct (reason + act): the model first thinks about what to do, then decides which tool to call.
• Function calling: the model emits structured "tool-call requests" executed by an external system.
• Multi-Agent collaboration: multiple Agents play different roles and coordinate to complete a task.

We will dive into this in the next chapter, Agent Evolution.

7. Chapter summary

• ✅ Transformer replaced recurrence with the attention mechanism, enabling parallel training and long-range modeling.
• ✅ Self-attention lets every word "see" every other word; multi-head attention captures multiple relationships at once.
• ✅ Pre-training + SFT + RLHF is the three-step recipe for training an LLM.
• ✅ 100B-parameter models demand GPU clusters and a combination of parallel strategies.

8. Hands-on: build a Mini Transformer from scratch

Now that the theory is covered, this section walks through building a 6-layer Decoder-only Transformer in PyTorch from zero, trained on a small text corpus. The full code is around 250 lines and runs on CPU or GPU — the goal is to make every gear of the Transformer visible to you.

8.1 Goals & environment

• Implement a 6-layer Decoder-only Transformer with ~1.2M parameters.
• Train a character-level language model on a small ~500-line corpus and watch the loss fall.
• Provide two runnable scripts: train.py and sample.py.

Environment setup (2 lines):

pip install torch numpy
python -c "import torch; print('torch', torch.__version__)"

8.2 Data preparation

We will use a small piece of Tang-poetry-style pseudo data (you can replace it with your own input.txt):

cat > data/input.txt << 'EOF'
Quiet night thoughts
Before my bed a pool of light,
Can it be frost upon the ground?
Eyes up, I find the moon so bright,
Head down, I think of home I've found.
Spring dawn
Spring slumber knows no dawn so soon,
The birds' song fills the air with tune.
Last night the wind and rain did blow,
How many flowers have let go?
EOF
wc -c data/input.txt

A character-level model just reads the whole file as one long string — no tokenizer needed.

8.3 Full code: train.py

"""Mini Transformer: 6-layer Decoder-only, character-level language model.

Usage:
    python train.py
"""
import math
import time
from pathlib import Path

import torch
import torch.nn as nn
import torch.nn.functional as F


# ---------- 1. Data loader ----------
class CharDataset:
    """Slice a text file into (block_size)-long training samples."""

    def __init__(self, path: str, block_size: int = 64):
        self.text = Path(path).read_text(encoding="utf-8")
        self.chars = sorted(set(self.text))
        self.vocab_size = len(self.chars)
        self.stoi = {c: i for i, c in enumerate(self.chars)}
        self.itos = {i: c for i, c in enumerate(self.chars)}
        self.block_size = block_size
        self.data = torch.tensor([self.stoi[c] for c in self.text], dtype=torch.long)
        print(f"[data] chars={self.vocab_size} len={len(self.data)}")

    def get_batch(self, batch_size: int = 32):
        ix = torch.randint(0, len(self.data) - self.block_size - 1, (batch_size,))
        x = torch.stack([self.data[i : i + self.block_size] for i in ix])
        y = torch.stack([self.data[i + 1 : i + 1 + self.block_size] for i in ix])
        return x, y


# ---------- 2. Model components ----------
class PositionalEncoding(nn.Module):
    """Sinusoidal positional encoding, frozen."""

    def __init__(self, d_model: int, max_len: int = 512):
        super().__init__()
        pe = torch.zeros(max_len, d_model)
        position = torch.arange(0, max_len).unsqueeze(1).float()
        div_term = torch.exp(torch.arange(0, d_model, 2).float() * -(math.log(10000.0) / d_model))
        pe[:, 0::2] = torch.sin(position * div_term)
        pe[:, 1::2] = torch.cos(position * div_term)
        self.register_buffer("pe", pe.unsqueeze(0))  # (1, max_len, d_model)

    def forward(self, x):
        return x + self.pe[:, : x.size(1)]


class CausalSelfAttention(nn.Module):
    """Multi-head causal self-attention (used in the Decoder)."""

    def __init__(self, d_model: int, n_heads: int, max_len: int = 512, dropout: float = 0.1):
        super().__init__()
        assert d_model % n_heads == 0
        self.n_heads = n_heads
        self.head_dim = d_model // n_heads
        self.qkv = nn.Linear(d_model, 3 * d_model)
        self.proj = nn.Linear(d_model, d_model)
        self.dropout = nn.Dropout(dropout)
        # causal mask: upper triangle (excluding diagonal) is -inf
        mask = torch.triu(torch.ones(max_len, max_len), diagonal=1).bool()
        self.register_buffer("mask", mask)

    def forward(self, x):
        B, T, C = x.shape
        qkv = self.qkv(x).reshape(B, T, 3, self.n_heads, self.head_dim).permute(2, 0, 3, 1, 4)
        q, k, v = qkv[0], qkv[1], qkv[2]  # (B, h, T, head_dim)
        att = (q @ k.transpose(-2, -1)) / math.sqrt(self.head_dim)
        att = att.masked_fill(self.mask[:T, :T], float("-inf"))
        att = F.softmax(att, dim=-1)
        att = self.dropout(att)
        out = (att @ v).transpose(1, 2).reshape(B, T, C)
        return self.proj(out)


class FeedForward(nn.Module):
    """Transformer FFN: GELU + Linear + Dropout."""

    def __init__(self, d_model: int, d_ff: int, dropout: float = 0.1):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(d_model, d_ff),
            nn.GELU(),
            nn.Linear(d_ff, d_model),
            nn.Dropout(dropout),
        )

    def forward(self, x):
        return self.net(x)


class Block(nn.Module):
    """Pre-LN Decoder block: Attention -> FFN, each with residual."""

    def __init__(self, d_model: int, n_heads: int, d_ff: int, max_len: int, dropout: float):
        super().__init__()
        self.ln1 = nn.LayerNorm(d_model)
        self.attn = CausalSelfAttention(d_model, n_heads, max_len, dropout)
        self.ln2 = nn.LayerNorm(d_model)
        self.ffn = FeedForward(d_model, d_ff, dropout)

    def forward(self, x):
        x = x + self.attn(self.ln1(x))
        x = x + self.ffn(self.ln2(x))
        return x


class MiniTransformer(nn.Module):
    """6-layer Decoder-only Transformer, character-level language model."""

    def __init__(
        self,
        vocab_size: int,
        d_model: int = 128,
        n_heads: int = 4,
        d_ff: int = 512,
        n_layers: int = 6,
        max_len: int = 256,
        dropout: float = 0.1,
    ):
        super().__init__()
        self.tok_emb = nn.Embedding(vocab_size, d_model)
        self.pos_enc = PositionalEncoding(d_model, max_len)
        self.blocks = nn.ModuleList(
            [Block(d_model, n_heads, d_ff, max_len, dropout) for _ in range(n_layers)]
        )
        self.ln_f = nn.LayerNorm(d_model)
        self.head = nn.Linear(d_model, vocab_size, bias=False)
        # Weight tying: input embedding shares weights with the output head (BERT/GPT trick)
        self.head.weight = self.tok_emb.weight
        self.apply(self._init_weights)

    def _init_weights(self, m):
        if isinstance(m, nn.Linear):
            nn.init.normal_(m.weight, std=0.02)
            if m.bias is not None:
                nn.init.zeros_(m.bias)
        elif isinstance(m, nn.Embedding):
            nn.init.normal_(m.weight, std=0.02)

    def forward(self, idx, targets=None):
        x = self.tok_emb(idx)
        x = self.pos_enc(x)
        for blk in self.blocks:
            x = blk(x)
        x = self.ln_f(x)
        logits = self.head(x)
        loss = None
        if targets is not None:
            loss = F.cross_entropy(logits.view(-1, logits.size(-1)), targets.view(-1))
        return logits, loss

    @torch.no_grad()
    def generate(self, idx, max_new_tokens: int = 100, temperature: float = 1.0, top_k: int = 20):
        """Autoregressive generation with temperature + top-k sampling."""
        for _ in range(max_new_tokens):
            idx_cond = idx[:, -self.pos_enc.pe.size(1) :]
            logits, _ = self(idx_cond)
            logits = logits[:, -1, :] / temperature
            if top_k:
                v, _ = torch.topk(logits, min(top_k, logits.size(-1)))
                logits[logits < v[:, [-1]]] = float("-inf")
            probs = F.softmax(logits, dim=-1)
            nxt = torch.multinomial(probs, num_samples=1)
            idx = torch.cat([idx, nxt], dim=1)
        return idx


# ---------- 3. Training loop ----------
def main():
    torch.manual_seed(0)
    ds = CharDataset("data/input.txt", block_size=64)
    device = "cuda" if torch.cuda.is_available() else "cpu"
    model = MiniTransformer(vocab_size=ds.vocab_size).to(device)
    print(f"[model] params={sum(p.numel() for p in model.parameters()) / 1e6:.2f}M device={device}")

    opt = torch.optim.AdamW(model.parameters(), lr=3e-4, weight_decay=0.1)
    steps = 1000
    batch_size = 32
    t0 = time.time()
    for step in range(steps):
        x, y = ds.get_batch(batch_size)
        x, y = x.to(device), y.to(device)
        _, loss = model(x, y)
        opt.zero_grad()
        loss.backward()
        torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)
        opt.step()
        if step % 100 == 0 or step == steps - 1:
            dt = time.time() - t0
            print(f"step {step:4d} loss={loss.item():.4f} elapsed={dt:.1f}s")
    torch.save({"model": model.state_dict(), "stoi": ds.stoi, "itos": ds.itos}, "ckpt.pt")
    print("[done] saved ckpt.pt")


if __name__ == "__main__":
    main()

8.4 Inference script: sample.py

"""Generate text from a trained ckpt.pt."""
import torch
from train import MiniTransformer

ckpt = torch.load("ckpt.pt", map_location="cpu")
stoi, itos = ckpt["stoi"], ckpt["itos"]
model = MiniTransformer(vocab_size=len(itos))
model.load_state_dict(ckpt["model"])
model.eval()

prompt = "Quiet night thoughts\n"
idx = torch.tensor([[stoi[c] for c in prompt]], dtype=torch.long)
out = model.generate(idx, max_new_tokens=120, temperature=0.8, top_k=10)[0].tolist()
print("=== generated ===")
print("".join(itos[i] for i in out))

8.5 Run it & expected output

mkdir -p data
# (write the input.txt from 8.2)
python train.py

Expected output (CPU ~30-60s, MPS/GPU a few seconds):

[data] chars=28 len=87
[model] params=1.21M device=cpu
step    0 loss=3.3321 elapsed=0.0s
step  100 loss=2.7102 elapsed=2.1s
step  200 loss=2.3504 elapsed=4.0s
...
step  900 loss=1.8421 elapsed=32.5s
step  999 loss=1.7903 elapsed=36.0s
[done] saved ckpt.pt

python sample.py
=== generated ===
Quiet night thoughts
Before my bed a pool of light,
Can it be frost upon the ground?
Eyes up, I find the moon so bright,
Head down, I think of home I've found.

Screenshot: training curve (loss vs step) — use tensorboard or matplotlib to plot loss.item() as loss_curve.png. Sample output — just copy the terminal text.

8.6 Tuning tips

• Loss won't drop: data too short → use a longer input.txt (e.g. a full Tang poetry corpus); tune LR to 1e-4 / 5e-4.
• Generation collapses to one character: temperature too low — bump to 0.8-1.2; raise top_k to 20-50.
• Out of memory: lower batch_size to 8, or run in half precision with torch.amp.autocast.
• Need more speed: set d_model=96, n_layers=4 to bring params to ~400K.

8.7 Where to go next

• A full GPT: add top-p (nucleus) sampling, KV-cache for faster inference.
• LLaMA architecture: replace LayerNorm with RMSNorm, sinusoidal PE with RoPE, GELU FFN with SwiGLU, and add GQA to cut memory.
• Multi-GPU training: torchrun --nproc_per_node=2 train.py with DistributedDataParallel.
• Use a real tokenizer: swap CharDataset for tiktoken (GPT-2 BPE) or sentencepiece.

9. Video learning — watch the masters explain it

Once the theory sinks in, watching an expert hand-code the whole thing pushes the rest of the chapter to a deeper level. Below is the single video most often recommended as the best visual introduction to how Transformers work.