Alok Didderobots + systems

Tools / models / mechanisms

Lab

Small interactive tools for seeing how models work inside: change one design choice and watch what it changes.

02 tools

Transformers

Attention is seven questions wearing one name

Change the mask, share the key-value cache, then assemble a mechanism from seven design choices. Causal attention, GQA, FlashAttention and RoPE each answer a different one.

Attention · Masks · KV cache · RoPE

Encoders

Inside a text encoder

Follow a sentence from token lookup, through attention that reads both ways, to one pooled vector you can compare with cosine similarity.

Embeddings · Bidirectional · Pooling · Similarity