Attention is seven questions wearing one name
Causal attention, GQA, FlashAttention and RoPE answer different questions. Seven categories show which part each one changes.
Field notes / ideas / experiments
Notes from the workshop: robotics, software, AI, and whatever I’m learning while building in the open.
04 pieces
Causal attention, GQA, FlashAttention and RoPE answer different questions. Seven categories show which part each one changes.
A loop-engineered robotics harness should route experiments through durable state, independent verification, simulation, replay and supervised hardware.
Mark Gaussian splats with object identity, retrieve the relevant scene evidence, and let geometry handle the drive.
Musk repeatedly presents unfinished engineering as nearly solved, then replaces missed deadlines with larger promises.