Why I annotate
research papers.

01 / Purpose

The details between the equations

Papers describe new results for readers who already know the field. They often leave intermediate calculations implicit, and put implementation details in a supplement or a separate codebase. That can make a model difficult to reconstruct from the paper alone.

I use this site to explain those details: what an equation computes, which dimensions a tensor has, and how an operation is implemented. The original paper remains the reference for the research; the annotation provides additional explanation and code.

The format is inspired by The Annotated Transformer, which presents an implementation alongside an explanation of the paper.

02 / Format

How the annotations are organized

Each annotation is divided into sections for the model's components. The table of contents links to individual sections, and the notes refer to the relevant equations and code.

1. Paper and notation

   The text explains the architecture and equations using notation close to the original. References let you compare the explanation with the paper and supplement.

2. Implementation

   Code excerpts come from a runnable implementation. They include tensor shapes and variables corresponding to the paper's notation. The repository contains the complete source.

3. Interactive figures

   Some sections include figures with adjustable views or parameters, so you can inspect the operation described in the text.

03 / Source code

Code and reproducibility

The companion implementations prioritize readable code and include unit and integration tests. Each annotation refers to a specific repository commit, so the excerpts can be checked against the exact version of the source.

You can clone the repository to run the implementation locally. The repository's documentation describes its setup and scope.