← avinash / experiments Source ↗
PLE-LAB / FIELD INSTRUMENT 02STATIC DATA · INTERACTIVE VIEW

THE TOKEN ATLAS

One token.
Six places.

Move through the six learned embedding spaces. Search a token, rotate its neighborhood, and read the actual 384 published vector values.

Trained small GPT on TinyStories; not Gemma 3n.

01 / EXPLORE THE TABLELOADING PUBLISHED DATA
Loading projection and quantized vectors…
02 / WHAT YOU ARE SEEINGLOOKUP, NOT INFERENCE

A small table,
not a model in your browser.

These are pre-exported rows from a trained small GPT on TinyStories, not Gemma 3n. Each token has six 64-dimensional learned PLE slices. The map stores three PCA coordinates per slice; the heatmap and cosine search reconstruct vector values from signed 8-bit integers multiplied by each row’s published float32 scale. Quantization means these approximate the original float vectors; similarity is exact for this published, quantized subset.

This page cannot generate text, run PyTorch, or perform live ablations. To try the full model-backed explorer locally, clone PLE-Lab ↗, install pip install -r requirements.txt, prepare a run (or python -m explorer.make_demo_run for a random demonstration), then launch python -m explorer.server --runs runs --port 8765 and open http://127.0.0.1:8765. For trained results, follow the training instructions ↗; the random demo is not the measured model.

← Back to the PLE-Lab results