Contents
Where this sits
This page is about choices, so it is pinned to the stage where the choices meet: F, static embeddings. There the exact, training-free tree of stages D and E first stands next to learned geometry, and the main route, a baseline and a control are decided together in one claim, L6.
Phase 0 contains no model code. Every route below that will be run is either part of a claim already registered in PREREGISTRATION.md (L1–L11), or will be registered by a dated amendment before any code for it is written. The routes come from the design spec’s comparison of alternatives. The descriptions and evidence grades come from the read-only literature survey and its adversarial cross-check; where the two disagreed, the cross-check wins.
Five roles, one table
Each route has exactly one role. The role says what the route is for here, not whether it is good research.
- MAIN
- What we build, in order. M1 (Stage 1) is built first. M2 (Stage 2) starts only after every Stage-1 claim is decided.
- BASELINE
- A comparator. It either runs in the benchmark ladder, or its published number is cited with its grade.
- CONTROL
- Built to fail. A control removes the one thing a claim credits, such as the curvature or the hierarchy. If a must-fail control does as well as the real arm, the metric cannot tell structure from noise and the test is void; if a comparison arm does as well, the claim fails.
- LATER
- Worth doing, but not now. Each is registered by amendment before any run.
- REJECTED
- Checked against the code or the literature and found unfit for the job it was proposed for. Some stay in use for another job, which the table names.
How the evidence is graded
Most primary hosts were blocked for the survey (arxiv.org, aclanthology.org, the NeurIPS proceedings, huggingface.co); only github.com and raw.githubusercontent.com could be fetched. So every literature claim carries the survey’s grade:
- verified checked against the paper’s abstract or proceedings page (through search results that quote it, since those hosts were blocked), or read directly from the official repository or its source code;
- likely taken from a search-engine snippet of the primary paper, such as a table number that was not read directly;
- secondary from a secondary summary, or a paper the survey did not open (the survey calls some of these “unverified”);
- ours a choice of this design, with no literature number behind it.
No first-round claim compares against a likely or secondary number. Those comparisons wait until the primary sources are read; arxiv.org and aclanthology.org are reachable since the network was widened at the start of Phase 1. On 5 Nov 2026 the outside numbers whose primary source had become reachable were read in it, and the grades on this page were updated to match (Explainer 7 records the check); the others are still the survey’s.
Models of hyperbolic space, in one paragraph each
Hyperbolic space Hd has constant negative curvature. Its key property here: the volume of a ball grows exponentially with its radius, just as the number of nodes in a tree grows exponentially with depth. So trees fit into H² or H³ with little distortion, where a flat (Euclidean, curvature 0) space of the same dimension runs out of room. The same space can be written down in several coordinate systems, called models:
Lorentz (hyperboloid) model
Points lie on one sheet of a hyperboloid, the surface x0² − x1² − … − xd² = 1 with x0 > 0, in d+1 coordinates. Distance is a monotone function of the Lorentz inner product, so ranking by that product ranks by distance with no acosh. Geodesics have closed forms, which makes it the standard model for training (Nickel & Kiela 2018).
Poincaré ball
All of Hd drawn inside the unit ball; distances blow up near the edge. It is the natural model for pictures (the disk and ball on our pages) and common for output heads (HiT). Near the edge, gradients vanish (Guo 2022).
Klein and upper half-space
The Klein model gives cheap averaging and is used in Gulcehre et al.’s hyperbolic attention (ICLR 2019). The upper half-space model hosts the multi-component floats of NeurIPS 2021. Neither is used by any route here.
Our exact tree
The norm-5 quaternion tree, whose generators 1±2i, 1±2j, 1±2k are the Lubotzky–Phillips–Sarnak (LPS) set for p = 5. A word in the generators is an address; its point on the hyperboloid has exact rational coordinates, in H³ (existing) and in H² (new). See the LPS link and placement.
All 28 routes
here runs in this container (4 CPU cores, 15 GB RAM, no GPU; PyPI and GitHub reachable). net needs hosts that were blocked here in Phase 0, such as huggingface.co, osf.io or zenodo.org; all three are reachable since the network was widened at the start of Phase 1, except osf.io’s file downloads. gpu needs a GPU, on your hardware or in the cloud.
| Route | Role | Model of hyperbolic space | Evidence | Needs | Decided by |
|---|---|---|---|---|---|
| M1 exact, training-free addresses and points | MAIN · Stage 1 | exact rational points on the hyperboloid, H³ and H²; Poincaré only to draw | Sarkar; Sala et al.; HS-DTE; Yu & De Sa verified | here | L3–L7 |
| M2 Euclidean encoder, hierarchy factor, hyperbolic head | MAIN · Stage 2 | Poincaré head (HiT); Lorentz exp-map lift (MERU, HyTE-H) | HiT and its F1 numbers verified | net gpu | amendment after the gate |
| B1 learned static embeddings | BASELINE | Lorentz and Poincaré against Euclidean, float64 | Nickel & Kiela verified | here | L6 |
| B2 published combinatorial embeddings | BASELINE | Delaunay trees in Hd; integer tiling | Sala 0.989 MAP verified | cited, not rerun | reported in L5 |
| B3 HiT on MiniLM | BASELINE | Poincaré ball | WordNet multi-hop F1 verified | net gpu | Phase 7 |
| B4 Euclidean BERT at matched size | BASELINE | none (flat) | BERT-Tiny–Medium, ELECTRA-Small verified | net gpu | Phase 7 |
| Shuffled addresses | CONTROL | same tree, words permuted | ours | here | L5 |
| Content-hash addresses | CONTROL | same tree, hash-like words | ours | here | L5 |
| Curvature 0 (F and F*) | CONTROL | the same construction in R² and R³ | ours | here | L5 |
| Learned Euclidean at matched dimension and bits | CONTROL | flat | Bansal & Benton verified | here | L6 |
| Float against exact ranking | CONTROL | the same exact points, read in float | Engram spike B; a planning check | here | L4 |
| Byte-level encoder (tiny ByT5-style) | CONTROL | none (flat) | ByT5 verified | net gpu | Stages 3–4 |
| engram/1 as an id | CONTROL | none | Engram’s spike A and E9 | here | L2 |
| L-a fully Lorentz encoders | LATER | Lorentz throughout | HyboNet, Hypformer, HELM verified; Hyperbolic PLM likely | net gpu | amendment |
| L-b product and mixed curvature | LATER | products of hyperbolic, spherical and flat factors | Gu et al. 2019 likely | the Stage-2 encoder | amendment |
| L-c hyperbolic tokenization | LATER | merge or morphology trees in H²/H³ | no prior work found secondary | here net | amendment |
| L-d gradient-free training | LATER | any; heads, curvature, adapters only | MeZO verified; our C1 refuted | here | amendment |
| L-e quaternion and hypercomplex layers | LATER | none (hypercomplex algebra, flat) | Quaternion Transformer, PHM, Compacter verified | the Stage-2 encoder | amendment |
| L-f distilling BERT into a hyperbolic student | LATER | Lorentz student | no BERT precedent; HyFL-CLIP secondary | net a BERT teacher | amendment |
| L-g Brown-cluster addresses | LATER | the same exact tree | ours | here | amendment |
| L-h Huffman-5 virtual letters | LATER | the same exact tree | ours | here | amendment |
| engram/1 as a token id or feature | REJECTED | — | Engram’s own README | — | kept as a checksum only |
| GROWTH/1 epochs as corpus growth | REJECTED | — | BioChain-AI code | — | shards + manifest instead |
| A Form nibble as namespace | REJECTED | — | layout census | — | record kinds instead |
| Hugging Face fingerprints as provenance | REJECTED | — | datasets source verified | — | SHA-256 instead |
| LPS mod-q reduction as integrity | REJECTED | — | Tillich–Zémor; Petit–Lauter–Quisquater | — | a shard key only |
| The float chiral weave as identity | REJECTED | — | Owl and BioChain code | — | the exact weave instead |
| Register clock as position; coupler as segment binding | REJECTED | — | Engram’s design | — | prefix embeddings instead |
The main routes
Two routes, built in order. M1 plays to what only this stack has: exact tree addresses. M2 is the route the literature supports best, and it starts only after every Stage-1 claim (L1–L11) is decided and reviewed.
Green stages are exact (integers, rationals, bytes; identical in Python and JavaScript). Gold stages are float, with exact snapshots and receipts. WordPiece is BERT’s subword tokenizer: it splits each word greedily into the longest pieces found in a fixed vocabulary, marking pieces after the first with ##.
M1 · Exact, training-free hierarchy addresses and points
- What it is. Each WordNet noun synset (a set of synonyms naming one concept) gets a word in the norm-5 tree, under a fixed rule: its parent is its first hypernym or instance-hypernym pointer, children are ordered by synset offset, and a node with c children gives each child max(1, ⌈log5 c⌉) letters. That word is placed as an exact point in H³ (the existing octahedral tree, step τ = ln 7) and in H² (a new exact tree, step τ = ln 12). Nothing is trained. The only learning in Stage 1 is the float64 static embeddings of B1, which are snapped to a 2−32 grid for certified storage.
- Model. Lorentz (hyperboloid) points with exact rational coordinates. The Poincaré disk and ball are used only to draw them. Ranking uses the exact Lorentz inner product, with ties counted pessimistically (L4).
- Evidence. This is the family of training-free tree embeddings, all verified at abstract level:
- Sarkar (2011) embeds any weighted tree in H² with arbitrarily low distortion, combinatorially, with no training.
- Sala, De Sa, Gu & Ré (ICML 2018) reach WordNet MAP 0.989 in 2 dimensions, but need about 500 bits of precision, or 10 dimensions at 32 bits. The cross-check confirmed both figures.
- HS-DTE (van Spengler & Mettes, ICML 2025) extends such Delaunay tree embeddings to any dimension, with floating-point expansions.
- Yu & De Sa (NeurIPS 2019) store hyperbolic points as an integer tiling with provably bounded error, at 2% of a Poincaré embedding’s size on WordNet nouns.
- Why it is main. It plays to what only this stack has: exact LPS-tree addresses, rational points, certificates (the integer quaternion has norm exactly 5length, and dividing it back returns the word), records committed by a Merkle root, and results identical across engines. It also produces the certified artifacts every later stage reuses.
- Costs. The branching is fixed at 5. WordNet’s noun tree has 3,291 nodes with more than 5 children, 396 with more than 25, and the largest has 659, whose children each need ⌈log5 659⌉ = 5 letters. These virtual letters put some siblings as close to a node as its parent, or closer, which distorts distances. And exactness does not avoid the precision cost that Sala et al. describe; it pays it in BigInt. An earlier measurement found H³ points at depth 28 need about 138-bit numerators and 79-bit denominators; a planning check at the same depth put H² at up to 217/143 bits for random words and 229/188 in the worst case.
- Needs. here CPU Python as the authority, Node and the browser for parity. No GPU and no blocked host: WordNet 3.0 comes from GitHub and is pinned (82,115 noun synsets, maximum spanning depth 19, 743,241 closure edges).
The literature survey assumed hyperbolic_core.js already had exact rational H² points. It does not: its H² TreeEmbedding uses Math.cos and Math.sin. Only H³ had exact tree points, so the exact H² tree is new work, built on the exact rational type alone (Explainer 3).
M2 · A Euclidean encoder with a hierarchy factor and a hyperbolic head
- What it is. A small BERT-style encoder in ordinary flat space. It takes in each token’s address as a hierarchy factor: the sum of learned embeddings of the address’s prefixes, up to 12 letters long, plus the frozen log coordinates of the exact point truncated to depth 12 or less (3 numbers from H³, or 2 from H²). On top sit a masked-LM head and a hyperbolic hierarchy head that scores a parent–child pair as −(d + w·(‖parent‖ − ‖child‖)), with feature clipping. See the encoder.
- Model. The head follows HiT, which uses a Poincaré ball. Related systems lift flat vectors into the Lorentz model with the exponential map, which treats a vector as a direction and length from the origin and walks that far along a geodesic.
- Evidence. This is the best-supported route in the literature:
- HiT, Hierarchy Transformers (He, Yuan, Chen & Horrocks, NeurIPS 2024) verified: a pretrained encoder’s outputs are placed in a Poincaré ball that encloses the encoder’s output space, and re-trained with hyperbolic clustering and centripetal losses. Code is Apache-2.0. The paper re-trains three bases, all-MiniLM-L6-v2, all-MiniLM-L12-v2 and all-mpnet-base-v2 (Sec. 4.3 of arXiv:2401.11374v4), and uses all-MiniLM-L12-v2 (about 33M parameters; 33.4M on its Hugging Face model page) alone for its transfer and SNOMED tests. The repository README names no base; the Hugging Face card of HiT’s L12 checkpoint does.
- WordNet multi-hop F1 (Table 2 of arXiv:2401.11374v4): HiT on all-mpnet-base-v2 0.916 (random negatives) and 0.899 (hard), against Poincaré embeddings (200-d) 0.864 and 0.830 and a fine-tuned all-mpnet-base-v2 0.615 and 0.687 verified. HiT on all-MiniLM-L12-v2 scores 0.903 and 0.908 in the same table.
- MERU (Desai et al., ICML 2023) verified lifts CLIP-style embeddings onto the hyperboloid with the exp map and adds an entailment loss. HyTE-H (HypRAG, ICML 2026 poster) likely projects pretrained embeddings into hyperbolic space.
- Guo et al. (CVPR 2022) verified: hyperbolic networks often underperform because embeddings drift to the Poincaré boundary, where gradients vanish. Clipping feature norms brings them level with Euclidean networks, and ahead on few-shot tasks. This is why M2’s head clips.
- Tree-like structure inside BERT: Hewitt & Manning (NAACL 2019) find that a linear map of ELMo and BERT vectors makes squared distance track parse-tree distance and squared norm track depth; a Poincaré probe (ICLR 2021) recovers syntax trees better than Euclidean probes verified.
- What ours adds. HiT has no exact addresses as input features and no certified heads. M2 adds both: the address enters as the hierarchy factor, and outputs can be snapped and assigned to tree words by exact decoding.
- Costs. 1–10M parameters on CPU in Phase 6, then BERT-mini and BERT-small on a GPU in Phase 7. Training happens in Python. transformers.js (Apache-2.0) can run a BERT-class encoder in the browser through ONNX Runtime, for inference only; it does not train verified.
- Needs. net BabyLM comes from Hugging Face (osf.io’s file downloads are blocked), and HiT’s models and data from huggingface.co and zenodo.org, all reachable since the network was widened in Phase 1. gpu for the Phase 7 scale. Its claims are registered by amendment after the Stage-1 gate; the roadmap’s first candidate, the hierarchy factor on BLiMP, is predicted to be a null result within ±1 point.
Baselines
Comparators that run in the ladder, or whose published numbers are cited with their grade. They are what M1 and M2 must match, beat, or honestly lose to.
B1 · Learned static hyperbolic embeddings
- What it is. One learned vector per node, trained so that a node’s neighbours in the hierarchy sit close to it, in hyperbolic or flat space. This is stage F.
- Model. Nickel & Kiela (NIPS 2017) used the Poincaré ball and beat Euclidean embeddings on WordNet nouns, most clearly in low dimensions. Nickel & Kiela (ICML 2018) found learning in the Lorentz model substantially more efficient. Both verified. Training uses Riemannian SGD: take a gradient step in the tangent space, then map it back onto the curved space.
- Evidence. Best Poincaré WordNet noun-closure reconstruction MAP 0.87 at 200 dimensions (Nickel & Kiela 2017, Table 1 of arXiv:1705.08039v2) verified. At 5 dimensions, Table 1 (reconstruction) gives Poincaré 0.823, Translational (a TransE-style score) 0.517 and Euclidean 0.024 verified.
- Role here. L6 trains Lorentz against Euclidean at matched dimension on the WordNet mammal closure (pinned: 1,182 nodes, 6,542 edges). Dimensions 2, 3 and 50 decide; 5 and 10 are reported. L6 also asks whether the exact H³ tree’s graph-edge MAP reaches learned Lorentz at d = 3, and whether learned Lorentz beats the exact H³ tree on closure MAP. Phase 4 decided it: (a), (b) and (d) are supported, as predicted, and (c) is supported against its registered prediction of refuted (Explainer 12).
- Costs. here. The recommended trainer is a dependency-free numpy float64 reference, because the PyPI torch wheel brings the CUDA libraries and download.pytorch.org is blocked here. geoopt (Apache-2.0) is an optional cross-check; Phase 4 could not install torch through the proxy and did not run it. A mammal run is estimated at about a minute, and the roughly 290 runs with tuning at 1–5 hours (estimates, not measured). Measured in Phase 4, four runs at a time on this container: from 30 s a run (Euclidean, d = 2) to 350 s (Lorentz, d = 50); registered tuning, 240 runs, took 6,089 s, and the decisive run, 100 trained tables with the floors, the exact arm and the controls, 3,051 s: about 2.5 hours (Explainer 12).
- Side baselines. Poincaré GloVe (Tifrea, Bécigneul & Ganea, ICLR 2019; LGPL-2.1) is reported only if its code runs. facebookresearch/poincare-embeddings is CC-BY-NC 4.0 and archived since 21 October 2025, so it is a protocol reference only and is not redistributed.
Floating-point limits cause NaNs and lost accuracy in hyperbolic models. Mishne et al. (ICML 2023) verified found that in float64 the Poincaré ball represents points more accurately, while the Lorentz model is better for optimisation. Lorentz points far from the origin carry large representation error (Yu & De Sa 2019). The published remedies: integer tiling; multi-component floats in the upper half-space (NeurIPS 2021); floating-point expansions (HypFPE, ICML 2025); clipping feature norms (Guo 2022); re-projecting onto the hyperboloid (HypLoRA); bounding norms (HiM).
This design splits the job. Learned hyperbolic tables train in the Lorentz model in float64 (the Euclidean ones in flat space); all are snapped to a 2−32 grid and then ranked exactly (L4(c)). Addresses and their points never pass through floats, except in the float control of L4(b). Pictures use the Poincaré model. Floats lose deep tree points: Engram’s spike B decoded float placements 50/50 to depth 12, 36/50 at depth 14 and 0/50 at depth 18.
B2 · Published combinatorial embeddings
- What it is. The numbers of Sala et al., HS-DTE and Yu & De Sa, the same family as M1.
- Model. Combinatorial tree embeddings: Sarkar’s in H², Sala et al.’s in two or more dimensions, HS-DTE’s in any dimension; Yu & De Sa store the points as an integer tiling with bounded error. Evidence verified at abstract level, as under M1.
- Role here. Cited, not rerun, unless HazyResearch’s
comb.jl(Julia, with a precision flag; licence not shown) becomes runnable. We expect to lose on MAP, because fixed branching adds virtual letters where Sala’s construction handles any degree. We expect to win on integer addressability, invertibility (the word decodes back to the synset) and compatibility with the LPS tree. L5 predicts the exact H³ tree’s noun graph-edge MAP falls below 0.989 and reports the value. - Two protocols, never equated. Graph-edge reconstruction (Sala-style) counts only a node’s tree parent and children as neighbours. Closure reconstruction (Nickel–Kiela) counts all ancestors and descendants. A number from one is never compared with a number from the other.
B3 · HiT on MiniLM
- What it is. The literature comparator for M2’s hierarchy head, scored by HiT’s multi-hop protocol (train on one-hop edges, test transitive ones) and mixed-hop protocol (an incomplete hierarchy, possibly with unseen entities), with random and hard negatives.
- Model. A Poincaré ball on a MiniLM encoder. On all-MiniLM-L12-v2 its WordNet multi-hop F1 is 0.903 (random negatives) / 0.908 (hard) (Table 2 of arXiv:2401.11374v4) verified; the often-quoted 0.916 / 0.899 is HiT on all-mpnet-base-v2. On SNOMED CT mixed-hop prediction (appendix Table 8, random negatives) the F-score rises from 0.297 (base all-MiniLM-L12-v2) to 0.888 (fine-tuned) to 0.954 (HiT) verified; with hard negatives, 0.213 / 0.743 / 0.899.
- Needs. net huggingface.co and zenodo.org, both blocked in Phase 0 and reachable since the network was widened at the start of Phase 1; it runs in Phase 7, on a GPU. Until then HiT’s splits are rebuilt from WordNet here, and any comparison with HiT’s published numbers is labelled indicative.
B4 · Euclidean BERT at matched size
- What it is. Ordinary BERT models of the same size, the Stage-2 control at matched parameters, tokens and FLOPs. Model: none, they are flat.
- Evidence. GLUE test scores from the google-research/bert README verified: BERT-Tiny (2 layers, width 128) 64.2, BERT-Mini (4/256) 65.8, BERT-Small (4/512) 71.2, BERT-Medium (8/512) 73.5. ELECTRA-Small (14M parameters) scores 77.4, about 4 days on one V100 verified.
- Needs. net gpu Phase 7.
Controls built to fail
A control keeps everything the same except the one thing a claim credits. Two kinds appear. A must-fail check tests the metric itself: if it does as well as the real arm, the metric cannot see what the claim is about, and the test is void. A comparison arm is the other side of the claim: if it does as well, the claim is refuted or inconclusive.
| Control | What it is | Expected | If it ties the real arm | Claim |
|---|---|---|---|---|
| Shuffled addresses | The same tree, with words permuted among synsets of equal word length (20 permutations on mammals, 5 on nouns). | Below the exact H³ tree, with the bootstrap interval of the difference above 0. | Void: the metric is blind to hierarchy. | L5 |
| Content-hash addresses | The first 8 bytes of SHA-256 of the first lemma, written as a 28-letter tree word by engram_core.encode. Closeness is then 5-adic and hash-like, not semantic. | Below H³, as above. | Void. | L5 |
| Curvature 0, F | The same words, directions and step rule in R² or R³ with unit edges. | Loses: it collides on its lattice. | L5 refuted or inconclusive. | L5 |
| Curvature 0, F* | A flat fractal tree whose edges shrink by a factor 1/4 per level. It is injective: sibling subtrees stay apart. | Loses: it ranks every descendant ahead of the parent. | L5 refuted or inconclusive. | L5 |
| Learned Euclidean at matched dimension and bits | The B1 trainer at curvature 0, reported dimension by dimension; bits compared as letters × log2 5 against d × 33 snapped bits. | Loses by at least 0.2 closure MAP at d = 2 and 3; draws within ±0.02, or wins, at d = 50. | L6(a) refuted or inconclusive. | L6 |
| Float against exact ranking | The same exact points, converted to float64 before ranking. Float32 is the must-fail check on the test itself. | At least one inversion of an exactly ordered pair: float ranking is not certifiable at WordNet depth. | No inversion refutes L4(b); no float32 inversion voids it. | L4 |
| Byte-level encoder | A tiny ByT5-style encoder that reads raw bytes, with no vocabulary and no addresses. | A matched-parameter floor for any address-factor claim. | Registered with those claims, by amendment. | Stages 3–4 |
| engram/1 as an id | Engram’s frozen 64-bit key used as a token id, on WordNet’s 147,306 lemma types. | At least 1% of types share a key with another type; 2–10% predicted. | Below 1% refutes L2; a SHA-256-prefix key that collides voids it. | L2 |
The table has eight rows because curvature 0 has two forms, F and F*; they count as one control in the role tally.
Bansal & Benton (Insights from Negative Results, 2021) verified (the abstract and Table 1 of arXiv:2109.07488v1) found that, in their own matched reproduction, Euclidean embeddings with 50 or more dimensions match or beat Poincaré and Lorentz embeddings on WordNet nouns. So a hyperbolic advantage at low dimension says nothing about high dimension, and every hyperbolic claim here carries a matched Euclidean arm at the same dimension: a learned one for learned embeddings (L6), and the same construction at curvature 0 for the exact tree (L5). L6(b) registers the expected vanishing at d = 50 as a prediction in its own right. On the mammals Phase 4 found the same: at d = 50 learned Euclidean was ahead (closure MAP 0.99997 against 0.968), which (b)’s rule counts as support (Explainer 12).
ByT5 (TACL 2022) verified runs a nearly unchanged T5 on raw bytes, is competitive with token-level models, and is more robust to noise and spelling. A model that needs no tokenizer and no hierarchy is the fairest floor for a claim that addresses help.
engram/1 fails as an id for a known reason, recorded in Engram’s README and in Explainer 1: the key keeps only the parity of each letter’s count. Its 26 letter grids are independent over GF(2), so the key has rank 26 and at most 226 of the 264 keys can occur. LISTEN = SILENT, and numbers and words in which every letter appears an even number of times land on the zero key. L2 measures how often that happens on real vocabulary, beside a 64-bit SHA-256-prefix key that must show no collisions at all.
Later, by amendment
Eight routes that are worth doing but not now. Each is registered by a dated amendment to the preregistration before any run, with its own controls.
L-a · Fully Lorentz encoders
- What. Encoders whose every layer (linear maps, attention, normalisation) works on the hyperboloid, instead of a flat encoder with a hyperbolic head.
- Evidence. HyboNet (Chen et al., ACL 2022) verified: Lorentz linear and attention layers that beat Euclidean baselines with fewer parameters on knowledge graphs, network embedding and translation; no licence file visible. Hypformer (KDD 2024, MIT) verified: graph benchmarks only, no text tasks. The Hyperbolic Pre-Trained Language Model (TASLP 2024) likely: a Lorentz BERT claimed to improve on GLUE, SuperGLUE and extractive QA tasks, but the survey could not see its numbers (secondary summaries disagree on the task count); a secondary survey says it runs about 1.3× slower than BERT. HELM (NeurIPS 2025, MIT) verified: claimed as the first billion-parameter fully hyperbolic LLMs; the abstract’s “up to 4%”, which the paper does not define, matches only its largest single-benchmark gain, about 4 points on OpenbookQA at about 120M parameters; at 1B (one run each) its gains over a DeepSeek-style baseline are +2.3 points on MMLU, +1.0 on ARC-Challenging and +1.0 on average, all near chance accuracy (Table 1 of arXiv:2505.24722v2) verified.
- Costs. gpu only. All of them would come through HyperCore (MIT), which pins torch 2.6 with CUDA 12.4. Phase 7 holds an exploratory HyperCore arm.
L-b · Product and mixed curvature
- What. Embeddings in a product of several spaces, each with its own curvature, beyond the single Hd factor M2 already feeds the encoder.
- Evidence. Gu, Sala, Gunel & Ré (ICLR 2019) likely report 32.55% lower distortion on a Facebook graph, and a 50-dimensional product of hyperbolic spaces that improved word similarity and analogy. HELM-MiCE mixes experts of different curvature. An ICLR 2026 paper argues that product manifolds have limited expressive power.
- Needs. It would extend the Stage-2 encoder, so it waits for M2 and inherits its needs.
L-c · Hyperbolic tokenization: an open gap
- What. Place the tokenizer’s own hierarchy in H² or H³. BPE (byte-pair encoding) builds a vocabulary by repeatedly merging the most frequent adjacent pair, so its merges form a forest of trees; morphology (stems and affixes) gives another tree. The variant A_merge would address
##pieces by the merge forest. In the registered design they get an orthographic trie instead, which carries spelling, not meaning. Model: the same exact trees in H² and H³. - Evidence. The survey found no published work that applies hyperbolic geometry at the tokenizer level. Two targeted searches found none secondary; that is an absence after searching, not proof. The nearest work: morphology-aware tokenizers (MorphPiece), a 2026 paper noting that BPE vocabularies form an implicit merge hierarchy, used there for pruning (not opened), and HyperHELM, which embeds a codon hierarchy for mRNA language models with HS-DTE.
- Why later, and how it would be judged. Intrinsic tokenizer scores predict downstream quality poorly: Ali et al. (2024), Schmidt et al. (EMNLP 2024) and Lotz et al. (ACL 2025); the first two verified, Lotz et al. likely. So a hyperbolic tokenizer cannot be justified by a nicer fertility or compression number; only a downstream test, Stage 3 or above, can. Placing the forest runs here; the test needs BabyLM.
L-d · Gradient-free training
- What. PolyStep, Hyperbolic Oracles’ gradient-free optimal-transport step, against MeZO, ZO-SGD and evolution strategies (ES), at an equal number of loss evaluations. These are gradient-free or zeroth-order methods: the update is estimated from loss values alone, with no backpropagation. Only for heads, curvature and adapters.
- Evidence. MeZO (Malladi et al., NeurIPS 2023, MIT) verified trains a 30B model on one A100-80GB, against about 2.7B with backpropagation, and brings OPT-13B within 1% of Adam on 7 of 11 tasks, but it relies on pretraining and prompts. ZO-Bench (ICML 2024) benchmarks the zeroth-order family. ES at scale (Qiu et al.) fine-tunes billion-parameter LLMs with all parameters, and beats PPO and GRPO by its authors’ account. None of these pretrains from scratch: the literature fine-tunes with them.
- Our own record. Hyperbolic Oracles’ C1, “the hyperbolic step beats the same step at curvature 0”, was REFUTED: 0 cells supported, 12 refuted, 1 inconclusive, of 13. On Hyperbolic Oracles’ benchmark B4 (test accuracy; not this page’s B4), hyper3 scored 0.325 against flat3’s 0.334 (the results). The largest model PolyStep has trained has 48 parameters. So here PolyStep only tunes heads of up to a few hundred parameters, in the browser.
L-e · Quaternion and hypercomplex layers
- What. Layers whose weights multiply as quaternions or learned hypercomplex numbers, which share parameters. A parameter-efficiency ablation, orthogonal to this design; no hyperbolic geometry is involved.
- Evidence. verified: the Quaternion Transformer (Tay et al., ACL 2019), about 75% fewer parameters with no significant loss; PHM layers (ICLR 2021), about 1/n of the parameters; Compacter (NeurIPS 2021), adapters training 0.047% of parameters that match full fine-tuning on GLUE.
- Needs. An ablation of the Stage-2 encoder, so it waits for M2.
L-f · Distilling BERT into a hyperbolic student
- What. Train a hyperbolic model to imitate a trained Euclidean BERT.
- Evidence. No BERT precedent was found. The closest, HyFL-CLIP, distils a Euclidean CLIP teacher into a Lorentz student by matching similarity distributions; it was not verified secondary. Model: a Lorentz student. The design gives it no cost estimate; it would need a trained BERT teacher, and none is pinned here (huggingface.co is reachable since Phase 1, but only BERT’s vocabulary, V1, has been pinned).
L-g · Brown-cluster addresses
- What. Brown clustering groups words bottom-up by how they co-occur, giving a tree of word clusters learned from text. Mapped to tree words, it would be the cheapest learned hierarchy, and it needs no gradients. It runs here on CPU. ours
L-h · Huffman-5 virtual letters
- What. A different rule for nodes with more than five children: like a Huffman code, give heavy subtrees shorter words and light ones longer words, instead of the fixed-length digits of L3. An address variant on the same exact tree, registered by amendment if it is pursued. ours
Rejected
Seven ideas that a survey, an earlier design or the existing systems suggested, each checked against the code or the literature. Each fails at the job it was suggested for. Several stay in use for a smaller job.
| Idea | Why it fails | What we do instead |
|---|---|---|
| engram/1 as a token id or semantic feature | It is rank-26 letter-count parity, and its own README says so: anagrams collide, numbers sit on the zero key, and Engram’s E9b found Hamming recall at chance for one-letter substitutions (E9). | Computed only as an Owl-compatibility checksum layer, off by default for corpora; L2 measures its collisions. Token ids come from the pinned WordPiece vocabulary (Explainer 3), and 59- and 64-bit address ids from a bijective codec (the address). |
| GROWTH/1 epochs as corpus growth | extendBiochain regrows the parent text plus the new text through growBiochain, which throws above 60,000 bytes (BioChain-AI biomesh.js:279-281, 388-400). So a whole lineage is capped at 60 KB. | Independent shards under an RFC 6962 manifest. WordNet’s noun synsets alone, one line each with lemmas and gloss, make 148 shards of at most 60,000 B (Explainer 4). |
| A Form nibble as a namespace | Form is a 4-bit field of the SHD-CCP word. There are at least eight incompatible 64-bit layouts, and every one of the 16 values is already claimed by some consumer. | Record kinds above the word (Explainer 2; layout census). |
| Hugging Face dataset fingerprints as provenance | They use XXH64, a fast 64-bit non-cryptographic hash, and fall back to a random value when anything cannot be hashed verified. Good for caching, not for tamper evidence. | SHA-256 commitments in typed records (Explainer 4). |
| LPS mod-q reduction as integrity | Reducing a tree word mod q lands on a vertex of the X(5,q) graph. Hash functions built on such graphs have published collision attacks (Tillich & Zémor 2008; Petit, Lauter & Quisquater 2008). A planning check also found girths of only about 8–9 for q ≤ 29, so distinct vertices are guaranteed only for 3–4-letter prefixes. | Used as a shard key only (L7); integrity comes from SHA-256 and the manifest. |
| The float chiral weave as a cross-language identity | shdccp/2’s weave is computed in floats (Math.hypot, toFixed(12)), a determinism risk between JavaScript and Python. | Kept only for shdccp/2 compatibility; the exact norm-5 weave, of norm exactly 528K, carries identity (L9, Explainer 4). |
| Engram’s register clock as a positional encoding, and its coupler as a segment binding | The clock is bookkeeping, not position meaning. The coupler gives integrity only and adds nothing to modelling. | Prefix embeddings of the address, a tree positional encoding in the style of Shiv & Quirk (NeurIPS 2019) (Explainer 3). |
A Merkle root is a single hash that commits to a whole list: items are hashed, then pairs of hashes are hashed again up a binary tree. An RFC 6962 tree tags leaves and nodes differently, splits at the largest power of two, and never duplicates an odd node. shdccp/2’s rule does duplicate it, so an odd-length list and the same list with its last item repeated get the same root (the CVE-2012-2459 class); L9 keeps that weakness as a control that must show it.
The position
Build the route only this stack can decide, here and now; enter the literature’s best route only through a gate; and keep every hyperbolic claim next to a flat one.
- M1 is decidable here, now. It runs on CPU and in the browser, needs no GPU and no blocked host, and directly answers one question: does exact LPS-tree geometry carry real linguistic hierarchy? Its claims (L3–L7) are registered; Phase 1 decided L3 (Python side), L5 and L7 (Explainer 9), Phase 2 L3(f) and L4(a, b) (Explainer 10), and Phase 4 L6 and L4(c) (Explainer 12), all supported, L6(c) against its registered prediction; and it produces the certified addresses, points and records every later stage reuses.
- The Stage-1 gate. All of L1–L11 are decided and reviewed with the project’s owner before any encoder code is written.
- M2 after the gate. The best-supported route in the literature, on CPU at 1–10M parameters, then on a GPU, each claim registered by amendment first, with B3 and B4 as comparators and the byte-level encoder as a floor.
- Later routes by amendment. L-a to L-h wait for evidence from the earlier stages or a GPU (the wider network arrived at the start of Phase 1).
Hyperbolic geometry clearly helps on explicit hierarchies: WordNet, ontologies, hypernymy, and low-dimensional embeddings. On general NLP it helps modestly or not at all. Dhingra et al. (2018) found gains on some tasks but not all; Leimeister & Wilson (2018) found hyperbolic skip-gram promising in low dimensions but not clearly better verified. The Hyperbolic PLM’s gains could not be checked, and HELM’s “up to 4%” matches only its largest single-benchmark gain, about 4 points on OpenbookQA at about 120M parameters; at 1B its gains are +2.3 points on MMLU, +1.0 on ARC-Challenging and +1.0 on average, near chance verified.
Our own record says the same. Hyperbolic Oracles’ C1 was refuted, 0/12/1, with hyper3 at 0.325 against flat3’s 0.334 on its benchmark B4; and engram/1, the one language key the stack had, is letter-count parity. So encoder-level gains are predicted small or null at this scale. H² and H³ are far below a language model’s width (128–768), so they serve as hierarchy factors, heads and views, never the main embedding. The bet is on representation, where low-dimensional trees favour curvature, and on certification: exactness, invertibility and results identical across engines.
The tracks that run each route are in Explainer 6; the ladder that scores them, with every baseline and its grade, is in Explainer 7; the claims and spikes are in Explainer 8 and on the claims ledger.