Limits RegistryPUBLIC RECORD
Limits RegistryPaLM was notable both for its dense (as opposed to sparse mixture-of-experts) 540-billion-parameter scale and for demonstrating that Google's Pathways system could efficiently train a single model across two TPU v4 Pods (over 6,000 chips) with high hardware utilization. Google reported PaLM showed 'breakthrough' performance on tasks requiring multi-step reasoning when combined with chain-of-thought prompting, exceeding average human performance on the BIG-bench suite as reported in its technical paper.
The question, scope, and sources behind this Registry record.
PaLM was notable both for its dense (as opposed to sparse mixture-of-experts) 540-billion-parameter scale and for demonstrating that Google's Pathways system could efficiently train a single model across two TPU v4 Pods (over 6,000 chips) with high hardware utilization. Google reported PaLM showed 'breakthrough' performance on tasks requiring multi-step reasoning when combined with chain-of-thought prompting, exceeding average human performance on the BIG-bench suite as reported in its technical paper.
Google's Pathways Language Model (PaLM), announced in April 2022, was a 540-billion-parameter dense transformer trained across two TPU v4 Pods, among the largest dense (non-mixture-of-experts) language models publicly documented at the time, and reported state-of-the-art results on the majority of the roughly 150 language benchmark tasks it was evaluated on.
Change a parameter to stress-test whether a proposed result is still inside the published specification. This is an audit aid, not a proof checker.
This record has no editable parameters. Read the formal question and assumptions before challenging it.
Current frontiers derived from accepted Claims.
An observed or demonstrated result; no opposing bound is implied.
The frontier is not sacred
Most progress starts with a disagreement that survives contact with evidence. If you can push the known lower bound up or pull the upper bound down, show us the work.
≥ when you have shown that at least this value is achievable.≤ when you have shown that anything above this value is impossible.No vibes. State the value, define the scope, and link the paper, proof, code, or reproduction that lets another person check it. Editors review every challenge before the public record changes.
Challenge this recordAssertions tied to evidence, attribution, and review.
The frontier as it changed over time.
Only accepted Claims matching the current specification contribute to the displayed bounds. Strict inequalities remain open; contradictory Claims require editorial review.
1 accepted Claim, with 1 linked evidence records.
Permanent ID limitsregistry.com/limits/LR-PALM-TRAINING-COMPUTE
No active verified bounties are linked to this Limit.
View verified bounty tracker ↗No accepted machine-checked reproductions are recorded for this Limit.
Limits Registry. LR-PALM-TRAINING-COMPUTE. PaLM — largest publicly documented dense model training compute of its era. 2026.