Researchers at the University of Zurich and Cologne University are creating large language models trained solely on historical documents up to specific dates. The team, led by Daniel Göttlich, Dominik Loibner, Guohui Jiang, and Hans-Joachim Voth, describes their project on GitHub.
The models, called Ranke-4B, contain four billion parameters and are trained on 80 billion tokens of historical data. Five versions exist with knowledge cutoffs in 1913, 1929, 1933, 1939, and 1946. The training uses a curated dataset of 600 billion tokens of time-stamped text.
These models cannot access information beyond their cutoff dates because that information does not exist in their training data. A model trained on texts up to 1913 cannot discuss World War I because the war had not yet occurred.
The researchers explain that modern language models suffer from hindsight contamination. Even when instructed to roleplay historical perspectives, they know how events unfolded. Time-locked models embody their training data rather than pretending.
The team positions these models as tools for exploring historical discourse patterns and complementing traditional archival research. They acknowledge the models will reproduce problematic views from historical sources, including racism and antisemitism, which the researchers consider essential for understanding how such ideologies took hold.
The project plans to release all training data, model checkpoints, and code repositories publicly. The team is developing a responsible access framework for scholarly use.
