AI context windows are much smaller than advertised, developer warns
AI companies advertise ever-larger context windows for their language models, but a developer argues that the usable portion is far smaller than the marketed figures suggest. Garrit Franke writes in his blog Garrit’s Notes that effective model performance begins to degrade around 100,000 tokens, regardless of whether the advertised limit is 200,000, one million, or …