Meta has developed a new technology called scalable memory layers to improve knowledge retention and reduce hallucinations in large language models. As reported by Ben Dickson in VentureBeat, this innovation adds parameters to AI models without increasing computational demands. The memory layers work by using sparse activations and key-value lookup mechanisms, making them more efficient than traditional dense layers for storing factual knowledge. In testing, a 1.3-billion-parameter memory model matched the performance of much larger models while using significantly less computing power. Meta’s researchers demonstrated that these memory layers can be distributed across multiple GPUs without performance loss. The technology shows particular promise in factual question-answering tasks, where it approaches the capability of models trained with ten times more computing resources.