Gary Marcus, a prominent AI expert, argues that pure scaling of AI systems without fundamental architectural changes is reaching a point of diminishing returns. He cites recent comments from venture capitalist Marc Andreesen and editor Amir Efrati confirming that improvements in large language models (LLMs) are slowing down, despite increasing computational resources. Marcus warns that …
Two recent studies provide the first empirical evidence that having AI models debate each other can help a human or machine judge discern the truth, reports Nash Weerasekera for Quanta Magazine. The approach, first proposed in 2018, involves two expert language models presenting arguments on a given question to a less-informed judge, who then decides …
Geoffrey Hinton, Jensen Huang, and Fei-Fei Li were instrumental in launching the deep learning revolution, despite facing skepticism from colleagues, Timothy B. Lee writes. Hinton spent decades promoting neural networks and developed the backpropagation algorithm for training them efficiently, as detailed in Cade Metz’s book “Genius Makers.” Huang, CEO of Nvidia, recognized the potential of …
Researchers have introduced OmniGen, the first diffusion model capable of unifying various image generation tasks within a single framework. Unlike existing models like Stable Diffusion, OmniGen does not require additional modules to handle different control conditions, according to the authors Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, et al. The model can perform text-to-image …
Google subsidiary DeepMind has introduced SynthID-Text, a system for watermarking text generated by large language models (LLMs). By subtly altering word probabilities during text generation, SynthID-Text embeds a detectable statistical signature without degrading the quality, accuracy, or speed of the output, as described by Pushmeet Kohli and colleagues in the journal Nature. While not foolproof, …
Researchers from UC San Diego and Tsinghua University have developed a method that improves artificial intelligence’s ability to understand when to use external tools versus relying on built-in knowledge, similar to how human experts approach problem-solving. Using a relatively small language model with 8 billion parameters, the team achieved a 28% improvement in answer accuracy …
The rise of AI could lead to a 3-12% increase in global electronic waste by 2030, amounting to an extra 2.5 million metric tons annually, according to a study by researchers at the Chinese Academy of Sciences and Reichman University in Israel published in the journal Nature Computational Science. The analysis, based on industry investment …
The research paper “Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worse” investigates the effectiveness of chain-of-thought (CoT) prompting in large language and multimodal models. While CoT has generally improved model performance on various tasks, the authors explore scenarios where it may actually hinder performance, drawing parallels from …
A new study by researchers from Technion, Google Research, and Apple reveals that large language models (LLMs) have a deeper understanding of truthfulness than previously thought. The researchers analyzed the internal representations of LLMs across various datasets and found that truthfulness information is concentrated in specific response tokens, VentureBeat reports. By training classifier models on …
Researchers at XJDR have developed a new technique called Entropix that aims to improve reasoning in language models by making smarter decisions when the model is uncertain, according to a recent blog post by Thariq Shihipar. The method uses adaptive sampling based on two metrics, entropy and varentropy, which measure the uncertainty in the model’s …