Why tiny AI models could outperform giant chatbots

In 2019, Nigerian entrepreneur Adebayo Alonge faced a technical failure during a product demo in Cape Town: his AI-powered pill scanner needed to reach a data center 14,000 kilometers away, and the connection was far too slow. David Berreby reports for IEEE Spectrum that this incident pushed Alonge to shrink his AI model within hours so it could run directly on a smartphone, without any internet connection.

That moment marked the beginning of a broader shift toward what experts call “small AI”: compact models with at most a few billion parameters that run on phones, Raspberry Pis, or low-power sensors. Unlike massive large language models, they require little electricity, no cloud access, and modest hardware.

According to a World Bank report cited in the article, only 0.7 percent of internet users in the world’s poorest countries have used ChatGPT, compared to a quarter of users in wealthy nations. World Bank president Ajay Banga said at the World Economic Forum in Davos that most countries outside India and China lack the resources for large AI systems. Small AI, he argued, can still deliver essential services.

Examples from around the world

• A drone developed at India’s Vellore Institute of Technology detects diseased cashew plants using onboard processing.
• A $50 Arduino device in Brazil analyzes sensor data to locate standing water where mosquitoes breed.
• Alonge’s phone-based scanner authenticates medication without any internet connection.

Brazilian professor Marcelo José Rovai explains that many small models are created by “pruning” or “distilling” larger ones, keeping only the parameters needed for a specific task. Improving smartphone hardware and open-weight models such as Google DeepMind’s Gemma 4 make this process increasingly accessible, he adds.

Still, experts caution that small AI cannot fix deeper structural problems. Reliable electricity, functioning supply chains, and technical education remain necessary for the technology to have a lasting impact, Alonge says.