A recent experiment demonstrates that Large Language Models (LLMs) can significantly improve code quality through iterative prompting. Max Woolf tested whether repeatedly asking an LLM to optimize code would yield better results. Using Claude 3.5 Sonnet, the experiment showed performance improvements of up to 100 times compared to initial implementations.
The test focused on a specific programming challenge: finding the difference between the smallest and largest numbers whose digits sum to 30 in a list of one million random integers. Through successive optimization requests, the LLM implemented increasingly sophisticated solutions, incorporating parallel processing, vectorization, and just-in-time compilation techniques.
Two different approaches were tested: casual prompting with simple “write better code” requests, and structured prompt engineering with specific optimization guidelines. The structured approach proved more effective, though it occasionally introduced subtle bugs requiring human intervention to fix.
Woolf concludes that while LLMs won’t replace software engineers, they can serve as valuable tools for code optimization when properly guided. The results also highlighted the importance of prompt engineering in achieving optimal results from LLM interactions.