The market for Nvidia’s H100 GPUs has gone from a shortage last year to a glut in 2024. According to Eugene Cheah, rental prices for the powerful GPUs have dropped from $8 per hour to less than $2. Cheah attributes this to a number of factors. For example, more companies are relying on fine-tuning existing open-source models, such as Llama 3, rather than training large language models themselves. These models also require significantly less computing power, reducing the need for large H100 clusters. In addition, many startups that relied on smaller, specialized models have reduced their investment in their own training infrastructure. Adding to the oversupply, large AI companies like Meta are building their own data centers, reducing demand from public cloud providers. On top of this, there are delivery delays from last year and cheaper alternatives from AMD and Intel.