Two UC Berkeley doctoral students have created an influential AI evaluation platform that has become the industry’s go-to resource for comparing chatbot performance. According to Miles Kruppa’s report in The Wall Street Journal, Anastasios Angelopoulos and Wei-Lin Chiang developed Chatbot Arena as a graduate project in April 2023, which now ranks over 170 AI models through user-based evaluations.
The platform lets users compare responses from two anonymous AI models and vote on their preference, generating rankings that major tech companies closely monitor. Major players including OpenAI, Google, and Meta actively participate in the rankings, with some companies even testing unreleased technologies through the platform. The system has collected two million votes to date and provides separate rankings for specific capabilities like coding and creative writing, while sharing 20% of its collected data with developers for analysis.
See also: How trustworthy is Chatbot Arena?