Schools and universities are increasingly relying on AI detectors to check student work for AI use. According to one survey, about two-thirds of teachers use such tools regularly. As Jackie Davalos and Leon Yin report in Bloomberg Businessweek, at that scale, even low error rates can lead to a large number of false positives.
The authors tested two leading services on 500 pre-ChatGPT Texas A&M University application essays. They found that 1-2% of the guaranteed human-generated texts were falsely marked as AI-generated. Students who speak English as a second language or who are neurodivergent seem to be particularly affected. The consequences range from zero grades to warnings. As a result, some educational institutions have already stopped using such tools, while others continue to do so. The situation leads to tensions between teachers and students, as well as costly accommodations for students.