On the 9th of this month at Seoul Girls’ Middle School in Mapo-gu, Seoul, about 20 first-year students taking a Korean language class in the library sat with the educational smart device “D-Bot” alongside their textbooks as they took a unit test. Instead of handing out test papers, teacher Sung Ye-eun displayed login IDs and passwords on a large monitor. When Sung said, “Please press the ‘take test’ button now,” the questions appeared on each smart device, and the students began entering their answers into the devices.

The unit test was conducted using the “AI Descriptive and Essay-Type Assessment Support System” (Chaeum-Ai), developed by the Seoul Metropolitan Office of Education. It is an AI-based assessment system developed by the office in August this year and has been used since September at 66 elementary, middle, and high schools in Seoul. Intense debate continues over how AI scores descriptive and essay-type questions and whether accuracy and fairness in scoring can be ensured.

● AI scores based on criteria set by teachers

The core of this system is that human teachers and AI grade together. Once students submit their answers, a notification that the test has ended and the answers submitted by each student immediately appear on the teacher’s page.

The teacher creates in advance a scoring rubric that includes evaluation elements, detailed scoring criteria, and point allocations. The AI grades according to this rubric. For example, for a question requiring the joint use of passages “A” and “B,” a response that clearly uses both passages is worth 2 points, a response that uses only one is worth 1 point, and a response unrelated to the passages is worth 0 points. The AI calculates the score based on these criteria and then automatically aggregates the results.

The AI also provides descriptive feedback such as, “The problem-solving process is consistent and persuasive, based on conceptual understanding and logical exposition.” Both the scores and the descriptive feedback generated by AI can be corrected or supplemented by the teacher.

The system is in a pilot phase and is currently used only up to the stage where students enter answers into the device. The answers entered by Seoul Girls’ Middle School students that day were also graded directly by the teacher. From the second half of next year, the plan is to implement a system in which students write their answers on a PC or similar device and AI performs the scoring.

● “Like gaining one more grader”

The Seoul Metropolitan Office of Education plans to expand the use of descriptive and essay-type assessments. By using AI, it aims to increase consistency in scoring and reduce teachers’ workload. “Using an AI grading system is like gaining one more grader,” Sung said. “It not only improves work efficiency but can also enhance fairness compared with having a single person grade all answers.”

When AI grading results differ from the teacher’s judgment, the teacher’s judgment takes precedence. The system is also expected to fulfill the role of a so-called “red-pen teacher.” Currently, when grading dozens of descriptive answers, simply assigning scores can be burdensome, but with AI, it becomes possible to provide feedback on strengths and areas for improvement.

Concerns about AI grading remain. Choi Ah-jung, a first-year student at Seoul Girls’ Middle School, said, “I am not sure AI can grade flexibly when a problem is solved in a way different from how it was trained,” adding, “It is hard to trust its basis for judgment because we do not know what data the AI has been trained on for grading.”

Popular News