Domestic researchers have developed a technology that overcomes a chronic limitation of large language models (LLMs), which is that they provide answers based on outdated information.

On the 14th, KAIST announced that a research team led by Professor Hwang Eui-jong of the School of Electrical Engineering, in collaboration with Microsoft Research, has developed an evaluation and diagnostic technology that automatically detects the “temporal errors” of LLMs.

Conventional AI evaluation has only checked whether AI produced the correct answer. For this reason, it has been difficult to properly filter out errors in situations where information changes over time. This is why cases repeatedly occur in which ChatGPT, when asked “Who is the minister who took office last month?”, answers with the name of the person from a year ago, or when asked “What is today’s KRW–USD exchange rate?”, responds with a figure from several months earlier, producing answers detached from current reality.

To address this, the research team applied the concept of a “temporal database” to AI evaluation for the first time. Based on how information changes over time, they implemented a system that automatically generates 13 types of time-related questions purely from data, without human-crafted items. The key feature is that the entire process—from question generation to deriving and verifying the correct answer—is automated, so that simply updating the data automatically refreshes the corresponding questions and correct answers.

In particular, this study goes beyond checking whether the answer is correct and separately verifies whether the dates or time points used by the AI in its explanation are also accurate. As a result, it can more effectively detect “temporal errors” in which the answer appears correct on the surface but is actually based on outdated information.

Popular News