On the 13th of this month (local time), global accounting and consulting firm KPMG urgently withdrew an internal report on the use of artificial intelligence (AI). It belatedly emerged that, due to so‑called “hallucinations” in which AI plausibly fabricates information, false statements had been mixed throughout the report. The report contained inaccurate claims about how several institutions, including Swiss bank UBS, were using AI. A report that was supposed to analyze AI ended up marred by fake information generated by AI itself.

As such AI hallucination incidents continue, companies that have introduced generative AI into their operations are focusing as much on “AI that filters well” as on “AI that creates well.” As AI increasingly penetrates core tasks such as code development, customer support, and data analysis, the reliability of its answers has become crucial. To raise productivity while preventing side effects, “AI verification solutions” that double‑check answers are becoming a key pillar of enterprise AI infrastructure.

● “AI checkpoints” attached to big tech cloud services

Global big tech companies such as Google and Microsoft (MS) are quickly adding verification functions to their enterprise AI services. Google Cloud, after introducing the security tool “Model Armor” that controls user prompts and AI responses, recently added a function that filters out risk factors in real time as the AI is generating answers. Both functions immediately block malicious prompt injections, personal data leaks, and hate content, helping companies use AI more safely in real‑world environments.

MS also provides an AI safety check function called “Prompt Shield” on its Azure cloud service. The questions or commands that users input into AI are called “prompts,” and Prompt Shield prevents safety mechanisms from being disabled by detecting and blocking sophisticated prompt manipulation designed to elicit prohibited responses or by finding hidden malicious instructions embedded in documents or web pages. It effectively serves as a checkpoint placed in front of AI. In January this year, MS updated Azure documentation to highlight its ability to detect harmful content generated by AI.

● Emerging as an essential safeguard for advertising and brands

The advertising and content market has long been identified as a sector where such AI verification systems are urgently needed. When corporate advertisements appear alongside low‑quality “AI slop” content mass‑produced by AI, brand trust can be immediately undermined. In fact, a survey conducted by global security company CHEQ together with media company Magna and IPG Media Lab found that when ads are displayed next to inappropriate content, brand trust can fall by up to 60%.

Startups are also joining the fray. AI startup Markvision operates “MarkAI,” an AI technology that scans online shopping malls and social network services (SNS) to identify counterfeit products, unauthorized use of images, and brand impersonation. Video understanding AI company Filer is targeting overseas markets with “AiD,” a multimodal AI‑based brand safety solution that filters harmful content by analyzing video, audio, and subtitles together. A Filer representative stated, “If unverified AI is used indiscriminately and an incident occurs, the cost of remediation can be enormous,” adding, “AI verification tools are becoming essential safeguards, much like cybersecurity.”

Choi Kyung-jin, president of the Korean Association for AI and Law, said, “Up to now, AI performance has been what divided the market, but as AI becomes more advanced, security and verification are emerging as even more important themes,” and predicted, “Reliability and safety will determine AI competitiveness going forward.”

Popular News