The domestically developed artificial intelligence (AI) semiconductor neural processing unit (NPU) is handling core functions in four AI services of SK Telecom, processing more than 4 billion tokens (the data processing unit of AI) per day. It is handling about 14 million user requests daily, ranging from call summaries to phishing text detection. As a domestic chip takes charge of key computations for AI customer services, assessments are emerging that the prospects for building an independent Korean AI ecosystem have improved.

● First step toward commercialization of ‘sovereign AI chip’

According to SK Telecom and AI semiconductor startup Rebellions on the 20th, Rebellions’ NPU “ATOMMAX” is being used for key computations in four SK Telecom services: call summarization and speech synthesis for its AI agent “A.” (A.Dot), its AI contact center, and its Scam Buster Guard (Scamvanguard) service. After first being used for A.’s call summarization in December last year, the application scope was expanded in June and July this year.

The NPU is a chip specialized for “inference,” in which a trained AI provides answers to user queries. Compared with graphics processing units (GPUs), which are used for both training and inference, NPUs can save power and cost in suitable inference workloads.

This is why SK Telecom allocates workloads between GPUs and NPUs according to service characteristics. Since June this year, the company has applied ATOMMAX to functions that summarize calls in a single line and read out AI responses sentence by sentence. It subsequently deployed the domestic chip in its AI contact center, which organizes and searches consultation content, and in Scam Buster Guard, which filters out phishing text messages.

The increasing use of domestic AI chips also means that there is now an option to reduce dependence on foreign chips. Dario Amodei, CEO of Anthropic, who recently advocated “slowing down AI” to curb the development pace of advanced AI models, has also called for blocking sales of advanced AI chips to China. With such concerns that access to chips and AI models could be cut off at any time, countries around the world are accelerating AI model development and infrastructure such as data centers, regardless of the slowdown debate. Yoo Hoi-joon, Dean of the KAIST Graduate School of AI Semiconductors, said, “If Korea relies entirely on foreign products, the entire AI ecosystem could be affected by changes on the other side,” adding, “Domestic AI semiconductors, which until now remained in the lab, have finally become actual commercial products.”

● Intensifying competition over AI chip sovereignty

To accelerate the commercialization of domestic AI chips, follow-up chips must also pass verification milestones. On the 8th, the Dong-A Ilbo became the first Korean media outlet to visit Rebellions’ data center in Magok, Gangseo District, Seoul, where preparations are underway to commercialize the next-generation NPU “Rebel 100” (Rebel100). Noise levels in the third-floor server room reached 77.5 dB, requiring raised voices to hold a conversation. When an engineer inserted eight Rebel100 chips into a test server, the server and cards were recognized normally on the screen about 30 minutes later. The chips, equipped with high-bandwidth memory (HBM3E) for fast data transmission, were undergoing testing ahead of deployment to client companies.

As such verification and operational experience accumulate, the speed of field deployment increases. It took about five to six months for SK Telecom to first apply a domestic NPU to A. in December last year, but when adding new applications this year, the period shortened to two to three months.

In the global race to secure “sovereign AI” capabilities by building domestic AI foundations, the ability to develop and use chips has emerged as a key factor. An analysis by the Dong-A Ilbo of the “Sovereign AI Index” of the Center for a New American Security (CNAS) in the United States found that as of the end of June this year, Nvidia was supplying GPUs to 45% of 117 government-backed infrastructure AI projects worldwide. While countries tout their own AI, the chips that actually run these systems often come from foreign companies.

Kim Yong-seok, Endowed Professor at the Gachon University College of Semiconductors, said, “Korea has the capability to develop AI semiconductors, but is still lacking in follow-through when it comes to mass production and commercialization,” diagnosing that “there is a need for the public sector and large corporations to serve as initial customers so that domestic AI chip manufacturers can accumulate delivery and deployment experience.”

Popular News