KAIST announced on the 27th that a research team led by Professor Kim Hyun-woo of the School of Computing, in joint research with a Korea University team, has developed a technology that can transplant learned knowledge between different artificial intelligence (AI) models.

KAIST explained that using this technology, knowledge can be transferred without having to retrain a new AI model from scratch with massive amounts of data and cost, much like transferring contacts and photos stored on an old smartphone to a newly purchased one.

The core of this technology is transferring the “adaptation experience” accumulated by one AI during training to another AI model. In the past, in order to pass on the “senior AI’s” experience to a “junior AI,” it was necessary to repeat for the junior AI the training process that the senior AI had undergone. This was because even a slight change in model architecture made it difficult to fully leverage previously accumulated experience.

However, by using the research team’s technology, knowledge can be delivered as if transplanting part of the artificial neural network in which the senior AI’s experience is accumulated into the junior AI. According to KAIST, this method enables precise knowledge transfer even between AI models with different architectures and sizes.

Professor Kim said, “If this research is extended, it will be possible to significantly reduce the cost of ‘post-training,’ which has had to be repeatedly performed whenever a new large-scale language model appears,” adding, “It will also make possible ‘knowledge patches’ that can easily add domain-specific expertise.”

Popular News