Models · Google DeepMind
Institute of Science Tokyo releases powerful Japanese-focused LLM with Gemma 2
Compiled by KHAO Editorial — aggregated from 1 outlet. See llms.txt for citation guidance.
★ Tier-1 Source
Institute of Science Tokyo together with National Institute of Advanced Industrial Science and Technology (AIST) is working to create large language models (LLMs) that excel in Japanese.
Key facts
- To put Gemma-2-Llama Swallow to the test, the team used 10 Japanese understanding and generation tasks, 10 English understanding and generation tasks, and the Japanese MT-bench to evaluate
- Institute of Science Tokyo will continue to refine Gemma-2-Llama Swallow following its initial launch in May 2025
- Since Gemma 2 already exhibited strong instruction-following and dialogue capabilities in Japanese, they were able to employ imitation learning for the instruction tuning of their model from Gemma 2
- Chart representing Gemma-2-Llama Swallow 27B IT v0.1 superior performance
Summary
Latest research efforts resulted in the creation of Gemma-2-Llama Swallow, a new LLM that delivers unparalleled Japanese language knowledge and performance, thanks also to Gemma’s high base-level proficiency in the language. The institute recognized that many of the world’s most-popular LLMs focus on western languages like English, and lack reliable utility in European languages, southeastern languages, and in this case, Japanese. The Swallow developer team began creating Japanese-focused iterations of popular models from Llama, Mistral, and Mixtral to varying degrees of success. “Gemma 2 already exhibited strong instruction-following and dialogue capabilities in Japanese,” said Naoaki Okazaki, professor at Institute of Science Tokyo. Chart representing Gemma-2-Llama Swallow 27B IT v0.1 superior performance.