The field of generative artificial intelligence, especially in the field of large language models (LLMs), has seen significant progress in recent years. Models such as GPT-4, LLaMA and Gemini have set new standards, demonstrating impressive levels of sophistication and functionality. However, such advances mainly cover highly accessible languages such as English, Chinese, Japanese and Russian. Lack of resources for other languages creates a language gap in AI, which impacts the quality of user experience, reinforces bias, and contributes to the digital divide. On December 11 in Astana, the KAZ-LLM national language model was presented to the President of Kazakhstan Kassym-Jomart Tokayev. The model was developed under the leadership of the Institute of Smart Systems and Artificial Intelligence (ISSAI NU) in partnership with Beeline Kazakhstan and its IT company QazCode, as well as Astana Hub. The project is coordinated by the Ministry of Digital Development, Innovation and Aerospace Industry of the Republic of Kazakhstan (MCRIAP RK). The model is of strategic importance for the entire country as it solves the problem of the language gap with the help of AI. How was the KAZ-LLM model developed? KAZ-LLM from ISSAI is based on 15
0 billion tokens, carefully collected from publicly available sources in four languages - Kazakh, Russian, English and Turkish. This allows the model to demonstrate high accuracy and versatility, providing improved quality of text processing in various languages and contributing to improved translation. Tokens are minimal units of text, such as words, parts of words, or even individual characters, that AI uses to analyze and understand information. The interface and functionality of the KAZ-LLM model were developed taking into account the most advanced world standards, which confirms the high technological maturity and wide potential of the model. To evaluate its performance, comprehensive benchmarks with question-answer pairs were used, covering a variety of knowledge areas. The benchmark package included the following tests: - ARC (AI2 Reasoning Challenge) - testing scientific thinking through multiple choice questions. - GSM8K - assessment of the ability to solve problems in mathematics for primary school. — HellaSwag — testing sentence continuation logic. — MMLU (Massive Multitask Language Understanding) — tests knowledge in 57 different subjects. — Winogrande — health assessment
new meaning in ambiguous sentences. — DROP — testing reading comprehension and logical thinking skills. The partnership between Beeline and QazCode accelerated the development. Key partners in its creation were Beeline Kazakhstan and its IT company QazCode, combining efforts and experience in creating language models such as Kaz-RoBERTA, as well as in developing AI solutions for small language groups in collaboration with foreign partners. Support in the form of provided servers with computing power of 8 DGX H100 significantly accelerated the learning process and expanded the capabilities of the model. By comparison, it would take a regular computer several days to analyze an archive of 1 million photos. While 8 DGX H100 servers used for training ISSAI KAZ-LLM will cope with this task in just a few seconds. Based on these servers, the developers trained two versions of the model - with 8 billion and 70 billion parameters; data scientists from QazCode joined the process. Alexey Sharavar, CEO of QazCode “Our team actively participated in the development and training of the KAZ-LLM model. When creating LLM, developers and partners used modern machine learning technologies, such as
to PyTorch and Torchtune, and also took into account the experience of previous projects on adapting open source LLM architectures for the Kazakh language. Over the course of training, which lasted 50 days of continuous computing, the model improved its ability to understand context and provide a high-quality user experience. Testing has shown that the model successfully solves technical problems, taking into account the cultural and linguistic characteristics of the Kazakh language,” said Alexey Sharavar, CEO of QazCode. About the results and prospects of KAZ-LLM Researchers note that the project is an important milestone in Kazakhstan’s path on the global stage of artificial intelligence: “This model reflects Kazakhstan’s desire for innovation, independence and growth of its technological ecosystem. Our team has prepared two versions of ISSAI KAZ-LLM with 8 billion and 70 billion parameters, built on the Meta Llama architecture and optimized for high-performance systems and resource-constrained environments. Models released under CC-BY-NC license are available for non-commercial use on the Hugging Face website, promoting global academic and research collaboration. This way, developers will be able to download
b and run our model both on complex servers and on laptops,” said ISSAI Director, NU Professor Hussein Atakan Varol. It is expected that ISSAI Kaz-LLM will open up new opportunities for creating startups and innovative projects based on AI. In the future, it is planned to develop next-generation models that will integrate linguistic and visual data, which will significantly expand the capabilities of AI. Adding support for the model for other Turkic languages is also being considered, which will strengthen ties between Turkic-speaking communities. Kaspi Gold is now in Apple Pay The Kazakhstan IT services market has almost doubled Huawei Watch Fit 5 Pro: a premium gadget for managing stress and burnout in a big city From Excel to a digital and AI enterprise: how medium-sized businesses in Kazakhstan can build a unified SAP ecosystem Alla Zatsepilova, Bereke Bank: for a digital balance in caring for clients Asel Marchenko, NPK of the National Bank of the Republic of Kazakhstan: we are iteratively developing digital infrastructure Digital kings: Evgeniy Shcherbinin Bolat Basheev: we begin to discuss technologies before we transformed business processes Kaspi.kz became the general partner in the filming of the film “Armor of God: Ultimatum” with Jackie Chan
om Elcore became an authorized reseller of Anthropic
