Azerbaijani LLM Trains Fast on AWS Cuts Memory

Six weeks, one language, a 23 percent throughput boost.

Azercell Telecom LLC teamed with AWS Generative AI Innovation Center to train an Azerbaijani language model on Amazon SageMaker AI, tackling the dual challenge of a morphologically rich language and limited training data. The project aims to power telecom use cases and a customer facing chatbot, delivering a production ready framework that can be adapted to other low resource languages. The team reports that the framework ran on an ml.p5.48xlarge instance and achieved a 23% higher training throughput while cutting peak GPU memory usage by 58% through kernel level optimizations. In addition, a custom tokenizer yielded a 2x improvement in tokens per word, effectively doubling the model’s context window and enabling more text to fit into memory during training.

The effort rests on a three stage pipeline that feeds artifacts from one stage into the next. The team highlights this design choice as essential for building robust foundations in languages with limited data. Stage 1 centers on tokenizer development, where the team built an efficient Azerbaijani-specific tokenizer. They evaluated three approaches: baseline English optimized tokenizers, vocabulary extensions, and a custom monolingual tokenizer, to measure encoding efficiency with standardized metrics. The project, which builds on open source tools such as PyTorch, Hugging Face Transformers, and Liger Kernels, seeks a production-ready path on SageMaker AI that scales beyond a single experiment. The framework’s architecture is designed to let practitioners replace or extend stages as data and models evolve, a practical constraint for teams racing toward deployment in telecoms where uptime and latency matter.

The collaboration shows how to bridge the gap between research style experiments and real world deployment for a morphologically complex language with scarce data. Azercell’s use case, customer interactions in Azerbaijani, drives the need for better tokenization and memory efficiency, two levers the team pulled aggressively. The results, the paper shows, are not merely academic: 23% more throughput translates into faster training cycles, and 58% less peak memory reduces hardware costs and enables more aggressive model scaling within existing infrastructure. The 2× tokens per word gain mirrors a broader strategy of squeezing more linguistic signal into each training pass, a key tactic when data is scarce and every byte of text matters.

From an engineering perspective, the project underscores a few concrete payoffs and tradeoffs. First, investing in language specific tokenization can yield outsized gains in efficiency, especially when training data is tight. Second, kernel level optimizations on a cutting edge instance can deliver meaningful gains in both speed and memory, but such gains come with increased maintenance and platform specificity. Third, a staged, artifact fed pipeline promotes reproducibility and modularity, which helps teams iterate quickly without starting from scratch for each new language or domain. Finally, aligning the technical plan with a concrete business objective, here a telecom chatbot, helps secure the resources and data needed to justify the engineering effort.

As the field leans into more languages and use cases with uneven data footprints, this six week sprint offers two practical takeaways: invest early in language tailored tokenization and be willing to tune at the kernel level to unlock memory and throughput headroom on modern AI accelerators. The approach also demonstrates that a production ready SageMaker AI framework, when coupled with a focused business goal, can deliver tangible gains even for languages that have historically flown under the radar of large scale LLM efforts.