Azercell cuts memory, boosts throughput for Azerbaijani LLM

Azercell slashed GPU memory use by 58 percent while boosting training throughput by 23 percent.

In six weeks, Azercell Telecom of Azerbaijan partnered with AWS’s Generative AI Innovation Center to adapt foundation models for Azerbaijani, a morphologically rich language with limited training data and no blueprint for efficient LLM training. The effort aimed to land a production ready framework on Amazon SageMaker AI for telecom use cases and a customer facing chatbot. The team reports that the result is a framework that can deliver real gains on cloud hardware while respecting the quirks of a language with rich inflection and limited labeled data.

The paper shows that the solution rests on a three stage pipeline, each producing artifacts that feed the next. Stage 1 centers on tokenizer development, building an efficient tokenizer specifically for Azerbaijani. The team evaluated three approaches, including baseline English optimized tokenizers, vocabulary extensions, and a custom monolingual tokenizer, measuring encoding efficiency with standardized metrics. The custom monolingual tokenizer is highlighted as a key part of the tokenizer effort, underscoring the importance of language tailored representation when data is scarce.

Beyond tokenizer work, the framework leverages open source tools such as PyTorch, Hugging Face Transformers, and Liger Kernels to stitch together the training process. The gains were realized on an ml.p5.48xlarge instance, where kernel level optimizations delivered the dramatic memory and throughput improvements. In addition to faster training, the approach achieved a 2x improvement in tokens per word with the custom tokenizer, effectively doubling the model’s context window and enabling Azerbaijani text to be processed more efficiently within a given compute budget.

The practical payoff is clear for teams facing similar constraints. For engineers, the Azeri project demonstrates the value of language specific tokenization in low resource settings and how careful kernel level tuning can yield measurable hardware efficiency. For product teams, the results point to faster iteration cycles when deploying language models to real world telecom use cases and chatbots, where latency and throughput directly affect user experience. And for infrastructure leaders, the experience underscores the importance of selecting hardware and software stacks that unlock memory savings and throughput without sacrificing model quality or safety guardrails.

The six week collaboration also provides a blueprint for other teams working with morphologically complex languages. By building a production ready framework on a managed platform, Azercell shows that cloud based tooling can scale specialized LLMs even when linguistic data is sparse. The duo’s emphasis on tokenizer design and kernel level optimizations offers concrete, actionable steps: start with language tailored tokenizers, experiment with memory efficient kernels on capable instances, and structure development around a three stage pipeline that yields reusable artifacts for the next iteration.

In the broader AI engineering context, the Azercell achievement is a reminder that progress in practical LLM deployment often comes not from new model sizes alone but from careful engineering choices that squeeze efficiency from existing architectures. The result is a tangible example of how a telecom operator can bring a local language model into production, with real gains in throughput and memory footprint, while expanding the practical reach of AI for a language spoken by millions but historically underrepresented in large-scale models.