Training of Large Language Models

Sarvam AI unveiled two indigenous LLMs at the India‑AI Impact Summit 2026.
New models use Mixture of Experts (MoE) architecture, activating only a subset of parameters per query, reducing compute cost.
IndiaAI Mission (launched March 2024) has an outlay of Rs 10,372 crore.
Target: 100,000 GPUs in Indian data centres by end‑2026 (currently >36,000 commissioned, 20,000 more to be added).
Sarvam AI received 4,096 GPUs and subsidies worth ~Rs 100 crore.
Talent development: support for 13,500+ students; establishment of India Data and AI Labs.

Large Language Models (LLMs) are transformer‑based neural networks trained on massive text corpora to understand and generate human language.
Training stages:

Data Collection & Pre‑processing – gathering diverse text, tokenisation, cleaning.
Pre‑training (Self‑Supervised Learning) – next‑token prediction using Transformer self‑attention.
Supervised Fine‑Tuning (Instruction Tuning) – human‑curated prompt‑response pairs.
Alignment using RLHF – Reinforcement Learning from Human Feedback to ensure safety and ethical outputs.

Traditional LLMs with hundreds of billions of parameters are compute‑intensive; MoE reduces activation to a few “expert” sub‑networks per query.

Data Sovereignty: Indigenous LLMs trained on Indian datasets mitigate dependence on foreign AI platforms and protect sensitive information.
Language Inclusivity: Tailoring models for Indian languages improves accessibility in education, healthcare, and public services.
Economic Boost: Subsidised GPU access and startup incentives foster a domestic AI ecosystem, creating jobs and encouraging open‑source innovation.
Strategic Autonomy: A robust AI infrastructure aligns with national security and digital governance objectives.

Article 19(1)(a) – Freedom of speech and expression; AI models must respect this while ensuring responsible content.
Data Protection Bill (proposed) – Emphasises consent and security for personal data; sovereign LLMs can be designed to comply.
National AI Strategy (IndiaAI Mission) – Government policy framework guiding AI research, capacity building, and ethical guidelines.