Llama 4

Meta released Llama 4 on April 5, 2025, as a family of natively multimodal models built on a mixture-of-experts architecture. Two models shipped immediately: Scout, with 17 billion active parameters drawn from 16 experts and 109 billion parameters in total, and Maverick, a higher-capacity variant aimed at reasoning and large-context enterprise workloads. A third model, Behemoth, was previewed at roughly two trillion parameters but held back as a research system still in training.

The headline number was Scout's context window: ten million tokens, by a wide margin the longest of any openly available model at launch. Where earlier long-context work had pushed toward one million tokens and treated that as a frontier, Llama 4 moved the ceiling by an order of magnitude and did so in a model small enough to run outside a hyperscale data center. Mixture-of-experts was central to this. By activating only a fraction of total parameters per token, the architecture decoupled the cost of inference from the total size of the model.

Both released models were multimodal from the start rather than having vision bolted on after the fact, processing text and images through a unified backbone. Distribution followed the pattern Meta had established with earlier Llama generations: weights published under the Llama 4 Community License, a permissive-in-practice but not OSI-approved license that allows broad commercial building while retaining specific controls on scale and attribution.

Llama 4 marked the point where the open-weight ecosystem stopped following the frontier on capability and started leading it on a specific axis. Context length had been a closed-lab advantage since Gemini 1.5 demonstrated a million-token window. Within a year, the longest context available to anyone was something you could download and run yourself. For builders, this changed the calculation on what had to be sent to an external API at all, and it made whole-repository and whole-corpus reasoning a local capability rather than a rented one.

The release also settled an architectural argument. Mixture-of-experts had been used quietly in frontier systems for years without being confirmed publicly at this scale, and Llama 4 made the approach both explicit and reproducible. Publishing the weights of a large MoE model gave the research community something to study rather than infer, and the design pattern of many experts with few activated per token became the default assumption for large models that followed, including the open Chinese systems that would close the remaining gap over the following year.