Gemma 4 Technical Report

The Gemma Team's own report on the gemma-4 suite. arXiv 2607.02770, submitted 2026-07-02, revised 2026-07-24. Reached through the awesome-gemma catalog. T1 for what the models are — sizes, architecture, training choices; its benchmark wins are the maker scoring its own work and are flagged as such below.

The abstract, verbatim

We introduce Gemma 4, a new generation of open-weight, natively multimodal language models in the Gemma model family. Designed to advance compute efficiency and reasoning, the Gemma 4 model suite features dense and Mixture-of-Experts architectures, ranging from 2.3B to 31B parameters. Alongside improved vision and audio encoders for all model sizes, we propose a unified, encoder-free architecture for our 12B model, which ingests raw audio and image patches. Furthermore, we integrate a thinking mode, enabling Gemma models to generate reasoning traces prior to responding. We improve inference speed, memory, and compute efficiency, as well as long-context abilities through critical design choices. Gemma 4 establishes a leap in performance across STEM, multimodal, and long-context benchmarks, and rivals larger, frontier open models in human-rated tasks.

What it settles

• The suite runs 2.3B to 31B. This closes an open flag on gemma-4, which had the 31B-it as reported, not confirmed — the size had entered the corpus only through a third-party derivative (gemma-4-31b-it-scotoma) with no Google source behind it. The maker now names the top of the range.
• Encoder-free is specific to the 12B. Every size got improved vision and audio encoders; only the 12B drops them for raw audio and image patches. gemma-4-12b-announcement described the 12B in isolation, so the wiki had no way to tell a family-wide change from a one-model one.
• Thinking mode ships in the open family. Reasoning traces before the answer — the first record in this spoke of Google putting that in the open-weight track rather than gemini. Bears directly on the wiki's live question of whether open-weight reasoning closes on the frontier.

What it does not settle

"Leap in performance", "rivals larger, frontier open models" and the human-rated result are the Gemma Team grading Gemma. Per ../QUALITY.md, self-run numbers stay flagged whatever the venue — an arXiv PDF does not make a vendor's own benchmark independent. The independent read on the 31B in the corpus is artificial-analysis; nothing here replaces it. The abstract also gives no per-variant parameter table, so which sizes are dense and which are MoE is still only partly known (the 26B-A4B is the MoE recorded on gemma-4).

Related

gemma-4 · awesome-gemma · gemma-variants · google · open-weight-models · artificial-analysis · gemma-4-12b-announcement