Speed Up Your Local Coding Agent: Running Gemma 4 with Multi-Token Prediction in Ollama

Ollama 0.31 introduced a major performance upgrade for Apple Silicon users: multi-token prediction (MTP) for Gemma 4. In practice, this means a coding agent backed by Gemma 4 can generate code noticeably faster without any change in output quality. This tutorial walks through setting it up from scratch.

Step 1: Check your hardware and OS requirements

MTP acceleration is currently exclusive to Macs with Apple Silicon (M-series chips), since it relies on Apple’s MLX framework. Confirm your Mac has an M1 chip or newer and that macOS is up to date before continuing. If you’re on Windows or Linux, you can still run Gemma 4 through Ollama, but you won’t get the MLX-specific speedup discussed here.

Step 2: Install or update Ollama to version 0.31 or later

Download the latest macOS build from Ollama’s official site and install it normally. If you already have Ollama installed, open it and let it auto-update, or reinstall the newest package to make sure you’re on 0.31+. You can confirm your version from a terminal:

ollama --version

If the version reported is older than 0.31, repeat the update before moving on, since the MTP engine work only ships in this release and later.

Step 3: Pull the MLX-optimized Gemma 4 model

Gemma 4 needs to be pulled in its MLX variant to take advantage of the draft-and-verify speedup. In your terminal, run:

ollama pull gemma4:12b-mlx

If you already had a copy of this model from before MTP support landed, re-pull it anyway. The underlying weights needed for multi-token prediction are only included in the updated model artifact, so an older local copy won’t get the speedup automatically.

Step 4: Launch a coding agent through Ollama

Rather than chatting with the model directly, the intended workflow is to wire it into a coding agent using Ollama’s launch command, which handles the setup for you. For example, to launch Claude Code backed by Gemma 4:

ollama launch claude --model gemma4:12b-mlx

The same command pattern works with other supported agents, including Codex, Droid, OpenCode, and Copilot, just swap out the agent name after ollama launch.

Step 5: Verify MTP is active and observe the speedup

Once the agent is running, ask it to perform a normal coding task, such as writing a small function, refactoring a file, or fixing a bug. You should notice generation feels snappier, especially during long, syntactically predictable stretches of code, since these are exactly the patterns MTP’s draft model tends to guess correctly.

There’s nothing you need to toggle: MTP runs automatically whenever conditions are favorable, and Ollama’s engine continuously adjusts how many tokens to speculatively draft based on real-time acceptance rates. If the draft model’s guesses stop being useful, Ollama quietly falls back to standard one-token-at-a-time decoding so performance never gets worse than the non-MTP baseline.

Step 6: Understand what’s happening under the hood

For readers who want the mechanics, the speedup comes from three coordinated pieces: a small draft model bundled with Gemma 4 proposes several upcoming tokens at once; the full Gemma 4 model then verifies that entire short sequence in a single pass instead of checking tokens one at a time; and a custom matrix-multiplication kernel makes that batch verification step itself faster by avoiding redundant weight reads. Correctly predicted tokens are kept, incorrect ones are cheaply rolled back using a checkpoint in the model’s cache, and the whole cycle repeats.

Step 7: Benchmark it yourself

If you want to quantify the improvement on your own machine, time a fixed coding task with gemma4:12b-mlx before and after confirming you’re on Ollama 0.31+. Your actual gain will vary by chip, model size, and how repetitive your codebase’s patterns are, so treat any published benchmark as a rough guide rather than a guarantee.

Troubleshooting tips

If you don’t notice any speedup, double check three things: that ollama –version reports 0.31 or higher, that you re-pulled gemma4:12b-mlx after updating, and that you’re actually invoking the MLX-tagged model rather than a plain GGUF variant of Gemma 4.