Netlify Tests 11 AI Models for Coding: Which Is Best?

Netlify has published a real-world comparison of 11 AI models running identical coding prompts, using their newly open-sourced evaluation tool, AXIS. The tests cover common web development tasks like building a coffee shop site, a to-do app, and an AI recipe app, giving you concrete data on model quality and credit costs.

This comparison comes right after Netlify's partnership with OpenRouter, which lets your projects use any model on OpenRouter through their AI Gateway. For their Agent Runners (the chat prompt box in Netlify that builds or iterates on projects), they've added the open-source OpenCode agent so you can drive a wider range of models, including Kimi K3, GLM 5.2, and DeepSeek V4.

What They Tested

Netlify used their in-house AXIS framework to run three straightforward use-cases:

• Coffee shop site – simple static site, plus a follow-up to add seat reservations.
• To-do list app – requires a shared database from the start, then adds optional photo uploads.
• "What can I cook" app – users enter ingredients, and the site uses the AI Gateway to generate recipes.

They evaluated on functionality (not design), checking things like: Does it use a database when needed? Does it properly use Netlify Database? Does it avoid over-engineering? Models that consistently fail the checks aren't offered in Agent Runners.

Key Findings

The full report, available at their blog, shows each model's generated site, notes on notable issues, and credit costs. While they didn't publish official scores in this post, they highlight major differences in results. For example, they mention running GPT 5.6 Sol on low-effort by default, offering a more economical alternative to Opus that still gives "pretty darn good results."

They also noted that credit cost varies wildly between models for the same task, so you might be paying a premium for a model that doesn't produce better output.

Takeaway

If you're choosing a coding model for your own agent-based projects, this is a useful data point. The test focuses on real-world web development scenarios and includes open-source models that are often hyped. The results are subjective (they admit it), but worth reviewing before you commit to a default model or spend credits.

Note: This is the first in a series; follow-up posts will dive into the other use cases.