Recursive Flint Explained: The 40B Coding Model That Runs on a Laptop (2026)

If you want an AI coding agent that reads your repository, edits files and runs tests without sending a single line of code to a cloud API, the usual trade has been painful: small local models that stall on real tasks, or large open models that need a terabyte of memory. Recursive's Flint, released on October 21, 2026, is a direct attempt to break that trade. It is a free, Apache 2.0 mixture-of-experts coding model with about 40 billion total parameters and about 2.5 billion active per token, and Recursive says it generates roughly 131 tokens per second on a MacBook Pro with an M4 Pro chip, about twice the 58 to 69 tokens per second TechCrunch reports for similar models such as Google's Gemma 4 and Nvidia's Nemotron 3.5 Lightning [1].

This post is the engineering read of that launch. It explains what the 40B-A2.5B label actually means, why decode speed on a laptop depends on active parameters and memory bandwidth rather than headline size, how low-bit training shrank the model by 71%, what the attention layout does to long-context memory, which benchmark numbers are checkable and which are not, how to run Flint with rec, Ollama, llama.cpp, vLLM or SGLang, and where the "built largely by AI" claim sits in the research literature that runs from The AI Scientist to AlphaEvolve. It ends with a small tested Python toolkit that reproduces the memory and bandwidth arithmetic and measures tokens per second on your own machine.

TL;DR

• What it is: a decoder-only mixture-of-experts model with 39.8B total parameters and about 2.5B active per token, 49 layers, 256 routed experts per MoE layer with 6 selected per token plus one always-on shared expert, and a 262,144-token context window [2].
• Speed: TechCrunch reports about 131 tokens per second on an M4 Pro MacBook Pro and over 150 on an Nvidia RTX Spark, while Recursive's own post states "more than 125" and "more than 150" for short contexts [1][3].
• Why it is fast: each token reads only about 1.4 GB of weights at 4.5 bits per value, which puts the theoretical ceiling near 194 tokens per second on the 273 GB/s of an M4 Pro, so the reported 131 is roughly two thirds of that bound (my arithmetic, shown below).
• Quality: 74.2% on SWE-bench Verified per the model card, best among sub-50B models there per TechCrunch, with only Qwen3.8 27B ahead on Recursive's six-benchmark average [1][2].
• Catch: it needs more explicit guidance than large paid coding assistants [3].

What Recursive Shipped on October 21, 2026

Recursive is a 2025 startup led by CEO Richard Socher, formerly Salesforce's chief scientist. It emerged from stealth in May 2026 with $650 million at a $4.65 billion valuation and later signed a $410 million compute deal with Amazon Web Services [1][4]. Flint is its first model. A small team of people and agents built it from scratch in under four months, and the company credits its automated research system, the Eureka Machine, with contributions to data, optimizers, training recipes, infrastructure and inference optimization [3].

The model card gives the cleanest spec sheet. I have collected the load-bearing numbers in one place because most of the rest of this post is about why each of them matters.

Two details in that table deserve a second look before going further. First, the "40B" in the name is a rounded designation: the card lists 39.8 billion total parameters, and the roughly 2.5 billion active figure includes the output head, while the active body alone is about 2.2 billion [2]. Second, the only checkpoint on the Hub is BF16 at about 80 GB, which is far too large for a 48 GB or 64 GB laptop. The laptop story depends entirely on the separately prepared quantized builds, which is why the low-bit section below is the heart of the launch.

The speed claim is worth stating carefully because two Recursive-linked sources phrase it differently. TechCrunch reports about 131 tokens per second on an M4 Pro MacBook Pro versus 58 to 69 for similar models, and over 150 on an Nvidia RTX Spark [1]. Recursive's own post says more than 125 tokens per second on the M4 Pro and more than 150 on a 64 GB RTX Spark for short contexts, and separately defines its headline speed as the average tokens per second over a 128k-token session with prefill excluded [3]. Treat 125 to 131 as the realistic single-user decode range on that MacBook, and remember that the comparison models were measured by Recursive, not by a neutral lab.

Step 1: Read "40B-A2.5B" the Right Way (Mixture of Experts Explained)

A dense transformer applies every weight to every token. A mixture-of-experts transformer replaces the feed-forward block in most layers with many parallel feed-forward "experts" and a small router that picks a few of them per token. The idea goes back to Shazeer and colleagues' sparsely-gated MoE layer, which showed that a network could gain more than 1000 times the model capacity with only minor losses in computational efficiency by routing each example through only a few experts [5]. Switch Transformers simplified routing to a single expert per token to make trillion-parameter training stable and efficient [6], and Mixtral showed that a decoder with 8 experts and 2 active per token (47B total, 13B active) could outperform or match Llama 2 70B across the benchmarks it evaluated [7].

Flint sits at a much finer grain than Mixtral. Per the model card, layers 3 through 49 are MoE layers with 256 routed experts each, the router selects 6 per token, and one shared expert is always active [2]. The shared expert follows the design DeepSeekMoE introduced, where many small routed experts specialize while a shared expert absorbs common knowledge so the routed ones do not each have to relearn it [8]. The arithmetic of sparsity is easy to see: 6 of 256 routed experts is 2.3% of the routed capacity touched per token, and the whole model touches about 2.5 of 39.8 billion parameters, which is 6.3%.

Total parameters set memory, active parameters set speed

This is the single most useful mental model for local inference. Every expert must be resident in memory because any token might route to any of them, so memory footprint follows the total parameter count of 39.8 billion. Per-token compute and, more importantly on a laptop, per-token memory traffic follow the active count of about 2.5 billion. Google's documentation for Gemma 4's 26B mixture-of-experts model says the same thing in plain words: all 26 billion parameters must be loaded even though only about 4 billion are active per token [9].

That asymmetry explains why Flint and a dense 27B model feel so different on the same Mac even though their file sizes are within a factor of two of each other. The dense model streams all 27 billion weights for every token, and Flint streams about a tenth as many. It also explains the one thing MoE cannot do: it does not make a model fit into less memory. A 40B MoE needs the same room as a 40B dense model, which is why Recursive's quantization work, not its sparsity, is what gets the model onto a 48 GB machine.

How Flint's shape compares

Flint's baseline started at 35B total and 3B active, and the Eureka Machine searched expert configurations and widths until it landed on 40B total and 2.5B active [3]. That is a notable design direction: more total capacity (more experts, so more stored knowledge) and fewer active parameters (so faster decode). Compare it with the two models TechCrunch names as speed rivals. Gemma 4's 26B mixture-of-experts variant has about 25.2B total and 3.8B active parameters, with 8 of 128 experts active plus one shared expert [10]. Nvidia's Nemotron 3.5 Lightning is a 30B model with about 3B active [11]. Flint has more total weights than either and fewer active ones, which is the shape that favors both stored knowledge and tokens per second.

For larger-scale versions of the same idea, see our explainers on Mistral Large 4's 1 trillion parameter MoE and Meta's 816B Muse Spark Open, which needs at least a terabyte of memory according to TechCrunch [12]. Flint is built explicitly against that contrast.

What can go wrong with fine-grained routing

Routers can collapse onto a few favorite experts unless training adds load-balancing pressure, a long-standing issue in the Switch Transformer and DeepSeekMoE lines of work [6][8]. At batch size 1 on a laptop each new token can touch a different set of 6 experts, so memory reads are scattered, and batching many users erodes the sparsity advantage because the union of experts touched grows. That is why rec is optimized for single-session use and handles one generation at a time [13].

Step 2: Why Tokens per Second Follow Active Parameters and Memory Bandwidth

Generating one token at batch size 1 is a memory-bound job. The processor can multiply faster than it can fetch weights, so the speed limit is how many bytes per second the memory system can deliver divided by how many bytes each token must read. The systems literature calls this out directly: the "Efficiently Scaling Transformer Inference" paper analyzes decode as dominated by weight and KV cache loading, with latency set by memory bandwidth at small batch sizes [14]. On Apple Silicon and on Nvidia's RTX Spark, the CPU and GPU share one pool of unified memory, so the relevant number is that pool's bandwidth: Apple publishes 273 GB/s for the M4 Pro [15], and Android Authority reports 273 GB/s for the RTX Spark's LPDDR5X as well [16].

The ceiling formula is simple: tokens per second is at most bandwidth divided by bytes read per token, and bytes per token is active parameters times bits per value divided by 8. Here are the numbers for Flint's 4.5-bit format, which I computed with the script at the end of this post.

Three observations come out of that table. A dense 27B model at the same 4.5 bits is capped near 18 tokens per second on an M4 Pro while Flint's ceiling is about 194, so Recursive's claim of more than 5 times the speed of dense 25B to 40B models [3] is what the physics predicts. The reported 131 tokens per second implies about 184 GB/s of weight traffic, roughly 68% of the 273 GB/s peak, a believable efficiency once attention reads, the shared expert and routing are counted. And my inference, not a Recursive statement, is that the RTX Spark's 150 tokens per second implies about 211 GB/s on the same 273 GB/s, a similar fraction, consistent with a bandwidth-bound workload.

Where the simple model breaks

The ceiling is an upper bound, and four effects keep real systems below it. Prefill, the pass that reads your prompt, is compute-bound rather than bandwidth-bound, which is why Recursive's headline speed excludes it and why a 100,000-token repository prompt still takes real time to ingest [3]. KV cache reads grow with context, so speed at 128k tokens is lower than at 2k (the next sections cover how Flint limits that). Dequantizing 4-bit values to the format the math units use costs cycles, which is why Recursive wrote specialized low-bit kernels, including a Metal megakernel on M4 and M5 chips that fuses the whole decode forward pass into one GPU program [3]. And the scattered expert reads described above cost more than a single sequential stream.

Recursive's rec engine claims 1.4 to 2.6 times the token throughput of llama.cpp on an M4 Pro, according to its repository README [17], a reminder that the same weights run at very different speeds in different runtimes. Compare models only in the same engine, quantization and context length. Speculative decoding attacks the same bottleneck by verifying several draft tokens per weight read [18]; our speculative decoding explainer covers when it helps, and it helps less on an MoE, where verifying several tokens touches many more experts.

Step 3: Low-Bit Training and Why the Model Shrank 71%

TechCrunch quotes Socher crediting the Eureka Machine with three things: a sparser model shape, low-bit training that shrank Flint by 71% with almost no quality loss, and custom kernels [1]. The 71% figure has a clean explanation in the format Recursive chose. The final checkpoint is NVFP4 [3], Nvidia's 4-bit floating-point format: each value is a 4-bit E2M1 number, each block of 16 values shares an FP8 E4M3 scale factor, and a second-level FP32 scale applies per tensor, which works out to about 4.5 effective bits per value [19]. Nvidia says this cuts the memory footprint by about 3.5 times relative to FP16 [19]. Going from 16 bits to 4.5 bits removes 72% of the bytes, and my script gives 79.6 GB for BF16 versus 22.4 GB for NVFP4. That lines up with the roughly 80 GB BF16 checkpoint on the Hub, a roughly 30 GB free-disk requirement for rec's model and runtime [13], and the 71% shrink Socher describes. Recursive did not publish the byte count behind 71%, so treat the match as consistent arithmetic rather than a confirmed derivation.

Post-training quantization versus quantization-aware training

Post-training quantization trains in 16 bits and then rounds weights to 4 bits, which hurts accuracy at that width, so the field moved toward quantization-aware training, where the forward pass simulates low precision so the weights learn to survive rounding. LLM-QAT showed this for large language models, including generating training data from the model itself [20]. Nvidia's NVFP4 pretraining work trained a 12-billion-parameter model on 10 trillion tokens in the 4-bit format, with loss and downstream accuracy comparable to an FP8 baseline [21], building on block-scaled formats from the Microscaling paper [22].

Flint's recipe, as Recursive describes it, is a hybrid. The Eureka Machine discovered an offline distillation stage in which the quantized model trains on pre-sampled responses from a self-discovered coreset, with the 16-bit model as the teacher, and Recursive says this nearly halved the training steps compared with a standard quantization-aware training baseline [3]. Using a teacher to transfer behavior to a smaller or lower-precision student is classic knowledge distillation [23]; our own guide to knowledge distillation walks through the loss and the practical pitfalls. Pre-sampling the teacher's answers offline matters for cost: you pay for the expensive 16-bit model once, then reuse its outputs across many quantized training runs.

Two details stand out. The output layer, the lm_head, is also quantized to 4 bits, which Recursive notes departs from common practice, where the head and embeddings are usually kept at higher precision because they directly shape the final probabilities [3]. With a 150,144-token vocabulary the head is large, and at batch size 1 it is read on every token, so shrinking it is a direct speed gain. And the reported accuracy cost is small: within 0.7 points of the 16-bit model across benchmarks [3]. That is a self-reported average, and the card itself cautions that results vary with checkpoint precision and runtime [2], so test the exact build you plan to run.

Native NVFP4 hardware support is a Blackwell-generation feature on the Nvidia side [19], so on Apple Silicon runtimes unpack the 4-bit values in software or custom Metal kernels, part of why Recursive built its own. The Ollama tag Recursive documents is named flint:40b-a2b-qat-int4 [24], which signals an int4 quantization-aware build, so the exact bit layout depends on the runtime you pick.

Step 4: The Attention Layout That Keeps Long Context Affordable

Weights are only half the memory story, because every context token adds keys and values to the KV cache. Of Flint's 49 layers, 35 use sliding-window attention with a 513-token window and 14 use full attention; it has 32 query heads sharing 4 key/value heads with head dimension 128, per-head QK RMSNorm and learned attention sinks, and uses RoPE only on the sliding-window layers [2].

Here is the worked example. Each layer stores 2 (keys and values) times 4 heads times 128 dimensions, which is 1,024 values or 2 KB per token at 16 bits. The 14 full-attention layers therefore cost 28 KB per token, while the 35 sliding layers never hold more than 513 tokens each, about 37 MB in total at any context length. That gives the following (assuming a 16-bit KV cache, which the card does not specify).

At the full 262,144-token context the cache is about 7.5 GB, versus about 26 GB if all 49 layers used full attention, a 3.5 times saving that follows directly from the 14 to 35 split. This is the same family of trick as the hybrid attention designs we covered in DeepSeek-V4.1-Flash's causal encoder-decoder, which cuts KV cache by 8x and in the hybrid attention architecture behind DeepSeek V4-Pro: keep a few global layers to move information across the whole window and let the rest look only nearby.

Attention sinks matter because sliding windows alone degrade badly once early tokens fall out of the window. The StreamingLLM paper found that models dump a lot of attention onto the first few tokens regardless of meaning, and that keeping those sink tokens stabilizes generation over very long streams [25]. Learned per-head sinks bake that behavior into the architecture instead of leaving it to be discovered. The model card also states that the 262,144-token figure is a capacity limit and not a guarantee of uniform accuracy [2], and Recursive recommends staying within 256k tokens for the best coding results on a runtime that advertises up to 512k [3]. In practice, keep the working context far below the maximum, because speed and quality both degrade as it fills.

Step 5: What the Benchmarks Do and Do Not Show

Recursive's launch material leans on three agentic benchmarks and a mean across six. The model card lists 54.9% on Terminal-Bench 2.1, 89.7% on SWE-bench Pro v2 and 74.2% on SWE-bench Verified, using the Terminus 2 harness for Terminal-Bench and mini-swe-agent for SWE-bench, with temperature 1.0, top-p 0.95, top-k 20 and average pass@1 over at least five attempts [2]. The official post says Flint leads mixture-of-experts models under 50B total parameters on SWE-Bench Pro v2, Terminal-Bench 2.1 and SWE-Bench Verified, rivals dense models of 25B to 40B parameters while running more than 5 times faster, and reports a mean across Terminal-Bench 2.1, SWE-Bench Pro v2, SWE-Bench Verified, LiveCodeBench v6, IFBench and IFEval [3]. TechCrunch adds that on SWE-Bench Verified Flint beat every other model under 50B parameters, and that on Recursive's six-test average only Alibaba's Qwen3.8 27B scored higher [1].

There are good reasons to read these numbers with care, and the card itself states several. Evaluation setups vary across sources, developer-reported comparison scores may not apply the same adjustments, and Flint's Terminal-Bench score counts solutions found by web retrieval as failures, which some other developers' reported scores may not [2]. The official post gives no numeric scores in text, only charts [3], so the only exact figures I can verify are the three on the card. The 89.7% figure is for SWE-bench Pro v2, which is a different benchmark version from the original SWE-bench Pro where Alibaba reports 61.7% for Qwen3.8-27B, so those two numbers should never be placed side by side [26].

The fair comparison I can make from public numbers is on Terminal-Bench 2.1: Alibaba reports 73.0% for Qwen3.8-27B in its own evaluation against Flint's 54.9% on the card, consistent with TechCrunch's statement that Qwen3.8 27B is the one model ahead on Recursive's average [26]. The gap is real, and so is the speed gap in the other direction, since a dense 27B model is capped near 18 tokens per second on an M4 Pro by the arithmetic above. Our Qwen3.8-27B explainer covers the dense side of that trade. For a skeptical frame on inflated coding leaderboards, see our guide to training data contamination in SWE-bench, LiveCodeBench and MLE-bench and our Terminal-Bench 4.0 write-up.

According to TechCrunch's summary of Wired's review, Will Knight tested Flint on a 48 GB Mac by building a to-do list app and a web research tool, found that it handled every coding request correctly, and saw smaller models often error out or run slowly [1][27]. Two small apps is an anecdote, not a benchmark, but it matches the intended use.

Step 6: How to Run Flint Locally

The official documentation lists several routes: rec for Apple Silicon, Ollama, Unsloth, LM Studio and llama.cpp for local use, SGLang and vLLM for GPU serving, and Hugging Face Transformers for Python, with MLX not mentioned on the getting-started page [28]. The table below summarizes what each needs. All figures come from the model card and Recursive's docs, and I could not test them on Apple hardware myself.

Option A: rec (recommended on an M4 Pro, M5 Pro or M5 Max)

rec is Recursive's own engine, built around the Metal megakernel and optimized for single-session use. It checks your chip and memory before downloading, refuses to download on a Mac with under 48 GB of unified memory, needs about 30 GB of free disk, and serves an OpenAI-compatible API on port 11500 with model id flint [13]. The commands below are taken from Recursive's documentation.

bash
/code
# Apple Silicon Mac (M4 Pro, M5 Pro or M5 Max) with 48 GB or more of unified memory.
# Commands follow Recursive's rec documentation. Read install.sh first if you prefer not to pipe it to bash.
curl -fsSL https://raw.githubusercontent.com/recursive-oss/rec/main/install.sh | bash
export PATH="$HOME/.local/bin:$PATH"
rec doctor            # checks chip, memory and disk
rec launch pi         # downloads the model on first use, starts the server, opens the Pi harness
# rec launch opencode # alternative harness
rec status

# Direct API call while the server is running (OpenAI-compatible, port 11500, model id "flint")
curl --fail-with-body --silent --show-error \
  http://127.0.0.1:11500/v1/chat/completions \
  -H "Authorization: Bearer $(rec auth token)" \
  -H 'Content-Type: application/json' \
  -d '{"model":"flint","messages":[{"role":"user","content":"What is 19 + 23?"}],"max_tokens":4096}'

rec stop

The docs recommend the Pi harness over OpenCode for local Flint work and cite informal reports of roughly 1.5 times faster workflows with Pi, with no formal benchmark behind that figure [13]. rec handles one generation at a time, so overlapping requests can receive HTTP 429; if you run an agent that fires parallel tool calls, expect to serialize them.

Option B: Ollama

Recursive documents an Ollama tag named flint:40b-a2b-qat-int4, along with launcher integrations for OpenCode, Pi, Codex and VS Code, and the local OpenAI-compatible endpoint at port 11434 [24].

bash
/code
# Tag taken from Recursive's Ollama documentation.
ollama run flint:40b-a2b-qat-int4        # downloads on first use, then opens a chat
/bye                                      # type this inside the chat to exit

# Coding harness integrations (run from your project directory)
ollama launch pi
ollama launch opencode

# OpenAI-compatible endpoint for your own clients (start with `ollama serve` only if the app is not running)
curl --fail-with-body --silent --show-error http://127.0.0.1:11434/v1/models

Ollama downloads the model on first use and reuses it afterward, and the docs warn that you should match your client's context budget to the context configured for the model in Ollama [24]. If a harness silently truncates tool results, a mismatched context setting is the first thing to check.

Option C: llama.cpp

The llama.cpp page in Recursive's docs describes the server settings but defers the build commands and quantized file names to separate recipes, and it insists on a Flint-compatible llama.cpp revision with a matching GGUF, the right chat template and a tool parser that matches it [29]. The model card says the Hub has more than 60 community quantizations but does not name a GGUF repository, so the snippet below uses a clearly named placeholder variable that you must set from Recursive's recipe. The flags themselves are standard llama-server options, and Hugging Face documents the -hf download syntax for GGUF repositories [30].

bash
/code
# Recursive's docs call for a Flint-compatible llama.cpp revision and a matching GGUF.
# FLINT_GGUF_REPO is a placeholder: set it from the recipe on docs.recursive.com, not from this post.
FLINT_GGUF_REPO="REPLACE_WITH_REPO_FROM_RECURSIVE_DOCS"

llama-server -hf "$FLINT_GGUF_REPO" \
  --alias flint-local \
  --host 127.0.0.1 \
  --port 8080 \
  --ctx-size 65536 \
  --jinja

# In a second terminal: confirm the server answers (llama-cli alone does not expose an HTTP endpoint)
curl --fail-with-body --silent --show-error http://127.0.0.1:8080/v1/models
# Client settings: base URL http://127.0.0.1:8080/v1, model flint-local, API key local

Option D: vLLM and SGLang on a GPU

If you have a B300 or H200, the model card gives the supported path: one GPU per replica using the SGLang kit in the repository, and standard vllm serve and sglang.launch_server commands for the Hub id [2]. The card also states that Transformers modeling code is not yet included, so the pipeline snippet auto-generated on the Hub page may not run as shown [2]. That makes Transformers a poor first choice today.

bash
/code
# GPU serving, from the Flint model card. The BF16 checkpoint is about 80 GB.
pip install -U vllm
vllm serve "RecursiveSI/flint"

# In a second terminal
curl -X POST "http://localhost:8000/v1/chat/completions" \
  -H "Content-Type: application/json" \
  --data '{"model": "RecursiveSI/flint", "messages": [{"role": "user", "content": "What is the capital of France?"}]}'

# Recommended SGLang kit on one B300 (use serve_h200.sh on an H200)
pip install -U huggingface_hub openai
hf download RecursiveSI/flint --local-dir flint && cd flint
docker build -t flint-sglang sglang
docker run -d --name flint --gpus '"device=0"' --ipc=host -p 127.0.0.1:30000:30000 \
  -v "$PWD:/model" -e MODEL=/model flint-sglang bash /model/sglang/serve_b300.sh
until curl -sf localhost:30000/health_generate >/dev/null; do sleep 15; done
python3 sglang/example_client.py --base-url http://localhost:30000/v1

Calling it from your own code

Every route above exposes an OpenAI-compatible endpoint, so one client works for all of them. Recommended sampling is temperature 1.0, top-p 0.95 and top-k 20; top_k is not a standard chat-completions field, so it goes through extra_body, and some providers reject it. Reasoning is always on, so do not send reasoning-effort or disable-thinking parameters, and avoid custom stop strings in first tests [31]. The card suggests an output budget of at least 32,768 tokens for hard problems and says to preserve returned reasoning_content in history [2].

python
/code
# pip install -U openai
# Works against rec, Ollama, llama-server, vLLM or SGLang because all expose an OpenAI-compatible API.
import os
from openai import OpenAI

BASE_URL = os.environ.get("FLINT_BASE_URL", "http://127.0.0.1:11434/v1")   # Ollama; rec uses :11500, llama-server :8080
API_KEY = os.environ.get("FLINT_API_KEY", "local")                           # rec: output of `rec auth token`
MODEL = os.environ.get("FLINT_MODEL", "flint:40b-a2b-qat-int4")              # rec: "flint"

client = OpenAI(base_url=BASE_URL, api_key=API_KEY)
messages = [{"role": "user", "content": "Write a Python function that merges overlapping intervals."}]

def ask(messages):
    resp = client.chat.completions.create(
        model=MODEL,
        messages=messages,
        max_tokens=32768,                 # reasoning tokens count against this budget
        temperature=1.0,
        top_p=0.95,
        extra_body={"top_k": 20},         # not a standard OpenAI field; drop it if your server rejects it
    )
    choice = resp.choices[0]
    msg = choice.message
    if choice.finish_reason == "length":
        print("Hit the token budget; raise max_tokens before retrying.")
    # Keep reasoning_content in the history, as the model card advises for multi-turn use.
    reasoning = getattr(msg, "reasoning_content", None) or (msg.model_extra or {}).get("reasoning_content")
    assistant = {"role": "assistant", "content": msg.content}
    if reasoning:
        assistant["reasoning_content"] = reasoning
    return msg.content, assistant

answer, assistant_turn = ask(messages)
print(answer)
messages.append(assistant_turn)
messages.append({"role": "user", "content": "Now add three pytest unit tests for it."})
answer, _ = ask(messages)
print(answer)

Measure your own tokens per second

Published speeds are averages on specific machines, so measure yours. The script below streams a coding prompt, records time to first token, and divides the server-reported completion token count by the decode time. I tested it against a local mock server that emits 20 chunks with a usage record, and it printed a plausible rate; I have not run it against a real Flint server. Run it with the same context length and runtime every time you compare builds.

python
/code
# Measure decode speed (tokens per second after the first token) against any OpenAI-compatible server.
# pip install -U openai
import os
import time
from openai import OpenAI

BASE_URL = os.environ.get("FLINT_BASE_URL", "http://127.0.0.1:11434/v1")  # Ollama default
API_KEY = os.environ.get("FLINT_API_KEY", "local")
MODEL = os.environ.get("FLINT_MODEL", "flint:40b-a2b-qat-int4")

client = OpenAI(base_url=BASE_URL, api_key=API_KEY)
prompt = "Write a Python function that merges overlapping intervals, with three unit tests."

start = time.perf_counter()
first = None
chunks = 0
usage_tokens = None
stream = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": prompt}],
    max_tokens=4096,
    temperature=1.0,
    top_p=0.95,
    extra_body={"top_k": 20},                 # not a standard OpenAI field; many local servers accept it
    stream=True,
    stream_options={"include_usage": True},
)
for event in stream:
    if event.usage:
        usage_tokens = event.usage.completion_tokens
    if event.choices:
        delta = event.choices[0].delta
        # Flint always reasons, so reasoning tokens may arrive in a separate field
        text = (delta.content or "") + (getattr(delta, "reasoning_content", None) or "")
        if text:
            first = first or time.perf_counter()
            chunks += 1
end = time.perf_counter()

if first is None:
    raise SystemExit("No tokens came back. Check FLINT_BASE_URL and FLINT_MODEL.")
n = usage_tokens or chunks                     # server-reported count beats counting chunks
decode_s = max(end - first, 1e-9)
print(f"time to first token: {first - start:.2f} s")
print(f"{n} tokens in {decode_s:.2f} s after the first token = {n / decode_s:.1f} tok/s")

Check the arithmetic yourself

The next script reproduces every number in the bandwidth and KV cache tables from the model card values, Apple's published bandwidth and Nvidia's NVFP4 description. I ran it locally and its output matches the tables above. Change BANDWIDTH to your own machine's figure to see your ceiling.

python
/code
# Back-of-envelope memory and decode-speed math for Flint, using numbers from the model card.
TOTAL_PARAMS = 39.8e9      # total parameters (model card)
ACTIVE_PARAMS = 2.5e9      # active per token, including the output head (model card)
NVFP4_BITS = 4.5           # 4-bit values plus one FP8 scale per 16 values (NVIDIA)
BANDWIDTH = 273            # GB/s, Apple's published M4 Pro figure; change to your machine
BF16_BITS = 16

def gb(n_bytes):
    return n_bytes / 1e9

def weights_gb(params, bits):
    return gb(params * bits / 8)

def decode_ceiling(active_params, bits, bandwidth_gbs):
    """Upper bound on tokens/s if every generated token streams its active weights once."""
    return bandwidth_gbs / weights_gb(active_params, bits)

def kv_cache_gb(tokens, full_layers=14, window_layers=35, window=513,
                kv_heads=4, head_dim=128, bytes_per_value=2):
    per_layer_per_token = 2 * kv_heads * head_dim * bytes_per_value   # K and V
    full = full_layers * tokens * per_layer_per_token
    local = window_layers * min(tokens, window) * per_layer_per_token
    return gb(full + local)

if __name__ == "__main__":
    bf16, fp4 = weights_gb(TOTAL_PARAMS, BF16_BITS), weights_gb(TOTAL_PARAMS, NVFP4_BITS)
    print(f"Total weights  BF16: {bf16:.1f} GB   NVFP4: {fp4:.1f} GB   saved: {1 - fp4 / bf16:.0%}")
    bw = BANDWIDTH
    print(f"Bytes read per token at NVFP4: {weights_gb(ACTIVE_PARAMS, NVFP4_BITS):.2f} GB")
    print(f"Flint 2.5B active, NVFP4  ceiling on {bw} GB/s: {decode_ceiling(ACTIVE_PARAMS, NVFP4_BITS, bw):.0f} tok/s")
    print(f"Dense 27B, 4.5-bit        ceiling on {bw} GB/s: {decode_ceiling(27e9, NVFP4_BITS, bw):.0f} tok/s")
    print(f"Dense 27B, BF16           ceiling on {bw} GB/s: {decode_ceiling(27e9, BF16_BITS, bw):.0f} tok/s")
    print(f"Observed 131 tok/s implies about {131 * weights_gb(ACTIVE_PARAMS, NVFP4_BITS):.0f} GB/s of weight traffic")
    for ctx in (32_768, 131_072, 262_144):
        all_full = kv_cache_gb(ctx, full_layers=49, window_layers=0)
        print(f"KV cache at {ctx:>7} tokens (16-bit): {kv_cache_gb(ctx):5.2f} GB   vs {all_full:5.2f} GB if all 49 layers were full attention")

Step 7: The Eureka Machine and the Rise of AI-Built AI

The most discussed line in the launch is that Flint was "built largely by AI." What Recursive actually claims is narrower and more interesting. The Eureka Machine is a system that proposes ideas, implements them, tests them, and uses the results to choose the next experiments, exploring many directions in parallel, building on earlier runs and combining promising ideas [3]. It contributed to the model's data, optimizers, training recipes, infrastructure and inference optimization, and in the three places Socher highlighted it found the sparser expert configuration, the distillation-based low-bit recipe and the custom kernels [1][3]. Recursive did not say how much researcher time it saved [1], so "largely" describes breadth of involvement, not a measured fraction of the work.

The research lineage

This is the latest step in a sequence of systems that automate parts of the research loop. Sakana AI's The AI Scientist was an early end-to-end attempt: it generates ideas, writes code, runs experiments, visualizes results, writes a paper and runs a simulated review, at a reported cost under $15 per paper, demonstrated on diffusion modeling, transformer language modeling and learning dynamics [32]. Its limits are worth remembering: the quality judgment came from an automated reviewer rather than human reviewers, and the scope was three machine learning subfields. Google DeepMind's AlphaEvolve took a narrower but more verifiable route, pairing Gemini models with automated evaluators in an evolutionary loop over code. Its reported results include a Borg scheduling heuristic that recovers on average 0.7% of Google's worldwide compute, a matrix multiplication kernel in Gemini's architecture made 23% faster (cutting Gemini's training time by 1%), up to a 32.5% speedup of a FlashAttention kernel implementation, and 48 scalar multiplications for 4x4 complex matrices [33].

Flint's three highlighted contributions map closely onto AlphaEvolve's strengths. Kernel optimization has a crisp automatic score (is it correct, and how fast is it), and so does a quantization recipe (how many benchmark points were lost, how many training steps were spent). Socher's AI Engineer World's Fair talk, "First Steps Toward Automated AI Research," described a research loop evaluated on nanochat training quality, nanoGPT training speed and CUDA kernel optimization, and framed them as first steps toward recursive self-improvement rather than proof of it [34]. Problems with a fast, trustworthy score are where automated research works first, which also tells you where it is least likely to help: open-ended choices such as what to build, which data is trustworthy, or when a benchmark is being gamed.

Other labs are moving the same way, as our posts on Tencent's Hy4 Preview, the 770B model that helped train itself and Anthropic's R&D automation index show, and TechCrunch notes OpenAI's statement that by mid-August its agents were logging 3.1 workdays for every one worked by a human researcher [1]. For where agents still fall short at real discovery, read our EurekaBench breakdown. The long-term aim Socher described to Wired is full recursive self-improvement with humans overseeing the process and able to stop it [1], a goal that Flint does not demonstrate by itself; it demonstrates that an automated loop can find usable engineering wins inside a real model.

Step 8: Prompting a 2.5B-Active Agent So It Finishes the Job

Recursive says plainly that Flint needs more explicit guidance than larger closed-source coding models, and recommends breaking complex tasks into smaller subtasks with explicit instructions [3]. That advice is easy to nod at and hard to apply, so here is a concrete before and after.

A vague request such as "add rate limiting to the API" asks the model to decide the algorithm, storage, configuration, tests and files to touch. A guided version runs as separate steps, each ending in a check: first ask Flint to list the files that register middleware without editing anything; then specify the algorithm (a token bucket keyed by API key, 60 requests per minute, in-memory) and ask for only that module plus a unit test; run the tests yourself and paste failures back with a fix limited to the failing function; and only then ask for the middleware wiring as a separate small diff.

Each step keeps the prompt and the tool outputs small, which helps a model with 2.5 billion active parameters and also keeps context well below the range where quality and speed drop. Because reasoning is always on and long reasoning can use up the output budget before an answer completes [2], set max_tokens generously and watch for a finish reason of length, which the docs say means you should raise the budget rather than retry the same request [31].

The card also reminds you to test generated code, review consequential changes and enforce your own permissions around tools, since structured tool-call grammar does not stop a model from choosing an unsafe action [2]. Run the agent in a project directory with version control, a throwaway container for shell commands, and no credentials in the environment.

If you route work between models, Flint suits the cheap, private, high-volume tier (scaffolding, tests, refactors) while a hosted model handles ambiguous design; our explainer on routing requests to the right model shows the pattern.

Where a Local Coding Model Fits for Creators and Builders

A local coding model writes the glue: renaming 400 clips, batch-resizing exports, pulling your channel analytics into a spreadsheet, or formatting prompts for a pipeline. The creative generation itself still runs better in a browser tool. Text2Shorts on Miraflow turns a topic into a script, scene visuals and a finished Short, the AI image generator and YouTube thumbnail maker cover stills and covers, AI Clipping finds highlights in long recordings, and the cinematic video generator and AI music generator handle B-roll and soundtrack. Start from the Miraflow home page and keep Flint for the code around it.

Common Mistakes When Evaluating or Running Flint

1. Comparing speeds across runtimes. A Flint run in rec against a rival in llama.cpp measures the engines; hold runtime, quantization and context constant.
2. Quoting benchmarks across versions. SWE-bench Pro v2, SWE-bench Pro and SWE-bench Verified are different tests, and other developers' scores may use different harness rules [2].
3. Forgetting that total parameters set memory. A 2.5B active count does not make a 2.5B-sized download; rec needs a Mac with at least 48 GB [13].
4. Filling the context window. The card calls the context length a capacity limit, not a guarantee of uniform accuracy [2].
5. Fighting the reasoning setup. Reasoning is always on, reasoning_content belongs in history, and the shipped chat template and tool parser should not be swapped [2][29][31].
6. Handing it a whole epic. The one limitation Recursive states outright is the need for explicit, decomposed instructions [3].

Production and Best-Practice Checklist

• Pin and report everything. Record model revision, quantization, runtime version, context length, sampling and tool-parser settings together, as Recursive's docs request [29][31].
• Evaluate the exact build you deploy. The card warns that results vary with precision and runtime [2]; run a 20 to 50 task regression set from your own repositories before trusting a new quant.
• Budget memory in three parts: quantized weights (about 22 GB for NVFP4 by my arithmetic), KV cache (about 1 GB at 32k and 7.5 GB at 262k tokens at 16 bit), and the OS plus your editor and browser.
• Sandbox tool execution. Use containers for shell commands and keep secrets out of the environment, as the card advises [2].
• Keep a hosted fallback. Escalate ambiguous multi-file design work to a larger model, for example Kimi K3 or a hosted frontier model.

Frequently Asked Questions

What is Recursive Flint?

Flint is a free, Apache 2.0 mixture-of-experts coding model from Recursive with 39.8 billion total parameters and about 2.5 billion active per token, designed to run locally on laptops and workstations. It was released on October 21, 2026 and is built for agentic coding tasks such as fixing bugs, implementing features and writing tests [1][2].

Where do I download Flint?

The weights are on Hugging Face as RecursiveSI/flint, a BF16 checkpoint of about 80 GB, with community and prepared quantizations linked from the page [2]. Recursive's rec engine and the Ollama tag flint:40b-a2b-qat-int4 download a quantized build for you [24][13].

How much memory do I need to run Flint?

For Recursive's rec engine the requirement is an Apple Silicon Mac with an M4 Pro, M5 Pro or M5 Max, at least 48 GB of unified memory and about 30 GB of free disk [13]. Recursive's post lists a 64 GB RTX Spark for the Nvidia speed figure [3]. The BF16 checkpoint needs one B300 or H200 GPU per replica [2].

Why is Flint faster than Gemma 4 or Nemotron 3.5 Lightning?

Decode speed is bounded by memory bandwidth divided by bytes read per token. Flint reads about 2.5 billion active parameters at roughly 4.5 bits each, uses sliding-window attention on 35 of 49 layers to limit KV cache reads, and runs custom low-bit kernels, including a Metal megakernel on M4 and M5 chips [3][2]. TechCrunch reports 131 tokens per second against 58 to 69 for similar models [1].

Is Flint better than Qwen3.8 27B?

Not on quality. TechCrunch reports that only Qwen3.8 27B scored higher than Flint on Recursive's six-test average, and Alibaba's own Terminal-Bench 2.1 figure for Qwen3.8-27B is 73.0% versus Flint's 54.9% [1][26]. Flint's advantage is speed, since a dense 27B model reads roughly ten times more weights per token. See our Qwen3.8-27B guide.

Does Flint support MLX?

Recursive's getting-started page lists rec, Ollama, Unsloth, LM Studio, llama.cpp, SGLang, vLLM and Transformers, and does not mention MLX [28]. On a Mac, use rec or Ollama.

Can I use Flint commercially?

The weights are released under Apache 2.0, which permits commercial use, while hosted API terms are separate [2]. Read the license file in the repository for the full terms.

What is the Eureka Machine?

It is Recursive's system for large-scale automated AI research. It proposes ideas, implements and tests them, and uses the results to pick the next experiments, and it contributed to Flint's data, optimizers, training recipes, infrastructure and inference optimization [3].

Does Flint prove recursive self-improvement?

No. Recursive's stated long-term goal is full recursive self-improvement with human oversight [1]. Flint shows an automated system contributing to specific engineering wins in a model built by people and agents together, and Recursive did not quantify the time saved.

Why does Flint need more guidance than ChatGPT-style coding assistants?

Recursive says it needs more explicit instructions than larger closed-source coding assistants and recommends splitting complex tasks into smaller subtasks [3]. With about 2.5 billion active parameters per token, there is less computation available per step to infer unstated requirements, so precise prompts and short feedback loops matter more.

Conclusion

Flint is a well-documented example of what happens when sparsity, low-bit weights and kernel engineering are all pushed in the same direction at once. A 39.8B-parameter mixture-of-experts model touches only about 2.5B parameters per token, 4.5-bit NVFP4 weights cut the file from about 80 GB to the low 20s of gigabytes, a 35-to-14 mix of sliding and full attention keeps the KV cache near 7.5 GB even at 262k tokens, and the resulting traffic of roughly 1.4 GB per token sits comfortably under the 273 GB/s of a current MacBook Pro. The 125 to 131 tokens per second Recursive reports follows from that arithmetic, which is the best reason to believe it.

The quality claims deserve the usual care. The checkable numbers are 74.2% on SWE-bench Verified, 54.9% on Terminal-Bench 2.1 and 89.7% on SWE-bench Pro v2 from the model card, all measured under settings Recursive documents, and a dense Qwen3.8-27B still scores higher overall at a fraction of the speed. The "built largely by AI" story is best read as the newest entry in a line of automated research systems from The AI Scientist to AlphaEvolve, applied to problems with fast, trustworthy scores such as kernels and quantization recipes.

If you try it, start with rec or Ollama, keep tasks small, measure your own tokens per second with the script above, and evaluate the exact build you deploy. Then pair it with the tools that handle the creative side: build the scripts and glue code locally, and use Miraflow for the videos, images, thumbnails, clips and music your channel actually publishes.

References

1. [1] TechCrunch, Recursive launches Flint, a laptop coding model it says was largely built by AI (Oct 21, 2026)
2. [2] Hugging Face, RecursiveSI/flint model card
3. [3] Recursive, Introducing Recursive Flint
4. [4] TechCrunch, Recursive Superintelligence signs compute deal with Amazon
5. [5] Shazeer et al., Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
6. [6] Fedus et al., Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
7. [7] Mixtral of Experts
8. [8] DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
9. [9] Google AI for Developers, Gemma 4 model overview
10. [10] Google, Gemma 4: Byte for byte, the most capable open models
11. [11] Guru3D, Nvidia Nemotron 3.5 Lightning runs locally on a single GPU
12. [12] TechCrunch, Meta launches an open-weight model that runs on Macs (Oct 15, 2026)
13. [13] Recursive docs, Run Flint with rec
14. [14] Pope et al., Efficiently Scaling Transformer Inference
15. [15] Apple Newsroom, Apple introduces M4 Pro and M4 Max
16. [16] Android Authority, Nvidia RTX Spark explained
17. [17] GitHub, recursive-oss/rec local inference engine
18. [18] Leviathan et al., Fast Inference from Transformers via Speculative Decoding
19. [19] NVIDIA, Introducing NVFP4 for Efficient and Accurate Low-Precision Inference
20. [20] LLM-QAT: Data-Free Quantization Aware Training for Large Language Models
21. [21] Pretraining Large Language Models with NVFP4
22. [22] Microscaling Data Formats for Deep Learning
23. [23] Hinton et al., Distilling the Knowledge in a Neural Network
24. [24] Recursive docs, Run Flint with Ollama
25. [25] Efficient Streaming Language Models with Attention Sinks (StreamingLLM)
26. [26] DataNorth, Alibaba releases Qwen3.8-27B
27. [27] Wired, Recursive self-improvement and Flint (review by Will Knight)
28. [28] Recursive docs, Getting started
29. [29] Recursive docs, Run Flint with llama.cpp
30. [30] Hugging Face Hub docs, GGUF with llama.cpp
31. [31] Recursive docs, Generation settings
32. [32] Lu et al., The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
33. [33] Google DeepMind, AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms
34. [34] AI Engineer World's Fair 2026, Richard Socher, First Steps Toward Automated AI Research