🔍 Read the full analysis: Can Inkling 2 Mini Deliver Cutting-Edge Performance Locally? on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Thinking Machines Lab released Inkling 2 Mini, a multimodal mixture-of-experts model that the company describes as having about 26 billion total parameters, with about 3 billion active per token. It supports several inference runtimes and has a research variant with adjustable layer passes, but independent benchmark checks and practical hardware requirements are not established in the supplied material.
Thinking Machines Lab has released Inkling 2 Mini, a multimodal model intended for local AI setups that the company says has about 26 billion total parameters and activates about 3 billion per token, as detailed in the original analysis. The release includes support for several inference runtimes and a research version with adjustable computation, but the supplied information does not establish independent benchmark results or the hardware needed to run it in practice.
Inkling 2 Mini accepts text and image input and is described as having a 1 million-token context window. Its architecture uses 24 physical decoder layers. The first and last six layers run once, while the 12 middle layers run three times, producing 48 logical layer visits per token. The middle layers reuse weights across passes rather than using a distinct set for each visit.
The model uses a mixture-of-experts architecture: at each layer visit, a routing system selects eight experts from a pool of 128. Thinking Machines says it is distributing expert weights in NVFP4 and MXFP8 formats and including a DSpark drafter intended to speed generation with a small memory cost. These design details describe the release, but do not by themselves establish actual memory use or speed on consumer devices.
The fixed-pass version is available through Transformers, llama.cpp and MLX; the company says SGLang supports day-one inference for datacenter deployments. A separate research checkpoint, TML-d24-varloop-research, lets users change the number of passes. The company says reducing passes can make inference faster, potentially at a cost to output quality.
Inkling 2 Mini for Local AI
Thinking Machines Lab’s new multimodal model combines reused layers with expert routing. Its practical speed, hardware needs and benchmark results still need outside validation.
A loop reuses model layers
The design reuses middle-layer weights to build capacity, while routing selects part of the expert pool at each layer visit.
The first six and last six physical decoder layers run once.
The same 12 middle layers run in three passes, for 48 logical layer visits per token.
At each visit, the model selects eight experts from a pool of 128.
Options for inference
The release offers several runtimes and a research checkpoint with adjustable loop passes.
Multiple runtimes
Transformers, llama.cpp and MLX support the fixed-pass model; SGLang supports datacenter inference.
Fewer passes
The research checkpoint allows one or two passes. The company says this can be faster, with a possible quality trade-off.
Claims need outside checks
The company reports strong benchmark comparisons, but the supplied material does not include independent replication or a full score table.
- Leads in 32 of 48 benchmarks against named models.
- Reported tie with Qwen3.8-27B on Terminal-Bench 4.0.
- Trails on AutomationBench-AA v1.0.6.
- Complete memory requirements and device-specific speed.
- Independent image and long-context performance.
- Quality at reduced loop counts.
What local users should assess
Runtime availability is a starting point; actual usefulness depends on hardware and workload.
A New Option for Local Model Users
The release adds another potential choice for people running AI models on their own machines or infrastructure. Its reused middle-layer weights and selective expert activation are intended to provide substantial model capacity without activating every expert for each token. Support across several established runtimes may also make it easier for developers to test the model in different environments.
Those features do not yet answer the central practical question: whether Inkling 2 Mini delivers useful performance on hardware that local users can access. The supplied release material gives no complete memory requirement or measured generation speeds across consumer devices. A model’s total parameter count, active parameter count and weight format all inform deployment, but none alone tells users whether a specific computer can run it at an acceptable speed.
For teams evaluating local multimodal systems, the company’s reported benchmark leads may be a reason to test the model, not a substitute for independent results. Image quality, long-context behavior, runtime compatibility and output quality at reduced pass counts all need practical evaluation.
Products Worth Considering

Apple AirPods Pro

DJI Mini 3 Drone with DJI RC Controller

Apple AirPods Pro (2nd Generation) Wireless Earbuds with MagSafe Charging Case (Lightning)
As an affiliate, we earn on qualifying purchases.
How Inkling Reuses Its Layers
The repeated-layer design is the release’s main architectural distinction. The model has 24 physical decoder layers, but runs its 12 middle layers three times. Learned messages carry information between passes, according to the technical description. The design therefore reuses weights, while still requiring computation for all 48 logical visits per token. The source says each visit also keeps its own key-value and short-convolution state, a detail that matters when estimating memory use.
Thinking Machines compares Mini with its Inkling 2 Small model. Both are described as performing 48 layer visits per token, but Mini has a 2,304 hidden width versus 4,608 for Small, and routes among 128 experts per layer rather than 384. The supplied material does not give a complete model file size or minimum hardware specification. It also notes that other work, including Nanbeige, has explored looping architectures; the release does not establish that this approach is new to the field.
The company reports that Inkling 2 Mini performs strongly on several evaluations, including the Artificial Analysis Intelligence Index v5 and comparisons with Gemma 4 31B, Muse Glimmer, Qwen3.8-27B and Nemotron 3.5 Super VL. It also reports a lead on FrontierCode v2.0, a tie with Qwen3.8-27B on Terminal-Bench 4.0 and a lower result on AutomationBench-AA v1.0.6. Those are company-reported comparisons; the provided material contains neither the full score table nor independent replication.
“Inkling 2 Mini is a mixture-of-experts model with 24 decoder layers, but the 12 middle layers run three times in a loop.”
— Thinking Machines Lab, in the supplied release material
Hardware and Benchmarks Still Unclear
The supplied information does not specify minimum memory, model file size or measured generation speeds on consumer hardware. It also does not show independent evaluations confirming the benchmark comparisons, or provide enough results to assess image understanding and the claimed million-token context in practice. A stated context window does not by itself show how well a model uses information across that length.
The quality trade-off from running the research checkpoint with one or two passes is also not fully assessed in the material. It is unclear which devices and runtime configurations can take advantage of the mixed-precision weights and DSpark drafter, or how performance varies among Transformers, llama.cpp, MLX and SGLang. These gaps mean local users cannot yet infer expected performance from the architecture description alone.
Model Cards and Reproducible Tests
The next evidence to watch for is a full model card and score table, followed by independent, reproducible benchmark results. Reports that identify hardware, runtime, quantization and generation settings would help users compare results rather than relying on broad performance claims.
Hands-on tests can also show whether image inputs and long prompts work as described, and whether fewer passes provide a useful speed gain without an unacceptable quality loss. Until those details are available, Inkling 2 Mini’s performance on specific local systems remains an open question.
Key Questions
What is Inkling 2 Mini?
It is a multimodal mixture-of-experts model released by Thinking Machines Lab. The company describes it as having about 26 billion total parameters, with about 3 billion active per token.
Can it run on a consumer PC?
The supplied material does not give a minimum hardware configuration or complete memory requirement. Whether it runs acceptably will depend on the device, runtime and model format; the release details alone do not settle that question.
Which inference tools are supported?
The fixed-pass model is listed for Transformers, llama.cpp and MLX. The company also says SGLang supports day-one inference for datacenter deployments.
Are the benchmark results independently verified?
Not in the material provided. Thinking Machines reports competitive results across several benchmarks, but the full score table and independent replication were not included.
What does the research checkpoint change?
The TML-d24-varloop-research checkpoint allows users to adjust how many times the middle layers run. The company says fewer passes can speed inference, with a possible quality trade-off, but the supplied material does not quantify that trade-off.
Primary source: Hugging Face · via ThorstenMeyerAI.com
