Can Inkling 2 Mini Deliver Cutting-Edge Performance Locally?
AIThis post was created with the assistance of artificial intelligence (AI).

🔍 Read the full analysis: Can Inkling 2 Mini Deliver Cutting-Edge Performance Locally? on ThorstenMeyerAI.com

Before you orderOffer from Amazon

Get the latest gadgets delivered free with Prime

  • Fast, free delivery on millions of items
  • Prime Video, Amazon Music and more included
  • Member-only deals all year
Start your free Prime trial Free trial for eligible customers · Cancel anytime
As an affiliate, we earn on qualifying purchases.

TL;DR

Thinking Machines Lab released Inkling 2 Mini, a multimodal mixture-of-experts model that the company describes as having about 26 billion total parameters, with about 3 billion active per token. It supports several inference runtimes and has a research variant with adjustable layer passes, but independent benchmark checks and practical hardware requirements are not established in the supplied material.

Thinking Machines Lab has released Inkling 2 Mini, a multimodal model intended for local AI setups that the company says has about 26 billion total parameters and activates about 3 billion per token, as detailed in the original analysis. The release includes support for several inference runtimes and a research version with adjustable computation, but the supplied information does not establish independent benchmark results or the hardware needed to run it in practice.

Inkling 2 Mini accepts text and image input and is described as having a 1 million-token context window. Its architecture uses 24 physical decoder layers. The first and last six layers run once, while the 12 middle layers run three times, producing 48 logical layer visits per token. The middle layers reuse weights across passes rather than using a distinct set for each visit.

The model uses a mixture-of-experts architecture: at each layer visit, a routing system selects eight experts from a pool of 128. Thinking Machines says it is distributing expert weights in NVFP4 and MXFP8 formats and including a DSpark drafter intended to speed generation with a small memory cost. These design details describe the release, but do not by themselves establish actual memory use or speed on consumer devices.

The fixed-pass version is available through Transformers, llama.cpp and MLX; the company says SGLang supports day-one inference for datacenter deployments. A separate research checkpoint, TML-d24-varloop-research, lets users change the number of passes. The company says reducing passes can make inference faster, potentially at a cost to output quality.

At a glance
announcementWhen: Released; the supplied material is date…
The developmentThinking Machines Lab released Inkling 2 Mini, a model designed for local inference, alongside runtime support and a research checkpoint with adjustable repeated-layer passes.
AI Models · Briefing

Inkling 2 Mini for Local AI

Thinking Machines Lab’s new multimodal model combines reused layers with expert routing. Its practical speed, hardware needs and benchmark results still need outside validation.

26B
Total parameters, about
3B
Parameters active per token, about
1M
Token context window

A loop reuses model layers

The design reuses middle-layer weights to build capacity, while routing selects part of the expert pool at each layer visit.

1
Start with 24 layers

The first six and last six physical decoder layers run once.

2
Repeat the middle

The same 12 middle layers run in three passes, for 48 logical layer visits per token.

3
Route to experts

At each visit, the model selects eight experts from a pool of 128.

Options for inference

The release offers several runtimes and a research checkpoint with adjustable loop passes.

Fixed model

Multiple runtimes

Transformers, llama.cpp and MLX support the fixed-pass model; SGLang supports datacenter inference.

Research variant

Fewer passes

The research checkpoint allows one or two passes. The company says this can be faster, with a possible quality trade-off.

Claims need outside checks

The company reports strong benchmark comparisons, but the supplied material does not include independent replication or a full score table.

Reported results
  • Leads in 32 of 48 benchmarks against named models.
  • Reported tie with Qwen3.8-27B on Terminal-Bench 4.0.
  • Trails on AutomationBench-AA v1.0.6.
Still unclear
  • Complete memory requirements and device-specific speed.
  • Independent image and long-context performance.
  • Quality at reduced loop counts.

What local users should assess

Runtime availability is a starting point; actual usefulness depends on hardware and workload.

Memory useGeneration speedImage handlingLong-context behaviorOutput quality
CaveatThe release does not specify a complete file size or minimum hardware configuration.
The takeaway
Inkling 2 Mini brings an interesting local inference design, but independent tests and hardware-specific measurements will show whether its reported promise holds in practice.

A New Option for Local Model Users

The release adds another potential choice for people running AI models on their own machines or infrastructure. Its reused middle-layer weights and selective expert activation are intended to provide substantial model capacity without activating every expert for each token. Support across several established runtimes may also make it easier for developers to test the model in different environments.

Those features do not yet answer the central practical question: whether Inkling 2 Mini delivers useful performance on hardware that local users can access. The supplied release material gives no complete memory requirement or measured generation speeds across consumer devices. A model’s total parameter count, active parameter count and weight format all inform deployment, but none alone tells users whether a specific computer can run it at an acceptable speed.

For teams evaluating local multimodal systems, the company’s reported benchmark leads may be a reason to test the model, not a substitute for independent results. Image quality, long-context behavior, runtime compatibility and output quality at reduced pass counts all need practical evaluation.

Products Worth Considering

Apple AirPods Pro (2nd Generation) Wireless Earbuds with MagSafe Charging Case (Lightning)

Apple AirPods Pro (2nd Generation) Wireless Earbuds with MagSafe Charging Case (Lightning)

As an affiliate, we earn on qualifying purchases.

How Inkling Reuses Its Layers

The repeated-layer design is the release’s main architectural distinction. The model has 24 physical decoder layers, but runs its 12 middle layers three times. Learned messages carry information between passes, according to the technical description. The design therefore reuses weights, while still requiring computation for all 48 logical visits per token. The source says each visit also keeps its own key-value and short-convolution state, a detail that matters when estimating memory use.

Thinking Machines compares Mini with its Inkling 2 Small model. Both are described as performing 48 layer visits per token, but Mini has a 2,304 hidden width versus 4,608 for Small, and routes among 128 experts per layer rather than 384. The supplied material does not give a complete model file size or minimum hardware specification. It also notes that other work, including Nanbeige, has explored looping architectures; the release does not establish that this approach is new to the field.

The company reports that Inkling 2 Mini performs strongly on several evaluations, including the Artificial Analysis Intelligence Index v5 and comparisons with Gemma 4 31B, Muse Glimmer, Qwen3.8-27B and Nemotron 3.5 Super VL. It also reports a lead on FrontierCode v2.0, a tie with Qwen3.8-27B on Terminal-Bench 4.0 and a lower result on AutomationBench-AA v1.0.6. Those are company-reported comparisons; the provided material contains neither the full score table nor independent replication.

“Inkling 2 Mini is a mixture-of-experts model with 24 decoder layers, but the 12 middle layers run three times in a loop.”

— Thinking Machines Lab, in the supplied release material

Hardware and Benchmarks Still Unclear

The supplied information does not specify minimum memory, model file size or measured generation speeds on consumer hardware. It also does not show independent evaluations confirming the benchmark comparisons, or provide enough results to assess image understanding and the claimed million-token context in practice. A stated context window does not by itself show how well a model uses information across that length.

The quality trade-off from running the research checkpoint with one or two passes is also not fully assessed in the material. It is unclear which devices and runtime configurations can take advantage of the mixed-precision weights and DSpark drafter, or how performance varies among Transformers, llama.cpp, MLX and SGLang. These gaps mean local users cannot yet infer expected performance from the architecture description alone.

Model Cards and Reproducible Tests

The next evidence to watch for is a full model card and score table, followed by independent, reproducible benchmark results. Reports that identify hardware, runtime, quantization and generation settings would help users compare results rather than relying on broad performance claims.

Hands-on tests can also show whether image inputs and long prompts work as described, and whether fewer passes provide a useful speed gain without an unacceptable quality loss. Until those details are available, Inkling 2 Mini’s performance on specific local systems remains an open question.

Key Questions

What is Inkling 2 Mini?

It is a multimodal mixture-of-experts model released by Thinking Machines Lab. The company describes it as having about 26 billion total parameters, with about 3 billion active per token.

Can it run on a consumer PC?

The supplied material does not give a minimum hardware configuration or complete memory requirement. Whether it runs acceptably will depend on the device, runtime and model format; the release details alone do not settle that question.

Which inference tools are supported?

The fixed-pass model is listed for Transformers, llama.cpp and MLX. The company also says SGLang supports day-one inference for datacenter deployments.

Are the benchmark results independently verified?

Not in the material provided. Thinking Machines reports competitive results across several benchmarks, but the full score table and independent replication were not included.

What does the research checkpoint change?

The TML-d24-varloop-research checkpoint allows users to adjust how many times the middle layers run. The company says fewer passes can speed inference, with a possible quality trade-off, but the supplied material does not quantify that trade-off.

Primary source: Hugging Face · via ThorstenMeyerAI.com

You May Also Like

AI Data Centers In Europe: The Debate Behind 218 Arrests In Amsterdam

Police say 218 people were arrested after protesters entered a Microsoft-leased data center site in Amsterdam. All have been released.

Sony’s New PlayStation Portal Handheld Includes ‘Additional’ Internal Storage

Sony’s Japan-only PlayStation Portal model adds internal storage for unspecified future features and launches December 10 for 39,980 yen.

A First Look At Images 2.6 And AI Image Making

OpenAI has published an announcement titled “Introducing Images 2.6,” but available details do not confirm its features, access or release timing.

Kindle Scribe 64GB: A Holiday Review for Readers and Note-Takers

A practical holiday review of the 64GB Kindle Scribe: its paper-like display, writing tools, trade-offs, and the readers it suits best.