VoltgridNews

First published Changes log

Samsung's LittleBit: Llama 2 70B Would Need 1.98 GB at 0.1 Bit per Weight

Samsung Research's first LittleBit paper sizes Llama 2 70B at 1.98 GB at 0.1 bit per weight, 69.72x smaller than at 16 bits, though neither it nor its follow-up trains a 70B model. At 0.1 bit, LittleBit-2 cuts Llama 2 7B's average accuracy on five common-sense tests to 42.06% from 68.88%.

Local AI Desk

9 min read

Llama 2 70B at 0.1 bit per weight, computed
1.98GBagainst 138.04 GB at 16 bits
Llama 3 8B at 0.1 bit; 2.1 GB is 16-bit head and embedding
2.2GB0.1 GB of compressed blocks
Llama 2 7B average accuracy, LittleBit-2 at 0.1 bit
42.06%against 68.88% at 16 bits
Llama 3 8B perplexity at 1 bit, LittleBit-2; lower is better
11.53OneBit 13.09; 16-bit model 6.10

As ofSourceLittleBit paper, LittleBit-2 paper

Samsung Research's LittleBit method would store Llama 2 70B in 1.98 GB at 0.1 bits per weight, against 138.04 GB at 16 bits, by the size accounting in its NeurIPS 2025 paper. A weight is one of the model's learned numbers; the original stores each in 16 bits, and bits per weight is the average storage per weight in the layers that are compressed.

Key-facts card: Llama 2 70B would need 1.98 GB at 0.1 bit by formula, against 138.04 GB at 16 bits; Llama 2 7B average accuracy 42.06% with LittleBit-2 at 0.1 bit

LittleBit's size formula puts Llama 2 70B at 1.98 GB at 0.1 bit per weight, against 138.04 GB at 16 bits; neither paper trains a 70B model.

Graphic: Voltgrid News

Llama 2 70B in 1.98 GB? Samsung's LittleBit, by formula
Samsung Research's first LittleBit paper puts Llama 2 70B at 1.98 GB at 0.1 bit per weight, against 138.04 GB at 16 bits. It is a calculation: neither LittleBit paper trains a 70B model, and at 0.1 bit LittleBit-2 cuts Llama 2 7B's average accuracy to 42.06% from 68.88%.

That 69.72x cut is a calculation, not a tested model. The paper's experiments span models of 1.3 billion to 32 billion parameters, and its authors write that their attempts at training 70B-class models ran into resource limits. Where the two LittleBit papers do measure, on models up to 32B, the method beats STBLLM, the earlier sub-1-bit method both papers test, in nearly every setting, yet a 0.1-bit model gives up much of the original's quality.

What did Samsung Research publish?

Two papers and one code base. The first, LittleBit, by Banseok Lee, Dongkyu Kim, Youngcheon You and Youngmin Kim of Samsung Research, went up on arXiv on May 30, 2025, was last revised on Feb. 5, 2026, and was accepted at NeurIPS 2025 (paper). The second, LittleBit-2, by Banseok Lee and Youngmin Kim of Samsung Research in Seoul, followed in February 2026 and appears in the ICML 2026 proceedings (paper). The code for both sits under the SamsungLabs account on GitHub (repository), and Samsung Research summarized the first paper on its own blog (post).

The method works on the transformer blocks, which hold 13.0 GB of Llama 2 7B's 13.5 GB at 16 bits. It replaces each weight matrix in their linear layers with two much thinner matrices whose product approximates it, a low-rank factorization, then stores every entry of the thin matrices as plus or minus one, a step called binarization. Three small sets of 16-bit scales, for rows, columns and the thin inner dimension, restore magnitude, and a second, residual path of the same shape corrects the first. The width of the thin matrices sets the budget, so one recipe yields 1, 0.55, 0.3 or 0.1 bits per weight.

LittleBit-2 keeps that stored format and changes how training starts. It turns the two thin matrices by one shared rotation, which leaves their product unchanged, and searches for the rotation that pushes their values away from zero before they are reduced to signs; the authors call the search Joint-ITQ. The repository offers it as an opt-in setting with no added cost at inference, and the second paper counts the same memory for both versions.

How much memory does each bit width take?

At 16 bits, the first paper's memory table puts Llama 2 7B at 13.49 GB, 13B at 26.06 GB and 70B at 138.04 GB. Ollama's model library lists Llama 2 70B at 138 GB in 16-bit form and 39 GB as its default 4-bit file, against 3.8 GB for 7B and 7.4 GB for 13B (library).

Below 4 bits, the sizes are the papers' own calculations, not file sizes. LittleBit-2 counts 2.4 GB for Llama 2 7B and 4.2 GB for 13B at 2 bits with GPTQ, a compression method that keeps a 16-bit scale and zero point for every 128 weights. LittleBit's table puts Llama 2 70B at 9.72 GB in the format of OneBit, an earlier 1-bit training method, 5.83 GB at 0.55 bit, 3.70 GB at 0.3 bit and 1.98 GB at 0.1 bit.

Llama 2 70B takes 138.04 GB at 16 bits and 39 GB as a 4-bit file; LittleBit's formula puts it at 1.98 GB at 0.1 bit
Bits per weightLlama 2 7BLlama 2 13BLlama 2 70BSize from
16 (FP16)13.49 GB26.06 GB138.04 GBLittleBit paper
4 (q4_0 file)3.8 GB7.4 GB39 GBOllama library
2 (GPTQ, group-wise scales)2.4 GB4.2 GBNot givenLittleBit-2 paper
1 (OneBit)1.36 GB2.28 GB9.72 GBLittleBit paper
0.55 (LittleBit)0.98 GB1.51 GB5.83 GBLittleBit paper
0.3 (LittleBit)0.79 GB1.15 GB3.70 GBLittleBit paper
0.1 (LittleBit)0.63 GB0.84 GB1.98 GBLittleBit paper

NoteThe 4-bit row is Ollama's file size. The 2-bit label is nominal: counting GPTQ's 16-bit group scales and zero points, LittleBit-2 puts it at 2.25 bits per weight. The other rows are the papers' formulas, which count the 16-bit embedding and output head. Neither paper trains a 70B model.

SourceLittleBit paper, memory table, LittleBit-2 paper, Table 1, Ollama library

As of

What a 0.1-bit model contains matters as much as the headline. The bits-per-weight figure covers the compressed linear layers, counting their 16-bit scales and both paths. The embedding layer, which turns each word piece into numbers, and the output head, which turns the model's result back into a choice of word, stay at 16 bits, and the totals include them. LittleBit-2 splits the two for Llama 3 8B: at 0.1 bit the compressed blocks take 0.1 GB, but the whole model takes 2.2 GB, of which 2.1 GB is the 16-bit head and embedding. Against 16.1 GB at 16 bits, that is a 7.3x cut, not the 160x that 16 bits against 0.1 would suggest. The same 2.1 GB is why the 0.1-bit Llama 3 8B, at 2.2 GB, comes out larger than LittleBit's 1.98 GB for Llama 2 70B. The first paper's limits section warns that the output head can become a significant bottleneck in total size at such settings, especially for models with large vocabularies.

Llama 2 7B carries a lighter load: its head and embedding take 0.5 GB at 16 bits, against 2.1 GB for Llama 3 8B, so its whole-model cut is larger. The first paper's 0.63 GB for Llama 2 7B at 0.1 bit is 21.41x smaller than at 16 bits.

How much quality survives at each step?

The papers' main score is perplexity on the WikiText-2 benchmark, measured at a sequence length of 2,048 tokens in LittleBit-2. It gauges how well a model predicts real text, roughly how many equally likely choices it weighs for each next word: lower is better, and it is not a share of right answers. The 16-bit Llama 3 8B scores 6.10.

At about 1 bit, LittleBit-2 leads the methods its paper tests on Llama 3 8B with 11.53, against 13.09 for OneBit, an earlier 1-bit training method, and 16.30 for the first LittleBit. ARB-LLM and BiLLM, which reach 1.1 bits on average by keeping some weights at higher precision, score 27.63 and 59.37. On Llama 2 7B the race is closer: 8.27 for LittleBit-2, 8.36 for OneBit and 9.08 for LittleBit, against 5.47 at 16 bits. The first paper's table also lists BinaryMoS, a 1-bit training method with dynamic scaling, at 10.83 on Llama 3 8B and 7.74 on Llama 2 7B, ahead of both LittleBit versions; the second paper does not test it.

At 1 bit, LittleBit-2 scores 11.53 on Llama 3 8B, ahead of OneBit's 13.09

WikiText-2 perplexity of Llama 3 8B at about 1 bit per weight; lower is better. The 16-bit model scores 6.10

LittleBit-2, 1 bitLittleBit-2, 1 bit: 11.5311.53OneBit, 1 bitOneBit, 1 bit: 13.0913.09LittleBit, 1 bitLittleBit, 1 bit: 16.316.3ARB-LLM, 1.1 bitsARB-LLM, 1.1 bits: 27.6327.63BiLLM, 1.1 bitsBiLLM, 1.1 bits: 59.3759.37

NoteLittleBit and LittleBit-2 store the same bits; the second changes only how training starts. ARB-LLM and BiLLM keep some weights at higher precision. BinaryMoS, which this paper does not test, scores 10.83 at 1 bit in the first paper.

SourceLittleBit-2 paper, Table 1

As of

Data table
CategoryPerplexity
LittleBit-2, 1 bit11.53
OneBit, 1 bit13.09
LittleBit, 1 bit16.3
ARB-LLM, 1.1 bits27.63
BiLLM, 1.1 bits59.37
Download the data (CSV)

Scroll sideways to see the whole chart

Below 1 bit, the earlier method comes apart. STBLLM, the sub-1-bit method both papers test against, scores about 1,800 on Llama 2 7B at 0.3 bit in the first paper, where LittleBit scores 12.00, and LittleBit's 15.92 at 0.1 bit beats STBLLM's 19.17 at 0.7 bit, the comparison the first paper's abstract cites. On Llama 3 8B at 0.55 bit, STBLLM scores 241.96 and LittleBit-2 14.01.

LittleBit-2 also reports zero-shot accuracy, the average share of questions answered correctly without worked examples, on five common-sense tests: HellaSwag, ARC-Easy, ARC-Challenge, PIQA and WinoGrande. All five are multiple choice, so a model that guessed at random would still score well above zero. Llama 2 7B scores 68.88% at 16 bits, 62.93% with EfficientQAT at 2 bits and 55.79% with LittleBit-2 at 1 bit, against 54.72% for OneBit. It keeps 50.97% at 0.55 bit and 42.06% at 0.1 bit, against 43.40% for STBLLM at 0.55 bit and 38.79% for GPTQ at 2 bits.

At 0.1 bit, Llama 2 7B keeps 42.06% average accuracy, against 68.88% at 16 bits

Average zero-shot accuracy of Llama 2 7B on five common-sense tests, in percent, by method and bits per weight

16-bit original16-bit original: 68.88%68.88%EfficientQAT, 2 bitsEfficientQAT, 2 bits: 62.93%62.93%LittleBit-2, 1 bitLittleBit-2, 1 bit: 55.79%55.79%OneBit, 1 bitOneBit, 1 bit: 54.72%54.72%LittleBit-2, 0.55 bitLittleBit-2, 0.55 bit: 50.97%50.97%STBLLM, 0.55 bitSTBLLM, 0.55 bit: 43.4%43.4%LittleBit-2, 0.1 bitLittleBit-2, 0.1 bit: 42.06%42.06%GPTQ, 2 bitsGPTQ, 2 bits: 38.79%38.79%

NoteHigher is better. The tests are HellaSwag, ARC-Easy, ARC-Challenge, PIQA and WinoGrande. All scores are the authors' own runs.

SourceLittleBit-2 paper, Table 1

As of

Data table
CategoryAverage accuracy (%)
16-bit original68.88
EfficientQAT, 2 bits62.93
LittleBit-2, 1 bit55.79
OneBit, 1 bit54.72
LittleBit-2, 0.55 bit50.97
STBLLM, 0.55 bit43.4
LittleBit-2, 0.1 bit42.06
GPTQ, 2 bits38.79
Download the data (CSV)

Scroll sideways to see the whole chart

The first paper calls 0.3 to 0.55 bit the sweet spot and finds a quantization cliff between 0.3 and 0.1 bit. Its own text samples show the cost: prompted with a sentence about the Mona Lisa, a 0.1-bit Phi-4 calls the painting a statue linked to Milan's fashion world. The largest models tested lose ground too: QwQ-32B's perplexity goes from 6.34 at 16 bits to 35.26 at 0.1 bit in the first paper, and Gemma 3 27B from 4.58 to 16.38 with LittleBit-2.

One caveat applies to LittleBit's perplexity scores. Its training mix combines WikiText-2 with web text from C4, and the headline perplexity is measured on WikiText-2's validation split, text of the same kind the models trained on. The first paper reports the same ordering against STBLLM on C4 and PTB text.

Which local machines could hold the weights?

By weights alone, bit width moves Llama 2 70B across the whole range of desk machines. At 16 bits, its 138.04 GB is 10.04 GB more than the 128 GB of an NVIDIA DGX Spark (specs) but fits a Mac Studio M5 Ultra configured with 256 GB or 512 GB (specs; our comparison). The 39 GB 4-bit file leaves 89 GB of a DGX Spark or AMD's 128 GB Ryzen AI Halo free, and 57 GB of a 96 GB RTX PRO 6000, but is 15 GB too big for the 24 GB card in Dell's Pro Max Micro, where a 27B model already did not fit in StorageReview's tests (our story).

At 1 bit and below, 70B sizes drop into laptop range, on paper. OneBit's 9.72 GB would leave 6.28 GB of the 16 GB RTX 5060 Ti in a WiCi One free, but is 1.72 GB too big for the 8 GB RTX PRO 2000 in Lenovo's ThinkPad P1 Gen 9, on which StorageReview ran an 8B model at 4 bits (our story). LittleBit's 5.83 GB at 0.55 bit would fit that laptop GPU with 2.17 GB to spare, and its 1.98 GB at 0.1 bit would fit every machine here.

By weights alone, a 128 GB box holds Llama 2 70B as a 4-bit file; an 8 GB laptop GPU would take LittleBit's computed 0.55-bit size
MachineMemoryLargest form in the size tableWeightsRoom leftSize from
Mac Studio M5 Ultra, 512 GB option512 GB unified16-bit138.04 GB373.96 GBShipping format
NVIDIA DGX Spark128 GB unified4-bit file; 16-bit is 10.04 GB over39 GB89 GBShipping format
NVIDIA RTX PRO 6000 Blackwell96 GB GDDR74-bit file39 GB57 GBShipping format
Dell Pro Max Micro, RTX PRO 4000 SFF24 GB GDDR71-bit (OneBit); 4-bit is 15 GB over9.72 GB14.28 GBPaper's size only
WiCi One, RTX 5060 Ti16 GB GDDR71-bit (OneBit)9.72 GB6.28 GBPaper's size only
ThinkPad P1 Gen 9, RTX PRO 20008 GB GDDR70.55-bit LittleBit; 1-bit is 1.72 GB over5.83 GB2.17 GBPaper's size only

NoteWeights only. Makers do not say whether a GB is decimal or binary, and the KV cache, runtime and system take room first, so thin margins are uncertain. Neither paper trains a 70B model at the paper-only sizes. Ollama also lists an 8-bit 70B file of 73 GB, which fits the 128 GB and 96 GB machines.

SourceNVIDIA, DGX Spark specifications, Apple, Mac Studio specifications, LittleBit paper, memory table, Ollama library

As of

Three gaps separate those sizes from a working 70B model on a desk. Neither LittleBit paper trains a 70B model, so they give no quality figure at these sizes. The weights are not the whole memory bill: the KV cache that holds each conversation's context needs room on top (our analysis), though the authors estimate their format shrinks it by up to 21.3x for Llama 2 7B at 0.1 bit. And the format needs software that can run it.

Can anyone run it today?

Not as a ready-made model from Samsung. The repository is training code, and its README names no released compressed checkpoint. Since Oct. 9, 2026, others have posted models on Hugging Face adapted from LittleBit's papers and public code, among them three conversions of a Qwen3.8 27B derivative named for 0.1, 0.55 and 1 bit. Those bit widths cover only the feed-forward layers, no training was done during or after the conversion, and the uploader says the checkpoints did not keep the original's task performance. None is a Samsung release. The code compresses an existing model by quantization-aware training, in which the compressed model learns to match the original's outputs, and its setup installs NVIDIA's CUDA toolkit and a CUDA build of PyTorch. The first paper's recipe, which LittleBit-2 reuses, runs 5 epochs over about 1 billion tokens in all; it trained the largest, QwQ-32B, on 32 A100 GPUs.

The README lists OPT, Llama, Llama 2 and Llama 3, Phi-4, Qwen2.5, QwQ, Qwen3, Gemma 2 and Gemma 3 as supported. Its instructions cover training and evaluation, and it lists no support for desktop engines such as llama.cpp or Ollama. The code, the official implementation of the two papers, is licensed under CC BY-NC 4.0, which rules out commercial use.

Speed figures are the authors' own. With a custom CUDA kernel on an NVIDIA A100, a matrix product the size of one Llama 2 70B feed-forward layer ran 11.6x faster at 0.1 bit than at 16 bits with a batch of one, the potential speedup the abstract cites. End to end, a 0.1-bit Llama 2 7B decoded 203.20 tokens per second against 82.56 at 16 bits when generating 128 new tokens, or 2.46x, with attention and other parts left at 16 bits. The authors call the kernel a proof of concept.

Measured in the papers

  • Perplexity and zero-shot accuracy for models of 1.3 billion to 32 billion parameters, from 1 bit down to 0.1 bit.
  • LittleBit-2 against LittleBit at equal memory: perplexity of 11.53 against 16.30 on Llama 3 8B at 1 bit, where lower is better.
  • Text samples at 0.1 bit that keep their grammar but lose facts.

Computed or claimed by the authors

  • Llama 2 70B at 1.98 GB: a size from the paper's formula.
  • 11.6x faster on one 70B-sized layer on an A100, with the authors' own kernel.
  • 2.46x faster decoding of Llama 2 7B, end to end, on an A100.
  • A KV cache up to 21.3x smaller: an estimate.

Not shown in the papers or code

  • A trained 70B model and its quality.
  • Support in desktop engines such as llama.cpp or Ollama.
  • Independent replication: every result here is the authors' own.

Terms in this story

bits per weight
Average storage per original weight. For LittleBit it counts the compressed layers with their 16-bit scales, not the embedding or output head; other methods' 2-bit labels leave out their scales.
low-rank factorization
Approximating a large matrix as the product of two thin ones, which hold far fewer numbers.
binarization
Reducing each stored value to plus or minus one, a single bit, with separate scales to restore magnitude.
perplexity
How well a model predicts held-out text; lower is better. It is not a share of right answers.
zero-shot accuracy
The share of test questions a model answers correctly with no worked examples in the prompt.
quantization-aware training
Training a model with its compressed format in place so it adapts to the lost precision; it needs GPUs and training data.
KV cache
Memory that holds attention keys and values for the text so far; it grows with context length and the number of sessions.

How we counted

Sizes are the papers' own: LittleBit's memory table for Llama 2 and LittleBit-2's for the 2-bit row and Llama 3 8B, both counting the 16-bit embedding and head. The 4-bit row is Ollama's q4_0 file. Scores come from each paper's tables; the papers differ on STBLLM for Llama 2 7B at 0.55 bit (perplexity of 30.67 in the first, 32.93 in the second), and each chart uses one table. Machine memory is each maker's spec; room left is memory minus weights, leaving out the KV cache, runtime and operating system. As of Nov. 20, 2026.

What to watch

  1. Scheduled

    NeurIPS 2026 main conference in Sydney

    Atlanta and Paris sites run Dec. 9 to 11. The venue took the first LittleBit paper; watch for 70B-scale sub-1-bit results or independent replications.

  2. Scheduled

    ICML 2027 full-paper deadline

    The conference that published LittleBit-2 closes submissions for its 2027 meeting.

  3. Scheduled

    ICML 2027 main conference

    Papers accepted from the January round are presented.

How it compares

Full table
Samsung's LittleBit: Llama 2 70B Would Need 1.98 GB at 0.1 Bit per Weight: how the machine compares with other local AI hardware
ProductMemoryBandwidthAI computeMeasured speedPrice
ASUS ExpertCenter Pro ET900N G3ASUS (NVIDIA DGX Station GB300 platform)748 GB coherent: 252 GB HBM3e (GPU) + 496 GB LPDDR5X (CPU)[1]7.1 TB/s (GPU HBM3e); 396 GB/s (CPU LPDDR5X)[1]up to 20 PFLOPS AI performance (ASUS gives no precision or sparsity label) (sparse)[2]GPT-OSS-120B MXFP4, vLLM, 512/512, 1-128 streams: 9,280 aggregate output tok/s at 128 streams; 5,023 on 8,192/1,024 (StorageReview, ASUS-supplied unit, no add-in GPU)[3]$111,999.99 at Central Computers, US (ET900N G3-2SSD-V2, read Nov 8, 2026); no ASUS list price[4]
HP ZGX Fury AI StationHP252 GB HBM3e GPU + 496 GB LPDDR5x CPU (4x 128 GB SOCAMM); 748 GB unified[5]7.1 TB/s (GPU HBM3e); 396 GB/s (CPU LPDDR5x)[5]up to 20 petaFLOPS FP4 (HP gives no sparsity label) (sparse)[5]Two-unit cluster with an ASUS ET900N G3 over two 400G DACs: GLM-5.3 NVFP4 5,018 output tok/s at 256 streams, 512/512, vLLM 0.27.1 TP2 with MTP (StorageReview); no single-unit HP result[6]$132,000.00 on HP Store US (ZGX Fury G1n, 3-yr warranty, no graphics card, 4x 2 TB SED SSDs; Add to cart; read Nov 11, 2026)[7]
MSI XpertStation WS300MSI (NVIDIA DGX Station platform)252 GB HBM3e GPU + 496 GB LPDDR5X CPU (748 GB coherent)[8]7.1 TB/s (GPU HBM3e); 396 GB/s CPU LPDDR5X[8]—StorageReview vLLM: gpt-oss-120b MXFP4 9,294 output tok/s at 128 streams (512/512), 5,038 on 8,192/1,024; MiniMax M3 NVFP4 1,041 at 32 streams with EAGLE3-GQA at a forced acceptance length of 3 (best-case)[8]$99,999.00 'Start From' at Newegg US (model WS300T60L, 2x 2 TB NVMe RAID1), request-a-quote listing sold by Newegg (read Nov 15, 2026)[9]
NVIDIA DGX StationNVIDIA (systems built by ASUS, Dell, GIGABYTE, HP, MSI, Supermicro and Exxact)748 GB coherent: 252 GB HBM3e (GPU) + 496 GB LPDDR5X (CPU)[10]GPU HBM3e 7.1 TB/s; CPU LPDDR5X 396 GB/s; NVLink-C2C 900 GB/s between them[10]15 PFLOPS FP4 (without sparsity)[10]gpt-oss-120b MXFP4, vLLM, 512/512 chat workload: 9,280 output tok/s at 128 streams (MSI WS300 9,294), single GB300 (StorageReview, ASUS ExpertCenter Pro ET900N G3)[3]No NVIDIA list price; lowest US partner listing read Nov 4, 2026: Exxact Valence VWS-158270643 from $98,450 (Tom's Hardware found $94,930 in Aug); Dell Pro Precision with GB300 lists $160,195[11]
Supermicro Super AI Station (ARS-511GD-NB-LCC)Supermicro (NVIDIA DGX Station platform)748 GB total coherent: 252 GB HBM3e GPU + 496 GB LPDDR5X system[12]7.1 TB/s (GPU HBM3E); 396 GB/s (CPU LPDDR5X)[13]up to 20 PFLOPS FP4 (Supermicro gives no sparsity label; ServeTheHome describes it as sparse FP4)[12]Qwen3.8 Flash Next NVFP4, vLLM, tool-loop 4,096-in/512-out: several scenarios exceeded 3,000 output tok/s at 32 concurrent users, still rising (ServeTheHome; RTX PRO 2000 installed)[14]$92,887.47 before tax and shipping on Supermicro's online store (ARS-511GD-NB-LCC-01-G2); about $90,101.05 with a 3% new-customer discount (TechRadar, Sep 1, 2026)[15]
NVIDIA RTX PRO 6000 Blackwell Workstation EditionNVIDIA (sold through PNY, TD SYNNEX and workstation makers)96 GB[16]1,792 GB/s[16]4,000 AI TOPS (FP4, with sparsity) (sparse)[16]gpt-oss-120b MXFP4, stock vLLM, 512/512 chat workload: 3,064 output tok/s at 128 streams (1,169 on 8,192/1,024 prompts); Server Edition at 600W (StorageReview)[17]$16,000 on NVIDIA's US marketplace (Workstation Edition), listed out of stock[18]
Apple Mac Studio (M5 Ultra)Appleup to 512 GB unified (96 GB on 30-core M5 Ultra; 256 GB or 512 GB on 36-core/80-core M5 Ultra)[19]1.2 TB/s[19]—GLM-5.3-Flash MLX 4-bit with MTP on TensorFold, ISL 2048/OSL 512: 94.0 tok/s at 0 context, 96.5 at 128K, 79.4 at 512K; TTFT 1.1 s at 0, 66.3 s at 128K (Tom's Hardware, 256 GB M5 Ultra)[20]$22,299 minimum with 512 GB ($6,799 36-core CPU/80-core GPU M5 Ultra config plus $15,500 memory upgrade; Oct 28, 2026)[21]
Corsair FlexPrime V20C Workstation (CSW-9040004-NA)Corsair32 GB GDDR6 on the Arc Pro B70 GPU + 32 GB (2x 16 GB) DDR5 system memory[22]608 GB/s (GPU GDDR6)[22]—Qwen3.8 27B Q4_K_M, llama.cpp server: 19.8 tok/s at 1 request, 33.5 at 4, 35.4 at 8; TTFT 9,005 ms on 8,192-token prompt (StorageReview)[23]$3,999.99 on Corsair US store (32 GB DDR5, 1 TB SSD; read Nov 15, 2026)[22]
Show the other 34 machines
Samsung's LittleBit: Llama 2 70B Would Need 1.98 GB at 0.1 Bit per Weight: how the machine compares with other local AI hardware: the other machines
ProductMemoryBandwidthAI computeMeasured speedPrice
ASUS ProArt GR1XASUS64 GB or up to 128 GB LPDDR5X unified memory (ASUS spec page, read Nov 18, 2026)[24]300 GB/s (ASUS spec page, read Nov 18, 2026)[24]WITHDRAWN, not a sparse figure: ASUS and NVIDIA say 'up to 1 petaFLOP FP4' with no sparse or dense label; no field asserts either[25]Qwen3.8 27B Q4_K_M, llama.cpp CUDA, Windows 11: 13.1 tok/s output at 1 request, 31.4 at 4, 40.0 at 8 requests; TTFT 8.6 s (StorageReview, 64 GB unit, sr-client 1.13)[26]$4,499 (64 GB, 1 TB Gen4, Windows 11 Home) at ASUS eShop US; $4,499.99 at Best Buy, which lists 64 GB/2 TB at $4,999.99 (StorageReview, Nov 18, 2026)[26]
Mechrevo Master Road X10Mechrevo64 GB (with 1 TB SSD) or 128 GB (with 2 TB SSD) unified memory[27]300 GB/s (unified memory)[27]up to 1 PFLOP FP4 (sparse)[27]—CNY 32,999 (64 GB/1 TB) to CNY 49,999 (128 GB/2 TB), China only[27]
NVIDIA RTX Spark (N1X) PCsNVIDIA (MediaTek collaborated on the custom CPU design)Up to 128 GB LPDDR5X (20-core CPU / 6,144-core GPU version); up to 64 GB (18-core / 5,120-core laptop version)[25]Up to 300 GB/s: LPDDR5X-9400 on a 256-bit bus (NVIDIA's product page lists no bandwidth)[28]—gpt-oss:20b via Ollama in a Codex CLI coding agent, 64K context: 58.5 tok/s generation, 3 of 4 tasks passed (Dell XPS 16, 20-core N1X, 64 GB; StorageReview)[29]Laptops from $2,599; pricing and configurations vary by region[30]
AMD Ryzen AI Halo (Ryzen AI Max+ PRO 495, 192 GB)AMD192 GB LPDDR5X (256-bit)[31]273 GB/s[31]——$7,299.99 for the Linux version (released the week of Oct 12, 2026); Windows version coming soon[31]
ASUS Ascent GX10ASUS64 GB or 128 GB LPDDR5X-8533 unified[32]273 GB/s (8533 MT/s), as listed by StorageReview; ASUS's spec page states no bandwidth[33]Up to 1 PFLOP FP4 (GB10; NVIDIA's footnote: theoretical, using the sparsity feature). ASUS's pages say 1 petaFLOP / FP4 with no sparsity label (sparse)[34]gpt-oss-120b, vLLM online serving, equal ISL/OSL: 70 tok/s at batch 1 rising to 680 tok/s aggregate at batch 64 (StorageReview)[33]$5,999 (128 GB, 1 TB, GX10-GG0010BN, ASUS eShop US, Oct 30, 2026)[35]
Dell Pro Precision with GB10 (model FCM1253; launched as Dell Pro Max with GB10)Dell128 GB LPDDR5x up to 8533 MT/s onboard (64 GB option also listed)[36]273 GB/s (128 GB LPDDR5x, 256-bit)[37]—gpt-oss-120b, vLLM online serving, equal ISL/OSL: 68.57 tok/s at batch 1 rising to 670.83 tok/s aggregate at batch 64 (StorageReview)[38]$7,901.00 (default FCM1253, 128 GB, 2 TB Gen4 QLC) on Dell.com US, unchanged; 64 GB memory option listed at -$2,150.00, 4 TB TLC SED SSD +$2,450.00 (read Nov 20, 2026)[39]
Framework Desktop (Ryzen AI Max+ PRO 495, 192 GB)Framework192 GB LPDDR5X-8533 soldered[40]273 GB/s (256-bit)[40]——$6,799 (Max+ PRO 495 192 GB DIY Edition pre-order, Nov 3, 2026; storage not included)[41]
GIGABYTE AI TOP ATOMGIGABYTE128 GB unified system memory[42]273 GB/s (LPDDR5x-8533, 256-bit)[43]—gpt-oss-120b, vLLM online serving, equal ISL/OSL: 60.71 tok/s at batch 1 rising to 649.62 tok/s aggregate at batch 64 (StorageReview)[43]$7,499.99 (128 GB, 4 TB PCIe 4.0) and $9,405.63 (128 GB, 4 TB PCIe 5.0; more options from $7,999.99); 128 GB from $5,999.99 (1 TB) and $6,999.99 (2 TB) at Newegg US, read Nov 9, 2026[44]
GMKtec EVO-X5 Pro AI Mini PCGMKtec192 GB LPDDR5X-8533 onboard (8 x 24 GB, non-upgradeable); up to 160 GB allocatable as VRAM[45]Up to 273 GB/s[46]—GMKtec Laboratory table: GLM-5.3-Flash UD-IQ4_XS (146.05 GB) decode 17.79 tok/s, prefill 70.18, TTFT 14.71 s; Qwen3.8-Flash-Next ROCmFP4-FAST (93.48 GB) decode 45.08 tok/s[45]$6,799 (192 GB/2 TB) and $7,099 (192 GB/4 TB), all variants available, GMKtec US store, read Nov 9, 2026[45]
HP ZGX Nano AI Station (G1n)HP128 GB coherent unified system memory[47]273 GB/s (128 GB LPDDR5x)[48]GB10: up to 1 PFLOP FP4, theoretical, using the sparsity feature; HP and reviewers list 1,000 TOPS FP4 (sparse)[34]gpt-oss-120b, vLLM online serving, equal ISL/OSL: 69.6 tok/s at batch 1 rising to 722.9 tok/s aggregate at batch 64 (StorageReview)[49]$7,399.00 at the HP Store for the 4 TB model; $4,759 on Amazon (Notebookcheck, review updated Oct 5, 2026)[50]
Lenovo ThinkStation PGXLenovo128 GB LPDDR5X unified (256-bit)[51]273 GB/s[51]Up to 1 PFLOP FP4 (Lenovo PSREF gives no sparsity label; XDA says NVIDIA's figure assumes structured sparsity) (sparse)[51]Qwen3-Coder-Next FP8 (FP8 KV cache), vLLM NGC container, one request at a time, 170K context: 25-40 tok/s (XDA Developers, Lenovo-loaned unit)[52]US from $5,079 (1 TB base model) on Lenovo's site, per TechRadar's review (Jun 9, 2026); UK from GBP 5,200[53]
MSI EdgeMesa N AI+ (128 GB, X-005US)MSI128 GB LPDDR5X unified system memory[54]273 GB/s[54]—Qwen3.8 27B Q4_K_M, llama.cpp CUDA, Windows 11: 13.0 tok/s output at 1 request, 31.1 at 4, 39.8 at 8 requests (StorageReview, 128 GB unit, default 62.5 GB GPU allocation)[55]$7,499.00 (128 GB, 4 TB, X-005US, MSI US store, out of stock, read Nov 21, 2026)[54]
MSI EdgeXpert (MS-C931)MSI64 GB or 128 GB LPDDR5x unified (256-bit)[56]273 GB/s[56]1 PFLOPS FP4 (sparse)[56]—No list price; MSI sells via quote (Get a quote)[56]
NVIDIA DGX SparkNVIDIA (partner GB10 systems from Acer, ASUS, Dell, GIGABYTE, HP, Lenovo and MSI)128 GB; a 64 GB configuration is sold only through OEM partners[34]273 GB/s[34]Up to 1 PFLOP FP4 (theoretical, using the sparsity feature) (sparse)[34]Qwen 3.8 27B (dense) with multi-token prediction: decode of about 20 tokens/s, with time-to-first-token staying interactive out to the full 262K context.[57]$6,950.00 (128 GB, 4 TB) on NVIDIA's enterprise marketplace with an Add To Cart button in a copy cached 07:59 UTC on 2026-10-31; stock unverified (listed out of stock on 2026-10-21); 64 GB partner systems from $4,999[58]
NVIDIA DGX Spark 64 GBNVIDIA (sold by Acer, ASUS, Dell, Gigabyte, HP and MSI)64 GB LPDDR5X unified (two units pool 128 GB over ConnectX-7)[59]273 GB/s (256-bit; NVIDIA spec table covers both 64 GB and 128 GB)[34]1 PFLOPS FP4 (sparse; NVIDIA spec table covers both memory configurations)[34]—from $4,999 (partner systems, Oct 2026; no Founders Edition)[59]
Sipeed MaixStationSipeed128 GB LPDDR5X (256-bit)[31]273 GB/s (Sipeed claim)[31]256 TOPS FP4 (native FP4, no sparsity; Axera/Sipeed claim)[31]—about $3,999 expected for 128 GB, with Kickstarter launch discounts (Sipeed to Tom's Hardware, Oct 2026)[31]
AMD Ryzen AI Halo Developer PlatformAMD128 GB LPDDR5x-8000 unified[60]256 GB/s[60]—gpt-oss-120b, vLLM online serving, equal ISL/OSL 256/256: 222 tok/s aggregate at batch 64 vs 701 on DGX Spark (StorageReview)[61]$3,999 with 2 TB SSD (Jul 2026)[61]
Framework Desktop (Ryzen AI Max+ 395)Framework128 GB LPDDR5x-8000 soldered (32 GB and 64 GB options)[62]256 GB/s[63]—gpt-oss-120b MXFP4 in LM Studio (out-of-the-box setup): roughly 45 tok/s (ServeTheHome; Ryzen AI Max+ 395, 128 GB, Windows 11 Pro)[64]$3,449 (DIY Edition, Max+ 395 with 128 GB, system only; out of stock, Nov 3, 2026)[65]
Acer Veriton GN100 AI Mini WorkstationAcer128 GB LPDDR5x coherent unified[66]——gpt-oss-120b, vLLM, total token throughput: prefill heavy 303.61 to 2,777.56 tok/s; decode heavy 38.38 to 292.41 tok/s (batch 1 to 64, StorageReview)[67]from $3,999 in North America (announced Sep 3, 2025; prices vary by region)[66]
ASRock Industrial AI BOX-A395ASRock Industrial128 GB LPDDR5X-8000 soldered[68]——AgentSTH V7, 32 threads: 15% ahead of the next-closest Ryzen AI Max+ 395 box (single-core within 1%); estimated SPEC CPU2026 int rate: top of a 44% spread across four 395 systems; measured by ServeTheHome, Ubuntu 26.04[69]—
Beelink GTR9 Pro (Ryzen AI Max+ 495, 128 GB)Beelink128 GB LPDDR5X-8533 unified (soldered)[70]—132 AI TOPS on Beelink's page; AMD lists the PRO 495 at Overall TOPS up to 132 and NPU TOPS up to 55; no precision stated by either[70]4.24 tok/s, LM Studio, Qwen3.8-35B-A3B (Beelink's own figure on its store page; quantization, context, backend and RAM size not stated)[70]$4,799 (128 GB/2 TB), Beelink store, pre-order (read Nov 13, 2026)[70]
Beelink GTR9 Pro (Ryzen AI Max+ 495, 192 GB)Beelink192 GB LPDDR5X-8533 unified (soldered)[70]—132 AI TOPS on Beelink's page; AMD lists the PRO 495 at Overall TOPS up to 132 and NPU TOPS up to 55; no precision stated by either[70]4.24 tok/s, LM Studio, Qwen3.8-35B-A3B (Beelink's own figure on its store page; quantization, context, backend and RAM size not stated)[70]$7,399 (192 GB/2 TB) or $7,799 (192 GB/4 TB), Beelink store, pre-order (read Nov 13, 2026)[70]
Bosgame M5 MAX AI Mini PCBosgame192 GB unified LPDDR5 (soldered); up to 160 GB allocatable to the GPU[71]———$6,199 introductory first-batch pre-order price (192 GB, 2 TB PCIe 4.0 SSD), from Nov 3, 2026, until the first allocation sells out[71]
Dell Pro Max Micro (FCM2250)Dell24 GB GDDR7 on the RTX PRO 4000 SFF (23.9 GB usable measured) + 96 GB DDR5-6400 CSODIMM system RAM in StorageReview's tested build[72]——Ministral 3 8B, llama.cpp server (CUDA), 4,096-in/1,024-out: 52.0 tok/s at 1 request, 108.9 at 4; TTFT 1,477 ms (StorageReview, RTX PRO 4000 SFF 24 GB, driver 595.99.02, Ubuntu 26.04.1)[72]From $1,449.00 on Dell.com US; RTX PRO 4000 SFF 24 GB option +$3,870.00 (read Nov 16, 2026)[73]
Dell Pro Precision 9 T4DellUp to 2x 96 GB GDDR7 (two RTX PRO 6000 Max-Q) + up to 4 TB DDR5 RDIMM in 16 slots, 8 channels[74]——MiniMax M2.7 Q4_K_M (129.3 GB) split across 2x RTX PRO 6000 Max-Q: 83.7 tok/s one user, 127.3 peak; 30.3 on one card plus system RAM (StorageReview, llama.cpp b11716, Ubuntu 26.04.1)[74]From $5,925.00 on Dell.com US; dual RTX PRO 6000 Max-Q option +$30,820.00 (read Nov 14, 2026). StorageReview's tested build (Xeon 698X, 256 GB, 2x Max-Q) about $66,600[75]
Dell Pro Precision with GB300 (model FCT6263; announced as Dell Pro Max with GB300)Dell252 GB HBM3e GPU + 496 GB LPDDR5X CPU[76]———$160,195.00 (Dell.com US default config FCT6263, read Nov 8, 2026)[76]
Gigabyte AI TOP Aeroup to 128 GB LPDDR5X unified[77]————
GMKtec EVO-X2 AI Mini PCGMKtec64 GB or 128 GB onboard LPDDR5X-8000 (non-upgradeable)[78]——Llama 3.3 70B Q6_K Instruct: 3.7-3.8 tok/s generation fully on the Radeon 8060S (about 2 tok/s CPU-only); Qwen3-32B 10.4-10.5 tok/s; 128 GB unit, short prompt, tool not named (ServeTheHome)[79]$3,499.99 (128 GB/1 TB) and $3,649.99 (128 GB/2 TB), each against a $3,999.99 compare-at price (GMKtec US store, read Nov 11, 2026)[78]
HP Z2 Mini G1a WorkstationHPup to 128 GB unified, up to 96 GB assignable to the GPU, up to 8000 MT/s[80]—up to 50 NPU TOPS[80]gpt-oss-120b GGUF (63.39 GB, full GPU offload) in LM Studio: close to 40 tok/s (StorageReview)[81]$4,781 list for StorageReview's review configuration; about $3,300 street (B&H top spec with 2 TB SSD at $3,342.65, Aug 2025)[81]
Microsoft Surface RTX Spark Dev BoxMicrosoft128 GB unified memory[82]—up to 1 PFLOP of AI compute (Microsoft; precision and sparsity not stated on the page) (sparse)[82]—From $5,999.99 (Microsoft Store US, pre-order, read Nov 1, 2026)[82]
MSI EdgeMesa N AI+ (64 GB)————$4,399 (64 GB / 1 TB, after MSI's cut from $4,699; Nov 2026)[83]
Tenstorrent TT-QuietBox 2 (Blackhole)Tenstorrent128 GB GDDR6 on the accelerators + 256 GB DDR5-5600 system memory[84]———$9,999 (Oct 30, 2026)[84]
WiCi One (RTX 5060 Ti edition)WiCi16 GB GDDR7 VRAM; 1 TB NVMe 4 SSD for TurboStream weight paging (WiCi's claim: NVMe used as working VRAM). System RAM not stated[85]———$2,599 list (WiCi shop, Oct 2026); a $600-off code (WICIWELCOME) is applied at checkout. Early-access price was $1,999 (Notebookcheck, Engadget, Sep 2026)[85]
WiCi One (RTX 5090 edition)WiCi32 GB GDDR7 VRAM; 4 TB NVMe 5 SSD for TurboStream weight paging (WiCi's claim: NVMe used as working VRAM). System RAM not stated[85]———$9,999 list (WiCi shop, Oct 2026); a $600-off code (WICIWELCOME) is applied at checkout[85]

Italic values are reported by an outlet or measured by an independent lab; the rest come from the maker. Measured speeds use different models, quantizations and tools, so read each with its source rather than against the others. Voltgrid News does not test hardware itself.

Sources (85)
  1. [1] ASUS
  2. [2] ASUS
  3. [3] StorageReview
  4. [4] Central Computers
  5. [5] HP
  6. [6] StorageReview
  7. [7] HP
  8. [8] StorageReview
  9. [9] Newegg
  10. [10] NVIDIA
  11. [11] Exxact
  12. [12] Supermicro
  13. [13] ServeTheHome
  14. [14] ServeTheHome
  15. [15] TechRadar
  16. [16] NVIDIA
  17. [17] StorageReview
  18. [18] NVIDIA
  19. [19] Apple
  20. [20] Tom's Hardware
  21. [21] MacRumors
  22. [22] Corsair
  23. [23] StorageReview
  24. [24] ASUS
  25. [25] NVIDIA
  26. [26] StorageReview
  27. [27] Notebookcheck
  28. [28] ServeTheHome
  29. [29] StorageReview
  30. [30] NVIDIA
  31. [31] Tom's Hardware
  32. [32] ASUS
  33. [33] StorageReview
  34. [34] NVIDIA
  35. [35] ASUS eShop
  36. [36] Dell
  37. [37] HotHardware
  38. [38] StorageReview
  39. [39] Dell
  40. [40] Framework Blog
  41. [41] Framework
  42. [42] GIGABYTE (PR Newswire)
  43. [43] StorageReview
  44. [44] Newegg
  45. [45] GMKtec
  46. [46] GMKtec
  47. [47] HP
  48. [48] HotHardware
  49. [49] StorageReview
  50. [50] Notebookcheck
  51. [51] Lenovo PSREF
  52. [52] XDA Developers
  53. [53] TechRadar
  54. [54] MSI
  55. [55] StorageReview
  56. [56] MSI
  57. [57] Tom's Hardware
  58. [58] NVIDIA Marketplace
  59. [59] NVIDIA Blog
  60. [60] AMD
  61. [61] StorageReview
  62. [62] Framework
  63. [63] Framework Blog
  64. [64] ServeTheHome
  65. [65] Framework
  66. [66] Acer
  67. [67] StorageReview
  68. [68] ServeTheHome
  69. [69] ServeTheHome
  70. [70] Beelink
  71. [71] Notebookcheck
  72. [72] StorageReview
  73. [73] Dell
  74. [74] StorageReview
  75. [75] Dell
  76. [76] Dell
  77. [77] Notebookcheck
  78. [78] GMKtec
  79. [79] ServeTheHome
  80. [80] HP
  81. [81] StorageReview
  82. [82] Microsoft
  83. [83] Notebookcheck
  84. [84] Tenstorrent
  85. [85] WiCi

Changes

  1. First published.

Sources

Published
Sources
6 cited