Llama · Samsung · ARM · Hugging Face
Apply the files with llama.cpp or any runtime that supports GGUF Q4_0 artifacts
Compiled by KHAO Editorial — aggregated from 1 source. See llms.txt for citation guidance.
★ Tier-1 Source
The QAD GGUFs are.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B.
Key facts
- The team measure decode throughput for the four models LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B across four targets: MacBook Pro, NucBox EVO-X2, Samsung Galaxy S26 Ultra
- The 230M and 350M QAD Q4_0 checkpoints match Q5_K_M quality within evaluation variance at a 4-33% higher decode throughput
- The 1.2B and 2.6B QAD Q4_0 checkpoints match Q4_K_M quality at a 3-14% higher throughput
- The team also add one scale-appropriate math evaluation: GSM8K for LFM2.5-230M and LFM2.5-350M, and AIME25 for LFM2.5-1.2B-Instruct and LFM2.5-2.6B
Summary
Trained with Quantization-Aware Distillation (QAD): a high-precision teacher model is distilled into a quantized student model. Same memory and speed as native Q4_0: They keep the low memory footprint and high throughput of Q4_0 GGUFs. For all four models, they compare their released GGUFs produced with post-training quantization (PTQ) against the trained QAD Q4_0 checkpoints on a benchmark suite spanning reasoning, instruction-following, tool use, and agentic capabilities: GPQA Diamond, MMLU-Pro, IFEval, IFBench, Multi-IF, and BFCLv4. Across all four models, QAD substantially improves the Q4_0 checkpoint. The team measure decode throughput for the four models LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B across four targets: MacBook Pro, NucBox EVO-X2, Samsung Galaxy S26 Ultra, and Raspberry Pi 5.