Compute · subq.ai
AutomationBench Finance at 13% places SubQ 1.1 Small close to the strongest models on that benchmark
Compiled by KHAO Editorial — aggregated from 1 source. See llms.txt for citation guidance.
◌ Single Source
SSA replaces the O(n²) dense attention pass with a learned sparse formulation that scales linearly with context length.
Key facts
- SubQ 1.1 Small scores near-perfect at 1M, 2M, 6M, and 12M tokens
- The model was trained predominantly at 1M tokens yet the retrieval held near perfectly at 12x that length, despite compressing attention to 0.13% of relationships
- At 1M tokens, SubQ requires 64.5x fewer compute than dense attention and runs 56x faster than FlashAttention-2 on a single attention layer
- SubQ uses 64.5x less compute than dense attention, and is 56× faster than FlashAttention-2 at 1M-token context
Summary
For years, the industry worked around this problem by building retrieval pipelines, chunking strategies, and agentic scaffolding, useful tools, but ultimately workarounds for context limitations of the model architecture. SubQ is built to remove that constraint. Near-perfect long-context retrieval up to 12M tokens on the needle-in-a-haystack test, with up to nearly 1,000x attention compute reduction. A balance of long-context optimization and general reasoning ability, with strong performance retained across knowledge, coding, and non-coding enterprise agent benchmarks. At 1M tokens, SubQ 1.1 Small requires 64.5x less compute than dense attention and runs 56x faster than FlashAttention-2.