AI Reasoning · Oracle · Apple Machine Learning
Arbitrage: Efficient Reasoning via Advantage-Aware Speculation
Compiled by KHAO Editorial — aggregated from 1 source + 4 references discovered via search. See llms.txt for citation guidance.
★ Tier-1 Source
Arbitrage: Efficient Reasoning via Advantage-Aware Speculation.
Key facts
- Authors Monishwaran Maheswaran†*, Rishabh Tiwari†*, Yuezhou Hu†*, Kerem Dilmen†, Coleman Hooper†, Haocheng Xi†, Nicholas Lee†, Mehrdad Farajtabar, Michael W
- Across multiple mathematical reasoning benchmarks, ARBITRAGE consistently surpasses prior step-level SD baselines, reducing inference latency by up to ∼ 2× at matched accuracy
- Arbitrage: Efficient Reasoning via Advantage-Aware Speculation
- However, due to unnecessary rejections caused by token mismatches in semantically equivalent steps, traditional token-level Speculative Decoding struggles in reasoning tasks
Summary
Authors Monishwaran Maheswaran†*, Rishabh Tiwari†*, Yuezhou Hu†*, Kerem Dilmen†, Coleman Hooper†, Haocheng Xi†, Nicholas Lee†, Mehrdad Farajtabar, Michael W. Modern Large Language Models achieve impressive reasoning capabilities with long Chain of Thoughts, but they incur substantial computational cost during inference, and this motivates techniques to improve the performance-cost ratio. This routing approximates an ideal ARBITRAGE ORACLE that always chooses the higher-quality step, achieving near-optimal efficiency–accuracy trade-offs. Across multiple mathematical reasoning benchmarks, ARBITRAGE consistently surpasses prior step-level SD baselines, reducing inference latency by up to ∼ 2× at matched accuracy.