ml-rsi-inference-bench

苹果放出一个让 agent 改推理引擎的基准 37 个任务跟着一个模型走完在 vLLM 或 SGLang 里的生命周期，上线当天支持、发布后调优、回归时修好，评分靠真的把引擎跑起来。 apple-aiml-research/ml-rsi-inference-bench t.co

@onloglogn great work - was just looking at some of the code, and it seems like your oracles just have the nop solution instead? github.com/apple-aiml-research/ml-rsi-inference-b…

Huge thanks to my brilliant collaborators @richard_baihe , @UnderGroundJeg , and @YizheZhangNLP , and to my apple advisor @onloglogn for their support and guidance! 🙏 🌐 Website: rsi-inference.com 💻 Code: apple-aiml-research/ml-rsi-inference-bench (4/4)

Introducing RSI-Inference Bench from 🍎 Apple ML Research 🌐 rsi-inference.com 💻 apple-aiml-research/ml-rsi-inference-bench Can AI agents improve the systems that run AI? We spent $100K API tokens, 50K GPU hours to find out. We built environments and benchmark from real-world #vLLM and #SGLang tasks, covering the full lifecycle of model inference engineering: 🚀 Day-0 Model Support ⚡ Inference Optimization 🔧 Bug Fix → Unsaturated: best agent at 26.7% pass@1 → Comprehensive coverage of the inference stack → Verifiable rewards: binary and continuous Watch the video to find out more!