neuphonic

you can try it here: huggingface.co model: huggingface.co repo: neuphonic/neudecide blogpost: neuphonic.com

We chose Apache 2.0 so you can download NeuDecide, run it yourself and build on it. We’d love your feedback, and appreciate the work TypeSafe, Cactus and others are doing in this space. Blog: neuphonic.com Python: neuphonic/neudecide Model: huggingface.co

We’re open sourcing NeuDecide: a 43 MB audio-to-tool model with a WASM browser demo We’re the team at Neuphonic, and we’re open sourcing NeuDecide under Apache 2.0. It takes audio and tool definitions and returns a tool call with arguments, without an intermediate transcription step. The model files total 43 MB, and inference runs on a single CPU thread. Try the WASM demo in your browser: select a preset or define your own tools, record or upload audio, and inspect the returned tool call. Demo link i.redd.it i.redd.it Performance On SLURP’s tool-only task with 10 tools available, NeuDecide achieves 72.4% tool accuracy directly from speech, without transcription: - ~3× that of Nvidia Parakeet + Google FunctionGemma (24.4%). - ~3.5× that of Cactus (Whistle + Needle) (20.7%). preview.redd.it preview.redd.it Running on a single CPU thread: - MacBook Pro M3: 46 ms time to call, 159 ms loading time, 149 MB peak RAM. - Samsung S24+: 82 ms time to call, 267 ms loading time, 174 MB peak RAM. - Raspberry Pi 5: 206 ms time to call, 499 ms loading time, 146 MB peak RAM. How it works The export contains three ONNX graphs: - An audio encoder processes the speech. - A tool encoder combines the audio representations with tokenised JSON tool definitions. - A decoder generates the tool call token by token, using cached keys and values. The tool list is an input to each request, so changing the available actions doesn’t require retraining. preview.redd.it preview.redd.it Try it with your own tools The project grew out of our work with robotics partners who needed voice control on limited hardware. The demo includes editable presets for robot vacuums, car controls and smart homes, alongside a custom option for testing your own tool definitions. We’ve also packaged NeuDecide for Python so you can run inference locally and test it with your own tool definitions. We chose Apache 2.0 to make it easier for people to build on the model and contribute. We’ve enjoyed seeing the work from TypeSafe, Cactus and others in this space, and hope this adds something useful. Technical write-up: neuphonic.com Python package: neuphonic/neudecide Model on Hugging Face: huggingface.co If you try it, we’d be interested in your hardware, tool definitions and any requests it struggles with.

We’re open sourcing NeuDecide: a 43 MB audio-to-tool model with a WASM browser demo We’re the team at Neuphonic, and we’re open sourcing NeuDecide under Apache 2.0. It takes audio and tool definitions and returns a tool call with arguments, without an intermediate transcription step. The model files total 43 MB, and inference runs on a single CPU thread. Try the WASM demo in your browser: select a preset or define your own tools, record or upload audio, and inspect the returned tool call. Demo link i.redd.it Performance On SLURP’s tool-only task with 10 tools available, NeuDecide achieves 72.4% tool accuracy directly from speech, without transcription: - ~3× that of Nvidia Parakeet + Google FunctionGemma (24.4%). - ~3.5× that of Cactus (Whistle + Needle) (20.7%). preview.redd.it Running on a single CPU thread: - MacBook Pro M3: 46 ms time to call, 159 ms loading time, 149 MB peak RAM. - Samsung S24+: 82 ms time to call, 267 ms loading time, 174 MB peak RAM. - Raspberry Pi 5: 206 ms time to call, 499 ms loading time, 146 MB peak RAM. How it works The export contains three ONNX graphs: - An audio encoder processes the speech. - A tool encoder combines the audio representations with tokenised JSON tool definitions. - A decoder generates the tool call token by token, using cached keys and values. The tool list is an input to each request, so changing the available actions doesn’t require retraining. preview.redd.it Try it with your own tools The project grew out of our work with robotics partners who needed voice control on limited hardware. The demo includes editable presets for robot vacuums, car controls and smart homes, alongside a custom option for testing your own tool definitions. We’ve also packaged NeuDecide for Python so you can run inference locally and test it with your own tool definitions. We chose Apache 2.0 to make it easier for people to build on the model and contribute. We’ve enjoyed seeing the work from TypeSafe, Cactus and others in this space, and hope this adds something useful. Technical write-up: neuphonic.com Python package: neuphonic/neudecide Model on Hugging Face: huggingface.co If you try it, we’d be interested in your hardware, tool definitions and any requests it struggles with.