@sriramk I'd go further and say you need a harness that watches over what your AI is doing. Otherwise you're asking for trouble. Check out this thing we build for electric: @QuestHarness electric-capital/quest
1 like · 32 views
We just turned on our github discussions page. Come say hi and feel free to ask any questions!
github.com/electric-capital/quest/discussions
2 likes · 2 reposts · 181 views
Privacy is not just about local inference
I'm guessing a lot of people here are interested in local models because it means you're not sending all your data up to some cloud inference provider. But what I find weird is that people will go through all this trouble to set up a model and then run a harness basically in yolo mode on top of that.
I'm at a small financial firm and when I was trying to figure out how to deploy AI to the company I kept running into this problem. Even if you could run the model locally, if the harness on top allowed the AI to do risky things with sensitive data, then you haven't really solved the privacy/confidentiality problem.
Coding-focused harnesses especially tend to have this problem, because they usually assume more liberal permissions and secured environments to run in.
We ended up building our own harness because we couldn't find a good solution. It runs on prem, has pluggable inference backends (including local, of course, we're running Qwen3.8 27b 4bit on an ninfer setup), but also has strict categorization of any integration tools that makes it so that the model can access your data, but can't autonomously send it anywhere without your permission.
Our ~30 person firm is using it successfully every day, and I thought it might be worth sharing here since many folks here might have the same goals: electric-capital/quest
45 points · 13 comments
@yozm_it Also make sure your harness protects your data too! We actually had to build one ourselves and we open sourced it so you can try it for free: @QuestHarness
electric-capital/quest
1 like · 54 views
@sundarpichai @googlecloud Now if you want all this, but self-hosted, true control over inference (including local models), not tied only to google-approved integrations, and open-source, check out Quest: electric-capital/quest
2 likes · 164 views
Now if you want all this, but self-hosted, true control over inference (including local models), not tied only to google-approved integrations, and open-source, check out Quest 😉
electric-capital/quest
2 likes · 302 views
Mentions
@sriramk I'd go further and say you need a harness that watches over what your AI is doing. Otherwise you're asking for trouble. Check out this thing we build for electric: @QuestHarness electric-capital/quest
We just turned on our github discussions page. Come say hi and feel free to ask any questions! github.com/electric-capital/quest/discussions
Privacy is not just about local inference I'm guessing a lot of people here are interested in local models because it means you're not sending all your data up to some cloud inference provider. But what I find weird is that people will go through all this trouble to set up a model and then run a harness basically in yolo mode on top of that. I'm at a small financial firm and when I was trying to figure out how to deploy AI to the company I kept running into this problem. Even if you could run the model locally, if the harness on top allowed the AI to do risky things with sensitive data, then you haven't really solved the privacy/confidentiality problem. Coding-focused harnesses especially tend to have this problem, because they usually assume more liberal permissions and secured environments to run in. We ended up building our own harness because we couldn't find a good solution. It runs on prem, has pluggable inference backends (including local, of course, we're running Qwen3.8 27b 4bit on an ninfer setup), but also has strict categorization of any integration tools that makes it so that the model can access your data, but can't autonomously send it anywhere without your permission. Our ~30 person firm is using it successfully every day, and I thought it might be worth sharing here since many folks here might have the same goals: electric-capital/quest
@yozm_it Also make sure your harness protects your data too! We actually had to build one ourselves and we open sourced it so you can try it for free: @QuestHarness electric-capital/quest
@sundarpichai @googlecloud Now if you want all this, but self-hosted, true control over inference (including local models), not tied only to google-approved integrations, and open-source, check out Quest: electric-capital/quest
Now if you want all this, but self-hosted, true control over inference (including local models), not tied only to google-approved integrations, and open-source, check out Quest 😉 electric-capital/quest