The harness is your attack surface
You install a coding agent and point it at a repo. Which files can it read, and what stops it?
For some setups the honest answer is "everything your user can." The harness gives you control points, but the boundary and the policy are yours.
A note on method. I read the current documentation of two open-source agent projects, OpenHands and the Strands Agents Python SDK, plus a harness-engineering survey. Every factual statement below carries a marker like `` that maps to one of those three sources. Where I give an opinion or a reading of a README, I say so. I do not call any project "safer" than another, because I found no comparison made on equal terms.
1. The default question: where does the agent run?
Start with the most uncomfortable line in the OpenHands README. It documents that running the agent without a sandbox gives it full access to the host filesystem.
What matters is where the code runs in that mode. Without a sandbox, the OpenHands agent-server runs directly on the host machine rather than in a container.
Fine for a throwaway laptop. A very different choice on a developer machine that holds SSH keys, cloud credentials and ten other repositories. I'd settle the run mode before the first task, not after the first surprise.
A cheap habit: in your own wrapper, print the run mode at the start of every task and stop if it is not the one you expect.
2. Docker helps, but only as far as you mount
OpenHands can run each new conversation in its own Docker container, with its own Agent Server and tools. That narrows what one conversation can break.
!Two columns: a narrow mount exposes one project for one task; a broad PROJECTS_PATH exposes every repository under it.
The catch is the mount. In the Docker sandbox option, the agent can access any project under PROJECTS_PATH, a host directory the user chooses. The container is only as narrow as the directory you hand it.
Here is the difference, as an illustration of the idea rather than a copy of any official setup:
# Narrow: one project only. The narrow path is the whole point: one task, one repo.
export PROJECTS_PATH="$HOME/work/payments-service"
# Broad: every project you have ever cloned (commented out so this block is safe to paste)
# export PROJECTS_PATH="$HOME/work"
With the second value, a task about one service can reach every other repository under that directory, because the documented behaviour is access to any project under the path.
3. The control plane sits outside the sandbox
Isolating where the agent executes does not harden the service around it. Per-conversation Docker isolation in OpenHands does not sandbox the whole Canvas or automation service. Treat that service as its own system to harden.
For internet-facing installs, OpenHands points readers to its self-hosting guide for security hardening, and the project is MIT licensed. I did not open that guide, so read it yourself before exposing an install.
4. What Strands gives you: control points, not a policy
The Strands Agents SDK takes a different shape. Its README lists a set of control points. I group them by the job they do:
!A three-step chain: set limits such as turn limits and budgets, add hooks to log each step, then use steering to recover.
- Observe. Hooks let you intercept any step to log it, validate it or redirect it.
- Limit. Guardrails catch mistakes before they run. Lifecycle controls include turn limits, token budgets, cancellation and stop reasons.
- Recover. Steering handlers let agents correct themselves instead of failing silently.
My opinion on order: set the limits first, then add hooks so you can see what the limits catch.
A single create_harness() call in Python, or createHarness() in TypeScript, returns an optimized agent, in the README's words. The SDK is Apache-2.0 licensed.
These are mechanisms. My reading of the README excerpt is that it states no default policy, so what to block, what budget to set and what to log would fall to whoever deploys it; check the repository before relying on that.
The README excerpt I used shows no hook example, so for working code, go to the repository.
5. What the survey says is still open
The harness-engineering survey gives an outside view. It says it covers 110+ papers and 23+ deployed systems, and identifies nine open challenges.
Security and sandboxing is challenge number one. The survey attributes 15–35% frontier-model container escape rates to a benchmark it calls SandboxEscapeBench, and I saw that figure only second-hand, so treat it as the survey's claim.
The survey's advice is blunt: treat sandboxing as a hard requirement and apply defense-in-depth. It also lists a harness-specific threat model, covering tool hijacking, context poisoning and agent impersonation, as future work.
For anyone planning multi-agent setups, one more line stands out. The survey says Byzantine fault tolerance remains unsolved for adversarial multi-agent settings. The term comes from the problem of getting generals who communicate only by messenger to agree on a plan when some may be traitors, per Lamport, Shostak and Pease.
6. What I would ask for before a pilot
This section is my opinion, not a sourced finding. If I were setting a baseline for coding agents in a company, I would ask for these before the first pilot:
!A six-item checklist: sandbox on by default, narrow mounts, hardened control plane, explicit budgets, logging hooks, human approval for high-stakes steps.
1. Sandbox on by default, with the run mode checked in tooling, so nobody discovers the host-level mode by accident.
2. Narrow mounts: one project per task, so one mistake stays small.
3. The control plane treated as its own system to harden, separate from the execution sandbox, since the sandbox does not cover it.
4. Budgets and turn limits set explicitly, so a runaway task stops.
5. Hooks that log every step, so you can see what happened.
6. Human approval for high-stakes steps, so a person owns the risky calls.
The last items line up with the survey's governance layer, which covers security constraints, human handoff points, HITL policy, audit trails and compliance enforcement. Separately, in its section on multi-agent fault tolerance, the survey recommends gating high-stakes coordination on human approval.
Each item is cheap to check before a pilot and expensive to retrofit. Ask each team to show, not describe, the run mode, the mount, the budget and the log. If any of the four is "we will look into it", the pilot is not ready. None of this needs a particular product; it needs someone to own the answers, hence the question below.
Draw the boundary before the agent runs. Which boundary do you enforce today for coding agents, and who owns it on your team?
2 comments
OpenHands Self-Hosted Runtime Review: Docker Isolation, Agent Cost, and the Production Reality
Why We Evaluated OpenHands
We brought OpenHands into our evaluation queue because buying another coding-agent seat does not solve the infrastructure question facing engineering teams. We wanted to know whether we could operate the execution layer ourselves, connect it to a model provider of our choice, and retain control over source code, runtime images, network policy, and usage accounting.
We often see OpenHands described as an open-source Devin alternative, but that description is too shallow for an architecture review. The interesting component is not the chat interface. It is the runtime responsible for turning model-generated actions into shell commands, file mutations, Python execution, and observations.
We traced that path through the OpenHands runtime architecture. The backend launches a purpose-built runtime container, starts an action execution server inside it, sends actions over an HTTP interface, and receives structured observations. The model remains outside that execution boundary. The container is where potentially destructive commands run.
That separation is useful. It gives us an explicit control point for image policy, workspace mounts, resource limits, process cleanup, and audit logging. It also means OpenHands is not merely an application that happens to use Docker. Docker is part of its security and scheduling model.
The image workflow is more sophisticated than a naïve build-per-task design. We found three levels of runtime tags:
- A versioned tag tied to the OpenHands version and base image.
- A lock tag derived from the base image and dependency files.
- A source tag incorporating the OpenHands source hash.
When an exact source image exists, the runtime can reuse it. When only the dependency-compatible layer exists, it can copy changed source without repeating the slow dependency installation path. That design should reduce repeated build work, although we would still pin immutable image digests rather than rely on a moving tag in production.
The project’s 90,000-plus-star GitHub repository made it impossible to dismiss as a weekend prototype. We treated that popularity as an adoption signal, not evidence of secure isolation or predictable operating cost.
Our central questions were therefore practical:
1. Can we reproduce the documented local deployment without hidden services?
2. Does the sandbox protect the host under a realistic threat model?
3. Can we measure model spend per conversation?
4. What happens when multiple sandboxes need ports, storage, and cleanup?
5. At what team size does self-hosting become cheaper than buying hosted seats?
We also wanted to separate what we could verify from what required a functioning Docker daemon and paid model credentials. That distinction became important almost immediately.
Hands-On Walkthrough: Setup, Execution & Output
We started with the supported local prerequisites: Python 3.12, uv, and a Docker installation with access to the Docker socket. We treated the 4 GB host-memory recommendation as a setup baseline, not a measured per-sandbox footprint. It is a minimum host recommendation, not a capacity-planning number.
The shortest launcher path was:
uv tool install openhands --python 3.12
openhands serve --mount-cwd
The --mount-cwd option mounts the current repository into the workspace. That convenience has a direct security consequence: an agent with write access can modify anything exposed through that mount.
For a more controlled reproduction, we prepared a command with explicit application and agent-server versions rather than using latest:
docker run -it --rm --pull=always \
-e AGENT_SERVER_IMAGE_REPOSITORY=ghcr.io/openhands/agent-server \
-e AGENT_SERVER_IMAGE_TAG=1.26.0-python \
-e LOG_ALL_EVENTS=true \
-v /var/run/docker.sock:/var/run/docker.sock \
-v "$HOME/.openhands:/.openhands" \
-p 3000:3000 \
--add-host host.docker.internal:host-gateway \
--name openhands-app \
docker.openhands.dev/openhands/openhands:1.8
This command exposes the web application on http://localhost:3000. More importantly, it mounts /var/run/docker.sock into the application container so OpenHands can create and manage sandbox containers.
We hit a hard environmental stop before a useful runtime benchmark: our available runner did not expose a Docker daemon. We could inspect the launch configuration, but we did not run a deployment-contract test or measure container startup latency, memory consumption, concurrent sandbox throughput, or cleanup behavior on that runner. We did not replace those missing measurements with invented numbers.
For a host with Docker available, the SDK-level sandbox path is straightforward:
import platform
from openhands.workspace import DockerWorkspace
def detect_platform() -> str:
machine = platform.machine().lower()
if "arm" in machine or "aarch64" in machine:
return "linux/arm64"
return "linux/amd64"
with DockerWorkspace(
server_image="ghcr.io/openhands/agent-server:1.26.0-python",
host_port=8010,
platform=detect_platform(),
) as workspace:
result = workspace.execute_command(
"printf 'sandbox-ready\\n' && pwd && python --version"
)
print(
{
"command": result.command,
"exit_code": result.exit_code,
"stdout": result.stdout,
"stderr": result.stderr,
}
)
The expected response shape below is simulated to make the reproduction contract explicit. It is not presented as a measurement from our Docker-unavailable runner:
{
"command": "printf 'sandbox-ready\\n' && pwd && python --version",
"exit_code": 0,
"stdout": "sandbox-ready\n/workspace\nPython 3.12.x\n",
"stderr": ""
}
DockerWorkspace is the right choice when we can use a prebuilt agent-server image. DockerDevWorkspace builds from a selected base image and is appropriate when we need additional compilers, system packages, or internal tooling. We would not use the development workspace on a latency-sensitive request path because image construction adds an avoidable startup stage.
A real agent conversation introduces another dependency: an LLM credential. For our API-backed conversation design, we use LLM_API_KEY and LiteLLM-style model identifiers such as anthropic/... or openai/..., with accumulated cost reporting available through:
cost = (
conversation.conversation_stats
.get_combined_metrics()
.accumulated_cost
)
We used that metric as the required accounting surface in our production design. We would record it by conversation, repository, model, tenant, and completion state. We would not treat it as a complete bill until reconciling it against provider invoices, retries, failed requests, cache behavior, and any model calls made outside the conversation object.
A local model endpoint can use a placeholder API-key value when its server does not authenticate requests. That removes the paid-provider key dependency, but it does not remove inference cost. GPU depreciation, idle capacity, model serving, and weaker task completion can easily cost more than API tokens.
Deployment Limitations and Operational Risks
The first failure was simple and decisive: without Docker, there was no meaningful OpenHands sandbox benchmark. A mock Docker client could verify Python control flow, but it could not verify namespaces, mounts, port binding, image pulls, process termination, or actual container cleanup. We rejected a mock-only result as production evidence.
The second issue was the Docker socket. The architecture isolates agent commands in a child container, but the parent OpenHands application requires control of the host Docker daemon in the direct deployment. On a conventional Linux host, access to that socket is effectively privileged host control. If the application container is compromised, the sandbox boundary does not protect the Docker host from the controller.
Our workaround is architectural rather than cosmetic:
- Run OpenHands workers on dedicated nodes or disposable virtual machines.
- Do not colocate them with databases, CI secrets, or customer-facing services.
- Restrict outbound network access from both controller and sandbox.
- Mount only the repository needed for the task.
- Prefer read-only mounts plus an explicit writable worktree where possible.
- Apply CPU, memory, process, and execution-time limits.
- Treat the Docker host as replaceable after hostile workloads.
OpenHands supports bind mounts, named volumes, and overlay copy-on-write behavior. Overlay mode requires a bind mount marked with :overlay and a writable SANDBOX_VOLUME_OVERLAYS host directory. If we do not configure that directory, OpenHands skips the overlay path. We would fail startup on that condition rather than silently continue with different write semantics.
When planning the reverse-proxy deployment, we identified another design problem. Each sandbox normally receives random host ports, while generated browser URLs default to a localhost pattern. Setting OH_WEB_URL does not rewrite those sandbox URLs; it affects the allowed origin rather than the hostname and port used to reach the container.
We found two imperfect options.
The first is to set OH_SANDBOX_CONTAINER_URL_PATTERN to a public hostname containing {port}. That retains a random port for each sandbox, forcing the edge layer to forward a dynamic range.
The second is host networking:
export AGENT_SERVER_USE_HOST_NETWORK=true
This exposes fixed ports, including agent-server port 8000 and editor-related port 8001. The trade-off is severe: every sandbox tries to bind the same ports, so only one sandbox can run on that host without collision. We would not use host networking for a multi-tenant production worker.
We also found that editor capabilities vary by image. We should not assume portable SDK images include VS Code, VNC, a fixed editor port, or /api/vscode/url. The extra_ports=True pattern belongs to selected legacy editor-capable images, where host_port + 1 and host_port + 2 may expose VS Code and VNC. We would feature-detect those services instead of deriving URLs unconditionally.
Finally, moving tags are a reproducibility risk. ghcr.io/openhands/agent-server:latest-python is convenient for exploration, but production rollouts need pinned versions or, preferably, image digests. We would test the application image, agent-server image, base image, SDK, and model configuration as one release unit.
What This Evaluation Could Not Verify
We could not measure OpenHands startup latency, throughput, or memory use because our runner lacked Docker. The available setup material also does not provide a reproducible production benchmark.
Scaling and Cost vs. Alternatives
What we can evaluate is the cost and scaling shape.
| Option | Execution ownership | Isolation model | Model cost | Operational burden | Best fit |
|---|---|---|---|---|---|
| OpenHands self-hosted | Our team | Docker sandbox managed by OpenHands | Paid API or local inference | High | Teams needing model and runtime control |
| E2B-style managed sandbox | Provider | Remote managed sandbox | Model plus sandbox usage | Medium | Fast deployment without worker operations |
| Custom Docker runner | Our team | Whatever we build and audit | Paid API or local inference | Very high | Specialized policies or narrow workflows |
| Hosted coding-agent seat | Vendor | Vendor-managed | Usually bundled or plan-dependent | Low | Teams prioritizing speed over control |
| IDE agent | Developer machine | Local permissions or editor controls | Subscription or API | Low to medium | Interactive individual development |
OpenHands removes per-seat runtime licensing from the equation, but it does not make agent execution free. Our monthly model is:
OpenHands monthly cost =
worker infrastructure
+ engineering maintenance
+ observability and storage
+ LLM or local inference
+ security and incident overhead
The corresponding hosted calculation is:
Hosted monthly cost =
number of seats × effective monthly seat price
+ overages
+ integration overhead
For an illustrative break-even model, assume:
- Worker infrastructure and logs: $600 per month.
- Runtime maintenance: 12 engineering hours per month.
- Loaded engineering rate: $150 per hour.
- Self-hosted model usage: $120 per active user per month.
- Hosted alternative: $500 per active user per month.
That produces $2,400 in fixed monthly self-hosting cost and a $380 per-user advantage:
Break-even users = $2,400 / ($500 - $120) = 6.32
Under those assumptions, self-hosting becomes cheaper at approximately seven active users. This is a planning example, not a quoted Devin price or an observed OpenHands bill. If maintenance rises to 30 hours, or if agents burn tokens through repeated tool loops, the break-even point moves sharply upward.
Agent-loop cost is the largest variable. A task is not one model call. The agent may inspect files, invoke tools, ingest observations, revise its plan, run tests, encounter errors, and ask the model again. Long command output also returns to context and can increase subsequent request cost.
We would enforce four controls before opening access:
1. Maximum model spend per conversation.
2. Maximum loop iterations and wall-clock duration.
3. Output truncation and artifact storage outside the prompt.
4. Per-team and per-repository budgets with hard rejection.
For teams building this into a broader internal platform, our [AI infrastructure services](/services) cover the worker isolation, observability, and cost-control layer that the default local launcher does not provide. We also maintain a broader set of implementation references in our [tools collection](/tools).
Our Final Verdict: When to Deploy, When to Skip
OpenHands has a credible runtime design. We like the separation between the agent backend and the in-container action server, the prebuilt DockerWorkspace path, the customizable development workspace, and the exposed conversation-cost metric.
We do not consider the default single-host Docker deployment a complete production security boundary. The Docker socket, repository mounts, dynamic ports, image compatibility, outbound network policy, and cleanup behavior all become our responsibility the moment we self-host.
Deploy this if:
- We need control over model providers, runtime images, and source-code location.
- We can dedicate isolated worker hosts to untrusted agent execution.
- We have enough active usage to absorb recurring platform maintenance.
- We can pin and test the full application and agent-server image set.
- We will enforce token, iteration, time, memory, and network limits.
- We accept that self-hosting transfers operational risk rather than eliminating cost.
Hold off or avoid it if:
- We need a turnkey service with contractual support and predictable seats.
- We plan to expose the host Docker socket on shared infrastructure.
- We cannot operate dynamic sandbox ports behind our edge architecture.
- We expect bundled VS Code or VNC capabilities in every SDK image.
- We lack per-conversation model budgets and provider-bill reconciliation.
- We need audited multi-tenant isolation stronger than ordinary Docker containers.
- We need benchmark-backed capacity numbers before running our own Docker tests.
Our recommendation is a guarded deploy for a controlled pilot, not an immediate shared production rollout. Start with a dedicated worker, one pinned image set, outbound-deny networking, disposable repository copies, and a small group of engineers. Measure cold and warm startup separately, record accumulated model cost, test cleanup after forced termination, and attempt deliberate workspace escapes before expanding.
OpenHands can reduce vendor lock-in and give us a useful coding-agent control plane. It does not remove the hard parts. It makes those parts visible—and assigns them to our infrastructure team. Teams that want help validating that trade-off against their own workload can [contact us](/contact) with their concurrency, repository, compliance, and model assumptions.
Mentions
🤖 Sadece sohbet etmeyen; kendi başına kod yazan, terminal kullanan ve proje geliştiren o #Shorts 🤖 Sadece sohbet etmeyen; kendi başına kod yazan, terminal kullanan ve proje geliştiren otonom yapay zeka mühendisi: OpenHands (61.4K+ Yıldız)! 📌 Yorum şartı yok, doğrudan bilgiler: 🔹 Proje Adı: OpenHands (All-Hands-AI/OpenHands) 🔗 Doğrudan GitHub Linki: All-Hands-AI/OpenHands ⚖️ Lisans: MIT (%100 Açık Kaynak) Kaydetmeyi ve @ugurakkaya_djitalpro hesabımı takip etmeyi unutma! 🚀 #github #openhands #yapayzeka #yazılım #kodlama #aiagent #teknoloji #ugurakkaya [en-US] 🤖 The AI that doesn't just chat: it codes, uses terminals, and builds projects on its own #Shorts 🤖 An autonomous AI engineer that doesn't just chat; it writes code, uses terminals, and develops projects on its own: OpenHands (61.4K+ Stars)! 📌 No comment required, direct information: 🔹 Project Name: OpenHands (All-Hands-AI/OpenHands) 🔗 Direct GitHub Link: All-Hands-AI/OpenHands ⚖️ License: MIT (100% Open Source) Don't forget to save and follow my account @ugurakkaya_djitalpro! 🚀 #github #openhands #artificialintelligence #software #coding #aiagent #technology #ugurakkaya
The harness is your attack surface You install a coding agent and point it at a repo. Which files can it read, and what stops it? For some setups the honest answer is "everything your user can." The harness gives you control points, but the boundary and the policy are yours. A note on method. I read the current documentation of two open-source agent projects, OpenHands and the Strands Agents Python SDK, plus a harness-engineering survey. Every factual statement below carries a marker like `` that maps to one of those three sources. Where I give an opinion or a reading of a README, I say so. I do not call any project "safer" than another, because I found no comparison made on equal terms. 1. The default question: where does the agent run? Start with the most uncomfortable line in the OpenHands README. It documents that running the agent without a sandbox gives it full access to the host filesystem. What matters is where the code runs in that mode. Without a sandbox, the OpenHands agent-server runs directly on the host machine rather than in a container. Fine for a throwaway laptop. A very different choice on a developer machine that holds SSH keys, cloud credentials and ten other repositories. I'd settle the run mode before the first task, not after the first surprise. A cheap habit: in your own wrapper, print the run mode at the start of every task and stop if it is not the one you expect. 2. Docker helps, but only as far as you mount OpenHands can run each new conversation in its own Docker container, with its own Agent Server and tools. That narrows what one conversation can break. !Two columns: a narrow mount exposes one project for one task; a broad PROJECTS_PATH exposes every repository under it. The catch is the mount. In the Docker sandbox option, the agent can access any project under PROJECTS_PATH, a host directory the user chooses. The container is only as narrow as the directory you hand it. Here is the difference, as an illustration of the idea rather than a copy of any official setup: # Narrow: one project only. The narrow path is the whole point: one task, one repo. export PROJECTS_PATH="$HOME/work/payments-service" # Broad: every project you have ever cloned (commented out so this block is safe to paste) # export PROJECTS_PATH="$HOME/work" With the second value, a task about one service can reach every other repository under that directory, because the documented behaviour is access to any project under the path. 3. The control plane sits outside the sandbox Isolating where the agent executes does not harden the service around it. Per-conversation Docker isolation in OpenHands does not sandbox the whole Canvas or automation service. Treat that service as its own system to harden. For internet-facing installs, OpenHands points readers to its self-hosting guide for security hardening, and the project is MIT licensed. I did not open that guide, so read it yourself before exposing an install. 4. What Strands gives you: control points, not a policy The Strands Agents SDK takes a different shape. Its README lists a set of control points. I group them by the job they do: !A three-step chain: set limits such as turn limits and budgets, add hooks to log each step, then use steering to recover. - Observe. Hooks let you intercept any step to log it, validate it or redirect it. - Limit. Guardrails catch mistakes before they run. Lifecycle controls include turn limits, token budgets, cancellation and stop reasons. - Recover. Steering handlers let agents correct themselves instead of failing silently. My opinion on order: set the limits first, then add hooks so you can see what the limits catch. A single create_harness() call in Python, or createHarness() in TypeScript, returns an optimized agent, in the README's words. The SDK is Apache-2.0 licensed. These are mechanisms. My reading of the README excerpt is that it states no default policy, so what to block, what budget to set and what to log would fall to whoever deploys it; check the repository before relying on that. The README excerpt I used shows no hook example, so for working code, go to the repository. 5. What the survey says is still open The harness-engineering survey gives an outside view. It says it covers 110+ papers and 23+ deployed systems, and identifies nine open challenges. Security and sandboxing is challenge number one. The survey attributes 15–35% frontier-model container escape rates to a benchmark it calls SandboxEscapeBench, and I saw that figure only second-hand, so treat it as the survey's claim. The survey's advice is blunt: treat sandboxing as a hard requirement and apply defense-in-depth. It also lists a harness-specific threat model, covering tool hijacking, context poisoning and agent impersonation, as future work. For anyone planning multi-agent setups, one more line stands out. The survey says Byzantine fault tolerance remains unsolved for adversarial multi-agent settings. The term comes from the problem of getting generals who communicate only by messenger to agree on a plan when some may be traitors, per Lamport, Shostak and Pease. 6. What I would ask for before a pilot This section is my opinion, not a sourced finding. If I were setting a baseline for coding agents in a company, I would ask for these before the first pilot: !A six-item checklist: sandbox on by default, narrow mounts, hardened control plane, explicit budgets, logging hooks, human approval for high-stakes steps. 1. Sandbox on by default, with the run mode checked in tooling, so nobody discovers the host-level mode by accident. 2. Narrow mounts: one project per task, so one mistake stays small. 3. The control plane treated as its own system to harden, separate from the execution sandbox, since the sandbox does not cover it. 4. Budgets and turn limits set explicitly, so a runaway task stops. 5. Hooks that log every step, so you can see what happened. 6. Human approval for high-stakes steps, so a person owns the risky calls. The last items line up with the survey's governance layer, which covers security constraints, human handoff points, HITL policy, audit trails and compliance enforcement. Separately, in its section on multi-agent fault tolerance, the survey recommends gating high-stakes coordination on human approval. Each item is cheap to check before a pilot and expensive to retrofit. Ask each team to show, not describe, the run mode, the mount, the budget and the log. If any of the four is "we will look into it", the pilot is not ready. None of this needs a particular product; it needs someone to own the answers, hence the question below. Draw the boundary before the agent runs. Which boundary do you enforce today for coding agents, and who owns it on your team?
值得注意的是一条很直白的安全提示：不装沙箱直接跑，agent 会拿到你整个文件系统的权限。 「常驻的 AI 工程团队」听着诱人，但团队意味着边界、权限和审计，而不是把 terminal 交出去就不管了。 All-Hands-AI/OpenHands
OpenHands Self-Hosted Runtime Review: Docker Isolation, Agent Cost, and the Production Reality Why We Evaluated OpenHands We brought OpenHands into our evaluation queue because buying another coding-agent seat does not solve the infrastructure question facing engineering teams. We wanted to know whether we could operate the execution layer ourselves, connect it to a model provider of our choice, and retain control over source code, runtime images, network policy, and usage accounting. We often see OpenHands described as an open-source Devin alternative, but that description is too shallow for an architecture review. The interesting component is not the chat interface. It is the runtime responsible for turning model-generated actions into shell commands, file mutations, Python execution, and observations. We traced that path through the OpenHands runtime architecture. The backend launches a purpose-built runtime container, starts an action execution server inside it, sends actions over an HTTP interface, and receives structured observations. The model remains outside that execution boundary. The container is where potentially destructive commands run. That separation is useful. It gives us an explicit control point for image policy, workspace mounts, resource limits, process cleanup, and audit logging. It also means OpenHands is not merely an application that happens to use Docker. Docker is part of its security and scheduling model. The image workflow is more sophisticated than a naïve build-per-task design. We found three levels of runtime tags: - A versioned tag tied to the OpenHands version and base image. - A lock tag derived from the base image and dependency files. - A source tag incorporating the OpenHands source hash. When an exact source image exists, the runtime can reuse it. When only the dependency-compatible layer exists, it can copy changed source without repeating the slow dependency installation path. That design should reduce repeated build work, although we would still pin immutable image digests rather than rely on a moving tag in production. The project’s 90,000-plus-star GitHub repository made it impossible to dismiss as a weekend prototype. We treated that popularity as an adoption signal, not evidence of secure isolation or predictable operating cost. Our central questions were therefore practical: 1. Can we reproduce the documented local deployment without hidden services? 2. Does the sandbox protect the host under a realistic threat model? 3. Can we measure model spend per conversation? 4. What happens when multiple sandboxes need ports, storage, and cleanup? 5. At what team size does self-hosting become cheaper than buying hosted seats? We also wanted to separate what we could verify from what required a functioning Docker daemon and paid model credentials. That distinction became important almost immediately. Hands-On Walkthrough: Setup, Execution & Output We started with the supported local prerequisites: Python 3.12, uv, and a Docker installation with access to the Docker socket. We treated the 4 GB host-memory recommendation as a setup baseline, not a measured per-sandbox footprint. It is a minimum host recommendation, not a capacity-planning number. The shortest launcher path was: uv tool install openhands --python 3.12 openhands serve --mount-cwd The --mount-cwd option mounts the current repository into the workspace. That convenience has a direct security consequence: an agent with write access can modify anything exposed through that mount. For a more controlled reproduction, we prepared a command with explicit application and agent-server versions rather than using latest: docker run -it --rm --pull=always \ -e AGENT_SERVER_IMAGE_REPOSITORY=ghcr.io/openhands/agent-server \ -e AGENT_SERVER_IMAGE_TAG=1.26.0-python \ -e LOG_ALL_EVENTS=true \ -v /var/run/docker.sock:/var/run/docker.sock \ -v "$HOME/.openhands:/.openhands" \ -p 3000:3000 \ --add-host host.docker.internal:host-gateway \ --name openhands-app \ docker.openhands.dev/openhands/openhands:1.8 This command exposes the web application on http://localhost:3000. More importantly, it mounts /var/run/docker.sock into the application container so OpenHands can create and manage sandbox containers. We hit a hard environmental stop before a useful runtime benchmark: our available runner did not expose a Docker daemon. We could inspect the launch configuration, but we did not run a deployment-contract test or measure container startup latency, memory consumption, concurrent sandbox throughput, or cleanup behavior on that runner. We did not replace those missing measurements with invented numbers. For a host with Docker available, the SDK-level sandbox path is straightforward: import platform from openhands.workspace import DockerWorkspace def detect_platform() -> str: machine = platform.machine().lower() if "arm" in machine or "aarch64" in machine: return "linux/arm64" return "linux/amd64" with DockerWorkspace( server_image="ghcr.io/openhands/agent-server:1.26.0-python", host_port=8010, platform=detect_platform(), ) as workspace: result = workspace.execute_command( "printf 'sandbox-ready\\n' && pwd && python --version" ) print( { "command": result.command, "exit_code": result.exit_code, "stdout": result.stdout, "stderr": result.stderr, } ) The expected response shape below is simulated to make the reproduction contract explicit. It is not presented as a measurement from our Docker-unavailable runner: { "command": "printf 'sandbox-ready\\n' && pwd && python --version", "exit_code": 0, "stdout": "sandbox-ready\n/workspace\nPython 3.12.x\n", "stderr": "" } DockerWorkspace is the right choice when we can use a prebuilt agent-server image. DockerDevWorkspace builds from a selected base image and is appropriate when we need additional compilers, system packages, or internal tooling. We would not use the development workspace on a latency-sensitive request path because image construction adds an avoidable startup stage. A real agent conversation introduces another dependency: an LLM credential. For our API-backed conversation design, we use LLM_API_KEY and LiteLLM-style model identifiers such as anthropic/... or openai/..., with accumulated cost reporting available through: cost = ( conversation.conversation_stats .get_combined_metrics() .accumulated_cost ) We used that metric as the required accounting surface in our production design. We would record it by conversation, repository, model, tenant, and completion state. We would not treat it as a complete bill until reconciling it against provider invoices, retries, failed requests, cache behavior, and any model calls made outside the conversation object. A local model endpoint can use a placeholder API-key value when its server does not authenticate requests. That removes the paid-provider key dependency, but it does not remove inference cost. GPU depreciation, idle capacity, model serving, and weaker task completion can easily cost more than API tokens. Deployment Limitations and Operational Risks The first failure was simple and decisive: without Docker, there was no meaningful OpenHands sandbox benchmark. A mock Docker client could verify Python control flow, but it could not verify namespaces, mounts, port binding, image pulls, process termination, or actual container cleanup. We rejected a mock-only result as production evidence. The second issue was the Docker socket. The architecture isolates agent commands in a child container, but the parent OpenHands application requires control of the host Docker daemon in the direct deployment. On a conventional Linux host, access to that socket is effectively privileged host control. If the application container is compromised, the sandbox boundary does not protect the Docker host from the controller. Our workaround is architectural rather than cosmetic: - Run OpenHands workers on dedicated nodes or disposable virtual machines. - Do not colocate them with databases, CI secrets, or customer-facing services. - Restrict outbound network access from both controller and sandbox. - Mount only the repository needed for the task. - Prefer read-only mounts plus an explicit writable worktree where possible. - Apply CPU, memory, process, and execution-time limits. - Treat the Docker host as replaceable after hostile workloads. OpenHands supports bind mounts, named volumes, and overlay copy-on-write behavior. Overlay mode requires a bind mount marked with :overlay and a writable SANDBOX_VOLUME_OVERLAYS host directory. If we do not configure that directory, OpenHands skips the overlay path. We would fail startup on that condition rather than silently continue with different write semantics. When planning the reverse-proxy deployment, we identified another design problem. Each sandbox normally receives random host ports, while generated browser URLs default to a localhost pattern. Setting OH_WEB_URL does not rewrite those sandbox URLs; it affects the allowed origin rather than the hostname and port used to reach the container. We found two imperfect options. The first is to set OH_SANDBOX_CONTAINER_URL_PATTERN to a public hostname containing {port}. That retains a random port for each sandbox, forcing the edge layer to forward a dynamic range. The second is host networking: export AGENT_SERVER_USE_HOST_NETWORK=true This exposes fixed ports, including agent-server port 8000 and editor-related port 8001. The trade-off is severe: every sandbox tries to bind the same ports, so only one sandbox can run on that host without collision. We would not use host networking for a multi-tenant production worker. We also found that editor capabilities vary by image. We should not assume portable SDK images include VS Code, VNC, a fixed editor port, or /api/vscode/url. The extra_ports=True pattern belongs to selected legacy editor-capable images, where host_port + 1 and host_port + 2 may expose VS Code and VNC. We would feature-detect those services instead of deriving URLs unconditionally. Finally, moving tags are a reproducibility risk. ghcr.io/openhands/agent-server:latest-python is convenient for exploration, but production rollouts need pinned versions or, preferably, image digests. We would test the application image, agent-server image, base image, SDK, and model configuration as one release unit. What This Evaluation Could Not Verify We could not measure OpenHands startup latency, throughput, or memory use because our runner lacked Docker. The available setup material also does not provide a reproducible production benchmark. Scaling and Cost vs. Alternatives What we can evaluate is the cost and scaling shape. | Option | Execution ownership | Isolation model | Model cost | Operational burden | Best fit | |---|---|---|---|---|---| | OpenHands self-hosted | Our team | Docker sandbox managed by OpenHands | Paid API or local inference | High | Teams needing model and runtime control | | E2B-style managed sandbox | Provider | Remote managed sandbox | Model plus sandbox usage | Medium | Fast deployment without worker operations | | Custom Docker runner | Our team | Whatever we build and audit | Paid API or local inference | Very high | Specialized policies or narrow workflows | | Hosted coding-agent seat | Vendor | Vendor-managed | Usually bundled or plan-dependent | Low | Teams prioritizing speed over control | | IDE agent | Developer machine | Local permissions or editor controls | Subscription or API | Low to medium | Interactive individual development | OpenHands removes per-seat runtime licensing from the equation, but it does not make agent execution free. Our monthly model is: OpenHands monthly cost = worker infrastructure + engineering maintenance + observability and storage + LLM or local inference + security and incident overhead The corresponding hosted calculation is: Hosted monthly cost = number of seats × effective monthly seat price + overages + integration overhead For an illustrative break-even model, assume: - Worker infrastructure and logs: $600 per month. - Runtime maintenance: 12 engineering hours per month. - Loaded engineering rate: $150 per hour. - Self-hosted model usage: $120 per active user per month. - Hosted alternative: $500 per active user per month. That produces $2,400 in fixed monthly self-hosting cost and a $380 per-user advantage: Break-even users = $2,400 / ($500 - $120) = 6.32 Under those assumptions, self-hosting becomes cheaper at approximately seven active users. This is a planning example, not a quoted Devin price or an observed OpenHands bill. If maintenance rises to 30 hours, or if agents burn tokens through repeated tool loops, the break-even point moves sharply upward. Agent-loop cost is the largest variable. A task is not one model call. The agent may inspect files, invoke tools, ingest observations, revise its plan, run tests, encounter errors, and ask the model again. Long command output also returns to context and can increase subsequent request cost. We would enforce four controls before opening access: 1. Maximum model spend per conversation. 2. Maximum loop iterations and wall-clock duration. 3. Output truncation and artifact storage outside the prompt. 4. Per-team and per-repository budgets with hard rejection. For teams building this into a broader internal platform, our [AI infrastructure services](/services) cover the worker isolation, observability, and cost-control layer that the default local launcher does not provide. We also maintain a broader set of implementation references in our [tools collection](/tools). Our Final Verdict: When to Deploy, When to Skip OpenHands has a credible runtime design. We like the separation between the agent backend and the in-container action server, the prebuilt DockerWorkspace path, the customizable development workspace, and the exposed conversation-cost metric. We do not consider the default single-host Docker deployment a complete production security boundary. The Docker socket, repository mounts, dynamic ports, image compatibility, outbound network policy, and cleanup behavior all become our responsibility the moment we self-host. Deploy this if: - We need control over model providers, runtime images, and source-code location. - We can dedicate isolated worker hosts to untrusted agent execution. - We have enough active usage to absorb recurring platform maintenance. - We can pin and test the full application and agent-server image set. - We will enforce token, iteration, time, memory, and network limits. - We accept that self-hosting transfers operational risk rather than eliminating cost. Hold off or avoid it if: - We need a turnkey service with contractual support and predictable seats. - We plan to expose the host Docker socket on shared infrastructure. - We cannot operate dynamic sandbox ports behind our edge architecture. - We expect bundled VS Code or VNC capabilities in every SDK image. - We lack per-conversation model budgets and provider-bill reconciliation. - We need audited multi-tenant isolation stronger than ordinary Docker containers. - We need benchmark-backed capacity numbers before running our own Docker tests. Our recommendation is a guarded deploy for a controlled pilot, not an immediate shared production rollout. Start with a dedicated worker, one pinned image set, outbound-deny networking, disposable repository copies, and a small group of engineers. Measure cold and warm startup separately, record accumulated model cost, test cleanup after forced termination, and attempt deliberate workspace escapes before expanding. OpenHands can reduce vendor lock-in and give us a useful coding-agent control plane. It does not remove the hard parts. It makes those parts visible—and assigns them to our infrastructure team. Teams that want help validating that trade-off against their own workload can [contact us](/contact) with their concurrency, repository, compliance, and model assumptions.
KỸ SƯ PHẦN MỀM AI TỰ TRỊ THAY THẾ DEVIN: OPENHANDS TỰ DUYỆT WEB, VIẾT CODE VÀ ĐÓNG GÓP PULL REQUEST THỰC TẾ Khi dự án Devin thương mại gây xôn xao dư luận với lời hứa hẹn về một AI Software Engineer có thể tự động nhận issue và giải quyết bug từ đầu đến cuối, cộng đồng mã nguồn mở đã lập tức đáp lời bằng dự án OpenHands (tiền thân là OpenDevin). OpenHands là nền tảng mã nguồn mở phát triển kỹ sư phần mềm AI tự trị, có khả năng tương tác với terminal, trình duyệt web và trình biên tập mã nguồn trong môi trường sandbox an toàn để thực hiện các nhiệm vụ phát triển phần mềm phức tạp. Tech Stack: Xây dựng trên nền tảng Python, Docker và FastAPI. Sử dụng kiến trúc Event-Stream phân tán cho phép agent lập kế hoạch, kiểm tra lỗi và tương tác với môi trường runtime được cách ly 100%. Tại sao nên dùng? - Khả năng thực thi tác vụ đầu-cuối (Full-lifecycle Development): Agent có thể đọc issue trên GitHub, duyệt web tìm tài liệu thư viện, viết code sửa lỗi, chạy test suite và tự động mở Pull Request hoàn chỉnh. - Môi trường Sandbox an toàn tuyệt đối: Toàn bộ thao tác code và dòng lệnh shell đều chạy bên trong một container Docker riêng biệt, đảm bảo agent không bao giờ gây ảnh hưởng đến máy chủ lưu trữ. - Độc lập mô hình hoàn toàn: Hoạt động xuất sắc với Claude 3.5 Sonnet, GPT-4o hoặc các mô hình mã nguồn mở hàng đầu như DeepSeek-Coder và Qwen 2.5 Coder. Dự án đã cán mốc hơn 44.000 stars trên GitHub và đang dẫn đầu cuộc đua dân chủ hóa công nghệ AI Software Engineer toàn cầu. Link tham khảo: All-Hands-AI/OpenHands #OpenSourceVN #VNOSS #opensourcevn #vnoss #DevCommunity #OpenHands #OpenDevin #AIAgent #SoftwareEngineer #Python #Docker