sentient-agi

EvoSkill v2 improves AI agents by rewriting skills, not model weights. It hit 40.4% on SpreadsheetBench v2, +7.7 points over SkillOpt, per Sentient Labs' release. macOS, Python 3.12+, uv, Daytona and Claude access required; Linux/Docker planned. github.com/sentient-agi/EvoSkill/releases/tag/v2.…

Paper: arxiv.org Code: sentient-agi/EvoSkill Strong work from Dastin Huang, Baran Nama, and Salaheddin Alzubi.

Test EvoSkill v2 on your own benchmarks: • github.com/sentient-agi/EvoSkill/releases/tag/v2.… (v2 branch) Read the full technical report: • alphaxiv.org • arxiv.org Work by Dastin Huang, Baran Nama and @salahalzubi401 at Sentient Labs.

Sentient Labs Introduces EvoSkill v2: A Three-Agent System That Rewrites Skills From Training Attempts Sentient Labs has released EvoSkill v2, an open-source (Apache-2.0) system that improves AI agents by rewriting their task instructions while keeping the underlying model and runtime unchanged. The current implementation requires a macOS host. Version 2.0.0 was released on October 26, 2026. The team's GitHub release notes state: "It improves an agent by rewriting its skills and leaves the model and the harness as they are." How the three agents work All three roles run as Claude Code sessions: - Benchmark adaptor: takes your benchmark description, data, split, and grader, then prepares a fixed task list and training material. - Solver: attempts each task three times with the available skills, with each attempt running in its own container. - Evolver: reviews training attempts and scores, investigates failures using a bash tool, and writes the next set of skills. A skill is a folder containing a SKILL.md instruction file and optional scripts. The solver loads the instructions when their description matches its task. What changed from v1 V2 combines v1's separate failure-analysis and skill-writing agents into one persistent evolver. Its notes, scripts, and earlier work remain available across rounds. It also automates benchmark adaptation and accepts every round's skill set, allowing later revisions instead of rejecting changes based on noisy validation scores. The final round's skills are evaluated on held-out tasks. Reported benchmark results With detailed training feedback, Sentient reports these held-out pass rates: |Benchmark|No skills|SkillOpt|EvoSkill v2| |:-|:-|:-|:-| |SpreadsheetBench v2|26.0%|32.7%|40.4%| |τ³-bench banking|28.0%|46.2%|51.6%| That is 7.7 and 5.4 percentage points above SkillOpt, respectively. These comparisons used Sonnet 5.5 for every optimizer and the same solver configuration across methods within each benchmark. In a separate banking experiment, Haiku 5.5 powered both solver and evolver, raising the pass rate from 28.0% to 49.5%. These are self-reported results from the authors' experiments, not an independent evaluation. Optimization cost For banking with detailed feedback, the paper estimates $0.99 for EvoSkill v2's optimizer, versus $5.55 for SkillOpt and $6.98 for SkillBoost. These are Claude Code list-price estimates for optimization only; solver runs are excluded. Limits - The detailed-feedback comparison gives EvoSkill richer training-answer access than the baselines. With pass/fail feedback alone, its spreadsheet result was roughly on par with SkillOpt. - The evolver cannot see validation or test tasks, but grader quality still matters. An earlier version exploited cached spreadsheet values; the reported experiments use a corrected grader that recalculates formulas. - Current task support covers workbook editing and τ³-style conversations. Other task types need additional solver support. Try it V2 lives on the v2 branch; main remains v1. git clone -b v2 sentient-agi/EvoSkill cd EvoSkill uv sync --extra daytona Setup requires Python 3.12+, uv, Daytona, and Claude access; banking experiments also require OpenRouter. Linux hosts and a local Docker backend are not yet supported. Sources - GitHub: EvoSkill v2.0.0 release and setup - arXiv: EvoSkill v2 technical report

Sentient Labs Introduces EvoSkill v2: A Three-Agent System That Rewrites Skills From Training Attempts Sentient Labs has released EvoSkill v2, an open-source (Apache-2.0) system that improves AI agents by rewriting their task instructions while keeping the underlying model and runtime unchanged. The current implementation requires a macOS host. Version 2.0.0 was released on October 26, 2026. The team's GitHub release notes state: "It improves an agent by rewriting its skills and leaves the model and the harness as they are." How the three agents work All three roles run as Claude Code sessions: - Benchmark adaptor: takes your benchmark description, data, split, and grader, then prepares a fixed task list and training material. - Solver: attempts each task three times with the available skills, with each attempt running in its own container. - Evolver: reviews training attempts and scores, investigates failures using a bash tool, and writes the next set of skills. A skill is a folder containing a SKILL.md instruction file and optional scripts. The solver loads the instructions when their description matches its task. What changed from v1 V2 combines v1's separate failure-analysis and skill-writing agents into one persistent evolver. Its notes, scripts, and earlier work remain available across rounds. It also automates benchmark adaptation and accepts every round's skill set, allowing later revisions instead of rejecting changes based on noisy validation scores. The final round's skills are evaluated on held-out tasks. Reported benchmark results With detailed training feedback, Sentient reports these held-out pass rates: |Benchmark|No skills|SkillOpt|EvoSkill v2| |:-|:-|:-|:-| |SpreadsheetBench v2|26.0%|32.7%|40.4%| |τ³-bench banking|28.0%|46.2%|51.6%| That is 7.7 and 5.4 percentage points above SkillOpt, respectively. These comparisons used Sonnet 5.5 for every optimizer and the same solver configuration across methods within each benchmark. In a separate banking experiment, Haiku 5.5 powered both solver and evolver, raising the pass rate from 28.0% to 49.5%. These are self-reported results from the authors' experiments, not an independent evaluation. Optimization cost For banking with detailed feedback, the paper estimates $0.99 for EvoSkill v2's optimizer, versus $5.55 for SkillOpt and $6.98 for SkillBoost. These are Claude Code list-price estimates for optimization only; solver runs are excluded. Limits - The detailed-feedback comparison gives EvoSkill richer training-answer access than the baselines. With pass/fail feedback alone, its spreadsheet result was roughly on par with SkillOpt. - The evolver cannot see validation or test tasks, but grader quality still matters. An earlier version exploited cached spreadsheet values; the reported experiments use a corrected grader that recalculates formulas. - Current task support covers workbook editing and τ³-style conversations. Other task types need additional solver support. Try it V2 lives on the v2 branch; main remains v1. git clone -b v2 sentient-agi/EvoSkill cd EvoSkill uv sync --extra daytona Setup requires Python 3.12+, uv, Daytona, and Claude access; banking experiments also require OpenRouter. Linux hosts and a local Docker backend are not yet supported. Sources - GitHub: EvoSkill v2.0.0 release and setup - arXiv: EvoSkill v2 technical report

Test EvoSkill v2 on your own benchmarks: • github.com/sentient-agi/EvoSkill/releases/tag/v2.… (v2 branch) Read the full technical report: • alphaxiv.org • arxiv.org Work by Dastin Huang, Baran Nama and @salahalzubi401 at Sentient Labs.

3) If you read the scores carefully, the board uses one attack on small 3B–4B models. A hundo p robustness score can just mean the attack also broke the model so badly nobody would use it. github.com/sentient-agi/mlprints/blob/main/leader…