Your Test Run Floods the Agent's Context. 9 Flags That Cut the Output It Reads
- Book: LLM Observability Pocket Guide
- Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go
- My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
- Me: xgabriel.com | GitHub
---
Your agent runs the test suite. One test fails. Hundreds pass, and the runner tells the agent about every one of them: a header, progress for each file, the names of passing tests, a summary, and somewhere in there the stack trace it actually needs.
Test reporters were written for a person scrolling a terminal. A coding agent doesn't scroll. It reads what the command printed.
The useful part can get cut. A command that fails reaches Claude Code inline only up to roughly 10,000 characters. Past that, Claude gets "a head-and-tail excerpt of that size", with no file path to the rest. A red test run exits non-zero, so it counts as a failure, and it is the run where the middle matters most.
And whatever fits can stay. Claude Code "sends your full conversation with every request", so the names of passing tests can ride along on later turns until the conversation is compacted or cleared.
You can't change how the agent reads. You can change what the runner prints.
Measure what your agent reads first
Run the suite the way an agent does: piped, not in an interactive terminal, with stderr merged into stdout. Then count.
npx vitest run 2>&1 | wc -c
npx vitest run --reporter=minimal 2>&1 | wc -c
wc -c counts bytes. For plain ASCII output that is also characters, the unit Claude Code's limits use. I'm not printing numbers here, because they depend entirely on your suite: how many tests, how much they log, how deep the stack traces go. Run it on your project, before and after each flag below, and keep the flags that move your count.
A small loop compares several variants in one go:
for cmd in "pytest" "pytest -q" "pytest -q --tb=short"; do
printf '%-24s' "$cmd"
sh -c "$cmd" 2>&1 | wc -c
done
sh -c runs each string as a full command, flags included, in bash or zsh. Run this from your own shell, not inside an agent session. Vitest changes its reporter by itself when it detects an AI coding agent (flag 1), so measuring inside the agent hides the "before".
JavaScript and Python: flags 1 to 4
1. Vitest: --reporter=minimal
Vitest 4.1 added a reporter "to reduce ai agent token usage". The current reporters guide calls it minimal, with agent as an alias. It prints only failed tests and their error messages, and suppresses console logs from passing tests and the summary section. When no reporter is configured and Vitest detects an AI coding agent, it uses this one automatically.
Pass it yourself when your config sets reporters, or when detection doesn't recognise your agent:
npx vitest run --reporter=minimal
If you want a progress signal, --reporter=dot prints "a single dot for each completed test", with details only for failures plus the summary. And the CLI's --silent=passed-only keeps console output from failing tests only.
2. Jest: --silent
If your code logs while tests run, every one of those lines lands in the agent's context. Jest's CLI has one flag for that: --silent will "Prevent tests from printing messages through the console."
npx jest --silent
Jest has a default that bites agents: verbose is false, "or true if there is only one test file to run". The quick rerun of one file after a fix, the run agents do all the time, is the run that reports each test individually.
3. pytest: -q
-q is pytest's "quiet - less verbose - mode". The flag to look for in your scripts and addopts is the opposite one: -v gives each test its own line, and -vv stops truncating assertion text entirely.
4. pytest: --tb=short
The default, --tb=auto, prints "'long' tracebacks for the first and last entry". --tb=short is the "shorter traceback format", and --tb=line is "only one line per failure". Short keeps the file, the line and the failing assertion, which is usually what the agent needs to act. Line is often too little.
pytest -q --tb=short
One more from the same page: if addopts turns on --showlocals, every traceback carries the local variables. --no-showlocals hides them for that run.
Go, Rust, PHP and Ruby: flags 5 to 8
5. Go: take -v off
go test ./... is already quiet: a passing package prints only the final ok summary line. The -v flag logs all tests as they run, and will "Also print all text from Log and Logf calls even if the test succeeds." If a Makefile target or the agent adds -v, every passing test and every t.Log comes back.
If you want a friendlier view, gotestsum wraps go test -json. Its default pkgname format prints one line per package, and dots prints one character per test. Flags for go test go after --:
go test ./...
gotestsum --format dots -- ./...
pkgname grows with packages and dots grows with tests, so measure both on your repo.
6. Cargo: -q, on both sides of --
cargo test -q -- --quiet
The first -q is Cargo's own and hides its log messages. Everything after -- goes to the test harness. There, libtest's --quiet "Displays one character per test instead of one line per test" and is an alias for --format=terse. The default, pretty, is one line per test.
7. PHPUnit: --no-progress
PHPUnit's TextUI docs list --no-progress: "Disable output of test execution progress." Failures and the result summary still print.
vendor/bin/phpunit --no-progress
Two neighbours to avoid in an agent loop: --testdox will "Replace default result output with TestDox format", which reports each test by name, and --no-output will "Disable all output", which takes the failure details with it.
8. RSpec: --format progress
RSpec defaults to the progress formatter: . for a pass, F for a failure, * for pending. The documentation formatter prints the text of every describe and it instead. Check the project's .rspec file: if it has --format documentation, change it to --format progress or delete the line.
Flag 9: stop at the first failure
Every runner above has a version of the same switch. When the agent is fixing one thing, the failures after the first are output it reads before it can act on any of them.
| Runner | Stop early | What the docs say |
|---|---|---|
| Vitest | --bail=1 | stop after N failed tests |
| Jest | --bail | N failing suites, default 1 |
| pytest | -x | stop after first failure |
| go test | -failfast | no new tests after one fails |
| PHPUnit | --stop-on-failure | stop on first failure |
| RSpec | --fail-fast | stop on first failed test |
Sources: Vitest CLI, Jest CLI, pytest, go test, PHPUnit, RSpec.
Cargo stops early at a coarser level: it exits after the first test executable fails, but libtest runs every test inside that executable. Libtest's own --fail-fast exists, and it needs -Z unstable-options.
Put together, the fast loop looks like this:
npx vitest run --reporter=minimal --bail=1
npx jest --silent --bail
pytest -q --tb=short -x
go test -failfast ./...
cargo test -q -- --quiet
vendor/bin/phpunit --no-progress --stop-on-failure
bundle exec rspec --format progress --fail-fast
Stopping early is for the loop, not the finish line. Before the agent says it's done, it should run the whole suite, quietly, and read every failure.
Pin the commands, then watch the meter
Flags help only if the agent uses them. Put the commands in the file your agent reads when a session starts. Claude Code loads CLAUDE.md "into context at session start". agents.md is the open format that Codex, Gemini CLI, Cursor, Aider and others read.
## Tests
- While fixing: `pytest -q --tb=short -x tests/test_orders.py`
- Before saying done: `pytest -q --tb=short`
Claude Code's cost guide lists two heavier options too: a PreToolUse hook that rewrites test commands so only failures come back, and running tests in a subagent so "the verbose output stays in the subagent's context while only a summary returns".
Then you want to see the effect. I build Hermes IDE, so weigh this part with that in mind. Hermes runs Claude Code, Codex and other CLI agents in real terminals, and version 2.3.4 added a context meter that shows how full the conversation is and offers Compact when it gets close to full.
!Hermes IDE with a Claude Code session showing its context breakdown, per-session context gauges in the sidebar and a status strip with token total and estimated cost
Hermes 2.3.4: Claude Code's /context breakdown after pressing the Context chip, per-session context gauges in the sidebar, and the token total and estimated cost in the status strip.
Run the suite once with default flags and once with the quiet ones, and watch the meter between turns. Compact is there when you need it, in the Commands list above each session.
!The Hermes action bar above a Claude Code session with the Commands list open, showing pinned Context, Model, Plan and Review and built-in commands such as Compact, Usage and Diff
The action bar above a Claude Code session, with the Commands list open: pinned Context, Model, Plan and Review, then built-in commands such as Compact, Usage and Diff.
Compacting is the fix after the fact, and not a free one: Claude Code's docs note that /compact "reads the conversation it summarizes, so compacting a large context is itself a large request." The flags above are the cheaper fix.
Hermes is free, source-available under BSL 1.1, and runs on macOS, Windows and Linux. It isn't affiliated with Nous Research, Anthropic, OpenAI or any agent vendor. Download it here, and the code is on GitHub.
---
If this was useful
Counting bytes with wc -c is the crude version of a question that gets harder as a team grows: what did the model actually read, and what did that cost? The LLM Observability Pocket Guide compares the tracing and eval tools that answer it across a whole system, not one terminal.

1 comment
We migrated 70,000 tests from Jest to Vitest. Yes, in 2 months
YOU got it right. We migrated 70,000 tests (reads: 70 thousand) from Jest to Vitest. The logical question would be: how? And here is where the story begins.
I work as a software engineer in the Frontend Developer Experience team at Miro. Our ownership covers various parts of the platform. One of the major focuses for us is frontend infrastructure and tooling – everything that makes the product teams at Miro fast and productive. 2026 was indeed a pivotal year for us: we were finally able to kill Angular.js (I know, right), switched from Prettier to Oxfmt, got rid of ESLint in favor of Oxlint, we are running the CI typechecking with TypeScript 7 and much more. But look, you are here to read about unit-tests! In this article, I will cover how we migrated the main Miro frontend application unit testing harness from Jest to Vitest.
The beginning
The main Miro application is constantly evolving and sits now at ~6 million LoC. At this scale, the regular approaches and tooling stop functioning. How can I tell? Here are some facts about our CI state, unit-test jobs specifically:
- We run 6 CI runners in parallel just for unit-tests. Each runner takes up to 20 minutes to finish
- For Jest 30 the average speed is ~13 tests per second, even with SWC as the code parser
- 25% of all runs are failing because of memory leaks, i.e. the CI runners are dying with OOM errors
- We had to apply specific patches for Jest core packages to maintain a better worker pool
(were never accepted upstream)
- Migrating from Jest 29 to Jest 30 (yes, the first major release in 3 years) gave us only around 5% speed and memory improvements
This does not look bright, and it has a big impact on the new releases for our application. But what could we do about that?
Lurking through the Slack messages, I found a few threads where we actually discussed if we want to migrate away from Jest. Back in November 2023, one of my colleagues wrote:
> Although I’ve spent some time also exploring an alternative (vitest) but I’ve found that in practice the extra set of vitest specific configurations and replicating our current build might be too risky, without knowing how much speed benefits we get now that we’re relying on @swc/jest
At that moment, the scale of the problem was manageable – we just started to run the unit-tests in parallel on CI, compensating for the waiting time with the compute power from extra runners. Worked pretty well.
In July 2024, another colleague of mine made a remark regarding the unit-test tooling one more time:
> I expect higher returns from better test selection (e.g. not running 35k unit tests each time). The cost of switching to vitest (for example) is high and unclear what the improvement is
Fair enough! 6 runners for 35,000 tests in parallel took just around 7 minutes each. With job durations like this, we had much bigger problems to solve, think of end-to-end tests, for example.
The problem
Fast-forward to March 2025. We started facing more and more issues with the Jest stack for testing. Let me give just one example that triggered me to explore the alternatives to Jest. Apparently, our engineers reported that with the VSCode Jest extension, multiple versions of fb-watchman (Jest’s in-house filesystem watcher library) were installed, which led to CPU saturation and, as a result, the inability to use their laptops. We figured out how to fix it with watchman: false. At that moment, Ahmed, a Principal engineer in the Frontend Developer Experience team, had enough:
> I want to say maybe it's time to switch to vitest, but that's a massive effort…
We had a conversation regarding this, but I had no capacity to lead the migration right away. We also saw a lot of traction regarding the AI models and how they rapidly evolve for more and more complex problems. All signs were to wait for a bit. And this is how we landed in 2026.
Now you may ask: “Sergey, why did you decide to explore the migration again, considering all the facts you just mentioned?” Don’t get me wrong, we are not solving problems of this scale just because it’s cool. We need to understand what we want to achieve. I will start from the facts first:
The main Miro application is a monorepo, that includes both web and native parts. It is managed by yarn and NX. At the beginning of 2026, we had 800+ packages in our codebase. The largest code share is located in the web core, which accounts for around 20% of the codebase. We also had ~70,000 unit-tests (doubled in 2 years, imagine that) and the unit-tests jobs were taking up to 20 minutes on average to finish, which made them the slowest CI jobs in our pipeline, even worse than building the application and running the e2e tests on the new version!
Now, we can already see a few pain points we are fighting against. I can compile the whole list for you:
1. CI runners memory limit. 25% of the runs are failing because of memory leaks
2. Even with the SWC parser, Jest is slow, at 13 tests/sec for us (we use JSDOM and canvas mock for testing)
3. The TypeScript compatibility is not excellent. There are still API methods that infer the any type.
4. There is no proper ESM support, no mock hoisting. We add the spying code and mock factories between the top-level imports, so the import statements are downleveled to require() and can pick the mocked implementation.
5. Because of the previous point, we still transform the code to CommonJS. This means we are not testing what we are delivering to our customers (we deliver the ESM bundles).
6. The test coverage report is inaccurate. The packages do not share the instrumentation outside of each project root directory. This effectively means that there is missing cross-package coverage.
7. No test isolation. The jest workers in the pool are not spawned for each module; rather, they are recycled once a worker hits the memory limit.
8. There are no guardrails for the test fixtures to prevent leaking into the production bundle (funny spoiler: some teams actually used the test fixtures in the production code, intentionally)
9. Let's be real, it took Jest 3 years to release a new major version. We have opened a few issues on GitHub, but we did not see any traction around it.
10. Not only us, did I mention the incompatibility with other popular libraries?
Prettier, for example.
I can go on and on, but I think this is already impressive.
Now, you cannot just take 70k tests and rewrite them with a random testing framework (imagine you could!). You also have to educate hundreds of product engineers, and the scale of the change would be massive. And also, how about all the custom plugins, matchers, and supporting dependencies? Think of jest-axe, jest-when, React Testing Library, and others. The answer to all of this is – we are looking for a Jest-compatible api. Compatible enough so we won’t turn our monorepo into chaos.
Where to migrate?
Now, the market research gave us two alternatives: Rstest and Vitest (versions as of March 2026):
RStest v0.9.8 (rstest.rs)
- Part of Rstack toolchain, so we can reuse the Rspack configuration from our application
- Modern ESM-based syntax (think of
import attributes and more explicit mocking syntax)
- Integration with Rsdoctor for debugging
Vitest v4.1.6 (vitest.dev)
- Wide community adoption
- Type safety and ESM-first
- Test isolation and rich, comprehensive api
You can guess Vitest won, but why? I did my homework: I created two branches and migrated around 10k tests to each of the frameworks. Then I tried to execute the tests in different modes. Here are the results (the values are tests per second, the bigger, the better):
| | Jest | Vitest | Rstest |
|:----------------------------|:-----------|:-----------|:----------|
| Local run | 14.7 t/sec | 16.8 t/sec | 7.6 t/sec |
| Local run with coverage | 5.9 t/sec | 8 t/sec | 2.9 t/sec |
| CI run | 12.9 t/sec | 13 t/sec | OOM |
| CI run with coverage | 5 t/sec | 8.6 t/sec | OOM |
You can already notice that Rstest failed to run on CI, mainly because of bad memory management in the worker pool. Keep in mind that it was pre-1.0 version and the runner is under active development, so the hopes were low (I encourage you to try the latest version and explore the current state of things). In the meantime, migrating this small test subset showed a huge difference in memory consumption for Vitest – more than 10GB less memory used on the runner compared to the same group of tests in Jest 🚀.
Just to be honest here, the speed and RAM consumption were indeed among the main factors. And although the research did not show any meaningful speed improvements, we decided to execute the migration as we saw opportunities to optimize Vitest configuration and try to make it faster. It also covers other pain points we were targeting to solve. Yes, we have a front-runner, and we are now coming to a plan!
The plan
As I said already, the Miro web application is a monorepo orchestrated by NX and Yarn. It combines the web core monolith, holding ~20% of all the code. Other parts are: 800+ packages and the part for native applications. And since the native part has its own CI and release pipeline, we decided to leave it out of scope.
On a separate note, the application accepts 60+ merges into the main branch on a daily basis. And as much as I wanted to “just” ask the AI agents to migrate everything in one go, it was not possible for us to execute. I decided to go with a gradual rollout:
1. Run both Jest and Vitest on separate runners on CI in parallel. As the migration goes, reallocate the runners depending on the share of migrated tests
2. Start with packages. This way, we will be able to deliver the meaningful parts, and it is easy to discover which tests should go to Jest and which to Vitest.
3. Use AI agents to migrate the tests. We create a skill for the agents in the repo, so the agents have all necessary context. After each migration phase, we update our patterns not mentioned in the skill files.
The last bit is the most powerful. Now, the agents learn and remember new patterns. This makes the migration faster and produces fewer hallucinations with each iteration. This also makes the migration more consistent – the agents apply the same patterns they saw before. Not only that, once completed, you will end up having a huge list of very specific code examples the agents found in the tests – an amazing source of insights to reflect on. It can easily open the room for follow-ups, good practices, and knowledge sharing.
Act 1 - Infrastructure
I started with infrastructure for Vitest. The basics include having the very first version of Vitest config to reuse in the application.
Then we added the compatibility bridges. Think about shared mocks and helpers that create the fake data. During the time of migration, we do not want to duplicate such files. Hence, we added the shim globalThis.jest = vi in Vitest config. And globalThis.vi = jest in Jest. Yes, the api is not 100% compatible, but this was enough in our case.
Now the whole idea of migration was built around a gradual switch package by package. Each package runs the scripts via the CLI commands, think of it as having a command miro-scripts test. It took me to add the runner’s opt-in argument, so we can distinguish which test runner to spawn. Looks like this: miro-scripts test --runner=vitest in the package.json file.
It was not just a local story. This flag allowed us to see how to run these tests on CI runners. As we specialized the runners per tool, we combined all Vitest projects into one pool of runners, and the rest was for the remaining Jest projects.
Finally, we adjusted the observability metrics in order to track the progress and test distribution. We added the extra runner label for test-related Prometheus metrics, which made it possible to visualize the migration.
Act 2 - First steps
Now we were all set to start the actual migration. All guardrails were there, the CI picked up all the tests, so we were not losing quality. The only missing part was the test coverage for Vitest, but we agreed to handle it after the transition.
I started with small packages. This allowed us to keep the pull-requests size manageable. This way I could:
1. Review the changes more precisely (as I said, in the later stages, the built context was so rich that it could hardly produce off-track changes)
2. Form the foundation of the AI skill. First instructions, patterns, and polyfills were missing in the config
3. Convince teams and build trust. We, as a Frontend Developer Experience team, knew we needed to modernise the tech stack. But it was not obvious for the product teams. I made sure to be vocal about the migration and tried to iterate on the reasoning behind the change
4. Start collecting issues. For example, there were some extensions in VSCode that tried to guess the test framework used in the repo by the root-level config. With both Jest and Vitest configs at the root, it gave priority to Vitest, which at that time had only a fraction of packages with completed migration – just one of those things that I could only assume at the beginning.
Once the first version of the AI skill was ready, I made sure to focus on the community work, presenting on it internally and explaining how each team could DIY their own migration. The skill is in the repo, just ask your agent to migrate the packages under your ownership! ✨
Act 3 - Goes well
We started off strong with the migration. The first 90 packages went pretty well. At this moment, we built the needed Vitest plugins to cover parts of Jest configuration and added a decent number of global polyfills. I intentionally did not ask the AI agents to mirror the Jest config – nobody touched it for years, and it absorbed so much legacy, I did not want to port it to Vitest and started from scratch. I also tried to run the migration based on ownership, i.e. in a single pull-request I would only transform tests for the single owning team. Unfortunately, this did not work as expected. We have so many cross-package helpers and fixture generators, so it was never a single team, rather a few owners here and there. But that was still acceptable to us.
At that point, I felt so confident that I decided to take one of the biggest packages in our repo (apart from the web monolith). ~600 test files all at once, a true battle test for the migration flow. BOOM, we landed it in the main branch! It was a huge milestone and proof we could continue.
On another side, we kept working with the frontend community and were very transparent about the effort. As a result, we started receiving positive feedback about the progress in general and how smoothly the migration was going.
Act 4 - It couldn’t be _that_ easy
Although the mechanical migration was progressing well, our responsibility never changed – stable CI processes. Not only did we need to migrate the code, but we also needed to follow the metrics. Unfortunately, the metrics started showing a coming disaster.
The CI runner started consuming more and more memory. I was confused! My initial research gave me the signal that Vitest consumed much LESS memory, but we were around 50% of the way through the migration, and it was getting only worse?! We had to pause everything until we figured it out. Imagine my mental state at that time: we were in the middle of a transition. I couldn’t roll everything back, but I also could not move on. What do I do now? Did I fail?!
I was buried in the problem for days until I came to the realization:
Each package in the repository is defined as a Vitest project (for simplicity, I will use the terms package and project interchangeably). Turns out, Vitest projects do not share the transformation cache! Every project builds up the cache with a unique identifier under the hood. If packages are heavily interconnected, Vitest spawns a Vite server for each package and transforms the same module graph again and again. So 10 linked packages will transform the same modules 10 times. Now scale it up to 800 packages, and you will get it. And in my initial research, I only used the subset of the tests inside a single Vitest project, thus, I could not actually see it coming.
The solution is to run fewer projects, not by ignoring some of them, but rather by merging the projects into a single instance. Fortunately, most of the packages in our application have the default shared Vitest config with no overrides. So it was easy to write a project discovery script that would combine all packages with default configuration into a single project. By doing this, we were able to run only 60 Vitest projects that actually account for 400 packages (59 custom projects + 1 project that includes the remaining files).
I will give you a sneak peek with the numbers:
| Metric | Before | After | Delta | % Change |
|:-------------------|:----------|:----------|:-----------|:---------|
| Heap Memory | 8,770 MB | 5,024 MB | -3,746 MB | -42.71% |
| Total Memory | 21,403 MB | 18,564 MB | -2,839 MB | -13.26% |
| Total Duration | 1059.83s | 611.06s | -448.77s | -42.34% |
| Transform Time | 10756.03s | 4327.69s | -6428.34s | -59.76% |
| Setup Time | 1495.36s | 1277.27s | -218.09s | -14.58% |
| Import Time | 13452.26s | 6984.71s | -6467.55s | -48.08% |
| Tests Time | 257.29s | 262.76s | +5.47s | +2.13% |
| Environment Time | 344.18s | 371.06s | +26.88s | +7.81% |
Curious readers may ask: why did Environment time and Test time increase? Imagine you run 10 Vitest workers in parallel. And only a single main process that runs Vite to transform the import tree. Before the change, the workers would just hang and wait when the main process transforms the same modules again and again. Once we merged the projects, the transformation time dropped significantly. Now the workers do not wait, they run more tests, which causes CPU saturation and system pressure in general. This effectively results in slower worker performance. But since the actual test run takes only a fraction of the overall test execution process, the degradation is negligible.
That being said, combining all standard projects into a single instance made an outstanding impact on the performance and confirmed my assumption. The most important thing is that we can now continue with migration!
Act 5 – Closer to the goal
The next 60 pull-requests were just mechanical. The AI agent has built up the comprehensive skill context. My job was to check the bad-looking and non-trivial changes. Then also hear the feedback from the reviewers, since I cannot in this world know each and every unit-test in the Miro application.
One nice cherry on top was the fact that with the Jest → Vitest migration, we started noticing some test fixtures that had accidentally leaked into our production bundle (remember point 8?). The logical question would be: how? How did we detect this, and how did the test data leak into the production bundle?
The first part was easy for us – we build the web application with Rspack with no overrides or patches in the config to support Node.js modules. In fact, Vitest uses the NodeJS api under the hood (it is a Node library in the end). As a result, Rspack would fail to build the application once any part of Vitest is imported in the dependency graph.
How does it usually happen for us?
1. Barrel files. Every package in the repository exports its API via the barrel file index.ts. Although it would work fine in small projects, barrel files give nothing but pain as the application gets bigger. Barrel files, and especially star exports (export * from ‘./foo.ts’) are hard to tree-shake, and you never know what exactly is being exported. Accidentally, some test shapes were exported transitively as well.
2. Some teams actually used test helpers to create artificial users and accounts in the production code. These were edge cases but still happened.
The Vitest migration clearly highlighted such cases and made us improve the codebase. This gave us approximately a 4MB cut from the full production build.
Act 6 - One-shotting the web core
We had migrated all packages at that point, and the AI agents now had a definitive playbook on how to run the migration. Now it was time for the final boss: the web core monolith. 1,800 test files, 16,500 tests. I realized it was too risky to run the migration during business hours – engineers merge pull-requests once every 10 minutes. I would spend more time resolving the conflicts. I decided to handle it during the weekend (no worries, I took two days off to compensate!)
On Saturday, I told the AI agent to run the migration. The result: 1,985 files changed, +22,776 / -19,998 lines diff. This pull-request tagged each and every team for review, but my plan was to merge it right away with an admin override approval. It wasn’t perfect, but it was decent enough. The rest we could fix later with smaller PRs during business hours. It took me an entire day to look at the changelog!
I clicked the Bypass rules and merge button and waited for the main branch CI to complete. I could finally go to sleep. We made it.
Act 7 – Aftermath breakthrough
I don’t want to give the impression that the transition was perfectly smooth. Once we completed the mechanical work, we had to clean up the configs, revert the hybrid setup, and start removing Jest dependencies. But this is fine. The bigger problem (ironically) was that Vitest is a much more reliable tool than Jest. It has its own timeout for module imports, it catches unhandled errors, it races the duplicated vi.mock blocks. All of this actually made a small subset of tests more flaky. We spent some time stabilizing the tests, exploring the actual cause, and working with the owners to see how we could address it. A fractional chunk of tests (think of 27 test cases) was completely skipped for the sake of CI stability.
But one question still stood out in my mind – we were able to speed up the run and save memory by merging the projects. Could we merge everything into a single Vitest project?
One of the advantages of the transition to Vitest is that we were able to drop tons of legacy configurations. This made all custom Vitest configs look like this:
import { defineProject, mergeConfig } from 'vitest/config';
import defaultCfg from '@shared/config/vitest/shared.config.js';
export default mergeConfig(
defaultCfg,
defineProject({
test: {
setupFiles: ['./setupVitest.ts']
}
})
);
Only the setup script is what makes us create the custom configs. Why does that matter? If the setup file is the only difference, maybe I can detect which test file is running and call the setup script relative to that file!
There is no documentation around it. The only hint I got is this page, which explains the interface for the expect api. Apparently, expect.getState: () => MatcherState, and then MatcherState has the field testPath - the absolute test path, which is exactly what we are looking for.
Now I quickly spun up the shared setup script, which takes the test path, derives the package the test is located in. and then dynamically executes the setup script attached to that specific test file. Here is the idea:
import { existsSync } from 'node:fs';
import { relative, resolve } from 'node:path';
import { expect } from 'vitest';
const repoRootDir = '%repo_root_path%';
const absoluteTestPath = expect.getState().testPath;
const relativeTestPath = absoluteTestPath ? relative(REPO_ROOT_DIR, absoluteTestPath) : '';
// you can manipulate the path to find the right location here
const vitestProjectRootDir = fetchProjectDirForTestFile(relativeTestPath);
const setupFile = vitestProjectRootDir ? resolve(repoRootDir, vitestProjectRootDir, 'setupVitest.ts') : '';
if (setupFile && existsSync(setupFile)) {
await import(setupFile);
}
You can make this script project-agnostic and execute it for each test file. Doing this, we were able to merge ALL 800+ Vitest projects in our application into a single test run. No transformation overhead, no cache duplications. To give you an idea of how impactful it was, I will give you a sample:
| Metric | Before | After | Delta | % Change |
|:-------------------|:--------------|:------------|:---------------|:---------|
| Heap Memory | 5,238 MB | 1,857 MB | -3,381 MB | -65% |
| Total Memory | 22,460 MB | 17,450 MB | -5,010 MB | -22.3% |
| Total Duration | 865.64s | 588.82s | -276,82s | -32% |
| Test speed | 12.6 t/sec | 19 t/sec | +6.4 t/sec | +50% |
Finally, we found the source of the speed ✨ And this concluded our road to landing Vitest in our main Frontend application.
Results
| Active migration time | About 80 days (May 4 to July 20) |
|:----------------------|:----------------------------------|
| Merged pull requests | 97 |
| Lines added/removed | +87,701 / -78,946 |
| Net lines | +8,755 |
| Files touched | 9,328 |
Notice we have added almost 9,000 net lines of code. The actual net is much lower, because we moved from globally injected api (think jest, it, describe, etc.) to locally imported objects (import {describe} from ‘vitest’). Plus, code formatting gave some diff. Now to the technical details:
- The test runs are 50% faster than they were with Jest (13 t/sec to 19 t/sec)
- Vitest consumes 30% less memory on the CI runners
- Vitest mocks are type-safe i.e. they inherit the signature from the mocked object, allowing engineers to write better unit-tests.
- ESM-first. The mocks are hoisted automatically, which gives again the better experience for the engineers.
- Production bundle saved around 4MB, the test fixtures were pulled out from the production barrel files
- Real test isolation – each test file is being run on an isolated worker in the pool, ensuring tests do not interfere with each other
- Accurate test coverage. allowExternal: true allows us to collect the test coverage, dropping the project boundary. We collect the full picture.
- We collected the test polyfills from all files, generalised them, and stored them in a shared configuration. Not only did it simplify the tests, but we also made sure engineers have a consistent experience with the polyfilled api’s
- The migration AI skill was transformed into the unit-tests creation skill. Now it gives the agents the common patterns from our codebase, which enabled better test generation
The total model usage during this period cost around $7,000. To put this into perspective, I estimated the annual savings from the migration on CI runners at approximately $75,000. We should also account for more application releases: as I mentioned at the beginning, we started facing more and more memory issues for Jest runs. Add here the faster feedback loop during the local development as well, which is, to be fair, hard to estimate. All of this made the investments a huge win for us.
Lessons learned
In 2026 here at Miro, we keep experimenting with large-scale migrations. Our goal is to provide engineers with the best-in-class tooling and infrastructure. We have already made a transition from Prettier to Oxfmt, from ESLint to Oxlint, from Webpack to Rspack. It was just about time to switch from Jest to Vitest. Each journey results in lots of artifacts for future improvements. And lessons learned along the way. Here are some of them:
1. Set a clear goal. Be specific about what you want to accomplish. Research the market and find the tools that can move you towards the goal. Experiment with a subset of the codebase to be able to estimate the scale and feasibility, and to get a feel for the planned work.
2. Plan the execution. Sometimes you just need to make a big bang migration. Sometimes you can be more gentle and have a gradual rollout. For Vitest, I made a safe bet to iterate and build up the context for the AI agents. It worked pretty well, and at some point we were able to one-shot the remaining tests.
3. Be specific in the prompts with AI agents. Give a deterministic way to reproduce the result. Set the constraints. It was not possible to run the subagents in parallel because Vitest takes so much memory in our case. I asked to run the migration sequentially, so it did not ruin my workstation.
4. Build trust. People resist change. My goal was to give the reasoning behind the change, acknowledge the struggle to understand where to use which runner, and help to adopt the new tooling.
5. You can not plan for everything. There are always issues you could not possibly expect during the preparation. You need to accept it and not be afraid to pivot and look up for the solution.
6. The scale of change does not matter at this point. With Fable 5, or Sol, AI can execute any volume of change. If you can formalise what needs to be done, if you can create a context for the agent, if you can give a deterministic reproduction – the implementation part is extremely democratised.
7. Trust the CI. Some changes during the migration were trivial. Others touched the production logic. I worked with the code owners to understand if the change I am introducing is legit. But at the end of the day, you have to build the release quality gates that are trustworthy, and a rollback process so fast, so even if you end up having a bad day, you can easily fix the mistake.
To wrap it up
We have completed the migration of our main frontend application from Jest to Vitest. The transition took 70,000 tests to transform (75,000 as of the last merged pull-request date). It took us just a bit over 2 months to execute. We used Opus 4.8 and Fable 5 (once it was released) to drive the migration.
Migration to Vitest gave us space and resources to grow further, modernised our codebase, and allowed us to start writing type-safe unit-tests. It also caught the test data leaks into the production bundle and actual production bugs.
This project paid off in all areas we were targeting. It also built a story of how useful the AI agents can be if done right.
Special credits go to Evgeny Evsyukhin, who has heavily contributed to the migration as well as the observability infrastructure around Vitest. I want to also mention Ivan Voronin, Ahmed El Gabri, Frederico Estrela, Vinicius Kiatkoski Neves, and my beloved Frontend Developer Experience team for making the migration possible. I also want to give a shoutout to Ben Makuh for proofreading and helping me refine this article.
PS: Since we created this Jest to Vitest migration skill, I thought it would be nice to create an open-source version as well. It can be a good foundation for your own contextual migration skill. Feel free to use and contribute here.
Mentions
React Compiler 導入で画面が更新されなくなった話 〜既存SPAでの主要ライブラリとの衝突と解決の記録〜 ! 検証環境 React 19.2.8 / babel-plugin-react-compiler 1.0.0 Vite 8.1.5 @vitejs/plugin-react 6.0.4 @rolldown/plugin-babel 0.2.4 @babel/core 8.0.7 React Hook Form 7.82.0 TanStack Router 1.170.18 TanStack Table 8.21.3 Vitest 4.1.10 Oxlint 1.75.0 記事中のコード・挙動・リンクはすべて上記バージョンでのものです。公開時点では TanStack Router・Oxlint などに新しいバージョンが出ており、一部の挙動や設定は変わっています。該当箇所ではその旨を注記しています。 はじめに こんにちは☀️ 株式会社AI Shiftでフロントエンドエンジニアをしているさっちゃんです!(X:おさわ / 吉岡 爽) 今回は、プロダクトに React Compiler を導入した際に UI に発生した不具合と、その対処について書きます。 私達のプロダクトでは React を採用し、Vite と TanStack Router を用いて、SPA を構築しています。フォームには React Hook Form、一覧には TanStack Table を使っています。 React Compilerの導入にあたり、導入前の実装をCompilerのON/OFFで比較しました。その中で、React Compiler を有効にしてテストを実行したところ、いくつかの箇所で異常が検知されました。これをきっかけに実際の画面上で動作確認を行った結果、サイドバーが画面遷移に追従しないなど、ユーザーの操作が意図通りにUIへ反映されない不具合が発覚しました。 この記事では、実際の画面の操作へ影響していたこのサイドバーの問題を中心に、設定・切り分け・原因・対処を紹介します。 この記事について 想定読者:React Compiler の導入を検討している方 書くこと 既存の SPA に React Compiler を導入したときの設定 導入前の実装に Compiler を適用し、ON/OFFで挙動を比較した方法 TanStack Routerの不具合と、React Hook Form の挙動変化、その原因・対処 テストの失敗と、実際の画面への影響を分けて確認したこと 書かないこと React Compiler 導入による性能改善の計測結果 React Compiler の内部実装の詳細 SSR / React Server Components 環境での挙動 導入設定 まずは、今回の Vite 8 + Babel プラグイン環境で React Compiler を導入した際の設定を紹介します。 パッケージの追加 React Compiler を導入するため、以下のパッケージをインストールします。 pnpm add -D -E babel-plugin-react-compiler@1.0.0 @rolldown/plugin-babel@0.2.4 @babel/core@8.0.7 Vite Ver.8 ではバンドラに Rolldown、JS/TS や JSX の変換に Oxc が使われています。 また、 @vitejs/plugin-react v6 では内部の Babel 依存が排除されました。そのため、今回使用した v6.0.4 で、Babelプラグインとして提供される React Compiler を適用するために、@rolldown/plugin-babel を組み合わせました。 ! @vitejs/plugin-react 6.1.0 以降では、oxc-transform-react を用いて Babel なしで Compiler を有効化する実験的な機能も追加されています。 react({ compiler: true }) この記事では、より実績があり安定している Babel プラグイン版を対象とします。 参考:@vitejs/plugin-react 6.1.0 (2026-08-19) CHANGELOG.md また、React Compiler はバージョンアップによってメモ化の挙動が変わる可能性があるため、予期せぬ不具合を防ぐ目的で -E フラグを使い、exact バージョンでインストールしています。 vite.config.ts の実装 Compilerに関係する部分を抜き出すと、次のようになります。 import { defineConfig } from "vite"; import react, { reactCompilerPreset } from "@vitejs/plugin-react"; import babel from "@rolldown/plugin-babel"; const compilerPreset = reactCompilerPreset(); compilerPreset.rolldown.filter = { ...compilerPreset.rolldown.filter, id: { exclude: [/node_modules/, /.(test|spec).[jt]sx?$/] }, }; export default defineConfig({ plugins: [ react(), ...(process.env.REACT_COMPILER === "false" ? [] : [babel({ presets: [compilerPreset] })]), ], }); @vitejs/plugin-react が提供する reactCompilerPreset() をベースに、この設定に加えてLintも導入しました。 1. node_modules とテストファイルを変換対象から外す まず、node_modules 配下のライブラリは自分たちが管理しているコードではないため、変換対象から除外しました。 また、.test.* や .spec.* などのテストファイルも外しています。テストコード内で定義したモックやラッパーコンポーネントまで Compiler の対象になってしまうと、テストが失敗したときに、本体の変換による影響なのか、テスト側の変換による影響なのかを切り分けることが難しくなるためです。 2. 環境変数で無効化できるようにする なにか不具合が起きたとき、それが元のコードの問題か、Compiler による変換の影響かを切り分けやすくするために、REACT_COMPILER=false という環境変数を渡すことで、Compiler を無効化した状態で起動・ビルド・テストができるようにしました。 Vitest でも Compiler が適用されているか確認する 先ほどテストファイル(.test.ts など)自体は変換対象から除外しましたが、テストを実行できても、テスト対象のコードにCompilerが適用されていなければ、導入後の挙動を検証したことにはなりません。 reactCompilerPreset() は、Vite の Environment API において「クライアント向けのビルド・変換時」にのみ Babel(Compiler)を適用するように初期設定されています。 Vitest 4 では、jsdom 環境は実際には Node.js 上で実行されるため、モジュールの読み込み自体は SSR(サーバー)側の仕組みを使います。しかし、Vitest 4 が Vite のプラグインパイプライン(Environment API)を呼び出す際、jsdom のようなブラウザの動きを再現する環境では 「コンポーネントの変換(Transform)処理においては Client 環境向けのプラグイン設定を適用する」 という設計になっています (Vitest のソースコード)。 そのため、開発者が vite.config.ts で applyToEnvironment などのフックを上書きして「常に全環境でBabelを動かす」といった設定をしなくても、自動的にテスト用のコンポーネントに Compiler が適用されます。 念のため、コンポーネントの関数を文字列化して、Compiler が挿入するキャッシュ(const $ = ...c(10) のような記述)が含まれているかを直接確認してみました。 import { CommandPalette } from "@/components/ui/CommandPalette"; test("Compiler が適用されている", () => { expect(CommandPalette.toString()).toMatch(/const \$ = .*.c)(\d+)/); }); 設定 結果 初期設定のまま 適用される REACT_COMPILER=false 適用されない 初めは、jsdom 環境には自動適用されないのではと疑い、applyToEnvironmentHook が常に true を返すように設定を上書きしていました。 しかし検証の結果、上書きしなくても正しく適用されることがわかりました。 3. Lint で Rules of React を守る React Compiler が正しく動くには、コードが Rules of React に沿って書かれていることが前提です。 あなたのアプリケーションでコンパイラを導入できるかどうかは、コードベースの健全性と React のルールをどの程度遵守しているかに依存します。 出典:React Compiler:安全に使用できるか? Compiler は、最適化を安全に適用できないと判断したコードについて、対象の最適化をスキップすることがあります。 ルールに違反している可能性のあるコードに遭遇した場合、アプリケーションの動作を変更するリスクを冒すのではなく、最適化を安全にスキップします。 出典:React Compiler:コンパイラの動作を理解する これを防ぐため、Oxlint の react/react-compiler ルールを error レベルで有効にして CI 等で検知できるようにしました。 { "plugins": ["react"], "rules": { "react/react-compiler": "error" } } これで Rules of React への違反をある程度未然に防げます。ただし、Lint が通ったからといって Compiler 由来の不具合が完全にゼロになるわけではありません。この記事で紹介する不具合も、Lint では検出できずテストで発覚したものでした。 ! ここで設定している react/react-compiler は、Oxlint 1.78.0 までのルールです。 1.79.0 で purity・refs・immutability などの個別ルールに分割され、react/react-compiler は削除されました(Oxlint CHANGELOG 1.79.0、Oxc 公式ブログ)。 1.79.0 以降でこの設定を書くと、Rule 'react-compiler' not found in plugin 'react' で設定ファイルの読み込みに失敗します。代わりに次のように設定してください。 { "plugins": ["react"], "categories": { "correctness": "error" } } 検証方法 テストが失敗した際、それが本当にReact Compiler の変換による影響なのかを正確に切り分ける必要がありました。 そこで、Compiler 導入前のコードベースに、設定・依存・比較用のテストを持ち込み、Compiler の ON/OFF 以外の条件を完全にそろえた状態を作って検証を行いました。 以下の手順は、Vitest に unit プロジェクトが定義され、jsdom 環境と、先ほどの Vite プラグイン設定を利用する構成を前提にしています。今回の設定では、extends: true で親の Vite 設定を引き継いでいます。関係する部分を抜き出すと、次のようになります。 // vite.config.ts // pluginsなどは、前述のCompiler設定と同じ export default defineConfig({ plugins: [ // 前述のViteプラグイン設定 ], test: { projects: [ { extends: true, test: { name: "unit", environment: "jsdom", globals: true, include: ["src/**/*.{test,spec}.{ts,tsx}"], }, }, ], }, }); 手順 導入直前のコミットを別ディレクトリに展開する git worktree add ../before <導入直前のコミット> cd ../before Compiler の設定・依存と、比較に使うテストを、導入後のコミットから持ってくる git checkout <導入後のコミット> -- package.json pnpm-lock.yaml vite.config.ts <対象のテストファイル> pnpm install Compiler 無効 REACT_COMPILER=false pnpm vitest run --project unit <対象のテストファイル> Compiler 有効 pnpm vitest run --project unit <対象のテストファイル> Compiler の切り替えには、前の章で用意した環境変数 REACT_COMPILER を使いました。 結果 フォーム・ナビゲーション・共通 UI に関わる77件のテストで比べた結果です(導入時点の計測)。 実装 Compiler 結果 導入前 無効 77件成功 導入前 有効 71件成功・6件失敗 対応後 有効 77件成功 失敗した6件は、次の2種類に分かれました。 種類 件数 症状 TanStack Router を使ったサイドバー 2件 SPA内の遷移後、子ページを含むグループが開かない React Hook Form を使ったダイアログ 4件 キャンセル直後、終了アニメーション中の入力欄に編集値が残る これらの失敗がテスト環境特有のエラーなのか、実際のUIに影響するものなのかを確かめるため、ブラウザ上で実際に操作して挙動を確認しました。 結果として、どちらも Compiler によって実際に画面の挙動が変わってしまった状態でした。ただし、ユーザーへの影響は異なります。サイドバーは現在地のリンクが見えなくなる一方、ダイアログは終了アニメーション中だけ値が残り、再表示時には初期化されていました。 対応の概要 見つかった2つの問題には、それぞれ次のように対処しました。 問題 今回の対処 次のステップ サイドバーのグループが開かない(TanStack Router) useMatchRoute を、毎回新しい関数を返すラッパーフック経由で使う 修正済みの TanStack Router v1.170.27 以降へ上げ、ラッパーを外す キャンセル直後に入力値が残る(React Hook Form) 対象のダイアログに "use no memo" を付け、Compiler の変換対象から外す reset() に keepFieldsRef: true を指定し、"use no memo" を外す どちらも、導入時点では既存の挙動を確実に保つことを優先した一時的な対応です。 特にダイアログの "use no memo" は、Compiler の最適化をあきらめることで不具合を避けている状態です。本来は、Compiler の最適化を受けたまま直せる keepFieldsRef に移行を検討しています。 なお、サイドバー側の "use no memo" は数行のラッパーフックにだけ付けており、サイドバーのコンポーネント自体は引き続き Compiler で最適化されています。 壊れたもの:サイドバーの外から移るとグループが開かない 症状 サイドバーでは、関連する子ページのリンクを、開閉できる見出しの下にまとめています(以下、グループと呼びます)。 現在のページがグループ内の子ページであれば、そのグループを自動で開き、現在地がわかるようにしています。 しかし、サイドバー以外の導線から、ページ全体を再読み込みせずに子ページへ遷移すると、ページ本体は切り替わるのに、サイドバーのグループが開きませんでした。 例えば、アプリ概要から設定グループ内の子ページへ遷移すると、ページ本体は切り替わるのに、サイドバーの設定グループは閉じたままになっていました。 👇 再現用の簡易アプリで、ダッシュボードのカードから「通知」へ移動したときの動き Before(修正前):ページは切り替わるが、サイドバーの「設定」は閉じたまま After(修正後):「設定」が開き、パンくずも「通知」に更新される サイドバーは、以下のように useMatchRoute() でリンクがアクティブかどうかを判定し、その判定関数をグループのコンポーネントに渡していました。 function NavEntryItem({ entry, params }: Props) { const matchRoute = useMatchRoute(); const isActive = (link: NavLink) => matchRoute({ to: link.to, params }) !== false; return <NavGroupItem entry={entry} isActive={isActive} />; } function NavGroupItem({ entry, isActive }: GroupProps) { const hasActiveChild = entry.children.some(isActive); // hasActiveChild が true になったらグループを開く // ... } 原因 useMatchRoute が返す関数は、URL が変わっても同じ参照のままだった 今回使用していた TanStack Router v1.170.18 では、useMatchRoute() の実装は次のようになっていました。型やオプションの処理を省略しています。 ! この節に掲載する Matches.tsx と router.ts のコードは、TanStack Router の実装から、説明に必要な部分を抜粋・簡略化したものです。 ライセンス:MIT License 著作権表記:Copyright (c) 2021-present Tanner Linsley 引用元コミットの LICENSE 全文 実際のソースコードMatches.tsx の実装 export function useMatchRoute() { const router = useRouter() useStore(router.stores.matchRouteDeps, (d) => d) // 👇 router インスタンスだけを依存配列にしている return React.useCallback((opts) => router.matchRoute(/* options を渡す */), [router]) } ブラウザ側ではルーターの状態を購読しているので、URLが変わると、このフックを使うコンポーネントは再レンダーされます。 一方、返す関数の依存配列は [router] なので、同じ router インスタンスを使っている間、URLが変わっても関数の参照は変わりません。 しかし matchRoute は、引数だけで結果が決まるわけではなく、「現在の URL」を読み取って判定を行っています。 ソースコードを確認してみると、router.matchRoute() は、呼び出された時点の現在地をルーターの内部状態から読み取っていました。 実際のソースコード router.ts の実装 // 👇 呼ばれた瞬間に、ルーターから現在地を取得している const baseLocation = this.stores.resolvedLocation.get() || this.stores.location.get() const match = findSingleMatch(next.pathname, /* ... / baseLocation.pathname, / ... */) 同じ matchRoute({ to: "/settings" }) という呼び出しでも、現在の URL によって戻り値が変わります。一致した場合はルートパラメータのオブジェクト、不一致なら false です。そのため、次の式は、アプリ概要ページでは false、設定ページでは true になります。 matchRoute({ to: "/settings" }) !== false; この「参照は同じでも、内部状態によって結果が変わる」という性質が、今回のメモ化と衝突しました。 Compilerによって何がキャッシュされたのかを確認する 実際のコンポーネントを babel-plugin-react-compiler@1.0.0 で変換すると、isActive の生成に次のようなキャッシュが入っていました。 // 【1】matchRoute や params の参照が変わったかチェック if ($[0] !== matchRoute || $[1] !== params) { t1 = (link) => matchRoute({ to: link.to, params }) !== false; // 新しく関数を作る $[0] = matchRoute; $[1] = params; $[2] = t1; // 作った関数をキャッシュに保存 } else { // どちらも変わっていなければ、前回の関数(キャッシュ)を使い回す! t1 = $[2]; } const isActive = t1; // 【2】NavGroupItem に渡す props(entry や isActive)が変わったかチェック if ($[3] !== entry || $[4] !== isActive) { t2 = <NavGroupItem entry={entry} isActive={isActive} />; // 新しいJSX要素を作る // ... } ここで登場する変数は、それぞれ次のものを表しています。 matchRoute:useMatchRoute() が返す、指定した遷移先が現在の URL に一致するか判定する関数です。一致するとルートパラメータのオブジェクト、不一致なら false を返します。 params:遷移先の URL を組み立てるためのルートパラメータです。IDやslug、環境などが入ります。同じID・環境内で概要ページから設定ページへ移る場合、これらの値は変わりません。 entry:サイドバーに表示する項目の定義です。グループの場合は、「設定」のような見出しや、配下に表示する子リンクの情報を持っています。現在のページが変わっても、グループの定義自体は変わりません。 $:Compiler がコンポーネントのインスタンスごとに用意するキャッシュ用の配列です。$[0] には前回の matchRoute、$[1] には前回の params、$[2] には生成した判定関数を保存しています。 t1・t2:Compiler が生成した一時変数です。t1 には isActive として使う判定関数、t2 には <NavGroupItem ... /> という JSX 要素が入ります。 今回の実装では、entry はコンポーネントの外に定義した固定の項目一覧から渡しています。また、Compiler によって、親が作る項目一覧の JSX もキャッシュされていました。今回のページ遷移ではそのキャッシュが再利用されたため、JSX に含まれる params オブジェクトも作り直されませんでした。このため、子が URL の変更を受けて再レンダーされても、entry と params は前回と同じ参照のままでした。 この結果を読み解くと、以下のようになります。 URL が変わり、useMatchRoute() の購読によって親が再レンダーされる matchRoute と params の参照が変わらないため、Compiler によって isActive が再利用される entry と isActive の参照も変わらないため、子へ渡す JSX が再利用される この経路では NavGroupItem の再レンダーが省略され、hasActiveChild が再計算されない 子ページへ移っても、グループが閉じたままになる Compiler なしなら、毎回のレンダーで matchRoute(...) が呼び直されるため、この問題は表に出ていませんでした。 ! この問題は TanStack Router 側でも報告されており(#4499)、v1.170.27 で修正されました(#8015)。 「対応の概要」のとおり、今後は、バージョンを上げて対応することも検討しています。 対処 毎回新しい関数を返すラッパーを作る useMatchRoute を直接使わず、次のラッパーフックを経由するようにしました。 import { useMatchRoute } from "@tanstack/react-router"; export function useReactiveMatchRoute() { "use no memo"; const matchRoute = useMatchRoute(); const reactiveMatchRoute: typeof matchRoute = (options) => matchRoute(options); return reactiveMatchRoute; } やっていることは、matchRoute を実行するだけの関数をレンダーのたびに新しく作って返すことだけです。 "use no memo" ディレクティブをつけることで、このラッパーフック自体が Compiler に最適化されるのを防いでいます。 "use no memo" は、関数単位で Compiler の変換を止めるディレクティブです。 これにより、URL が変わって useMatchRoute の購読によって再レンダーが走るたびに、呼び出し元には必ず前回と違う参照を持った新しい関数が返るようになります。 呼び出しは以下のように変更しました。 function NavEntryItem({ entry, params }: Props) { const matchRoute = useReactiveMatchRoute(); const isActive = (link: NavLink) => matchRoute({ to: link.to, params }) !== false; return <NavGroupItem entry={entry} isActive={isActive} />; } URLが変わると、元の useMatchRoute() の購読によって再レンダーが発生します。その際、ラッパーからは新しい関数が返ります。 その結果、呼び出し側のコンポーネントでは Compiler のキャッシュ判定 ($[0] !== matchRoute) が正しく外れるようになり、isActive の再計算も、<NavGroupItem> の再描画も正常に行われるようになりました。 挙動が変わったもの:終了アニメーション中の入力リセット 症状 各種フォームのダイアログにおいて、値を入力してからキャンセルでダイアログを閉じるアニメーションの中で、入力値が残ってしまっていてテストが落ちていました。 再度ダイアログを開くと残っていなかったため、ユーザーへの影響は少ないと判断しましたが、こちらも原因を調査して対処しました。 function AccountCreateDialog({ isOpen, onClose }: Props) { const { register, reset } = useForm({ defaultValues: { name: "" } }); const close = () => { onClose(); reset({ name: "" }); // ← ここでリセットしている }; return ( <Dialog isOpen={isOpen}> <input {...register("name")} /> <button onClick={close}>キャンセル</button> </Dialog> ); } 原因 原因は、React Compiler のキャッシュ、React Hook Form の reset() の仕組み、React の ref コールバックの仕組みの3つが組み合わさったことでした 実際のダイアログコンポーネントを React Compiler で変換すると、register("name") の呼び出しは次のようにキャッシュされていました(出力コードの抜粋)。 if ($[25] !== register) { // register の参照が前回と違う場合(初回レンダーなど) t12 = register("name"); // 関数を実際に呼び出して結果を取得 $[25] = register; // 今回の register の参照をキャッシュに保存 $[26] = t12; // 呼び出した結果をキャッシュに保存 } else { // register の参照が変わっていない場合 t12 = $[26]; // 関数を呼び出さず、前回の結果を使い回す } $: 今回はインデックス 25 の枠に register 関数の参照を、26 の枠にその実行結果が保存されていました。 t12: register("name") の実行結果(onChange や ref などが含まれたオブジェクト)が代入され、最終的に JSX 側で <input {...t12} /> のように展開されます。 Compiler は register 関数の参照が前回と同じなら、関数を呼び出さずに前回の結果を使い回す という最適化を行ってくれます。 この最適化が、React Hook Form の2つの仕様とぶつかりました。 React Hook Form が持つ2つの仕様 register 関数は常に同じ参照だが、呼んだ結果は毎回新しい useForm が返す register 関数は、最初のレンダーで一度だけ作られ、その後は再レンダーされても同じ参照のままです。 一方、register("name") を呼んだ結果(戻り値)は、呼ぶたびに新しく作られます。戻り値に含まれる ref コールバック(ref に渡す関数)が、毎回新しい関数になるためです。 今回の reset() は登録済みフィールドの参照を破棄する 今回は保持オプションを指定していないため、フォームの値をリセットするとともに、React Hook Form が内部で保持するフィールドの参照も破棄されます。 Reset the entire form state, fields reference, and subscriptions. (出典:React Hook Form 公式ドキュメント - reset) React Hook Form は、「直後の再レンダーで再び register("name") が呼ばれ、入力欄が登録し直されること」 を前提に動いています。 keepFieldsRef: true を指定すると、今回のリセットでフィールド参照を保持できます。後ほど、この方法も紹介します。 なぜ画面に古い値が残るのか Compiler が register("name") の呼び出しをスキップする register("name") の引数 "name" は固定値であり、register 関数の参照も再レンダー間で変わりません。 Compiler は「同じ関数を同じ引数で呼ぶなら結果も同じはず」と判断し、2回目以降のレンダーで register("name") の実行をスキップして前回の戻り値(ref コールバック等を含むオブジェクト)をそのまま使い回します。 React が ref コールバックを呼び直さない 同じ入力 DOM が維持されている再レンダーでは、ref コールバックの参照も同じなら、React は ref コールバックを呼び直しません。今回、終了アニメーション中はこの条件に当てはまり、入力欄の再登録と値の書き込みが行われませんでした。 React Hook Form が画面を更新できない React Hook Form は、この ref コールバックが実行されるタイミングで入力欄を内部に再登録し、reset() 後の値を DOM へ書き込んでいます。ref が呼ばれないため、DOM への書き込みも再登録も行われません。 その結果、終了アニメーション中の入力欄には編集した値が残りました。一方、閉じ切った後は入力 DOM がアンマウントされます。再表示時には同じ ref コールバックでも呼び出されるため、初期値が反映されました。 対処 今回のPRでは、従来の「キャンセル直後に入力をリセットする」挙動を維持するため、対象ダイアログを Compiler の変換対象から外しました。 register() と reset() を組み合わせて使うダイアログは、コンポーネントの関数本体の先頭に "use no memo" を書くように対処しました。 function AccountCreateDialog({ isOpen, onClose }: Props) { "use no memo"; const { register, reset } = useForm({ defaultValues: { name: "" } }); // ... } "use no memo" を指定したコンポーネントは、その内部で生成される JSX や関数(子コンポーネントへ渡す props など)も Compiler のキャッシュ対象外となります。 局所的に最適化の恩恵は受けられなくなりますが、これ以外の独立したコンポーネントは引き続き Compiler で最適化されます。 あわせて、「閉じてもう一度開いたあとも、入力と検証が正しく動く」ことを確かめるテストを追加し、キャンセル直後に入力欄が空になっているかだけでなく、フィールドが登録し直されているかまで確認するようにしました。 今回はダイアログがアンマウントされるため再表示時には正常に戻っていましたが、仮にページに残り続けるフォームだった場合、reset() 以降は React Hook Form と DOM の紐付けが完全に切れてしまい、送信(submit)や値の取得ができなくなるという、より深刻な不具合に発展していた可能性があります。 本来望ましい対処:keepFieldsRef 「対応の概要」で触れたとおり、今回の "use no memo" は一時的な対応です。 今回の挙動変化は、次の流れで起きていました。 reset() がフィールドの参照を破棄する そのため、次のレンダーで register("name") を呼んで登録し直す必要がある ところが Compiler が register("name") の呼び出しをスキップするため、登録し直されない "use no memo" は、3. の Compiler の変換を止めて、登録し直しが起きるようにする 対処です。 一方、1. の破棄そのものをやめて、登録し直しを不要にする 方法もあります。それが reset() の keepFieldsRef オプションです(v7.60.0 以降)。 reset(defaultValues, { keepFieldsRef: true }); このオプションを付けると、reset() は入力欄の登録を残したまま、登録済みの入力欄に直接値を書き込みます。 今回失敗した4つのダイアログでは、対応する reset() に keepFieldsRef: true を指定すると、"use no memo" を外してもキャンセル直後の入力値がリセットされました。 対処 よい点 気になる点 "use no memo" コンポーネントの先頭に1行足すだけ。React Hook Form の挙動を変えない そのコンポーネントは Compiler の最適化を受けられない keepFieldsRef: true Compiler の最適化を受けたまま直せる 対象フォームでフィールド参照を保持する設定が必要。条件によって表示・非表示が切り替わる入力欄がある場合は、影響の確認が必要 ! React 公式ドキュメントでは、"use no memo" は一時的な回避策であり、恒久的な解決策ではないとされています 恒久的な解決策ではなく、一時的なデバッグツールとして使用することを意図しています。 (公式ドキュメント)。 keepFieldsRef への移行や、React Hook Form 側の対応状況を見ながら、将来的には外していくのが望ましいと考えています。 壊れなかったけど先に手を打ったもの テストが落ちたり実際に画面が壊れたりしたわけではありませんが、React Compiler 導入にあたって、事前に手を打った(コードを変更した)箇所もあります。 特に、React Compiler の互換性のないライブラリとして、互換性が現在確認されていないライブラリについては、先行事例(React Compilerについて調べてみた、React Compiler導入で得られた効果と気をつけたいポイント)も参考にしながら、バグを踏む前に対策を入れています。 TanStack Table には "use no memo" を指定 TanStack Table(@tanstack/react-table)は、Compiler との互換性に問題があるライブラリとして名前が挙がっています。 そのため、DataTable コンポーネントには "use no memo" ディレクティブを指定し、安全のために Compiler の変換対象から外しました。これにより、このコンポーネント内で定義するカラム定義や配下の子コンポーネントへの props も Compiler の最適化対象外となりますが、 予期せぬバグを防ぐため現時点では互換性を優先しています。 なお、検証時点の TanStack Table は v8 です。公開時点では v9 がリリースされていますが、v9 と Compiler の組み合わせは今回検証していません。 RHF の watch() を useWatch() に変更 React Hook Form の watch() メソッドも、Compiler との組み合わせで問題が起きやすいことが公式から指摘されています。 そのため、フォームの値を購読してUIを切り替えるような箇所は、フックである useWatch() を使用する形へすべて書き換えました。 手動の useMemo の削除 CommandPalette などのコンポーネントで過去に手動で書いていた useMemo は削除しました。 これはバグ対策というより、「今後のメモ化は Compiler に任せる」という方針に基づいたコードの整理です。 まとめ React Compiler を導入し、一部の手動メモ化を Compiler に任せる形へ整理しました。しかし今回の検証で見えてきたように、外部ライブラリとの組み合わせでは、過剰なキャッシュが原因でバグを引き起こすことがあります。 今回は、Compiler の ON/OFF で挙動を比較し、変換結果とライブラリの実装を確認することで、必要な互換性対応を判断できました。 また、テストが失敗した場合も、どのタイミングで挙動が変わり、ユーザーの操作にどう影響するのかまで確かめることが大切だと感じました。 この記事での調査の過程や回避策のアイデアが、これから React Compiler をプロダクトに導入しようとしている方の参考になれば幸いです! 最後に AI Shiftではエンジニアの採用に力を入れています! 少しでも興味を持っていただけましたら、カジュアル面談でお話しませんか? (オンライン・19時以降の面談も可能です!) 【面談フォームはこちら】
Your Test Run Floods the Agent's Context. 9 Flags That Cut the Output It Reads - Book: LLM Observability Pocket Guide - Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go - My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools - Me: xgabriel.com | GitHub --- Your agent runs the test suite. One test fails. Hundreds pass, and the runner tells the agent about every one of them: a header, progress for each file, the names of passing tests, a summary, and somewhere in there the stack trace it actually needs. Test reporters were written for a person scrolling a terminal. A coding agent doesn't scroll. It reads what the command printed. The useful part can get cut. A command that fails reaches Claude Code inline only up to roughly 10,000 characters. Past that, Claude gets "a head-and-tail excerpt of that size", with no file path to the rest. A red test run exits non-zero, so it counts as a failure, and it is the run where the middle matters most. And whatever fits can stay. Claude Code "sends your full conversation with every request", so the names of passing tests can ride along on later turns until the conversation is compacted or cleared. You can't change how the agent reads. You can change what the runner prints. Measure what your agent reads first Run the suite the way an agent does: piped, not in an interactive terminal, with stderr merged into stdout. Then count. npx vitest run 2>&1 | wc -c npx vitest run --reporter=minimal 2>&1 | wc -c wc -c counts bytes. For plain ASCII output that is also characters, the unit Claude Code's limits use. I'm not printing numbers here, because they depend entirely on your suite: how many tests, how much they log, how deep the stack traces go. Run it on your project, before and after each flag below, and keep the flags that move your count. A small loop compares several variants in one go: for cmd in "pytest" "pytest -q" "pytest -q --tb=short"; do printf '%-24s' "$cmd" sh -c "$cmd" 2>&1 | wc -c done sh -c runs each string as a full command, flags included, in bash or zsh. Run this from your own shell, not inside an agent session. Vitest changes its reporter by itself when it detects an AI coding agent (flag 1), so measuring inside the agent hides the "before". JavaScript and Python: flags 1 to 4 1. Vitest: --reporter=minimal Vitest 4.1 added a reporter "to reduce ai agent token usage". The current reporters guide calls it minimal, with agent as an alias. It prints only failed tests and their error messages, and suppresses console logs from passing tests and the summary section. When no reporter is configured and Vitest detects an AI coding agent, it uses this one automatically. Pass it yourself when your config sets reporters, or when detection doesn't recognise your agent: npx vitest run --reporter=minimal If you want a progress signal, --reporter=dot prints "a single dot for each completed test", with details only for failures plus the summary. And the CLI's --silent=passed-only keeps console output from failing tests only. 2. Jest: --silent If your code logs while tests run, every one of those lines lands in the agent's context. Jest's CLI has one flag for that: --silent will "Prevent tests from printing messages through the console." npx jest --silent Jest has a default that bites agents: verbose is false, "or true if there is only one test file to run". The quick rerun of one file after a fix, the run agents do all the time, is the run that reports each test individually. 3. pytest: -q -q is pytest's "quiet - less verbose - mode". The flag to look for in your scripts and addopts is the opposite one: -v gives each test its own line, and -vv stops truncating assertion text entirely. 4. pytest: --tb=short The default, --tb=auto, prints "'long' tracebacks for the first and last entry". --tb=short is the "shorter traceback format", and --tb=line is "only one line per failure". Short keeps the file, the line and the failing assertion, which is usually what the agent needs to act. Line is often too little. pytest -q --tb=short One more from the same page: if addopts turns on --showlocals, every traceback carries the local variables. --no-showlocals hides them for that run. Go, Rust, PHP and Ruby: flags 5 to 8 5. Go: take -v off go test ./... is already quiet: a passing package prints only the final ok summary line. The -v flag logs all tests as they run, and will "Also print all text from Log and Logf calls even if the test succeeds." If a Makefile target or the agent adds -v, every passing test and every t.Log comes back. If you want a friendlier view, gotestsum wraps go test -json. Its default pkgname format prints one line per package, and dots prints one character per test. Flags for go test go after --: go test ./... gotestsum --format dots -- ./... pkgname grows with packages and dots grows with tests, so measure both on your repo. 6. Cargo: -q, on both sides of -- cargo test -q -- --quiet The first -q is Cargo's own and hides its log messages. Everything after -- goes to the test harness. There, libtest's --quiet "Displays one character per test instead of one line per test" and is an alias for --format=terse. The default, pretty, is one line per test. 7. PHPUnit: --no-progress PHPUnit's TextUI docs list --no-progress: "Disable output of test execution progress." Failures and the result summary still print. vendor/bin/phpunit --no-progress Two neighbours to avoid in an agent loop: --testdox will "Replace default result output with TestDox format", which reports each test by name, and --no-output will "Disable all output", which takes the failure details with it. 8. RSpec: --format progress RSpec defaults to the progress formatter: . for a pass, F for a failure, * for pending. The documentation formatter prints the text of every describe and it instead. Check the project's .rspec file: if it has --format documentation, change it to --format progress or delete the line. Flag 9: stop at the first failure Every runner above has a version of the same switch. When the agent is fixing one thing, the failures after the first are output it reads before it can act on any of them. | Runner | Stop early | What the docs say | |---|---|---| | Vitest | --bail=1 | stop after N failed tests | | Jest | --bail | N failing suites, default 1 | | pytest | -x | stop after first failure | | go test | -failfast | no new tests after one fails | | PHPUnit | --stop-on-failure | stop on first failure | | RSpec | --fail-fast | stop on first failed test | Sources: Vitest CLI, Jest CLI, pytest, go test, PHPUnit, RSpec. Cargo stops early at a coarser level: it exits after the first test executable fails, but libtest runs every test inside that executable. Libtest's own --fail-fast exists, and it needs -Z unstable-options. Put together, the fast loop looks like this: npx vitest run --reporter=minimal --bail=1 npx jest --silent --bail pytest -q --tb=short -x go test -failfast ./... cargo test -q -- --quiet vendor/bin/phpunit --no-progress --stop-on-failure bundle exec rspec --format progress --fail-fast Stopping early is for the loop, not the finish line. Before the agent says it's done, it should run the whole suite, quietly, and read every failure. Pin the commands, then watch the meter Flags help only if the agent uses them. Put the commands in the file your agent reads when a session starts. Claude Code loads CLAUDE.md "into context at session start". agents.md is the open format that Codex, Gemini CLI, Cursor, Aider and others read. ## Tests - While fixing: `pytest -q --tb=short -x tests/test_orders.py` - Before saying done: `pytest -q --tb=short` Claude Code's cost guide lists two heavier options too: a PreToolUse hook that rewrites test commands so only failures come back, and running tests in a subagent so "the verbose output stays in the subagent's context while only a summary returns". Then you want to see the effect. I build Hermes IDE, so weigh this part with that in mind. Hermes runs Claude Code, Codex and other CLI agents in real terminals, and version 2.3.4 added a context meter that shows how full the conversation is and offers Compact when it gets close to full. !Hermes IDE with a Claude Code session showing its context breakdown, per-session context gauges in the sidebar and a status strip with token total and estimated cost Hermes 2.3.4: Claude Code's /context breakdown after pressing the Context chip, per-session context gauges in the sidebar, and the token total and estimated cost in the status strip. Run the suite once with default flags and once with the quiet ones, and watch the meter between turns. Compact is there when you need it, in the Commands list above each session. !The Hermes action bar above a Claude Code session with the Commands list open, showing pinned Context, Model, Plan and Review and built-in commands such as Compact, Usage and Diff The action bar above a Claude Code session, with the Commands list open: pinned Context, Model, Plan and Review, then built-in commands such as Compact, Usage and Diff. Compacting is the fix after the fact, and not a free one: Claude Code's docs note that /compact "reads the conversation it summarizes, so compacting a large context is itself a large request." The flags above are the cheaper fix. Hermes is free, source-available under BSL 1.1, and runs on macOS, Windows and Linux. It isn't affiliated with Nous Research, Anthropic, OpenAI or any agent vendor. Download it here, and the code is on GitHub. --- If this was useful Counting bytes with wc -c is the crude version of a question that gets harder as a team grows: what did the model actually read, and what did that cost? The LLM Observability Pocket Guide compares the tracing and eval tools that answer it across a whole system, not one terminal. 
Vitest 5 advisory (Oct 16): Browser Mode with test.projects replaces Vite's default deny list, letting malicious sites read some .env values. Upgrade vitest and @vitest/browser to 5.0.5. Plain vitest run is unaffected. github.com/vitest-dev/vitest/security/advisories/…
Vitest 遅い件についてコアチームにベンチ送りつけて対応してもらっている vitest#11534
We migrated 70,000 tests from Jest to Vitest. Yes, in 2 months YOU got it right. We migrated 70,000 tests (reads: 70 thousand) from Jest to Vitest. The logical question would be: how? And here is where the story begins. I work as a software engineer in the Frontend Developer Experience team at Miro. Our ownership covers various parts of the platform. One of the major focuses for us is frontend infrastructure and tooling – everything that makes the product teams at Miro fast and productive. 2026 was indeed a pivotal year for us: we were finally able to kill Angular.js (I know, right), switched from Prettier to Oxfmt, got rid of ESLint in favor of Oxlint, we are running the CI typechecking with TypeScript 7 and much more. But look, you are here to read about unit-tests! In this article, I will cover how we migrated the main Miro frontend application unit testing harness from Jest to Vitest. The beginning The main Miro application is constantly evolving and sits now at ~6 million LoC. At this scale, the regular approaches and tooling stop functioning. How can I tell? Here are some facts about our CI state, unit-test jobs specifically: - We run 6 CI runners in parallel just for unit-tests. Each runner takes up to 20 minutes to finish - For Jest 30 the average speed is ~13 tests per second, even with SWC as the code parser - 25% of all runs are failing because of memory leaks, i.e. the CI runners are dying with OOM errors - We had to apply specific patches for Jest core packages to maintain a better worker pool (were never accepted upstream) - Migrating from Jest 29 to Jest 30 (yes, the first major release in 3 years) gave us only around 5% speed and memory improvements This does not look bright, and it has a big impact on the new releases for our application. But what could we do about that? Lurking through the Slack messages, I found a few threads where we actually discussed if we want to migrate away from Jest. Back in November 2023, one of my colleagues wrote: > Although I’ve spent some time also exploring an alternative (vitest) but I’ve found that in practice the extra set of vitest specific configurations and replicating our current build might be too risky, without knowing how much speed benefits we get now that we’re relying on @swc/jest At that moment, the scale of the problem was manageable – we just started to run the unit-tests in parallel on CI, compensating for the waiting time with the compute power from extra runners. Worked pretty well. In July 2024, another colleague of mine made a remark regarding the unit-test tooling one more time: > I expect higher returns from better test selection (e.g. not running 35k unit tests each time). The cost of switching to vitest (for example) is high and unclear what the improvement is Fair enough! 6 runners for 35,000 tests in parallel took just around 7 minutes each. With job durations like this, we had much bigger problems to solve, think of end-to-end tests, for example. The problem Fast-forward to March 2025. We started facing more and more issues with the Jest stack for testing. Let me give just one example that triggered me to explore the alternatives to Jest. Apparently, our engineers reported that with the VSCode Jest extension, multiple versions of fb-watchman (Jest’s in-house filesystem watcher library) were installed, which led to CPU saturation and, as a result, the inability to use their laptops. We figured out how to fix it with watchman: false. At that moment, Ahmed, a Principal engineer in the Frontend Developer Experience team, had enough: > I want to say maybe it's time to switch to vitest, but that's a massive effort… We had a conversation regarding this, but I had no capacity to lead the migration right away. We also saw a lot of traction regarding the AI models and how they rapidly evolve for more and more complex problems. All signs were to wait for a bit. And this is how we landed in 2026. Now you may ask: “Sergey, why did you decide to explore the migration again, considering all the facts you just mentioned?” Don’t get me wrong, we are not solving problems of this scale just because it’s cool. We need to understand what we want to achieve. I will start from the facts first: The main Miro application is a monorepo, that includes both web and native parts. It is managed by yarn and NX. At the beginning of 2026, we had 800+ packages in our codebase. The largest code share is located in the web core, which accounts for around 20% of the codebase. We also had ~70,000 unit-tests (doubled in 2 years, imagine that) and the unit-tests jobs were taking up to 20 minutes on average to finish, which made them the slowest CI jobs in our pipeline, even worse than building the application and running the e2e tests on the new version! Now, we can already see a few pain points we are fighting against. I can compile the whole list for you: 1. CI runners memory limit. 25% of the runs are failing because of memory leaks 2. Even with the SWC parser, Jest is slow, at 13 tests/sec for us (we use JSDOM and canvas mock for testing) 3. The TypeScript compatibility is not excellent. There are still API methods that infer the any type. 4. There is no proper ESM support, no mock hoisting. We add the spying code and mock factories between the top-level imports, so the import statements are downleveled to require() and can pick the mocked implementation. 5. Because of the previous point, we still transform the code to CommonJS. This means we are not testing what we are delivering to our customers (we deliver the ESM bundles). 6. The test coverage report is inaccurate. The packages do not share the instrumentation outside of each project root directory. This effectively means that there is missing cross-package coverage. 7. No test isolation. The jest workers in the pool are not spawned for each module; rather, they are recycled once a worker hits the memory limit. 8. There are no guardrails for the test fixtures to prevent leaking into the production bundle (funny spoiler: some teams actually used the test fixtures in the production code, intentionally) 9. Let's be real, it took Jest 3 years to release a new major version. We have opened a few issues on GitHub, but we did not see any traction around it. 10. Not only us, did I mention the incompatibility with other popular libraries? Prettier, for example. I can go on and on, but I think this is already impressive. Now, you cannot just take 70k tests and rewrite them with a random testing framework (imagine you could!). You also have to educate hundreds of product engineers, and the scale of the change would be massive. And also, how about all the custom plugins, matchers, and supporting dependencies? Think of jest-axe, jest-when, React Testing Library, and others. The answer to all of this is – we are looking for a Jest-compatible api. Compatible enough so we won’t turn our monorepo into chaos. Where to migrate? Now, the market research gave us two alternatives: Rstest and Vitest (versions as of March 2026): RStest v0.9.8 (rstest.rs) - Part of Rstack toolchain, so we can reuse the Rspack configuration from our application - Modern ESM-based syntax (think of import attributes and more explicit mocking syntax) - Integration with Rsdoctor for debugging Vitest v4.1.6 (vitest.dev) - Wide community adoption - Type safety and ESM-first - Test isolation and rich, comprehensive api You can guess Vitest won, but why? I did my homework: I created two branches and migrated around 10k tests to each of the frameworks. Then I tried to execute the tests in different modes. Here are the results (the values are tests per second, the bigger, the better): | | Jest | Vitest | Rstest | |:----------------------------|:-----------|:-----------|:----------| | Local run | 14.7 t/sec | 16.8 t/sec | 7.6 t/sec | | Local run with coverage | 5.9 t/sec | 8 t/sec | 2.9 t/sec | | CI run | 12.9 t/sec | 13 t/sec | OOM | | CI run with coverage | 5 t/sec | 8.6 t/sec | OOM | You can already notice that Rstest failed to run on CI, mainly because of bad memory management in the worker pool. Keep in mind that it was pre-1.0 version and the runner is under active development, so the hopes were low (I encourage you to try the latest version and explore the current state of things). In the meantime, migrating this small test subset showed a huge difference in memory consumption for Vitest – more than 10GB less memory used on the runner compared to the same group of tests in Jest 🚀. Just to be honest here, the speed and RAM consumption were indeed among the main factors. And although the research did not show any meaningful speed improvements, we decided to execute the migration as we saw opportunities to optimize Vitest configuration and try to make it faster. It also covers other pain points we were targeting to solve. Yes, we have a front-runner, and we are now coming to a plan! The plan As I said already, the Miro web application is a monorepo orchestrated by NX and Yarn. It combines the web core monolith, holding ~20% of all the code. Other parts are: 800+ packages and the part for native applications. And since the native part has its own CI and release pipeline, we decided to leave it out of scope. On a separate note, the application accepts 60+ merges into the main branch on a daily basis. And as much as I wanted to “just” ask the AI agents to migrate everything in one go, it was not possible for us to execute. I decided to go with a gradual rollout: 1. Run both Jest and Vitest on separate runners on CI in parallel. As the migration goes, reallocate the runners depending on the share of migrated tests 2. Start with packages. This way, we will be able to deliver the meaningful parts, and it is easy to discover which tests should go to Jest and which to Vitest. 3. Use AI agents to migrate the tests. We create a skill for the agents in the repo, so the agents have all necessary context. After each migration phase, we update our patterns not mentioned in the skill files. The last bit is the most powerful. Now, the agents learn and remember new patterns. This makes the migration faster and produces fewer hallucinations with each iteration. This also makes the migration more consistent – the agents apply the same patterns they saw before. Not only that, once completed, you will end up having a huge list of very specific code examples the agents found in the tests – an amazing source of insights to reflect on. It can easily open the room for follow-ups, good practices, and knowledge sharing. Act 1 - Infrastructure I started with infrastructure for Vitest. The basics include having the very first version of Vitest config to reuse in the application. Then we added the compatibility bridges. Think about shared mocks and helpers that create the fake data. During the time of migration, we do not want to duplicate such files. Hence, we added the shim globalThis.jest = vi in Vitest config. And globalThis.vi = jest in Jest. Yes, the api is not 100% compatible, but this was enough in our case. Now the whole idea of migration was built around a gradual switch package by package. Each package runs the scripts via the CLI commands, think of it as having a command miro-scripts test. It took me to add the runner’s opt-in argument, so we can distinguish which test runner to spawn. Looks like this: miro-scripts test --runner=vitest in the package.json file. It was not just a local story. This flag allowed us to see how to run these tests on CI runners. As we specialized the runners per tool, we combined all Vitest projects into one pool of runners, and the rest was for the remaining Jest projects. Finally, we adjusted the observability metrics in order to track the progress and test distribution. We added the extra runner label for test-related Prometheus metrics, which made it possible to visualize the migration. Act 2 - First steps Now we were all set to start the actual migration. All guardrails were there, the CI picked up all the tests, so we were not losing quality. The only missing part was the test coverage for Vitest, but we agreed to handle it after the transition. I started with small packages. This allowed us to keep the pull-requests size manageable. This way I could: 1. Review the changes more precisely (as I said, in the later stages, the built context was so rich that it could hardly produce off-track changes) 2. Form the foundation of the AI skill. First instructions, patterns, and polyfills were missing in the config 3. Convince teams and build trust. We, as a Frontend Developer Experience team, knew we needed to modernise the tech stack. But it was not obvious for the product teams. I made sure to be vocal about the migration and tried to iterate on the reasoning behind the change 4. Start collecting issues. For example, there were some extensions in VSCode that tried to guess the test framework used in the repo by the root-level config. With both Jest and Vitest configs at the root, it gave priority to Vitest, which at that time had only a fraction of packages with completed migration – just one of those things that I could only assume at the beginning. Once the first version of the AI skill was ready, I made sure to focus on the community work, presenting on it internally and explaining how each team could DIY their own migration. The skill is in the repo, just ask your agent to migrate the packages under your ownership! ✨ Act 3 - Goes well We started off strong with the migration. The first 90 packages went pretty well. At this moment, we built the needed Vitest plugins to cover parts of Jest configuration and added a decent number of global polyfills. I intentionally did not ask the AI agents to mirror the Jest config – nobody touched it for years, and it absorbed so much legacy, I did not want to port it to Vitest and started from scratch. I also tried to run the migration based on ownership, i.e. in a single pull-request I would only transform tests for the single owning team. Unfortunately, this did not work as expected. We have so many cross-package helpers and fixture generators, so it was never a single team, rather a few owners here and there. But that was still acceptable to us. At that point, I felt so confident that I decided to take one of the biggest packages in our repo (apart from the web monolith). ~600 test files all at once, a true battle test for the migration flow. BOOM, we landed it in the main branch! It was a huge milestone and proof we could continue. On another side, we kept working with the frontend community and were very transparent about the effort. As a result, we started receiving positive feedback about the progress in general and how smoothly the migration was going. Act 4 - It couldn’t be _that_ easy Although the mechanical migration was progressing well, our responsibility never changed – stable CI processes. Not only did we need to migrate the code, but we also needed to follow the metrics. Unfortunately, the metrics started showing a coming disaster. The CI runner started consuming more and more memory. I was confused! My initial research gave me the signal that Vitest consumed much LESS memory, but we were around 50% of the way through the migration, and it was getting only worse?! We had to pause everything until we figured it out. Imagine my mental state at that time: we were in the middle of a transition. I couldn’t roll everything back, but I also could not move on. What do I do now? Did I fail?! I was buried in the problem for days until I came to the realization: Each package in the repository is defined as a Vitest project (for simplicity, I will use the terms package and project interchangeably). Turns out, Vitest projects do not share the transformation cache! Every project builds up the cache with a unique identifier under the hood. If packages are heavily interconnected, Vitest spawns a Vite server for each package and transforms the same module graph again and again. So 10 linked packages will transform the same modules 10 times. Now scale it up to 800 packages, and you will get it. And in my initial research, I only used the subset of the tests inside a single Vitest project, thus, I could not actually see it coming. The solution is to run fewer projects, not by ignoring some of them, but rather by merging the projects into a single instance. Fortunately, most of the packages in our application have the default shared Vitest config with no overrides. So it was easy to write a project discovery script that would combine all packages with default configuration into a single project. By doing this, we were able to run only 60 Vitest projects that actually account for 400 packages (59 custom projects + 1 project that includes the remaining files). I will give you a sneak peek with the numbers: | Metric | Before | After | Delta | % Change | |:-------------------|:----------|:----------|:-----------|:---------| | Heap Memory | 8,770 MB | 5,024 MB | -3,746 MB | -42.71% | | Total Memory | 21,403 MB | 18,564 MB | -2,839 MB | -13.26% | | Total Duration | 1059.83s | 611.06s | -448.77s | -42.34% | | Transform Time | 10756.03s | 4327.69s | -6428.34s | -59.76% | | Setup Time | 1495.36s | 1277.27s | -218.09s | -14.58% | | Import Time | 13452.26s | 6984.71s | -6467.55s | -48.08% | | Tests Time | 257.29s | 262.76s | +5.47s | +2.13% | | Environment Time | 344.18s | 371.06s | +26.88s | +7.81% | Curious readers may ask: why did Environment time and Test time increase? Imagine you run 10 Vitest workers in parallel. And only a single main process that runs Vite to transform the import tree. Before the change, the workers would just hang and wait when the main process transforms the same modules again and again. Once we merged the projects, the transformation time dropped significantly. Now the workers do not wait, they run more tests, which causes CPU saturation and system pressure in general. This effectively results in slower worker performance. But since the actual test run takes only a fraction of the overall test execution process, the degradation is negligible. That being said, combining all standard projects into a single instance made an outstanding impact on the performance and confirmed my assumption. The most important thing is that we can now continue with migration! Act 5 – Closer to the goal The next 60 pull-requests were just mechanical. The AI agent has built up the comprehensive skill context. My job was to check the bad-looking and non-trivial changes. Then also hear the feedback from the reviewers, since I cannot in this world know each and every unit-test in the Miro application. One nice cherry on top was the fact that with the Jest → Vitest migration, we started noticing some test fixtures that had accidentally leaked into our production bundle (remember point 8?). The logical question would be: how? How did we detect this, and how did the test data leak into the production bundle? The first part was easy for us – we build the web application with Rspack with no overrides or patches in the config to support Node.js modules. In fact, Vitest uses the NodeJS api under the hood (it is a Node library in the end). As a result, Rspack would fail to build the application once any part of Vitest is imported in the dependency graph. How does it usually happen for us? 1. Barrel files. Every package in the repository exports its API via the barrel file index.ts. Although it would work fine in small projects, barrel files give nothing but pain as the application gets bigger. Barrel files, and especially star exports (export * from ‘./foo.ts’) are hard to tree-shake, and you never know what exactly is being exported. Accidentally, some test shapes were exported transitively as well. 2. Some teams actually used test helpers to create artificial users and accounts in the production code. These were edge cases but still happened. The Vitest migration clearly highlighted such cases and made us improve the codebase. This gave us approximately a 4MB cut from the full production build. Act 6 - One-shotting the web core We had migrated all packages at that point, and the AI agents now had a definitive playbook on how to run the migration. Now it was time for the final boss: the web core monolith. 1,800 test files, 16,500 tests. I realized it was too risky to run the migration during business hours – engineers merge pull-requests once every 10 minutes. I would spend more time resolving the conflicts. I decided to handle it during the weekend (no worries, I took two days off to compensate!) On Saturday, I told the AI agent to run the migration. The result: 1,985 files changed, +22,776 / -19,998 lines diff. This pull-request tagged each and every team for review, but my plan was to merge it right away with an admin override approval. It wasn’t perfect, but it was decent enough. The rest we could fix later with smaller PRs during business hours. It took me an entire day to look at the changelog! I clicked the Bypass rules and merge button and waited for the main branch CI to complete. I could finally go to sleep. We made it. Act 7 – Aftermath breakthrough I don’t want to give the impression that the transition was perfectly smooth. Once we completed the mechanical work, we had to clean up the configs, revert the hybrid setup, and start removing Jest dependencies. But this is fine. The bigger problem (ironically) was that Vitest is a much more reliable tool than Jest. It has its own timeout for module imports, it catches unhandled errors, it races the duplicated vi.mock blocks. All of this actually made a small subset of tests more flaky. We spent some time stabilizing the tests, exploring the actual cause, and working with the owners to see how we could address it. A fractional chunk of tests (think of 27 test cases) was completely skipped for the sake of CI stability. But one question still stood out in my mind – we were able to speed up the run and save memory by merging the projects. Could we merge everything into a single Vitest project? One of the advantages of the transition to Vitest is that we were able to drop tons of legacy configurations. This made all custom Vitest configs look like this: import { defineProject, mergeConfig } from 'vitest/config'; import defaultCfg from '@shared/config/vitest/shared.config.js'; export default mergeConfig( defaultCfg, defineProject({ test: { setupFiles: ['./setupVitest.ts'] } }) ); Only the setup script is what makes us create the custom configs. Why does that matter? If the setup file is the only difference, maybe I can detect which test file is running and call the setup script relative to that file! There is no documentation around it. The only hint I got is this page, which explains the interface for the expect api. Apparently, expect.getState: () => MatcherState, and then MatcherState has the field testPath - the absolute test path, which is exactly what we are looking for. Now I quickly spun up the shared setup script, which takes the test path, derives the package the test is located in. and then dynamically executes the setup script attached to that specific test file. Here is the idea: import { existsSync } from 'node:fs'; import { relative, resolve } from 'node:path'; import { expect } from 'vitest'; const repoRootDir = '%repo_root_path%'; const absoluteTestPath = expect.getState().testPath; const relativeTestPath = absoluteTestPath ? relative(REPO_ROOT_DIR, absoluteTestPath) : ''; // you can manipulate the path to find the right location here const vitestProjectRootDir = fetchProjectDirForTestFile(relativeTestPath); const setupFile = vitestProjectRootDir ? resolve(repoRootDir, vitestProjectRootDir, 'setupVitest.ts') : ''; if (setupFile && existsSync(setupFile)) { await import(setupFile); } You can make this script project-agnostic and execute it for each test file. Doing this, we were able to merge ALL 800+ Vitest projects in our application into a single test run. No transformation overhead, no cache duplications. To give you an idea of how impactful it was, I will give you a sample: | Metric | Before | After | Delta | % Change | |:-------------------|:--------------|:------------|:---------------|:---------| | Heap Memory | 5,238 MB | 1,857 MB | -3,381 MB | -65% | | Total Memory | 22,460 MB | 17,450 MB | -5,010 MB | -22.3% | | Total Duration | 865.64s | 588.82s | -276,82s | -32% | | Test speed | 12.6 t/sec | 19 t/sec | +6.4 t/sec | +50% | Finally, we found the source of the speed ✨ And this concluded our road to landing Vitest in our main Frontend application. Results | Active migration time | About 80 days (May 4 to July 20) | |:----------------------|:----------------------------------| | Merged pull requests | 97 | | Lines added/removed | +87,701 / -78,946 | | Net lines | +8,755 | | Files touched | 9,328 | Notice we have added almost 9,000 net lines of code. The actual net is much lower, because we moved from globally injected api (think jest, it, describe, etc.) to locally imported objects (import {describe} from ‘vitest’). Plus, code formatting gave some diff. Now to the technical details: - The test runs are 50% faster than they were with Jest (13 t/sec to 19 t/sec) - Vitest consumes 30% less memory on the CI runners - Vitest mocks are type-safe i.e. they inherit the signature from the mocked object, allowing engineers to write better unit-tests. - ESM-first. The mocks are hoisted automatically, which gives again the better experience for the engineers. - Production bundle saved around 4MB, the test fixtures were pulled out from the production barrel files - Real test isolation – each test file is being run on an isolated worker in the pool, ensuring tests do not interfere with each other - Accurate test coverage. allowExternal: true allows us to collect the test coverage, dropping the project boundary. We collect the full picture. - We collected the test polyfills from all files, generalised them, and stored them in a shared configuration. Not only did it simplify the tests, but we also made sure engineers have a consistent experience with the polyfilled api’s - The migration AI skill was transformed into the unit-tests creation skill. Now it gives the agents the common patterns from our codebase, which enabled better test generation The total model usage during this period cost around $7,000. To put this into perspective, I estimated the annual savings from the migration on CI runners at approximately $75,000. We should also account for more application releases: as I mentioned at the beginning, we started facing more and more memory issues for Jest runs. Add here the faster feedback loop during the local development as well, which is, to be fair, hard to estimate. All of this made the investments a huge win for us. Lessons learned In 2026 here at Miro, we keep experimenting with large-scale migrations. Our goal is to provide engineers with the best-in-class tooling and infrastructure. We have already made a transition from Prettier to Oxfmt, from ESLint to Oxlint, from Webpack to Rspack. It was just about time to switch from Jest to Vitest. Each journey results in lots of artifacts for future improvements. And lessons learned along the way. Here are some of them: 1. Set a clear goal. Be specific about what you want to accomplish. Research the market and find the tools that can move you towards the goal. Experiment with a subset of the codebase to be able to estimate the scale and feasibility, and to get a feel for the planned work. 2. Plan the execution. Sometimes you just need to make a big bang migration. Sometimes you can be more gentle and have a gradual rollout. For Vitest, I made a safe bet to iterate and build up the context for the AI agents. It worked pretty well, and at some point we were able to one-shot the remaining tests. 3. Be specific in the prompts with AI agents. Give a deterministic way to reproduce the result. Set the constraints. It was not possible to run the subagents in parallel because Vitest takes so much memory in our case. I asked to run the migration sequentially, so it did not ruin my workstation. 4. Build trust. People resist change. My goal was to give the reasoning behind the change, acknowledge the struggle to understand where to use which runner, and help to adopt the new tooling. 5. You can not plan for everything. There are always issues you could not possibly expect during the preparation. You need to accept it and not be afraid to pivot and look up for the solution. 6. The scale of change does not matter at this point. With Fable 5, or Sol, AI can execute any volume of change. If you can formalise what needs to be done, if you can create a context for the agent, if you can give a deterministic reproduction – the implementation part is extremely democratised. 7. Trust the CI. Some changes during the migration were trivial. Others touched the production logic. I worked with the code owners to understand if the change I am introducing is legit. But at the end of the day, you have to build the release quality gates that are trustworthy, and a rollback process so fast, so even if you end up having a bad day, you can easily fix the mistake. To wrap it up We have completed the migration of our main frontend application from Jest to Vitest. The transition took 70,000 tests to transform (75,000 as of the last merged pull-request date). It took us just a bit over 2 months to execute. We used Opus 4.8 and Fable 5 (once it was released) to drive the migration. Migration to Vitest gave us space and resources to grow further, modernised our codebase, and allowed us to start writing type-safe unit-tests. It also caught the test data leaks into the production bundle and actual production bugs. This project paid off in all areas we were targeting. It also built a story of how useful the AI agents can be if done right. Special credits go to Evgeny Evsyukhin, who has heavily contributed to the migration as well as the observability infrastructure around Vitest. I want to also mention Ivan Voronin, Ahmed El Gabri, Frederico Estrela, Vinicius Kiatkoski Neves, and my beloved Frontend Developer Experience team for making the migration possible. I also want to give a shoutout to Ben Makuh for proofreading and helping me refine this article. PS: Since we created this Jest to Vitest migration skill, I thought it would be nice to create an open-source version as well. It can be a good foundation for your own contextual migration skill. Feel free to use and contribute here.