Show HN: I built a tool that finds reliability bugs in your code
Hey HN,
I’m Abhishek. I'm building Opslane, a tool that finds reliability issues in your code. On every PR, it spins up your entire app and then tries to find weak spots that would break in production.
Demo: youtu.be
Example: We tested Opslane on Lago (open-source billing) and found a bug where customers get double charged. Opslane made Stripe take the money and then return a 500. Lago marked the payment failed. A retry from the dashboard charged the customer again. lago-api#6903
Why I built this
I used to work on reliability at Robinhood. We used to run chaos testing in our staging environments: inject faults in various parts of the system and see what breaks. We found issues like Kafka or Redis being unavailable for a few seconds could take the whole app down.
But that kind of testing was expensive. It needed a well maintained staging environment. Engineers needed to know what to test. Most teams never actually did it.
Over the last year, coding agents have gone writing some of my code to almost all my code now. Teams are shipping way more code than they were 6 months ago.
Code review tools read the diff, but a lot of reliability bugs aren't in the diff. They're in how your code behaves when Stripe, Redis or the network misbehaves. And e2e tests usually mock those services out and test the happy path.
Agents have made this type of testing a lot cheaper. And we will see more of this (kinda like security scanning / testing)
How it works
A full copy of your app per PR: Postgres, Redis, Kafka, SQS, S3.
Working copies of your third-party dependencies: Stripe, Slack, Gmail and more. Traffic goes through our proxy, which lets us inject failures.
The agent picks what to test: It reads every PR and writes down invariants that need to hold (for e.g. a customer should never be charged twice). It then injects faults likely to break the invariants.
Bugs we've found
We ran Opslane on some popular open-source apps and here are some of the bugs we have found:
1/ Twenty: deleting a workspace while Stripe is failing locks everyone out, and billing keeps going. Twenty merged a fix. (twenty#28043)
2/ Dub: a restart in the middle of a payout paid a partner $240 for $120 owed. (dub.link#4707)
Full list: opslane.com/open-source
Opslane is pretty early - we have a couple of early customers. We have found bugs in their system that their other code review tools missed.
Would love to get feedback from folks and how they are seeing the reliability of their systems evolve over time with agents.
6 points
Mentions
Show HN: I built a tool that finds reliability bugs in your code Hey HN, I’m Abhishek. I'm building Opslane, a tool that finds reliability issues in your code. On every PR, it spins up your entire app and then tries to find weak spots that would break in production. Demo: youtu.be Example: We tested Opslane on Lago (open-source billing) and found a bug where customers get double charged. Opslane made Stripe take the money and then return a 500. Lago marked the payment failed. A retry from the dashboard charged the customer again. lago-api#6903 Why I built this I used to work on reliability at Robinhood. We used to run chaos testing in our staging environments: inject faults in various parts of the system and see what breaks. We found issues like Kafka or Redis being unavailable for a few seconds could take the whole app down. But that kind of testing was expensive. It needed a well maintained staging environment. Engineers needed to know what to test. Most teams never actually did it. Over the last year, coding agents have gone writing some of my code to almost all my code now. Teams are shipping way more code than they were 6 months ago. Code review tools read the diff, but a lot of reliability bugs aren't in the diff. They're in how your code behaves when Stripe, Redis or the network misbehaves. And e2e tests usually mock those services out and test the happy path. Agents have made this type of testing a lot cheaper. And we will see more of this (kinda like security scanning / testing) How it works A full copy of your app per PR: Postgres, Redis, Kafka, SQS, S3. Working copies of your third-party dependencies: Stripe, Slack, Gmail and more. Traffic goes through our proxy, which lets us inject failures. The agent picks what to test: It reads every PR and writes down invariants that need to hold (for e.g. a customer should never be charged twice). It then injects faults likely to break the invariants. Bugs we've found We ran Opslane on some popular open-source apps and here are some of the bugs we have found: 1/ Twenty: deleting a workspace while Stripe is failing locks everyone out, and billing keeps going. Twenty merged a fix. (twenty#28043) 2/ Dub: a restart in the middle of a payout paid a partner $240 for $120 owed. (dub.link#4707) Full list: opslane.com/open-source Opslane is pretty early - we have a couple of early customers. We have found bugs in their system that their other code review tools missed. Would love to get feedback from folks and how they are seeing the reliability of their systems evolve over time with agents.