keploy

接口一改，回归测试全靠手补？ Keploy 把真实 API 流量、数据库和队列交互录下来，直接回放成测试和 mock，应用代码一行不用改。 适合接口多、依赖杂的后端团队，staging 那套麻烦能省就省了。 你现在的回归测试，是录出来的还是手写的？ #API测试 #开源工具 @grok keploy/keploy

Regression Testing Vs Integration Testing: Key Differences Here is a situation most backend teams have lived through. A developer changes how the user service formats dates. It is a tiny change: one line, a cleaner ISO format, every unit test still green. The pull request gets approved in five minutes. Two days later, the billing service starts failing to generate invoices, because it was still expecting the old date format from the user service. Two kinds of testing could have caught that, and they are the subject of this post. Integration testing would have checked that the user service and the billing service still understand each other. Regression testing would have re-run that check after the "tiny" change, before it shipped. They sound like competitors. They are not. Once you see what each one actually answers, it becomes clear that you want both, and that they work best together. The short answer Integration testing checks that separate components, such as services, databases and external APIs, work correctly together. Regression testing checks that features which already worked still work after a change. Integration testing is about what you test: the connections between parts. Regression testing is about when and why you test: after every change, to make sure nothing that used to work is now broken. That is also why the two overlap so much. Many of the tests in a good regression suite are integration tests. What integration testing actually checks Integration testing looks at the seams of your system. A unit test might confirm that a function formats a date correctly. An integration test confirms that when the user service sends that date to the billing service, the billing service can read it and do its job. Typical integration tests check things like: - an API writing the right data to its database - one service calling another with the right payload and handling the response - your code talking to a third-party API, a message queue or a cache - what happens when a dependency is slow, returns an error or sends unexpected data You usually write integration tests when you build or change a connection between components. If you want the full picture, including types and examples, our integration testing guide goes deeper. What regression testing actually checks Regression testing asks a simpler, slightly paranoid question: did we just break something that used to work? "Regression" literally means going backwards. A regression bug is a feature that worked yesterday and does not work today, usually because of a change that looked unrelated. New features, bug fixes, refactors, dependency upgrades and config changes can all cause one. A regression test is not a special kind of test. It is any existing test that you re-run after a change. Unit tests, integration tests, API tests and end-to-end tests can all be part of a regression suite. What makes them "regression" is the purpose: protecting behaviour that already exists. For the full background, see our guide on what regression testing is, and the different types of regression testing teams use. [caption id="attachment_8592" align="aligncenter" width="1200"]<img class="size-full wp-image-8592" src="wp.keploy.io" alt="Diagram showing integration testing as checking the connections between services, and regression testing as re-running tests after each release" width="1200" height="680" /> Integration testing is about what you test. Regression testing is about when you test it again.[/caption] Regression testing vs integration testing at a glance | | Integration testing | Regression testing | |---|---|---| | Question it answers | Do these components work together? | Does everything that worked before still work? | | What defines it | Its scope: the connections between components | Its purpose: re-checking after a change | | When you create the tests | When a connection between components is built or changed | Usually reuses tests that already exist | | When it runs | During development, often on every pull request | After every change, before every release | | Scope | Interfaces, contracts and data flow | Can cover anything: units, integrations, full user flows | | Typical bugs found | Wrong payloads, broken contracts, mismatched data formats, timeouts | Features broken by changes elsewhere, side effects of fixes and refactors | | What makes it hard | Setting up dependencies and test environments | Keeping a growing suite fast and up to date | | Automation | Strongly recommended | Practically required, because the suite runs so often | Back to the date format bug Let us replay the story from the start, this time with both kinds of tests in place. When the user and billing services were first connected, someone wrote an integration test: create a user, generate an invoice, check that the invoice has the right customer name and date. It passed, and it stayed in the test suite. Months later, the developer makes the "tiny" date format change. The CI pipeline runs the full regression suite, which includes that old integration test. The test fails, because the billing service can no longer parse the date. The bug is caught on the pull request, not in production. Notice what happened. The integration test is what knew how to detect the problem. The regression run is what made sure the check happened at the right moment. Neither one alone would have been enough: an integration test that nobody re-runs protects nothing, and a regression suite with no integration tests would have missed a bug that only exists between two services. How they work together in a real pipeline In most teams, the relationship looks like this: 1. You write integration tests while building a feature, to confirm the new connections work. 2. Once they pass, those tests join the regression suite, next to unit tests and a few end-to-end tests. 3. Every pull request runs the regression suite. Fast tests run first, slower ones after. 4. Before a release, a wider regression run covers the critical user flows. So the question is rarely "regression testing or integration testing?". It is usually "which integration tests belong in our regression suite, and how do we keep that suite fast?". Our guide to automated regression testing covers how to keep a growing suite under control, and CI/CD testing covers where it fits in the pipeline. [caption id="attachment_8593" align="aligncenter" width="1200"]<img class="size-full wp-image-8593" src="wp.keploy.io" alt="Diagram of a regression suite made of unit tests, integration tests and end-to-end tests, re-run on every pull request and before every release" width="1200" height="620" /> Integration tests become part of the regression suite once they pass.[/caption] When you need more integration testing You probably need more integration tests if: - most of your production bugs show up between services, not inside them - you have recently split a monolith into services - you depend on third-party APIs that change without much warning - your unit tests are green but staging keeps breaking When you need more regression testing You probably need a stronger regression process if: - fixing one bug regularly creates another one somewhere else - releases feel risky, so the team ships less often - refactoring is avoided because "nobody knows what it will break" - tests exist, but they are not run automatically on every change If several of these sound familiar, the fix is usually the same: automate the tests you already have and run them on every pull request. A real example: one recording, both jobs Here is where it gets practical. With Keploy, the same recorded tests can act as integration tests and as your regression suite, without writing either by hand. Using Keploy's Express and MongoDB sample app, you start the app through Keploy and use it like a normal client: keploy record -c "docker compose up" --container-name "nodeMongoApp" --build-delay 50 curl --request POST \ --url http://localhost:8000/students \ --header 'content-type: application/json' \ --data '{"name":"John Do","email":"john@xyiz.com","phone":"0123456799"}' Keploy saves the API request and response as a test case, and the MongoDB calls behind it as mocks. That recorded test is already an integration test: it checks that the API and the database work together to produce the right response. Now, every time someone changes the code, you replay the recordings: keploy test -c "docker compose up" --container-name "nodeMongoApp" --build-delay 50 --delay 10 That replay is your regression test. If a change alters the response or the way the app talks to MongoDB, the test fails and shows you the difference. Add the same command to your CI pipeline and every pull request is checked against real, previously working behaviour. Setup for other languages and frameworks is in the Keploy docs. Common mistakes Treating them as either/or. Teams sometimes say "we do regression testing, so we don't need integration tests". If the regression suite has no tests that cross service boundaries, it cannot catch bugs that live there. Writing integration tests and never re-running them. A test that only ran once, when the feature was built, gives no protection against the next change. Letting the regression suite grow until nobody runs it. If the full suite takes an hour, people stop waiting for it. Split it: fast unit and integration tests on every pull request, the slower end-to-end tests before release. Relying only on end-to-end tests for regression. They are slow and flaky, and when they fail they rarely tell you where the problem is. Integration tests point much closer to the cause. Tools that help with both - Keploy: records real API traffic into test cases and mocks, then replays them on every change, so the same tests cover integration and regression. - Postman: API checks that can be re-run as a regression collection. - Testcontainers: real databases and services in containers for integration tests. - Selenium and Playwright: end-to-end regression checks through the UI. - Jenkins, GitHub Actions and GitLab CI: run the whole suite automatically on every change. For longer lists, see our roundups of regression testing tools and integration testing tools. Related comparisons If you are sorting out testing terms, these comparisons are worth reading next: - Unit testing vs regression testing - Integration testing vs functional testing - Smoke testing vs regression testing Conclusion Integration testing and regression testing answer different questions. Integration testing asks whether your components work together. Regression testing asks whether your latest change broke something that used to work. In practice, they meet in the middle: your integration tests become part of your regression suite, and running that suite on every change is what stops "tiny" changes from turning into production incidents. If you want both without writing hundreds of tests by hand, try Keploy for API testing and turn real traffic into tests you can replay on every pull request. FAQs What is the difference between regression testing and integration testing? Integration testing checks that separate components work correctly together. Regression testing checks that existing features still work after a change. Integration testing is defined by its scope, the connections between components, while regression testing is defined by its purpose, re-checking after changes. Is integration testing a type of regression testing? No, but integration tests are often part of a regression suite. An integration test becomes a regression test when you re-run it after a change to make sure the connection it covers still works. Which comes first, integration testing or regression testing? Integration testing usually comes first, while a feature is being built. Regression testing comes after changes are made, and it re-runs existing tests, including those integration tests, to catch anything that broke. Can regression testing be done at the integration level? Yes. Integration regression testing means re-running integration tests after a change, to make sure components still work together. It is one of the most effective kinds of regression testing for microservices and API-heavy systems. What is the difference between unit, integration and regression testing? Unit testing checks one function or class in isolation. Integration testing checks that connected components work together. Regression testing re-runs existing tests, at any of these levels, after a change to make sure nothing that worked before is now broken. Should regression tests be automated? Yes, in almost every case. A regression suite runs after every change, so running it by hand quickly becomes too slow and too easy to skip. Automated regression tests in a CI/CD pipeline catch problems on the pull request, before they reach users.

Bounding CI Cost Without Weakening Trust in Keploy Pull Requests Bounding CI Cost Without Weakening Trust in Keploy Pull Requests It's 2:47 AM and your GitHub Actions budget notification fires: 90% consumed with eleven days left in the billing cycle. You check the console and find forty-three overlapping CI runs—someone pushed a quick typo fix eight times in fifteen minutes, and every push triggered a full matrix build with no cancellation policy. Each job compiles Go packages, runs the test suite, exercises privileged recording infrastructure, and uploads artifacts. None of them will finish before the next push arrives and starts the cycle again. That incident reveals two independent failures: there was no concept of superseded work, and different trust boundaries were collapsed into a single expensive job. Fixing only one leaves the other intact. For keploy/keploy adoption, where production-sandbox testing involves compiling Go, testing application behavior, and exercising privileged recording or replay infrastructure, those boundaries matter. Putting them in one oversized workflow makes cancellation expensive and failures difficult to interpret. The configuration below is an adoption baseline—not a claim about Keploy's current workflows or measured coverage. It assumes a root Go module and requires validation against the repository's actual toolchain and test prerequisites before becoming a required check. The Failure Is Usually Multiplication Consider a representative scenario: a maintainer adds a third-party reporting action by mutable tag. Later, that tag resolves to compromised code. Meanwhile, contributors repeatedly push fixes, creating overlapping matrix runs with no concurrency policy. These failures compound. The compromised action executes wherever it is referenced; redundant runs multiply both exposure and runner consumption. A workflow timeout limits each job, but it does not stop twenty obsolete jobs from starting. The root causes are independent: dependency identity was mutable, and scheduling had no concept of superseded work. For Keploy adoption, I separate the architecture into three layers: an unprivileged pull-request gate, isolated Linux integration execution, and trusted publishing. Only the first belongs below. Recording, replay, privileged networking, production-derived artifacts, and release credentials stay outside this gate. One Workflow, With an Embedded Validation Script Save this as .github/workflows/pr-go.yml. The single configuration creates a Bash script so local validation and CI policy have an explicit, auditable implementation. Once adopted, move that script into a tracked file without changing its behavior. name: PR Go gate on: pull_request: push: branches: [main] permissions: contents: read concurrency: group: go-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }} cancel-in-progress: true jobs: validate: name: Go / ${{ matrix.os }} runs-on: ${{ matrix.os }} timeout-minutes: 20 strategy: fail-fast: true max-parallel: 2 matrix: os: [ubuntu-22.04, ubuntu-24.04] env: MIN_COVERAGE: "60" steps: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - uses: actions/setup-go@0a12ed9d6a96ab950c8f026ed9f722fe0da7ef32 # v5 with: go-version-file: go.mod cache: false - name: Locate Go caches id: paths shell: bash run: | echo "mod=$(go env GOMODCACHE)" >> "$GITHUB_OUTPUT" echo "build=$(go env GOCACHE)" >> "$GITHUB_OUTPUT" echo "version=$(go env GOVERSION)" >> "$GITHUB_OUTPUT" - uses: actions/cache@0400d5f644dc74513175e3cd8d07132dd4860809 # v4.2.4 with: path: | ${{ steps.paths.outputs.mod }} ${{ steps.paths.outputs.build }} key: go-${{ runner.os }}-${{ runner.arch }}-${{ matrix.os }}-${{ steps.paths.outputs.version }}-${{ hashFiles('go.sum') }} - name: Validate source shell: bash run: | cat > "$RUNNER_TEMP/validate.sh" <<'BASH' #!/usr/bin/env bash set -euo pipefail mapfile -d '' files < <(git ls-files -z '*.go') if ((${#files[@]})); then unformatted="$(gofmt -l "${files[@]}")" if [[ -n "$unformatted" ]]; then printf 'Unformatted Go files:\n%s\n' "$unformatted" exit 1 fi fi go mod download go mod verify go vet ./... go build ./... go test -race -count=1 -timeout=8m \ -covermode=atomic -coverprofile=coverage.out ./... actual="$(go tool cover -func=coverage.out | awk '$1 == "total:" {gsub(/%/, "", $3); print $3}')" awk -v actual="$actual" -v minimum="$MIN_COVERAGE" ' BEGIN { printf "Coverage: %s%%; required: %s%%\n", actual, minimum exit !(actual != "" && actual + 0 >= minimum + 0) }' BASH bash "$RUNNER_TEMP/validate.sh" What This Boundary Actually Guarantees The matrix checks two Linux environments without inventing an unsupported Go-version range; go.mod supplies the toolchain request. Confirm that both runner images support the repository's required native libraries. Go's toolchain selection can still download a newer toolchain requested by the module. The script enforces formatting, static analysis, compilation with type checking, race-enabled tests, and a coverage floor. Sixty percent is an example policy, not Keploy's reported coverage. Measure the baseline before enabling branch protection. Package-default coverage also does not prove that replay paths or failure recovery received meaningful exercise. Tests requiring privileges or external services will fail here. Classify those tests explicitly using the repository's supported package selection or build tags; do not quietly replace the command with a narrower suite and retain a misleading "full validation" label. The Trade-Offs Worth Keeping Exact action SHAs prevent a moving tag from silently changing execution. They do not certify the pinned code or freeze every dependency it downloads. Review updates, retain version comments for maintainability, and audit transitive execution separately. The cache key includes the operating system, architecture, runner image, Go version, and dependency checksum file. Omitting broad restore prefixes sacrifices some cache hits for simpler provenance. Cache entries accelerate work; checksum verification and successful validation remain necessary. Concurrency cancellation keeps only the newest run for a pull request. max-parallel bounds this matrix, not the entire repository's runner consumption. Separate pull requests still run concurrently, and cancellation is not instantaneous. Organization-level budgets and runner limits remain necessary. Finally, keep publishing out of this workflow. Read-only permissions and disabled persisted Git credentials reduce exposure, but pull-request tests still execute contributor-controlled code. Avoid pull_request_target for that execution path, and never assume a self-hosted runner is safe simply because the token is read-only. For Keploy, this boundary preserves a useful distinction: cheap source feedback can be aggressive and disposable; privileged integration evidence must be isolated and deliberate. The Hardest Trade-Off The configuration above makes one deliberate sacrifice: it prefers cache simplicity over hit rate. A more aggressive restore strategy could reuse module caches across Go version bumps or checksum updates, improving cold-start latency at the cost of provenance clarity. That trade-off is testable—measure your cache hit rate and typical download overhead, then decide whether the complexity is worth the seconds saved. What's your tolerance for cache complexity versus build speed? Have you seen a setup where aggressive fallback keys caused more confusion than they prevented, or the opposite—where conservative caching created unnecessary pain during dependency churn?

From Mexico: Keploy turns real API traffic into tests and data mocks. Run your app with keploy record and it captures API calls, DB queries and streams via eBPF, then replays them as tests. No code changes. AI expands API coverage. 18873 stars. keploy/keploy t.co

keploy ⭐ 19k Keploy generates unit tests for your code using LLMs, aimed at developers who want automated test coverage. 🔗 keploy/keploy #opensource t.co

I followed Keploy's quickstart as a first-time user. Here's where it broke. --- title: I followed Keploy's quickstart as a first-time user. Here's where it broke. published: true tags: go, testing, devops, wsl --- A couple of months ago I came across a product called Keploy on LinkedIn. I went to their site to read about it, and the pitch stopped me: it writes your integration tests for you by watching your app run. I went like -- !impressive image gif And it did that by Not generating them from your code. Watching actual traffic. You run your app, use it normally, and Keploy records what came in and what went out. Later it replays those requests against your app and checks it still responds the same way. I wanted to see whether that held up, !curious/mischevious image gif so I ran one of their Go quickstarts end to end on Windows through WSL. This is what it was like. <br> How it works Integration tests usually mean keeping a database alive, writing mocks by hand, and updating those mocks every time a schema changes. It's a lot of upkeep for tests that mostly verify things you already know work. Keploy's approach is to record once and replay forever. It supports several languages — I used one of their Go quickstarts because that's what I wanted to try. When you start a recording, Keploy moves your app to another port and takes the original one for itself, so every request passes through it. At the same time, an eBPF probe watches the syscalls your app makes to the database. !hide and seek image gif RECORD your curl -> Keploy (:8082) -> app (:35555) -> Postgres | | test case mock REPLAY Keploy -> replays recorded request -> app -X- Postgres | answered from mocks So one recording run captures two things: the HTTP exchange becomes a test case, and the database exchange becomes a mock. On replay, your app starts and runs for real — your router matches the path, your handler executes, your code does its work. Only the database calls are faked, answered from the recording. Postgres never gets touched. Because the probe sits outside your code rather than inside it, you don't import a library or change a line of your app. That's also why Keploy needs sudo, and why it only runs on Linux — hence WSL on Windows. <br> What I built Their quickstart docs offer several Go samples. I picked Echo + Postgres — a URL shortener. The setup is: install the CLI, start Postgres in Docker, build the app, and run it under keploy record. Then you just use the API. I made two curl calls — a POST creating a short URL, and a GET following it. Both became test cases, written to keploy/test-set-1/, with the Postgres exchanges saved alongside them as mocks. A few small things didn't match as I went. The docs site, its screenshots, and the binary I installed were on three different versions. The README pointed at a line number that had moved — and the quickstart pointed at a different wrong one. A flag the docs use had been removed. The quickstart skips a login step that everything after it depends on, so the first failure is a 401 timeout that never mentions authentication. None of these are dramatic on their own, but following instructions literally means hitting each one and stopping to work out why. Then I hit a real one. !sad and shocked gif <br> What I'd fixed first, and what actually broke I ran the test command straight from the docs. Keploy started my app, and then the run died. The errors talked about the app being unreachable and the database connection failing. So I went looking at Postgres, at ports, at the binary — all of it fine. The real cause was a folder I'd never opened. The sample repo ships with a test-set-0 already committed, recorded by someone else a long time ago. My own recording had created test-set-1. Without naming a test set, Keploy picked up both and started with theirs. Its mocks were in an older format that the current version can't read — eighteen failed to unmarshal errors. With no mocks loaded, nothing answered my app's startup database ping, so the app exited. And with the app gone, the whole run collapsed. My two tests were never reached, let alone run. --test-sets test-set-1 One flag, and it skips theirs entirely. Then a smaller oddity: with that flag everything passed, but the summary still reported "across 2 test-set(s)" and threw a red error about a missing report for test-set-0 — a set that never ran. The filter controls which tests execute, but not what gets counted. <br> What convinced me !here you go image gif COMPLETE TESTRUN SUMMARY. Total tests: 2 Total test passed: 2 Total test failed: 0 Total time taken: "10.09 s" And higher up in the log: {"testsRan": 2, "mocksConsumed": 4} Four mocks consumed. My app pinged the database, created its table, and ran both queries — and Postgres answered none of it. Every one of those calls came from the recording. That's the whole idea in a single line. Real code running, real database work happening, no database. Where it fits Keploy is most useful when you already have a working service and want regression coverage without maintaining a pile of database mocks by hand. The recorded traffic gives you a practical starting point — but it doesn't replace deliberately testing paths you haven't exercised. If you never send a request through a particular part of your app, Keploy can't record a test for it. !here you go image gif I wrote up the full walkthrough here — every version mismatch and workaround, in the order you'll hit them. If you're going down the same path, it should save you some time.

Keploy – an open-source API and integration testing tool

“Real Developers Test in Production.” But Should They? !test-in-prod-meme You've probably heard this joke before. It pokes fun at developers who skip testing, push code straight to production, and let real users discover the bugs. But there's a reason the joke exists. Software that works on a developer's machine can behave very differently when real requests, databases, and multiple services get involved. A broken update endpoint could corrupt customer data. An unhandled error could take down a critical feature. A bad deployment could turn a routine release into hours of debugging, incident response, and frustrated users. And fixing the code is only part of the problem. Someone has to identify what broke, reproduce the issue, assess its impact, roll back or deploy a fix, and verify that the same bug doesn't return. Depending on the organization, that work can involve developers, QA engineers, SREs, and incident responders. So, should developers test in production? Their are different types of Production testing which are done like A/B testing, Canary Releases, Feature Flags, Shadow Testing, Smoke testing etc. But these testings are done for a specific purpose and that is to test the application in a production environment. But these testings are not enough to catch all the bugs that our existing tests miss before they affect real users. Let's build a small application, break it deliberately, explore the testing techniques that help prevent these failures, and see how Keploy can turn recorded API interactions into reusable regression tests so we dont have to write the test cases and guess what scenarios to test. Sample Application Setup We have a sample Bookmark application with Go, Echo and PostgreSQL. It has all the CRUD operations for bookmarks along with a simple UI to interact with the application. You need: - Go - Docker - Keploy CLI Clone and run this sample application locally using: git clone https://github.com/Namanv0509/bookmark.git && cd bookmark go mod download To start the application run: docker compose up --build -d Check the health of endpoint: curl http://localhost:8080/health Run the following command to stop it: docker compose down Let's Break Something First We'll use bookmark, a simple bookmark manager built with Go, Echo, and PostgreSQL. Users can save bookmarks, add notes and tags, view their saved links, update them, and delete them. The application has a small browser interface and a REST API. Start the application locally and Open http://localhost:8080 and create a bookmark. You can also interact with the API directly. !save-bookmark curl -i -X POST http://localhost:8080/links \ -H "Content-Type: application/json" \ -d '{ "url": "https://www.google.com/", "title": "Google Search", "notes": "Sounds like a good search engine. It helps me find good blogs like this.", "tags": ["Search Engine"] }' The application should save the bookmark and return its details. Now let's imagine we're making a small improvement to the update endpoint. We change its SQL query, test that the application starts, and deploy the new version without checking whether the update behavior still works. Here's the mistake we introduce: UPDATE links SET url = $1, title = $3, notes = $2, tags = $4, updated_at = NOW() WHERE id = $5; The title and notes parameters are swapped. We intended to assign the title from $2 and the notes from $3, but accidentally did the opposite. Let's send an update request with distinct values to see the effect. !bug-bookmark curl -i -X PUT http://localhost:8080/links/1 \ -H "Content-Type: application/json" \ -d '{ "url": "https://www.google.com/", "title": "Title", "notes": "Notes", "tags": ["updated"] }' The endpoint still return a successful response. The application is running and everything looks fine. Yet the bookmark's title can now become notes. We've introduced a bug without necessarily causing a crash. Imagine the same mistake in a production application. Users might save incorrect information, downstream services might consume the corrupted data, and the problem might go unnoticed until someone reports it. Let's imagine a different mistake: a database connection failure that causes requests to time out. That could make a feature unavailable, potentially affecting every user who relies on it. These are different failure modes, but they reveal the same problem: a successful deployment doesn't guarantee correct behavior. What Does “Testing in Production” Actually Mean? The phrase gets used to describe several different practices, but they aren't interchangeable. Skipping tests and deploying directly This is the reckless interpretation of the joke: code goes straight to production without sufficient verification. It might appear faster initially, but failures can become expensive. Developers may have to investigate unfamiliar production states, QA teams may need to reproduce issues, and operations teams may have to mitigate an outage. Testing in a staging environment !Staging A staging environment is a separate environment intended to resemble production. Teams can deploy a release candidate there and run tests before exposing it to users. Staging is useful, but it isn't a perfect copy of production. Data volumes, configurations, integrations, and traffic patterns can differ. Testing in production deliberately Some teams perform controlled tests in production because certain conditions are difficult to reproduce elsewhere. For example: - Canary releases send a small portion of real traffic to a new version before a wider rollout. - Feature flags let teams enable or disable features for selected users and gradually expand access. - A/B testing compares different versions of a feature with different user groups to measure outcomes such as engagement or conversion. - Shadow testing mirrors real requests to a candidate version so its behavior can be evaluated without using its responses for normal user-facing results.results. The Different Layers of Software Testing Different tests answer different questions. A practical testing strategy combines them rather than relying on a single tool. | Type | Question it answers | bookmark example | | --- | --- | --- | | Unit testing | Does this piece of logic work? | Does URL validation reject ftp:// URLs? | | Integration testing | Do components work together? | Can the API save a bookmark in PostgreSQL? | | API testing | Does the HTTP interface behave correctly? | Does an invalid request return the expected status and response? | | End-to-end testing | Does a complete user workflow work? | Can a user create a bookmark through the browser and see it in the list? | | Regression testing | Did a change break existing behavior? | Does updating a bookmark still preserve its title and notes? | | Load testing | How does the application behave under traffic? | Can the list endpoint handle concurrent requests? | These categories can overlap. For example, an API test that exercises the actual database is also an integration test. Regression testing describes the goal of detecting unintended changes, not a particular test level. Let's look at what each approach gives us. Unit Testing: Check the Small Pieces First A unit test checks a small, isolated piece of code. For our bookmark application, we want to verify that the validation logic accepts valid HTTP URLs, rejects unsupported schemes, and requires a title. The test cases are already available in main_test.go. Run the following commands from your project directory: go test ./... go test -v ./... go test -cover ./... The first command runs the tests, the second displays individual test results, and the third reports code coverage. You should see output similar to this: === RUN TestValidateInput === RUN TestValidateInput/valid_input === RUN TestValidateInput/missing_title === RUN TestValidateInput/unsupported_scheme === RUN TestValidateInput/missing_URL --- PASS: TestValidateInput (0.00s) PASS ok linkvault 0.110s All four test cases pass, confirming that our validation logic handles these inputs as expected. However, our overall code coverage is only 7% because we've tested just a small part of the application. Unit tests help us verify individual pieces of logic, but they don't tell us whether our HTTP handlers and SQL queries work correctly together. For that, we need to test how the application interacts with its dependencies. Dependencies: What Does Your Application Rely On? A dependency is something your code relies on to perform its job. bookmark has dependencies at several levels: - Echo handles HTTP routing and middleware. - A PostgreSQL driver lets Go communicate with the database. - PostgreSQL stores bookmark data. - The frontend depends on the API returning data in the expected format. Some dependencies are libraries. Others are external systems that must be running and correctly configured. This distinction matters when testing. Our validation function can run without PostgreSQL, but creating a bookmark requires a database connection, a valid schema, and a working SQL query. When a test fails, we want to know whether our own logic is wrong or a dependency is unavailable or behaving differently than expected. One way to investigate this is to test with the real dependency. Another is to replace it with a test double. Integration and API Testing: Bring the Pieces Together Let's return to the running application. An integration test can send a request to bookmark, let the actual handler execute its SQL query, and verify that PostgreSQL stores the expected data. An API test focuses on the behavior exposed through HTTP. We might check that: - Creating a valid bookmark returns the expected response. - An invalid URL produces a validation error. - Retrieving an unknown ID returns the appropriate not-found response. - Updating a bookmark changes its fields correctly. - Deleting a bookmark removes it from subsequent results. For example: curl -i -X POST http://localhost:8080/links \ -H "Content-Type: application/json" \ -d '{ "url": "https://keploy.io", "title": "Keploy", "notes": "API testing tutorial", "tags": ["testing"] }' Then inspect the result: curl http://localhost:8080/links These requests exercise more of the real application than our isolated unit tests. The problem is that manually repeating these requests after every code change quickly becomes tedious. We also need to ensure the same scenarios are tested consistently. Regression Testing How do we know a code change hasn't broken behavior that already worked? That's where regression testing comes in. It helps us verify that existing functionality continues to behave as expected after changes to the code. Their are severl tools we can use for it like: - Keploy - Diffy - Pact For our demo, we'll use Keploy to record API interactions and replay them after code changes. This lets us catch unexpected changes in application behavior without manually writing every test case. Keploy can also mock supported dependency interactions during replay, reducing the need to run every dependency. 1. Record the API interactions Start Keploy's recording workflow: keploy record -c "docker compose up --build" \ --container-name "linkvault-app" \ --build-delay 60 Your Keploy logs should look something like this in keploy-logs.txt 🐰 Keploy: 2026-10-22T12:00:59.071063+05:30 INFO Starting Keploy {"version": "3.8.79", "apiServerURL": "https://api.keploy.io", "inDocker": false} 🐰 Keploy: 2026-10-22T12:01:01.555659+05:30 INFO AppName in config (bookmark) does not match current directory name (linkvault). using current directory name as appName 🐰 Keploy: 2026-10-22T12:01:01.556065+05:30 INFO agent replay posture resolved {"replaying": false, "cmd": "record", "mode": "", "daemonset": false} 🐰 Keploy: 2026-10-22T12:01:01.577954+05:30 INFO Running application using a temporary Keploy-generated Docker Compose file (will be cleaned up automatically) {"cmd": "docker compose -f ./docker-compose-tmp.yaml up --abort-on-container-exit --exit-code-from app --build", "composePath": "docker-compose-tmp.yaml"} 🐰 Keploy: 2026-10-22T12:01:01.578072+05:30 INFO Waiting for keploy-agent to be ready for docker compose... {"Agent-uri": "http://localhost:61321/agent"} 🐰 Keploy: 2026-10-22T12:01:01.654583+05:30 INFO Starting Application: {"executing_cmd": "/bin/sh -c docker compose -f ./docker-compose-tmp.yaml up --abort-on-container-exit --exit-code-from app --build"} 🐰 Keploy: 2026-10-22T12:01:03.650059+05:30 INFO Keploy agent is ready to record test cases and mocks. With recording active, open http://localhost:8080 and exercise the application: - Create a bookmark. - Retrieve the bookmark list and an individual bookmark. - Update its title and notes. - Delete a bookmark. !update-bookmark We just performed an update to bookmark 🐰 Keploy: 2026-10-22T12:20:36.086862+05:30 INFO 🟠 Keploy has captured test cases for the user's application. {"path": "/Users/namanverma/Downloads/linkvault/keploy/test-set-0/tests", "testcase name": "post-links-1"} 🐰 Keploy: 2026-10-22T12:20:36.089196+05:30 INFO 🟠 Keploy has captured test cases for the user's application. {"path": "/Users/namanverma/Downloads/linkvault/keploy/test-set-0/tests", "testcase name": "get-links-1"} 🐰 Keploy: 2026-10-22T12:30:01.330101+05:30 INFO 🟠 Keploy has captured test cases for the user's application. {"path": "/Users/namanverma/Downloads/linkvault/keploy/test-set-0/tests", "testcase name": "post-links-2"} 🐰 Keploy: 2026-10-22T12:30:01.333557+05:30 INFO 🟠 Keploy has captured test cases for the user's application. {"path": "/Users/namanverma/Downloads/linkvault/keploy/test-set-0/tests", "testcase name": "get-links-2"} Use distinct values for the title and notes so that incorrect updates are easier to detect. Stop recording with Ctrl+C. Keploy captures the interactions and generates test cases according to the configured setup. keploy can be configured to mask senstitive information and handle noise. 2. Introduce a regression Let's deliberately introduce a bug in the update handler's HTTP response. In main.go, locate the end of updateLink(). Just before return c.JSON(200, link) add the following line: // Intentional regression: swap title and notes in the response. link.Title, link.Notes = link.Notes, link.Title return c.JSON(200, link) This simulates a response-mapping bug: the API returns the title as the notes and the notes as the title. 3. Replay the captured tests With the regression in place, run: keploy test -c "docker compose up --build" \ --container-name "linkvault-app" \ --build-delay 60 \ --delay 10 > Heads up: non-deterministic fields. Fields like created_at, updated_at (set by NOW()), and auto-generated IDs change on every run, keploy detects them and mark them as noise fields. Keploy replays the captured requests against the rebuilt application and compares the resulting HTTP responses with the recorded responses. Because the values differ, Keploy should flag the response mismatch, provided those fields are included in its response comparison. and for sure you can see that in the test summary as well <=========================================> COMPLETE TESTRUN SUMMARY. Total tests: 2 Total test passed: 1 Total test failed: 1 Total time taken: "28.14 s" Test Suite Name Total Test Passed Failed Time Taken Failed Testcases "test-set-0" 2 1 1 "28.14 s" <=========================================> Your summary can vary depending on what and how many requests you have sent and your endpoint configs We never had to write the test cases and we could catch the regression by replaying our traffic recorded previously. We can automate this testing with CI/CD, we will cover that in next article. 🐰 Conclusion There's no single testing technique that catches every bug. Understanding when to use each approach helps us build more reliable software and catch problems before they reach production. In the upcoming articles, we'll dive deeper into different types of testing, how they work, and when to use them. If you'd like to explore further, here are two useful resources: - Microsoft Engineering Playbook — Automated Testing - Keploy Blog — Technology Until next time, see you in the next one!

Keploy: API tests faster than unit tests, from user traffic (18.6k stars) What it does All text below is quoted directly from the project README. >The must-have tool for developers in the AI-Gen era for 90% test coverage >Keploy is a developer-centric API and integration testing tool that auto-generates tests and data-mocks faster than unit tests. >It records API calls, database queries, and streaming events, then replays them as tests. Under the hood, Keploy uses eBPF to capture traffic at the network layer, but for you it's completely code-less and language-agnostic. Key Highlights: - No code changes: Just run your app with keploy record. Real API + integration flows are automatically captured as tests and mocks. (Keploy uses eBPF under the hood to capture traffic, so you don't need to add any SDKs or modify code.) - Record and Replay complex Flows: Keploy can record and replay complex, distributed API flows as mocks and stubs. It's like having a very light-weight time machine for your tests, saving you tons of time! - Complete Infra-Virtualization (beyond HTTP mocks): Unlike tools that only mock HTTP endpoints, Keploy records databases (Postgres, MySQL, MongoDB), streaming/queues (Kafka, RabbitMQ), external APIs, and more. It replays them deterministically so you can run tests without re-provisioning infra. - Combined Test Coverage: If you're a developer, you probably care about statement and branch coverage, Keploy calculates that for you. If you're a QA, you focus more on API schema and business use-case coverage, Keploy calculates that too. This way coverage isn't subjective anymore. - Expand API Coverage using AI: Keploy uses existing recordings, Swagger/OpenAPI Schema to find: boundary values, missing/extra fields, wrong types, out-of-order sequences, retries/timeouts. This helps expand API Schema, Statement, and Branch Coverage. Other Capabilities: - CI/CD Integration: Run tests with mocks anywhere you like, locally on the CLI, in your CI pipeline (Jenkins, Github Actions..) , or even across a Kubernetes cluster. - Multi-Purpose Mocks: You can also use Keploy-generated Mocks, as server Tests! - Reporting: Unified reports for API, integration, unit, and e2e coverage with insights directly in your CI or PRs. - Console: A developer-friendly console to view, manage, and debug recorded tests and mocks. - Time Freezing: Deterministically replay tests by freezing system time during execution. - Mock Registry: Centralized registry to manage, reuse, and version mocks across teams and environments. >Because Keploy intercepts at the network layer (eBPF), it works with any language, framework, or runtime, no SDK required. Repository keploy/keploy Repository stats |Metric|Value| |:-|:-| |Stars|18,575| |Forks|2,411| |Last release|v3.6.126 (2026-10-14)| |Languages|Go 99.7%| |Contributors|146| |AI contributor accounts (Claude / Codex / Cursor / Copilot)|claude (96 commits), Copilot (22)| |AI co-authored commits|Claude: 96, Copilot: 24| Main contributor |Field|Value| |:-|:-| |Name|Shubham Jain| |GitHub|@slayerjain| |Commits to this repo|570 (top contributor)| |Bio|Helping developers release at light speed!| |Company / Location|@keploy / New Delhi| |Followers|107| |Public repos|107| |On GitHub since|2015-06-10| ----------------------------------------------------------------------------------------------- Discover the best GitHub repos without the endless scrolling. Get top repos organized by category (AI, DevOps, Databases & more), plus key takeaways from developers' most useful comments - all in one free, twice-weekly newsletter. 👉 Get the best repos & community insights - Free