nuclei

OpenCode+Docker+GitHub Actionsで全自動AIペネトレーションテストはじめました はじめに Zennチームのdyoshikawaです。 ZennではAIによるセキュリティチェックの仕組みを急ピッチで構築しました。現在はPull Request（以下PR）ごとのAIレビューに加えて、AIエージェントによる全自動のペネトレーションテストを毎週GitHub Actionsで回しています。 背景は、みなさんもご存知のとおり昨今のサイバー攻撃の激化です。 scan.netsecurity.ne.jp 攻撃する側もAIをフル活用しはじめているといわれており、脆弱性が見つかってから悪用されるまでの時間が非常に短くなっています。防御側としては、人間が定期的に診断する従来のやり方では攻撃に追いつかない時代になってきました。 本記事では、ZennでどのようにAIセキュリティレビューと自動AIペネトレーションテストを回しているかを紹介します。特別な製品は使っておらず、GitHub ActionsとDocker、OSSのツールを組み合わせて作っています。何かお役に立つものがあれば幸いです。 まずはPR単位のセキュアコードレビューを整備 最初に取り組んだのは、PRごとのAIレビューです。 PRがReady for reviewになると、GitHub ActionsがAIエージェントを起動します。エージェントは差分と周辺のコードを読み、コード品質に加えてセキュリティとコンプライアンスの観点でレビューします。結果はPRのコメントとして投稿されます。 AIによるPRレビューのコメントの例 モデルが非常に低コストなので、1回のレビューにつき$0.01〜$0.1程度で済みます。 エージェントにはOpenCodeを使い、セキュリティレビューの観点はSkillとしてリポジトリに置いています。手元のコーディングエージェントからも同じSkillを呼び出せるので、PRを出す前にClaude Codeなど手元のツールでローカルからレビューすることもできます。 なお、PRレビューのエージェントも後述するペネトレーションテストと同じDockerのサンドボックスとネットワーク隔離の中で動かしています。 また、レビュー中にWeb検索が必要な場面では、OpenAIのWeb検索ツールをAPI経由で使っています。OpenCodeにも組み込みのWeb検索ツールがあり、ExaまたはParallelを使ってAPIキーなしで検索できます。 opencode.ai ただ、Exaのプライバシーポリシーには、送信された検索クエリをモデルの学習やファインチューニングを含む製品の改善に使うと書かれています。検索語にはレビュー中の文脈が入りうるので、学習に使われることは避けたいところです。そこで、組み込みのWeb検索は使わず、OpenAIのAPIを使うようにしています。 exa.ai こうしたWeb検索のAPIを呼び出すCLIを、webseekとしてOSSにもしています。OpenAIのResponses APIの web_search などを、複数のプロバイダーで共通のインターフェースから呼び出せるツールです。検索ごとに使ったトークン数と推定コストも出力されます。 dyoshikawa/webseek サンドボックスとWeb検索の制限は、どちらも詳しくは後述します。 レビューの過程と結果をGitHub上に残したかった 外部のAIレビューサービスを使う選択肢もありました。それでも自前で作ったのは、レビューの過程と結果をGitHub上に残したかったためです。 結果はPRのコメントとして残るので、後から「このPRで何が指摘されたか」を追えます。実行のログはGitHub Actionsに残るので、どのような観点でどのファイルを読んだのかも確認できます。レビューの観点もSkillとしてリポジトリで管理しているので、観点の変更そのものをPRでレビューできます。 AIはレビュー結果をファイルに書くだけで、コメントの投稿はGitHub Actionsのstepが行います。AIのコンテナにはGitHubのトークンを渡していません。この「AIには書かせるだけ、外への操作は決まった処理で行う」という形は、後述のペネトレーションテストでもそのまま使っています。 Fireworksのオープンウェイトモデルを活用 モデルはFireworks AI上のオープンウェイトモデルを使っています。PRのたびにレビューを回すので、1回あたりのコストはできるだけ下げたいところです。 モデルの選び方やコストについては、以前の記事で詳しく紹介しています。 zenn.dev 実際のコストは、PRコメントや週次検査のIssueの末尾に毎回載せています。たとえば、ある週の全体を対象にしたペネトレーションテストでは、1億以上のトークンを使って、OpenCodeのコストは$3.02でした。コストだけの話でいうと、その気になれば毎日でも回せそうです（人間の対応が追いつくかは別として……）。 OpenCodeのトークン使用量とコスト Fireworks AIは明示的なオプトインなしに入力データをモデルの学習に使わないとしています。レビューではソースコードを送るので、ここは導入前に確認しておきたいポイントです。 さらにAIによる全自動ペネトレーションテストの構築へ PRレビューが回るようになったので、次は実際にアプリを動かして攻撃を試すペネトレーションテストに取り組みました。 毎週、GitHub Actionsの1つのジョブの中で上の流れを実行しています。AIが動くのは試験の部分だけで、依存関係の監査、アプリの起動、テストデータの準備、Issueへの集約は決まった処理で行います。 試験の範囲として、直近の変更を中心に見る「差分」と全体を見る「全体」の2種類を用意しています。 なぜAIによるペネトレーションテストが必要か PRレビューは差分を読むので、変更の中に入った問題を見つけるのは得意です。一方で、実際に動かさないとわからない問題もあります。 代表的なのは認可の不備です。たとえば「有料のChapterを買っていない人が本文を取得できないか」「Publicationを抜けた人が非公開の内容を見られないか」といった問題は、実際にユーザーを用意してリクエストを送らないと確かめにくいです。複数の変更が組み合わさって初めて起きる問題や、レスポンスヘッダーやリダイレクトのようにコードを読むだけでは気づきにくい問題もあります。 こうした問題は、外部の診断を定期的に受けるのが一般的だと思います。ただ、毎週のように受けるのは現実的ではありません。そこで、AIエージェントに実際にHTTPリクエストを送らせ、毎週自動で試験することにしました。 AIエージェントには試験用のユーザーとデータの組み合わせを順に試させています。OpenCodeの上で動くAIエージェントは、次のツールを使えます。 curlでは、ユーザーごとの認可を1件ずつ確認します。 nucleiでは、Zenn向けに自作したテンプレートで、開発用APIの露出やセキュリティヘッダーの不足などを確認します。 projectdiscovery/nuclei OWASP ZAPでは、spiderとactive scanで広く確認します。 zaproxy.org Playwrightでは、ヘッドレスブラウザでXSSやopen redirectを確認します。 playwright.dev ペネトレーションテストでもオープンウェイトモデルを使う ペネトレーションテストのAIエージェントにも、Fireworks AI上のオープンウェイトモデルを使っています。コストのほかにもう1つ理由があります。米国のクローズドモデルと比べて、セーフガードがゆるいことです。 ペネトレーションテストでは、AIエージェントに攻撃者と同じ操作をさせます。米国のクローズドモデルでは、こうした操作を断られることがあります。実際、このペネトレーションテストの仕組みを作る作業そのものを、手元のClaude Codeに断られたことがありました。 だからこそ攻撃する側に使われているのだと思いますが、防御する側も使わない手はありません。 Dockerでサンドボックスとネットワーク隔離を実現 AIエージェントに攻撃者役をやらせるとなると、エージェント自身が暴走したときのことを考える必要があります。認証情報を持ち出す、テスト対象以外を攻撃する、といったケースです。 プロンプトで「テスト対象以外にリクエストしないでください」と指示することはできます。ですが、プロンプトの指示はあくまでお願いです。そこで、DockerでAIが到達できる宛先そのものを絞ることにしました。 コンテナごとにネットワークだけを持つコンテナを用意してnetwork namespaceを共有させ、そのコンテナのiptablesでデフォルト拒否にしたうえで必要な宛先とポートだけを許可することで、ネットワーク境界を作っています。AIエージェントから外への通信はAIゲートウェイのコンテナを経由する経路だけで、その先も許可したAIプロバイダーのAPIに限っています。以下はイメージを伝えるために簡略化した例です。 services: ai-agent-network: image: firewall cap_add: [NET_ADMIN] # firewallを設定するコンテナだけが持つ healthcheck: test: ["CMD", "test", "-f", "/run/network-ready"] ai-agent: image: ai-agent network_mode: service:ai-agent-network cap_drop: [ALL] security_opt: ["no-new-privileges:true"] depends_on: ai-agent-network: condition: service_healthy 実際に処理を行うコンテナにはcapabilityを渡さないので、AIエージェントが自分でfirewallを書き換えることはできません。 AIエージェントが通信できる先は、試験対象のローカル起動のAppサーバーとAIゲートウェイだけです。AIゲートウェイは、LLMのAPIへのリクエストを中継する、Node.jsで書いた小さなプロキシです。LLMのAPIキーはゲートウェイだけが持ち、AIエージェントにはダミーの値を渡しています。DBに直接つないだり、外部にデータを送ったりはできません。 Web検索もこのゲートウェイを経由します。OpenAI APIのWeb検索ツールだけを許可し、検索語にURLや長いトークンのような文字列が含まれる場合は拒否しています。コードや内部の情報を検索語に含めないよう、プロンプトでも指示しています。 このサンドボックスとネットワーク隔離は、前述のPRレビューでも共通です。PRレビューではアプリを起動しないので、AIエージェントが通信できる先はAIゲートウェイだけになります。 ほかにも、エージェントが実行しようとするコマンドをAIの分類器で審査しています。こちらは後ほど詳しく紹介します。 GCSやBigQueryはOSSエミュレータを活用 試験対象は本番環境ではなく、GitHub Actionsのrunner上に立ち上げたZennです。 ZennのbackendはGoogle Cloud StorageやBigQuery、Stripeなどの外部サービスとつながっています。これらはOSSのエミュレーターに置き換えています。 fsouza/fake-gcs-server goccy/bigquery-emulator stripe/stripe-mock こうすることで本物の外部サービスに一切つながない状態で試験できます。AIが有料のAPIを叩いてしまう心配もありません。 もともとローカルの開発環境で使っていたものなので、そのほかにペネトレーションテスト用に改修したものはほとんどありません。開発環境をDockerで整えているプロダクトなら同じように流用できると思います。 サンドボックスとAIの分類器で多重にガード ここまで、Dockerのサンドボックスとネットワーク隔離の話をしてきました。ただ、（以前の自分であればそういう考え方自体しなかったのですが）サンドボックスが絶対に安全とも言い切れなくなってきました。 2026年7月、OpenAIのモデルが評価用のサンドボックスを脱出したという事例が公表されました。 openai.com 今回の構成でも設定の漏れやミドルウェアの未知の脆弱性が絶対にないとは言い切れません。 そこで、サンドボックス環境の整備だけでなく、AIの分類器によるチェックも用いた多重ガードの仕組みにしています。Claude CodeのAuto modeやCodex CLIのAuto-reviewのようなものであるとお考えください。 claude.com alignment.openai.com ZennのAIレビューとペネトレーションテストではOpenCodeを使っています。OpenCodeはベンダーフリーなAIコーディングエージェントであるため、オープンウェイトモデルの使用と相性が良いと感じています。 OpenCodeには --auto という起動オプションがありますが、自分が調べた範囲ではこれはClaude Codeの --dangerously-skip-permissions 相当の設定でした。現状、分類器を使った自動承認の仕組みはOpenCode本体にないと思われます。幸い、OpenCodeにはプラグイン機構があるため、自動承認プラグインを自作することで対応できました。これは別途OSSとしても公開しています。 dyoshikawa/opencode-auto-approval-plugin 分類器にJev/Clef/OpenAI Decisions APIといったDecisionモデルを使うこともできます。興味があれば見てみてください。 もちろんオープンウェイトモデルも（no censorを謳っているモデルでもない限り）明らかにペネトレーションテストの範囲を超えたクラッキングのような指示は拒否するはずですが、ペネトレーションテストの文脈を与えたAIは通常の開発作業時のコンテキストと比べると暴走しやすいのではないかという懸念を持っています。どこかの層が破られる前提で考え、慎重を期するに越したことはないと思います。 ペネトレーションテストの実行環境もGitHub Actionsに ペネトレーションテストの実行環境にもGitHub Actionsを選びました。すでにCIで使っていてチームが使い慣れている点、また、簡単な設定変更だけでログを最大400日まで残せるのも嬉しい点です。 docs.github.com 一応ペネトレーションテストをするにあたりGitHubの規約も確認したのですが、試験の対象はrunnerの中で立ち上げた自分たちのアプリだけで、外部のサービスには一切リクエストを送らない構成にしているため、規約上も問題ないと判断しています。 docs.github.com 検出結果への対応と次回ペネトレーションテストの改善のサイクルを回す ペネトレーションテストは回して終わりではなく、結果への対応こそが重要です。 診断結果はPrivateリポジトリのIssueとして投稿されるようになっています。 週次検査の診断結果Issue このIssueは、手元のコーディングエージェントのSkillで片付けています。Skillは週次検査のIssueをすべて読み、同じ指摘をまとめたうえで、指摘ごとに今のコードでも成立するかを確かめます。偽陽性や修正済みのものはここで落とします。対応が必要なものはAI Coding Agentに指示を出して修正していきます。 そして、未検証で終わった試験や脆弱性診断ツール実行の失敗がないかを見て、次回の検査の改善点を洗い、それもコーディングエージェントで修正をします。 おわりに 以上、ZennでのAIによるセキュリティチェックの取り組みを紹介しました。 まずはPRごとのAIレビューで、変更に入った問題を見つけます。さらにAIエージェントに毎週ペネトレーションテストをさせ、動かして初めてわかる問題を見つけます。AIの暴走はDockerのネットワーク隔離とAIの分類器で多重に抑え、外部サービスはOSSのエミュレーターに置き換えています。実行はGitHub Actionsで行い、結果への対応と検査の改善のサイクルを回しています。 どれもGitHub ActionsとDocker、OSSのツールの組み合わせなので、ローカルの開発環境をDockerで動かしているプロダクトなら、比較的手軽に始められると思います。いきなりペネトレーションテストまでやらなくても、まずはPRごとのAIレビューから始めるのもおすすめです。 今後は試験する範囲を広げつつ、継続して検出精度を上げていきたいと考えています。何かひとつでも参考になることがあれば幸いです。

WAFっていいな。簡単なペネトレ全部防げそう root@dev-bot:~# nmap example.com Starting Nmap 7.95 ( https://nmap.org ) at 2026-11-03 13:22 UTC Nmap scan report for nmap example.com (104.21.20.26) Host is up (0.0046s latency). Other addresses for nmap example.com (not scanned): 172.67.190.251 2606:4700:3037::6815:141a 2606:4700:3035::ac43:befb Not shown: 998 filtered tcp ports (no-response) PORT STATE SERVICE 80/tcp open http 443/tcp open https Nmap done: 1 IP address (1 host up) scanned in 4.74 seconds curl -I example.com #--- HTTP/1.1 200 OK Date: Tue, 03 Nov 2026 13:24:38 GMT Content-Type: text/html Connection: keep-alive CF-Cache-Status: HIT Cache-Control: public, max-age=0, must-revalidate Report-To: {"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=Pbe8W5nVe0oNQVKN8w0qKmjlbI...略"}]} Nel: {"report_to":"cf-nel","success_fraction":0.0,"max_age":604800} Server: cloudflare CF-RAY: a54bb34f3c92d472-NRT alt-svc: h3=":443"; ma=86400 何のページがあるのかわからない..そんなときはこれ。 zenn.dev note.com github.com/matteo741/Gobuster/blob/main/wordlist.… sudo apt update && sudo apt install -y gobuster dirb #gobuster --help gobuster dir -u http://study-hack-come.com -w /usr/share/wordlists/dirb/common.txt wordlist1.txtにワードのリストをはりつけ。 gobuster dir -u https://example.com -w wordlist1.txt 結果 =============================================================== Gobuster v3.6 by OJ Reeves (@TheColonial) & Christian Mehlmauer (@firefart) =============================================================== [+] Url: https://example.com [+] Method: GET [+] Threads: 10 [+] Wordlist: wordlist1.txt [+] Negative Status codes: 404 [+] User Agent: gobuster/3.6 [+] Timeout: 10s =============================================================== Starting gobuster in directory enumeration mode =============================================================== /robots.txt (Status: 200) [Size: 1248] /wp-config.php (Status: 403) [Size: 4917] /admin (Status: 401) [Size: 27] /index (Status: 307) [Size: 0] [--> /] Progress: 1828 / 1829 (99.95%) =============================================================== Finished =============================================================== gobuster dir -u https://example.com -w wordlist1.txt -x php,html,txt -o gobuster-basic.txt 結果 # gobuster-basic.txtからでも確認できる。 =============================================================== Gobuster v3.6 by OJ Reeves (@TheColonial) & Christian Mehlmauer (@firefart) =============================================================== [+] Url: https://example.com [+] Method: GET [+] Threads: 10 [+] Wordlist: wordlist1.txt [+] Negative Status codes: 404 [+] User Agent: gobuster/3.6 [+] Extensions: php,html,txt [+] Timeout: 10s =============================================================== Starting gobuster in directory enumeration mode =============================================================== /robots.txt (Status: 200) [Size: 1248] /wp-config.php (Status: 403) [Size: 4917] /wp-config.php.html (Status: 403) [Size: 4917] /wp-config.php.php (Status: 403) [Size: 4917] /wp-config.php.txt (Status: 403) [Size: 4917] /admin (Status: 401) [Size: 27] /index.html (Status: 307) [Size: 0] [--> /] /index (Status: 307) [Size: 0] [--> /] Progress: 7312 / 7316 (99.95%) =============================================================== Finished =============================================================== robots.txtを確認する。 検索エンジンのクローラーにここは検索結果に載せるなと指示する設定ファイル curl -s https://example.com/robots.txt 特になにもなかった。 # As a condition of accessing this website, you agree to abide by the following # content signals: # (a) If a content-signal = yes, you may collect content for the corresponding # use. # (b) If a content-signal = no, you may not collect content for the # corresponding use. # (c) If the website operator does not include a content signal for a # corresponding use, the website operator neither grants nor restricts # permission via content signal with respect to the corresponding use. # The content signals and their meanings are: # search: building a search index and providing search results (e.g., returning # hyperlinks and short excerpts from your website's contents). Search does not # include providing AI-generated search summaries. # ai-input: inputting content into one or more AI models (e.g., retrieval # augmented generation, grounding, or other real-time taking of content for # generative AI search answers). # ai-train: training or fine-tuning AI models. # ANY RESTRICTIONS EXPRESSED VIA CONTENT SIGNALS ARE EXPRESS RESERVATIONS OF # RIGHTS UNDER ARTICLE 4 OF THE EUROPEAN UNION DIRECTIVE 2019/790 ON COPYRIGHT # AND RELATED RIGHTS IN THE DIGITAL SINGLE MARKET. WordPress専用スキャンを試す cubepath.com sudo apt update sudo apt install curl unzip jq NUCLEI_VERSION=$(curl -fsSL https://api.github.com/repos/projectdiscovery/nuclei/releases/latest | jq -r .tag_name | sed 's/^v//') echo "$NUCLEI_VERSION" cd "$(mktemp -d)" curl -fsSLO "https://github.com/projectdiscovery/nuclei/releases/download/v${NUCLEI_VERSION}/nuclei_${NUCLEI_VERSION}_linux_amd64.zip" curl -fsSLO "https://github.com/projectdiscovery/nuclei/releases/download/v${NUCLEI_VERSION}/nuclei_${NUCLEI_VERSION}_checksums.txt" sha256sum --check --ignore-missing "nuclei_${NUCLEI_VERSION}_checksums.txt" # nuclei_3.11.1_linux_amd64.zip: OK を期待 unzip nuclei_*_linux_amd64.zip nuclei sudo install -m 0755 nuclei /usr/local/bin/nuclei nuclei -version #バージョン表示を確認 テンプレートをダウンロードする nuclei -update-templates ファイルを確認 ls /root/nuclei-templates #--- CODE_OF_CONDUCT.md TEMPLATES-STATS.md headless CONTRIBUTING.md TOP-10.md helpers LICENSE.md cloud http PULL_REQUEST_TEMPLATE.md code javascript README_CN.md contributors.json network README_JA.md cves.json profiles README_KR.md cves.json-checksum.txt ssl TEMPLATE-CREATION-GUIDE.md dast templates-checksum.txt TEMPLATE-REVIEW-GUIDE.md dns workflows TEMPLATES-STATS.json file 試しに実行 nuclei -u https://example.com -severity high,critical -rate-limit 50 -stats 結果 [INF] Scan completed in 3m. 0 matches found. [INF] HTTP connections: 5911 total, 1894 new, 4017 reused (68.0%) [0:03:06] | Templates: 4457 | Hosts: 1 | RPS: 57 | Matched: 0 | Errors: 22 | Requests: 10615/10782 (98%) なにもなしと。 本題のwordpress topscoder/nuclei-wordfence-cve export GITHUB_TEMPLATE_REPO=topscoder/nuclei-wordfence-cve nuclei -update-templates 期待する結果 [INF] No new updates found for nuclei templates [INF] Repo topscoder/nuclei-wordfence-cve cloned successfully at /root/nuclei-templates/github/topscoder/nuclei-wordfence-cve 早速試す nuclei -t github/topscoder/nuclei-wordfence-cve -u https://example.com -o results.jsonl # Deepl # 特定の CVE に関連する脆弱性をスキャンするには # nuclei -t github/topscoder/nuclei-wordfence-cve -template-id cve-2023-32961 -u https://target.com # 重大なものだけスキャン # nuclei -t github/topscoder/nuclei-wordfence-cve -severity critical -u https://target.com # WordPress コアの脆弱性のみをスキャンするには、次のコマンドを実行します。 # nuclei -t github/topscoder/nuclei-wordfence-cve -tags wp-core -u https://target.com # WordPress プラグインの脆弱性のみをスキャンするには、次のコマンドを実行してください： # nuclei -t github/topscoder/nuclei-wordfence-cve -tags wp-plugin -u https://target.com nuclei -t github/topscoder/nuclei-wordfence-cve -tags wp-plugin -u https://target.com nuclei -t github/topscoder/nuclei-wordfence-cve -tags wp-plugin -u rico-diary.com jqを使って深刻度ごとに表示 jq -r '.info.severity' results.jsonl | sort | uniq -c CMSeekを試す WebサイトのCMS検出・脆弱性スキャンツール Tuhinshubhra/CMSeeK sudo apt update sudo apt install git python3 python3-pip -y git clone https://github.com/Tuhinshubhra/CMSeeK cd CMSeeK curl -LsSf https://astral.sh/uv/install.sh | sh source ~/.bashrc #source ~/.bashrc uv venv && source .venv/bin/activate uv pip install -r requirements.txt 試す python3 cmseek.py -u example.com Niktoを試す omomuki-tech.com sudo apt update sudo apt install -y perl libxml-parser-perl　libxml-writer-perl git clone https://github.com/sullo/nikto.git cd nikto/program perl nikto.pl -Version 試す perl nikto.pl -h https://example.com -ssl # 80の時は perl nikto.pl -h http://example.com 保存形式 # テキスト形式で保存 perl nikto.pl -h http://example.com -o scan_result.txt -Format txt # HTML形式で保存 perl nikto.pl -h http://example.com -o scan_result.html -Format htm # CSV形式で保存 perl nikto.pl -h http://example.com -o scan_result.csv -Format csv # XML形式で保存 perl nikto.pl -h http://example.com -o scan_result.xml -Format xml # JSON形式で保存 perl nikto.pl -h http://example.com -o scan_result.json -Format json 結果をみる。 - Nikto v2.6.1/ + Target Host: example.com + Target Port: 443 + GET /: An alt-svc header was found which is advertising HTTP/3. The endpoint is: ':443'. Nikto cannot test HTTP/3 over QUIC. See: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/alt-svc: + GET /: Cloudflare detected via cf-ray header. Recommend proxying via Burp or mitmproxy to avoid TLS fingerprint blocks. See: https://github.com/sullo/nikto/wiki/Using-a-Proxy: + SSL Info: Subject: /CN=example.com CN: example.com SAN: example.com, www.example.com, *.www.example.com Ciphers: TLS_AES_256_GCM_SHA384 Issuer: /C=US/O=Google Trust Services/CN=WE1 + GET /: cloudflare - Cloudflare detected via banner. Recommend proxying via Burp or mitmproxy to avoid TLS fingerprint blocks if not already proxying. SSL/TLS証明書検索 platform.censys.io

cvENGINE: the most advanced external attack surface assessment you can get for free. hevnsnt/CVEngine It exists because of Black October. I wanted a self-hosted tool, built from the best open source tools there are, that answers one question: of all the vulns released every 30 minutes, do I have any of this exposed? Point it at a domain. cvENGINE auto-discovers every subdomain, IP, and open port, fingerprints every service, and matches what it finds against today’s exploited CVEs. Then an AI reproduces what it can with read-only checks and tells you which CVEs can actually hit you. One Docker Compose stack. Open source (GPL-3.0). Self-hosted. Bring your own AI keys, or run a local model. No license fee, no per-asset pricing, and nothing is sent to me. Under the hood: Discovery and scanning - bbot (Black Lantern Security): blacklanternsecurity/bbot - Nuclei and its templates (ProjectDiscovery): projectdiscovery/nuclei - naabu (ProjectDiscovery): projectdiscovery/naabu - nmap: nmap.org - WPScan: wpscan.com - node2shell scanner (Assetnote): assetnote/node2shell-scanner - MailFail (Black Hills Information Security): ACK-J/MailFail Exploitation intelligence behind the fire, smoke, and ice ratings - cve.now: cve.now - CISA KEV: cisa.gov - VulnCheck KEV: vulncheck.com - EPSS (FIRST): first.org - Exploit-DB: gitlab.com - Metasploit: rapid7/metasploit-framework Thank you to every maintainer who builds in the open. None of this exists without you.

2️⃣ Nuclei (@pdnuclei) ⚡ Version-matching scanners drown you in false positives. Nuclei runs simple YAML templates that fire real-world exploit requests across HTTP, DNS, TCP, and SSL to prove if a target is actually vulnerable. (32k+ Stars) 🔗 projectdiscovery/nuclei

I used local open-weight AI to avoid three phantom GitHub bounties I keep seeing bounty directories where a clean-looking $50 or $100 reward is still advertised even after the maintainer has closed the original task. For someone trying to earn their first real developer income, that is a dangerous trap: your code may pass, your PR may exist, and your dashboard may look great — but no person or payment process owes you money. I built SlothProof, a small, new, open-source, local-first AI tool that deliberately says no before an agent wastes days on the wrong bounty. - Try the working public report - Read the complete source and run the tests The three real issues I checked I ran the program against the original GitHub issue API, not an aggregator badge. | Original issue | What the first party actually says | Result | | --- | --- | --- | | Invidious #5957 | The advertised $50 alias bounty was closed by a maintainer after repetitive, unwanted submissions | SKIP_CLOSED | | Nuclei #6674 | A formerly advertised $100 bounty is closed/rewarded | SKIP_CLOSED | | Agent Bounties #1368 | The issue can appear open, but the underlying verification is unavailable/archived | SKIP_UNVERIFIABLE | Observed outcome: three listings excluded, zero new rewards received. I did not claim that filtering them earned me $250. It saved potential effort, which is not the same as a bank deposit. A surprisingly important design decision My original instinct was to feed each advertised task to an AI model and ask: "Is this worth doing?" That would be a bad architecture. A language model may find a closed bounty semantically exciting — especially if it mentions "funded", "verified" or a big prize. Instead, SlothProof makes the original source decisive: 1. Accept only a canonical public GitHub issue URL, with a strict parser; do not follow arbitrary user-supplied addresses. 2. Read the issue from GitHub's public REST API. 3. Classify closed, verifier-unavailable, and bond-required issues with deterministic checks. 4. Use an open-weight AI embedding model to add a separate advisory signal. It cannot override the source state. 5. Never convert a claimed bounty amount into income without an independent paid receipt. The actual open-weight model is Xenova/all-MiniLM-L6-v2, loaded locally through the open-source Transformers.js library. The demonstration did run real quantized inference on VM1; the public report contains the model name, top signal and similarity. No external closed-model API is needed once weights are cached. An example live invocation: npm ci --ignore-scripts --no-audit --no-fund npm test npm run scan -- --live \ invidious#5957 \ nuclei#6674 \ agent-bounties#1368 \ --output docs/report.json On the tested VM, the eight original deterministic regression tests all passed and the three public source issues produced the stop verdicts above. The online dashboard also loaded the first-party report correctly. Why open-weight AI is useful here "Slow and Steady" is a good design constraint. The semantic layer can flag terms related to a closed task, risky bond, or missing verifier. It is reproducible, swappable and locally inspectable — without delegating the financial decision to the model. The source checks still work when model loading fails, and the output explicitly reports whether AI inference really ran. The browser dashboard is static HTML, CSS and a local JSON fetch, with no analytics and no payment credentials. GitHub issue text is treated as untrusted data and inserted as text rather than executable HTML. There are important limitations. A fresh issue marked "open" might still be unfunded. A successful PR might never be merged. An approved sponsor payment may remain unpaid. Even a genuine USDC Transfer doesn't automatically prove a given invoice, buyer identity or obligation. SlothProof is a work-avoidance helper, not an escrow service. Does it make money? Not yet. The SlothProof project has zero customers, zero accepted paid contracts and zero received USDC at publication. Its underlying owner had separate historic transactions on other products, which I am not presenting as this project's sales. If someone has a list of public software bounties and wants a second look, contact me through the live project page for a conditional 2 USDC fixed-scope review of up to ten public issues. The work would begin only after the buyer confirms the scope, deliverables and lawful payout terms. This is an offer, not a previously paid job or an earnings guarantee. The most honest growth metric for this experiment is not the number of badges or repo stars. It is: did an independent buyer sign off and actually pay for a useful result? --- Disclosure: This project and article were prepared with substantial AI assistance for one actual human beneficial owner. It uses openly identified third-party open-weight model software; it is not a claim of manually written, AI-free engineering. This is an ordinary technical article, not an official contest submission or a claim of prize eligibility.

When copy-paste goes wrong: Bugs in structurally isomorphic Go code You might've done this many times: you found code in a project that seemed like a good fit for your task, copied it, replaced the variables or function calls with the ones you needed, and then got behavior you didn't expect when you ran the code. It's likely that you missed something while making those changes. This is exactly how the errors described in this article occur. !Image description Introduction PVS-Studio 8.00 was recently released, introducing several new major features, including the Go analyzer that comes with several new and cool diagnostic rules. It includes a rule that detects one of the trickiest types of copy-paste bugs, the kind that can easily slip past a regular code review. Okay, now let's take a look at the code: if process.ProcessName != nil && (len(*process.ProcessName) == 0 || len(*process.ProcessName) > common.NameFieldMaxLength) { return kit.CCError.CCErrorf(common.CCErrCommParamsInvalid, common.BKProcessNameField) } .... if process.FuncName != nil && (len(*process.FuncName) == 0 || len(*process.ProcessName) > common.NameFieldMaxLength) { return kit.CCError.CCErrorf(common.CCErrCommParamsInvalid, common.BKFuncName) } You've probably noticed that the expressions in both if statements consist of the same checks, but they aren't completely identical: the first one uses process.ProcessName, while the second one uses process.FuncName in the same positions. Or... does it? The last checks suspiciously break this pattern: they're exactly the same, and both use the process.ProcessName field. Based on the logic the code has consistently followed so far, the check in the second if should have used a different field, process.FuncName. Since such typos can easily slip past manual review, the best solution is to delegate the task of catching them to a static analyzer. But how can we teach a program to compare similar yet distinct code fragments and use those comparisons to detect bugs the way humans do? Is AI necessary for that? In the rest of this article, we'll explore these questions by looking at how the diagnostic rule works in PVS-Studio for Go. How does the analyzer find these bugs? Modern AI models can indeed detect such bugs. However, integrating artificial intelligence into a static analyzer for such a purpose is too expensive and resource-intensive. Fortunately, we don't need AI here. We can handle the task with a relatively simple heuristic algorithm. We'll take a closer look at how it works later, but first, let's go over a few basic concepts from static code analysis: - Statements are the elements of code that form its skeleton and describe the execution flow (if, switch, for, and {...} blocks). - Expressions are code elements that manage data (function calls, variables, and operators). - Nodes is a general term for statements, expressions, and other types of code elements. - Abstract Syntax Tree (AST) represents code as a hierarchical structure where some elements, like folders in a file system, contain other elements. An AST contains the same information you can get from the code itself, but its structured format makes it better suited for program analysis. If you're interested in delving deeper into static analysis, we have the perfect article for you. Well, now that we have enough context, we're ready to face the boss algorithm. Here's how the V8033 diagnostic rule in PVS-Studio's Go analyzer works: 1. Within a function, the analyzer takes individual statements and groups of statements, separated by blank lines (hereinafter referred to as code fragments) and builds a simplified AST for them. Unlike a regular AST, it contains only the information needed to describe the code structure and omits specific names of variables, functions, and other semantic information. 1. The fragments are grouped by their simplified AST. This is how fragments with similar structure are identified, even though they may still differ in their details (for example, in the variables they use). 1. Within each group, the fragments are paired and compared with one another. Let's look at how the algorithm works for a single pair. For convenience, let's call the two fragments in one pair A and B. 1. The analyzer identifies the variables that appear in both A and B (common variables) as well as the variables that appear in only one of the fragments (specific variables). 1. Let's assume that A is higher in the code than B. If one of the common variables occurs less frequently in B than in A, but still at the same relative positions, the uses of that variable in B are treated as potential typos. 1. Next, a replacement candidate is sought among the B variables. This variable must be used in the same relative position as the previously considered general variable in A, but not in B. 1. If the analyzer finds such a candidate, it performs a series of final checks to determine whether the variables are interchangeable. If all the checks pass, the developer receives warnings about possible typos in the code. Of course, this is just a conceptual outline. A significant amount of important logic remains behind the scenes, especially the logic that filters out false positives. Without it, the diagnostic rule would be far less useful. > It is only 5% of the task to create the diagnostic. It's not hard for the analyzer to complain about suspicious code, it's much harder to not complain about the correct code. We spend 95% of our time "teaching" the analyzer to detect various programming techniques, which may seem suspicious for the diagnostic, but in reality they are correct. From the article "How PVS-Studio does the bug search: methods and technologies" Now that we've taken a quick look at how the analyzer works, let's exercise in looking for similar errors on our own. Find an error I invite you to challenge yourselves by finding errors in this code, which was taken from real open-source projects. The closer you get to the end, the harder the examples become. Are you up for the challenge? Let's get started, then! Yao Agents Yao Agents is an open-source platform for creating and running autonomous AI agents. Its key feature is that the agents run directly on users' devices, while all operations are centralized and controlled via a single interface. We found this code in the v1.0.0-rc15 release: func Generate(option Option) (string, string) { if option.Width == 0 { option.Width = 240 } if option.Height == 0 { option.Width = 80 } if option.Length == 0 { option.Length = 6 } if option.Lang == "" { option.Lang = "zh" } .... } {% spoiler The analyzer's hint %} V8033 Two similar code fragments were found. The second fragment may contain a typo: the "option.Height" variable should be used instead of the "option.Width". captcha.go 48 {% endspoiler %} CDS CDS is an enterprise-grade platform for automating DevOps processes, continuous integration (CI), and continuous delivery (CD). We found this code in the 0.54.1 release: if conf.GraylogFieldCDSServiceName != "" { extra["CDSName"] = conf.GraylogFieldCDSServiceName } if conf.GraylogFieldCDSServiceName != "" { extra["CDSService"] = conf.GraylogFieldCDSServiceType } if conf.GraylogFieldCDSVersion != "" { extra["CDSVersion"] = conf.GraylogFieldCDSVersion } if conf.GraylogFieldCDSOS != "" { extra["CDSOS"] = conf.GraylogFieldCDSOS } if conf.GraylogFieldCDSArch != "" { extra["CDSArch"] = conf.GraylogFieldCDSArch } {% spoiler The analyzer's hint %} V8033 Two similar code fragments were found. The second fragment may contain a typo: the "conf.GraylogFieldCDSServiceType" variable should be used instead of the "conf.GraylogFieldCDSServiceName". log.go 107 {% endspoiler %} BFE BFE is a high-performance, application-level load balancer and reverse proxy server. We found this code in the 1.8.4 release: if cfg.Basic.WasmPluginPath == "" { .... cfg.Basic.WasmPluginPath = "mod_wasm" } cfg.Basic.WasmPluginPath = bfe_util.ConfPathProc(....) if cfg.Basic.DataPath == "" { .... cfg.Basic.WasmPluginPath = "mod_wasm/wasm.data" } cfg.Basic.DataPath = bfe_util.ConfPathProc(cfg.Basic.DataPath, confRoot) {% spoiler The analyzer's hint %} V8033 Two similar code fragments were found. The second fragment may contain a typo: the "cfg.Basic.DataPath" variable should be used instead of the "cfg.Basic.WasmPluginPath". conf_mod_wasmplugin.go 63 {% endspoiler %} Hertz Hertz is a high-performance HTTP framework written in Go by ByteDance (the creators of TikTok), designed for building high-load microservices. It combined the concepts of popular Go frameworks (Gin, FastHTTP) with ByteDance's internal requirements. We found this code in the 0.10.6 release: if goOptMapAlias[sb.Scope.GetOptions().GetGoPackage()] != "" { reqName = goOptMapAlias[sb.Scope.GetOptions().GetGoPackage()] + "." + inputGoType.GoIdent.GoName } if goOptMapAlias[sb.Scope.GetOptions().GetGoPackage()] != "" { respName = goOptMapAlias[st.Scope.GetOptions().GetGoPackage()] + "." + outputGoType.GoIdent.GoName } {% spoiler The analyzer's hint %} V8033 Two similar code fragments were found. The second fragment may contain a typo: the "st" variable should be used instead of the "sb". ast.go 233 {% endspoiler %} Nuclei Nuclei is a modern, high-performance vulnerability scanner that uses simple YAML-based templates. It enables users to create their own vulnerability detection scenarios that simulate real-world conditions, which eliminates false positives. We found this code in the 3.11.1 release: for _, tag := range config.ExcludeTags { for _, val := range splitCommaTrim(tag) { if _, ok := filter.block[val]; !ok { filter.block[val] = struct{}{} } } } for _, tag := range config.Severities { if _, ok := filter.severities[tag]; !ok { filter.severities[tag] = struct{}{} } } .... for _, tag := range config.ExcludeProtocols { if _, ok := filter.excludeTypes[tag]; !ok { filter.excludeTypes[tag] = struct{}{} } } for _, id := range config.ExcludeIds { for _, val := range splitCommaTrim(id) { if _, ok := filter.block[val]; !ok { filter.excludeIds[val] = struct{}{} } } } {% spoiler The analyzer's hint %} V8033 Two similar code fragments were found. The second fragment may contain a typo: the "filter.excludeIds" variable should be used instead of the "filter.block". tag_filter.go 434 {% endspoiler %} LakeFS LakeFS is an open-source data version control system for managing data stored in an object storage system. It enables Git-like operations like branching, committing, merging, and rolling back for large-scale data, stored in systems including Amazon S3, Azure Blob Storage, and Google Cloud Storage, as well as on other S3-compatible object storage platforms. We found this code in the 1.86.0 release: usersKey := model.GroupUserPath(groupID, "") it, err := kv.NewSecondaryIterator(....) if err != nil { return err } defer it.Close() for it.Next() { entry := it.Entry() user := entry.Value.(*model.UserData) .... } if err = it.Err(); err != nil { return err } policiesKey := model.GroupPolicyPath(groupID, "") itr, err := kv.NewSecondaryIterator(....) if err != nil { return err } defer it.Close() for itr.Next() { entry := itr.Entry() policy := entry.Value.(*model.PolicyData) .... } if err = itr.Err(); err != nil { return err } {% spoiler The analyzer's hint %} V8033 Two similar code fragments were found. The second fragment may contain a typo: the "itr" variable should be used instead of the "it". service.go 476 {% endspoiler %} Vault Vault is an open-source (BSL) tool for centralized secret management, data encryption, and for the protection of confidential information in cloud and hybrid infrastructures. We found this code in the 2.0.4 release: for name, crl := range h.CRLs { .... ratio := time.Duration( (int64(total) * int64(h.CRLExpiryPercentage)) / int64(100) ) if used >= ratio { expWhen := crl.ThisUpdate.Add(ratio) ret.Status = ResultCritical ret.Message = fmt.Sprintf( ...., crl.NextUpdate.Format("2006-01-02"), h.CRLExpiryPercentage, crl.ThisUpdate.Format("2006-01-02"), expWhen.Format("2006-01-02") ) .... } results = append(results, &ret) } for name, crl := range h.DeltaCRLs { .... ratio := time.Duration( (int64(total) * int64(h.DeltaCRLExpiryPercentage)) / int64(100) ) if used >= ratio { expWhen := crl.ThisUpdate.Add(ratio) ret.Status = ResultCritical ret.Message = fmt.Sprintf( ...., crl.NextUpdate.Format("2006-01-02"), h.CRLExpiryPercentage, crl.ThisUpdate.Format("2006-01-02"), expWhen.Format("2006-01-02") ) .... } results = append(results, &ret) } {% spoiler The analyzer's hint %} V8033 Two similar code fragments were found. The second fragment may contain a typo: the "h.DeltaCRLExpiryPercentage" variable should be used instead of the "h.CRLExpiryPercentage". pki_crl_validity_period.go 195 {% endspoiler %} Conclusion Did you spot all the errors without looking at the hints? If so, that's great! If not, don't worry—that's where a static code analyzer comes into play! By the way, you can download it from our official website and try it for free on your own projects. In addition to the Go analyzer, PVS-Studio 8.00 brought several other exciting features, including an analyzer for JavaScript and TypeScript. You can find more details about them in our article. Happy coding!

Defending your codebase from AI slop: guardrails that keep you in control AI coding agents are fast. They also write code that compiles, looks fine at a glance, and slowly wears down your architecture. A component imports the database layer directly. A function grows to 200 lines with three nested ternaries. A dependency shows up that nobody asked for. A routes file picks up a formatDate helper, then a slugify, then a retryWithBackoff. Tests run every line but never check a result. None of this is new. Junior developers in a hurry do the same things. What's new is the volume. If an agent opens ten PRs a day, careful human review alone won't catch everything. The answer isn't to stop using AI. It's to write your standards down in a form machines can enforce, so the rules hold no matter who or what wrote the code. Here's the stack I usually build for my projects. 1. Write the rules down first Tools can only enforce rules that exist. Before adding any tooling, I decide how I want my code to look, define the rules, and write them in two files: - ARCHITECTURE.md covers the layers, which layer may import which, where state lives, and how data flows. - AGENTS.md, essential nowadays, is usually created automatically and then edited by me. It covers conventions for AI agents: naming, patterns to use, patterns to avoid, and tools that are off-limits (for example: "we use Biome, don't add ESLint"). Agents read these files. So do new hires. And every tool below points back to a section in one of them, so when a check fails, the message explains _why_ and not just _what_. 2. Guard the boundaries: dependency-cruiser Something I've discovered quite recently: dependency-cruiser checks which files import which. You describe the allowed directions between layers, and it fails the build when something crosses a line: - UI components can't import the database layer - modules can't import each other circularly - features can't reach into another feature's internals - orphaned files that nothing imports get flagged This is the kind of mistake AI makes most, because it takes whatever import path is closest. A rule that always gives the same answer stops it before review. 3. Guard the shape of the code: Semgrep dependency-cruiser tells you _which files talk to each other_. It can't tell you _what the code inside a file is doing_. That's what Semgrep is for. How Semgrep works Semgrep is a pattern matcher that understands code. You write a rule that says "code shaped like X, in files matching Y, is a violation." Semgrep parses each file into a syntax tree and reports every match. It doesn't run your code or try to understand your whole app. It only checks shapes. A rule is a short YAML entry: rules: - id: api-routes-no-helpers languages: [python] severity: ERROR message: > routes.py is for route handlers only. Move helpers to their own module (see ARCHITECTURE.md → API). paths: include: [packages/api/src/api/routes.py] patterns: - pattern: | def $FUNC(...): ... - pattern-not-inside: | @$ROUTER.$METHOD(...) def $F(...): ... - pattern is the code to match. $FUNC matches any name, and ... matches anything. - pattern-not-inside is the exception. Here, functions with a route decorator are allowed. - paths limits the rule to certain files. - message is what the developer (or the AI agent) sees when the rule fails. Write it as an instruction. Running semgrep --config .semgrep/rules/ checks the whole repo against every rule in that folder. We keep one YAML file per area (app, API, mq) so rules stay easy to find, and each rule gets a small test fixture with code that should match and code that shouldn't, so we know the rule catches what we meant. Read their docs (or ask your agent to) on what's possible and get creative. One-purpose files This was the rule we wanted most. Some files exist for one job, and AI agents love to drop "just one small helper" into them. After a few months, your routes file is half utilities. So we made it impossible. Each of these files may only contain its one kind of thing: - API route modules (routes.py): route handlers only - App services/: one class, methods only, no loose functions - App functions/: TanStack server functions only - Hooks: hooks only - Route files: route definitions only - DB schema: table definitions only - API models/schemas: models only - API repositories: repository classes only Helpers go in their own module, where they can be named, tested and reused. The rule message tells the agent exactly that, so it usually fixes itself on the next try. Other patterns worth enforcing Once you think in shapes, a lot of review comments turn into rules: - query and mutation keys written inline instead of coming from a key factory - data fetching inside components - raw fetch in routes and server functions - Drizzle queries outside db/queries - exported query functions that default to tx: DbClient = db (this hides transaction bugs) - queue names written as string literals - raw SQL built from strings in the Python API - API routes importing models directly instead of going through a repository Semgrep works on many languages, so it covers anything you want. > Rule of thumb: if a review comment can be written as a code pattern, make it a Semgrep rule. Then it gets checked the same way on every PR. Adding it to an existing codebase You probably have violations already. Don't let that stop you. In CI, Semgrep can compare against a baseline: - On PRs, only violations _added in that PR_ fail the check. - On main, the full scan runs, so you can see the existing debt and pay it down over time. New code is held to the standard from day one, and nobody has to fix everything first. 4. Scan for vulnerabilities: SAST, SCA and DAST Architecture rules keep the code tidy. They don't tell you whether it's safe. AI agents write the same security bugs humans do, just faster: SQL built from strings, an endpoint that forgets to check who's calling it, a token pasted into a config file, user input rendered as HTML. Security tools fall into three groups, and each sees something the others can't: | Kind | Looks at | Finds | Misses | | --------------------------------------- | ------------------------------ | ------------------------------------------------------------------ | ------------------------------------------------------- | | SAST (static application security) | Your source code, not running | Injection, unsafe APIs, hardcoded secrets, tainted data flows | Runtime config, auth bugs that depend on real data | | SCA (software composition analysis) | Your dependencies and lockfiles | Known CVEs, malicious packages, licenses | Bugs in your own code | | DAST (dynamic application security) | The running app, from outside | Missing auth, exposed endpoints, bad headers, misconfigured servers | Where in the code the bug is; code paths it never reaches | SAST and SCA run on every PR in seconds or minutes. DAST needs a deployed app, so it runs against a preview environment or staging. Most of the tools below cover more than one group: | Tool | SAST | SCA | DAST | Other | | ------------------------ | :--: | :-: | :--: | ----------------------------- | | Semgrep | ✅ | ✅ | | Secrets | | Trivy | | ✅ | | Containers, IaC, secrets | | Snyk | ✅ | ✅ | ✅ | Containers, IaC | | Checkmarx One | ✅ | ✅ | ✅ | Containers, IaC, secrets, API | | GitHub CodeQL | ✅ | | | Dependabot covers SCA | | ZAP | | | ✅ | Free, open source | I am an OSS supporter, so I usually go with Semgrep and Trivy. Companies I've worked for went with whatever suited them best. Choose any; no judgement there. SAST: your code Good SAST tools track data flow: they follow user input from a request handler and flag it if it reaches SQL, a shell or HTML unescaped. Ask an agent to "add search" and you may get: @router.get("/search") def search(q: str, db: Session = Depends(get_db)): return db.execute(text(f"SELECT * FROM products WHERE name LIKE '%{q}%'")) It works, and it's SQL injection. Semgrep Code catches it with the engine you already run for architecture (add --config p/owasp-top-ten). CodeQL is free for public repos and shows results in the PR. Snyk Code and Checkmarx cover the same ground in their platforms. Fail the build on high-confidence rules only. A noisy check gets switched off. SCA: your dependencies Agents add packages freely, sometimes outdated, sometimes made up and squatted ("slopsquatting"). Trivy is the free baseline for vulnerable packages, secrets, container images and IaC: trivy fs . --scanners vuln,secret,misconfig --severity HIGH,CRITICAL --exit-code 1 Trivy only matches versions, so it flags vulnerable packages whether or not you use the vulnerable part. Semgrep Supply Chain checks reachability: does your code actually call the vulnerable function? Snyk opens fix PRs and alerts when a new CVE hits code you already shipped. Checkmarx One also detects malicious packages such as typosquats. Require CODEOWNERS approval on package.json and lockfiles, so every new dependency is a human decision. DAST: the running app DAST sends real requests to a deployed app, so it catches what code scanning can't: an endpoint the agent left without an auth check, one user reading another's data by changing an ID, missing security headers, exposed debug pages. Run it against a preview deploy or staging: docker run -t ghcr.io/zaproxy/zaproxy:stable zap-api-scan.py \ -t https://preview.example.com/openapi.json -f openapi ZAP is free. Nuclei checks for known exposures. StackHawk, Snyk API & Web and Checkmarx DAST are paid options. Give the scanner credentials and an API spec, or it will scan your login page and little else. DAST, I'd say, is a nice-to-have, not a must-have. Folks at Semgrep think so too. Occasional manual testing or pentesting is more than enough. 5. Measure complexity and test quality Something I've learned about from Uncle Bob. I never thought about it until AI became a thing. CRAP stands for Change Risk Anti-Patterns, a metric used to identify risky, complex, and poorly tested code. The CRAP Index measures the maintenance risk of a specific function or method. Rule of thumb: a score above 30 indicates a high-risk, "CRAPpy" method that needs attention or refactoring. I found two tools I could integrate into my pipelines that would help to keep AI-generated code under control: - crap4ts computes a CRAP score from complexity and coverage. Complex code with weak tests scores high. Set a limit and fail functions that go over it. - ArchUnitTS lets you write architecture rules as unit tests, if you'd rather keep them next to your test suite. 6. AI reviewing AI: CodeRabbit with custom rules Some rules can't be written as patterns: "does this name match what the code does?" or "is this abstraction premature?" For those, we can use CodeRabbit (or one of many other AI code review tools) with a strict config file, like .coderabbit.yaml: - the strictest review profile, and it requests changes when it finds problems - it reads ARCHITECTURE.md and AGENTS.md as guidelines - instructions for each folder, each linked to the doc section it comes from - pre-merge checks: most only warn, but a few block the merge (for example, a schema change with no migration) The important part is that CodeRabbit is the last layer, not the first. AI review isn't consistent, so anything that _can_ be checked by a tool that always gives the same answer should be. CodeRabbit handles judgement calls. 7. More you can add What we've set up covers most of it. Here are a few more tools that can help you keep even tighter control over your repo: - Knip finds unused files, exports and dependencies. AI leaves a lot of dead code behind. - Mutation testing (which Uncle Bob is also looking at and seems to think is an essential part of CI/CD checks) (Stryker) changes your code on purpose and checks whether tests fail. It's the best way to catch tests that run code but never check results. - PR size limits: fail or warn on PRs over a few hundred lines. Small PRs actually get reviewed. Not something I'd do, though. - Pre-commit hooks: such as Lefthook or, most commonly in JS-based projects, husky. You can configure them to run anything, including Biome, Semgrep and type checks, locally, so you get feedback before you push. The principle Every layer follows the same idea: move each rule to the most reliable tool that can enforce it. 1. Write the rule in ARCHITECTURE.md / AGENTS.md. 2. If it's about imports → dependency-cruiser. 3. If it's about code shape → Semgrep. 4. If it's a security bug in your code → SAST (Semgrep, CodeQL, Snyk Code, Checkmarx). 5. If it's about dependencies or secrets → SCA (Trivy, plus Semgrep Supply Chain, Snyk or Checkmarx). 6. If it only shows up at runtime → DAST (ZAP, Nuclei, StackHawk, Snyk, Checkmarx) against a preview deploy. 7. If it needs judgement → CodeRabbit, and then a human. AI can write as much code as it likes. It just has to pass the same checks as everyone else. That's how you stay in control.

・nuclei projectdiscovery/nuclei YAMLテンプレートベースの高速脆弱性スキャナー。コミュニティが大量のテンプレートを提供しており、CVEや設定ミスを効率よく検出できる定番ツール。