withastro

I Import 11 Packages I Never Declared; Seven Vanish on an Upgrade I sat down to upgrade this blog's dependencies. Ordinary work: astro was one minor behind, and @astrojs/mdx was a whole major behind — I had 7.0.5, the registry had 8.1.3. Before touching anything I opened the release notes, as I always do. Astro's 8.0.0 entry says: > If you haven't explicitly installed a Markdown processor, you don't need to > do anything. I hadn't installed one. So for me this was a no-action major. I was going to run the upgrade and move on. I didn't. Because I tried the upgrade in a scratch directory first, and the first step of my publishing pipeline died before reading its second line: Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'js-yaml' imported from .../scripts/render-article.mjs js-yaml is not in my package.json. None of the eleven third-party packages that file imports is; js-yaml only blows up first because it comes first. All of them work today — because Astro brings them along in its own tree and npm spreads everything into the root of node_modules. The upgrade changes Astro's tree, and seven of the eleven disappear. What this was measured on: macOS 26 (Darwin 25.6.0, arm64), Node v22.16.0, npm 10.9.2, repository pin a3c41e82 (6 Nov 2026). Everything was measured in this blog's own repository and in isolated lab directories under /tmp; the main node_modules tree was never modified. My first count was wrong My first job was answering "how many are there". I wrote a forty-line scanner: walk every .ts/.mjs/.astro/.mdx file in the repository, collect every bare specifier inside import and require, compare against what package.json declares. The scanner found 32 undeclared packages. Reading the list was a relief, because the list contained these: dart:async (2 files) package:flutter (2 files) fmt (2 files) aws-sdk (2 files) fmt is a Go package. package:flutter is a Dart package. None of this is my code — those are example lines inside the code blocks of the blog's own articles. The scanner walked every content file on the blog and took each code sample for a real import. Of the 3,079 files it scanned, 2,821 were content; 92% of the measurement was irrelevant, and 19 of the 32 packages it reported came purely from article code blocks — the Go and Dart lines above, but also innocent-looking JavaScript names like express and redis. This is not my first fall into that trap on this blog — I have written the same sentence four times before, and one of them was in an alt-text audit where the pattern ^#{1,6}\s counted shell comments inside code blocks as headings. The mechanism is identical; only the scanner changed. Four times is apparently not enough to learn something, and all four times it bit me in the same place: code blocks. I narrowed the scan to scripts/ and ran it again. The real numbers: | Measurement | Value | |---|---| | Files scanned | 97 | | Packages declared in package.json | 20 | | Third-party packages imported | 20 | | Undeclared | 13 | Eleven of the thirteen live in a single file: scripts/render-article.mjs. The other two are @mdx-js/mdx (in two separate validation scripts) and node-html-parser. Node never looks at dependencies while resolving Why do these thirteen imports not fail? The answer is in Node's resolution algorithm. The PACKAGE_RESOLVE step in the ESM documentation walks up the filesystem for a bare specifier: > While parentURL is not the file system root … Let packageURL be the URL > resolution of "node_modules/" concatenated with packageName, relative to > parentURL. So when scripts/render-article.mjs asks for js-yaml, Node checks scripts/node_modules/js-yaml, then node_modules/js-yaml, then the parent directory's node_modules, all the way to the filesystem root. First match wins. What the documentation does not contain matters more: no step of the algorithm reads the dependencies field of package.json. It reads exports, main, imports, name, type. It does not read dependencies. As far as resolution is concerned, a declaration is no more binding than a comment. !Diagram How the package reaches the root is npm's business. npm's own folders documentation describes it as an optimization: "Since version 3, npm hoists dependencies by default." The page offers no guarantee that every transitive dependency will be reachable from the root. What it offers is a default behaviour, and the difference between those two things is exactly this article's subject. In this repository: the lockfile holds 755 package entries, 687 of them at the root and only 68 nested. Alongside the 20 packages I declared, there are 667 more on the same shelf that I never declared. Every one of them is one import line away, and none of them will tell me when it leaves. When npm does not hoist: the 68 exceptions in my own tree If "no guarantee" sounds theoretical, the counter-example sits in my own tree. When npm cannot place a package at the root it puts it under its dependent instead, and that happens on version conflicts. Here is the count for this repository: 52 packages are installed at more than one version. Examples: chokidar 4.0.3 and 5.0.0 readdirp 4.1.2 and 5.0.0 @astrojs/internal-helpers 0.10.2 and 0.11.0 @astrojs/markdown-remark 7.2.2 and 7.3.1 In each conflict one version goes to the root and the other stays nested. So hoisting is not a rule that always fires; it is an optimization applied when there is no conflict — and which version wins the root is not determined by any line I wrote. When I write import 'js-yaml' I take a dependency on an install artifact. And that artifact is shaped not by my declarations but by the relationships among packages I never declared. Who carries them: ownership chains from the lockfile Which declared package brings which? I first tried reading npm ls, and the tree output drowned in deduped lines. The better route: compute each declared package's dependency closure from the packages map in package-lock.json. | Package | Direct dependent | Contained in the closure of | |---|---|---| | remark-gfm | @astrojs/mdx | @astrojs/mdx | | remark-smartypants | @astrojs/mdx | @astrojs/mdx | | unist-util-visit | @astrojs/mdx | mdx, node, astro | | @mdx-js/mdx | @astrojs/mdx | @astrojs/mdx | | shiki | astro | mdx, node, astro | | js-yaml | astro | mdx, node, astro | | github-slugger | astro | @astrojs/mdx, astro | | unified | — | mdx, node, astro | | remark-parse | — | @astrojs/mdx | | remark-rehype | — | @astrojs/mdx | | rehype-stringify | — | @astrojs/mdx | | remark-mdx | — | @astrojs/mdx | | node-html-parser | rettiwt-api | rettiwt-api | The five packages in the middle of the table are not a direct dependency of any declared package — they sit two or more levels deep. For them to be at the root, npm has to hoist across two tiers, which it is under no obligation to do. The last row is its own story. The only package that brings node-html-parser is rettiwt-api, this blog's Twitter client. I borrow my HTML parser from my social-posting library. The day I post to Twitter some other way, my render-comparison script will find itself unable to parse HTML. There is no conceptual link between these two; the only link is that they happen to sit in the same directory. The upgrade experiment: four installs, the same eleven imports To turn the theory into a measurement I started with two clean directories. Each one declares exactly two packages and nothing else: # lab-now: the versions this repo has today npm i astro@7.3.5 @astrojs/mdx@7.0.5 # lab-next: the pending upgrade npm i astro@latest @astrojs/mdx@latest # 7.4.7 + 8.1.3 Then I tried to resolve, one by one with import(), the eleven packages render-article.mjs needs: === lab-now (astro 7.3.5 + @astrojs/mdx 7.0.5) === RESOLVED 11/11 packages at root (scopes expanded): 307 === lab-next (astro 7.4.7 + @astrojs/mdx 8.1.3) === RESOLVED 4/11 MISSING: remark-parse, remark-rehype, rehype-stringify, remark-gfm, remark-mdx, remark-smartypants, js-yaml packages at root (scopes expanded): 188 Seven of eleven gone. My explanation was ready: @astrojs/mdx's dependency list shrank. The registry data supports it — 7.0.5 declares 14 dependencies (@astrojs/markdown-remark, @mdx-js/mdx, remark-gfm, remark-smartypants, unist-util-visit, rehype-raw, hast-util-to-html and others), while 8.1.3 declares 3: @astrojs/internal-helpers, @astrojs/markdown-satteri, es-module-lexer. The explanation looked right, but I had upgraded two packages together and never separated what each one did. So I set up two more labs, this time changing one thing at a time: | Install | Resolved | Lost | |---|---|---| | astro 7.3.5 + mdx 7.0.5 (today) | 11/11 | — | | astro 7.4.7 + mdx 7.0.5 (minor only) | 11/11 | — | | astro 7.3.5 + mdx 8.1.3 (major only) | 5/11 | 6 remark/rehype packages | | astro 7.4.7 + mdx 8.1.3 (both) | 4/11 | those six + js-yaml | The table refuted half of my explanation. The major does take six packages away, true. But the package in my error message, the js-yaml from this article's first line, disappears in neither change on its own. The reason: js-yaml had two separate providers. astro@7.3.5 declares it directly (js-yaml: ^4.3.2), and @astrojs/mdx@7.0.5 brings it a second time through the @astrojs/markdown-remark chain. Upgrade the minor and the first provider goes while the second holds; upgrade the major and the second goes while the first holds. Upgrade both and I cut away both legs at once. Writing this annoys me, because it strengthens my own thesis: no release note could have protected me here. The major's note says nothing about js-yaml, because the major alone does not remove it. The minor has no note at all; minors do not get one. The breakage lives at the intersection of two changes, and only I could have seen that intersection — for which I would have had to write down somewhere that I use those packages. The four survivors are worth a look too, because they also survived by accident: unified now arrives via @astrojs/internal-helpers; shiki and github-slugger come straight from astro; and unist-util-visit arrives by two routes — mdast-util-to-hast and retext-smartypants. That last one is a text package; the fact that my tree-walking package can also come from there says plenty about how much of this rests on coincidence. One experiment along the way was wasted. To see the same result by disabling hoisting, I copied the repository's own package-lock.json into a third lab directory and ran npm ci --install-strategy=nested. The layout did not change: 516 directories were still at the root, and only 30 nested node_modules appeared. I did not chase down why the flag was ineffective, because the table above already answered the real question with real versions. I am writing up the experiment that failed too — assuming a measurement measures what you expect is the most expensive habit in this line of work, and it caught me twice in this one article. End to end: real script, real error So the labs would not stay abstract, I copied scripts/render-article.mjs into both directories unchanged — along with src/lib/blog-frontmatter-schema.mjs, because the script imports it by relative path — and handed it a real article file: tutorials/3583-commite-not-yazdim-klon-hicbir-notu-getirmedi.mdx. In lab-now the script did what it should: parsed the frontmatter, extracted that post's 16 headings, produced HTML. In lab-next the output was: node:internal/modules/package_json_reader:256 throw new ERR_MODULE_NOT_FOUND(packageName, fileURLToPath(base), null); Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'js-yaml' imported from .../lab-next/scripts/render-article.mjs On the first import line, before reading a single character of MDX. I should say what this script is, because it is not some helper tool: - package.json's prebuild step runs scripts/migrate-content.mjs, which imports renderArticle from render-article.mjs. Every build goes through this file. - deploy-only.yml, content-generate.yml, translate-archive.yml and runner-selftest.yml call migrate-content.mjs directly — so they reach render-article.mjs indirectly, through that import. - deploy/content-fast-deploy.sh — the build-free fast publishing path — runs the same line. - One of the two files importing @mdx-js/mdx is validate-mdx-staged.ts, one of the hard gates that blocks a commit. Note this: the name render-article.mjs appears in none of those workflows. Grepping for the link does not work; it sits behind one import line and only shows up if you follow the call chain. Had I run the upgrade and pushed, the blog would have stopped publishing new posts, and the first thing to tell me would have been a red deploy run. tsc stays silent, because this is not a type error. The tests stay silent, because none of the 17 test files under scripts/tests/ looks at dependency declarations. I read the release note wrong I do not want to be unfair to the release note. The 8.0.0 entry's main change opens like this: > Moves MDX file processing to the Markdown processors. … If you haven't > explicitly installed a Markdown processor, you don't need to do anything. And it continues — this is the part I skipped on first reading — by saying plainly: if you use unified(), move @astrojs/markdown-remark to 7.3.0 or later. So the note does address unified() users, and unified() is exactly what I use. What failed to warn me was not the note but the sentence I formed while reading it: I thought "I never installed a markdown processor", when in fact I had not installed one but was using it. The first sentence is still perfectly true for someone using Astro as Astro. MDX files are now delegated to the markdown processor; the default processor is Sätteri, and anyone who wants the remark/unified chain installs @astrojs/markdown-remark themselves — 8.1.3 now asks for it as a peerDependency, which is to say it expects you to declare it. My situation is different. I do not use that chain through Astro's configuration; I build the unified() chain in my own script and borrow the packages from Astro's node_modules. Astro telling me "you don't need to do anything" is correct, because Astro does not know I use those packages too. There is no way for it to know: I never declared them. There is an interesting contrast here with the dependency confusion piece. There the danger was that the name I asked for fetched the wrong package — right name, wrong source. Here the danger is the reverse: the name and the source are both right, but I never said I wanted it. In the first case an attacker does something; in the second nobody does anything, a maintainer tidies up their own dependencies, and my pipeline falls over. I wrote the gate, and it went red on my own repository The lasting fix has two parts: declare the missing packages, and stop them from accumulating silently again. The second part is forty lines: // scripts/tests/declared-imports.test.mjs const SCAN = ['scripts', 'src/lib', 'src/components', 'src/pages']; const EXT = new Set(['.ts', '.tsx', '.js', '.mjs', '.cjs', '.astro']); const SPEC = /(?:^|[\s;{(=])(?:import\s+(?:[\w*{},\s]+\s+from\s+)?|export\s+[\w*{},\s]+\s+from\s+|require\s*\(\s*|import\s*\(\s*)['"]([^'"]+)['"]/g; function walk(dir, out = []) { let ents; try { ents = readdirSync(dir); } catch { return out; } for (const e of ents) { const p = path.join(dir, e); if (statSync(p).isDirectory()) { if (e !== 'node_modules') walk(p, out); } else if (EXT.has(path.extname(e))) out.push(p); } return out; } test('every bare import must be declared in package.json', () => { const pj = JSON.parse(readFileSync('package.json', 'utf8')); const declared = new Set([ ...Object.keys(pj.dependencies || {}), ...Object.keys(pj.devDependencies || {}), ...Object.keys(pj.optionalDependencies || {}), ]); const offenders = new Map(); for (const base of SCAN) { for (const file of walk(base)) { for (const m of readFileSync(file, 'utf8').matchAll(SPEC)) { const s = m[1]; if (/^[./]/.test(s)) continue; if (/^(node|astro|virtual):/.test(s)) continue; const name = s.startsWith('@') ? s.split('/').slice(0, 2).join('/') : s.split('/')[0]; if (declared.has(name)) continue; if (!offenders.has(name)) offenders.set(name, []); offenders.get(name).push(file); } } } const lines = [...offenders].map( ([n, f]) => ` ${n} <- ${f[0]}${f.length > 1 ? ` (+${f.length - 1})` : ''}`, ); assert.equal( offenders.size, 0, `${offenders.size} undeclared packages are imported:\n${lines.join('\n')}`, ); }); Two details matter. src/content is not in the SCAN list — that is precisely what produced the wrong count of 32 above, the code blocks inside articles. And the skip list carries the node:, astro: and virtual: prefixes; those are not packages but a Node builtin, an Astro virtual module and a Vite virtual module respectively. I ran the gate against its own repository: not ok 1 - every bare import must be declared in package.json error: |- 13 undeclared packages are imported: node-html-parser <- scripts/_compare-render.mjs @mdx-js/mdx <- scripts/generate-content.ts (+1) js-yaml <- scripts/render-article.mjs ... 13 !== 0 The gate went red on its first run, which is what you want from a gate. If your first measurement after writing a rule comes back green, check your measurement rather than the rule. Declaring has a cost too The first part — declaring what is missing — sounds like a one-command job. You read the installed versions and write them into package.json: for p in js-yaml unified remark-parse remark-mdx remark-gfm \ remark-smartypants remark-rehype rehype-stringify \ unist-util-visit shiki github-slugger \ @mdx-js/mdx node-html-parser; do printf '%s@%s\n' "$p" \ "$(node -p "require('./node_modules/$p/package.json').version")" done The ./node_modules/ prefix is mandatory here. Write node -p "require('unified/package.json')" instead and eight of the thirteen lines fail with ERR_PACKAGE_PATH_NOT_EXPORTED: unified, the remark-* packages and rehype-stringify publish an exports map, and that map does not expose the ./package.json path. Reading the file by path sidesteps the gate. (I wrote this line without the prefix the first time, and eight lines of the output came back as stack traces.) There is a decision here, and I would like to pick the easy side but cannot: the moment I declare these packages I also own their upgrades. Right now shiki going from 4.0.2 to 4.6.0 is none of my business, because I use whatever Astro brings. Declared, it means thirteen new lines, thirteen new Renovate PRs and thirteen new "will this minor break something" questions. It is still the right trade. Because in today's arrangement those questions do not disappear, they are merely not asked of me — Astro's maintainers answer them on my behalf, and when their answers change I learn it from a red deploy run rather than a release note. When writing the versions I prefer exact pins over ^, because these thirteen are not tools I picked but tools I inherited; a range buys me nothing here. This repository already pins astro, @astrojs/mdx and @astrojs/node exactly, for a reason from the same family: upgrading those three separately can produce a build that passes tsc and then dies when the server boots. Before you write forty lines: existing tools and the structural fix I wrote the test above myself, but presenting it as "the right way" would be dishonest — especially in an article about a hand-rolled regex scanner giving the wrong answer. There are three real alternatives. A lint rule. eslint-plugin-import's import/no-extraneous-dependencies does exactly this job, and its documentation defines it as: "Forbid the import of external modules that are not declared in the package.json's dependencies, devDependencies, …". It works off a real parser, not my regex. Two conditions: the packages have to be installed, and type imports are ignored by default. If you already run ESLint, this is the way; this repository does not, which is why I wrote the forty lines. Inventory tools. depcheck and knip find both undeclared imports and the reverse direction — packages declared but never used. This repository has thirteen candidates in that direction too (@astrojs/check, tailwindcss, tsx and other config-driven packages), which is a good illustration of why such a tool still needs human judgement. The structural fix. The lasting answer is to change package managers. pnpm's documentation names the problem directly: "When installing dependencies with npm or Yarn Classic, all packages are hoisted to the root of the modules directory. As a result, source code has access to dependencies that are not added as dependencies to the project." pnpm builds a "non-flat" layout instead, symlinking only the project's direct dependencies into the root directory. In that tree my thirteen imports would have failed on day one, without waiting for an upgrade. If you take one thing from this article, look at your package manager before you write the gate test. The gate catches the symptom; the layout removes the problem. And there is a dimension that gets too little airtime: an undeclared package is invisible on the security side too, not only at upgrade time. The scope of npm audit, the SBOM you generate, the PRs Dependabot opens and the emergency version pin you would apply with overrides — all of them read the declaration list. If a package your code actually runs is missing from that list, a vulnerability advisory for it has no path to reach you. The cost of my thirteen packages is not thirteen Renovate PRs; it is thirteen unreviewed supply-chain items. Three questions to ask your own repository 1. How many packages are at the root of node_modules, and how many in package.json? Get the first with ls node_modules | grep -vc '^\.' and the second from the length of your declaration list. The gap is the number of packages you can import silently. That command says 515 for me while the lockfile counts 687 entries at the root; the difference comes from scoped packages, since everything under @astrojs shows up as a single directory. Count it either way, and there is a crowd standing next to my 20. 2. Which of your files are on the production path, and what do they import? An undeclared import in a helper tool is annoying; the same import on the deploy path stops publishing. To map the critical path, start from the lifecycle steps in package.json and from your workflows: # 1) lifecycle steps: prebuild, postinstall, prepare... node -p "JSON.stringify(require('./package.json').scripts, null, 2)" # 2) scripts called directly by CI and deploy grep -rhoE 'node scripts/[a-z0-9._-]+' .github/workflows/ deploy/ \ | sort -u # 3) everything those files import — packages AND neighbouring scripts grep -rhoE "from '[^']+'" scripts/migrate-content.mjs \ scripts/render-article.mjs | sort -u In the third step, do not filter out the relative paths; that is precisely the line showing the chain. For me, the from './render-article.mjs' line in migrate-content.mjs's output was the only clue leading to a file whose name never appears in any workflow. 3. Have you tried a major upgrade in an isolated directory before doing it? Two npm i calls and an import() loop take two minutes. For me those two minutes stood in for a publishing outage. A note on the Renovate dependency automation side as well: in a repository with automated upgrades enabled, this failure arrives through the merge of a PR you never looked at. Without a gate test, automation does nothing but accelerate this class of error. A frozen tree is a delay, not a protection This blog is fine today. Because npm ci reads the lockfile, and the lockfile reconstructs the 7.0.5 tree exactly — including the thirteen packages I never declared. The break arrives with the first command that regenerates that lock. With one exception. The server installs with npm ci --omit=dev, and node-html-parser is marked dev: true in the lockfile — the package that brings it, rettiwt-api, is a development dependency. So the production tree holds twelve of the thirteen, not thirteen; the script using the thirteenth cannot run on the server anyway. I was not looking for this; it fell out while reading the lockfile for something else entirely. Every package you leave undeclared turns into a delayed question like that one. I want to state the distinction plainly, because I spent years treating the two as the same thing: a lockfile is a freezer; a declaration is a contract. The freezer carries yesterday's tree into today; it does not tell you why anything is in there. The contract says "I want this package", and if nobody brings it tomorrow, npm stops and tells you. My thirteen packages were in the freezer and absent from the contract. The day I opened the freezer I was going to learn all of their freshness at once. The good news is that this is a measurable gap. It is not a code smell or an architectural preference; it is a hole you can find and close by comparing two lists. On my side the next job is clear — declare the thirteen packages with exact versions and move the gate into scripts/tests/. The upgrade comes after that; in the universe where I reversed the order, this article would be the one the blog could not publish. Official Sources The @astrojs/mdx version comparison in the body is based on registry data for 7.0.5 and 8.1.3 read on 6 November 2026; the list below points to the current form of the same documents. - Node.js ESM — PACKAGE_RESOLVE and bare specifier resolution - withastro/astro — @astrojs/mdx CHANGELOG, the 8.0.0 entry - withastro/astro#17262 — moving MDX processing to the Markdown processors - Astro Docs — @astrojs/mdx, the processor option and the Sätteri default

全球AI赚钱案例 超简单单页建站，一人曾公开到约 72 万/月量级（约 2022 访谈档，年化约 860 万语境）。 Carrd：AJ 把「一个页面就够」做到极致，年费很低也能养活一人公司——少功能、少硬广、靠口碑。 同类开源可收藏：Astro——免费自己搭内容站/落地页，静态优先、很快。 withastro/astro Solo one-page builder ~¥720k/mo class figures discussed ~2022. 落地页你选极简付费，还是自己静态生成？

@puruvjdev @npmjs Congratulations! I was curious about the neotraverse stats. So, had a quick look and noticed Astro uses one of its functions, might explain its big usage 👍 github.com/withastro/astro/blob/main/packages/ast…

The @astrodotbuild LLM factory works great. Found a bug, clanker wrote a simple repro + short description, I then (manually) filed a ticket. Their bot wrote a fix in ~20 mins. Few days later, after @matthewcp approved the changes, fix was merged in ~3 h. astro#18507

Building a blog shouldn't feel like building a whole app. 🌐 Astro is designed for content-focused sites, with Markdown and UI framework support. Great for portfolios and docs. 🚀 withastro/astro #WebDev #浇朋友 #蓝朋友 #互关 #fo

搭博客还要先搞一整套复杂前端？🌐 Astro 适合内容型网站，支持 Markdown 和多种前端组件。建作品集或文档站可以试试。🚀 withastro/astro #WebDev #浇朋友 #蓝朋友 #互关 #fo

only tangentially related, but a few weeks back I was trying to build an astro site as an SEA roadmap#1442 yeah maybe astro is overkill for my use-case (digital signage kiosk app), but having a single binary would be useful in a variety of cases, same as with CLI apps

roadmap#1450 Composition: so hot right now

Proposal to Break Up Astro! Matt is drafting a proposal to break Astro into composable Vite plugins. This could get interesting. 👀 roadmap#1450 Short version: right now Astro is like a set meal. You get the whole combo whether you want every item or not. This proposal would let you order a la carte instead. But it only got accepted yesterday as an idea (Stage 2, no RFC, no code yet), so you can’t use any of it today.

Happy spooky season @astro.build It wasn't immediately obvious to me why Houston had a bone... github.com/withastro/astro/blob/c75e89711a4935a0b… At least now I know it's a tie 💀