nginx

I Asked for 499 Bytes, 51,101 Came Back This block sits in my blog's nginx configuration: location /_astro/ { proxy_pass http://127.0.0.1:3040; expires 1y; add_header Cache-Control "public, immutable"; } So even the site's static assets don't come off disk — they pass through the Node process, and nginx merely carries them. I knew that, and still, reading the config today, a question came up that I couldn't answer: if a client asks for only part of that file, who answers? There are three candidates — Node, nginx, and Cloudflare out front — and the request touches all of them. Asking with curl took five seconds. The table that came out wasn't what I expected: my own stack answers this question with a single voice, while the edge in front of it runs a different rulebook altogether. The most expensive answer the edge gave was this one — I asked for 499 bytes and 51,101 came back. That this piece opens on proxy_pass is no accident: over the past three weeks my measurements of open_file_cache and gzip_static both snagged on the same line. This is the third. What snags this time isn't a tuning knob, it's a defense nginx added this year. First measurement: the page doesn't do ranges, the asset does Two requests, same site. The first is an article page, the second is the CSS that page loads. I asked both for their first 100 bytes. # the article page curl -s -o /dev/null -w '%{http_code}/%{size_download}\n' \ -H 'Accept-Encoding: identity' -H 'Range: bytes=0-99' \ https://mustafaerbay.com.tr/blog/tutorials/node-u-tek-dizine-kilitledim-bes-kapi-acik-kaldi/ # 200/269938 # that page's CSS curl -s -o /dev/null -w '%{http_code}/%{size_download}\n' \ -H 'Range: bytes=0-99' \ https://mustafaerbay.com.tr/_astro/BaseLayout.CGi080bn.css # 206/100 The page answered my 100-byte request with 269,938 bytes — 2,699 times what I asked for. The CSS sent exactly what I asked: 100 bytes, with a `Content-Range: bytes 0-99/114436` header beside it. Same server, same certificate, same second: one of them has no notion of ranges, the other does. The difference shows up in the headers too. The CSS response carries Accept-Ranges: bytes; the page response doesn't. HTTP is lenient here — a server may ignore the Range header field, and RFC 9110 says so explicitly. So my page is standards-compliant. What interested me is why it behaves that way. One missing header explains the page The answer is inside my own server. On the VPS I asked the Node process directly: curl -s -D- -o /dev/null -H 'Range: bytes=0-99' \ 'http://127.0.0.1:3040/blog/tutorials/node-u-tek-dizine-kilitledim-bes-kapi-acik-kaldi/' # HTTP/1.1 200 OK # Transfer-Encoding: chunked Astro's SSR output streams the page: the body length isn't known when the response starts, so there is no Content-Length and there is Transfer-Encoding: chunked. nginx's range filter, for its part, wants three things before it does anything — a 200 status, a known content_length_n, and the allow_ranges flag. [The filter's entry condition](github.com/nginx/nginx/blob/cb9252ae6537d8c24addc…) hands the response straight on when the length is unknown. I wanted to verify that in a lab too, because I didn't want to write "because of a missing header" without measuring it. I ran nginx 1.31.8 in a container on my Mac and put a tiny two-ended Python server behind it: the /chunked path streams in chunks, the /cl path serves the same body with a Content-Length. As a fourth variable I also tried the proxy_force_ranges directive. | path | upstream | proxy_force_ranges | result | |---|---|---|---| | /p-chunked/ | chunked | off | 200, whole body | | /p-cl/ | Content-Length | off | 200, whole body | | /p-chunked-force/ | chunked | on | 200, whole body | | /p-cl-force/ | Content-Length | on | 206, 100 bytes | Four rows, one summary: nginx does not slice a proxied response by itself. It takes two conditions together — the upstream saying Accept-Ranges (or you writing proxy_force_ranges on) and a Content-Length on the response. Without the length, the directive doesn't save you either. My pages ignoring ranges isn't a misconfiguration, it's the price of streaming. I think that price is fair: no real client asks for 100 bytes from the middle of a page. Multiple ranges: the edge manufactures multipart Things get strange not with one range but with many. HTTP lets a client ask for several pieces in a single request, and the server answers in multipart/byteranges form, wrapping each piece in its own little header. curl -s -D- -o /dev/null -H 'Range: bytes=0-9,20-29,40-49' \ https://mustafaerbay.com.tr/_astro/BaseLayout.CGi080bn.css # HTTP/2 206 # content-type: multipart/byteranges; boundary=a331107a9f3d76fe # content-length: 346 I asked for 30 bytes and 346 came back. The surplus isn't data, it's envelope: each piece gets a boundary string, a Content-Type and a Content-Range line in front of it. At this scale nobody cares. But the envelope has a fixed length while the requested piece doesn't — so the ratio falls apart as you shrink the pieces and raise their count. So I broke it: I asked for 499 separate bytes in one request. R=$(python3 -c "print(','.join(f'{i*7}-{i*7}' for i in range(499)))") curl -s -o /dev/null -w '%{http_code}/%{size_download}\n' \ -H "Range: bytes=$R" https://mustafaerbay.com.tr/_astro/BaseLayout.CGi080bn.css # 206/51101 I asked for 499 bytes and 51,101 bytes came down. Same number on all three runs. The payload grew 102-fold; I paid 101.4 bytes of envelope per piece. When I ran the same experiment with 200-byte pieces, the number went somewhere even funnier: I asked for 99,800 bytes and got 151,707 — while the entire file is 114,436 bytes. Asking for the pieces cost 37 KB more than downloading the whole thing. Then I went up to 500 and it was over: # 500 x 1 byte -> 200/114436 # 500 x 200 bytes -> 200/114436 # 499 x 200 bytes -> 206/151707 The edge's rule is clear: yes up to 499 ranges, and at 500 it behaves as though it never saw the Range header and sends the whole body. The size of the pieces doesn't change the decision at all — this is a count limit, not a budget. Whether you ask for one byte or 200 per piece, the line is at 499. What would my own server say? Everything past this point would have been guesswork, so instead of guessing I asked the server. The origin's 443 is open to Cloudflare only, but from inside the VPS I can reach my own nginx with --resolve. Same requests, with the edge out of the loop: | request | Node (:3040) | origin nginx 1.30.0 | Cloudflare edge | |---|---|---|---| | single range, 100 bytes | 206 / 100 | 206 / 100 | 206 / 100 | | 3 ranges | 200 / 114,436 | 200 / 114,436 | 206 / 346 (multipart) | | 499 × 1 byte | 200 / 114,436 | 200 / 114,436 | 206 / 51,101 | | article page + range | chunked, 200 | 200 / 268,824 | 200 / 269,938 | The first thing that jumps out is that the first two columns are identical: whatever Node says, nginx carries, on all four rows. So this isn't "three layers, three answers" — it's "my own stack speaks with one voice, the edge with another". Every multipart response in these tests came from Cloudflare; my own stack can't do a multi-range request at all, because it delegates ranges to Node, and the static file server Astro uses (send 1.2.1) only knows single ranges: its ranges.length === 1 condition treats a multi-range request as a "regular response" and sends the whole body. Second: the same page is 268,824 bytes at the origin and 269,938 at the edge. The 1,114-byte difference is most likely the cached copy being an older render; I didn't chase it, that wasn't today's question. !Diagram What nginx alone would do: one budget, three predictions What if nginx served the file itself? Trying that at the origin would have meant changing the configuration; instead I put a file of the same size into the lab container (114,436 bytes) and raised the range count. n=453 -> 206/43294 n=454 -> 206/43390 n=455 -> 200/114436 At 454 it produces multipart; at 455 it ignores the Range header. An odd number. It has nothing to do with header length: by requesting pieces at offsets above 100,000 I pushed the header to 6,369 bytes and the break stayed at 455. So the decision isn't made by the count of ranges but by something else. The answer is in the range filter's source, in the last ten lines of ngx_http_range_parse(): if (max_ranges == NGX_MAX_INT32_VALUE) { size += ctx->ranges.nelts * 256; content_length += 4096; if (size > content_length) { return NGX_DECLINED; } } size is the total of the requested bytes, nelts the number of ranges. nginx assumes 256 bytes of envelope per piece, grants the source 4 KB of slack, and refuses a request that comes close to the whole body. So it's a budget: requested bytes + 256 × number of ranges > file size + 4096 means there are no ranges. I tested the formula on two more file sizes, computing the break point first and measuring afterwards: | file | range size | prediction | measurement | |---|---|---|---| | 114,436 bytes | 5 bytes | 454 accepted / 455 refused | 454 accepted / 455 refused | | 10,000 bytes | 5 bytes | 54 accepted / 55 refused | 54 accepted / 55 refused | | 1,048,576 bytes | 10,000 bytes | 102 accepted / 103 refused | 102 accepted / 103 refused | A direct hit all three times. This budget hasn't always been in nginx: it arrived with commit cb9252ae on 13 January 2026, titled "Range filter: reasonable limit on multiple ranges". The release notes list it as a bugfix — 1.29.5, 4 February 2026: "a response with multiple ranges might be larger than the source response". The 1.30.0 at my origin and the 1.31.8 in my lab both come after that fix. The trap: writing max_ranges cancels the budget Look at that if condition again: the budget only runs while max_ranges is at its default. The moment you set the directive to a number, the code never enters that block and all you have left is a flat count limit. Here is what came out of the lab: default n=455 -> 200/114436 (budget refused it) max_ranges 1000 n=455 -> 206/43486 (no budget, accepted) So writing a generous max_ranges is weaker than not writing it at all. nginx's documentation still says "By default, the number of ranges is not limited" — true as a count, but since 1.29.5 the default carries a size budget, and writing the directive takes it away. The commit message says the limit "can be altered with the max_ranges directive", which is technically right: you alter it, and a flat count takes its place. There's an important distinction here, so don't skip it: writing max_ranges doesn't leave you defenseless. Right above the envelope-aware budget sits a cruder check that has been in nginx since 1.1.2 — if the sum of the requested bytes exceeds the file itself, the request is refused — and that check lives outside the max_ranges block. With max_ranges 1000 in place, the line falls here: n=572 requested 114,400 bytes (file 114,436) -> 206/167597 n=573 requested 114,600 bytes -> 200/114436 The break is exactly at the file's size. So with the directive written, the ceiling of amplification becomes "file size + piece count × envelope": that 572-range request pulled 167,597 bytes out of a 114,436-byte file, 1.46 times over. The envelope budget had already cut the same request off at 400 ranges. Your three real options behave like this (all measured): - default: a single range always passes; multiple ranges pass up to the budget. - max_ranges 1: single range 206, multiple ranges 200. Resumable downloads and video seeking keep working; multipart closes entirely. - max_ranges 0: the Accept-Ranges header is never sent and every response is a 200. There's a third limit, and I hit it by accident. Generating ranges as 0-4,10-14,20-24… (f'{i*10}-{i*10+4}') and raising the count with max_ranges 1000 in place, the response flipped to 400 at range 841: n=840 Range line 8,190 bytes -> 206 n=841 Range line 8,200 bytes -> 400 The cause isn't the ranges but the request itself: the Range line no longer fits the default large_client_header_buffers (4 8k). That's where the natural ceiling of range-based amplification sits — the request has to fit into a single header line. What's in the envelope, and what nginx assumes Opening up a two-range response byte by byte shows what the tax actually is: \r\n--f822d9a65821054d\r\n Content-Type: text/css; charset=utf-8\r\n Content-Range: bytes 0-9/114436\r\n\r\n @keyframes \r\n--f822d9a65821054d\r\n ... Ninety-six bytes of envelope for ten bytes of data. The averages came out at 101.4 bytes per piece on the edge and 90.6 bytes on the lab's nginx. All 10.8 bytes of that gap come from the Content-Type line — the edge appends ; charset=utf-8, the local nginx doesn't. The boundary string actually works the other way: nginx's boundary is a zero-padded twenty-character number, the edge's a sixteen-character hex string. nginx's budget, meanwhile, assumes 256 bytes per piece, two and a half times reality. The envelope isn't quite constant either: it grows with the digit count of the offsets in each Content-Range line. Pieces asked for at the end of a file cost a few bytes more than pieces at the beginning. That generosity isn't accidental. For the same file, the break point moves like this depending on the assumption: | assumed envelope | break point | |---|---| | 91 bytes (actual) | 1,234 ranges | | 101 bytes | 1,118 ranges | | 256 bytes (nginx) | 454 ranges | So nginx also refuses requests it could have served without trouble. The commit message says as much: a client asking for small pieces of a large source is penalized, because there the processing cost dominates the data. That reads right to me — a browser has no reason to ask for 450 separate bytes. The moment I slice, I lose compression A side effect I wasn't looking for turned up here. When I ask for the same CSS with brotli accepted, 20,634 bytes come down. The moment I add a Range header: curl -s -D- -o /dev/null -H 'Accept-Encoding: br' -H 'Range: bytes=0-99' \ https://mustafaerbay.com.tr/_astro/BaseLayout.CGi080bn.css # HTTP/2 206 # content-length: 100 # content-range: bytes 0-99/114436 The content-encoding header disappears and the total in Content-Range is 114,436 — the length of the uncompressed body. Ranges are served over the raw representation, not the compressed one. A client that tries to fetch this file piece by piece carries 5.5 times what brotli would have cost. An earlier measurement turned on which representation compression picks; once ranges enter the picture, that choice is cancelled outright. A single range is privileged, and a stale copy gets the whole body One last detail about the budget: if the range count is exactly one, the source returns NGX_OK without consulting the budget at all. So a single-range request — a resumable download, seeking in a video, pulling the last page of a PDF — passes at any size and on any file. Asking for 100 bytes of the 1 MiB file gave me a 206 too. Alongside that path there is a validator. A client holding half a copy can say "continue where I left off if my copy is still valid" with If-Range: ET=$(curl -sI https://mustafaerbay.com.tr/_astro/BaseLayout.CGi080bn.css \ | grep -i '^etag' | sed 's/etag: //I') curl -s -o /dev/null -w '%{http_code}/%{size_download}\n' \ -H 'Range: bytes=0-99' -H "If-Range: $ET" .../BaseLayout.CGi080bn.css # 206/100 <- tag matches, the piece arrives curl -s -o /dev/null -w '%{http_code}/%{size_download}\n' \ -H 'Range: bytes=0-99' -H 'If-Range: "bogus-etag"' .../BaseLayout.CGi080bn.css # 200/114436 <- tag didn't match, the whole body arrives Getting a 200 rather than a 412 on a non-matching tag is by design: the server sends the new body directly so the client doesn't have to make a second request. In practice that means a client trying to finish a half-finished download will quietly pull the entire file if the asset changed during a deploy. Files under /_astro/ carry a content hash in their name, so this doesn't happen to me; it happens if you serve a file with a stable name. Two requests the edge won't take RFC 9110 grants the server a right: it may reject overlapping ranges, or a set of many small ranges that aren't listed in ascending order, since both point at either a broken client or a deliberate request. The standard advises this without requiring it. Each layer got the question separately. | request | lab nginx | Cloudflare edge | |---|---|---| | bytes=0-99,200-299 (ascending) | 206 | 206 | | bytes=200-299,0-99 (unsorted) | 206 / 403 bytes | 200 / whole body | | bytes=0-199,100-299 (overlapping) | 206 / 604 bytes | 200 / whole body | | bytes=999999-1000099 (past the end) | 416 | 416 | The edge follows the RFC's advice and nginx doesn't: unsorted or overlapping, it slices and sends. On 416 they agree — the response carries Content-Range: bytes */114436, telling the client the file's real size. That table also explains the 499-range measurement above: the bytes requested were in ascending order, which is why the edge took them. Shuffled, the same 499 ranges would have brought back the whole body without a single slice. The decision for my own setup Here's the table I'm left with: my origin's nginx is on a post-fix version and carries the budget, but the budget never fires — because Node already refuses the multi-range request, and the face turned outward is Cloudflare, whose limit is a count of 499. I don't own the defense. So I changed nothing on the origin side today, and I'm not going to set max_ranges to a number — the moment I write it I lose the only smart limit I have. The real limit is at the edge and I have no knob there; my one lever, if it ever matters, is lowering the range count at the edge. The scale deserves to stay in the right place too. 102-fold sounds bad, but the absolute figure is 51 KB; the amplification is bounded by the file size and the request has to fit in a single header. This is a tax, not a weapon. Still, it's good to know it isn't free: while marking the files under /_astro/ as immutable for a year, I had never thought about how they get sliced. You can measure your own stack from the same spot with four commands: # 1) does the page do ranges? curl -sI https://<site>/ | grep -i accept-ranges # 2) does a static asset serve a single range? curl -s -o /dev/null -w '%{http_code}/%{size_download}\n' -H 'Range: bytes=0-99' https://<site>/<asset> # 3) is multipart produced? curl -sI -H 'Range: bytes=0-9,20-29' https://<site>/<asset> | grep -i content-type # 4) up to how many ranges? R=$(python3 -c "print(','.join(f'{i*7}-{i*7}' for i in range(499)))") curl -s -o /dev/null -w '%{http_code}/%{size_download}\n' -H "Range: bytes=$R" https://<site>/<asset> If the second request returns 206, slicing is on; if the third says multipart/byteranges, you're paying the envelope tax; if the fourth returns 206, the layer in front of you doesn't honour nginx's budget. What this measurement taught me isn't really about ranges. Every layer of my stack can answer the same request differently, and I only get to see which answer counts by interrogating the layers one at a time. That ten-line budget nginx added in January sits on my server — unused. It's nice when defaults improve; whether that default ever sees your request is a separate question. The gzip_static measurement taught me the same lesson more cheaply: a static file module never touches a response coming through proxy_pass — writing that directive would not have affected a single path carrying traffic. This time the thing that doesn't touch my traffic isn't a speed knob, it's a defense. Official Sources - nginx — the max_ranges directive (ngx_http_core_module) - nginx CHANGES — 1.29.5 (4 February 2026) multi-range response fix - nginx commit cb9252ae — "Range filter: reasonable limit on multiple ranges" - RFC 9110 — the Range header, a server's right to ignore it (§14.2) and the security note on multipart range requests (§17.15)

Keepalive Is On by Default, 100 Requests Opened 200 Connections I ran a two-line measurement on my own server. First I counted the TIME_WAIT sockets pointing at port :3040, then I sent 100 requests to one of this blog's endpoints, then I counted again: N=20 requests -> TIME_WAIT(:3040) delta=44 N=50 requests -> TIME_WAIT(:3040) before=10 after=112 delta=102 N=100 requests -> TIME_WAIT(:3040) before=112 after=312 delta=200 A hundred requests, two hundred fresh sockets. And yet the nginx on that server is 1.30.0 — one of the versions where upstream keepalive is supposed to be on by default. The pool should have been open, connections should have been reused, these numbers should not exist. They do. The reason is that I had misread exactly what the change that landed in nginx on 24 March 2026 covers — and what it does not. And I did not keep that misreading to myself; I published it. In my TIME_WAIT article back in July I listed the three 1.29.7 changes correctly and then added that the 1.30.0 on my server "has this behaviour". The version was right, the sentence was incomplete: that behaviour arrives for an upstream block, not for a server. What follows is the record of me refuting my own sentence. What changed on 24 March The nginx 1.29.7 changelog sums it up in three lines: *) Feature: the "local" parameter of the "keepalive" directive in the "upstream" block. *) Change: now the "keepalive" directive in the "upstream" block is enabled by default. *) Change: now ngx_http_proxy_module supports keepalive by default; the default value for "proxy_http_version" is "1.1"; the "Connection" proxy header is not sent by default anymore. For years there was a three-line recipe copied into every nginx config: keepalive 32 in the upstream block, proxy_http_version 1.1 and proxy_set_header Connection "" in the location. Without all three, the upstream pool did nothing. 1.29.7 made that recipe the default: the documented default for keepalive is now keepalive 32 local, and for proxy_http_version it is 1.1. So the thing older articles tell you to "always add" is already there on new versions. Written from memory, the recipe can now produce a different result than you expect — I will come back to that. How I count whether the pool works I did not want to publish claims without measuring them first, so a small lab went up: two nginx containers, one proxy and one backend. The trick is in the backend's log format: log_format up '$connection|$http_connection|$server_protocol|$uri'; access_log /dev/stdout up; $connection is nginx's connection serial number. Every request arriving over the same TCP connection carries the same number. So the count of unique $connection values in the backend's log is the number of TCP connections the proxy opened. I count requests from the same log, which lets me divide one figure by the other. Two free extras ride along: $http_connection is the Connection header the backend received, and $server_protocol is the HTTP version the request arrived with. Together, those three explain in a single line why the pool did or did not open. The driver side is one curl call with a braced range — 200 requests: curl -s -o /dev/null "http://ka-px/req?i=[1-200]" The measurement: two versions, ten configurations Each row is 200 requests. The number is the count of TCP connections the backend saw for those 200 requests. I ran v1, v2 and v4 three times each; all three runs produced identical figures, so I quote a single number. The remaining rows are single runs — I did not repeat those three times. | # | Configuration | nginx 1.31.8 | nginx 1.28.3 | |---|---|---|---| | v1 | proxy_pass http://up:8080 — no upstream block | 200 | 200 | | v2 | upstream block present, no keepalive written | 1 | 200 | | v3 | Classic recipe: keepalive 32 + 1.1 + Connection "" | 1 | 1 | | v4 | upstream + keepalive_requests 10 | 20 | — | | v5 | upstream + proxy_http_version 1.0 | 200 | — | | v6 | upstream + keepalive 0 | 200 | — | | v7 | proxy_pass straight to an IP, no upstream block | 200 | — | | v8 | upstream + keepalive 32 local (default written by hand) | 1 | — | | v9 | Two locations sharing one upstream, no directive | 2 | 200 | | v10 | Two locations, keepalive 32 written by hand | 1 | — | The headers the backend saw changed with the version too: | | protocol | Connection header | |---|---|---| | 1.31.8, default | HTTP/1.1 | not sent | | 1.28.3, default | HTTP/1.0 | close | | 1.31.8, explicit proxy_http_version 1.0 | HTTP/1.0 | not sent | That second row shows which rule from RFC 9112 section 9.3 kicks in: a recipient decides whether a connection is persistent from the version and the Connection header of the most recent message. If close is present, the connection will not persist; if the protocol is HTTP/1.0 without keep-alive, it will not persist either. 1.28.3 did both at once. 1.31.8 stopped doing both. The default is not for everyone — only for those who wrote an upstream block v1 and v7 are the ones that stung. On nginx 1.31.8, writing proxy_pass http://up:8080 with no upstream block gives 200 connections for 200 requests. Writing an IP instead of a name also gives 200. The protocol is HTTP/1.1 and no Connection header is sent — so half the version change is clearly in effect — but there is no pool. I did not want to guess at this, so I read the source. In ngx_http_upstream_keepalive_module.c, the defaults are applied in ngx_http_upstream_keepalive_init_main_conf(), and the first thing it does is this: /* skip implicit upstreams */ if (uscfp[i]->srv_conf == NULL) { continue; } The default is applied while looping over the upstream list in the configuration, and implicit upstreams are skipped. When you write proxy_pass http://host:port, nginx builds an upstream for you behind the scenes; that upstream has no srv_conf, because no upstream { } block was ever written for it. The loop skips it, max_cached stays unset, and the pool never opens. Further down the same loop there is one more gate, and then the defaults: if (kcf->max_cached == 0) { continue; } ngx_conf_init_msec_value(kcf->time, 3600000); ngx_conf_init_msec_value(kcf->timeout, 60000); ngx_conf_init_uint_value(kcf->requests, 1000); if (kcf->max_cached == NGX_CONF_UNSET_UINT) { kcf->local = 1; kcf->max_cached = 32; } That middle continue explains row v6: write keepalive 0 and max_cached becomes zero, the loop leaves there too, and no pool is built. So one function holds three separate gates — implicit upstreams, keepalive 0, and an unset max_cached. One hour, 60 seconds, 1000 requests, 32 connections: all four are here. But note they are not under the same condition. The timeouts and the 1000 are assigned unconditionally; only local = 1 and max_cached = 32 sit inside the if. Writing keepalive by hand does not change the 1000 or the 60 seconds — it only takes the 32 and the local away from you. I downloaded the file at three release tags: release-1.29.7 and release-1.31.8 are byte-identical (16,146 bytes), while release-1.28.3 is 14,923 bytes and contains neither that if nor the implicit upstream skip. Reduced to one sentence: on nginx after 1.29.7 upstream keepalive is free, but only if you wrote an upstream { } block. Not writing a block is not the same as accepting the default; it is writing in a place the default never looks. Your own keepalive 32 is not the same thing as the default v9 and v10 are a subtler trap. Two separate locations use the same upstream, and I split the 200 requests between them: - With nothing written: 2 connections. - With keepalive 32 written: 1 connection. That looks backwards at first. Does writing the setting by hand make things better? No — it makes them different. Go back to the code above: the local = 1 assignment only happens if max_cached was unset. So the default is keepalive 32 local, while what you wrote by hand is keepalive 32 — without local. The local parameter (new in 1.29.7) separates the pool per location; hence two locations, two pools, two connections. Without local, every location pointing at that upstream shares a single pool. There is no "correct" option here, only a trade-off. A shared pool opens fewer connections. Separate pools stop a connection established with one location's settings — a different proxy_set_header, a different timeout, a different certificate — from being handed to another location. nginx picked the second, cautious option as the default. I think that is the right call: a feature that arrives for free should not produce surprises, and that matters more than saving one socket. The practical consequence: if the keepalive 32 line is still sitting in your old config, your behaviour differs from the default once you move to 1.29.7. Deleting that line is not a no-op. The pool is not infinite: where 1000 comes from The run with keepalive_requests 10 gave exactly 20 connections for 200 requests. The division explains itself: each connection carries 10 requests and closes. Because the default is 1000, you will not notice that recycling in daily life — but that 1000 exists for a reason, and the nginx docs state it plainly: closing connections periodically is necessary to free per-connection memory allocations, and setting the number too high is not recommended because of memory use. The same page holds one more line, and to my mind it is the most misread sentence in it: the keepalive directive does not limit the total number of connections an nginx worker can open to the upstream. 32 is the number of idle connections kept in the cache. If 300 concurrent requests arrive at peak, 300 connections are opened; when the work is done at most 32 are kept and the rest are closed. "I set keepalive 32, so at most 32 connections reach my backend" is simply not true. Worth noting too: the default for keepalive_requests was 100 before 1.19.10. If you find 100 in an older article, the article is not wrong — just old. Where the documentation contradicts itself One thing stopped me mid-measurement: on 1.31.8 the backend never sees a Connection header at all. Yet nginx's proxy_set_header page, as I read it today, still lists the defaults like this: proxy_set_header Host $proxy_host; proxy_set_header Connection close; What is more, a few screens up the same page, under proxy_http_version, it says "Since 1.29.7, version 1.1 is used by default". So the page knows about 1.29.7; the proxy_set_header box does not. The page contradicts itself. The changelog says the Connection proxy header is no longer sent by default; the default box says close. Both cannot be true at once, so I went to a third witness — the source. The default header table in ngx_http_proxy_module.c reads like this on 1.28.3: { ngx_string("Connection"), ngx_string("close") }, and like this on 1.29.7: { ngx_string("Connection"), ngx_string("") }, In nginx an empty value means "do not pass this header". In the same file the default for proxy_http_version has moved from NGX_HTTP_VERSION_10 to NGX_HTTP_VERSION_11. So the source and the changelog agree with each other, my measurement is on their side, and the only dissenting witness is the default box on a documentation page. Lining up five tags shows the second line of that same box has gone stale too: the Host value is $proxy_host on 1.28.3, $proxy_internal_host on 1.29.7 and 1.30.0, and empty on 1.31.8. The $proxy_host the documentation prints exists in no version today. One caveat while you are here: do not file Proxy-Connection under 1.29.7 — it was added to that table in 1.31.0 and is not part of the 1.29.7 entry. My point here is not that nginx's docs are bad — you will rarely find a server documented this well. The issue is that in a release where defaults move, not every corner of the documentation moves at the same speed. Collecting three witnesses is still the cheapest route. So what about my blog Back to the start. My server runs nginx 1.30.0, so the change is in there. But when I looked at my configuration I found this: six separate locations, every one of them with proxy_pass http://127.0.0.1:3040, and not a single upstream block anywhere. Exactly the situation in v1. Rather than infer from the version number, I put my server's exact version in the lab. On nginx 1.30.0, 200 requests opened 200 connections without an upstream block and 1 with one. So the change is present on my server; what is missing is a configuration that reaches it. A small sourcing note: do not look for the stable 1.30 branch in the main CHANGES file — it holds no 1.30 entries at all. The stable branch keeps its own changelog in CHANGES-1.30. I also worked out why the figures came to double, because the thing listening on :3040 is not the Node process: LISTEN 0 65535 127.0.0.1:3040 0.0.0.0:* users:(("docker-proxy",pid=1091705,fd=8)) The blog runs in a container, and the published port puts docker-proxy in the middle. Two separate rows pile up in the TIME_WAIT list: one for 127.0.0.1:random -> 127.0.0.1:3040 (nginx to docker-proxy) and one for 172.31.0.1:random -> 172.31.0.2:3040 (docker-proxy to the container). So an unpooled request costs me not one socket but two, and that is where 100 requests becoming exactly 200 comes from. The extra +2 and +4 on the 50- and 20-request runs I did not isolate; most likely it is real traffic arriving at the site, but I did not measure that. So base the "two sockets per request" ratio on the 100-request run, not on the short ones. !Diagram The obvious question: does adding an upstream block help the second hop too, or only collapse the first? Rather than guess, I rebuilt that same architecture in the lab — ran the backend container with a published port so docker-proxy sat in the middle, sent 100 requests, and counted the connections the container itself saw: | Proxy configuration | Connections the container saw | |---|---| | proxy_pass http://host:18080 (no block) | 100 | | upstream block (no directive written) | 1 | I ran it twice, with the same result twice. Pooling the first hop collapses the second as well, which makes sense: docker-proxy pairs an inbound connection with the connection it opens to the backend for that connection's lifetime — one inbound connection means one outbound. Here is where I owe you honesty: I have not applied this fix to the live server yet. My measurement holds in the lab and on a copy of the blog's architecture; I did not want to change a live configuration just to write an article. Six locations on the blog point at the same backend, which drags local behaviour into the picture too, and changing this config is a maintenance-window job in its own right. If you are a step ahead of me as you read this, congratulations — I have put it in the queue. Writing the fix, and what it hits The fix for everything described so far is three lines. Before: location / { proxy_pass http://127.0.0.1:3040; } After: upstream blog_backend { server 127.0.0.1:3040; } location / { proxy_pass http://blog_backend; } That is all of it — you do not need to write keepalive, because on a post-1.29.7 version the default takes over once the block exists. But know three things before making that change. First, name resolution. If your backend is a name rather than 127.0.0.1 — app, web, a Docker service name — moving that name into an upstream block makes nginx resolve it only at startup. When the container is recreated with a new IP, nginx keeps talking to the old one; that is exactly the failure I wrote up on this blog as nginx's DNS trap. With an IP you are fine; with a name, look into resolver and a variable proxy_pass. Second, the other side's timeout. A pool is a two-party agreement. If your backend closes idle connections sooner than nginx expects, nginx will pull a closing connection out of the pool, write a request onto it, and you will see occasional 502s. The nginx-side default is 60 seconds; find out your backend's and keep it shorter. This is the first failure anyone who has just enabled a pool meets. Third, the local multiplication. Because the default pool is kept per location, the idle-connection ceiling is number of locations × 32, and that is per worker. For my six locations that means 192 idle connections per worker, multiplied again by worker_processes. Written by hand as keepalive 32 — that is, without local — they all share one pool and the ceiling stays at 32. That number is also why I have not rushed the fix on the blog: whether to point six locations at one shared pool or accept the default and raise the ceiling is not a decision to make while writing an article. There is a reverse direction too: installations that upgrade to 1.29.7 and already have an upstream block get the pool switched on by itself. If that is not what you want, the way out is keepalive 0, row v6. Checklist The order I followed, if you want to measure the same thing on your own proxy: 1. Learn your version. nginx -v. Older than 1.29.7 and the classic three-line recipe is still mandatory; newer and it is already there — but keep reading the condition. 2. Check whether an upstream block exists. grep -n "upstream \|proxy_pass" conf. If it goes straight to proxy_pass http://host:port, there is no pool whatever your version. 3. Count, do not guess. If the backend is nginx, log $connection and count unique values. Otherwise look at the delta of ss -tanH state time-wait | awk '{print $4}' | grep -c ":<port>$" on the proxy side — but first see who is listening on that port with ss -tlnp, because if a proxy sits in between, the numbers multiply. 4. Account for caching. My first attempt produced zero difference across 50 requests, because they came back from proxy_cache and an HTTPS redirect and never reached the backend at all. Pick a path that genuinely traverses the proxy. 5. Choose keepalive_requests deliberately. The default is 1000. Lowering it improves memory behaviour but increases connection churn; raising it costs memory. 6. Know that your old keepalive 32 line is no longer local. Its behaviour differs from the default; add local if that is not what you wanted. Conclusion What surprised me most while writing this is that the measurement did not lead me somewhere wrong — it led me to my own configuration. The change nginx made is a good one: turning a decade-old copy-paste recipe into the default protects the person who never heard the recipe most of all. But whether a default actually reaches you is a finer question than the default itself. My configuration was living on the assumption "I wrote no setting, so I must be getting the default", and because of a srv_conf == NULL check it never entered the default's field of view at all. Put more generally: when a release note says "now the default", the unanswered question is always default for whom. Learning that from a measurement rather than from documentation — and confirming it with a single continue in the source — was the best twenty minutes of my week. Ask your own server the same question; the answer is probably one grep -c away. There is another end of connection reuse — why closed connections wait 60 seconds in TIME_WAIT, and which setting does not touch that — over in the TIME_WAIT article; if you are wondering where the sockets in this piece pile up, it is there. The story of another nginx default misleading me is in the config that sent 100 bytes of a 5000-byte file. Official Sources - nginx — ngx_http_upstream_module: keepalive, keepalive_requests, keepalive_time - nginx — CHANGES (the 1.29.7 entry) - nginx — CHANGES-1.30 (changelog of the stable 1.30 branch) - nginx — ngx_http_proxy_module: proxy_http_version and proxy_set_header - nginx source — ngx_http_upstream_keepalive_module.c (release-1.31.8) - nginx source — ngx_http_proxy_module.c (release-1.31.8) - RFC 9112 — HTTP/1.1, Section 9.3 Persistence

Seems like you’re stuck. nginx.org nginx/nginx-prometheus-exporter

I Set TCP_DEFER_ACCEPT to 10; 15 Seconds Later, an Empty Connection I wanted to spare a listening socket's accept() call from dealing with empty connections. Linux has a switch for exactly that: TCP_DEFER_ACCEPT. The promise is simple — a client that has finished the handshake but has not yet sent a single byte will not reach you; I will wait until data arrives. I set the switch to 10 seconds. Then, out of habit, I read back what I had written: getsockopt() said 15. If the number you write and the number you read are not the same, there is a third number too: how long the connection actually waits. That one got a stopwatch too. At the median of three runs, accept() returned after 15.4 seconds — and what I held was a connection with not one byte of data. The deferral window had expired, and the kernel had not broken its promise; nobody had simply told me that the promise comes with an expiry date. Two machines, the same oddity So as not to draw a general rule from one machine's quirk, I ran the measurement in two places. Machine A: the linuxkit virtual machine of Docker Desktop on my Mac — aarch64, kernel 6.10.14, an ubuntu:24.04 container inside it, Python 3.12.3, iproute2 6.1.0. Defaults: tcp_synack_retries=5, tcp_syncookies=1, somaxconn=4096. Machine B: VPS3, the server that hosts this blog — x86_64, kernel 6.8.0-147-generic, Python 3.12.3. Different defaults: tcp_synack_retries=2, somaxconn=65535. On both machines the measurement ran over 127.0.0.1 with a single listening socket. The measuring code is three pieces: a listener, a client, and a loop that samples state from the outside with ss. What I write is not seconds, it is a retransmission count The first measurement is the simplest: open a socket, write a number into TCP_DEFER_ACCEPT, read it straight back. I tried 26 values. Eight came back unchanged, eighteen changed — and both machines produced exactly the same table. | What I wrote | What getsockopt() answered | |---|---| | 1 | 1 | | 2 | 3 | | 3 | 3 | | 4 | 7 | | 7 | 7 | | 10 | 15 | | 15 | 15 | | 30 | 31 | | 60 | 63 | | 128 | 247 | | 300 | 367 | | 3600 | 3607 | Of the 26 values I tried, the eight that came back unchanged are no coincidence: 0, 1, 3, 7, 15, 31, 63, 127. One less than a power of two. After 127 the series breaks and continues in steps of 120 — 3607 in the table also maps to itself, so it is simply a later stop on the same series. The reason is written in two small functions in net/ipv4/tcp.c. On the setsockopt() path the kernel converts the seconds you give into a retransmission count with secs_to_retrans(); on the getsockopt() path it turns that count back into seconds with retrans_to_secs(). The conversion's first step is TCP_TIMEOUT_INIT, that is 1 second; it doubles each round and tops out at TCP_RTO_MAX_SEC, which is 120 seconds. The resulting series: 1, 1+2=3, 1+2+4=7, 15, 31, 63, 127, then +120, +120… I downloaded the source of these two functions from the 6.8, 6.10 and 7.3 tags and compared them: byte for byte identical. The two machines producing the same table is therefore not a surprise but the expected result. What the socket actually stores is a single u8 named rskq_defer_accept. Seconds do not fit in there; what fits is "defer for this many retransmission rounds". The ceiling used by the conversion is a compile-time constant too: TCP_RTO_MAX_SEC. Yet 7.3 has a separate option, TCP_RTO_MAX_MS, that changes that upper bound per socket, and the timer that paces the rounds reads it. The two know nothing about each other: on a listener whose bound is lowered, getsockopt() still says 15 while the wait is not 15 seconds long. That one I read in the code — the kernels on my measuring machines do not know the option yet. When you write 10, the kernel asks this: what is the smallest round count whose cumulative duration does not fall below 10 seconds? The answer is 4, and four rounds take 15 seconds. The code does exactly that: a while (seconds > period) loop adds rounds until the accumulated duration passes the seconds you asked for. This behavior is not undocumented either. The tcp(7) manual page says it in one sentence: the value is rounded up to the next larger TCP retransmit duration. The sentence sits there, but I have never seen anyone translate it as "write 10, get 15" — and I would have missed it too, had I not read back what I wrote. The paragraph right after that manual entry holds a second surprise: the very same socket option does something entirely different on a client socket. There, a nonzero value delays the final ACK of the handshake until the socket has data to send; the ACK then rides on the first data segment, or waits for a delayed-ACK timeout. One name, two behaviors at the two ends — it is easy to read the listener half and assume the client half matches. The real lesson here is about my own practice: reading a socket option back after setting it is thirty seconds of work. Every time I skip it, I am trusting a guess — the post about the neighbouring container that shrank my pipe buffer eightfold came from the same place: F_SETPIPE_SZ handed back a rounded value rather than the one I asked for, and code that never reads the return value does not know its own budget. A silent client: 15 seconds later, empty The second measurement tests the promise itself. I open the listener with a value D, a client calls connect() and sends nothing. When does accept() return, and what do I hold when it does? On machine A, with the tcp_synack_retries=5 default — each row is three separate runs, so that no number rests on a single measurement: | D | getsockopt() | accept() returned | Data ready | |---|---|---|---| | 1 | 1 s | 1.013 / 1.025 / 1.029 s | 0 bytes | | 2 | 3 s | 3.066 / 3.091 / 3.096 s | 0 bytes | | 10 | 15 s | 15.186 / 15.377 / 15.505 s | 0 bytes | On machine B, on a different architecture and a different kernel: 1.022 seconds for D=1, 15.587 seconds for D=10. Again 0 bytes. accept() really is delayed, and the delay lands on the number getsockopt() reported. The point is what happens at the end of the window: the connection is delivered anyway. Empty. Looking with ss, the connection sits in SYN-RECV throughout the window and moves to ESTAB when it expires. (I did not convert the sampler loop's counter index into seconds: on a busy server ss itself slows down and the index-to-time mapping drifts. ss shows the state transition; the duration comes from accept()'s timestamp.) The client side, meanwhile, believes it is connected; its connect() returned in under a millisecond. So there is no broken connection here, just fifteen seconds in which the two ends answer the question "are we connected?" differently. What the kernel does during that window nstat showed two counters: TcpExtTCPDeferAcceptDrop went up by one, and TcpExtTCPSynRetrans went up by one. Both describe exactly how the mechanism works. The ACK that completes the handshake is a bare ACK, carrying no data. tcp_check_req() in tcp_minisocks.c sees that packet and drops it if the deferral rounds are not exhausted — its comment says precisely that: while TCP_DEFER_ACCEPT is active, the bare ACK is dropped. While dropping it, the kernel leaves a mark on the request saying "this client did send its ACK" (acked = 1) and bumps the TCPDeferAcceptDrop counter. The request thus stays in SYN-RECV and never enters the accept queue. The rest is the job of syn_ack_recalc() in inet_connection_sock.c. On every timer round the function asks two questions: should this request be deleted, should the SYN-ACK be resent? With deferral on, the resend decision goes like this: if the client has not ACKed yet, resend; if it has, stay quiet until the deferral's last round. The comment in the source states the intent plainly — to give data or an ACK one last chance. On that last round the kernel resends the SYN-ACK. The client, receiving a SYN-ACK for a connection it considers long established, follows the rule and sends its ACK again. But this time the deferral counter is exhausted: that same bare ACK is no longer dropped, a child socket is created, the connection moves to ESTAB and accept() returns. You hold zero bytes of data. !Diagram So TCP_DEFER_ACCEPT is not a filter, it is a delay. It does not weed out the connection that brings no data; it postpones its delivery, and then delivers it anyway. When the promise is kept To be fair, the other direction went on the bench as well. Same listener, D=1, but this time the client talks. When the client sent 16 bytes right after connect(), accept() returned in 0.002 seconds with the data ready. When it waited half a second before sending, accept() returned in 0.504 seconds, again with data. In both cases TCPDeferAcceptDrop went up by one — so the handshake ACK was dropped either way; what created the connection was the first data packet. That detail is the nice side of the mechanism: with deferral on, the server side never sees a connection that has done nothing but shake hands. The connection is born the moment the first data arrives. For protocols whose clients speak immediately, this is a free win — one wake-up per accept(), one epoll round, one wasted read attempt, all gone. And that is exactly where it is free: where the client speaks immediately. Protocols where the server speaks first There is a common sentence about this trap: "don't use deferred for SSH, the server speaks first." Going to the source, I found the sentence is wrong. RFC 4253 says that when the connection is established, both sides must send an identification string. Rather than stop at the document, I pointed my Mac's OpenSSH 10.3p1 client at a listener that never speaks; the client sent its SSH-2.0-OpenSSH_10.3 line the moment the handshake finished. So SSH is in the "client speaks immediately" class as far as deferral goes. SMTP is not. RFC 5321 says a session begins when the client opens a connection and the server responds with an opening message; the client only sends EHLO after receiving that greeting. Combine that order with deferral: the server process waits in accept(), the client waits for the 220 line, nobody speaks. The "silent client" in my lab is exactly this SMTP client — and the measurement shows that in this case the connection sits dead for 1 second with D=1 and 15.4 seconds with D=10, then gets delivered to the server empty. It is sturdier to frame the rule as a question rather than a list of protocol names: who sends the first byte on this port? If the answer is "the client", deferral helps you. If it is "the server", you add the entire deferral window to every connection as latency. If you do not know the answer, watching one connection with tcpdump takes five minutes; guessing is not free, as that common sentence about SSH shows. The second gate: tcp_synack_retries Reading the source raised a suspicion. In syn_ack_recalc() the condition for deleting a request depends on two things: the tcp_synack_retries threshold and the deferral counter. If the client has sent its ACK — as it has in my scenario — the request is not deleted until both are exceeded. (If the client never ACKs at all, the !acked branch kicks in and tcp_synack_retries suffices on its own; that path is not this post's subject.) So does trimming tcp_synack_retries also shorten the deferral? So I ran the same test on machine A with tcp_synack_retries=1: for D=2, accept() still returned at 3.077 seconds. So, no. On machine B the setting is already 2, below the default; the durations there landed in the same range as machine A's (1.022 seconds for D=1, 15.587 seconds for D=10). The logic settled afterwards: tcp_synack_retries governs the deletion of the request, not its delivery. Delivery happens via the ACK the client sends in reply to the SYN-ACK of the last round; when that ACK arrives the request is not deleted, it is promoted. The two settings do not interfere. The post about who inherits a closing socket's queue had a similar distinction: how full the queue is and what happens to the things in it are two different mechanisms. Under load, deferral turns itself off The last measurement was the one that bothered me most. I sent 200 silent clients all at once — non-blocking sockets, all 200 SYNs on the wire within 3 milliseconds — at a listener whose listen() backlog was 10. Three runs, machine A, D=1, a 20-second observation window: | Observation | Run 1 | Run 2 | Run 3 | |---|---|---|---| | TCPDeferAcceptDrop — passed through the deferral gate | +10 | +10 | +10 | | SyncookiesSent / SyncookiesRecv — never saw the gate | +190 | +190 | +190 | | TCPReqQFullDoCookies | +295 | +306 | +326 | | ListenOverflows | +205 | +230 | +271 | | Empty connections accept()ed within 20 s | 100 | 86 | 65 | The first two rows are identical to the digit across all three runs, and they add up to exactly 200: 10 of the connections went through the deferral gate, 190 never saw it. The 10 is no coincidence. Deferred connections live in the request queue, and that queue's ceiling is the listen() backlog. Because the comparison in the code uses >, the real ceiling is one more than the backlog; all three runs came out at 10. Once the queue fills, the kernel falls back to SYN cookies. The SYN cookie path does not use a pending request socket: once the cookie in the incoming ACK validates, syncookies.c builds a request structure on the spot, turns it into a child socket immediately and puts it straight into the accept queue. The deferral gate lives inside tcp_check_req(), so it never runs on that path — those 190 connections went to accept() directly, and empty. The bottom three rows depend on timing and move from run to run. TCPReqQFullDoCookies exceeding 200 is not a contradiction: that counter is bumped inside tcp_syn_flood_action(), before the accept-queue fullness check. A SYN that arrives while the queue is full is dropped after the counter has already moved (ListenOverflows), and the client resends its SYN — so one connection can bump the counter several times. That is also why the number of empty connections accept()ed in 20 seconds wanders between 65 and 100: the accept queue seats ten, and everyone else keeps retrying for a place in line. The practical meaning: deferral switches itself off exactly when you would need it — under load. It is not a defense layer against a malicious load; SYN queue pressure still has to be managed with the tools I described in the TCP backlog and SYN flood runbook. A second path where the deferral gate never runs The bypass under load is not the only exception. A few lines before the deferral check in tcp_check_req() there is an early exit: if the connection was established with TCP Fast Open, the function returns right there. With TFO the child socket is already born at SYN time, so there is no pending request to hold back — meaning that if you turn on fastopen=3 and deferred on the same listener, deferral never runs for connections that arrive via TFO. I read that in the code rather than measuring it; the post where I put TCP Fast Open on the bench shows that TFO's own conditions are fiddly enough as it is. One note on scope as well: all of my measurements are over IPv4, on 127.0.0.1. The two files where the deferral logic lives (tcp_minisocks.c, inet_connection_sock.c) are address-family agnostic, so I would not expect IPv6 to behave differently — but I did not measure it, and you should know that. What nginx writes, and what it does nginx's listen ... deferred parameter gets one line in the documentation: on Linux it uses the TCP_DEFER_ACCEPT socket option. The docs do not say which value it uses; the source does. In src/core/ngx_connection.c the value is hardcoded to 1. The comment next to it explains why — there is no way to learn how long a connection waited in the queue, and besides, a connection may bypass the deferred queue entirely if SYN cookies were used. What nginx admits in its own source comment showed up as numbers across three runs here. It is a good example of why also looking at the source while reading the docs is such a cheap habit. Out of curiosity I looked at HAProxy too: in src/proto_tcp.c the defer-accept option sets accept_delay = 1, with a "defer accept by up to one second" comment beside it. Two independent proxies landed on the same number. So "the deferral window is one second" is not an nginx-specific preference; it is the industry's shared call. What you get when you write deferred, in the language of these measurements: a one-second window. Reasonable for HTTP, whose clients speak immediately. If the window expires the connection arrives anyway, empty — so deferred does not say "empty connections never arrive", it says "it waits about a second". I also checked what I use on my own server: there is not a single listen line containing deferred in VPS3's nginx configuration. And I will not be adding one after this measurement — for the nginx in front of this blog, one accept() wake-up is cheaper than the complexity a one-second window brings. A decision frame The questions I ask when weighing this switch, in order: 1. Who sends the first byte on this port? If it is not the client, stop here; deferral adds dead latency to every connection. 2. What will I gain? The gain is the accept() + epoll + first read attempt spent on connections that bring no data. Measure that first: if most of your listener's wake-ups do bring data, there is nothing to gain. 3. Did I read back the value I wrote? With getsockopt(). Trust the 1-3-7-15-31 series, not the number you wrote. 4. Do I accept what happens when the window expires? The empty connection still arrives. If your application was written on a "data guaranteed" assumption, deferral does not save that assumption. 5. What happens under load? The request queue is bounded by the listen() backlog, and once it fills, SYN cookies bypass the deferral logic. Deferral is not a security measure. 6. Which counter tells me the truth in monitoring? nstat -az TcpExtTCPDeferAcceptDrop. If it is not rising, deferral never engages; if it rises alongside SyncookiesSent, it is being bypassed under load. Conclusion TCP_DEFER_ACCEPT is not a bad option; it just speaks its own language in two places rather than ours. The first is its unit: we write seconds, the kernel reads retransmission rounds — and because it has no stopwatch of its own, it rounds up to the nearest round boundary. The second is its duration: at the end of "wait until data arrives" there is an "enough already", and at that point it delivers the connection empty. The lesson I am left with goes beyond the unit. A kernel switch's documentation, its code and its behavior do not tell three different stories — but they tell the story at three different levels of detail. The docs said "rounded up", the code said which series it rounds to, and the measurement showed that this means 15.4 seconds when you write 10. Reading all three together took thirty minutes. How long chasing a mysterious fifteen-second delay in production would have taken is harder to guess. Official Sources - net/ipv4/tcp.c — secs_to_retrans(), retrans_to_secs() and the TCP_DEFER_ACCEPT sockopt paths (v7.3) - net/ipv4/tcp_minisocks.c — dropping the bare ACK inside tcp_check_req() (v7.3) - net/ipv4/inet_connection_sock.c — the delete/resend decision in syn_ack_recalc() (v7.3) - net/ipv4/syncookies.c — tcp_get_cookie_sock() adding the child socket straight to the accept queue (v7.3) - net/ipv4/tcp_input.c — tcp_syn_flood_action() bumping the counter before the accept-queue check (v7.3) - tcp(7) manual page — the TCP_DEFER_ACCEPT definition and the rounding-up sentence - nginx listen directive — the deferred parameter - nginx source — the hardcoded value 1 and the SYN cookie comment in ngx_connection.c - HAProxy source — accept_delay = 1 for defer-accept in proto_tcp.c - RFC 4253 — both sides sending an identification string in the SSH version exchange - RFC 5321 — an SMTP session beginning with the server's opening message

nginx drops roughly half its HTTP/3 connections on reload. quic_bpf bug, open for over a year. nginx#1323

Today in DevOps: GitHub AI Scan for pull requests is now GA Today in DevOps: 1. AI Scan for pull requests generally available (github) github.blog 2. hashicorp/vault v2.2.0 github.com/hashicorp/vault/releases/tag/v2.2.0 3. Multi-repo support and other improvements for GitHub Copilot in Slack and Microsoft Teams github.blog 4. Agentic autofix for code scanning alerts general availability (github) github.blog 5. nginx/nginx-gateway-fabric v2.8.0 github.com/nginx/nginx-gateway-fabric/releases/ta… 6. istio/istio 1.31.2 github.com/istio/istio/releases/tag/1.31.2 [ar] جديد ديف أوبس (DevOps): إطلاق ميزة الفحص بالذكاء الاصطناعي لطلبات السحب (Pull Requests) في GitHub... في عالم DevOps اليوم: ١. أصبح فحص الذكاء الاصطناعي لطلبات السحب متاحًا للجميع (GitHub) github.blog ٢. hashicorp/vault الإصدار ٢.٢.٠ github.com/hashicorp/vault/releases/tag/v2.2.0 ٣. دعم المستودعات المتعددة وتحسينات أخرى لـ GitHub Copilot في Slack وMicrosoft Teams github.blog ٤. أصبح الإصلاح التلقائي الآلي لتنبيهات فحص التعليمات البرمجية متاحًا للجميع (جيت هاب) github.blog 5. nginx/nginx-gateway-fabric الإصدار 2.8.0 github.com/nginx/nginx-gateway-fabric/releases/ta… 6. istio/istio الإصدار 1.31.2 github.com/istio/istio/releases/tag/1.31.2 [bn] ডেভঅপস-এ আজকের আপডেট: গিটহাব এআই স্ক্যান ফর পুল রিকোয়েস্ট এখন সাধারণের জন্য উন্মুক্ত (GA) আজ ডেভঅপস-এ: ১. পুল রিকোয়েস্টের জন্য এআই স্ক্যান এখন সর্বসাধারণের জন্য উপলব্ধ (গিটহাব) github.blog ২. হ্যাশিকর্প/ভল্ট সংস্করণ ২.২.০ github.com/hashicorp/vault/releases/tag/v2.2.0 ৩. স্ল্যাক এবং মাইক্রোসফট টিমস-এ গিটহাব কপাইলটের জন্য মাল্টি-রেপো সাপোর্ট এবং অন্যান্য উন্নতি github.blog ৪. কোড স্ক্যানিং অ্যালার্টের জন্য এজেন্টিক অটোফিক্স এখন সর্বসাধারণের জন্য উপলব্ধ (গিটহাব) github.blog ৫. এনজিনএক্স/এনজিনএক্স-গেটওয়ে-ফ্যাব্রিক ভি২.৮.০ github.com/nginx/nginx-gateway-fabric/releases/ta… ৬. ইস্টিও/ইস্টিও ১.৩১.২ github.com/istio/istio/releases/tag/1.31.2 [de-DE] DevOps heute: GitHub AI Scan für Pull Requests jetzt allgemein verfügbar Heute in DevOps: 1. AI Scan für Pull Requests allgemein verfügbar (GitHub) github.blog 2. hashicorp/vault v2.2.0 github.com/hashicorp/vault/releases/tag/v2.2.0 3. Unterstützung für mehrere Repositories und weitere Verbesserungen für GitHub Copilot in Slack und Microsoft Teams github.blog 4. Agentic Autofix für Code-Scanning-Warnungen allgemein verfügbar (GitHub) github.blog 5. nginx/nginx-gateway-fabric v2.8.0 github.com/nginx/nginx-gateway-fabric/releases/ta… 6. istio/istio 1.31.2 github.com/istio/istio/releases/tag/1.31.2 [es-US] Hoy en DevOps: El análisis de IA de GitHub para pull requests ya está disponible de forma general... Hoy en DevOps: 1. Escaneo con IA para solicitudes de extracción ya disponible (github) github.blog 2. hashicorp/vault v2.2.0 github.com/hashicorp/vault/releases/tag/v2.2.0 3. Compatibilidad con múltiples repositorios y otras mejoras para GitHub Copilot en Slack y Microsoft Teams github.blog 4. Corrección automática de Agentic para alertas de escaneo de código ya disponible (github) github.blog 5. nginx/nginx-gateway-fabric v2.8.0 github.com/nginx/nginx-gateway-fabric/releases/ta… 6. istio/istio 1.31.2 github.com/istio/istio/releases/tag/1.31.2 [fr-FR] L'actu DevOps : l'analyse IA de GitHub pour les pull requests est désormais disponible Aujourd'hui dans le DevOps : 1. Analyse IA des demandes d'extraction disponible (GitHub) github.blog 2. HashiCorp/Vault v2.2.0 github.com/hashicorp/vault/releases/tag/v2.2.0 3. Prise en charge multi-dépôts et autres améliorations pour GitHub Copilot dans Slack et Microsoft Teams github.blog 4. Correction automatique des alertes d'analyse de code disponible (GitHub) github.blog 5. nginx/nginx-gateway-fabric v2.8.0 github.com/nginx/nginx-gateway-fabric/releases/ta… 6. istio/istio 1.31.2 github.com/istio/istio/releases/tag/1.31.2 [hi] डेवऑप्स (DevOps) में आज: GitHub का पुल रिक्वेस्ट्स के लिए AI स्कैन अब GA है आज के DevOps अपडेट: 1. पुल रिक्वेस्ट के लिए AI स्कैन अब उपलब्ध है (github) github.blog 2. hashicorp/vault v2.2.0 github.com/hashicorp/vault/releases/tag/v2.2.0 3. Slack और Microsoft Teams में GitHub Copilot के लिए मल्टी-रेपो सपोर्ट और अन्य सुधार github.blog 4. कोड स्कैनिंग अलर्ट के लिए Agentic ऑटोफिक्स अब उपलब्ध है (गिटहब) github.blog 5. nginx/nginx-gateway-fabric v2.8.0 github.com/nginx/nginx-gateway-fabric/releases/ta… 6. istio/istio 1.31.2 github.com/istio/istio/releases/tag/1.31.2 [id] Hari Ini dalam DevOps: Pemindaian AI GitHub untuk pull request kini sudah tersedia secara umum (GA) Hari Ini di DevOps: 1. Pemindaian AI untuk permintaan tarik tersedia secara umum (github) github.blog 2. hashicorp/vault v2.2.0 github.com/hashicorp/vault/releases/tag/v2.2.0 3. Dukungan multi-repo dan peningkatan lainnya untuk GitHub Copilot di Slack dan Microsoft Teams github.blog 4. Perbaikan otomatis Agentic untuk peringatan pemindaian kode tersedia secara umum (github) github.blog 5. nginx/nginx-gateway-fabric v2.8.0 github.com/nginx/nginx-gateway-fabric/releases/ta… 6. istio/istio 1.31.2 github.com/istio/istio/releases/tag/1.31.2 [it] Oggi nel DevOps: lo scanner AI di GitHub per le pull request è ora disponibile (GA) Oggi in DevOps: 1. Scansione AI per le pull request generalmente disponibile (github) github.blog 2. hashicorp/vault v2.2.0 github.com/hashicorp/vault/releases/tag/v2.2.0 3. Supporto multi-repository e altri miglioramenti per GitHub Copilot in Slack e Microsoft Teams github.blog 4. Agentic autofix per gli avvisi di scansione del codice generalmente disponibile (github) github.blog 5. nginx/nginx-gateway-fabric v2.8.0 github.com/nginx/nginx-gateway-fabric/releases/ta… 6. istio/istio 1.31.2 github.com/istio/istio/releases/tag/1.31.2 [iw] היום ב-DevOps: סריקת ה-AI של GitHub לבקשות משיכה (Pull Requests) זמינה כעת לכלל המשתמשים היום ב-DevOps: 1. סריקת בינה מלאכותית לבקשות משיכה זמינה באופן כללי (github) github.blog 2. hashicorp/vault v2.2.0 github.com/hashicorp/vault/releases/tag/v2.2.0 3. תמיכה במאגרים מרובים ושיפורים נוספים עבור GitHub Copilot ב-Slack וב-Microsoft Teams github.blog 4. זמינות כללית של Agentic להתראות סריקת קוד (github) github.blog 5. nginx/nginx-gateway-fabric גרסה 2.8.0 github.com/nginx/nginx-gateway-fabric/releases/ta… 6. istio/istio 1.31.2 github.com/istio/istio/releases/tag/1.31.2 [ja] 今日のDevOps：GitHubのプルリクエスト向けAIスキャンが一般提供開始（GA） 本日のDevOps関連情報： 1. プルリクエストのAIスキャンが一般提供開始（GitHub） github.blog 2. hashicorp/vault v2.2.0 github.com/hashicorp/vault/releases/tag/v2.2.0 3. SlackおよびMicrosoft TeamsにおけるGitHub Copilotのマルチリポジトリサポートおよびその他の改善 github.blog 4. コードスキャンアラートのエージェントによる自動修正が一般提供開始(GitHub) github.blog 5. nginx/nginx-gateway-fabric v2.8.0 github.com/nginx/nginx-gateway-fabric/releases/ta… 6. istio/istio 1.31.2 github.com/istio/istio/releases/tag/1.31.2 [ko] 오늘의 데브옵스(DevOps): GitHub AI 풀 리퀘스트 스캔, 정식 출시(GA) 오늘 DevOps 소식: 1. 풀 리퀘스트 AI 스캔 기능 정식 출시 (GitHub) github.blog 2. hashicorp/vault v2.2.0 github.com/hashicorp/vault/releases/tag/v2.2.0 3. Slack 및 Microsoft Teams용 GitHub Copilot에 멀티 리포지토리 지원 및 기타 개선 사항 추가 github.blog 4. 코드 스캔 알림에 대한 Agentic 자동 수정 기능 정식 출시 (깃허브) github.blog 5. nginx/nginx-gateway-fabric v2.8.0 github.com/nginx/nginx-gateway-fabric/releases/ta… 6. istio/istio 1.31.2 github.com/istio/istio/releases/tag/1.31.2 [ml] ഇന്നത്തെ ഡെവ്ഓപ്സ്: ഗിറ്റ്‌ഹബ് എഐ സ്കാൻ ഫോർ പുൾ റിക്വസ്റ്റ് ഇപ്പോൾ ലഭ്യമാണ് (GA) ഇന്ന് DevOps-ൽ: 1. പൊതുവെ ലഭ്യമായ പുൾ അഭ്യർത്ഥനകൾക്കായുള്ള AI സ്കാൻ (github) github.blog 2. hashicorp/vault v2.2.0 github.com/hashicorp/vault/releases/tag/v2.2.0 3. സ്ലാക്കിലും മൈക്രോസോഫ്റ്റ് ടീമുകളിലും GitHub കോപൈലറ്റിനുള്ള മൾട്ടി-റെപ്പോ പിന്തുണയും മറ്റ് മെച്ചപ്പെടുത്തലുകളും github.blog 4. കോഡ് സ്കാനിംഗിനുള്ള ഏജന്റ് ഓട്ടോഫിക്സ് പൊതു ലഭ്യത അലേർട്ടുകൾ (github) github.blog 5. nginx/nginx-gateway-fabric v2.8.0 github.com/nginx/nginx-gateway-fabric/releases/ta… 6. istio/istio 1.31.2 github.com/istio/istio/releases/tag/1.31.2 [nl-NL] DevOps vandaag: GitHub AI-scan voor pull requests nu algemeen beschikbaar Vandaag in DevOps: 1. AI-scan voor pull-requests algemeen beschikbaar (github) github.blog 2. hashicorp/vault v2.2.0 github.com/hashicorp/vault/releases/tag/v2.2.0 3. Ondersteuning voor meerdere repositories en andere verbeteringen voor GitHub Copilot in Slack en Microsoft Teams github.blog 4. Agentic autofix voor code-scanwaarschuwingen algemeen beschikbaar (github) github.blog 5. nginx/nginx-gateway-fabric v2.8.0 github.com/nginx/nginx-gateway-fabric/releases/ta… 6. istio/istio 1.31.2 github.com/istio/istio/releases/tag/1.31.2 [pa] ਅੱਜ ਡੇਵ-ਓਪਸ (DevOps) ਵਿੱਚ: GitHub AI ਸਕੈਨ ਹੁਣ ਪੁੱਲ ਰਿਕਵੈਸਟਾਂ ਲਈ ਆਮ ਤੌਰ 'ਤੇ ਉਪਲਬਧ (GA) ਹੈ ਅੱਜ DevOps ਵਿੱਚ: 1. ਪੁੱਲ ਬੇਨਤੀਆਂ ਲਈ AI ਸਕੈਨ ਆਮ ਤੌਰ 'ਤੇ ਉਪਲਬਧ ਹੈ (github) github.blog 2. hashicorp/vault v2.2.0 github.com/hashicorp/vault/releases/tag/v2.2.0 3. ਸਲੈਕ ਅਤੇ ਮਾਈਕ੍ਰੋਸਾਫਟ ਟੀਮਾਂ ਵਿੱਚ GitHub Copilot ਲਈ ਮਲਟੀ-ਰੇਪੋ ਸਹਾਇਤਾ ਅਤੇ ਹੋਰ ਸੁਧਾਰ github.blog 4. ਕੋਡ ਸਕੈਨਿੰਗ ਚੇਤਾਵਨੀਆਂ ਲਈ ਏਜੰਟਿਕ ਆਟੋਫਿਕਸ ਆਮ ਉਪਲਬਧਤਾ (github) github.blog 5. nginx/nginx-ਗੇਟਵੇ-ਫੈਬਰਿਕ v2.8.0 github.com/nginx/nginx-gateway-fabric/releases/ta… 6. istio/istio 1.31.2 github.com/istio/istio/releases/tag/1.31.2 [pl] Dzisiaj w DevOps: Skanowanie AI w GitHub dla pull requestów już dostępne Dzisiaj w DevOps: 1. Skanowanie AI dla żądań ściągnięcia jest ogólnie dostępne (GitHub) github.blog 2. hashicorp/vault v2.2.0 github.com/hashicorp/vault/releases/tag/v2.2.0 3. Obsługa wielu repozytoriów i inne ulepszenia dla GitHub Copilot w Slacku i Microsoft Teams github.blog 4. Automatyczne poprawki agentów dla alertów skanowania kodu są ogólnie dostępne (github) github.blog 5. nginx/nginx-gateway-fabric w wersji 2.8.0 github.com/nginx/nginx-gateway-fabric/releases/ta… 6. istio/istio 1.31.2 github.com/istio/istio/releases/tag/1.31.2 [pt-BR] Hoje no DevOps: Verificação por IA do GitHub para pull requests agora está disponível ao público ... Hoje em DevOps: 1. A função AI Scan para pull requests está disponível para o público em geral (GitHub) github.blog 2. hashicorp/vault v2.2.0 github.com/hashicorp/vault/releases/tag/v2.2.0 3. Suporte a múltiplos repositórios e outras melhorias para o GitHub Copilot no Slack e no Microsoft Teams github.blog 4. Correção automática de agentes para alertas de verificação de código disponível para o público em geral (github) github.blog 5. nginx/nginx-gateway-fabric v2.8.0 github.com/nginx/nginx-gateway-fabric/releases/ta… 6. istio/istio 1.31.2 github.com/istio/istio/releases/tag/1.31.2 [ru] Новости DevOps: AI-сканирование пулл-реквестов в GitHub теперь общедоступно Сегодня в DevOps: 1. AI Scan для запросов на слияние (github) стал общедоступным github.blog 2. hashicorp/vault v2.2.0 github.com/hashicorp/vault/releases/tag/v2.2.0 3. Поддержка нескольких репозиториев и другие улучшения для GitHub Copilot в Slack и Microsoft Teams github.blog 4. Agentic autofix для оповещений о сканировании кода стал общедоступным (github) github.blog 5. nginx/nginx-gateway-fabric v2.8.0 github.com/nginx/nginx-gateway-fabric/releases/ta… 6. istio/istio 1.31.2 github.com/istio/istio/releases/tag/1.31.2 [ta] இன்றைய DevOps: GitHub AI ஸ்கேன், புல் ரெக்வெஸ்டுகளுக்கு (pull requests) பொது பயன்பாட்டிற்கு வந்து... இன்று DevOps-இல்: 1. புல் ரிக்வெஸ்ட்களுக்கான AI ஸ்கேன் பொதுவாகக் கிடைக்கிறது (கிட்ஹப்) github.blog 2. hashicorp/vault v2.2.0 github.com/hashicorp/vault/releases/tag/v2.2.0 3. ஸ்லாக் மற்றும் மைக்ரோசாஃப்ட் டீம்ஸில் உள்ள கிட்ஹப் கோபைலட்டிற்கான மல்டி-ரெப்போ ஆதரவு மற்றும் பிற மேம்பாடுகள் github.blog 4. கோட் ஸ்கேனிங் எச்சரிக்கைகளுக்கான ஏஜென்டிக் ஆட்டோஃபிக்ஸ் பொதுவாகக் கிடைக்கிறது (கிட்ஹப்) github.blog 5. nginx/nginx-gateway-fabric v2.8.0 github.com/nginx/nginx-gateway-fabric/releases/ta… 6. istio/istio 1.31.2 github.com/istio/istio/releases/tag/1.31.2 [te] నేటి డెవలప్స్: గిట్‌హబ్ (GitHub) AI పుల్ రిక్వెస్ట్ స్కాన్ ఇప్పుడు అందుబాటులో ఉంది ఈరోజు DevOpsలో: 1. పుల్ రిక్వెస్ట్‌ల కోసం AI స్కాన్ సాధారణంగా అందుబాటులో ఉంది (github) github.blog 2. hashicorp/vault v2.2.0 github.com/hashicorp/vault/releases/tag/v2.2.0 3. Slack మరియు Microsoft Teamsలో GitHub Copilot కోసం మల్టీ-రెపో మద్దతు మరియు ఇతర మెరుగుదలలు github.blog 4. కోడ్ స్కానింగ్ హెచ్చరికల కోసం ఏజెంటిక్ ఆటోఫిక్స్ సాధారణంగా అందుబాటులో ఉంది (github) github.blog 5. nginx/nginx-gateway-fabric v2.8.0 github.com/nginx/nginx-gateway-fabric/releases/ta… 6. istio/istio 1.31.2 github.com/istio/istio/releases/tag/1.31.2 [uk] Сьогодні в DevOps: GitHub AI Scan для запитів на злиття (pull requests) тепер доступний для всіх ... Сьогодні в DevOps: 1. AI Scan для запитів на зміщення доступний для всіх (github) github.blog 2. hashicorp/vault v2.2.0 github.com/hashicorp/vault/releases/tag/v2.2.0 3. Підтримка кількох репозиторіїв та інші покращення для GitHub Copilot у Slack та Microsoft Teams github.blog 4. Загальна доступність Agentic autofix для сповіщень про сканування коду (github) github.blog 5. nginx/nginx-gateway-fabric версії 2.8.0 github.com/nginx/nginx-gateway-fabric/releases/ta… 6. istio/istio 1.31.2 github.com/istio/istio/releases/tag/1.31.2

I Left a Stale .gz and Nginx 1.31.7 Served Two Bodies at One URL This blog's origin config has carried gzip on for years: the same CSS gets recompressed on every request. The well-known way to make that cheaper is obvious enough — compress the file once, drop it next to the original as .gz, write gzip_static on, and let nginx ship what is already there. Before flipping that switch, I set the mechanism up on my own machine, because what I wanted to know was not the gain. It was the price. The price fits in one sentence. When nginx serves a precompressed file, it never looks at the original. And because it never looks, it can never learn that the .gz next door has gone stale. I ran that as a measurement, and the result came out sharper than I expected: one address, two different bodies — and which one you get depends solely on your Accept-Encoding header. The setup: five server blocks, five questions I installed nginx from Homebrew on my Mac. The version is the freshest mainline there is as I write this: $ nginx -v nginx version: nginx/1.31.7 $ nginx -V 2>&1 | tr ' ' '\n' | grep gzip_static --with-http_gzip_static_module 1.31.7 is the top entry in nginx's own CHANGES file, dated 14 October 2026 — so I am measuring a release that is two days old. The module itself has been sitting still for years: the last behavioural change recorded against ngx_http_gzip_static_module in the changelog is byte-range support in 1.23.4. This is not an old version's quirk, then. It is today's design. The trick in the config is that I deliberately turn off on-the-fly compression and push the thresholds to absurd heights: gzip off; # on-the-fly compression OFF gzip_min_length 100000; # do not compress anything under 100 KB gzip_comp_level 1; gzip_types text/plain; # text/html is NOT in the list gzip_vary on; server { listen 127.0.0.1:8091; root .../www; gzip_static on; } server { listen 127.0.0.1:8092; root .../www; gzip_static always; } server { listen 127.0.0.1:8093; root .../www; gzip_static off; } server { listen 127.0.0.1:8094; root .../www; gzip_static on; open_file_cache max=100 inactive=60s; open_file_cache_valid 30s; open_file_cache_min_uses 1; open_file_cache_errors on; } server { listen 127.0.0.1:8095; root .../www; gzip_static on; gzip_vary off; } I put two cases in the docroot: a stale companion file, and a .gz with no original at all. $ ls -lT www/a.html www/a.html.gz www/b.html.gz -rw-r--r--@ 19 Oct 16 08:47:59 2026 www/a.html # "SURUM-2 taze govde" (fresh) -rw-r--r--@ 40 Oct 15 08:00:00 2026 www/a.html.gz # "SURUM-1 bayat govde" (stale) -rw-r--r--@ 36 Oct 16 08:47:59 2026 www/b.html.gz # NO original First measurement: one address, two bodies I fetched a single URL as two different clients. The second request sends no Accept-Encoding header at all — in curl, -H 'Accept-Encoding:' does not send an empty value, it removes the header entirely: $ curl -s --compressed http://127.0.0.1:8091/a.html SURUM-1 bayat govde $ curl -s -H 'Accept-Encoding:' http://127.0.0.1:8091/a.html SURUM-2 taze govde $ cat www/a.html SURUM-2 taze govde The real content on disk is SURUM-2. Your browser — that is, every client that asks for gzip — gets yesterday's body. The headers make it even clearer: | Header | gzip-capable client | client sending no Accept-Encoding | |---|---|---| | Content-Length | 40 | 19 | | Content-Encoding | gzip | (none) | | Last-Modified | Thu, 15 Oct 2026 05:00:00 GMT | Fri, 16 Oct 2026 05:47:59 GMT | | ETag | "6ad05dd0-28" | "6ad1ba8f-13" | | Vary | Accept-Encoding | Accept-Encoding | Nginx reports the size and the date of the file whose body it is sending. The .gz is a day old, so Last-Modified is a day old too. The ETag decodes as well: 0x6ad05dd0 is 1792040400, that is 15 October 08:00, and 0x28 is 40 bytes. So the ETag is derived straight from that .gz file's mtime and size — in the source the ETag format is "%xT-%xO", fed by last_modified_time and content_length_n (the ETag exists at all only because etag on is the default; turn it off and the line is never produced). Everything is consistent. Consistent with the stale file, not with reality. The most informative row in that table is the last one: Vary is there in the non-gzip response too. I'll come back to it shortly, because that header is what gives away when the module actually runs. Why nginx does not notice: the single path in the code This is not a bug, it is the module's design. In ngx_http_gzip_static_module.c at the 1.31.7 tag, the handler maps the URI to a filesystem path, appends .gz, and opens only that path. There is no stat of the original, no attempt to open it, and no mtime comparison. !Diagram The order in that diagram is a claim too, so I kept it faithful to the code, with two branches simplified away: a URI ending in / makes the module back off at the very start, and a .gz that is not a regular file returns 404. The gzip_vary branch on the right is the surprising part. When the client does not ask for gzip, the module does not back off immediately: if gzip_vary is on it still opens the .gz path, flags the response to carry Vary: Accept-Encoding, and only then declines. The early exit exists only when gzip_vary is off. Two blocks settle it — 8091 with gzip_vary on, 8095 with it off, neither request asking for gzip: # 8091: gzip_vary on HTTP/1.1 200 OK Content-Length: 19 Vary: Accept-Encoding # 8095: gzip_vary off HTTP/1.1 200 OK Content-Length: 19 The body is the original file in both cases, but the first one carries Vary. That header is the proof that gzip_static ran even on a request that wanted no gzip. In practice it means this: with gzip_vary on, every request — asking for gzip or not — tries to open the .gz path, so the open_file_cache entry and the file descriptor are held for those requests too. The official documentation papers over the real gap with a recommendation — it says it is recommended that the modification date and time of the original and compressed files be the same. A recommendation. Not a requirement, and not a check. The moment those two timestamps drift apart on disk, nothing warns you. The stale copy that says "not modified" You might pause here and think caches will refresh at some point anyway. The numbers say otherwise: $ curl -s --compressed -D - \ -H 'If-Modified-Since: Thu, 15 Oct 2026 05:00:00 GMT' \ http://127.0.0.1:8091/a.html | head -1 HTTP/1.1 304 Not Modified $ curl -s -H 'Accept-Encoding:' -D - \ -H 'If-Modified-Since: Thu, 15 Oct 2026 05:00:00 GMT' \ http://127.0.0.1:8091/a.html | head -1 HTTP/1.1 200 OK Same moment, same address, same date: to the gzip client it says "not modified", to the plain client it says "here, take the new one". Sending the stale ETag with If-None-Match gets the same: $ curl -s --compressed -D - -H 'If-None-Match: "6ad05dd0-28"' \ http://127.0.0.1:8091/a.html | grep -Ei 'HTTP/|ETag' HTTP/1.1 304 Not Modified ETag: "6ad05dd0-28" The stale copy, in other words, declares itself fresh, and every intermediate cache in front of it takes that word and keeps what it has. There is no self-healing here. This file goes away only when a human notices it and deletes it. Put a CDN in the path and the picture tilts further: thanks to Vary: Accept-Encoding the two representations are stored separately, one right and one wrong, and both are "fresh". No bug report arrives either, because the complaining user reloads the page, the server says 304 again, and the problem degrades into "it doesn't show up for me". An address that exists only for gzip clients If the handler never opens the original, what happens when the original does not exist? There is no b.html in the docroot, only b.html.gz: $ curl -s --compressed -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8091/b.html 200 $ curl -s --compressed http://127.0.0.1:8091/b.html yalnizca-gz-var $ curl -s -H 'Accept-Encoding:' -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8091/b.html 404 Same address, same second: 200 and 404. Whether the page exists depends on the client's header. In the block with gzip_static off, both are 404 — so the only thing bringing that address into existence is the module itself: # 8093, gzip_static off gzip client : 404 plain client : 404 This turns into a trap on the monitoring side. If your uptime checker does not send Accept-Encoding, it sees 404 while browsers see 200 — or the other way around, the checker goes green and the user tries to describe a page they cannot find. Monitoring and users living in different worlds is the most tiring kind of debugging there is. Four knobs I had tuned do nothing, the other four still work Remember the lab config: gzip off, gzip_min_length 100000, gzip_types text/plain. All three should have blocked a 40-byte .gz of a text/html file. None did: $ curl -s --compressed -o /dev/null \ -w '%{size_download} bytes, enc=%header{content-encoding}\n' \ http://127.0.0.1:8091/a.html 40 bytes, enc=gzip Which makes sense, because nginx is not compressing anything here — it is shipping a ready-made file. gzip_comp_level is meaningless for the same reason: the compression level is now chosen by whoever runs the gzip command, which is you. The documentation's list of "the following directives are also taken into account" is short and clear: gzip_http_version, gzip_proxied, gzip_disable, gzip_vary. Anything not on that list is quietly inert while gzip_static is on. One of the directives on that list does take effect, and here it is at work: $ curl -s --http1.0 --compressed -D - http://127.0.0.1:8091/a.html | grep -i 'content-' Content-Length: 19 Requesting over HTTP/1.0 triggered the gzip_http_version 1.1 default: the module opened the .gz, flagged Vary, and backed off; the static module served the body — 19 bytes, the fresh one. A strange little irony: the shortest route to protection from the stale copy is for your client to speak an older protocol. | Knob | While gzip_static is on | |---|---| | gzip on/off | inert (measured) | | gzip_min_length | inert (measured) | | gzip_types | inert (measured) | | gzip_comp_level | inert (no compression happens) | | gzip_http_version | effective (measured) | | gzip_proxied | effective (docs and source) | | gzip_disable | effective (docs and source) | | gzip_vary | effective (measured) | One caveat matters here: by "inert" I mean inert for the response gzip_static sends. If another path on the same server is still compressed by nginx itself, every knob keeps working for that response. The two live side by side; what gets confusing is which response obeys which rule. Opening my own config surprised me on exactly this point. The origin carries gzip on, gzip_proxied any, gzip_types ..., gzip_min_length 1024 — but every path, /_astro/ included, goes to proxy_pass http://127.0.0.1:3040. The only place that serves a file from disk is the 404 fallback pointing at the previous build. gzip_static is a static-file module; it never touches a proxied response. So in my configuration that line would change literally nothing — and all four directives keep full authority, because they are compressing proxied responses. gzip_proxied any is sitting there for exactly that reason. There is also always. In that mode the module does not ask the client at all. Here you have to be careful, because not sending the header is not the same as saying you do not want gzip. With no header present, HTTP treats any content coding as acceptable, so gzip breaks no rule. The real test therefore needs a client that speaks up — Accept-Encoding: identity, meaning "give me the uncompressed one": # 8091, gzip_static on $ curl -s -H 'Accept-Encoding: identity' -D - http://127.0.0.1:8091/a.html \ | grep -Ei 'HTTP/|Content-Length|Content-Encoding' HTTP/1.1 200 OK Content-Length: 19 $ curl -s -H 'Accept-Encoding: identity' http://127.0.0.1:8091/a.html SURUM-2 taze govde # 8092, gzip_static always $ curl -s -H 'Accept-Encoding: identity' http://127.0.0.1:8092/a.html | xxd | head -1 00000000: 1f8b 0800 0000 0000 0203 0b0e 0d0a f5d5 ................ The on mode behaved: the client said it would not take gzip, the module backed off, the original went out. always sent raw gzip bytes starting with 1f8b — to a client that had explicitly said otherwise. The documentation gives two justifications for this mode: when there are no uncompressed files on disk anyway, or when ngx_http_gunzip_module is used. That second clause is the answer to my own question about who rescues the client that cannot decode gzip: that module decompresses the response on the server on its way to such a client. Turn always on without it, and the answer is you. How this file shows up in production The lab fixture came together by hand; the real question is how such a file comes to exist in an actual deployment. The most common route is copying build output into the docroot without --delete: $ printf 'v1 kisa govde\n' > dist/sayfa.html $ gzip -9 -nc dist/sayfa.html > dist/sayfa.html.gz $ rsync -a dist/ pub/ # second round: the build no longer emits .gz, only .html $ rm -rf dist && mkdir dist $ printf 'v2 govde biraz daha uzun oldu\n' > dist/sayfa.html $ rsync -a dist/ pub/ $ ls -l pub/ | awk 'NR>1 {print $5, $9}' 30 sayfa.html 34 sayfa.html.gz $ cat pub/sayfa.html; gzip -dc pub/sayfa.html.gz v2 govde biraz daha uzun oldu v1 kisa govde Nobody made a mistake: the build produced the right file, rsync copied the right file. The .gz that no longer has a counterpart in the source simply stayed where it was, and from that moment on it became the body nginx sent to gzip clients. With --delete added, the same round left only sayfa.html in pub2; the orphan concept disappeared. I walked into my own trap while measuring, and noting it is a debt of honesty: on the first attempt both versions had 9-byte bodies and I had written them within the same second. rsync's quick check skipped the file entirely because size and timestamp matched, and for a moment I was close to concluding "not even the original gets updated" — a wrong finding. Once I gave the bodies different lengths and put 1.2 seconds between them, the measurement started measuring the mechanism I actually wanted. The measuring rig produces findings of its own; you have to make sure it isn't. After you fix it, the job is not over Say you spot the orphan and refresh the .gz. When does nginx see that? With open_file_cache on, not immediately: alongside file descriptors, that cache also holds sizes and modification times, and open_file_cache_valid decides when they get revalidated. That mechanism got a full end-to-end treatment in a separate article three days ago: growing a file to 5,000 bytes while nginx sent 100 — the stale cached size goes into Content-Length, the body gets cut, and the status code is still 200. Both lessons from there apply here unchanged, except that this time the truncated body is a gzip stream: rewrite the file in place (same inode) and the client gets half a stream; replace it atomically with mv and the client gets the whole of the old version. This blog's deploy also mvs the entire dist directory into place, so it falls in the second group. One detail from that measurement is worth repeating, because it runs against intuition: the clock starts not when you fix the file but when the cache entry was born, and each worker carries its own entry. So you cannot say "I fixed the .gz, it will clear in 30 seconds". What you can say is "it stays stale for at most valid, and I do not know which second it turns". inactive is in play too: on a rarely requested path the entry is dropped anyway, so the window may never fill up in practice. What would I actually gain on my own site Now to the real question: is the risk worth it? I pulled my own site's real assets uncompressed and put them on the scale. | File | Raw | gzip -1 | gzip -9 | Gain | |---|---|---|---|---| | Home page HTML* | 122,826 | 35,129 | 29,724 | 5,405 bytes | | BaseLayout CSS | 114,436 | 25,417 | 20,237 | 5,180 bytes | | ClientRouter JS | 13,888 | 5,308 | 4,763 | 545 bytes | | TermHome CSS | 5,645 | 1,513 | 1,321 | 192 bytes | * The HTML row is there for comparison only; that body is produced by SSR, there is no file on disk, so it is not something gzip_static could ever win. The only real candidates are the assets under /_astro/. The level in my config is 1. Precompression's appeal is that it makes level 9 free: on my largest CSS, 20,237 bytes instead of 25,417 — a gain worth about a fifth of the compressed size. The CPU side went on the clock too: compressing that 114 KiB CSS 200 times spent 0.20 seconds of user time at level 1 and 0.45 seconds at level 9. Those numbers include the cost of starting the gzip process each time; I ran the same loop against a 1-byte file, measured that share as 0.11 seconds, and subtracted it. The values I derived from there: roughly 0.45 ms per compression at level 1 and 1.70 ms at level 9. What went on the clock is not nginx's in-process zlib call but gzip in a separate process, so treat these as an upper bound rather than a precise figure. Then I looked at my own site's live headers, and the arithmetic ended there: $ curl -sI --compressed https://mustafaerbay.com.tr/_astro/BaseLayout.CGi080bn.css \ | grep -Ei 'cf-cache-status|cache-control|age:' cache-control: max-age=31536000 cache-control: public, immutable age: 1713 cf-cache-status: HIT Hashed assets are immutable for a year and HIT at Cloudflare. The moments when my origin recompresses that CSS amount to the edge cache warming up. The HTML side is SSR: the body is produced from content.db, and there is no file on disk to precompress — gzip_static can do nothing on that path. As in the measurement where I grew a file and watched how many bytes nginx sent, the gain here is decided by the real distribution of traffic, not by theory. So my table reads like this: a few thousand bytes of gain, a CPU saving that rarely triggers, and in exchange the orphan-file risk plus the cache's staleness window. On top of that there is no switch for me to flip — I would first have to take /_astro/ off the proxy and serve it from disk, which means changing the shape of the deploy. The interesting part is that without measuring, I would have turned it on as a "well-known optimization". A decision frame The list I wrote for myself, on when I would turn gzip_static on: - Does the build itself produce the compressed file? A gzip pass run by hand is a pass that gets skipped in the first rush. - Is the copy into the docroot done with --delete, or can orphan .gz files survive it? - Is there a check after deploy that every X.gz still has its X? A three-line find does the job, and as long as nobody looks, no warning ever comes. - Are the mtimes of the .gz and the original kept in sync? Nginx does not verify it, but your own check can. - Do monitoring requests send Accept-Encoding: gzip? If not, monitoring never sees the body users get. - If open_file_cache is on, open_file_cache_valid is the worst-case staleness you are accepting — it does not tell you when the window ends after your fix. Use an atomic swap rather than an in-place write: at least the client gets the old body instead of half a body. - If there is a CDN in front, look for the gain there first: if the edge is already compressing, the level difference at the origin may never reach a user. What is left of it Precompression means putting derived data on disk. Disk has no idea what that data was derived from: the a.html.gz file does not know it is a.html's child, and nginx never asks. That is why every "compute the result once and store it" optimization brings a freshness debt along with it — and the one who has to pay that debt is standing in the deployment pipeline, not in the config file. My favourite detail across the whole measurement was that 304. Nginx was sending the wrong body, yet it was perfectly consistent about its wrongness: the size, the date, the ETag, all of them matched the file it was sending, byte for byte. That is how systems usually break — not through inconsistency, but through flawless loyalty to the wrong thing. Official Sources - nginx: ngx_http_gzip_static_module - nginx: ngx_http_gzip_module - nginx: open_file_cache and etag directives - nginx 1.31.7 source: ngx_http_gzip_static_module.c - nginx CHANGES

😤 一台服务器装三个服务，端口就开始打架。Nginx 3.1 万星，最常用的反向代理，一个域名挂多少服务都行，顺便把 HTTPS 证书也配了。 nginx/nginx

It is supported now, but only a very basic version of it nginx#771

I Grew a File to 5,000 Bytes, nginx Sent 100 Wandering through nginx configurations, I keep running into the same four lines. Always the same values, always the same order: open_file_cache max=1000 inactive=20s; open_file_cache_valid 30s; open_file_cache_min_uses 2; open_file_cache_errors on; Looking for why this quartet comes out identical everywhere, I found it: this block is nginx's own documentation Example:, verbatim — same values, same order. That's also why min_uses is 2 in every template; nobody picked that number for their own traffic, we are all copying the same example. The name is lovely too. "Open file cache" — it sounds like something every static file server ought to turn on. I considered adding it to my own blog's nginx, and before doing so I asked a question: what exactly does this buy me, and what am I giving up in return? I read the documentation; it wasn't enough. I read the source code, and something quite different came out of it. Then I ran it on my own machine, and the numbers put a behavior in front of me that the docs never mention: a file I had grown in place to 5,000 bytes was served by nginx for 28 seconds as 200 OK with Content-Length: 100. No error log, no warning. As far as the browser was concerned, everything was fine. This article is the anatomy of those 28 seconds. And at the end, the story of why I did not write those four lines into my own blog. The lab I ran the measurements inside the linuxkit virtual machine of Docker Desktop on my own Mac. There's no point in changing files on a production server and breaking live traffic; here, the only thing I can break is my own container. Docker Engine 27.4.0 (Docker Desktop) · Linux 6.10.14-linuxkit aarch64 14 vCPU · MemTotal 5,157,470,208 bytes · root filesystem: overlay nginx:alpine → nginx/1.31.6 I opened two server blocks in a single nginx instance. Same process, same kernel, same disk — the only difference being whether the cache is on. That way I eliminated the question "am I measuring the cache, or the machine's mood at that moment?" from the start: # :8081 — open_file_cache off (nginx's default) server { listen 8081; root /srv/a; open_file_cache off; access_log off; } # :8082 — cache enabled server { listen 8082; root /srv/b; open_file_cache max=1000 inactive=20s; open_file_cache_valid 30s; open_file_cache_min_uses 1; open_file_cache_errors on; access_log off; } I deliberately set min_uses to 1; the value of 2 that circulates in templates makes the first request of a measurement invisible, and I wanted to see the mechanism itself. First I checked the benefit If I'm going to defend a setting, I should know its gain first. I dropped to a single worker, fired 300 requests at the same 4,096-byte file, and listened to the worker with strace -c. | Syscall | open_file_cache off | open_file_cache on | |---|---|---| | openat | 300 | 0 | | fstat | 300 | 0 | | close | 599 | 300 | | sendfile | 300 | 300 | The gain is clear: three syscalls per request. openat and fstat disappear entirely, and close drops from 599 to 300 — with the cache off every request opens and closes a file descriptor, with it on there is nothing left to close and only the connections' own closes remain. The three together come to 300 + 300 + 299 = 899 calls; divided by three hundred requests, 2.997. The reason the close count is 599 rather than 600 is that one close fell outside the strace window; what's missing is the observation, not the call. It is very tempting to stop here and say "so it's three times faster." strace reported times between 10 and 22 µs for these calls. But I don't trust those numbers, because strace slows down the very thing it measures: it stops and inspects every syscall through ptrace. The usecs/call figures I have aren't the real cost, they're an upper bound inflated by the observer's own overhead — which is why I derive no timing claim from them. So I removed the observer and measured end to end. ab with 20,000 requests, keepalive on, concurrency 1. Three rounds with off first, three rounds with on first — to eliminate ordering bias as well: rounds with off first : off avg 16,711 rps | on avg 17,635 rps (+5.53%) rounds with on first : off avg 17,486 rps | on avg 17,914 rps (+2.45%) ──────────────────────────────────────────────────────────────────────── all 12 runs : off avg 17,098 rps | on avg 17,775 rps (+3.95%) off range: 16,441 – 17,932 rps (spread 1,491) on range: 17,194 – 18,577 rps (spread 1,383) The averages favor on, and so do five of the six pairings — all but one, where in L8b's second round off at 17,497 beats on's 17,194. And looking a little closer at the numbers: off's best run (17,932) is faster than on's worst run (17,194). The two groups' ranges overlap. The difference between the averages is 676 rps, whereas the spread within off alone is 1,491 rps — the noise is 2.2 times the effect I'm trying to measure. The conclusion I have to accept: in this setup I could not measure open_file_cache's end-to-end gain. The syscall count is certain; its effect on time stayed below my machine's own fluctuation. Three syscalls per request is a real saving, but for it to mean anything the rate of file opens has to be very high — how high, I did not measure, so I'm not going to invent a threshold number. The one certain thing: on a blog that sees a few hundred static file requests a day, you can't even measure what you gained. So is the cost unmeasurable too? That part turned out quite differently. I asked for 5,000 bytes and got 100 The scenario is as ordinary as I could make it: there's a 100-byte file, nginx served it once, and two seconds later I grew the file in place to 5,000 bytes. In place meaning with the shell's > redirection — that is, with the thing most scripts, most editors, and most "update this file" commands do. First I proved that this really is an in-place write, because the whole story hangs on it. I ran the check on a separate /tmp file so as not to disturb the one being served — the two numbers below belong to that file: # the > redirection inode before = 34071674 inode after = 34071674 → SAME # replacement via mv inode before = 34071674 inode after = 34071675 → CHANGED The file is now 5,000 bytes on disk, its mtime updated, its inode unchanged (34071666). I sent one request per second to both ports and recorded what came back: open_file_cache off open_file_cache on t=+0s 200 cl=5000 received=5000 first=B 200 cl=100 received=100 first=B t=+1s 200 cl=5000 received=5000 first=B 200 cl=100 received=100 first=B ... (28 consecutive samples) ... t=+27s 200 cl=5000 received=5000 first=B 200 cl=100 received=100 first=B t=+28s 200 cl=5000 received=5000 first=B 200 cl=5000 received=5000 first=B The left side is what I expected: with the cache off, the file changed and nginx gave its new state. The right side did this for 28 seconds — first byte B, so the new content; length 100, so the old size. nginx made no new open call — the descriptor it already held still pointed at the same inode, so it could read the new content; it sent the first 100 bytes from that and threw away 4,900. This is also why openat came out at zero in the syscall count. And since Content-Length is also 100, there is no inconsistency from the client's point of view: 100 bytes were promised, 100 bytes arrived, 200 OK. Nothing says that 98% of the content is missing. I checked the error log. Not a single line from start to finish of this experiment — no [error], no [warn], no [alert]. nginx isn't even aware it did this, because from its own point of view there is no error: it was asked to send 100 bytes, and it sent 100 bytes. The timing isn't a coincidence either. The request that populated the cache went out 2 seconds before I grew the file. open_file_cache_valid is 30 seconds. 30 − 2 = 28. I saw the transition at exactly t=+28s. The arithmetic lines up precisely, which tells me what I captured is mechanism, not noise. The source code says what it caches I went back to the documentation. nginx.org lists what open_file_cache stores: open file descriptors, their sizes and modification times; information on the existence of directories; and file lookup errors. For open_file_cache_valid there is a single sentence: it sets a time after which elements "should be validated." Validated how? The docs don't say. What gets compared — mtime, size? That gap is exactly where my 28 seconds live, so I opened ngx_open_file_cache.c. Here is the body of the cache hit: if (file->use_event || (file->event == NULL && (of->uniq == 0 || of->uniq == file->uniq) && now - file->created < of->valid /* ... symlink checks omitted ... */ )) { if (file->err == 0) { of->fd = file->fd; of->uniq = file->uniq; of->mtime = file->mtime; of->size = file->size; I shortened one condition in that quote: the leading file->use_event branch skips the valid check entirely and trusts file events coming from the kernel. On Linux that branch never runs (vnode events belong to kqueue systems), which is why my measurements are clean — but I should say that every result in this article holds for Linux/epoll. There's the answer. As long as now - file->created < of->valid holds, there is no syscall at all here. The file descriptor comes from the cache — but so do size and mtime. nginx writes the response's Content-Length not by looking at the file on disk, but by recalling the size at the moment it created the entry. A descriptor open on the same inode reads the new content — the file changed, but that descriptor still points at the same file. So nginx takes the body from the new content and the length from the old bookkeeping. When the two conflict, the bookkeeping wins. So where exactly does that cached size enter the response? In the static file module, a single number determines two separate places at once: /* ngx_http_static_module.c — the same of.size goes to two places */ r->headers_out.content_length_n = of.size; r->headers_out.last_modified_time = of.mtime; /* ... */ b->file_pos = 0; b->file_last = of.size; of.size writes the Content-Length header, and it also draws the b->file_last boundary that tells sendfile how many bytes to send. The promise made to the client and the number of bytes actually sent are fed from the same source; when that source goes stale, both go stale together. That's why the response looks internally consistent while being incomplete relative to the file. In the same block last_modified_time comes from of.mtime too, so the Last-Modified header also stays old throughout the window. More than that, nginx builds the ETag out of those same two fields: /* ngx_http_core_module.c — ngx_http_set_etag */ ngx_sprintf(etag->value.data, "\"%xT-%xO\"", r->headers_out.last_modified_time, r->headers_out.content_length_n); The ETag is the hexadecimal pairing of a stale mtime with a stale size. So that truncated 100-byte response goes out carrying an old Last-Modified and an old ETag; a client or an intermediate cache arriving with If-None-Match or If-Modified-Since is told "unchanged." That's why nothing looks inconsistent on the client side: the response's entire bookkeeping agrees with itself, and disagrees only with the file. When I first read this, I thought: the benefit was saving three syscalls. The cost is that same saving. The fstat I gave up was precisely the call that would have told me how much the file had grown — by skipping it, the cache skips the time and the truth together. nginx doesn't look at mtime, it looks at the inode What happens when the valid period expires? The code re-examines the path while still holding the cached descriptor: of->fd = file->fd; of->uniq = file->uniq; rc = ngx_open_and_stat_file(name, of, pool->log); And inside ngx_open_and_stat_file, the place where the decision is made: if (of->fd != NGX_INVALID_FILE) { if (ngx_file_info_wrapper(name, of, &fi, log) == NGX_FILE_ERROR) { of->fd = NGX_INVALID_FILE; return NGX_ERROR; } if (of->uniq == ngx_file_uniq(&fi)) { goto done; } } What is ngx_file_uniq? src/os/unix/ngx_files.h says it in one line: #define ngx_file_uniq(sb) (sb)->st_ino The inode. On revalidation nginx does not compare mtime, does not compare size; it looks only at the inode number. mtime and size are copied from the fresh stat result at the done: label, but they never enter the decision. This produces a counterintuitive result. When you change a file in place, the inode stays fixed, which means nginx's validation criterion never budges. When you replace a file with mv, the inode changes and nginx sees the difference at once. !Diagram The measurement confirms it. I wrote 900 bytes into a temporary file and moved it over a 300-byte file with mv — the classic atomic replacement. The inode went from 34071676 to 34071674. On the cache-enabled side, the result: t=+0s ... t=+27s 200 cl=300 received=300 first=E ← old file, complete t=+28s 200 cl=900 received=900 first=F ← new file 28 seconds again, but this time a consistent staleness. The body is the whole of the old file, the length is the old file's length. A stale response, not a broken one. I rather enjoy the irony here: the move that looks "safe," updating a file in place, produces a broken response. The move that looks "aggressive," deleting a file and moving a new one into its place, produces only staleness. We've been using atomic rename for years so that half-written files never become visible; it turns out it has a second benefit in front of an nginx with open_file_cache enabled. Shrinking is loud, growing is quiet I set up the same experiment in reverse: I shrank the 5,000-byte file in place to 100 bytes. The size in the cache is now larger than the file on disk. t=+0s ... t=+27s 200 cl=5000 received=100 curl_rc=18 t=+28s 200 cl=100 received=100 curl_rc=0 (The loop skipped a second, so the 28-second window holds 27 samples rather than 28; the alert count below matches it exactly.) curl exits with 18 — the response was cut short. nginx promised 5,000 bytes, could find 100, and closed the connection. And this time it wrote to the error log: [alert] 87#87: *86 sendfile() reported that "/srv/b/shrink.html" was truncated at 100 while sending response to client, client: 127.0.0.1, server: _, request: "GET /shrink.html HTTP/1.1", host: "127.0.0.1:8082" I counted this line 27 times within the 28-second window; in the growth scenario there is no counterpart to it at all. The logic makes sense: when shrinking, the kernel tells nginx "there aren't as many bytes as you asked for," and nginx complains. When growing, the 100 bytes it asked for are more than available, and nobody objects. The upshot is that open_file_cache has two failure modes, and the dangerous one is the quiet one. If your file shrinks, you get an alert-level log entry, the client sees an error, and someone notices. If your file grows, nothing happens — the response looks valid, it just has no ending. Picture the first 100 bytes of an HTML file: a page that starts with <!doctype html> and stops in the middle. 200 OK. A file that didn't exist went on existing for 28 seconds I wanted to measure the open_file_cache_errors on line too, because it always comes bundled in the templates. I requested a nonexistent path, got a 404, then created the file: file created: 200 bytes off on t=+0s 200 cl=200 received=200 404 cl=153 received=153 ... (28 consecutive samples) t=+27s 200 cl=200 received=200 404 cl=153 received=153 t=+28s 200 cl=200 received=200 200 cl=200 received=200 The file is on disk, 200 bytes, readable. nginx said it wasn't there for 28 seconds. Thinking "maybe it keeps trying but can't get permission," I listened to the worker with strace: I sent 20 requests to the existing file, all returned 404, and the number of openat, newfstatat, statx and fstat calls touching that path came out zero. The decision is made entirely from the cached error record; the disk is never consulted. One more detail, the kind that will confuse you in operations: nginx writes a line to the error log for these cached 404s as well — [error] 87#87: *238 open() "/srv/b/sonra.html" failed (2: No such file or directory) The open() call was never made, but the record says open() failed, because the error text is produced from the cached state. So while the log says "no such file," the file is sitting right there when you look with ls. You don't want to be untangling that pair at midnight. The clock starts at first open, not last access The last question I had left was this: if a file is requested continuously, does the valid window get refreshed? If it does, a heavily trafficked file might never update; if it doesn't, the window is a fixed deadline from the first open. I ran it with two files. I first requested X at t=+0s and Y at t=+10s; at t=+14s I grew both in place to 900 bytes — same instant, same change, same worker. From t=+14s onward I requested both of them every second, so throughout the measurement their traffic was identical. The only difference between them was the moment their entries were born. t=+29s X=300B Y=300B t=+30s X=900B Y=300B ← X refreshed t=+39s X=900B Y=300B t=+40s X=900B Y=900B ← Y refreshed X refreshed exactly 30 seconds after its first open, and Y exactly 30 seconds after its own first open. The 10-second gap between them is identical to the gap between the two entries' first-open times. Both were receiving uninterrupted requests while this happened — so traffic simply doesn't concern the created counter. inactive enters here too, and most write-ups skip it. The block actually carries two separate clocks: valid is an expiry counted from the entry's birth, while inactive is an abandonment timer counted from its last access. An entry nobody requests for the template's 20 seconds falls out of the cache, and the next request does a fresh fstat. So the real staleness window is the smaller of the two: on a busy path valid decides, while on a rarely requested path the entry drops out via inactive before the window ever fills. In my measurements I sent at least one request per second, which is why valid always decided. The practical consequence matters: X stayed stale for 16 seconds, Y for 26. The same file change, the same configuration, two different staleness durations. valid is not a "freshness interval" but an expiry date counted from each entry's own birth. So the sentence "it stays stale for at most 30 seconds" is true, but "it fixes itself after 30 seconds" is not — which second it fixes itself depends on when that entry was born, and you don't know when that was. Fourteen workers, fourteen separate expiry dates Running with worker_processes auto, my VM starts 14 workers. After requesting the file for a while, I scanned every worker's /proc/<pid>/fd directory: workers holding an fd for flap.html: 4029:1 4030:1 4031:1 4032:1 4033:1 4034:1 4035:1 4036:1 4037:1 4038:1 4039:1 4040:1 4041:1 4042:1 workers holding an fd = 14 total fds = 14 A single file, fourteen separate open file descriptors. Because open_file_cache is not a shared zone; every worker has its own private cache. The documentation doesn't say this outright, but the fd table does. Combining that with the previous section gives the following — and I did not measure this, I derived it from the two measurements: since each worker's entry is born at that worker's own first open, a single file change produces as many different expiry dates as there are workers. In theory, then, some requests could get the new content while others get the old, and what the client sees could jump back and forth across consecutive requests. Here's where I have to be honest: I did not see that jumping in my own run. After the rename I sampled at 12 requests per second for 50 seconds; up to t=+13s every response was old (300 bytes), and from t=+14s onward every response was new (900 bytes). Not a single mixed second. In my priming traffic the 14 workers must have created their entries at moments very close to one another, so their windows nearly coincided. The mechanism allows a mixed window; I couldn't trigger one. The second line item of the cost: open file descriptors With min_uses 1 I requested 50 separate files once each and counted the worker's fd table: fds held for the cache-ON root (/srv/b) : 50 fds held for the cache-OFF root (/srv/a): 0 the worker's total fd count : 64 The cache really does what its name says: it keeps files open. That max=1000 in the template deserves attention — but not in the way it looks. The documentation says max sets the maximum number of elements in the cache, not of open descriptors; entries that never pass the min_uses threshold, and cached error records, hold no descriptor at all yet still occupy that thousand-seat quota. So 1,000 is a ceiling on descriptors rather than an equality: at most 1,000 open files per worker, at most 14,000 across 14 workers. Even as a ceiling, it is a number that intersects directly with worker_rlimit_nofile and system limits; I previously measured who sets file descriptor limits on fs.file-max and saw that in containerized environments these numbers come from unexpected places. When writing max into a configuration, it's worth remembering that this is not a memory setting but a file descriptor budget. It's also worth touching on what min_uses actually does, because its name misleads. According to the source code, if an entry doesn't reach the min_uses threshold its descriptor is closed, but the entry itself stays in the cache; on the next access the file is reopened. So min_uses 2 means "don't hold a descriptor for one-off files" — it does not mean closing the staleness window. A decision framework After the measurements, these are the rules I set for myself: - Don't enable it if the content of the files you serve can change at the same path. A script that writes in place, a directory synced with rsync, a file produced by a CMS — all of them are candidates for the growth scenario, which is to say candidates for silent truncation. - If you do enable it, only ever replace files with an atomic rename. That downgrades a broken response to a stale response. A free improvement, and a good habit anyway. - Paths whose contents never change are the best candidates. Content-hashed asset files (app.a1b2c3.js) by definition never change; there is no staleness there, only gain. - Think about errors on separately. Its gain is real during 404 storms, but its cost is a support ticket of the form "the file I just uploaded isn't showing." It should enter the configuration as a decision independent of valid. - Choose valid according to your deploy frequency. Thirty seconds is far too long on a system that deploys every minute; on one that deploys hourly it's as harmless as the gain is small. - If you use try_files, proxying won't save you. I had assumed "a request that goes upstream opens no file"; the source code corrected me. ngx_http_try_files_module.c probes paths by calling ngx_open_cached_file with the location's own open_file_cache settings. It sets of.test_only = 1, but ngx_open_cached_file honors that flag only when there is no cache; with a cache configured the file really is opened and cached. So in the most common pattern of all — location / { try_files $uri @backend; } — the cache is live, and together with errors on a static file you just uploaded will keep falling through to the proxy for as long as valid. The only place the cache truly doesn't reach is a plain proxy_pass block that fetches the response body from upstream. - If you get stuck, the exit is nginx -s reload. Because the cache is per-worker, every entry is wiped the moment a reload spawns new workers; that's the one-command way out of a stale window without waiting for valid. - Set up something that can detect it. Growth is quiet, so it won't announce itself: a small post-deploy check comparing the Content-Length from curl -sI against stat -c %s on disk catches this article's whole scenario. For the shrinking side, grepping the error log for truncated at is enough. What I did on my own blog The last few points led me to my own configuration, and there I ran into an unpleasant surprise: the first file I opened was the wrong one. The repository holds deploy/nginx-mustafaerbay-final.conf, and the name sounds self-assured enough; but deploy/nginx-live/README.md says that file is the clean-install template, and that the copy actually running on the server is deploy/nginx-live/sites-enabled_mustafaerbay.conf. I came close to ending this article with a decision made from the wrong file. Here is what the live file looks like: seventeen location blocks, seven of them going through proxy_pass to the Astro SSR process at 127.0.0.1:3040, and a group of them closing off scanner paths outright with return 444. open_file_cache appears nowhere, so the default is off. Up to this point my guess held: on my site nginx mostly doesn't serve files, it relays a conversation. But there are three places that read a file from disk: location /.well-known/acme-challenge/ { root /var/www/letsencrypt; } location /_admin/analytics/ { alias /var/www/analytics/; index mustafaerbay.html; auth_basic "Analytics"; auth_basic_user_file /etc/nginx/.htpasswd-analytics; try_files $uri $uri/ =404; } location @astro_prev { root /opt/mustafaerbay/dist-old/client; try_files $uri =404; expires 1y; add_header Cache-Control "public, immutable" always; } The first is certbot's validation directory; a handful of files get read for a few seconds while a certificate is renewed, and there's nothing for a cache to win there. The last is the fallback block that kicks in when an asset under /_astro/ returns 404 at the origin. There's no staleness risk there, because the files are content-hashed and immutable: the same name always means the same content. There's no gain either; that block only runs in a narrow window after a deploy, while old HTML asks for a new asset, and it has expires 1y and Cloudflare in front of it. What actually stopped me was the one in the middle. My ops panel's HTML sits on disk under a fixed name — no content hash, no immutable, and unlike the assets, the content at that same address changes over time. Had I enabled the cache, which of this article's two failure modes I'd meet would depend entirely on how that file gets refreshed: written in place means a truncated panel, replaced by a move means a stale one. I can't tell which from the repository, because the file lives only on the server. I'd rather not tie a setting to the update habits of a file whose behavior I can't see. On top of that, both of these blocks use try_files — and as I just learned, the cache reaches in there too. My deploy script also works by moving directories: rm -rf "$DEST/dist-old" [ -d "$DEST/dist" ] && mv "$DEST/dist" "$DEST/dist-old" mv "$DEST/dist-new" "$DEST/dist" dist-old becomes an entirely different directory on every deploy. The key in the cache, meanwhile, is the full file path — the path stays the same while the file behind it changes. Exactly the rename scenario from the lab: a consistent but old response, for as long as valid. My decision: I didn't enable it. The gain is three syscalls per request and I can't even measure those three on my own traffic; in exchange the configuration would acquire a trap contingent on how my panel gets updated, plus four lines someone could later copy into the wrong place. I had already learned how easily you can lose track of what actually gets read in an nginx configuration the day I left sixty-eight backup files behind. Today I refreshed that lesson by mixing up which file was the live one. Closing While working through this setting, one question got clearer in my head: when we evaluate a cache, we usually ask "how much does it speed things up?" The better question is this one — what truth does this cache forget, and for how long does it forget it? The truth open_file_cache forgets is the file's size. The duration of the forgetting is valid. Once you learn both at once, the setting's name suddenly reads more honestly: this isn't an accelerator, it's an agreement about how long to remember. nginx is telling you, "I won't ask again about this file for 30 seconds." And in return you're accepting this: if the truth about that file changes within those 30 seconds, I will tell the client what I remember rather than what is true. The sentence in the documentation that says it "can store open file descriptors, their sizes and modification times" was stating the whole agreement from the start. If it stores the size, it will say the wrong thing when the size changes. I didn't understand that by reading the sentence; I understood it by going looking for where 4,900 bytes had gone. Official Sources - nginx — ngx_http_core_module: open_file_cache - nginx source — src/core/ngx_open_file_cache.c - nginx source — src/os/unix/ngx_files.h (ngx_file_uniq) - nginx — ngx_core_module: worker_processes - nginx source — src/http/modules/ngx_http_try_files_module.c - nginx source — src/http/ngx_http_core_module.c (ngx_http_set_etag) - nginx source — src/http/modules/ngx_http_static_module.c