redis

This is a cool addition coming to Redis 8.12 redis#15649

Redis 8.12 RC2's SEXPIRE command cuts a set membership check from two round trips to one --- title: "Redis 8.12 RC2's SEXPIRE command cuts a set membership check from two round trips to one" published: true published_at: 2026-11-02 13:00 +0200 description: "Redis 8.12 RC2 adds per-member TTLs to sets. A membership check dropped from roughly 585 to 290 microseconds over 2,000 calls, but setting the TTL itself wasn't faster, and converting a small intset cost 15 times more memory." tags: redis, database, performance, devops ai_disclosure_level: ai_assisted --- A Redis set has always had one expiry for the whole key. If you wanted one member of a set to disappear on its own, say a session token in a set of active sessions, or a temporary grant in a set of permissions, you built a side index yourself: a sorted set with the expiry as the score, checked on every read, swept by a cron job. Redis 8.12, currently at release candidate 2, adds that directly to the set type. I built it from source, since there's no Docker image for an RC yet, and measured what changes when the server does this instead of your application code. The new commands are SEXPIRE, SPEXPIRE, SEXPIREAT, SPEXPIREAT to set an expiry on specific members, SPERSIST to clear it, and STTL, SPTTL, SEXPIRETIME, SPEXPIRETIME to read it back. There's no official command reference page for any of them yet; I pieced the syntax together from COMMAND DOCS and the release notes. The headline number The actual value of this feature isn't in setting the expiry. It's in checking membership afterwards. Under the old pattern, "is this member still valid" means SISMEMBER plus a ZSCORE lookup against your own expiry index, compared against the current time client-side. Under the new one, SISMEMBER alone already reflects the TTL, because the server has removed the member once it expired. I populated two 2,000-member sets, gave every member the same TTL, then ran 2,000 validity checks against each pattern, three times: | run | old pattern (SISMEMBER + ZSCORE) | new pattern (SISMEMBER alone) | difference | |---|---|---|---| | 1 | 997.7 ms | 475.8 ms | 52.3% faster | | 2 | 889.0 ms | 444.6 ms | 50.0% faster | | 3 | 994.3 ms | 423.6 ms | 57.4% faster | That's one round trip removed from every single read, and it held steady across all three runs: roughly half the time, every time. On loopback, where a round trip is already cheap, that's still a 50% cut. Over a real network to a managed Redis instance, removing a round trip matters more, not less, though I only measured loopback here and I'm not going to claim a number I didn't run. The number that argues against it Setting the expiry doesn't get any faster. I benchmarked writing the TTL itself, 2,000 members, three runs, old pattern being a plain ZADD into a side index versus the new SEXPIRE call: | run | old pattern (ZADD to a side index) | new pattern (SEXPIRE) | difference | |---|---|---|---| | 1 | 497.2 ms | 506.7 ms | 1.9% slower | | 2 | 430.0 ms | 438.9 ms | 2.1% slower | | 3 | 483.7 ms | 599.7 ms | 24.0% slower | SEXPIRE was never faster to call than a plain ZADD, and in one run it was a quarter slower. That's not surprising once you know what it has to do: find the member inside the set's internal structure, possibly convert the set's whole encoding to accommodate per-member metadata, and schedule it for the active-expiry cycle. A ZADD into an unrelated sorted set does none of that. The saving in this feature is entirely on the read side, where you stop needing the side index at all. If your workload only ever writes expiring members and never re-checks them, this feature costs you a little and gives nothing back. What it costs Nobody mentions this part. A set that has never had a member TTL stores members as compactly as possible: a plain intset for small sets of integers, a listpack for small sets of short strings. The moment any member gets a TTL, the set needs a separate expiry slot for every member, TTL or not, and the encoding changes to carry that. For a listpack-encoded set of short strings, the conversion to listpackex cost 42-75% more memory, worse for smaller sets since the per-entry overhead is proportionally larger: | set size | before | after SEXPIRE on one member | overhead | |---|---|---|---| | 10 members | 70 bytes | 122 bytes | +74.3% | | 50 members | 270 bytes | 402 bytes | +48.9% | | 100 members | 521 bytes | 753 bytes | +44.5% | | 128 members | 689 bytes | 977 bytes | +41.8% | That's expected, if undocumented. What I didn't expect: a small intset, 300 plain integers, has no TTL-aware equivalent at all. Giving one member a TTL doesn't convert it to some integer-specific expiring encoding, it jumps straight to a full hashtable: encoding before: intset, 636 bytes encoding after SEXPIRE on one member: hashtable, 10388 bytes delta: +9752 bytes (+1533.3%) A 15-times memory increase, for giving one out of 300 integers an expiry. And it doesn't come back. I called SPERSIST to remove the TTL again, and the set stayed on listpackex/hashtable rather than reverting to listpack/intset: > SPERSIST t MEMBERS 1 a (integer) 1 > OBJECT ENCODING t "listpackex" If you're going to use this on a set that's mostly stable and rarely touched by expiry, know that touching it once is permanent, memory-wise, until the key is rewritten some other way. What it refuses SEXPIRE won't take a negative TTL, which is a real departure from the key-level EXPIRE command, where a negative or past expiry deletes the key immediately: > SEXPIRE myset -1 MEMBERS 1 x (error) ERR invalid expire time, must be >= 0 If you want to delete a member immediately, you still use SREM; SEXPIRE with 0 works and deletes it on the next expiry pass, but a negative value is simply rejected. It also refuses to run on the wrong type: > SET strkey hello > SEXPIRE strkey 100 MEMBERS 1 x (error) WRONGTYPE Operation against a key holding the wrong kind of value And a nonexistent key or nonexistent member both return -2 from STTL, same as the hash-field TTL commands that came before this. Under concurrency I fired fifty concurrent SEXPIRE calls at the same member from fifty threads, each setting a different TTL. No errors, no corruption, and the final TTL matched whichever call the server happened to execute last, since Redis serialises command execution regardless of how many clients are connected. This isn't really a test of the feature so much as a reminder that the usual Redis guarantee holds here too: nothing about per-member TTLs introduces a new race inside the server. The race that matters is the one on your side, between reading a member and acting on it, and that hasn't changed. A documented claim, checked I couldn't find a documented statement about whether SINTERSTORE and SUNIONSTORE preserve member TTLs on the destination set, so I checked directly. They do: > SADD s1 x y z > SEXPIRE s1 100 MEMBERS 1 x > SINTERSTORE dest s1 > STTL dest MEMBERS 1 x (integer) 100 The TTL on x carried through to dest with the same remaining time. I'd half expected a store operation to rebuild the set fresh and drop per-member metadata the way some operations silently reset a counter or a type. It doesn't. Watching it in production INFO stats has two new counters: expired_subkeys and expired_subkeys_active. INFO keyspace grew a subexpiry field showing how many keys in the database currently have at least one member or field with a TTL. I set a short TTL on two members of a five-member set, never read the key again, and watched the counters change on their own: keyspace right after: subexpiry=1 expired_subkeys_active before: 3 (... 3 second wait, no access to the key at all ...) expired_subkeys_active after: 5 keyspace after: subexpiry=0 That confirms the background active-expiry cycle genuinely sweeps expired set members without anything ever reading the key, which matters if your workload sets TTLs on keys it then rarely touches. Without that active sweep, memory for expired-but-unread members would sit around until something happened to read the key and trigger lazy cleanup. What I got wrong on the way I tried to check whether SINTERSTORE preserves TTLs by running SINTERSTORE dest 1 s1, copying the numkeys argument style from SINTERCARD. The result came back as an empty set every time, with or without a TTL anywhere near it. I nearly wrote that down as "SINTERSTORE silently drops everything when any source member has a TTL", which would have been a dramatic finding and also completely wrong. SINTERSTORE doesn't take a numkeys argument at all; its syntax is just SINTERSTORE destination key [key ...]. My 1 was being read as a literal key named "1", which doesn't exist, so the intersection was always empty, TTL or no TTL. Running the same broken command against a set with zero TTL activity produced the identical empty result, which is what gave it away: a real TTL-related bug wouldn't also break on a completely untouched set. Run it yourself There's no Docker image for 8.12 RC2 yet, so building from source is the fastest path: git clone --depth 1 --branch 8.12-rc2 https://github.com/redis/redis.git cd redis && make -j4 BUILD_TLS=no ./src/redis-server --daemonize yes --save "" --port 6379 The commands behind the numbers above, run against that server: redis-cli SADD myset a b c redis-cli SEXPIRE myset 100 MEMBERS 1 a redis-cli STTL myset MEMBERS 1 a # 100 redis-cli STTL myset MEMBERS 1 b # -1, no TTL set redis-cli OBJECT ENCODING myset # listpackex, once any member has a TTL The two timing benchmarks, with redis-py: import redis, time r = redis.Redis(port=6379) N = 2000 r.sadd('newset', *[f'm{i}' for i in range(N)]) for i in range(N): r.execute_command('SEXPIRE', 'newset', 600, 'MEMBERS', 1, f'm{i}') r.sadd('oldset', *[f'm{i}' for i in range(N)]) now = time.time() for i in range(N): r.zadd('oldset:idx', {f'm{i}': now + 600}) t0 = time.perf_counter() for i in range(N): r.sismember('newset', f'm{i}') print('native:', time.perf_counter() - t0) t0 = time.perf_counter() for i in range(N): r.sismember('oldset', f'm{i}') r.zscore('oldset:idx', f'm{i}') print('side-index:', time.perf_counter() - t0) If you're maintaining a side index purely to expire individual set members, and you check membership more often than you set it, moving to SEXPIRE on 8.12 removes a round trip from every read for roughly double the read throughput, at the cost of a permanent jump in the set's memory footprint and a write path that's no faster than what you had before. Check which of those two costs you more before switching, and don't bother if you're only ever setting expiries and never checking them back.

Looks like Redis is finally adding the ability to expire items in a set. They've had this for hashes (via commands like `HSETEX` and `HEXPIRE`) since 8.0, so I wonder what's been keeping them from adding them to sets. github.com/redis/redis/releases/tag/8.12-rc1

Redis MCP 能让 AI 把对话存进事件流，再按记录读回来，MIT 开源，适合给客服助手留聊天记录。 留意官方源码：范围读取工具默认只取1条。要总结整段对话，得指定读取条数，别让 AI 拿一条消息概括全场。 redis/mcp-redis

Been getting back into exploring and contributing to open source projects lately. Three of my PRs ended up driving the v1.1.0 release of Redis’s go-redis-entraid. Pretty cool to see my contributions turn into a release, once again! github.com/redis/go-redis-entraid/releases/tag/v1…

@RosuGrigore @Redisinc @antirez @valkey_io I had also run Lean and TLA plus on Redis and found some crash bugs like redis#15844 , but still waiting for any action from the Redis team

RedisのKEYSとSCANはどう使い分ける？本番で安全にキーを列挙する実装 Redisで「cache:product:*に一致するキーを探したい」とき、KEYSとSCANのどちらを使っていますか。 KEYSは簡単ですが、データ量が増えた本番環境で無造作に実行すると、ほかのコマンドの応答を遅らせる可能性があります。一方のSCANは少しずつ列挙できますが、同じキーが複数回返ることがあるなど、独特の注意点があります。 この記事ではRedis公式ドキュメントとNode.js公式クライアントの実装を確認し、運用調査・バッチ処理での使い分けを整理します（確認日：2026年10月14日）。 先に結論 方法 向く用途 注意点 KEYS cache:product:* 小規模な検証環境での調査 全体を一度に走査し、処理中にほかの要求へ影響し得る SCAN 0 MATCH cache:product:* COUNT 100 本番での段階的な列挙 カーソルが0になるまで繰り返す。重複・空配列に注意 キーを別のSetなどで管理 アプリケーションが頻繁に一覧を取得する場合 インデックスの登録・削除・整合性を設計する キーの列挙自体が頻繁な要件なら、毎回Redis全体を検索する設計も見直すのがポイントです。 KEYSが本番向きではない理由 例えば、Redis CLI（Redisの操作用コマンドラインツール）で次を実行できます。 KEYS cache:product:* *は任意の文字列に一致するパターンです。KEYSは条件に合うキーの一覧をまとめて返します。Redisの公式コマンド仕様では、KEYSの計算量はO(N)（格納されているキーの数Nに比例）です。 Redisはコマンド処理に共有の実行資源を使うため、大規模なキー空間の走査は応答遅延につながります。公式ドキュメントでも通常のアプリケーションコードでのKEYS利用を避け、SCANや集合型を検討するよう注意しています。 ※少量のローカルデータを調べるときまで禁止する必要はありません。重要なのは環境・件数・実行頻度です。 SCANはカーソルで分割して読む SCANは一度にすべてのキーを返さず、カーソル（次にどこから読み進めるかを示す値）を使って処理を続けます。 SCAN 0 MATCH cache:product:* COUNT 100 0：最初の読み取りを開始するカーソル。 MATCH cache:product:*：取得した候補から該当キー名を絞り込む条件。 COUNT 100：各回の作業量を調整するヒント。100件ちょうどを保証しません。 応答は「次のカーソル」と「今回のキー配列」の2要素です。たとえば次のように返ります。 1) "42" 2) 1) "cache:product:100" 2) "cache:product:205" 続けてSCAN 42 MATCH cache:product:* COUNT 100を呼びます。返されたカーソルが0になったときに一巡が完了します。 MATCHに一致するキーが少ない場合、配列が空でも次のカーソルが0ではないことがあります。「0件返ったから終了」と判定するのは誤りです。 Redisの公式仕様では、SCANの計算量はおおむね1回の呼び出しがO(1)、一巡全体ではO(N)です。負荷がなくなるわけではなく、処理を小分けにできるのが利点です。 redis-cliで少しずつ調べる Redisに接続できる環境なら、次のコマンドでキー名を列挙できます。 redis-cli --scan --pattern 'cache:product:*' redis-cliはRedisを操作するCLI、--scanは内部でSCANを繰り返すオプション、--patternは名前を絞る指定です。引用符はシェルが*を勝手に展開しないようにするために付けます。 運用中の本番環境では、まず読み取りだけで確認してください。一致したものを無条件に削除するコマンドへパイプでつなぐのは避けましょう。 Redisの接続先が意図した開発・ステージング・本番のどれなのかも、実行前に明示的に確認します。 Node.jsのバッチ処理例 公式のnode-redisクライアントを使うと、SCANのカーソル管理をscanIterator()に任せられます。 npm install redis npm installは依存パッケージを追加するコマンド、redisは公式Node.jsクライアントのパッケージ名です。次の例は単一Redisインスタンスを想定します（redis-scan.mjsとして保存）。 import { createClient } from "redis"; const client = createClient({ url: process.env.REDIS_URL ?? "redis://localhost:6379", }); client.on("error", (error) => console.error("Redis:", error)); await client.connect(); try { for await (const keys of client.scanIterator({ MATCH: "cache:product:*", COUNT: 100, TYPE: "string", })) { // 現行node-redisのscanIteratorはキーの配列を順次返す。 if (keys.length === 0) continue; // この例は値が文字列のキーだけを対象とする。 const values = await client.mGet(keys); for (let i = 0; i < keys.length; i++) { if (values[i] === null) continue; // 走査中に期限切れとなる場合 console.log(keys[i], values[i]); } } } finally { await client.close(); } 実行するときはnode redis-scan.mjsです。nodeはNode.jsでファイルを実行するコマンドです。REDIS_URL環境変数で接続先を切り替えられます。 TYPE: "string"は文字列型のキーに絞る指定です。ただし、キー名の接頭辞だけでは値の型は決まりません。型が混在する用途では、型ごとに処理を分けます。 mGetは複数の文字列キーを一度に取得するコマンドです。ここでの例は単一インスタンス向けであり、Redis Clusterの複数スロットにまたがる複数キー処理にはそのまま流用しません。 また、クライアントのバージョンによりイテレーターが返す形が異なる場合があります。利用中のredisパッケージのREADMEや型定義を確認してください。 SCANを使っても残る3つの落とし穴 1. 同じキーが複数回返る SCANはキーが重複して返る可能性があると公式に明記されています。単純なログ出力や読み取りなら許容できる場合もありますが、課金・集計・通知・ジョブ実行でキーを1回ずつ処理したい場合は重複対策が必要です。 処理側を冪等（べきとう）にする、つまり同じ入力を繰り返し処理しても結果が壊れない設計にするのが基本です。全キーをSetへ保持して除外する方式はメモリを使うため、大きなデータセットでは慎重に検討します。 2. SCANは一瞬の正確なスナップショットではない 走査中に新たに追加・削除されたキーが、返るかどうかは確定しません。走査の開始から終了まで存在し続けたキーには保証がありますが、「ある時点の全キーを厳密に一覧化する処理」には向きません。 業務上の正確な一覧が必要なら、データ登録時に別の索引を維持する、DBに正本を持つなどの設計が適しています。 3. COUNTは上限ではなくヒント COUNT 100は100件ずつ固定で返す意味ではありません。0件の回や100件を超える回があり得ます。また、MATCHは検索対象をあらかじめ索引で絞り込むものではありません。極端に一致件数が少ないパターンでは、空の応答が続く場合もあります。 大量のデータを処理するなら、応答時間・CPU使用率・処理間隔を測りながらCOUNTを調整します。多数のバッチを一斉に走らせるのも避けます。 削除処理はUNLINKを理解してから UNLINK keyはキーを名前空間から外し、主なメモリ解放をバックグラウンドへ任せるコマンドです。DELより大きな値の削除で応答が安定しやすい場合があります。 ただし、UNLINKも削除操作そのものです。SCAN結果を無条件に削除するのではなく、接続先・対象パターン・件数・実行時間帯を確認し、ロールバック方針を決めてから使用します。 まとめ KEYSはデバッグ時に便利。ただし大量データの本番運用では慎重に扱う。 SCANはカーソルで分割して列挙し、0へ戻るまで継続する。 空配列、重複、走査中の更新、COUNTの非保証を理解する。 頻繁な厳密一覧が必要なら、SCANで毎回探索せず設計を見直す。 公式参考資料（2026年10月14日確認） Redis: KEYS Redis: SCAN Redis: Redis CLI / --scan Redis: UNLINK node-redis: クライアントREADME node-redis: scanIterator実装

Processing 10k Concurrent RFID Payments at the Edge: Go, SQLite WAL, and EPC Gen 2 When engineering fintech and access control infrastructure for large-scale physical events, standard cloud architectures eventually hit a hard physical wall. Under the Saudi Vision 2030 events strategy, mega-summits in Riyadh and Jeddah regularly process upwards of 10,000 delegates. At these scales, the operational realities of a crowded convention center break traditional web systems: 1. Saturated Networks: As thousands of delegates stream through the doors, local cellular macrocells congest and venue Wi-Fi suffers catastrophic packet loss. 2. Concurrency Bursts: 80% of gate traffic and 60% of food and beverage (F&B) purchases happen during narrow 30-to-45-minute windows (e.g., morning ingress and keynote breaks)[cite: 14]. If your event analytics platform or POS system relies on a synchronous HTTP handshake to a remote PostgreSQL database to open a turnstile or process a payment, it will fail under burst load. To guarantee sub-second transaction times and zero gate downtime, we redesigned our RFID attendee tracking and cashless event payments architecture entirely around Edge Persistence and Dual-Frequency hardware. Here is how we built it. --- 1. Dual-Frequency Credentials (UHF + HF) The foundation of our architecture is a single physical smart credential (either a sub-3-second on-demand thermal badge or a durable silicone wristband). We abandoned optical QR codes entirely. Instead, each credential contains a dual-frequency inlay with two distinct memory chips: - Passive UHF EPC Gen 2 (865–868 MHz): Used strictly for zero-contact spatial tracking. Overhead RFID portals read these tags at distances up to 8 meters. Using dynamic Q-algorithm arbitration (Frame-Slotted ALOHA), a single portal can resolve up to 400 tag collisions per second, tracking venue capacity and sponsor dwell times without attendees ever breaking stride. - Near-Field HF (13.56 MHz / NFC): Used for explicit intent actions, such as tapping to process a cashless payment at a vendor terminal or tapping to access a restricted ministerial VIP lounge. --- 2. Offline-First POS Architecture with SQLite WAL If a vendor's Point-of-Sale terminal has to query an external cloud API to verify a wristband's wallet balance, the F&B queue will grind to a halt the moment venue Wi-Fi drops. We pushed the financial ledger down to the edge. Each physical POS terminal and gate controller runs an embedded SQLite database in Write-Ahead Logging (WAL) mode[cite: 14]. -- POS Edge Node Initialization PRAGMA journal_mode = WAL; PRAGMA synchronous = NORMAL; PRAGMA busy_timeout = 5000; PRAGMA temp_store = MEMORY; -- Local Ledger Schema CREATE TABLE IF NOT EXISTS edge_transactions ( txn_id TEXT PRIMARY KEY, wallet_uid TEXT NOT NULL, vendor_id TEXT NOT NULL, amount INTEGER NOT NULL, -- Stored in base currency units (Halalas) epoch_ms INTEGER NOT NULL, sync_state INTEGER DEFAULT 0 ); Sub-Millisecond Tap-to-PayWhen a delegate taps their wristband to buy a coffee:The local NFC reader validates the cryptographically signed offline balance directly from the HF chip's secure memory sector. The transaction is appended to the local SQLite WAL companion file in under 4 milliseconds. The physical UI updates instantly, completing the transaction. A background Go daemon running on the terminal continuously monitors network health. When the internet connection is stable, it asynchronously flushes the committed transactions in batches to our central cloud API. If the internet drops, the terminal keeps selling. 3. Cloud Ingestion with Go & Redis StreamsTo handle the asynchronous telemetry bursts coming from hundreds of offline-resilient edge nodes, we use a Go-based ingestion layer backed by Redis Streams[cite: 14].Gopackage main import ( "context" "encoding/json" "net/http" "github.com/redis/go-redis/v9" ) var ctx = context.Background() var rdb *redis.Client type SyncBatch struct { Transactions []CashlessTxn json:"transactions" } func ingestEdgeBatch(w http.ResponseWriter, r *http.Request) { var batch SyncBatch if err := json.NewDecoder(r.Body).Decode(&batch); err != nil { http.Error(w, err.Error(), 400) return } pipe := rdb.Pipeline() for _, txn := range batch.Transactions { // Push to in-memory Redis Stream buffer pipe.XAdd(ctx, &redis.XAddArgs{ Stream: "stream:cashless_ingestion", Values: map[string]interface{}{ "txn_id": txn.TxnID, "wallet": txn.Wallet, "amount": txn.Amount, "vendor": txn.VendorID, }, }) } if _, err := pipe.Exec(ctx); err != nil { http.Error(w, "Ingestion Failed", 500) return } w.WriteHeader(http.StatusAccepted) } By pushing telemetry directly into an in-memory stream buffer, we completely decouple our primary PostgreSQL database from the extreme write contention caused by hardware synchronization bursts.4. Live Command via WebSocketsA worker pool consumes the Redis Streams, applies idempotent deduplication logic based on the txn_id, and computes rolling spatial and financial aggregations. These metrics are then broadcasted via WebSockets directly to our KSA custom reporting dashboard. Event directors and financial controllers gain instant visibility into live vendor settlements, peak F&B purchasing windows, and civil defense hall capacities, all without aggressively polling the database. ConclusionBuilding physical event technology requires acknowledging that venue networks will fail[cite: 1, 14]. By treating cloud connectivity as an eventual luxury rather than a strict requirement, we guarantee continuous ingress and revenue generation. To explore how these architectures are deployed in the field across GCC exhibitions, review the hardware integration models behind StampIQ's cashless payments for events and their real-time event analytics platform.

Architecting Zero-Latency RFID Attendee Tracking: Edge SQL and Go for High-Density Events When building an event analytics platform for Saudi Vision 2030 mega-events (like LEAP Riyadh or the Global AI Summit), standard web architecture paradigms completely break down[cite: 1, 14]. If you build a typical gate access control system—where an optical scanner reads a QR code, sends an HTTP request to a remote PostgreSQL database, waits for validation, and opens a turnstile—you will fail in production[cite: 1]. Why? Because when 8,000 delegates enter a convention center in Riyadh between 8:15 AM and 9:00 AM, the venue’s commercial fiber connection and local 5G macrocells instantly saturate. Your 150ms HTTP handshake balloons to 8 seconds, and your physical gates freeze. To achieve zero-latency entry and continuous crowd flow, we redesigned our RFID attendee tracking architecture around two principles: Offline-First Edge Persistence and Passive UHF Telemetry[cite: 1, 14]. --- 1. Ditching Barcodes for Passive UHF Telemetry Optical scanners require line-of-sight, which limits gate throughput to roughly 10 delegates per minute per lane. To process enterprise crowds, we moved to passive Ultra-High Frequency (UHF) radio telemetry. Instead of a phone screen, attendees are issued a smart badge encoded with an EPC Gen 2 (ISO 18000-6C) microchip. We place overhead RFID portals across the venue archways. The Physics of Anti-Collision: When a group of 15 attendees walks through a portal simultaneously, their backscattered radio signals collide. To resolve this, our edge readers use dynamic Q-algorithm arbitration (Frame-Slotted ALOHA): 1. The reader broadcasts a slot parameter $Q$. 2. Tags randomly select a slot $C = 2^Q$. 3. If tags collide in the same slot, the reader increments $Q$ to expand the slot pool for the next millisecond round. This allows a single overhead portal to resolve up to 400 distinct tag reads per second—capturing precise zone access and sponsor dwell times without anyone ever stopping to scan. --- 2. Offline-First Edge Resilience (SQLite WAL) When the reader successfully decodes an EPC payload, it passes the binary frame to a local industrial edge controller (e.g., an ARM-based SBC running Linux). This controller must not wait for a cloud HTTP response to grant access. Instead, we run a localized admission engine using SQLite in Write-Ahead Logging (WAL) mode[cite: 14]. // Local Node.js Edge Runtime (Simplified) const Database = require('better-sqlite3'); const db = new Database('/var/data/gate_telemetry.db'); // WAL decouples reads from writes, preventing thread locks db.pragma('journal_mode = WAL'); db.pragma('synchronous = NORMAL'); db.pragma('temp_store = MEMORY'); // Pre-load the expected attendee manifest into memory-mapped pages const validateScan = db.prepare(` SELECT delegate_uuid, access_level FROM local_manifest WHERE epc_hash = ? `); const logEntry = db.prepare(` INSERT INTO telemetry_buffer (epc_hash, portal_id, direction, epoch_ms, synced) VALUES (?, ?, ?, ?, 0) `); function processRFIDFrame(frame) { // 1. Local sub-millisecond lookup const delegate = validateScan.get(frame.epc_hash); if (delegate && delegate.access_level >= REQUIRED_LEVEL) { // 2. Fire physical relay (turnstile/light) instantly hardwareRelay.open(); // 3. Commit to local buffer (avg ~2ms write time) logEntry.run(frame.epc_hash, frame.portal_id, frame.direction, Date.now()); } } Because SQLite WAL allows concurrent readers and writers, the continuous influx of RFID reads never blocks the background daemon, which is responsible for syncing data back to the cloud. 3. High-Throughput Cloud Ingestion (Go & Redis) When the venue's wide-area network is stable, edge nodes continuously flush their local telemetry_buffer to our central cloud API. To handle bursts of telemetry from dozens of portals simultaneously without overwhelming our relational database, our cloud ingestion layer is written in Go and buffers directly into Redis Streams[cite: 14]. Go package main import ( "context" "encoding/binary" "net/http" "github.com/redis/go-redis/v9" ) var ctx = context.Background() var rdb *redis.Client func ingestTelemetryBatch(w http.ResponseWriter, r *http.Request) { // Parse compacted binary batch from edge node batch := parseBinaryPayload(r.Body) pipe := rdb.Pipeline() for _, scan := range batch { // Append to immutable stream buffer O(1) pipe.XAdd(ctx, &redis.XAddArgs{ Stream: "stream:rfid_ingress", Values: map[string]interface{}{ "epc": scan.EPC, "portal": scan.PortalID, "ts": scan.EpochMS, }, }) } if _, err := pipe.Exec(ctx); err != nil { http.Error(w, "Ingestion Failed", 500) return } w.WriteHeader(http.StatusAccepted) } 4. Powering the Live Dashboard A cluster of Go worker nodes consumes the stream:rfid_ingress via Redis Consumer Groups[cite: 14]. These workers apply deduplication logic, aggregate spatial headcounts, and push the live metrics via WebSockets directly to the operations team's KSA custom reporting dashboard[cite: 14]. This architecture provides event directors with real-time visibility into gate influx velocity and civil defense hall capacities, while guaranteeing that physical ingress never halts[cite: 14]. Engineering for Physical Scale If you are building IoT telemetry pipelines or developing access control software, remember: physical hardware cannot wait on the cloud[cite: 14]. By pushing persistence to the edge with SQLite WAL and decoupling cloud ingestion with Redis Streams, you can build event infrastructure that scales effortlessly to tens of thousands of users[cite: 14]. To see these systems in production, check out StampIQ's event technology deployments across the GCC.

Building a High-Throughput Cashless Payment & RFID Ticketing System for Saudi Mega-Events When engineering the infrastructure for an event ticketing and management system in Saudi Arabia, the physical reality of the venue dictates the software architecture. Under the Saudi Vision 2030 events strategy, mega-summits in Riyadh and Jeddah frequently process upwards of 10,000 delegates. These environments are hostile to standard cloud-dependent web applications: 1. Network Saturation: Cellular towers congest and venue Wi-Fi degrades under massive attendee density. 2. Fragmented Stacks: Venues traditionally run three isolated systems—one API for ticketing, a separate local database for gate turnstiles, and independent POS terminals for food/merchandise. 3. High Concurrency Peaks: 80% of gate entries and 60% of F&B purchases happen in tight 30-to-45-minute windows (e.g., morning ingress and keynote breaks). To eliminate venue queues and financial reconciliation errors, we built a unified, edge-resilient architecture that merges secure ticketing, RFID attendee tracking, and cashless event payments into a single cryptographic smart credential. Here is how we architected the system to guarantee sub-second gate scans and offline-resilient financial transactions. --- 1. The Unified Credential: UHF RFID + NFC Instead of relying on fragile optical QR codes that require line-of-sight scanning, the foundation of the system is a physical smart badge or RFID wristband. When a delegate arrives, their digital ticket triggers an on-demand thermal print cycle (sub-3 seconds). Simultaneously, an integrated encoder writes an encrypted payload to a dual-frequency inlay: - UHF EPC Gen 2 (865–868 MHz): Used for passive, zero-contact spatial tracking. Overhead portals read these tags at distances up to 8 meters to monitor civil defense hall capacities and VIP dwell times without requiring the user to stop. - HF NFC (13.56 MHz / ISO 14443): Used for deliberate, secure intent actions—specifically, tapping to pay at a vendor terminal or tapping to open a restricted ministerial lounge door. --- 2. Edge-First Gate & POS Architecture (SQLite WAL) If a vendor's Point-of-Sale (POS) terminal requires a round-trip HTTP request to a cloud server to verify a wristband's wallet balance, the transaction will inevitably fail when venue Wi-Fi drops. To achieve guaranteed sub-second tap-to-pay transactions, we decoupled the physical read layer from the wide-area network. Both our gate controllers and vendor POS devices act as autonomous edge nodes. Local Persistence with Write-Ahead Logging Each edge terminal runs an embedded SQLite database configured for high-concurrency appends: -- POS Edge Node Initialization PRAGMA journal_mode = WAL; PRAGMA synchronous = NORMAL; PRAGMA busy_timeout = 5000; PRAGMA temp_store = MEMORY; CREATE TABLE IF NOT EXISTS local_transactions ( txn_id TEXT PRIMARY KEY, wallet_uid TEXT NOT NULL, vendor_id TEXT NOT NULL, amount INTEGER NOT NULL, -- Stored in base currency units (Halalas) epoch_ms INTEGER NOT NULL, sync_state INTEGER DEFAULT 0 ); When an attendee taps their wristband to purchase a coffee, the NFC reader validates the cryptographically signed wallet balance directly from the chip's secure memory sector. The ledger deduction is written to the local SQLite WAL companion file in under 4 milliseconds. A background Go daemon continuously monitors network health and asynchronously flushes committed transactions to the central cloud API when connectivity permits. 3. High-Throughput Ingestion Pipeline (Go & Redis)When network connectivity is stable, hundreds of edge portals and POS terminals continuously stream telemetry and financial ledgers to the central cloud. To prevent our primary PostgreSQL relational database from locking under heavy concurrent writes, we use Redis Streams as a high-speed ingestion buffer. Our Go ingestion gateway handles incoming TCP payloads and pushes them to Redis in memory:Gopackage main import ( "context" "encoding/json" "log" "net/http" "github.com/redis/go-redis/v9" ) var ctx = context.Background() var rdb *redis.Client type CashlessTxn struct { TxnID string json:"txn_id" Wallet string json:"wallet" Amount int json:"amount" // Halalas VendorID string json:"vendor_id" EpochMS int64 json:"epoch" } func ingestTransaction(w http.ResponseWriter, r *http.Request) { var txn CashlessTxn if err := json.NewDecoder(r.Body).Decode(&txn); err != nil { http.Error(w, err.Error(), http.StatusBadRequest) return } // Push to in-memory Redis Stream buffer err := rdb.XAdd(ctx, &redis.XAddArgs{ Stream: "stream:cashless_transactions", Values: map[string]interface{}{ "txn_id": txn.TxnID, "wallet": txn.Wallet, "amount": txn.Amount, "vendor": txn.VendorID, "epoch": txn.EpochMS, }, }).Err() if err != nil { http.Error(w, "Ingestion Failed", http.StatusInternalServerError) return } w.WriteHeader(http.StatusAccepted) } 4. Live Financial & Spatial Command DashboardsA worker pool consumes the Redis Streams, applying deduplication (based on TxnID) and writing the settled records to TimescaleDB for permanent storage. Simultaneously, the workers compute rolling 60-second aggregations and broadcast them via WebSockets to our KSA custom reporting dashboards. This provides event operations teams with real-time, actionable intelligence:Live Vendor Settlements: Financial controllers can monitor live POS revenue across the entire exhibition floor, tracking exact spending velocity without waiting for end-of-day batch processing. Civil Defense Density Adherence: Security teams receive immediate alerts if passive UHF portals detect that a specific keynote hall is nearing its municipal capacity limit. Architecting for Scale in the GCCBy unifying ticketing, spatial tracking, and digital wallets into a single edge-resilient architecture, we eliminate the physical queues and data silos that plague legacy event operations. If you are developing solutions for physical venues or exploring hardware-integrated fintech, review the deployment models powering StampIQ's cashless payments for events and their enterprise event registration platform