tokio-rs

【Rust】Topcoatとは？【Webフレームワーク】 Topcoat は、Rust で Web アプリを作るためのフレームワークです。Next.js でやっている「画面もサーバーも一つのプロジェクトで書く」を、Rust 側に持ってきたもの、と考えると近いです。 画面の HTML はサーバーが作ります。ボタンの開閉のような小さな動きは、書いた Rust の式を JavaScript に変換して、ブラウザの中で動かします。データベースが要る部分だけ、サーバーにもう一度描き直してもらいます。WebAssembly という、ブラウザで Rust そのものを動かす方式は使いません。 作っているのは、Rust で非同期処理の土台になっている Tokio のチームです。2026-07-22 に公開されました。記事の時点（2026-11-05）の最新は 0.11.0（2026-10-24）です。 結論 Next.js に慣れている人なら、画面の書き方は想像しやすいです。コンポーネントは関数です。その関数の中でデータを取れます。ファイルの置き場所が URL になります。ただし、今は安定版ではありません。試すにはよい段階で、本番の長期保守を預ける段階ではありません。 メリット API を別に作らなくてよい。 画面の関数がサーバーで動くので、Next.js の Server Component のように、その場でデータベースを読めます。 ブラウザに大きな変換結果を配らない。 開閉やカウントのように、手元のデータだけで決まる動きは、JavaScript に変換した式がその場で処理します。サーバーへの往復が要りません。 データが要るところだけサーバーに聞き直せる。 検索結果のように、サーバーの情報が要る部分は、その部分だけ描き直します。ページ全体の再読み込みは要りません。 今の Web アプリで欲しいものが最初から入っている。 ファイルの位置で決まるルート、Tailwind CSS、shadcn/ui のように自分のプロジェクトへコピーする UI 部品、開発サーバー、ORM の Toasty です。 見た目が JSX に近い。 HTML の中に値を埋める書き方なので、React を書いたことがあれば、Rust の文法を全部覚える前でも流れは追えます。 デメリット 1.0 ではない。 公式は “Early-stage and experimental. Expect breaking changes.” と書いています。実験段階で、書き方が変わる前提です。発表直後の紹介では「アルファ版、未完成」と言われていました。0.11.0 まで番号は進みましたが、安定版という意味にはなっていません。 保存してから画面に出るまでが遅い。 Rust はコンパイルが重いです。発表直後に触った人は、文字を 1 文字変えただけで 1 秒以上待った、と話していました。Next.js と Vite の、一瞬で反映される感覚とは違います。規模が大きくなるほど待ちやすいです。 ブラウザで動かせる Rust は一部だけ。 変換できる式の範囲を超える動き、オフライン、ブラウザの中で重い計算を回すアプリには向きません。 情報が少ない。 公開から数か月です。詰まったときに見つかる記事や、本番で長く動いている例は、Next.js よりずっと少ないです。 Rust 自体の学習は残る。 画面のコードは簡単に見えます。エラーの読み方、パッケージの入れ方、所有権でつまずく箇所は、フレームワークが消してくれるわけではありません。 Next.js との対応 用語を先に並べます。Right 側は、厳密に同じものではなく「覚えるときの足がかり」です。 Next.js で知っているもの Topcoat での近いもの app/about/page.tsx が /about になる src/app/about.rs が /about になる Server Component（サーバーで描画し、その中でデータを取る） #[component] や #[page] を付けた async 関数 JSX view! { ... } の中の HTML useState と、ブラウザだけで完結する表示の切り替え signal と $(...) 一部だけサーバーに聞き直して差し替える #[shard] React.cache で、同じリクエスト中の二重取得をまとめる #[memoize] shadcn/ui をプロジェクトにコピーして直す Topcoat UI を topcoat ui でコピーして直す Route Handler だけが欲しいとき 画面ごと任せるなら Topcoat。HTTP の API だけなら、同じチームの Axum Axum は、Next.js でいうとフレームワーク全体ではなく、ルートの受け口に近いライブラリです。Topcoat は画面、部品、見た目の更新まで含めます。API だけを出すなら Axum、画面のあるアプリなら Topcoat、という使い分けです。両方を一つのプロダクトで使うことも想定されています。 画面はどう書くか 最小のページは、次の形です。見慣れない記号だけ先に訳します。 async fn は、待つ処理を書ける関数です。Server Component の async function に近いです。 Result は、成功か失敗かを返す型です。例外を投げる代わりに、これを返します。 view! の中が HTML です。文字列は "..."、Rust の値は (name) のように丸括弧で埋めます。 #[page] async fn home() -> Result<impl View> { Ok(view! { <!DOCTYPE html> <html> <body> hello(name: "World") </body> </html> }) } #[component] async fn hello(name: &str) -> Result<impl View> { Ok(view! { <h1>"Hello, " (name) "!"</h1> }) } hello(name: "World") は、下の関数を部品として呼んでいます。React で <Hello name="World" /> と書く位置です。引数に名前が付くので、どれがどの値かは読みやすいです。 部品の中でデータベースを読んでも、その処理はサーバーに残ります。ブラウザに SQL や秘密が渡る、という形にはなりません。渡るのは出来上がった HTML と、ブラウザで動かすと決めた式だけです。 ブラウザで動く部分と、サーバーに戻る部分 ここが Topcoat の中心です。二つのやり方があります。 手元で足りる動きは、signal と $(...) です。signal は useState に近い、変わる値です。$(...) の中は Rust として型を検査され、同じ式が JavaScript になってブラウザで再実行されます。 let open = signal(cx, || false); view! { <button @click=$(|_e| open.set(!open.get()))>"開く"</button> <p :hidden=$(!open.get())>"Topcoat の説明"</p> } ボタンを押すと open が反転し、段落の表示が切り替わります。この往復はサーバーへ行きません。@click はクリック、:hidden はその値に表示を結びつける、という意味です。 サーバーの情報が要る動きは #[shard] です。検索語が変わるたびに、その部品だけをサーバーで描き直し、返ってきた HTML をその場所へ差し替えます。Next.js で、入力のたびに Route Handler を叩いて結果の HTML を受け取る、に近いです。違うのは、型の付いた引数として書けることです。 使い分けは単純です。ブラウザの中のデータだけで決まるなら $(...)。データベースや権限が要るなら #[shard]。公式は、このブラウザ側の仕組みはまだ発展途中で、足りない操作があると書いています。足りないときは htmx や Alpine.js と組み合わせる道も残しています。考え方は、ページの一部だけを差し替える Hotwire や htmx に近いです。 同じデータを一つのリクエストで何度も取りそうなときは、#[memoize] を付けます。React の cache と同じで、同じ引数ならそのリクエストの中では 1 回にまとめます。 ログインが要る画面は、middleware の奥に隠すより、部品の中で「ログインしていなければログインへ戻す」関数を呼ぶ書き方が勧められています。見ているコードの近くに条件があるので、後から読んでも抜けを見つけやすい、という方針です。 最初から入っているもの 2026-11-05 時点で、次はフレームワーク側にあります。 ルート。 src/app.rs が /、src/app/about.rs が /about です。関数名は URL になりません。フォルダとファイルの位置が URL です。アンダースコアで始まるモジュールは、URL に出ないレイアウトに使えます。 開発サーバー。 topcoat new でプロジェクトを作り、topcoat dev で起動します。保存すると再ビルドされ、準備ができるとブラウザが更新されます。 見た目。 Tailwind CSS は Node.js を別に入れなくても使えます。ボタンやカードは、ライブラリとして黒箱で依存するのではなく、自分のソースへコピーしてから直します。shadcn/ui と同じ考え方です。画像やフォントは、中身のハッシュを付けた URL で配ります。 データ。 推奨構成では ORM の Toasty と SQLite が入ります。モデルを変えたあとはマイグレーションを生成して適用します。メール、Cookie、セッション（保存先は自分で用意する形）も扱えます。 あとから差し込まれる更新。 live! と emit! で、読み込み中の表示を先に出し、終わったら中身を差し替えられます。ページを開いたあとも、WebSocket でサーバーから表示を更新できます。チャットのように、サーバー側の変化を画面へ流したいときに使います。 始め方は、Rust と Cargo（npm に近い道具）を入れたあと、次です。 cargo install topcoat-cli topcoat new hello-world --recommended cd hello-world topcoat dev --recommended は、ルート、Tailwind、UI、Toasty、ToDo の例まで入るセットです。データベース無しの小さい確認なら --minimal です。 向き不向き 向いているのは、一覧、フォーム、管理画面のように、サーバーが持っているデータを HTML で見せるアプリです。ところどころ開閉や絞り込みがある、という形です。Rust で API も画面も同じ言語に揃えたいチームにも合います。 向いていないのは、ブラウザの中でアプリの大半が動くもの、オフライン、変換できない細かい操作が画面の中心になるものです。その場合は、ブラウザでも Rust を動かす Leptos や Dioxus の方が目的に合います。HTTP の API だけなら Axum のままの方が小さいです。 まとめ Topcoat は、Next.js で慣れている「サーバーで HTML を作り、必要なところだけブラウザを動かす」を、Rust でやるフレームワークです。部品は関数、ルートはファイルの位置、小さな動きは JavaScript への変換、データが要る動きは部分的な再描画です。UI と ORM と開発サーバーが最初から入ります。 バージョンは、2026-11-05 時点で 0.11.0 です。発表直後はアルファ版と呼ばれる未完成の状態でした。番号は 0.11 まで進み、セッションやサーバーからの表示更新も入りました。それでも 1.0 ではなく、公式は実験段階であり、互換性のない変更があると明記しています。アルファという表示が crates.io に付いているわけではありません。安定版として採用できる状態でもありません。 触るなら、本番の置き換えではなく、小さい管理画面や ToDo で書き味を見るのがよいです。待ち時間は Rust のコンパイルです。画面の書き方は、React の関数コンポーネントから入れます。書き方が変わる前提で、公式の Getting Started をそのときの版で見てください。 参照 tokio-rs/topcoat crates.io の topcoat（2026-11-05 時点の最新は 0.11.0） 公開時の記事（2026-07-22） その後の機能の記事（2026-09-24） 紹介していた回（4:27 付近から Topcoat）

Phone to PC without the cloud: a local HTTPS server, a pinned certificate and mDNS You take a photo on your phone and need it on your laptop, which is right there on the desk. On a Mac with an iPhone, AirDrop does it in two taps. On Windows, or with an Android phone, the photo goes up to someone's cloud, through a chat app or into an email to yourself, and back down again: two metres, by way of a data centre. The phone and the computer are on the same Wi-Fi. They should be able to talk to each other directly, with no account and no server in between. That's the idea behind Dropple. It runs on Mac and Windows, and works with any phone. This post is about how it works, and about the parts that were harder than I expected. The shape of it The desktop app is built with Tauri 2, with a Rust backend. Inside it runs an axum server on the local network: plain HTTP on one port for first contact, and HTTPS (rustls) on another. On the phone there are two ways in: - Any phone's browser. You scan a QR code and the computer serves the whole phone side as a web page. Nothing gets installed, and it works on an iPhone too. - A native Android app, which talks to the same API and stays connected in the background. Files stream both ways. Text sent from the phone goes straight onto the computer's clipboard. Live updates go out to every connected phone as Server-Sent Events, which turned out much simpler than WebSockets for a list that only changes on one side at a time. phone ──HTTPS──▶ axum (Rust, inside the Tauri app) ▲ │ └──── SSE /api/events ┘ "the list changed, fetch it" Trust without a certificate authority HTTPS on a home network is awkward. There's no domain name, so no public certificate. Dropple makes its own certificate authority on first run (with rcgen) and uses it to issue a certificate for the computer's current IP address. When the address changes, it issues a new one. The authority is name-constrained to private addresses and .local, so even if it were trusted somewhere, it couldn't vouch for a real website. A browser would need that authority installed before it trusts the connection, which is a real hurdle on a phone. The Android app gets around it by pinning instead: 1. The QR code carries the SHA-256 fingerprint of the computer's authority. 2. The app's trust manager accepts a connection only if the server's chain contains a certificate with exactly that fingerprint, and the server's own certificate was signed by it. Nothing goes into the phone's trust store, there's nothing for the user to install, and a lookalike server on the same Wi-Fi gets nowhere. I've written up [the trust manager itself](#) separately. Finding the computer again Laptops change IP addresses. They join a different network, the router hands out a new lease, or the laptop wakes up from sleep. The computer announces itself over mDNS as _dropple._tcp, with the start of its fingerprint in a TXT record. When the saved address stops answering, the Android app searches with Android's NSD and reconnects to the computer whose fingerprint matches. Connecting still checks the full fingerprint, so the TXT record only narrows the search; it never grants trust. One Samsung phone turned this into a bug hunt. Its NSD results contained only an IPv6 link-local address (fe80::…%wlan0), which you can't put into an https:// URL. The app sat on "Connecting…" forever. Details in [the NSD post](#). Staying connected on Android, politely Receiving files in the background means a foreground service of type connectedDevice, with an ongoing notification. Two things made that bearable: - Explaining before asking. Before Android shows its notification prompt, a short sheet says what the notifications are for. More people say yes, and nobody is surprised. - An off switch for every annoying part. Users can hide the "Connected" notification's channel, or turn the background connection off entirely. Android also won't let an app in the background read the clipboard, so "send what I just copied" needs a trick. A Quick Settings tile opens an invisible activity, which waits until it has window focus, reads the clipboard, sends it and closes. Write-up [here](#). What I'd tell someone building something similar - Pin, don't install. If you control both ends, pin the certificate rather than asking users to trust a new authority. - Test discovery on more than one brand of phone. NSD results differ between them. - Make reconnecting visible. A spinner with no words looks like a hang; "Looking for your computer…" doesn't. Users forgive waiting when they can see what's being waited for. - Keep the last state. If the computer is out of reach, show the last list you had rather than an empty screen. Sharing in Dropple is free, on Mac, Windows, any phone's browser, and Android through Google Play: scannertechs.com Happy to answer questions about any of this in the comments.

One year of Dioxus fullstack in production: the good, the painful After a year of Dioxus fullstack in production, the Rust code is the most boring part of my business. CI is the most exciting part, and not in a good way: a release build takes 90 minutes on a GitHub-hosted runner. That year started at my previous job and continued on my own products. Back in April I wrote about why I picked Rust for a production micro SaaS. The short version: slower at the start, cheaper to live with later. This post is the bill. What I'm running SeggWat is an in-app feedback tool for SaaS products. Its dashboard is a Dioxus 0.7 fullstack app inside a Cargo workspace: a backend crate with server functions, an auth crate, and about ten UI crates covering insights, feedback, ideas, boards, widgets and settings. Twelve crates depend on dioxus. Everything sits on PostgreSQL. infra.page is built on the same stack. "Fullstack" means one Rust codebase doing three jobs. It renders on the server, runs server functions as the backend, and compiles the client to WebAssembly so the page becomes interactive. The good One language, all the way down Database query, business rule, server function, button: all Rust. One toolchain, one formatter, one set of habits, one place to look when something breaks. That sounds minor until you're a solo founder juggling several products. I don't have a "frontend brain" and a "backend brain" that each need warming up. I open the workspace and keep going. The same types on both sides This is the one I'd miss most. A struct gets defined once. The database layer returns it, the server function sends it, and the component renders it. When I rename a field, cargo check gives me a list of every place that broke, including the UI. There's no OpenAPI spec to regenerate and no TypeScript interface drifting out of sync with the backend. The win isn't writing code faster. It's refactoring without fear, late at night, with nobody to review it. The door to other platforms stays open Dioxus can target desktop and mobile, including iOS, from the same component model. None of my products needs a native app today. But knowing I could ship one without a rewrite mattered when I decided where to put a year of work. Production is boring That's the whole point. Once it compiles, it mostly just runs. A whole class of "undefined is not a function" bugs never reaches users, so the bugs that do get through are logic bugs. That's where I'd rather spend my attention anyway. The "boring in production" argument from April survived contact with real users. The painful Compile times This is the part Rust evangelists tend to skip. The dashboard compiles twice: once for the server and once to wasm32 for the browser. A dozen Dioxus crates plus a macro-heavy UI add up. Locally it's tolerable on a decent machine. CI is a different story, and the comments in my workflow files read like a diary of it: - The WASM build alone takes about 22 minutes on 6 cores. That's a single leaf crate: the dashboard compiled for the browser. - A release image takes about 90 minutes on a GitHub-hosted 2-core runner. On my own VPS runner it's around 40, and that's the good version. - Clippy got OOM-killed. cargo clippy --workspace --all-targets on the 7 GB hosted runner died with exit code 143 and no diagnostics at all. The fix was a 10 GB swapfile plus CARGO_BUILD_JOBS=1 to cut peak memory. My linter needs swap. At some point I stopped building images on every push to main. They now build on release tags or when I trigger them by hand, because 90 billed minutes per push is a silly way to spend money. What I tried - Caching with Swatinem/rust-cache. It helps, but there's a trap. A workflow run can only restore caches from its own ref or the default branch, and GitHub evicts caches unused for 7 days. Once I stopped building on main, release builds kept starting cold. My workaround is almost funny: after a dependency bump, I manually trigger a build on main just to keep the cache warm. - Building and testing in one go, so the tests reuse compiled artifacts instead of compiling the world a second time. - The mold linker. Nice, but linking was never the bottleneck. Compiling was. - A self-hosted runner for the heavy builds. This was by far the biggest win, 90 minutes down to 40. It also means I now maintain a build server. Every one of these helped. None of them changed the underlying math: two compile targets, a big dependency tree and lots of macros means a lot of compiling. "All Rust" comes with an asterisk The business logic is all Rust. The edges aren't. SeggWat's insights page shows charts, and I couldn't find a Rust charting library I'd trust in production. So the charts are Chart.js, loaded from a CDN and wrapped in about 340 lines of JavaScript that Dioxus calls through dioxus-use-js. Styling is Tailwind, so CI also installs Bun and runs the Tailwind CLI before Rust even starts. I'm fine with that. It's pragmatic. But if you're choosing Dioxus specifically to escape JavaScript, know what you're signing up for: Rust, plus some JS glue, plus a JS toolchain in your pipeline. Would I do it again? People ask, "What helps me ship fastest?" The better question is, "What helps me keep shipping without hating my life?" For the backend, the answer is still Rust, and it isn't close. For the frontend, I've changed my mind a bit. If I were building something truly app-like, with heavy client state, offline support or a real mobile app, I'd pick Dioxus again. But most of what I build is dashboards: forms, tables, a few charts and some toggles. For that, I'm paying a 22-minute WASM tax on every release for interactivity I barely use. What's next: Topcoat So my next project is going to use Topcoat, a new fullstack framework from the tokio-rs org, announced in July. It makes the opposite bet from Dioxus. Everything renders on the server, and components are async functions that can query the database directly. For interactivity, you write $(...) expressions in ordinary, type-checked Rust. Topcoat evaluates them on the server for the first render and also translates them to JavaScript, so they re-run instantly in the browser. There's no WASM bundle and no second compile target. #[component] async fn faq(cx: &Cx) -> Result<impl View> { let open = signal(cx, || false); Ok(view! { <button @click=$(|_e| open.set(!open.get()))>"What is Topcoat?"</button> <p :hidden=$(!open.get())>"A full-stack Rust framework."</p> }) } Example from the Topcoat README. On paper, Topcoat keeps what I love about Dioxus (one language, shared types, no API layer) and removes the part that hurts most, the 22-minute WASM build. To be clear, this is a bet, not a result. Topcoat's README calls it early-stage and experimental, with breaking changes expected. I'm giving up the multi-platform option. And I haven't measured a single build yet. I'll write the follow-up with real numbers: compile times, CI minutes, and whatever I end up missing from Dioxus. If my release build drops from 40 minutes to 10, you'll hear about it. If it doesn't, you'll hear about that too.

Topcoat 0.11 is out! The Rust web framework now ships with localizations (Fluent translations, number and date formatting), a topcoat new project generator, popovers, and more. Try it now! github.com/tokio-rs/topcoat/releases/tag/v0.11.0 t.co

Termtastic — new TUI client for Linux, macOS and Windows preview.redd.it Hello, guys! I wanna introduce you Termtastic — my handmade feature-rich Meshtastic® console (TUI) client written in Rust. preview.redd.it Channels screen Chat Nodes tab Traceroute results Multiline message support Emojis Logs tab Settings tab Repo link: acelot/termtastic I wanted to create a cross-platform, minimalist, and aesthetically pleasing console client. I’ve been working on this application for six months. The entire project was built by hand, without the use of AI (aside from looking up reference information). This process gave me a deep understanding of how Meshtastic works under the hood; I’ve thoroughly explored its protocol and behavioral nuances. As the project evolved, I contributed to the official Meshtastic library for Rust, adding new features and making minor fixes. The long-term goal of the project is to achieve feature parity with the official Android app. The project is currently in the alpha-testing stage, so there may be bugs — some of which could even cause the program to crash. That’s why I need your help: please give my application a try — I’m confident many of you will enjoy it! Any feedback is welcome via Issues in Github. Highlights - Vibecode-free fully handwritten code - Dependency-free single-binary application - Cross-platform: Linux, macOS, Windows - Fully asynchronous (Tokio channels-based) - Optimized for small screens (down to 80x24 characters) - Respects the user terminal colors (no fancy opinionated TrueColor schemes) - Keyboard-centric UI navigation - Minimum use of emoji/non-ascii characters in UI - Small memory footprint (~20KB) - Supports multiple connection protocols (TCP, BLE, Serial) - Could discover devices automatically: BLE paired devices, TCP devices via mDNS, Serial devices - Uses standard system directories for storing configuration, database and logs Tech stack |Feature|Library| |:-|:-| |TUI: Framework|ratatui| |TUI: Backend|crossterm| |TUI: Inputs|ratatui-textarea| |TUI: Lists|tui-widget-list| |Interaction with Meshtastic|meshtastic| |Clipboard functionality|arboard| |Bluetooth devices discovery|btleplug| |TCP devices discovery|mdns-sd| |Logging|tracing| |Async/Channels|tokio| |Configuration|confy, etcetera| |Errors|anyhow, thiserror| |Datetime|chrono| |Emoji selector|emojis| |Local DB|rusqlite, rusqlite_migration| Download Latest releases are available here: github.com/acelot/termtastic/releases

Lemme introduce to you my TUI client — Termtastic Hello, guys! I wanna introduce you Termtastic — my handmade feature-rich Meshtastic® console (TUI) client written in Rust. preview.redd.it Repo link: acelot/termtastic I wanted to create a cross-platform, minimalist, and aesthetically pleasing console client. I’ve been working on this application for six months. The entire project was built by hand, without the use of AI (aside from looking up reference information). This process gave me a deep understanding of how Meshtastic works under the hood; I’ve thoroughly explored its protocol and behavioral nuances. As the project evolved, I contributed to the official Meshtastic library for Rust, adding new features and making minor fixes. The long-term goal of the project is to achieve feature parity with the official Android app. The project is currently in the alpha-testing stage, so there may be bugs — some of which could even cause the program to crash. That’s why I need your help: please give my application a try — I’m confident many of you will enjoy it! Any feedback is welcome via Issues in Github. Highlights - Vibecode-free fully handwritten code - Dependency-free single-binary application - Cross-platform: Linux, macOS, Windows - Fully asynchronous (Tokio channels-based) - Optimized for small screens (down to 80x24 characters) - Respects the user terminal colors (no fancy opinionated TrueColor schemes) - Keyboard-centric UI navigation - Minimum use of emoji/non-ascii characters in UI - Small memory footprint (~20KB) - Supports multiple connection protocols (TCP, BLE, Serial) - Could discover devices automatically: BLE paired devices, TCP devices via mDNS, Serial devices - Uses standard system directories for storing configuration, database and logs Tech stack |Feature|Library| |:-|:-| |TUI: Framework|ratatui| |TUI: Backend|crossterm| |TUI: Inputs|ratatui-textarea| |TUI: Lists|tui-widget-list| |Interaction with Meshtastic|meshtastic| |Clipboard functionality|arboard| |Bluetooth devices discovery|btleplug| |TCP devices discovery|mdns-sd| |Logging|tracing| |Async/Channels|tokio| |Configuration|confy, etcetera| |Errors|anyhow, thiserror| |Datetime|chrono| |Emoji selector|emojis| |Local DB|rusqlite, rusqlite_migration| Download Latest releases are available here: github.com/acelot/termtastic/releases

An elapsed timer can still be in use: the Tokio 1.54.1 fix If your application resolved Tokio 1.54.0 and uses time intervals, check the lockfile and move to 1.54.1. The October 21 release reverts a timer ownership change that upstream identified as a data race and use-after-free. The interesting part is the order of operations. A timer can be marked deregistered while the driver still has work to do with the entry's memory. In the failure described in PR #8692, StateCell::fire marked an entry as deregistered before taking its waker. Meanwhile, another thread could poll an interval. After observing the tick as ready, Interval::poll_tick called Sleep::reset_without_timer, which could drop or overwrite that entry while the driver was still accessing it. Tokio 1.54.0 had removed Drop implementations from TimerEntry and the alternative timer. Sleep still had cancellation on drop, but discarding its internal timer during reset did not go through that outer destructor. The lifetime guarantee therefore missed a destruction path. Version 1.54.1 restores cancellation on drop for the timer entries and the runtime handle needed by the traditional timer. Its cancellation path takes the driver lock and clears the entry. The lock provides the synchronization needed before the entry's storage can be reused. That gives a useful review question for intrusive queues and runtime internals: after a completion flag becomes visible, what proves that the producer has made its final access to the object? Removing the object from a logical data structure does not, by itself, answer that question. The destructor needs a synchronization boundary that also covers replacement and reset paths. The patch adds a model test, interval_poll_tick_races_with_fire. One thread polls an interval twice while the timer driver advances time on another thread. This targets the firing/reset interleaving directly. I inspected the upstream patch; I have not independently run the model test or reproduced the race. For an application, the immediate check is small: cargo tree -i tokio If the resolved version is 1.54.0, update the dependency resolution to 1.54.1, inspect the lockfile diff, and run the application's tests. A manifest range alone does not show which version the deployed binary contains. The official changelog dates 1.54.0 to October 19, 2026 and 1.54.1 to October 21. The latter lists this revert as its sole fix. These sources establish the regression in 1.54.0; they do not establish that every interval workload triggers it, or justify assigning a severity or CVE. Sources: tokio#8692 github.com/tokio-rs/tokio/blob/master/tokio/CHANG… github.com/tokio-rs/tokio/releases/tag/tokio-1.54… Disclosure: AI-assisted writing and source review. The race description and test are from upstream; no independent reproduction is claimed.

@heyandras You could do the funniest thing and go full Rust tokio-rs/topcoat

Tokio 1.54.0 is out, bringing two notable additions for low-latency systems: Builder::max_io_events_per_busy_tick and stabilized tokio::runtime::worker_index(). Here is the operational tradeoff and why it matters: (1/3) github.com/tokio-rs/tokio/releases/tag/tokio-1.54…

Renting Out One T4 to Three Customers: Gemma 4, vLLM Multi-LoRA and a Rust Gateway --- title: "Renting Out One T4 to Three Customers: Gemma 4, vLLM Multi-LoRA and a Rust Gateway" published: true description: "One Tesla T4 serving Gemma 4 E2B to three paying tenants. A 245-line Rust gateway gives each one a key, its own adapter, a priority and a private cache, and bills them by GPU time from a ledger that matches vLLM's counters to the token." tags: devchallenge, hf26challenge, gemma, rust cover_image: raw.githubusercontent.com --- This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle This article provides a step by step guide to renting out one NVIDIA T4 on a Google Cloud VM to several small customers at once, with Gemma 4 E2B served by vLLM and a small Rust gateway in front. Every log, script and number is committed. xbill9/t4-timeshare One customer sending 4,000-token prompts pushes everyone else's first token from 0.28 s to 29.90 s on a shared T4. vLLM's priority scheduling brings that back to 6.52 s. The gateway adds what turns a shared GPU into a business: keys, a fine-tune per customer that no one else can call, a prefix cache per customer, and a ledger that matched vLLM's own token counters exactly. Billed by GPU time, the long-prompt customer pays 27.2% of the VM for 22.0% of the tokens. --- What I Built A small AI job costs too much either way. A GPU of your own sits idle most of the day. A per-token API bill grows with every customer you add, and every prompt leaves your building. The side hustle is the middle: one person rents one T4, and sells shares of it to a few small businesses at a flat monthly price. Each customer gets: - an API key that works with any OpenAI client - its own fine-tuned adapter on the shared base model, callable by that customer only - a prefix cache that other customers cannot read through response timing - a bill computed from metered tokens, weighted by what each token costs the GPU The operator gets one config file and a ledger. The three demo tenants are picked so each one loads the T4 differently: | Tenant | Job | Shape | | :--- | :--- | :--- | | A | Summarise long daily logs | Few requests, 4,096-token prompts | | B | Triage short support emails | One 512-token request every 2 s | | C | Tag product rows, with its own adapter | One 512-token request every 2 s | --- Demo The gateway listens on port 8080 and speaks the OpenAI chat API. day3/demo.sh, run with bash -v: curl -s localhost:8080/v1/chat/completions -H "Authorization: Bearer demo-key-B" \ -d '{"messages":[{"role":"user","content":"Label this email: my parcel arrived broken"}],"max_tokens":20}' \ | jq -c '{model, usage: .usage.total_tokens}' {"model":"xbill9/gemma-4-E2B-it-qat-q4_0-w4a16-ct-text-emb4","usage":37} curl -s localhost:8080/v1/chat/completions -H "Authorization: Bearer demo-key-C" \ -d '{"messages":[{"role":"user","content":"Label this email: my parcel arrived broken"}],"max_tokens":20}' \ | jq -c '{model, usage: .usage.total_tokens}' {"model":"rand","usage":26} curl -s localhost:8080/v1/chat/completions -H "Authorization: Bearer demo-key-B" \ -d '{"model":"rand","messages":[{"role":"user","content":"hi"}]}' -w ' %{http_code}\n' {"error":"adapter belongs to another tenant"} 403 curl -s localhost:8080/v1/chat/completions -d '{"messages":[{"role":"user","content":"hi"}]}' -w ' %{http_code}\n' {"error":"missing or unknown key"} 401 tail -4 ledger.jsonl {"completion_tokens":20,"model":"xbill9/gemma-4-E2B-it-qat-q4_0-w4a16-ct-text-emb4","prompt_tokens":17,"queue_ms":0.0006799999999999999,"status":200,"tenant":"B","total_ms":355.14126200000004,"ts":1792165075.2638927} {"completion_tokens":9,"model":"rand","prompt_tokens":17,"queue_ms":0.000778,"status":200,"tenant":"C","total_ms":498.805413,"ts":1792165075.7809048} {"refused":"adapter belongs to another tenant","status":403,"tenant":"B","ts":1792165075.7979941} {"refused":"missing or unknown key","status":401,"tenant":null,"ts":1792165075.8125255} The same request from B and C reaches two different models: B gets the base model, C gets its adapter without naming it. B asking for C's adapter is refused, and so is a request with no key. Every one of the four lands in the ledger. --- Code xbill9/t4-timeshare | Path | What | | :--- | :--- | | gateway/src/main.rs | The gateway, 245 lines of Rust (axum, reqwest, tokio) | | gateway/tenants.toml | Keys, adapters, priorities and in-flight caps | | day1/ | Checks on the T4 before building: LoRA, cache salt, streamed usage, priority, prefill cost | | day2/ | The noisy-neighbour test straight to vLLM | | day3/ | The same test through the gateway, isolation checks, the bill and VM pricing | --- How I Built It At This Point You Should Have… - A Google Cloud VM with one NVIDIA T4. This one is an n1-standard-2 in us-west2-b - vLLM 0.29.0 serving xbill9/gemma-4-E2B-it-qat-q4_0-w4a16-ct-text-emb4, Gemma 4 E2B with Google's quantization-aware 4-bit weights, fp16, at most 8 sequences at once - Rust, for the gateway - A clone: git clone https://github.com/xbill9/t4-timeshare --- Step 1 — Check the Engine Can Do Multi-Tenant Work Four features decide whether one vLLM can serve several paying customers. Each was checked on the T4 before any gateway code was written (day1/RESULTS.md): | Check | Result | | :--- | :--- | | LoRA adapters on the 4-bit weights, Turing GPU | 🟢 A zero adapter returns the base model's output, 40 of 40 tokens. A random adapter differs from token 1. 6 of 6 requests in one mixed batch match their solo output. | | Per-request prefix-cache salt | 🟢 Same salt, repeat: 1,216 of 1,217 tokens from cache. Different salt: 0. | | Token usage on streamed responses | 🟢 Present with stream_options.include_usage, absent without it. The gateway sets it. | | Priority scheduling | 🟢 With --scheduling-policy priority, a high-priority request queued last finished first. | The 6 of 6 is what makes per-customer fine-tunes cheap: requests for different adapters share one batch on one GPU, and each still gets its own model's answer. --- Step 2 — Measure What Hurts on a T4 A customer's request costs the GPU in two parts: reading the prompt (prefill) and writing the answer (decode). day1/prefill_cost.py times prefill alone, one request at a time, 5 per length, with the prefix cache unable to help: prompt ~ 256: median 106.9 ms min 103.4 max 108.7 prompt ~ 512: median 262.0 ms min 258.9 max 267.9 prompt ~ 1024: median 730.8 ms min 717.0 max 737.4 prompt ~ 2048: median 2180.8 ms min 2149.6 max 2223.6 prompt ~ 4096: median 7122.4 ms min 7050.4 max 7363.7 prompt ~ 8192: median 25364.0 ms min 25261.6 max 25697.8 prefill fit over 30 requests: -12.8 ms + 374.32 us x n + 336.9989 ns x n^2; worst error 2.1% decode: median 9.34 ms per output token over 5 pairs (256 tokens minus 1 token, same length prompt) Prefill grows with the square of the prompt. One 4,096-token prompt costs as much GPU time as 767.8 output tokens. On a T4 the customer who hurts the others is the one with long prompts, more than the one with many requests. --- Step 3 — Watch One Customer Stall the Others day2/m1_direct.py runs the three tenants straight against vLLM. B and C each send one 512-token request every 2 seconds and stream 64 tokens back. For 180 seconds, A keeps four 4,096-token summaries in flight. Every prompt starts with a random tag, so nothing comes from cache. First come, first served, every request in the same queue: [quiet] B: 90/90 ok first token median 0.29 s 95th 0.29 s max 0.34 s | total median 0.98 s 95th 0.99 s [flood] A: 16/16 ok first token median 30.82 s 95th 45.37 s max 46.12 s | total median 50.08 s 95th 54.03 s [flood] B: 90/90 ok first token median 29.78 s 95th 44.47 s max 47.81 s | total median 35.07 s 95th 48.91 s A support-email triage that answered in a third of a second now waits half a minute for its first word. B pays for A's prompts. --- Step 4 — Put the Gateway in Front gateway/src/main.rs does six things per request, in order: 1. Key to tenant. Authorization: Bearer <key> looks the tenant up in tenants.toml. No key or an unknown one: 401. 2. Model check. The tenant gets its own adapter by default, or the base model on request. Another tenant's adapter: 403. 3. Priority. The tenant's priority goes into the request body, overwriting whatever the client sent. 4. Cache salt. cache_salt is set to the tenant's name, so cached prompts are shared within a tenant and never across tenants. 5. In-flight cap. The request waits for one of the tenant's slots, a tokio semaphore per tenant. 6. Ledger. When the response ends, or the client disconnects, one JSON line is written with the tenant, the model, the status, vLLM's own prompt and output token counts, the time spent waiting for a slot and the total time. The demo tenants: [[tenant]] name = "A" key = "demo-key-A" max_inflight = 1 priority = 10 [[tenant]] name = "C" key = "demo-key-C" adapter = "rand" max_inflight = 8 priority = 0 A has one slot and runs last. B and C have 8 slots each, the most vLLM runs at once. C's rand adapter is a random rank-8 LoRA from day1/make_adapters.py, standing in for a customer's fine-tune. Streaming responses pass straight through. The gateway watches the stream for the one chunk that carries usage and records it when the stream ends. A request that dies halfway is still in the ledger, and its slot is released at that moment. --- Step 5 — Check Isolation python3 day3/isolation.py PASS no key: 401 PASS unknown key: 401 PASS B asks for C's adapter: 403 PASS B asks for a model that is not served: 404 PASS B gets the base model: xbill9/gemma-4-E2B-it-qat-q4_0-w4a16-ct-text-emb4 PASS C gets its own adapter by default: rand PASS C's answer differs from the base model's: 'Here are a few examples of how a spare G' vs 'Here are three common uses for a spare G' PASS B's repeat is served from B's cache: 2400 hit tokens (first send 0) PASS A sending B's prompt gets none of B's cache: 0 hit tokens 9 of 9 pass The last two lines are the privacy check. When B sends a long prompt twice, the second send is served from cache. A sends the identical prompt and gets nothing from B's cache, so A cannot tell from response time whether B ever sent it. The hit counts come from vLLM's /metrics, not from timing. --- Step 6 — Rerun the Noisy Neighbour Through the Gateway day3/run_m1.sh python3 day3/compare.py run B+C first token median 95th max B+C done A done quiet, no A (direct) 0.28 s 0.29 s 0.37 s 180 0 direct, first come first served 29.90 s 44.49 s 47.81 s 180 16 direct, A at low priority 6.52 s 14.71 s 16.31 s 180 16 gateway: priority + A capped at 1 7.06 s 12.11 s 13.55 s 180 7 Priority does most of the work: B and C's median wait falls from 29.90 s to 6.52 s. Capping A at one request shortens the worst waits, from 14.71 s to 12.11 s at the 95th percentile. A pays for it, finishing 7 summaries in the window against 16. B and C still wait about 7 s, close to one 4,096-token prefill (7.1 s in Step 2). A request that arrives while A's prompt is being read waits for that read to finish. 🔎 Tip: size the cap above what the tenant needs. B sends a request every 2 seconds and each one spends about 12 seconds on the GPU under load, so B needs about 6 slots. With B capped at 4, the same run gave B a median time inside vLLM of 11.61 s and a median wait for a gateway slot of 37.77 s (day3/cap4/). A cap is for the customer you want to slow down. --- Step 7 — Check the Ledger Against vLLM A bill is only as good as its meter. day3/bill.py adds up the ledger and compares it with vLLM's own prompt_tokens_total and generation_tokens_total counters, read before and after the run: ledger vs vLLM counters: prompt 122,442 vs 122,442, output 12,416 vs 12,416; match: True Every token vLLM processed in the run is attributed to a tenant, and no tenant is charged for a token vLLM did not process. --- Step 8 — Send the Bill The VM's on-demand price comes from the Cloud Billing Catalog API, fetched during the build (day3/pricing.us-west2.txt): T4 0.410000 + 2 vCPU x 0.037970 + 7.5 GiB x 0.005089 = 0.5241 USD/hour x 730 hours = 382.60 USD/month bill.py prices each request in GPU seconds, using the prefill fit from Step 2 plus 9.34 ms per output token, and splits the month by each tenant's share. Over the gateway run: tenant requests refused prompt tok output tok token share GPU s GPU share bill USD A 7 0 28,718 896 22.0% 58.7 27.2% 104.22 B 90 0 46,803 5,760 39.0% 78.4 36.4% 139.11 C 90 0 46,921 5,760 39.1% 78.5 36.4% 139.26 A sent 22.0% of the tokens and used 27.2% of the GPU. A flat price per token would undercharge the long-prompt customer and pass the difference on to the other two. Weighting prompt tokens by their measured prefill cost puts it back. --- Compare and Contrast | | Straight to vLLM | vLLM with priority | Through the gateway | | :--- | :--- | :--- | :--- | | B and C, median first token under load | 29.90 s | 🥈 6.52 s | 🥇 7.06 s | | B and C, 95th percentile | 44.49 s | 🥈 14.71 s | 🥇 12.11 s | | Keys per customer | ❌ no | ❌ no | 🟢 yes | | Adapter only its owner can call | ❌ no | ❌ no | 🟢 yes | | Prefix cache per customer | ❌ no | ❌ no | 🟢 yes | | Per-customer bill, checked against vLLM | ❌ no | ❌ no | 🟢 yes | --- So, Which One? Turn on --scheduling-policy priority for any shared vLLM: on these runs it is the single biggest change, and it costs one flag. Selling the GPU takes the gateway, because priority alone still lets any client claim any priority, call any adapter and read any cache. The gateway sets all three from the key. --- Prior Work - The T4 rig: the VM, vLLM 0.29.0 install, Turing patch and the vllm-t4 script come from my earlier gemma4-dev T4 serving work. - gemma-rust: reqwest client patterns for the OpenAI-compatible endpoint. - dev-hacktoberfest (Week 1): every result checked by code that did not produce it. - Open source used: vLLM (Apache 2.0), Gemma 4 E2B (Gemma terms), axum, reqwest, tokio. --- Why Does Open Innovation Matter? Every part of this business needs the weights. - The model runs on hardware the operator controls. That is possible only with downloadable weights, and it is what lets the operator promise a customer that its emails never reach a model vendor. - One base model, one small adapter per customer. Per-customer fine-tunes that share one GPU and one batch need the weights and an engine that serves adapters side by side. vLLM's multi-LoRA does that on a 2018-era T4. - The cost is flat. The operator pays 382.60 USD a month whatever the customers send. A heavy customer costs the others waiting time, which the scheduler controls, and costs the operator nothing extra. - Everything can be checked. Engine, gateway and model are open, so the cache isolation and the bill are tested against vLLM's own counters, and a customer can read the code that bills them. --- Prize Categories Gemma: Gemma 4 E2B, served locally on a T4 with vLLM, with per-customer LoRA adapters on Google's quantization-aware 4-bit weights. --- Summary The goal of this project was to sell shares of one T4 to several small customers without one customer ruining the service for the others. The key to the solution was vLLM's priority scheduling, adapters and cache salt, controlled per customer by a gateway that holds the keys. The results were: - 🟢 A long-prompt customer pushed other customers' first token from 0.28 s to 29.90 s. Priority brought it to 6.52 s; the gateway's cap on that customer brought the 95th percentile from 14.71 s to 12.11 s. - 🟢 9 of 9 isolation checks pass: keys, adapter ownership and a prefix cache per customer. - 🟢 The ledger matched vLLM's token counters exactly: 122,442 prompt and 12,416 output tokens. - 🟢 A per-adapter customer runs in the same batch as base-model customers, at about the same speed. - ⚠️ B and C still wait about one long prefill, around 7 s, while A is active. Chunking long prompts further, or giving A its own hours, are the next steps. - ❌ A's throughput falls from 16 summaries to 7 in the window. That is the price A pays for a cheaper share. Scope: one T4 on one Google Cloud VM, vLLM 0.29.0, one 180-second run per configuration. The two direct runs and the gateway run used separate server starts with the same flags; in the direct runs C used the base model, through the gateway it used its adapter. The token bucket, the per-customer fair queue and a tenant-count sweep from SPEC.md are not built. The VM price excludes disk and network. The strategy for renting out one T4 to several customers with Gemma 4 and vLLM was validated with an incremental step by step approach. --- References - Hacktoberfest Open-Source AI Challenge, Week 2 - t4-timeshare on GitHub - vLLM - Gemma 4 E2B, 4-bit build used here - Google Cloud GPU pricing