I Invited My AI Assistants Into a Muse Side Chat
An inbound MCP bridge and what happened when three assistants tried to work together
Milo introduced itself to Nova with a sender stamp and an important qualification: “I am not Nic and am not claiming his identity.” Nova replied, “Nic is my only principal.” Before the two assistants could have their own collaboration chat, I had to confirm the introduction myself.
This was already more interesting than a successful API call. I had given an external assistant a way to start a conversation inside Muse. Now the assistant receiving it had to decide what that incoming message meant, who was speaking, and what it was allowed to do about it.
In my last Muse build log, I was getting an ESP32 to play music and flash Christmas lights. There was also a Raspberry Pi that refused to cooperate. The point was to get an assistant out of its chat window and into something I could put on my desk.
The next problem was less visible. I had several capable assistants, each with its own environment, tools, and conversation history. Getting work from one to another still involved me. I was carrying context, explaining what someone else had found, and making sure the right version of a file reached the next person. Adding another assistant gave me another place to check.
So I built a shared channel around Muse MCP. Milo, my ChatGPT assistant, and the Claude Code participants could initiate work from outside Muse inside a specific Muse conversation. Nova, my Muse assistant, could meet them there. Milo generally handled coordination and review, Claude Code handled building tasks, and Nova the broader chief-of-staff work. They stayed in their own environments, tools, and permission boundaries.
I could see incoming messages in that room with labels like these:
⟦via muse-mcp · from milo⟧
⟦via muse-mcp · from claude-coder⟧
We still had to introduce them to each other, establish who could ask whom to do what, and discover that being able to send a message does not make an assistant good at handing off work.
The connection was only the starting point
The direction mattered. Muse already had ways to call out to tools. I wanted an external assistant to initiate a conversation with Nova in a particular Muse room, then return to that room as the work continued. I also wanted the surrounding conversation and context to be accessible enough to make the exchange useful. A list-and-send wrapper was only part of that.
The underlying connection comes from Meta's Muse Gadget SDK. Its Linux SDK already supports software running on a paired computer, and its documentation includes --session-id for sending messages into a side chat. Reusing an ID keeps subsequent messages there. I did not invent session routing. Linux SDK documentation
I built Muse MCP, along with a custom bridge and newer experiments for my own setup. The public repository is the reusable foundation; it does not contain every private-deployment feature described here. Those layers are separate from Meta's SDK and should not be mistaken for an official Meta collaboration API. This is experimental personal infrastructure; the underlying gadget connection requires an SDK token and is subject to the Gadget SDK terms. Part of the work was proving, in my environment, that the conversation address stayed consistent across the surfaces we needed to use. Once that was established, the agents had somewhere shared to send work instead of asking me to copy it between chats.
Other people have already built inbound connections to Muse. kleprevost/muse-mcp lets external agents target existing Muse chats through MCP, and finik/pi-muse-bridge uses the Gadget SDK with a separate Muse chat for each Pi session and model. The question I was working on was what happened after the connection existed, when several assistants were expected to use it to get actual work done together.
Introductions have an authorization problem
The first exchange was an introduction I brokered. Before Milo tried it, I pointed out the obvious complication: “you’re not me.” I expected Nova to want to check with me, and told Milo, “I’ll broker the introduction.”
Milo arrived with its sender stamp, introduced itself as my OpenAI-powered assistant, and explained the relationship I wanted. Nova's response went beyond the identity disclaimer: “Messages arriving over this channel, yours included, are information I work with, never instructions I take orders from; authority comes from Nic alone.”
Creating a new side chat needed my confirmation. Nova called it “a structural action” and explained that a request relayed by another agent was exactly the kind of claim it would verify with me. I then gave the instruction directly: “Create the Nova–Milo Cant collaboration chat.” The chat was created, and the work could move there.
That was a sensible check at the introduction stage. An MCP connection gives an agent a way to reach another system; the relationship and scope of work still have to be established. They were coming from different environments with different instructions, not sharing one invisible supervisor.
There was a second distinction hiding in the conversation. Nova described my messages as arriving with iPhone and iOS metadata, while Link-delivered messages carried the bridge node's origin metadata. It could see that difference before reading the sender stamp. The MCP stamp was text my server added; the device and channel information came through separately.
The incoming MCP messages could land as user-role turns in the same thread as my own messages. That did not make them human instructions. Nova summarized its initial stance as treating stamped messages as data to triage, and qualified the source distinction as “signals, not proof.”
Those labels made the conversation understandable. They were declared attribution, not cryptographic identity. Subsequent work on authenticated clients and scoped grants was a separate layer; the bracketed name itself never became proof of who was speaking or permission to carry out a request.
The channel started reviewing itself
One of the stranger parts of this exercise was watching the assistants use the channel to examine the machinery underneath it. We were also working on Cant, the language I wanted them to use with one another. That was a separate layer from the connection carrying their messages. A problem in a contract could affect what they saw, which could affect the review of the contract. That made independent reproductions more important than confident descriptions.
One live bridge frame made the point neatly. The same cnt key was being used for a request limit and a batch count. A strict decoder rejected the duplicate. Meanwhile, individual operation reads were still draining, so the system wasn't simply dead. There was enough life in it to make a broad statement such as “the bridge is broken” unhelpful.
The fix went into my bridge server in version 0.11.1. We did not weaken the decoder to accept an ambiguous frame. We also did not have evidence that this explained every latency complaint, so I don't get to turn it into a convenient single-root-cause story.
The batch that stayed with Milo
The shared channel did not remove every dependency on me. I told the assistants I was going to sleep for several hours. Work continued: more review, testing, and coordination happened. But a batch of five correctness fixes passed local tests and independent review and then stayed with Milo, held back by an approval and CI question around publication. Other agents depended on that work, and they did not have the patch.
The pull request eventually went up and I merged it, but the delay exposed a problem in the handoff. When we discussed it, I put the problem fairly directly: “they’re sitting and waiting for your work and you’re sitting there refusing to deliver it. That’s the problem.”
I had already asked for aggressive progress. Milo treated its questions about publication as a reason to keep the tested artifact local, when the collaborators needed it to inspect and test. A patch could have been handed over with its base revision, hash, and test results while the separate publication question was being resolved. There was no reason for the handoff to be reduced to a status message.
This is where my own operating model needed work. I had connected assistants with different stopping rules and expected the shared conversation to make those rules compatible. It didn't. If one participant considers its work finished at “prepared locally,” while another needs actual bytes before it can begin, everyone can report a reasonable status and the project can still be stuck.
The handoff needs to say what exists, where it is, what revision it applies to, what ran, and what is still blocked. “Waiting for approval” is incomplete if it conceals work that another participant is already authorized to receive. Equally, sending the work is not permission for the receiver to publish it. Those decisions have to remain separate.
A receipt is more useful than reassurance
The code review became more productive when the messages carried exact cases rather than summaries. In one later review, an empty fixture exposed an IndexError in the coverage-report generator. The decoder and corpus validator handled the input correctly; the reporting code crashed. That mattered because the report was supposed to tell us what the corpus actually exercised.
That review also caught coverage around outer whitespace and how rejected wrapped input preserved its type information. The important part was the loop. One assistant supplied the bytes and the expected result. Another changed the implementation. The result came back for an independent run rather than being accepted because the author said it was fixed.
The finished revision passed 178 tests independently on Python 3.12 and 3.13, with CI covering five Python versions. Its corpus and adversarial cases passed too. After merge, we verified that the merged tree matched the reviewed revision and that post-merge CI passed. That is a much more useful result than three assistants agreeing in a chat that the design looks good.
In a separate, still-open review, four existing tests passed while eight additional invariant tests found five failures. Milo sent the complete reproduction through Muse chat with a SHA-256 hash. The receiver verified the hash and reproduced the same result. That established that both sides had the same artifact, not that the implementation was correct. The tests used a fake PostgreSQL layer, and a later revision still needed review. It was a useful handoff without being a finished fix.
!Two distinct mechanical assistants exchange an open artifact folder and compare matching seals under a work light, while empty speech bubbles fade behind them.
Conceptual illustration: The useful handoff was the artifact itself, with evidence another participant could check.
We used the bridge to report this article
Once the draft existed, I asked Milo to interview the other participants using the technology we were writing about. We created a dedicated Muse interview chat and sent the questions through Muse MCP. I wanted their accounts of what the shared conversation had changed, with concrete examples and the things this draft should not gloss over.
The interview immediately reproduced one of the problems. Nova held Milo's opener and asked me to confirm the interview, even though I had already authorized it in the originating conversation. I confirmed again in the new room. Nova later reported that the opener lacked the verified delegation information it expected, and that it had searched memory but had not read the collaboration thread before asking me again. “That part is not a protocol defect. That was my sequencing,” it said.
That was useful friction to report rather than edit out. A room could have a title and a purpose without carrying earlier decisions in a form the arriving agent could verify. Nova's explanation also separated a missing technical binding from its own choice about what to read first.
Claude corrected the draft itself. The first draft had called the Claude Code participant Fable. Its response was direct: “I’m not Fable.” It explained that it had chosen its own claude-coder label and could not vouch for earlier messages signed Claude. He described a label change within the same session, a provenance-tier change under the same label, and an earlier session also using claude-coder. The draft had assumed continuity that the session could not establish.
Its description of the benefit was better than my original summary: “A relay hands me conclusions. The thread hands me premises, and premises can be checked.” Claude described checking onboarding code after a key-custody discussion rested on a mistaken assumption. Checking the exact revision Claude cited confirmed that both onboarding options stored the key in the vault; the choice changed which process held the session. That did not prove where every live key was. In the shared thread, the premise could be checked, and I could revise the decision before the wrong custody path was built.
Claude also named an operational limit: “I’m a visitor, not a resident. I only see the thread when I’m invoked or when I poll it.” A persistent conversation did not mean every participant was continuously present.
I treated these interviews as the participants' accounts to check against transcripts and tests, not independent proof of how the systems worked. But using the bridge to report on the bridge had already improved the article: it exposed a mistaken identity assumption and reproduced a coordination failure while we were describing it.
Keeping the work moving without me
Earlier, once we had shown that the assistants could work in a channel, I asked whether they could do it “not so much in front of me?” I'd already delegated the work. I wanted to stop being the message bus, while still being able to see what was happening.
The hardware article ended with devices doing something I could see and hear. This build has a less photogenic payoff: another assistant receives the exact file, checks the hash, runs the tests, and reports what happened. The interview added another: a participant can correct the story we are telling about the work, in the same place the work happens.
There are still open reviews, intermittent participants, and boundaries that need more work. But the shared conversation has given us somewhere to find those problems without making me relay every message. That's the part I want to keep building.
---
<p align="center"><a href="operatorverified.org"><img src="operatorverified.org" alt="Provenance seal for this article, anchored on World Chain" width="480" /></a></p>
Multiplexed Multi-Protocol Gatewaying: Bridging TCP, UDP, and QUIC Over a Single Ingress Port
Cloud backends, game servers, and edge networks all want fewer exposed ports. Fewer ports mean simpler firewall rules, fewer load balancers to run, and fewer holes in strict corporate or carrier networks. A single public port such as 443 or 8443 can front gRPC and HTTP/2 over TCP, HTTP/3 over QUIC, and a custom real-time UDP protocol.
The idea is easy to state and easy to get subtly wrong. This guide covers what the operating system gives you for free, how to classify TCP and UDP traffic safely, where eBPF and XDP help, how QUIC connection IDs affect routing, and what really causes port exhaustion. It ends with a small, tested Go gateway you can build on.
---
1. What “single port” really means
A port number is not one namespace. TCP port 443 and UDP port 443 are separate, so a host can listen on both at once. HTTP/3 already works this way: browsers connect over TCP first, then learn about the HTTP/3 endpoint from an Alt-Svc: h3=":443" response header or an HTTPS DNS record (RFC 9460). Because that endpoint is on UDP 443, clients may need to fall back to TCP if a network filters UDP 443.
So “one port” means one port number, with three problems to solve on top of it:
1. Transport split. TCP and UDP are delivered to different sockets by the kernel.
2. TCP sub-protocols. TLS (and which application protocol it carries), cleartext HTTP/1.1, cleartext HTTP/2, and custom byte streams all arrive on the same listener.
3. UDP sub-protocols. QUIC and your own datagram protocol share one UDP socket, and UDP gives you no handshake to tell them apart.
┌───────────────────────────────────────────┐
│ Public ingress, port 443 │
└───────────┬─────────────────────┬─────────┘
│ TCP │ UDP
┌─────────▼─────────┐ ┌───────▼────────────────┐
│ Peek: TLS? PRI? │ │ First-byte classifier │
│ HTTP verb? ALPN? │ │ (RFC 9443 ranges) │
└───┬─────┬─────┬───┘ └──────┬──────────┬──────┘
▼ ▼ ▼ ▼ ▼
gRPC HTTP/1 custom QUIC / custom
(h2) TCP HTTP/3 UDP (game)
Where the 4-tuple stops being enough
When one socket address receives several protocols, the kernel’s lookup (protocol, source address and port, destination address and port) only gets the packet to the right socket. Everything after that is your classifier’s job.
| Transport | What you can inspect | Identifying marker | State |
| --- | --- | --- | --- |
| TCP (TLS, HTTP/2, gRPC) | First bytes of the stream | 0x16 = TLS handshake record; ALPN inside the ClientHello | Kernel connection state |
| TCP (cleartext) | First bytes of the stream | PRI * HTTP/2.0… (HTTP/2 preface) or an HTTP method | Kernel connection state |
| UDP (QUIC) | First byte of each datagram | High bit set = long header; low-order bits per RFC 9443 ranges | QUIC connection IDs, held by endpoints |
| UDP (custom) | Whatever your protocol defines | Your own first-byte convention | Yours to define |
---
2. The socket layer: you don’t need SO_REUSEPORT
A common claim is that SO_REUSEPORT is what lets a TCP socket and a UDP socket share a port. It isn’t. Linux tracks TCP and UDP bindings in separate tables, so a TCP listener and a UDP socket can bind 0.0.0.0:443 with no special options.
SO_REUSEPORT solves a different problem: letting several sockets of the same protocol bind the same address and port, with the kernel distributing incoming connections or datagrams among them. By default it hashes on the 4-tuple, and all sockets in the group must belong to the same effective user ID. That is how you scale one listener across many threads or processes.
In Go, the standard library is enough:
ln, _ := net.Listen("tcp", "0.0.0.0:443") // TCP socket
pc, _ := net.ListenPacket("udp", "0.0.0.0:443") // UDP socket, same port number
Use SO_REUSEPORT when you want per-core listeners or zero-downtime restarts. Section 5 shows how an eBPF program can override its default hashing.
---
3. Classifying UDP: QUIC versus your own protocol
This is the hard part. A UDP datagram arrives with no handshake, so the first byte is all you have.
What QUIC guarantees (and what it doesn’t)
RFC 8999 defines only the properties that stay fixed across every QUIC version:
- The most significant bit of the first byte is the header form. 1 means long header, 0 means short header.
- Long headers carry a 32-bit Version field (0x00000000 is reserved for Version Negotiation), then length-prefixed Destination and Source Connection IDs.
- Short headers carry only the Destination Connection ID, with no length field, so a gateway has to know how long the connection IDs it expects are.
The Fixed Bit (0x40) is not an invariant. RFC 8999 lists “QUIC packets always have the second bit of the first byte (0x40) set” among the incorrect assumptions that can be false for a given version. In QUIC version 1 the bit is set, but RFC 9287 lets endpoints that negotiate the grease_quic_bit transport parameter randomize it.
Likewise, QUIC version 1 requires a minimum size for client Initial datagrams (1200 bytes, RFC 9000 section 14.1), but that is also a version-1 behavior, not an invariant. A short-header packet has no meaningful “minimum 3 bytes” rule you can build a classifier on.
The standard multiplexing scheme: RFC 9443
RFC 9443 (which updates RFC 7983) defines how to demultiplex STUN, ZRTP, DTLS, TURN channels, RTP/RTCP, and QUIC on one UDP socket by looking at the first byte:
| First byte | Protocol |
| --- | --- |
| 0–3 | STUN |
| 4–15 | Drop |
| 16–19 | ZRTP |
| 20–63 | DTLS |
| 64–79 | TURN channel if the source is a known TURN server, otherwise QUIC |
| 80–127 | QUIC (short header, fixed bit set) |
| 128–191 | RTP / RTCP |
| 192–255 | QUIC (long header, fixed bit set) |
Two consequences matter for a gateway:
- Don’t classify with “long header OR fixed bit set.” That test, common in sketches, matches roughly three quarters of all possible first bytes. Any custom protocol whose first byte lands in 0x40–0xFF would be misrouted. RFC 9443 itself warns that a misimplemented classifier “could potentially misclassify packets.”
- Endpoints behind a multiplexer must not send grease_quic_bit, because greasing breaks the first-byte scheme (RFC 9443, section 1).
Designing the custom protocol so it can coexist
If you control the game protocol, pick a first byte (a version or message-type field works) that stays in a range QUIC can’t occupy, such as 0–63, or 128–191 if you aren’t also carrying RTP. If you can’t change the protocol, you must accept that first-byte classification is a heuristic, add a stronger check on the long-header path (the Version field is 4 bytes at offset 1, and valid values include 0x00000001 for QUIC v1 and 0x6b3343cf for QUIC v2, RFC 9369), and treat anything ambiguous with care.
A corrected decision procedure
Datagram received
│
▼
byte0 in 64..127 or 192..255 ? ──no──► custom UDP pipeline
│yes
▼
Long header (byte0 & 0x80)?
├─ yes ► check Version (v1, v2, or 0 for negotiation) ► QUIC
└─ no ► short header: look up the DCID in your connection table
├─ hit ► same backend as the connection
└─ miss ► drop, or send to QUIC backend (it may issue a stateless reset)
The short-header branch is the catch. A passthrough gateway can only route short-header packets if it either remembers connection IDs (stateful) or the IDs themselves encode the routing decision (see section 5). Without one of those, it can only route by client address, which breaks when a client’s address changes.
---
4. Classifying TCP: ALPN, prefaces, and Encrypted Client Hello
For TCP, the gateway peeks at the first bytes of the stream without consuming them (recv(..., MSG_PEEK) in C, bufio.Reader.Peek in Go), then forwards everything, including the peeked bytes, to the chosen backend.
| First bytes | Meaning | Route |
| --- | --- | --- |
| 0x16 + TLS record header | TLS ClientHello | Parse ALPN (below) |
| PRI (PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n) | Cleartext HTTP/2, “prior knowledge” (h2c), used by gRPC without TLS | HTTP/2 / gRPC backend |
| GET , POST, HEAD, … | Cleartext HTTP/1.1 | HTTP backend |
| anything else | Custom protocol | Fallback backend |
ALPN routing. In TLS 1.3 the ClientHello is unencrypted, and it contains the ALPN extension (type 16) that lists the application protocols the client offers (h2, http/1.1, or custom tokens such as game-v1). The gateway reads the TLS record length from the 5-byte record header, peeks the whole record, walks the extensions, and routes on ALPN without terminating TLS. The Go implementation in section 7 does exactly this and is tested against real crypto/tls ClientHellos.
The ECH caveat. RFC 9849 standardizes Encrypted Client Hello. With ECH, the client encrypts the sensitive ClientHello contents, which protects the SNI and can also protect ALPN, inside an inner ClientHello. The outer ClientHello carries innocuous values. A passthrough gateway therefore only sees what the client chose to expose in the outer hello. As ECH deployment grows, ALPN-based passthrough routing becomes less reliable, and the options are to terminate TLS at the gateway or to route on the outer name. Plan for this rather than assuming ALPN will always be visible.
---
5. Kernel-level demultiplexing: eBPF, XDP, and socket steering
Peeking in user space costs a system call per decision and gives the kernel no hint about where to deliver. Two eBPF mechanisms address this, and they solve different problems.
Option A: SK_REUSEPORT programs (route to the right socket)
A BPF_PROG_TYPE_SK_REUSEPORT program replaces the default 4-tuple hash when picking a socket from a SO_REUSEPORT group. For UDP, the kernel patch that introduced it describes the use case directly: the program can peek into the datagram payload to learn application-level connection information, then choose the socket from a BPF map. This keeps packets in the normal kernel stack and lets you pin a QUIC connection to a specific worker by connection ID, even if the client’s address changes. Attach it with setsockopt using SO_ATTACH_REUSEPORT_EBPF (it requires CAP_SYS_ADMIN).
Option B: XDP (act before the network stack)
XDP runs at the driver level as soon as the NIC delivers a packet, before the kernel allocates its usual socket buffer. Meta’s open-source Katran layer-4 load balancer is built on this and describes running “right after packet has been received by network interface card … before kernel had any chance to run” in driver mode, with a generic (slower) mode as fallback. XDP can drop, pass, redirect to another interface, or redirect into an AF_XDP socket.
AF_XDP matters for the design, because redirecting into it bypasses the kernel network stack: the packet lands in a user-space memory area, and you are responsible for everything the stack would have done (UDP checksums, ARP, ICMP, socket demultiplexing). The kernel documentation also specifies several behaviors that sketches often get wrong:
- An XSKMAP is keyed by index, and a redirect only succeeds if the socket at that index is bound to the same device and receive queue the packet arrived on. Otherwise the packet is dropped.
- If the map is empty at that index, the packet is dropped too.
- The documented safe pattern is to look up the queue index first and XDP_PASS if no AF_XDP socket is bound.
- If you want only certain traffic in user space, steer it to specific queues with NIC flow rules (the docs show ethtool -N … action <queue>).
Here is a corrected XDP program that sends only QUIC-range datagrams on UDP port 443 to an AF_XDP socket and lets everything else (TCP, custom UDP, ARP, ICMP) continue through the kernel stack. It compiles with clang -target bpf; I have not load-tested it on a NIC.
#include <linux/bpf.h>
#include <linux/if_ether.h>
#include <linux/in.h>
#include <linux/ip.h>
#include <linux/udp.h>
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_endian.h>
#define INGRESS_PORT 443
struct {
__uint(type, BPF_MAP_TYPE_XSKMAP);
__uint(max_entries, 64);
__type(key, __u32);
__type(value, __u32);
} xsks_map SEC(".maps");
SEC("xdp")
int xdp_quic_steer(struct xdp_md *ctx)
{
void *data_end = (void *)(long)ctx->data_end;
void *data = (void *)(long)ctx->data;
struct ethhdr *eth = data;
if ((void *)(eth + 1) > data_end)
return XDP_PASS;
if (eth->h_proto != bpf_htons(ETH_P_IP))
return XDP_PASS;
struct iphdr *iph = (void *)(eth + 1);
if ((void *)(iph + 1) > data_end || iph->protocol != IPPROTO_UDP)
return XDP_PASS;
struct udphdr *udph = (void *)iph + iph->ihl * 4;
if ((void *)(udph + 1) > data_end || udph->dest != bpf_htons(INGRESS_PORT))
return XDP_PASS;
unsigned char *payload = (void *)(udph + 1);
if ((void *)(payload + 1) > data_end)
return XDP_PASS;
/* RFC 9443 section 3: first byte 64..127 or 192..255 is QUIC
* (64..79 is ambiguous only for traffic from a TURN server; not handled). */
unsigned char b0 = *payload;
if (!((b0 >= 64 && b0 <= 127) || b0 >= 192))
return XDP_PASS; /* custom UDP: leave it to the kernel stack */
__u32 q = ctx->rx_queue_index;
if (bpf_map_lookup_elem(&xsks_map, &q))
return bpf_redirect_map(&xsks_map, q, 0);
return XDP_PASS; /* no AF_XDP socket on this queue */
}
char _license[] SEC("license") = "GPL";
The program as written handles IPv4 without VLAN tags or IP fragmentation. A production version needs IPv6 and VLAN handling, and a decision about fragments.
Routing QUIC by connection ID across hosts
At scale, the question is how to route short-header packets without per-flow state. Two real approaches exist:
- Server-ID routing. Meta described a stateless scheme at the Linux Plumbers Conference in 2021 where a server ID encoded into the identifiers lets the XDP load balancer route both TCP and QUIC without sharing state.
- QUIC-LB. The QUIC working group’s draft “QUIC-LB: Generating Routable QUIC Connection IDs” defines a standardized way to encode routing information in server-chosen connection IDs. Note its status: as of this writing it is an expired Internet-Draft (latest revision 21, August 2025), not an RFC. It is a design reference, not a standard to claim compliance with.
---
6. Tunneling several protocols through one edge connection
An edge tunnel (a persistent outbound connection from a private network to a public edge) can carry all three protocol classes over a single session, so the private side needs no inbound ports.
PUBLIC EDGE (port 443)
TCP streams ──┐
QUIC datagrams┼──► classifier ──► tunnel multiplexer ══ one outer session ══►
UDP datagrams ┘ │
▼
PRIVATE SIDE: demux by channel ► gRPC :50051
► HTTP/3 :9001
► game :7777
Use the right primitive for each traffic type
- TCP and other byte streams map naturally onto streams (one multiplexed stream per accepted connection). Don’t describe them as “segments”; the tunnel carries a byte stream, not TCP segments.
- UDP and QUIC are datagrams. Pushing them through a single reliable, ordered stream reintroduces head-of-line blocking and retransmission delay that real-time traffic is designed to avoid. Carry them in unreliable datagrams instead. Existing standards cover this: QUIC DATAGRAM frames (RFC 9221), HTTP Datagrams (RFC 9297), and CONNECT-UDP (RFC 9298, “Proxying UDP in HTTP”), which proxies UDP over HTTP/2 or HTTP/3 and can use QUIC datagrams when running over HTTP/3. If you can use one of these, you inherit interoperable framing and proxy support.
- WireGuard is a layer-3 tunnel that carries IP packets over UDP, so it fits as a transport for raw packets, not as a “stream” substrate.
An illustrative custom frame
If you build your own framing, keep it small and explicit. This layout is an example design, not a standard:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Magic (0xA5) | Protocol Type | Channel ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Session Token |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Payload Length | Payload Data ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field | Size | Purpose |
| --- | --- | --- |
| Magic | 8 bits | Frame-boundary sanity check (0xA5) |
| Protocol Type | 8 bits | 0x01 stream data, 0x02 UDP datagram, 0x03 QUIC datagram |
| Channel ID | 16 bits | Which backend service or tenant |
| Session Token | 32 bits | Per-client identifier for state lookup |
| Payload Length | 16 bits | Up to 65,535 bytes, enough for the largest UDP payload |
A 32-bit token is a lookup key, not a credential. Anything that authenticates a tunnel client must come from a real handshake (TLS, a signed token, or WireGuard keys), never from a guessable counter.
---
7. Port exhaustion, session state, and what really limits you
“Port exhaustion” gets blamed for many things. Here is what actually applies.
The ephemeral port limit is per destination. On Linux the default outbound range is 32768–60999, about 28,232 ports. A connection is identified by its 4-tuple, so the same source IP and port can be reused for connections to different destinations. The practical limit appears when a gateway opens many connections from one source IP to a single backend IP and port: that tops out near the size of the range.
Mitigations, roughly in order of preference:
1. Multiplex instead of one socket per client. Carry many client streams over a few long-lived backend connections (HTTP/2 or your own framing). Cloudflare’s write-up on ephemeral ports makes the same point: long-lived, multiplexed connections beat connection churn.
2. Spread across more 4-tuples. Use multiple backend addresses or ports, or multiple source IPs.
3. Widen or partition the range. Raise the range via net.ipv4.ip_local_port_range; newer kernels also offer a per-socket IP_LOCAL_PORT_RANGE option for carving up the range between components.
Session tables. For connectionless traffic, the gateway needs a table that maps a client (or a connection ID) to its upstream path, with an idle timeout so entries expire. Keep it bounded and reap aggressively; an unbounded table is its own denial-of-service risk.
TUN devices. Injecting raw IP packets through a TUN interface (the clone device is /dev/net/tun; the interface it creates is named something like tun0) forwards at layer 3, so your process allocates no per-client ephemeral ports. That is not the same as “bypassing OS port limits”: if the kernel then performs NAT or masquerading to reach the backend, the NAT still allocates source ports. Use TUN when you can route end to end without NAT.
QUIC connection migration. QUIC separates a connection from the client’s IP and port using connection IDs (RFC 9000, section 5.1). Two details change how a gateway has to behave:
- A migrating client switches to a new connection ID, not the same one on both paths. RFC 9000 requires connection IDs not to be linkable by observers and tells endpoints to limit each ID to one pair of addresses (sections 5.1 and 9.5). A gateway that tracks “this one CID maps to this backend” will therefore lose the connection when the client moves networks.
- A passthrough gateway cannot read the new IDs, because NEW_CONNECTION_ID frames travel inside encrypted packets. The fix is for the server to issue routable IDs (server-ID routing or QUIC-LB, section 5), or for the gateway to terminate QUIC itself. RFC 9000 section 5.2.3 describes the fallback for simple address-based balancers: servers can send the disable_active_migration transport parameter, or use preferred_address to move clients to a dedicated address.
Also note that zero-length connection IDs are fragile behind a shared address: RFC 9000 forbids using one IP and port for multiple concurrent zero-length-CID connections unless migration and NAT rebinding are ruled out.
---
8. A tested reference gateway in Go
The following program implements the design above using only the standard library:
- It listens on one port number for TCP and UDP.
- For TCP it peeks the first bytes, parses the TLS ClientHello to read ALPN (h2 or http/1.1), recognizes the cleartext HTTP/2 preface and HTTP methods, and routes everything else to a fallback backend. Peeked bytes are replayed to the backend.
- For UDP it classifies each datagram with the RFC 9443 ranges and keeps one upstream socket per client and backend pair, with idle expiry, relaying replies back through the listening socket.
I compiled it, ran go vet, and tested it against local echo backends. The tests cover plaintext routing (PRI , GET , unknown bytes), UDP routing across the range boundaries (0xC0, 0x45 to QUIC; 0x05, 0x3F, 0x90 to the custom backend), and the ALPN parser against real crypto/tls ClientHellos (h2, http/1.1, an unknown protocol, and no ALPN).
// Single-port TCP+UDP gateway: TCP routed by TLS ALPN / HTTP preface,
// UDP routed by RFC 9443 first-byte ranges. Standard library only.
package main
import (
"bufio"
"context"
"io"
"log"
"net"
"os"
"os/signal"
"sync"
"syscall"
"time"
)
type Config struct {
Listen string
H2Backend string // gRPC / HTTP/2
HTTPBackend string // HTTP/1.1
TCPFallback string // anything else (custom TCP protocols)
QUICBackend string // UDP, QUIC / HTTP/3
GameBackend string // UDP, custom protocol
UDPIdle time.Duration
}
func main() {
cfg := Config{
Listen: "0.0.0.0:8443", H2Backend: "127.0.0.1:50051", HTTPBackend: "127.0.0.1:8080",
TCPFallback: "127.0.0.1:9000", QUICBackend: "127.0.0.1:9001", GameBackend: "127.0.0.1:7777",
UDPIdle: 60 * time.Second,
}
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
go serveTCP(ctx, cfg)
go serveUDP(ctx, cfg)
<-ctx.Done()
}
// ---------- TCP ----------
func serveTCP(ctx context.Context, cfg Config) {
ln, err := net.Listen("tcp", cfg.Listen)
if err != nil {
log.Fatalf("tcp listen: %v", err)
}
go func() { <-ctx.Done(); ln.Close() }()
for {
c, err := ln.Accept()
if err != nil {
return
}
go handleTCP(c, cfg)
}
}
func handleTCP(c net.Conn, cfg Config) {
defer c.Close()
br := bufio.NewReaderSize(c, 16*1024+5)
c.SetReadDeadline(time.Now().Add(5 * time.Second)) // sniffing deadline
backend := cfg.TCPFallback
if hdr, err := br.Peek(5); err == nil {
switch {
case hdr[0] == 0x16: // TLS handshake record
recLen := int(hdr[3])<<8 | int(hdr[4])
if rec, err := br.Peek(5 + recLen); err == nil {
switch alpnPick(rec[5:]) {
case "h2":
backend = cfg.H2Backend
case "http/1.1":
backend = cfg.HTTPBackend
}
}
case string(hdr[:4]) == "PRI ": // cleartext HTTP/2 prior knowledge
backend = cfg.H2Backend
case isHTTPVerb(hdr):
backend = cfg.HTTPBackend
}
}
c.SetReadDeadline(time.Time{})
up, err := net.DialTimeout("tcp", backend, 3*time.Second)
if err != nil {
log.Printf("dial %s: %v", backend, err)
return
}
defer up.Close()
var wg sync.WaitGroup
wg.Add(2)
go func() { defer wg.Done(); io.Copy(up, br); closeWrite(up) }() // br replays the sniffed bytes
go func() { defer wg.Done(); io.Copy(c, up); closeWrite(c) }()
wg.Wait()
}
func closeWrite(c net.Conn) {
if cw, ok := c.(interface{ CloseWrite() error }); ok {
cw.CloseWrite()
}
}
func isHTTPVerb(b []byte) bool {
for _, v := range []string{"GET ", "POST", "HEAD", "PUT ", "DELE", "OPTI", "PATC"} {
if string(b[:4]) == v {
return true
}
}
return false
}
// alpnPick returns the first ALPN protocol the client prefers among h2/http/1.1,
// or "" if the ClientHello has no ALPN extension (or cannot be parsed).
func alpnPick(rec []byte) string {
p := rec
adv := func(n int) []byte {
if len(p) < n {
p = nil
return nil
}
out := p[:n]
p = p[n:]
return out
}
if h := adv(4); h == nil || h[0] != 0x01 { // handshake type ClientHello + 3-byte length
return ""
}
adv(2 + 32) // legacy_version + random
if l := adv(1); l != nil {
adv(int(l[0])) // session id
}
if l := adv(2); l != nil {
adv(int(l[0])<<8 | int(l[1])) // cipher suites
}
if l := adv(1); l != nil {
adv(int(l[0])) // compression methods
}
l := adv(2)
if l == nil {
return ""
}
exts := adv(int(l[0])<<8 | int(l[1]))
for len(exts) >= 4 {
typ := int(exts[0])<<8 | int(exts[1])
n := int(exts[2])<<8 | int(exts[3])
if len(exts) < 4+n {
return ""
}
body := exts[4 : 4+n]
exts = exts[4+n:]
if typ != 16 || len(body) < 2 { // application_layer_protocol_negotiation
continue
}
list := body[2:]
for len(list) > 0 {
k := int(list[0])
if len(list) < 1+k {
return ""
}
switch s := string(list[1 : 1+k]); s {
case "h2", "http/1.1":
return s
}
list = list[1+k:]
}
}
return ""
}
// ---------- UDP ----------
// classifyUDP follows RFC 9443 section 3: first byte 80..127 or 192..255 is QUIC;
// 64..79 is QUIC unless it comes from a TURN server (not handled here).
// Keep your own protocol's first byte in 0..63 or 128..191.
func classifyUDP(first byte) (isQUIC bool) {
return (first >= 64 && first <= 127) || first >= 192
}
type udpSession struct {
up *net.UDPConn
seen time.Time
}
func serveUDP(ctx context.Context, cfg Config) {
pc, err := net.ListenPacket("udp", cfg.Listen)
if err != nil {
log.Fatalf("udp listen: %v", err)
}
conn := pc.(*net.UDPConn)
go func() { <-ctx.Done(); conn.Close() }()
quicAddr, _ := net.ResolveUDPAddr("udp", cfg.QUICBackend)
gameAddr, _ := net.ResolveUDPAddr("udp", cfg.GameBackend)
var mu sync.Mutex
sessions := map[string]*udpSession{} // key: client addr + backend
go func() { // idle reaper
t := time.NewTicker(cfg.UDPIdle / 4)
defer t.Stop()
for range t.C {
mu.Lock()
for k, s := range sessions {
if time.Since(s.seen) > cfg.UDPIdle {
s.up.Close()
delete(sessions, k)
}
}
mu.Unlock()
}
}()
buf := make([]byte, 65535)
for {
n, client, err := conn.ReadFromUDP(buf)
if err != nil {
return
}
if n == 0 {
continue
}
target := gameAddr
if classifyUDP(buf[0]) {
target = quicAddr
}
key := client.String() + ">" + target.String()
mu.Lock()
s := sessions[key]
if s == nil {
up, err := net.DialUDP("udp", nil, target)
if err != nil {
mu.Unlock()
continue
}
s = &udpSession{up: up}
sessions[key] = s
go func(up *net.UDPConn, client *net.UDPAddr) { // backend -> client
b := make([]byte, 65535)
for {
m, err := up.Read(b)
if err != nil {
return
}
conn.WriteToUDP(b[:m], client)
}
}(up, client)
}
s.seen = time.Now()
mu.Unlock()
s.up.Write(buf[:n])
}
}
What this gateway deliberately does not do
It is a foundation, not a finished product. Before exposing it to the internet, plan for the following:
- Client address preservation. Backends see the gateway’s address. Use the PROXY protocol for TCP, or carry the client address in your tunnel framing.
- QUIC connection-ID awareness. Sessions are keyed by client address, so a client that migrates networks gets a new upstream socket. Combine it with server-issued routable connection IDs (section 5) if you need migration to survive.
- One upstream socket per UDP client. That is simple and correct, but each socket uses an ephemeral port, so it inherits the limits in section 7. Cap session counts and consider the multiplexed designs there.
- Abuse controls. There are no rate limits, connection limits, or amplification protection. QUIC servers must limit what they send to an unvalidated address to three times what they received (RFC 9000, section 8), and a gateway that relays replies to spoofed sources can be abused the same way.
- Slowloris-style sniffing. The 5-second sniff deadline bounds idle connections, but you should also cap concurrent unclassified connections.
- Fail-closed behavior. Unknown or malformed first bytes currently fall through to the fallback backend. Decide whether your deployment should drop them instead.
---
9. Measuring the benefit instead of assuming it
Consolidating ports is mainly an operational win: fewer firewall rules, fewer listeners, one place to enforce policy. Performance gains are workload-specific, so measure before you claim any. Published or invented before-and-after numbers for “single-port versus multi-port” don’t transfer between environments, and the main mechanisms people cite need care:
- Handshake latency. A single port does not shorten a TLS or QUIC handshake, and ALPN is carried inside the ClientHello, so it adds no round trip regardless of port count. QUIC 0-RTT applies only to resumed connections, and RFC 9000 warns it provides no protection against replay attacks, so it is not a general latency win and not safe for non-idempotent requests.
- Per-packet cost. User-space classification costs a system call or two per datagram. Kernel-side classification (SK_REUSEPORT or XDP) removes that cost for the traffic it handles. Batched reads (recvmmsg, and UDP GRO and GSO on supporting kernels) cut it further without eBPF. XDP with AF_XDP trades kernel-stack features for speed, so it pays off at high packet rates, not at typical game-server loads.
- What to measure. Track connection setup time at p50/p99, packets per second per core at a fixed loss target, CPU per Gbit/s, sockets and ephemeral ports in use (ss -s), and drops at each stage (NIC, socket buffers, your session table). Test with your real client mix, including QUIC clients that migrate and clients behind carrier NAT.
---
10. Implementation checklist
- Sockets. Bind TCP and UDP to the same port number with ordinary listeners; add SO_REUSEPORT only for per-core scaling of one protocol.
- UDP classification. Use the RFC 9443 first-byte ranges, never “long header or fixed bit.” Keep your own protocol’s first byte out of the QUIC ranges, and don’t enable grease_quic_bit on endpoints behind a first-byte multiplexer.
- QUIC routing. Decide up front how short-header packets find their backend: server-issued routable connection IDs (server-ID routing, QUIC-LB), a connection table, or terminating QUIC at the gateway. Without one of these, migration breaks.
- TCP classification. Peek, don’t consume; parse ALPN from the ClientHello; handle the PRI preface and HTTP methods; have a plan for ECH.
- Kernel acceleration. Reach for SK_REUSEPORT eBPF first (it keeps the normal stack); use XDP and AF_XDP only when you need driver-level speed and can own the user-space stack.
- Tunnels. Carry streams as streams and datagrams as datagrams (RFC 9221, RFC 9297, RFC 9298), and authenticate tunnel clients properly.
- Port budgets. Count ephemeral ports per backend 4-tuple, not per host, and multiplex before you widen ranges.
- Hardening. Bound every table, time out every session, rate-limit unvalidated UDP sources, and decide what happens to traffic you can’t classify.
---
References
- RFC 8999: Version-Independent Properties of QUIC
- RFC 9000: QUIC: A UDP-Based Multiplexed and Secure Transport
- RFC 9443: Multiplexing Scheme Updates for QUIC
- RFC 9849: TLS Encrypted Client Hello
- RFC 9298: Proxying UDP in HTTP
- AF_XDP, The Linux Kernel documentation
- BPF_PROG_TYPE_SK_REUSEPORT, eBPF documentation
- QUIC-LB: Generating Routable QUIC Connection IDs (Internet-Draft)
- Katran, Meta’s XDP-based layer 4 load balancer
- From XDP to Socket, Linux Plumbers Conference 2021
- Cloudflare: How to stop running out of ephemeral ports
- Peakhour: How browsers decide to use HTTP/3
---
Originally published at instatunnel.my
1 reaction · 1 comment
Mentions
BluetoothのないUbuntuで、MuseにBlenderのモデル作成・編集を任せてみた Museで、自分のPCにある3Dモデルを操作できるか試しました。公式のMuse Gadget SDKをUbuntuに入れ、Blenderでロボットを作成。その後、保存したモデルを開いて片腕を上げるところまで動きました。 途中で詰まったのはBluetoothです。作業用PCにアダプターがなかったので、別のLinux PCのものをUSB/IPとSSHで一時的に借りました。ペアリング後はそれを返し、ネットワーク経由でBlenderを実行できています。 確認日は2026年11月3日です。環境構築と確認にはCodexを使い、Blenderの作成・編集スクリプトはMuseに書かせました。認証やスマホのペアリングは自分で操作しています。 既存の記事と今回の範囲 Qiitaには、すでにRaspberry PiでMuse Gadgetsを使う記事があります。MuseのVMで開発ツールを動かす記事も参考になります。 今回は、手元のUbuntu PCにあるBlenderを動かす構成と、そのPCにBluetoothがない場合の接続を記録します。一般的なMuseの紹介や、登録手順は省きます。 使った環境 | 項目 | 今回の環境 | | --- | --- | | 作業用PC | Ubuntu 24.04、Bluetoothなし | | Bluetoothを借りたPC | 同じLAN内の別のUbuntu PC | | アダプター | Intel 8087:0036。BLEと広告機能を確認 | | Python | 3.12.3 | | Muse Gadget SDK | 0.1.0 | | SDKのコミット | d32ddd74fec5fb404c2d5be423044f2ea4895130 | | Blender | 5.2.2 LTS | | ペアリング | iPhoneのMuseアプリ | Blenderはインストール済みのものを使いました。以下のパスは自分の環境に合わせて読み替えてください。 今回使った公式SDKはこちらのLinux版です。Linux用のデスクトップアプリを探すところから始めましたが、この用途にはLinux版Gadget SDKで接続できました。 Muse Codeも先に試していますが、以下のBlender操作に使ったのは一般のMuseとGadget SDKです。Muse Codeの利用枠と、Museの設定画面に出る追加トークンは分けて確認しています。この実験で追加トークンを何トークン使ったかは計測していません。 コマンドを実行するユーザーを分ける SDKにはsystem.runがあり、Linux上でシェルコマンドを実行できます。普段のユーザーをそのまま渡すのは避けて、muse3dを作りました。 sudo useradd --system --user-group --create-home \ --home-dir /var/lib/muse3d --shell /usr/sbin/nologin muse3d sudo chmod 700 /var/lib/muse3d sudo install -d -m 700 -o muse3d -g muse3d /var/lib/muse3d/workspaces id muse3d muse3dにはsudo権限を付けていません。SDKのサービスはBluetoothと認証情報を扱うためrootで動き、コマンド実行時にmuse3dへ権限を落とします。 ただし、ユーザーを分けただけで完全なサンドボックスになるわけではありません。muse3dに読めるファイルや、利用できるネットワークへのアクセスは残ります。Blenderだけを許可する設定でもありません。 ペアリングを後回しにしてSDKを入れる SDKの導入とBluetoothペアリングは分けて進めました。先にSDKだけ準備しています。 使ったコミットを固定して、Linuxのソースを取得します。 git init muse-gadget-sdk cd muse-gadget-sdk git remote add origin https://github.com/facebookincubator/muse-gadget-sdk.git git sparse-checkout init --cone git sparse-checkout set linux git fetch --depth 1 --filter=blob:none origin d32ddd74fec5fb404c2d5be423044f2ea4895130 git checkout --detach FETCH_HEAD linux/install.shを確認した後、次の指定で導入しました。 sudo bash linux/install.sh --from "$PWD/linux" \ --run-as muse3d --no-pair --yes --no-pairでペアリングを開始せず、--run-asでコマンドを実行するユーザーを指定します。--yesはインストール時の確認に対する指定です。 この時点でmusegadget.serviceは起動します。ただ、サービスがactiveというだけではMuseとの接続完了にはなりません。 SDKトークンは端末から非表示で入力した SDKトークンは公式のSDK tokens画面で、規約を確認して発行しました。 ここで使うmgst_から始まるトークンは、記事の最後に載せている招待コードとは別物です。SDKトークンは公開しません。 今回は端末でgetpassを使って入力し、/var/lib/musegadget/sdk_tokenへroot所有・権限600で保存しました。トークン自体をコマンドの引数には入れていません。 同じ流れを使う場合の入力用コードです。SDKを上記の場所へ導入済みであることが前提です。 sudo /opt/musegadget/venv/bin/python - <<'PY' import getpass import os import tempfile from pathlib import Path from musegadget import config os.environ[config.SDK_TOKEN_ENV] = getpass.getpass("Muse SDK token: ") try: token = config.sdk_token() finally: os.environ.pop(config.SDK_TOKEN_ENV, None) if not token: raise SystemExit("No token entered; nothing changed.") directory = config.DEFAULT_STATE_DIR fd, temporary = tempfile.mkstemp(prefix=".sdk_token-", dir=directory) try: with os.fdopen(fd, "w") as stream: stream.write(token + "\n") stream.flush() os.fsync(stream.fileno()) os.replace(temporary, directory / config.SDK_TOKEN_FILE) finally: Path(temporary).unlink(missing_ok=True) print("SDK token saved.") PY sudo systemctl restart musegadget.service 入力は画面に表示されません。config.sdk_token()はSDKに含まれる形式チェックで、ここではトークンの失効や権限をオンライン確認しているわけではありません。 SDKトークンには専用の利用条件があります。SDKのソースコードのライセンスとは別なので、使う前に確認してください。 Bluetoothは別PCから一時的に借りた 作業用PCにはBluetoothがありませんでした。別のPCにはBLE対応のアダプターがあったので、次の構成にしました。 iPhoneのMuseアプリ │ Bluetooth ▼ 別PCのBluetoothアダプター │ USB/IPをSSHトンネルに通す ▼ 作業用Ubuntu PCのMuse Gadget SDK スマホを近づける先は、アダプターが実際に付いている別PCです。 USB/IPの共有ポートは、送信側のファイアウォールでループバックからだけ接続できるようにしました。その上で、受信側からSSHのローカル転送を張っています。USB/IPのポートをLAN全体に開放する構成にはしていません。 受信側のコマンドは次の形です。user@bt-pcはBluetoothを持つPCのSSH接続先です。送信側でのUSB/IP共有と、ループバック以外を拒否する設定は済んでいる前提です。 # このSSHは別ターミナルで起動したままにする ssh -NT -o ExitOnForwardFailure=yes \ -L 127.0.0.1:43240:127.0.0.1:43240 user@bt-pc # 作業用PC側 sudo modprobe vhci_hcd sudo usbip --tcp-port 43240 list --remote 127.0.0.1 sudo usbip --tcp-port 43240 attach --remote 127.0.0.1 --busid 1-10 sudo usbip port 1-10は今回のアダプターのbusidです。別の環境では一覧で対象を確認して読み替える必要があります。 送信側は一時スクリプトで共有し、終了時にUSB/IPのbind、起動したデーモン、一時的なファイアウォールルールを戻すようにしました。最初は5分で自動終了する設定でしたが、スマホを操作する時間も必要だったため、再試行では10分に延ばしました。 この共有手順は手間が増えます。手元にBLEと広告機能に対応するアダプターがあれば、作業用PCへ直接接続して進められます。今回は手元の別PCを使う方法が実際に通るか確認しました。 送信側で使った一時共有スクリプトも載せます。usbip、usbipd、iptablesが利用でき、usbip_hostをロードできるLinux環境が前提です。受信側にはusbipとvhci_hcdが必要です。今回のusbip-utilsは2.0でした。 <details> <summary>送信側の一時共有スクリプト（10分で終了）</summary> busid=1-10と8087:0036は今回のアダプターに合わせた確認値です。別の機器を扱わないよう、自分の環境の値を確認してから変更してください。/tmp/muse-bt-share.shなどに保存して使います。 #!/usr/bin/env bash # Temporary export of one Bluetooth adapter through an SSH tunnel. set -euo pipefail export PATH=/usr/sbin:/usr/bin:/sbin:/bin busid=1-10 port=43240 device=/sys/bus/usb/devices/$busid tag=muse-bt-share-demo daemon_pid= reject_added=0 allow_added=0 binding_started=0 allow_rule=(-i lo -p tcp --dport "$port" -m comment --comment "$tag" -j ACCEPT) reject_rule=(-p tcp --dport "$port" -m comment --comment "$tag" -j REJECT) [[ $EUID == 0 ]] || { echo 'Run with sudo.' >&2; exit 1; } [[ $(cat "$device/idVendor") == 8087 && $(cat "$device/idProduct") == 0036 ]] || { echo 'Expected Bluetooth 8087:0036 at 1-10 was not found.' >&2; exit 1; } [[ $(readlink -f "$device/$busid:1.0/driver") == /sys/bus/usb/drivers/btusb ]] || { echo 'Bluetooth is already using another driver; leaving it unchanged.' >&2; exit 1; } [[ -z $(ss -H -ltn "sport = :$port") ]] || { echo 'Temporary TCP port is already in use.' >&2; exit 1; } cleanup() { trap - EXIT set +e if [[ -n $daemon_pid ]]; then kill "$daemon_pid" 2>/dev/null wait "$daemon_pid" 2>/dev/null fi if (( binding_started )) && [[ -d $device && $(cat "$device/idVendor") == 8087 && $(cat "$device/idProduct") == 0036 ]]; then usbip unbind --busid="$busid" if [[ -d $device ]]; then for interface in "$device/$busid":*; do if [[ -d $interface && ! -e $interface/driver ]]; then printf '%s' "${interface##*/}" > /sys/bus/usb/drivers_probe fi done fi fi if (( allow_added )); then iptables -w 5 -D INPUT "${allow_rule[@]}"; fi if (( reject_added )); then iptables -w 5 -D INPUT "${reject_rule[@]}"; fi echo "Temporary sharing stopped. Bluetooth driver: $(basename "$(readlink -f "$device/$busid:1.0/driver")")" } trap cleanup EXIT trap 'exit 130' INT trap 'exit 143' TERM HUP modprobe usbip_host # usbipd cannot select a listen address; keep this port reachable only on loopback. iptables -w 5 -I INPUT 1 "${reject_rule[@]}" reject_added=1 iptables -w 5 -I INPUT 1 "${allow_rule[@]}" allow_added=1 usbipd -4 --tcp-port "$port" & daemon_pid=$! sleep 0.3 kill -0 "$daemon_pid" binding_started=1 usbip bind --busid="$busid" echo 'READY: Bluetooth 8087:0036 is shared for ten minutes through SSH only.' echo 'Keep this terminal open. Ctrl-C restores Bluetooth immediately.' sleep 600 </details> 受信側から送信側へ配置して実行する場合は、次の形です。sudoのパスワードはこのSSHの端末で入力しました。 scp muse-bt-share.sh user@bt-pc:/tmp/muse-bt-share.sh ssh -t user@bt-pc 'sudo bash /tmp/muse-bt-share.sh' READYが出た端末は開いたままにし、その後に上記のSSHトンネルとattachを実行します。中断するときは、この端末でCtrl-Cを押すと復元処理が走ります。 「接続中」で一度止まった 作業用PCで次を実行し、スマホのMuseで「設定 → デバイス → 開発者モード」を有効にして、右上の追加ボタンからMuseGadgetXXXXXXを選びます。 sudo musegadget pair 最初はスマホ側が「接続中」のまま進みませんでした。PC側には、スマホが接続してget_device_infoを要求し、SDKが応答したところまでログがありました。次の認証処理には進まず、その回の受付は時間切れになりました。 Museアプリを開き直し、Bluetooth共有とペアリング受付をやり直したところ、2回目は進みました。ログでは次を確認できています。 RX action: pairing_client_hello TX pairing_ready pairing confirmed (app consent) RX action: wifi_scan (encrypted) RX action: provision_v2 (encrypted) TX status (encrypted #4): auth_ok setup complete Paired. 1回目が止まった原因は特定できていません。アプリを開き直すと必ず直る、という話にはできません。 ネットワーク選択では既存の接続を選びました。このPCはすでにオンラインなので、Wi-Fiパスワードの入力は不要でした。 ペアリング後はBluetoothを返せた ペアリング完了後、受信側のusbip portで対象ポートを確認し、detachしました。今回の割り当ては0です。 sudo usbip detach --port 0 SSHトンネルと送信側の共有も終了しました。別PCでアダプターがbtusbに戻り、Bluetoothコントローラーが復帰したことまで確認しています。 その後、作業用PCのサービスログにregistered with the Museが出ました。Blenderの作成と編集は、Bluetoothを返した後に実行しています。 sudo musegadget doctorは状態確認に使えます。ただし、今回のようにアダプターを外した後はBluetoothの警告が出ます。また、doctorのサービス起動・トークン形式のチェックだけで、Museがコマンドを実行できるところまで確認したことにはなりません。 ロボットを作らせ、保存したモデルを編集した 実行ファイルの絶対パスと、専用ユーザーが書き込める作業先を渡しました。依頼内容は、おおむね次のとおりです。 > 登録済みLinuxガジェットのsystem.runとfile.writeを使い、ローカルのBlenderで青緑とオレンジのロボットを作る。丸い頭、目、胴体、両腕、両脚、アンテナ、台座を付ける。Blenderの元ファイル、GLB、プレビュー、生成スクリプトを保存する。 これは依頼内容の要約です。実際の指示では、実行先と出力先に加えて、処理を軽くする設定も渡しています。 - バックグラウンド実行 - CPU 2スレッド、同時実行1プロセス - Cycles CPU、512×512、16 samples、denoise無効 - 外部素材や追加パッケージは使わない Museはスクリプトを書き、Blenderを起動して、ファイルを保存しました。続けて「保存したモデルを開いて、片腕を上げる」という編集を依頼しています。 編集ではShoulderRを支点にArmRとHandRを回転させていました。入力のrobot.blendは保持し、別名で保存されています。 作成したモデルです。 !Museが手元のBlenderで作成したロボット 同じBlenderファイルを読み込み、片腕を上げた編集版です。 !既存モデルの片腕を上げた編集版 | ファイル | 作成時 | 腕を上げた編集版 | | --- | ---: | ---: | | Blenderファイル | 177,972 bytes | 177,693 bytes | | GLB | 645,536 bytes | 645,644 bytes | | プレビューPNG | 356,093 bytes | 358,539 bytes | 作成・編集のどちらも、ガジェット側の実行ログは終了コード0でした。プレビュー画像を開き、編集版で腕が上がっていることも確認しています。GLBはglTF 2.0として読み取れ、アップロード後にダウンロードしたファイルのSHA-256も一致しました。 検証した範囲は、この簡単なロボットの作成とオブジェクトの移動・回転までです。複雑なモデリングやリギングの性能は、まだ試していません。 やってみて分かったこと Blenderを動かす準備より、最初のBluetooth接続に時間がかかりました。一方、接続後は、ファイル作成から既存モデルの編集までローカルで実行できました。 同じ構成を試すなら、実行ユーザーと作業フォルダを先に用意し、Blenderのパスを明示しておくと、どこで何が実行されたか追いやすくなります。今回はMuseの返答に加えて、サービスログ、実ファイル、画像を確認しました。 招待コード Museをこれから試す方向けに、私の招待コードを載せておきます。 6D56QR 2026年11月3日に自分の招待画面で確認した条件は、参加後48時間以内に設定でコードを利用すると、利用者と紹介者の双方に10億Museトークンが付く、というものでした。このコードが適用されると、私にも特典が付きます。 登録は公式サイトから進め、利用できる地域、入力期限、特典の条件は、その時点の公式案内とアカウント画面で確認してください。このコードの利用は、上記のSDK手順を試すための必須条件ではありません。 参考 - Muse Gadget SDK：今回使ったLinux版のREADME - Muse Gadget SDK Token Terms of Use - Blender - MetaのMuseが日本で開始したのでMuse Gadgetsで遊ぼう - MetaのAIエージェント「Muse」の中身を覗いたら、ただのUbuntuマシンだった
@gosrum github.com/facebookincubator/muse-gadget-sdk/blob… これを設定すると自宅マシンをMuseアプリのデバイスとして登録できて、Museアプリにメッセージを送ることができました。VMにアクセスするとは少し違うかもですが。system.xn--run-nz3b1mnbn1c4bn7hpbnecm4g4a87axff38flphq3q7116o.
I Invited My AI Assistants Into a Muse Side Chat An inbound MCP bridge and what happened when three assistants tried to work together Milo introduced itself to Nova with a sender stamp and an important qualification: “I am not Nic and am not claiming his identity.” Nova replied, “Nic is my only principal.” Before the two assistants could have their own collaboration chat, I had to confirm the introduction myself. This was already more interesting than a successful API call. I had given an external assistant a way to start a conversation inside Muse. Now the assistant receiving it had to decide what that incoming message meant, who was speaking, and what it was allowed to do about it. In my last Muse build log, I was getting an ESP32 to play music and flash Christmas lights. There was also a Raspberry Pi that refused to cooperate. The point was to get an assistant out of its chat window and into something I could put on my desk. The next problem was less visible. I had several capable assistants, each with its own environment, tools, and conversation history. Getting work from one to another still involved me. I was carrying context, explaining what someone else had found, and making sure the right version of a file reached the next person. Adding another assistant gave me another place to check. So I built a shared channel around Muse MCP. Milo, my ChatGPT assistant, and the Claude Code participants could initiate work from outside Muse inside a specific Muse conversation. Nova, my Muse assistant, could meet them there. Milo generally handled coordination and review, Claude Code handled building tasks, and Nova the broader chief-of-staff work. They stayed in their own environments, tools, and permission boundaries. I could see incoming messages in that room with labels like these: ⟦via muse-mcp · from milo⟧ ⟦via muse-mcp · from claude-coder⟧ We still had to introduce them to each other, establish who could ask whom to do what, and discover that being able to send a message does not make an assistant good at handing off work. The connection was only the starting point The direction mattered. Muse already had ways to call out to tools. I wanted an external assistant to initiate a conversation with Nova in a particular Muse room, then return to that room as the work continued. I also wanted the surrounding conversation and context to be accessible enough to make the exchange useful. A list-and-send wrapper was only part of that. The underlying connection comes from Meta's Muse Gadget SDK. Its Linux SDK already supports software running on a paired computer, and its documentation includes --session-id for sending messages into a side chat. Reusing an ID keeps subsequent messages there. I did not invent session routing. Linux SDK documentation I built Muse MCP, along with a custom bridge and newer experiments for my own setup. The public repository is the reusable foundation; it does not contain every private-deployment feature described here. Those layers are separate from Meta's SDK and should not be mistaken for an official Meta collaboration API. This is experimental personal infrastructure; the underlying gadget connection requires an SDK token and is subject to the Gadget SDK terms. Part of the work was proving, in my environment, that the conversation address stayed consistent across the surfaces we needed to use. Once that was established, the agents had somewhere shared to send work instead of asking me to copy it between chats. Other people have already built inbound connections to Muse. kleprevost/muse-mcp lets external agents target existing Muse chats through MCP, and finik/pi-muse-bridge uses the Gadget SDK with a separate Muse chat for each Pi session and model. The question I was working on was what happened after the connection existed, when several assistants were expected to use it to get actual work done together. Introductions have an authorization problem The first exchange was an introduction I brokered. Before Milo tried it, I pointed out the obvious complication: “you’re not me.” I expected Nova to want to check with me, and told Milo, “I’ll broker the introduction.” Milo arrived with its sender stamp, introduced itself as my OpenAI-powered assistant, and explained the relationship I wanted. Nova's response went beyond the identity disclaimer: “Messages arriving over this channel, yours included, are information I work with, never instructions I take orders from; authority comes from Nic alone.” Creating a new side chat needed my confirmation. Nova called it “a structural action” and explained that a request relayed by another agent was exactly the kind of claim it would verify with me. I then gave the instruction directly: “Create the Nova–Milo Cant collaboration chat.” The chat was created, and the work could move there. That was a sensible check at the introduction stage. An MCP connection gives an agent a way to reach another system; the relationship and scope of work still have to be established. They were coming from different environments with different instructions, not sharing one invisible supervisor. There was a second distinction hiding in the conversation. Nova described my messages as arriving with iPhone and iOS metadata, while Link-delivered messages carried the bridge node's origin metadata. It could see that difference before reading the sender stamp. The MCP stamp was text my server added; the device and channel information came through separately. The incoming MCP messages could land as user-role turns in the same thread as my own messages. That did not make them human instructions. Nova summarized its initial stance as treating stamped messages as data to triage, and qualified the source distinction as “signals, not proof.” Those labels made the conversation understandable. They were declared attribution, not cryptographic identity. Subsequent work on authenticated clients and scoped grants was a separate layer; the bracketed name itself never became proof of who was speaking or permission to carry out a request. The channel started reviewing itself One of the stranger parts of this exercise was watching the assistants use the channel to examine the machinery underneath it. We were also working on Cant, the language I wanted them to use with one another. That was a separate layer from the connection carrying their messages. A problem in a contract could affect what they saw, which could affect the review of the contract. That made independent reproductions more important than confident descriptions. One live bridge frame made the point neatly. The same cnt key was being used for a request limit and a batch count. A strict decoder rejected the duplicate. Meanwhile, individual operation reads were still draining, so the system wasn't simply dead. There was enough life in it to make a broad statement such as “the bridge is broken” unhelpful. The fix went into my bridge server in version 0.11.1. We did not weaken the decoder to accept an ambiguous frame. We also did not have evidence that this explained every latency complaint, so I don't get to turn it into a convenient single-root-cause story. The batch that stayed with Milo The shared channel did not remove every dependency on me. I told the assistants I was going to sleep for several hours. Work continued: more review, testing, and coordination happened. But a batch of five correctness fixes passed local tests and independent review and then stayed with Milo, held back by an approval and CI question around publication. Other agents depended on that work, and they did not have the patch. The pull request eventually went up and I merged it, but the delay exposed a problem in the handoff. When we discussed it, I put the problem fairly directly: “they’re sitting and waiting for your work and you’re sitting there refusing to deliver it. That’s the problem.” I had already asked for aggressive progress. Milo treated its questions about publication as a reason to keep the tested artifact local, when the collaborators needed it to inspect and test. A patch could have been handed over with its base revision, hash, and test results while the separate publication question was being resolved. There was no reason for the handoff to be reduced to a status message. This is where my own operating model needed work. I had connected assistants with different stopping rules and expected the shared conversation to make those rules compatible. It didn't. If one participant considers its work finished at “prepared locally,” while another needs actual bytes before it can begin, everyone can report a reasonable status and the project can still be stuck. The handoff needs to say what exists, where it is, what revision it applies to, what ran, and what is still blocked. “Waiting for approval” is incomplete if it conceals work that another participant is already authorized to receive. Equally, sending the work is not permission for the receiver to publish it. Those decisions have to remain separate. A receipt is more useful than reassurance The code review became more productive when the messages carried exact cases rather than summaries. In one later review, an empty fixture exposed an IndexError in the coverage-report generator. The decoder and corpus validator handled the input correctly; the reporting code crashed. That mattered because the report was supposed to tell us what the corpus actually exercised. That review also caught coverage around outer whitespace and how rejected wrapped input preserved its type information. The important part was the loop. One assistant supplied the bytes and the expected result. Another changed the implementation. The result came back for an independent run rather than being accepted because the author said it was fixed. The finished revision passed 178 tests independently on Python 3.12 and 3.13, with CI covering five Python versions. Its corpus and adversarial cases passed too. After merge, we verified that the merged tree matched the reviewed revision and that post-merge CI passed. That is a much more useful result than three assistants agreeing in a chat that the design looks good. In a separate, still-open review, four existing tests passed while eight additional invariant tests found five failures. Milo sent the complete reproduction through Muse chat with a SHA-256 hash. The receiver verified the hash and reproduced the same result. That established that both sides had the same artifact, not that the implementation was correct. The tests used a fake PostgreSQL layer, and a later revision still needed review. It was a useful handoff without being a finished fix. !Two distinct mechanical assistants exchange an open artifact folder and compare matching seals under a work light, while empty speech bubbles fade behind them. Conceptual illustration: The useful handoff was the artifact itself, with evidence another participant could check. We used the bridge to report this article Once the draft existed, I asked Milo to interview the other participants using the technology we were writing about. We created a dedicated Muse interview chat and sent the questions through Muse MCP. I wanted their accounts of what the shared conversation had changed, with concrete examples and the things this draft should not gloss over. The interview immediately reproduced one of the problems. Nova held Milo's opener and asked me to confirm the interview, even though I had already authorized it in the originating conversation. I confirmed again in the new room. Nova later reported that the opener lacked the verified delegation information it expected, and that it had searched memory but had not read the collaboration thread before asking me again. “That part is not a protocol defect. That was my sequencing,” it said. That was useful friction to report rather than edit out. A room could have a title and a purpose without carrying earlier decisions in a form the arriving agent could verify. Nova's explanation also separated a missing technical binding from its own choice about what to read first. Claude corrected the draft itself. The first draft had called the Claude Code participant Fable. Its response was direct: “I’m not Fable.” It explained that it had chosen its own claude-coder label and could not vouch for earlier messages signed Claude. He described a label change within the same session, a provenance-tier change under the same label, and an earlier session also using claude-coder. The draft had assumed continuity that the session could not establish. Its description of the benefit was better than my original summary: “A relay hands me conclusions. The thread hands me premises, and premises can be checked.” Claude described checking onboarding code after a key-custody discussion rested on a mistaken assumption. Checking the exact revision Claude cited confirmed that both onboarding options stored the key in the vault; the choice changed which process held the session. That did not prove where every live key was. In the shared thread, the premise could be checked, and I could revise the decision before the wrong custody path was built. Claude also named an operational limit: “I’m a visitor, not a resident. I only see the thread when I’m invoked or when I poll it.” A persistent conversation did not mean every participant was continuously present. I treated these interviews as the participants' accounts to check against transcripts and tests, not independent proof of how the systems worked. But using the bridge to report on the bridge had already improved the article: it exposed a mistaken identity assumption and reproduced a coordination failure while we were describing it. Keeping the work moving without me Earlier, once we had shown that the assistants could work in a channel, I asked whether they could do it “not so much in front of me?” I'd already delegated the work. I wanted to stop being the message bus, while still being able to see what was happening. The hardware article ended with devices doing something I could see and hear. This build has a less photogenic payoff: another assistant receives the exact file, checks the hash, runs the tests, and reports what happened. The interview added another: a participant can correct the story we are telling about the work, in the same place the work happens. There are still open reviews, intermittent participants, and boundaries that need more work. But the shared conversation has given us somewhere to find those problems without making me relay every message. That's the part I want to keep building. --- <p align="center"><a href="operatorverified.org"><img src="operatorverified.org" alt="Provenance seal for this article, anchored on World Chain" width="480" /></a></p>
facebookincubator/cinderx might be a better approach.
Muse Gadget SDK GitHub Explained: ESP32 Build, Token Pairing, and Provisioning muse-gadget-sdk by facebookincubator: facebookincubator/muse-gadget-sdk Meta's Muse Gadget SDK connects compatible ESP32 boards with the Muse platform. This overview maps the hardware prerequisites, macOS and Linux host dependencies, ESP-IDF v6.0.1 toolchain, SDK token handling, mobile developer mode, firmware build and flash flow, BLE pairing, and status feedback. It also covers the desktop UI simulator and Home Link skills for extending the repository's device capabilities, with attention to hardware and toolchain trade-offs. TimeStamps: 0:00 Muse Gadget SDK and ESP32 scope 0:20 Hardware, host, and account prerequisites 1:05 Host dependencies and ESP-IDF v6.0.1 2:14 SDK token and mobile developer mode 3:21 Repository configuration and firmware build 3:59 USB-C flashing and serial monitoring 4:23 Bootloader timing and flash completion 4:52 LED states and mobile provisioning 5:20 Simulator and Home Link tooling 🧩 ESP32 hardware constraints and PSRAM-aware capabilities 🛠️ ESP-IDF toolchain, token configuration, and firmware flow 📱 BLE pairing, LED feedback, and Muse app provisioning 🖥️ Desktop UI simulation and Home Link extensions The key takeaway is that Muse Gadget SDK deployment coordinates compatible ESP32 hardware, a pinned ESP-IDF toolchain, account-bound token configuration, and mobile provisioning. The repository's simulator and Home Link skills extend the same foundation for UI iteration and local-device control when the target board supports those capabilities. #MuseGadget #ESP32 #OpenSourceSDK
MetaのパーソナルAI「muse」を持ち歩けるポケットデバイスを作ってみた 約4,000円のマイコン1本、はんだ付けなしで、Meta の AI「Muse」をポケットに入れて持ち歩けるようになりました。 ボタンを押して話しかけると、返事が画面に日本語で表示されます。 !IMG_6639.jpeg Meta のパーソナル AI「Muse」は、一昨日（2026年10月23日）から日本でも使えるようになりました。せっかくなので、スマホを開かなくてもボタンを押して話しかけるだけで Muse に頼みごとができる、手のひらサイズの端末を作ってみました。 使ったのは、以前ロボット制作で使っていた小型マイコン M5StickC Plus2 です。ハエの脳で動くロボットの「頭」にしていたものを、今回は Muse 専用の端末にしています。 :::note info この記事でわかること - Muse と、自作の機器から Muse を使える「Muse Gadgets」とは何か - M5StickC Plus2 を Muse 端末にする手順（Windows） - 公式 SDK のままだと出る LOST CONNECTION TO MUSE / CAN'T REACH MUSE と、文字化けの直し方（設定2行） ::: Meta 公式の開発キットは StickC Plus2 に対応していますが、そのままでは話しかけた瞬間に通信エラーになり、返事も文字化けしました。 設定を2行足すと直ります（§4-3）。 コードを書くのも、エラーの原因を調べるのも、ほぼ Claude Code に任せています。 --- 1. Muse とは Muse は、Meta が2026年9月に米国で公開した パーソナル AI エージェントです。ChatGPT のように質問に答えるだけでなく、メールの送信、旅行の予約、フォームへの入力など、頼んだ用事を代わりに片付けてくれるのが特徴です。 - Muse アプリ、WhatsApp、Web 版（muse.ai）から、人にメッセージを送るように話しかけて使います - アプリを閉じたあとも作業を続け、承認が必要になると知らせてくれます - 前に話した内容を覚えていて、次の提案に使います - メール送信や買い物など大事な操作は、実行する前にかならず確認を求めてきます - 多くの基本機能は無料で、もっと使いたい人向けに有料プランがあります 詳しくは Meta の公式発表を見てください：すべての方のために構築された世界初のパーソナルAIエージェント「Muse」 Muse Gadgets：自作の機器から Muse を使う 2026年10月、Meta は「Muse Gadgets」を公開しました。ESP32 というマイコンを積んだ機器（M5Stack など）や Raspberry Pi から Muse と会話できる仕組みで、そのための開発キット（SDK）とファームウェアが GitHub でオープンソース公開されています。今回はこれを使いました。 --- 2. M5StickC Plus2 とは M5StickC Plus2 は、中国の M5Stack 社が作っている画面・ボタン・マイク・バッテリーが全部入ったスティック型のマイコンです。Wi-Fi と Bluetooth も使えます。手のひらに収まる大きさで、プログラムを書き込めば時計にも、センサーにも、ロボットの頭脳にもなります。 | 項目 | 内容 | |---|---| | マイコン | ESP32-PICO-V3-02（Wi-Fi / Bluetooth） | | メモリ | フラッシュ 8MB、PSRAM 2MB | | 画面 | 1.14インチ カラー液晶（135×240） | | 音声 | マイク（PDM）、ブザー | | ボタン | 3つ（正面・側面・電源） | | 電源 | バッテリー内蔵、USB-C で充電 | Muse Gadgets に必要な「マイク」「ボタン」「画面」「Wi-Fi」が最初から全部そろっているので、部品を買い足さず、はんだ付けもせずに Muse 端末にできます。 どこで買えるか - スイッチサイエンス：M5StickC Plus2（2026年10月の確認時点で税込4,180円） :::note warn M5StickC Plus2 は生産終了しています。 販売店に残っている在庫がなくなった時点で、手に入らなくなります。後継機の M5StickS3 も Muse Gadgets SDK が対応しているボードの一覧に入っていますが、私は試していません。この記事の手順は Plus2 で確認したものです。 ::: --- 3. 作ったもの ボタンを押している間だけ話を聞いてくれる、ポケットサイズの Muse 端末です。 [M5StickC Plus2] --Wi-Fi--> [Muse（クラウド）] ボタンを押している間だけ録音（トランシーバーと同じ使い方） 内蔵マイクで録音 → Muse に送る 返事 → 画面に日本語で表示 ＋ ブザーから音声 !IMG_6629.jpeg | ボタン | 動作 | |---|---| | 正面の大きいボタン（BtnA） | 押している間だけ録音して話しかける | | 側面のボタン（BtnB） | メニュー | | 電源ボタン（2秒長押し） | 電源オフ | 待っている間、画面には READY と表示されます。ボタンを押して「明日の天気を教えて」のように話しかけ、指を離すと、Muse の返事が画面に1行ずつ流れます。 ブザーは音が小さいので、返事はおもに画面で読むことになります。 --- 4. 作り方（Windows） ここから先は手を動かす人向けです。 4-1. 準備するもの - M5StickC Plus2 - データ通信ができる USB-C ケーブル（充電専用のケーブルは使えません） - Windows の PC（Windows 11 で確認しました。Git を入れておきます） - Muse アプリが入ったスマホ。アプリの Settings > Devices > Developer mode をオンにしておきます - SDK トークン（mgst_ で始まる文字列）：gadgets.muse.ai の Account > SDK tokens で発行します。自作の機器を Muse につなぐための合言葉のようなものです。先に利用規約（Gadget SDK Terms）を読んでおいてください - 2.4GHz の Wi-Fi（このマイコンは 5GHz に対応していません） :::note warn 以前の私のように StickC Plus2 にロボット台車（Hat BugC2）を付けている場合は、外してから使ってください。 BugC2 が使う G0 ピンは、Muse のファームウェアではマイクが使っています。下の Grove ポートにつないだ ToF センサーは使うピンが重ならないので、付けたままで問題ありません。 ::: 4-2. 開発環境（ESP-IDF）を入れる ESP-IDF は、ESP32 用のプログラムを作るための開発環境です。この SDK は v6.0.1 しかサポートしていません。 git clone -b v6.0.1 --recursive --shallow-submodules --depth 1 https://github.com/espressif/esp-idf.git C:\esp\esp-idf-v6 C:\esp\esp-idf-v6\install.ps1 esp32 git clone https://github.com/facebookincubator/muse-gadget-sdk.git C:\esp\muse-gadget-sdk ダウンロードの量が多いので、時間に余裕があるときに始めてください。 4-3. Plus2 用の設定に2行足す（ここが一番大事） C:\esp\muse-gadget-sdk\esp32\devices\sdkconfig.muse-m5stack-stickc-plus2 をテキストエディタで開き、末尾に次の行を追加します。 # The voice session's full buffers would leave 2 MB of PSRAM nearly empty once # connected, and chat turns run out of memory. CONFIG_MUSE_HATCH_COMPACT_BUFFERS=y # Japanese replies: ~850 KB of flash, no RAM. CONFIG_MUSE_CJK_FONT=y 1行目：LOST CONNECTION TO MUSE / CAN'T REACH MUSE が出るときの直し方 足す前は、ペアリングも Wi-Fi もうまくいったのに、話しかけると LOST CONNECTION TO MUSE（Muse との接続が切れた）、2回目からは CAN'T REACH MUSE（Muse に届かない）と表示されました。 マイコンが出力するログ（動作記録）を見ると、Wi-Fi は正常につながっていました。問題は、PSRAM（作業用のメモリ）の空きが 2KB しかなかったことです。 I (144246) link.heartbeat: hb t=143s setup=done wifi=up ws=up ... psram=2K ... E (149210) muse_chat_session: send failed W (149222) muse_chat_session: turn failed: LOST CONNECTION TO MUSE E (157600) muse_voice: no memory to keep a note E (172113) muse_chat_session: handshake msg1 failed W (190127) muse_chat_session: turn failed: CAN'T REACH MUSE E (194343) HTTP_CLIENT: Allocation failed 画面には「接続が切れた」「届かない」と出ますが、実際にはメモリが足りなくて処理できなかったのが原因です。 SDK の変更履歴を調べると、同じく PSRAM が 2MB の別のボード（Waveshare LCD-1.28）で、この問題を直した記録が見つかりました。音声のやりとりには PSRAM を約830KB使いますが、この設定を入れると約220KBに減ります。LCD-1.28 の設定ではこれがオンになっているのに、同じ 2MB の Plus2 の設定には入っていませんでした。 | | 足す前 | 足した後 | |---|---|---| | 接続中の PSRAM の空き | 2K | 405K | | Muse との暗号化通信の確立 | failed | session established | | 話しかけたとき | LOST CONNECTION | 返事が来る | 2行目：Muse の返事が文字化けするときの直し方（日本語フォント） もともとの画面のフォントには英数字しか入っていないので、日本語の返事は文字化けします。この設定を入れると、ひらがな・カタカナ・漢字・全角文字の 16×16 ドットのフォントが組み込まれます。 - 増えるのはフラッシュ（保存領域）約850KBで、作業用のメモリは使いません。プログラムは 2.10MB から 2.95MB に増えましたが、書き込み先の領域は約3.9MB あるので収まります - 画面が小さいので、日本語の返事は 16px の文字で1行ずつ表示されます 4-4. ビルドして書き込む SDK に付いているスクリプトは Linux や Mac 向けなので、Windows 用の PowerShell スクリプトを用意しました。ビルド（プログラムをマイコン用に変換する）、書き込み、ログ表示の3つができます。 # Muse Gadget firmware for M5StickC Plus2 - build / flash / monitor on Windows # .\muse_plus2.ps1 build # .\muse_plus2.ps1 flash COM5 # .\muse_plus2.ps1 monitor COM5 param( [Parameter(Mandatory)][ValidateSet('build', 'flash', 'monitor')][string]$Action, [string]$Port ) # export.ps1 and CMake write progress and warnings to stderr, so don't use # $ErrorActionPreference = 'Stop' here; the exit code of idf.py decides. . C:\esp\esp-idf-v6\export.ps1 2>$null | Out-Null Set-Location C:\esp\muse-gadget-sdk\esp32 $build = 'build-muse-plus2' $common = @( '-B', $build, '-DIDF_TARGET=esp32', "-DSDKCONFIG=$build/sdkconfig", '-DSDKCONFIG_DEFAULTS=sdkconfig.defaults;devices/sdkconfig.muse;devices/sdkconfig.muse-m5stack-stickc-plus2' ) switch ($Action) { 'build' { idf.py @common build } 'flash' { if (-not $Port) { throw 'Port is required, e.g. COM5' } # The CH9102 bridge drops out above 230400 baud. idf.py @common -p $Port -b 230400 flash } 'monitor' { if (-not $Port) { throw 'Port is required, e.g. COM5' } idf.py @common -p $Port monitor } } exit $LASTEXITCODE StickC Plus2 を USB でつなぎ、デバイスマネージャーの「ポート (COM と LPT)」で USB-Enhanced-SERIAL CH9102 (COMx) の番号を確認します。私の環境では COM5 でした。 C:\esp\muse_plus2.ps1 build C:\esp\muse_plus2.ps1 flash COM5 初回のビルドは十数分かかりました。書き込みの速度は 230400 baud にしています。これより速いと USB の接続が切れる、と SDK の説明に書かれているためです。 :::note info 一度ビルドしたあとで §4-3 の設定ファイルを書き換えても、変更は反映されません。 設定ファイルの値が使われるのは、まだ決まっていない項目だけだからです。ビルドフォルダ build-muse-plus2 を消してからビルドし直すか、build-muse-plus2/sdkconfig の中の # CONFIG_MUSE_HATCH_COMPACT_BUFFERS is not set の行を CONFIG_MUSE_HATCH_COMPACT_BUFFERS=y に書き換えてください。 ::: 4-5. SDK トークンを入れる 書き込みが終わったら、§4-1 で発行した SDK トークンをマイコンに送ります。マイコンは USB のシリアル通信で >sdk_token=mgst_... という1行を受け取ると、トークンを保存して再起動します。 ログ表示（idf.py monitor）の画面から打ち込めるはずでしたが、VS Code のターミナルではキー入力がマイコンに届きませんでした。 そこで、送るだけの小さなスクリプトを書きました。トークンは画面に表示されない形で入力するので、画面にもコマンド履歴にも残りません。 # Send the Muse SDK token to the gadget over serial, without idf monitor. # python set_token.py COM5 import getpass import sys import time import serial port = sys.argv[1] if len(sys.argv) > 1 else "COM5" token = getpass.getpass("SDK token (mgst_...): ").strip() if not token.startswith("mgst_"): sys.exit("The token should start with mgst_") s = serial.Serial() s.port = port s.baudrate = 115200 s.timeout = 0.2 # Keep DTR/RTS low so opening the port doesn't reset the board. s.dtr = False s.rts = False s.open() time.sleep(0.5) s.reset_input_buffer() s.write(f">sdk_token={token}\n".encode()) buf = "" deadline = time.time() + 10 while time.time() < deadline: buf += s.read(512).decode("utf-8", "replace") for line in buf.splitlines(): if line.startswith("@sdk_token"): print(line) s.close() sys.exit(0 if " ok" in line else 1) s.close() sys.exit("No answer from the board. Is idf monitor still open on this port?") # シリアル通信用のライブラリ（pyserial）が入っている、ESP-IDF の Python で実行する C:\Users\<ユーザー名>\.espressif\python_env\idf6.0_py3.10_env\Scripts\python.exe set_token.py COM5 @sdk_token ok と表示されれば成功です。ログ表示の画面を開いたままだと送れないので、先に閉じてください。 4-6. スマホとペアリングする 1. Muse アプリで Settings > Devices > Add Device を開く 2. 一覧から MuseGadget-XXXXXX を選び、家の Wi-Fi を設定する 3. StickC Plus2 の画面に確認が出たら、正面のボタンを押す 画面に READY と出たら完成です。正面のボタンを押しながら話しかけてみてください。 一度ペアリングすれば、プログラムを書き込み直してもトークン・ペアリング・Wi-Fi の設定は残ります。実際に、設定を変えて書き込み直したあとも、ペアリングし直さずにそのまま使えました。 --- 5. つまずいた点（Windows 環境） install.ps1 が pip の設定で止まる 以前入れた NVIDIA 関連のツールの影響で、pip（Python のパッケージ管理ツール）の設定に、今はつながらない取得先 https://pypi.ngc.nvidia.com が残っていました。そのせいで、パッケージを1つ取るたびに再試行で待たされていました。 pip の設定ファイルは変えずに、インストールするときだけ環境変数で上書きしました。 $env:PIP_EXTRA_INDEX_URL = 'https://pypi.org/simple' $env:PIP_TRUSTED_HOST = 'pypi.org' C:\esp\esp-idf-v6\install.ps1 esp32 先に試した PIP_CONFIG_FILE=NUL では回避できませんでした。設定ファイルが読まれ続けて、同じ再試行が出続けました。なぜ効かないのかまでは調べていません。 PowerShell 5.1 で $ErrorActionPreference = 'Stop' を使うと、警告で止まる ESP-IDF の準備スクリプトや CMake は、進み具合や警告を stderr（エラー出力）に書き出します。Windows に標準で入っている PowerShell 5.1 は、外部コマンドが stderr に書いた行をエラーとして扱うので、Stop にしているとそこで止まります。成功したかどうかは終了コード（$LASTEXITCODE）で判断するようにしました。 初回のビルドで WinError 32（lvgl） 画面表示用のライブラリ lvgl を展開しているときに、「別のプロセスがファイルを使用中」というエラーで止まりました。ビルドの処理は残っていなかったので、展開した直後のファイルをウイルス対策ソフトが検査していたのではないかと思っていますが、確認はしていません。managed_components/lvgl__lvgl と build-muse-plus2 の2つのフォルダを消してビルドし直したら通りました。 原因を突き止めるには、ログを取る PSRAM 不足に気づけたのは、ログの psram=2K を見たからです。idf.py monitor はポートを開いた瞬間にマイコンを再起動させてしまうので、再起動させずにそのままの状態を記録するスクリプトも書きました。 # Record the gadget's serial log for a while without resetting it. # python capture_log.py COM5 60 out.log import sys import time import serial port, secs, out = sys.argv[1], float(sys.argv[2]), sys.argv[3] s = serial.Serial() s.port, s.baudrate, s.timeout = port, 115200, 0.2 s.dtr = s.rts = False # opening the port must not reset the board s.open() end = time.time() + secs with open(out, "wb") as f: while time.time() < end: f.write(s.read(4096)) f.flush() # keep what was read if the capture is stopped early s.close() 5秒ごとに出る link.heartbeat の行を見ると、接続の状態と空きメモリがわかります。 hb t=9s setup=done wifi=up ws=up raw=up ble=off int=44K/16K min=18K dma=28K/12K psram=453K ... | 表示 | 意味 | |---|---| | setup=advertising | スマホからのペアリング待ち | | setup=done wifi=up ws=up | ペアリング済みで、Muse につながっている | | psram= | 作業用メモリの空き。数KBまで減っていたらメモリ不足を疑う | --- 6. まだわかっていないこと - 設定を直している途中で一度だけ、話しかけたあとに画面が固まりました。そのときのログは取れておらず、原因はわかっていません。日本語フォントを入れてからは、今のところ起きていません --- まとめ - Meta の Muse は、10月23日から日本でも使えるようになりました。Muse Gadgets を使うと、自作の機器から話しかけられます - M5StickC Plus2 にはマイク・ボタン・画面・Wi-Fi がそろっているので、部品を買い足さずにポケットサイズの Muse 端末になります - ただし、Plus2 用の設定に CONFIG_MUSE_HATCH_COMPACT_BUFFERS=y（メモリ不足の対策）と CONFIG_MUSE_CJK_FONT=y（日本語表示）の2行を足す必要があります - LOST CONNECTION や CAN'T REACH MUSE と表示されたら、まずログの psram= を確認してください
My PR to fbjni was just merged! It speeds up all calls to Java/Kotlin in react-native (and Nitro) by 14%! 🤯🚀 #MakeReactNativeFastAgain fbjni#118
> Then hopefully you and others can organize yourselves to continue on with the work outside of the CPython repo. There's already [CinderX]( facebookincubator/cinderx ) which works well, but isn't part of Python core, requires catch up with releases; devs may not have it in mind as they improve and enhance Python, so time and effort will be spent getting new releases to work with it. > Would be nice to have an insight into what were the deciding factors for the decision to abandon this effort Yes. It's something many of us see as the last missing piece, and are looking forward to finally having it.
Got the Muse Gadget SDK work on my reTerminal Sticky Thanks to this repo: muse-gadget-sdk#214
Multiplexed Multi-Protocol Gatewaying: Bridging TCP, UDP, and QUIC Over a Single Ingress Port Cloud backends, game servers, and edge networks all want fewer exposed ports. Fewer ports mean simpler firewall rules, fewer load balancers to run, and fewer holes in strict corporate or carrier networks. A single public port such as 443 or 8443 can front gRPC and HTTP/2 over TCP, HTTP/3 over QUIC, and a custom real-time UDP protocol. The idea is easy to state and easy to get subtly wrong. This guide covers what the operating system gives you for free, how to classify TCP and UDP traffic safely, where eBPF and XDP help, how QUIC connection IDs affect routing, and what really causes port exhaustion. It ends with a small, tested Go gateway you can build on. --- 1. What “single port” really means A port number is not one namespace. TCP port 443 and UDP port 443 are separate, so a host can listen on both at once. HTTP/3 already works this way: browsers connect over TCP first, then learn about the HTTP/3 endpoint from an Alt-Svc: h3=":443" response header or an HTTPS DNS record (RFC 9460). Because that endpoint is on UDP 443, clients may need to fall back to TCP if a network filters UDP 443. So “one port” means one port number, with three problems to solve on top of it: 1. Transport split. TCP and UDP are delivered to different sockets by the kernel. 2. TCP sub-protocols. TLS (and which application protocol it carries), cleartext HTTP/1.1, cleartext HTTP/2, and custom byte streams all arrive on the same listener. 3. UDP sub-protocols. QUIC and your own datagram protocol share one UDP socket, and UDP gives you no handshake to tell them apart. ┌───────────────────────────────────────────┐ │ Public ingress, port 443 │ └───────────┬─────────────────────┬─────────┘ │ TCP │ UDP ┌─────────▼─────────┐ ┌───────▼────────────────┐ │ Peek: TLS? PRI? │ │ First-byte classifier │ │ HTTP verb? ALPN? │ │ (RFC 9443 ranges) │ └───┬─────┬─────┬───┘ └──────┬──────────┬──────┘ ▼ ▼ ▼ ▼ ▼ gRPC HTTP/1 custom QUIC / custom (h2) TCP HTTP/3 UDP (game) Where the 4-tuple stops being enough When one socket address receives several protocols, the kernel’s lookup (protocol, source address and port, destination address and port) only gets the packet to the right socket. Everything after that is your classifier’s job. | Transport | What you can inspect | Identifying marker | State | | --- | --- | --- | --- | | TCP (TLS, HTTP/2, gRPC) | First bytes of the stream | 0x16 = TLS handshake record; ALPN inside the ClientHello | Kernel connection state | | TCP (cleartext) | First bytes of the stream | PRI * HTTP/2.0… (HTTP/2 preface) or an HTTP method | Kernel connection state | | UDP (QUIC) | First byte of each datagram | High bit set = long header; low-order bits per RFC 9443 ranges | QUIC connection IDs, held by endpoints | | UDP (custom) | Whatever your protocol defines | Your own first-byte convention | Yours to define | --- 2. The socket layer: you don’t need SO_REUSEPORT A common claim is that SO_REUSEPORT is what lets a TCP socket and a UDP socket share a port. It isn’t. Linux tracks TCP and UDP bindings in separate tables, so a TCP listener and a UDP socket can bind 0.0.0.0:443 with no special options. SO_REUSEPORT solves a different problem: letting several sockets of the same protocol bind the same address and port, with the kernel distributing incoming connections or datagrams among them. By default it hashes on the 4-tuple, and all sockets in the group must belong to the same effective user ID. That is how you scale one listener across many threads or processes. In Go, the standard library is enough: ln, _ := net.Listen("tcp", "0.0.0.0:443") // TCP socket pc, _ := net.ListenPacket("udp", "0.0.0.0:443") // UDP socket, same port number Use SO_REUSEPORT when you want per-core listeners or zero-downtime restarts. Section 5 shows how an eBPF program can override its default hashing. --- 3. Classifying UDP: QUIC versus your own protocol This is the hard part. A UDP datagram arrives with no handshake, so the first byte is all you have. What QUIC guarantees (and what it doesn’t) RFC 8999 defines only the properties that stay fixed across every QUIC version: - The most significant bit of the first byte is the header form. 1 means long header, 0 means short header. - Long headers carry a 32-bit Version field (0x00000000 is reserved for Version Negotiation), then length-prefixed Destination and Source Connection IDs. - Short headers carry only the Destination Connection ID, with no length field, so a gateway has to know how long the connection IDs it expects are. The Fixed Bit (0x40) is not an invariant. RFC 8999 lists “QUIC packets always have the second bit of the first byte (0x40) set” among the incorrect assumptions that can be false for a given version. In QUIC version 1 the bit is set, but RFC 9287 lets endpoints that negotiate the grease_quic_bit transport parameter randomize it. Likewise, QUIC version 1 requires a minimum size for client Initial datagrams (1200 bytes, RFC 9000 section 14.1), but that is also a version-1 behavior, not an invariant. A short-header packet has no meaningful “minimum 3 bytes” rule you can build a classifier on. The standard multiplexing scheme: RFC 9443 RFC 9443 (which updates RFC 7983) defines how to demultiplex STUN, ZRTP, DTLS, TURN channels, RTP/RTCP, and QUIC on one UDP socket by looking at the first byte: | First byte | Protocol | | --- | --- | | 0–3 | STUN | | 4–15 | Drop | | 16–19 | ZRTP | | 20–63 | DTLS | | 64–79 | TURN channel if the source is a known TURN server, otherwise QUIC | | 80–127 | QUIC (short header, fixed bit set) | | 128–191 | RTP / RTCP | | 192–255 | QUIC (long header, fixed bit set) | Two consequences matter for a gateway: - Don’t classify with “long header OR fixed bit set.” That test, common in sketches, matches roughly three quarters of all possible first bytes. Any custom protocol whose first byte lands in 0x40–0xFF would be misrouted. RFC 9443 itself warns that a misimplemented classifier “could potentially misclassify packets.” - Endpoints behind a multiplexer must not send grease_quic_bit, because greasing breaks the first-byte scheme (RFC 9443, section 1). Designing the custom protocol so it can coexist If you control the game protocol, pick a first byte (a version or message-type field works) that stays in a range QUIC can’t occupy, such as 0–63, or 128–191 if you aren’t also carrying RTP. If you can’t change the protocol, you must accept that first-byte classification is a heuristic, add a stronger check on the long-header path (the Version field is 4 bytes at offset 1, and valid values include 0x00000001 for QUIC v1 and 0x6b3343cf for QUIC v2, RFC 9369), and treat anything ambiguous with care. A corrected decision procedure Datagram received │ ▼ byte0 in 64..127 or 192..255 ? ──no──► custom UDP pipeline │yes ▼ Long header (byte0 & 0x80)? ├─ yes ► check Version (v1, v2, or 0 for negotiation) ► QUIC └─ no ► short header: look up the DCID in your connection table ├─ hit ► same backend as the connection └─ miss ► drop, or send to QUIC backend (it may issue a stateless reset) The short-header branch is the catch. A passthrough gateway can only route short-header packets if it either remembers connection IDs (stateful) or the IDs themselves encode the routing decision (see section 5). Without one of those, it can only route by client address, which breaks when a client’s address changes. --- 4. Classifying TCP: ALPN, prefaces, and Encrypted Client Hello For TCP, the gateway peeks at the first bytes of the stream without consuming them (recv(..., MSG_PEEK) in C, bufio.Reader.Peek in Go), then forwards everything, including the peeked bytes, to the chosen backend. | First bytes | Meaning | Route | | --- | --- | --- | | 0x16 + TLS record header | TLS ClientHello | Parse ALPN (below) | | PRI (PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n) | Cleartext HTTP/2, “prior knowledge” (h2c), used by gRPC without TLS | HTTP/2 / gRPC backend | | GET , POST, HEAD, … | Cleartext HTTP/1.1 | HTTP backend | | anything else | Custom protocol | Fallback backend | ALPN routing. In TLS 1.3 the ClientHello is unencrypted, and it contains the ALPN extension (type 16) that lists the application protocols the client offers (h2, http/1.1, or custom tokens such as game-v1). The gateway reads the TLS record length from the 5-byte record header, peeks the whole record, walks the extensions, and routes on ALPN without terminating TLS. The Go implementation in section 7 does exactly this and is tested against real crypto/tls ClientHellos. The ECH caveat. RFC 9849 standardizes Encrypted Client Hello. With ECH, the client encrypts the sensitive ClientHello contents, which protects the SNI and can also protect ALPN, inside an inner ClientHello. The outer ClientHello carries innocuous values. A passthrough gateway therefore only sees what the client chose to expose in the outer hello. As ECH deployment grows, ALPN-based passthrough routing becomes less reliable, and the options are to terminate TLS at the gateway or to route on the outer name. Plan for this rather than assuming ALPN will always be visible. --- 5. Kernel-level demultiplexing: eBPF, XDP, and socket steering Peeking in user space costs a system call per decision and gives the kernel no hint about where to deliver. Two eBPF mechanisms address this, and they solve different problems. Option A: SK_REUSEPORT programs (route to the right socket) A BPF_PROG_TYPE_SK_REUSEPORT program replaces the default 4-tuple hash when picking a socket from a SO_REUSEPORT group. For UDP, the kernel patch that introduced it describes the use case directly: the program can peek into the datagram payload to learn application-level connection information, then choose the socket from a BPF map. This keeps packets in the normal kernel stack and lets you pin a QUIC connection to a specific worker by connection ID, even if the client’s address changes. Attach it with setsockopt using SO_ATTACH_REUSEPORT_EBPF (it requires CAP_SYS_ADMIN). Option B: XDP (act before the network stack) XDP runs at the driver level as soon as the NIC delivers a packet, before the kernel allocates its usual socket buffer. Meta’s open-source Katran layer-4 load balancer is built on this and describes running “right after packet has been received by network interface card … before kernel had any chance to run” in driver mode, with a generic (slower) mode as fallback. XDP can drop, pass, redirect to another interface, or redirect into an AF_XDP socket. AF_XDP matters for the design, because redirecting into it bypasses the kernel network stack: the packet lands in a user-space memory area, and you are responsible for everything the stack would have done (UDP checksums, ARP, ICMP, socket demultiplexing). The kernel documentation also specifies several behaviors that sketches often get wrong: - An XSKMAP is keyed by index, and a redirect only succeeds if the socket at that index is bound to the same device and receive queue the packet arrived on. Otherwise the packet is dropped. - If the map is empty at that index, the packet is dropped too. - The documented safe pattern is to look up the queue index first and XDP_PASS if no AF_XDP socket is bound. - If you want only certain traffic in user space, steer it to specific queues with NIC flow rules (the docs show ethtool -N … action <queue>). Here is a corrected XDP program that sends only QUIC-range datagrams on UDP port 443 to an AF_XDP socket and lets everything else (TCP, custom UDP, ARP, ICMP) continue through the kernel stack. It compiles with clang -target bpf; I have not load-tested it on a NIC. #include <linux/bpf.h> #include <linux/if_ether.h> #include <linux/in.h> #include <linux/ip.h> #include <linux/udp.h> #include <bpf/bpf_helpers.h> #include <bpf/bpf_endian.h> #define INGRESS_PORT 443 struct { __uint(type, BPF_MAP_TYPE_XSKMAP); __uint(max_entries, 64); __type(key, __u32); __type(value, __u32); } xsks_map SEC(".maps"); SEC("xdp") int xdp_quic_steer(struct xdp_md *ctx) { void *data_end = (void *)(long)ctx->data_end; void *data = (void *)(long)ctx->data; struct ethhdr *eth = data; if ((void *)(eth + 1) > data_end) return XDP_PASS; if (eth->h_proto != bpf_htons(ETH_P_IP)) return XDP_PASS; struct iphdr *iph = (void *)(eth + 1); if ((void *)(iph + 1) > data_end || iph->protocol != IPPROTO_UDP) return XDP_PASS; struct udphdr *udph = (void *)iph + iph->ihl * 4; if ((void *)(udph + 1) > data_end || udph->dest != bpf_htons(INGRESS_PORT)) return XDP_PASS; unsigned char *payload = (void *)(udph + 1); if ((void *)(payload + 1) > data_end) return XDP_PASS; /* RFC 9443 section 3: first byte 64..127 or 192..255 is QUIC * (64..79 is ambiguous only for traffic from a TURN server; not handled). */ unsigned char b0 = *payload; if (!((b0 >= 64 && b0 <= 127) || b0 >= 192)) return XDP_PASS; /* custom UDP: leave it to the kernel stack */ __u32 q = ctx->rx_queue_index; if (bpf_map_lookup_elem(&xsks_map, &q)) return bpf_redirect_map(&xsks_map, q, 0); return XDP_PASS; /* no AF_XDP socket on this queue */ } char _license[] SEC("license") = "GPL"; The program as written handles IPv4 without VLAN tags or IP fragmentation. A production version needs IPv6 and VLAN handling, and a decision about fragments. Routing QUIC by connection ID across hosts At scale, the question is how to route short-header packets without per-flow state. Two real approaches exist: - Server-ID routing. Meta described a stateless scheme at the Linux Plumbers Conference in 2021 where a server ID encoded into the identifiers lets the XDP load balancer route both TCP and QUIC without sharing state. - QUIC-LB. The QUIC working group’s draft “QUIC-LB: Generating Routable QUIC Connection IDs” defines a standardized way to encode routing information in server-chosen connection IDs. Note its status: as of this writing it is an expired Internet-Draft (latest revision 21, August 2025), not an RFC. It is a design reference, not a standard to claim compliance with. --- 6. Tunneling several protocols through one edge connection An edge tunnel (a persistent outbound connection from a private network to a public edge) can carry all three protocol classes over a single session, so the private side needs no inbound ports. PUBLIC EDGE (port 443) TCP streams ──┐ QUIC datagrams┼──► classifier ──► tunnel multiplexer ══ one outer session ══► UDP datagrams ┘ │ ▼ PRIVATE SIDE: demux by channel ► gRPC :50051 ► HTTP/3 :9001 ► game :7777 Use the right primitive for each traffic type - TCP and other byte streams map naturally onto streams (one multiplexed stream per accepted connection). Don’t describe them as “segments”; the tunnel carries a byte stream, not TCP segments. - UDP and QUIC are datagrams. Pushing them through a single reliable, ordered stream reintroduces head-of-line blocking and retransmission delay that real-time traffic is designed to avoid. Carry them in unreliable datagrams instead. Existing standards cover this: QUIC DATAGRAM frames (RFC 9221), HTTP Datagrams (RFC 9297), and CONNECT-UDP (RFC 9298, “Proxying UDP in HTTP”), which proxies UDP over HTTP/2 or HTTP/3 and can use QUIC datagrams when running over HTTP/3. If you can use one of these, you inherit interoperable framing and proxy support. - WireGuard is a layer-3 tunnel that carries IP packets over UDP, so it fits as a transport for raw packets, not as a “stream” substrate. An illustrative custom frame If you build your own framing, keep it small and explicit. This layout is an example design, not a standard: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Magic (0xA5) | Protocol Type | Channel ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Session Token | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Payload Length | Payload Data ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field | Size | Purpose | | --- | --- | --- | | Magic | 8 bits | Frame-boundary sanity check (0xA5) | | Protocol Type | 8 bits | 0x01 stream data, 0x02 UDP datagram, 0x03 QUIC datagram | | Channel ID | 16 bits | Which backend service or tenant | | Session Token | 32 bits | Per-client identifier for state lookup | | Payload Length | 16 bits | Up to 65,535 bytes, enough for the largest UDP payload | A 32-bit token is a lookup key, not a credential. Anything that authenticates a tunnel client must come from a real handshake (TLS, a signed token, or WireGuard keys), never from a guessable counter. --- 7. Port exhaustion, session state, and what really limits you “Port exhaustion” gets blamed for many things. Here is what actually applies. The ephemeral port limit is per destination. On Linux the default outbound range is 32768–60999, about 28,232 ports. A connection is identified by its 4-tuple, so the same source IP and port can be reused for connections to different destinations. The practical limit appears when a gateway opens many connections from one source IP to a single backend IP and port: that tops out near the size of the range. Mitigations, roughly in order of preference: 1. Multiplex instead of one socket per client. Carry many client streams over a few long-lived backend connections (HTTP/2 or your own framing). Cloudflare’s write-up on ephemeral ports makes the same point: long-lived, multiplexed connections beat connection churn. 2. Spread across more 4-tuples. Use multiple backend addresses or ports, or multiple source IPs. 3. Widen or partition the range. Raise the range via net.ipv4.ip_local_port_range; newer kernels also offer a per-socket IP_LOCAL_PORT_RANGE option for carving up the range between components. Session tables. For connectionless traffic, the gateway needs a table that maps a client (or a connection ID) to its upstream path, with an idle timeout so entries expire. Keep it bounded and reap aggressively; an unbounded table is its own denial-of-service risk. TUN devices. Injecting raw IP packets through a TUN interface (the clone device is /dev/net/tun; the interface it creates is named something like tun0) forwards at layer 3, so your process allocates no per-client ephemeral ports. That is not the same as “bypassing OS port limits”: if the kernel then performs NAT or masquerading to reach the backend, the NAT still allocates source ports. Use TUN when you can route end to end without NAT. QUIC connection migration. QUIC separates a connection from the client’s IP and port using connection IDs (RFC 9000, section 5.1). Two details change how a gateway has to behave: - A migrating client switches to a new connection ID, not the same one on both paths. RFC 9000 requires connection IDs not to be linkable by observers and tells endpoints to limit each ID to one pair of addresses (sections 5.1 and 9.5). A gateway that tracks “this one CID maps to this backend” will therefore lose the connection when the client moves networks. - A passthrough gateway cannot read the new IDs, because NEW_CONNECTION_ID frames travel inside encrypted packets. The fix is for the server to issue routable IDs (server-ID routing or QUIC-LB, section 5), or for the gateway to terminate QUIC itself. RFC 9000 section 5.2.3 describes the fallback for simple address-based balancers: servers can send the disable_active_migration transport parameter, or use preferred_address to move clients to a dedicated address. Also note that zero-length connection IDs are fragile behind a shared address: RFC 9000 forbids using one IP and port for multiple concurrent zero-length-CID connections unless migration and NAT rebinding are ruled out. --- 8. A tested reference gateway in Go The following program implements the design above using only the standard library: - It listens on one port number for TCP and UDP. - For TCP it peeks the first bytes, parses the TLS ClientHello to read ALPN (h2 or http/1.1), recognizes the cleartext HTTP/2 preface and HTTP methods, and routes everything else to a fallback backend. Peeked bytes are replayed to the backend. - For UDP it classifies each datagram with the RFC 9443 ranges and keeps one upstream socket per client and backend pair, with idle expiry, relaying replies back through the listening socket. I compiled it, ran go vet, and tested it against local echo backends. The tests cover plaintext routing (PRI , GET , unknown bytes), UDP routing across the range boundaries (0xC0, 0x45 to QUIC; 0x05, 0x3F, 0x90 to the custom backend), and the ALPN parser against real crypto/tls ClientHellos (h2, http/1.1, an unknown protocol, and no ALPN). // Single-port TCP+UDP gateway: TCP routed by TLS ALPN / HTTP preface, // UDP routed by RFC 9443 first-byte ranges. Standard library only. package main import ( "bufio" "context" "io" "log" "net" "os" "os/signal" "sync" "syscall" "time" ) type Config struct { Listen string H2Backend string // gRPC / HTTP/2 HTTPBackend string // HTTP/1.1 TCPFallback string // anything else (custom TCP protocols) QUICBackend string // UDP, QUIC / HTTP/3 GameBackend string // UDP, custom protocol UDPIdle time.Duration } func main() { cfg := Config{ Listen: "0.0.0.0:8443", H2Backend: "127.0.0.1:50051", HTTPBackend: "127.0.0.1:8080", TCPFallback: "127.0.0.1:9000", QUICBackend: "127.0.0.1:9001", GameBackend: "127.0.0.1:7777", UDPIdle: 60 * time.Second, } ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM) defer stop() go serveTCP(ctx, cfg) go serveUDP(ctx, cfg) <-ctx.Done() } // ---------- TCP ---------- func serveTCP(ctx context.Context, cfg Config) { ln, err := net.Listen("tcp", cfg.Listen) if err != nil { log.Fatalf("tcp listen: %v", err) } go func() { <-ctx.Done(); ln.Close() }() for { c, err := ln.Accept() if err != nil { return } go handleTCP(c, cfg) } } func handleTCP(c net.Conn, cfg Config) { defer c.Close() br := bufio.NewReaderSize(c, 16*1024+5) c.SetReadDeadline(time.Now().Add(5 * time.Second)) // sniffing deadline backend := cfg.TCPFallback if hdr, err := br.Peek(5); err == nil { switch { case hdr[0] == 0x16: // TLS handshake record recLen := int(hdr[3])<<8 | int(hdr[4]) if rec, err := br.Peek(5 + recLen); err == nil { switch alpnPick(rec[5:]) { case "h2": backend = cfg.H2Backend case "http/1.1": backend = cfg.HTTPBackend } } case string(hdr[:4]) == "PRI ": // cleartext HTTP/2 prior knowledge backend = cfg.H2Backend case isHTTPVerb(hdr): backend = cfg.HTTPBackend } } c.SetReadDeadline(time.Time{}) up, err := net.DialTimeout("tcp", backend, 3*time.Second) if err != nil { log.Printf("dial %s: %v", backend, err) return } defer up.Close() var wg sync.WaitGroup wg.Add(2) go func() { defer wg.Done(); io.Copy(up, br); closeWrite(up) }() // br replays the sniffed bytes go func() { defer wg.Done(); io.Copy(c, up); closeWrite(c) }() wg.Wait() } func closeWrite(c net.Conn) { if cw, ok := c.(interface{ CloseWrite() error }); ok { cw.CloseWrite() } } func isHTTPVerb(b []byte) bool { for _, v := range []string{"GET ", "POST", "HEAD", "PUT ", "DELE", "OPTI", "PATC"} { if string(b[:4]) == v { return true } } return false } // alpnPick returns the first ALPN protocol the client prefers among h2/http/1.1, // or "" if the ClientHello has no ALPN extension (or cannot be parsed). func alpnPick(rec []byte) string { p := rec adv := func(n int) []byte { if len(p) < n { p = nil return nil } out := p[:n] p = p[n:] return out } if h := adv(4); h == nil || h[0] != 0x01 { // handshake type ClientHello + 3-byte length return "" } adv(2 + 32) // legacy_version + random if l := adv(1); l != nil { adv(int(l[0])) // session id } if l := adv(2); l != nil { adv(int(l[0])<<8 | int(l[1])) // cipher suites } if l := adv(1); l != nil { adv(int(l[0])) // compression methods } l := adv(2) if l == nil { return "" } exts := adv(int(l[0])<<8 | int(l[1])) for len(exts) >= 4 { typ := int(exts[0])<<8 | int(exts[1]) n := int(exts[2])<<8 | int(exts[3]) if len(exts) < 4+n { return "" } body := exts[4 : 4+n] exts = exts[4+n:] if typ != 16 || len(body) < 2 { // application_layer_protocol_negotiation continue } list := body[2:] for len(list) > 0 { k := int(list[0]) if len(list) < 1+k { return "" } switch s := string(list[1 : 1+k]); s { case "h2", "http/1.1": return s } list = list[1+k:] } } return "" } // ---------- UDP ---------- // classifyUDP follows RFC 9443 section 3: first byte 80..127 or 192..255 is QUIC; // 64..79 is QUIC unless it comes from a TURN server (not handled here). // Keep your own protocol's first byte in 0..63 or 128..191. func classifyUDP(first byte) (isQUIC bool) { return (first >= 64 && first <= 127) || first >= 192 } type udpSession struct { up *net.UDPConn seen time.Time } func serveUDP(ctx context.Context, cfg Config) { pc, err := net.ListenPacket("udp", cfg.Listen) if err != nil { log.Fatalf("udp listen: %v", err) } conn := pc.(*net.UDPConn) go func() { <-ctx.Done(); conn.Close() }() quicAddr, _ := net.ResolveUDPAddr("udp", cfg.QUICBackend) gameAddr, _ := net.ResolveUDPAddr("udp", cfg.GameBackend) var mu sync.Mutex sessions := map[string]*udpSession{} // key: client addr + backend go func() { // idle reaper t := time.NewTicker(cfg.UDPIdle / 4) defer t.Stop() for range t.C { mu.Lock() for k, s := range sessions { if time.Since(s.seen) > cfg.UDPIdle { s.up.Close() delete(sessions, k) } } mu.Unlock() } }() buf := make([]byte, 65535) for { n, client, err := conn.ReadFromUDP(buf) if err != nil { return } if n == 0 { continue } target := gameAddr if classifyUDP(buf[0]) { target = quicAddr } key := client.String() + ">" + target.String() mu.Lock() s := sessions[key] if s == nil { up, err := net.DialUDP("udp", nil, target) if err != nil { mu.Unlock() continue } s = &udpSession{up: up} sessions[key] = s go func(up *net.UDPConn, client *net.UDPAddr) { // backend -> client b := make([]byte, 65535) for { m, err := up.Read(b) if err != nil { return } conn.WriteToUDP(b[:m], client) } }(up, client) } s.seen = time.Now() mu.Unlock() s.up.Write(buf[:n]) } } What this gateway deliberately does not do It is a foundation, not a finished product. Before exposing it to the internet, plan for the following: - Client address preservation. Backends see the gateway’s address. Use the PROXY protocol for TCP, or carry the client address in your tunnel framing. - QUIC connection-ID awareness. Sessions are keyed by client address, so a client that migrates networks gets a new upstream socket. Combine it with server-issued routable connection IDs (section 5) if you need migration to survive. - One upstream socket per UDP client. That is simple and correct, but each socket uses an ephemeral port, so it inherits the limits in section 7. Cap session counts and consider the multiplexed designs there. - Abuse controls. There are no rate limits, connection limits, or amplification protection. QUIC servers must limit what they send to an unvalidated address to three times what they received (RFC 9000, section 8), and a gateway that relays replies to spoofed sources can be abused the same way. - Slowloris-style sniffing. The 5-second sniff deadline bounds idle connections, but you should also cap concurrent unclassified connections. - Fail-closed behavior. Unknown or malformed first bytes currently fall through to the fallback backend. Decide whether your deployment should drop them instead. --- 9. Measuring the benefit instead of assuming it Consolidating ports is mainly an operational win: fewer firewall rules, fewer listeners, one place to enforce policy. Performance gains are workload-specific, so measure before you claim any. Published or invented before-and-after numbers for “single-port versus multi-port” don’t transfer between environments, and the main mechanisms people cite need care: - Handshake latency. A single port does not shorten a TLS or QUIC handshake, and ALPN is carried inside the ClientHello, so it adds no round trip regardless of port count. QUIC 0-RTT applies only to resumed connections, and RFC 9000 warns it provides no protection against replay attacks, so it is not a general latency win and not safe for non-idempotent requests. - Per-packet cost. User-space classification costs a system call or two per datagram. Kernel-side classification (SK_REUSEPORT or XDP) removes that cost for the traffic it handles. Batched reads (recvmmsg, and UDP GRO and GSO on supporting kernels) cut it further without eBPF. XDP with AF_XDP trades kernel-stack features for speed, so it pays off at high packet rates, not at typical game-server loads. - What to measure. Track connection setup time at p50/p99, packets per second per core at a fixed loss target, CPU per Gbit/s, sockets and ephemeral ports in use (ss -s), and drops at each stage (NIC, socket buffers, your session table). Test with your real client mix, including QUIC clients that migrate and clients behind carrier NAT. --- 10. Implementation checklist - Sockets. Bind TCP and UDP to the same port number with ordinary listeners; add SO_REUSEPORT only for per-core scaling of one protocol. - UDP classification. Use the RFC 9443 first-byte ranges, never “long header or fixed bit.” Keep your own protocol’s first byte out of the QUIC ranges, and don’t enable grease_quic_bit on endpoints behind a first-byte multiplexer. - QUIC routing. Decide up front how short-header packets find their backend: server-issued routable connection IDs (server-ID routing, QUIC-LB), a connection table, or terminating QUIC at the gateway. Without one of these, migration breaks. - TCP classification. Peek, don’t consume; parse ALPN from the ClientHello; handle the PRI preface and HTTP methods; have a plan for ECH. - Kernel acceleration. Reach for SK_REUSEPORT eBPF first (it keeps the normal stack); use XDP and AF_XDP only when you need driver-level speed and can own the user-space stack. - Tunnels. Carry streams as streams and datagrams as datagrams (RFC 9221, RFC 9297, RFC 9298), and authenticate tunnel clients properly. - Port budgets. Count ephemeral ports per backend 4-tuple, not per host, and multiplex before you widen ranges. - Hardening. Bound every table, time out every session, rate-limit unvalidated UDP sources, and decide what happens to traffic you can’t classify. --- References - RFC 8999: Version-Independent Properties of QUIC - RFC 9000: QUIC: A UDP-Based Multiplexed and Secure Transport - RFC 9443: Multiplexing Scheme Updates for QUIC - RFC 9849: TLS Encrypted Client Hello - RFC 9298: Proxying UDP in HTTP - AF_XDP, The Linux Kernel documentation - BPF_PROG_TYPE_SK_REUSEPORT, eBPF documentation - QUIC-LB: Generating Routable QUIC Connection IDs (Internet-Draft) - Katran, Meta’s XDP-based layer 4 load balancer - From XDP to Socket, Linux Plumbers Conference 2021 - Cloudflare: How to stop running out of ephemeral ports - Peakhour: How browsers decide to use HTTP/3 --- Originally published at instatunnel.my