openpi

openpi ⭐ 14k Openpi provides pre-trained robotics models trained on 10,000 hours of data for developers building robot control systems. 🔗 Physical-Intelligence/openpi #opensource t.co

【10月13日β公開】ロボットの脳はクラウドに置けるか？Physical Intelligenceの論文値で試算したら、日米の往復を足して約208msで、学習時に想定した上限200msの境目だった。推論の置き場所3択と、今すぐ試せる代替5選 ロボットの「脳」をクラウドに置こうとしているなら、申込フォームを出す前に読んでほしい。 Hugging Faceのロボティクス部門には今、3万6,917個のモデルが並んでいる（10月20日に筆者が確認）。手元で動かせるロボット用のAIは、すでにこれだけ出回っている。 - 名前に「smolvla」を含むモデルが6,102個 - 名前に「pi05」（Physical Intelligenceのπ0.5）を含むモデルが3,520個 - 2つを合わせると26.1%。4つに1つを超える ところが、そのπ0.5を作ったPhysical Intelligence（以下PI）は、10月13日に逆の道を選んだ。最新モデルを、クラウドのAPIで開いたのだ。発表の書き出しはこうだ。 We are making our latest models available for training and inference through the Pi API beta. 自分のロボットのデータを上げ、PIが管理する計算資源で微調整し、ホスト型のゲートウェイ越しに推論する。事前に使った会社は日本、スイス、米国各地に広がり、ユタ州では運転席に誰もいないホイールローダーが、資材をすくってダンプに積んでいる。発表によれば、公開した事例の大半がクラウドで推論していた。そして一部では通信の問題が起きたことも、PI自身が認めている。 そのAPIの申込フォームには、こんな質問がある。 Where do you need to run inference? Cloud / Local (on your own hardware) / Both / Not sure yet ロボットの脳を、どこに置くか。この1問に、数字で答えられるだろうか。 :::note alert 結論から言うと 1. 「Cloud」を選ぶ前に、自分のロボットからGPUまでの遅延を測り、行動の何ステップ分にあたるかを計算すること。 計算は d = ⌊遅延 ÷ 制御周期⌋ だけだ。PIの論文の設定（50Hz、1回の推論で50ステップ＝1秒分）なら、遅延200msで10ステップになる。 2. PIの論文値で試算すると、日米の往復を足した遅延は約208ms。PIが新しい学習方式で想定した上限200msの、ちょうど境目だ。 遠隔のH100で測った108msに、Azureの東日本から米国西部への往復100ms（中央値）を足した筆者の計算だ。一方、古い方式は+200msの遅延を足した実験でも劣化しなかった。 3. ただし怖いのは、平均の遅さより途切れることだ。1回の推論で手に入る行動は1秒分しかない。 しかもAPIの価格、GPUの場所、どちらの方式を使っているかは、どれも公表されていない。 ::: !半円のメーター図。目盛りは0〜300ms。論文の実測108msを青、日米の往復100msをオレンジで積み上げ、合計約208msが学習時に想定した上限200msの線をわずかに越えている。右側には、GPUの場所ごとの試算として国内119ms（d＝5、余裕）、米国西部208ms（d＝10、境目）、米国東部270ms（d＝13、想定の外）を並べている 図: PIの学習時RTCの論文の実測108msに、日米の往復100msを足した試算。GPUの場所ごとの遅延と合わせて筆者が作図 :::note info 数字は2026年10月20日時点。APIの内容は、PIの発表（10月13日）と申込フォームによる。遅延の実測値はPIの2本の論文（2025年6月の推論時RTC、2025年12月の学習時RTC）から、往復時間はMicrosoftが公表しているAzureのリージョン間の中央値（2026年7月30日までの30日間）から借りた。APIで使われるモデルの名前、行動チャンクの長さ、どちらのRTCを使うか、GPUの場所、価格はいずれも公表されておらず、試算はすべて筆者の単純な足し算だ。筆者はPIのAPIもロボットも動かしていない。 ::: 数字で見るPi APIとロボットの遅延 | 項目 | 値 | | --- | --- | | Pi API β | 2026年10月13日公開。少数の会社から順番に受け入れ | | 使い方 | データを上げる → PIの計算資源で微調整 → ホスト型ゲートウェイで推論 | | 公開事例の推論 | 大半がクラウド（日本、スイス、米国各地） | | モデル名・価格・GPUの場所 | 非公開 | | 制御周期と行動チャンク | 50Hz（1ステップ20ms）、1回の推論で50ステップ＝1秒分（π0.5の論文設定） | | LAN越しの総遅延（π0.5、RTX 4090） | 移動ロボット139ms、据え置き109ms | | 遠隔H100での総遅延（π0.6、学習時RTC） | 108ms | | 学習時RTCが想定する遅延 | 最大200ms（0〜10ステップ） | | 東日本→米国西部2の往復（Azure） | 100ms（中央値） | | Hugging Faceのロボティクス部門 | 3万6,917モデル（smolvla 6,102、pi05 3,520） | 同じ建物の中のGPUにつないでも、移動ロボットは推論1回に139msかかっている。50Hzなら139÷20＝6.95で、切り捨てて6ステップ。新しい指令が届くまでの間、ロボットは前の計画のまま6歩ぶん動いている。 ここまでは「ロボットのAIもAPIで借りられるようになった」という話。面白いのはその先だ。 なぜロボットは「1秒先」までまとめて動くのか 50Hzの世界で、VLAの推論は間に合わない VLA（Vision-Language-Action）は、カメラの画像と指示文を受け取り、関節の動きを出力するモデルだ。PIのπ0.5は、1回の推論で50ステップ分の行動をまとめて出す。これを行動チャンクと呼ぶ。 理由は単純だ。π0.5の実験で使った制御周期は50Hz、つまり20msごとに次の指令が要る。ところがRTCの論文によると、RTX 4090で推論すると、モデル本体だけで76msかかる。1ステップごとに推論していたら間に合わない。 だから1秒分をまとめて出し、その間に次の1秒分を考える。問題はつなぎ目だ。 | つなぎ方 | 何が起きるか | | --- | --- | | 同期（使い切ってから次を推論） | 推論のたびにロボットが止まる。遅延が増えるほど作業が遅くなる | | 重なりを平均する（temporal ensembling） | +100msの遅延で振動し、ロボットの保護停止が作動して動けなかった | | そのまま切り替える | 新旧のチャンクが食い違い、急な加速やカクつきが出る | 平均でつなぐ方式について、論文はこう書いている。 > both TE variants do not run at all due to causing such high oscillations that the robot’s protective stop is triggered RTCは「もう変えられない未来」を固定して、残りを描き直す PIが2025年6月に出したRTC（Real-Time Chunking）は、このつなぎ目を「穴埋め」の問題として解いた。 推論にかかる時間をδ、制御周期をΔtとすると、論文は遅延をステップ数で定義する。 d = ⌊δ / Δt⌋ （次のチャンクが届くまでに、前のチャンクのまま実行してしまうステップ数） 先頭のdステップは、新しいチャンクが届く前に必ず実行されるので、もう変えられない。RTCはここを前のチャンクの値に固定し、残りを新しい観測に合わせて描き直す。重みは固定部分で1、その先で1から0へ指数的に減らし、チャンクの後ろ側は0（自由に描き直す）にする。 もう1つ大事な制約がある。1本のチャンクから実行するステップ数をs、チャンクの長さをHとすると、非同期で回すには d ≤ s ≤ H − d が必要だ。ここから d ≤ H/2 が出る。H＝50なら d ≤ 25、遅延にして520ms未満が理論上の天井になる（論文の制約からの筆者の計算）。 # 推論時RTCの流れ（論文の説明を筆者が簡略化した擬似コード・未実行） H, DT = 50, 0.020 # 1チャンク50ステップ、制御周期20ms（50Hz） chunk = policy.sample(robot.observe()) while True: obs = robot.observe() d = floor(p99_latency() / DT) # 推論の間に進んでしまうステップ数 future = start_async( # 別スレッドで次のチャンクを推論 policy.inpaint, obs, frozen=chunk[:d], # 必ず実行される部分は固定 weights=soft_mask(d, H)) # 1から0へ指数的に減る重み execute(chunk[:d]) # 待つ間も、前のチャンクで動き続ける chunk = future.result()[d:] # 届いたら、まだ実行していない部分から使う 実機の結果はこうだ。6種類の作業で、作業と方式の組ごとに10回ずつ試し、計480エピソード（約28時間）を比べた。+100ms（d≈11）、+200ms（d≈16）の遅延を足しても、RTCの作業効率は落ちなかった。 > RTC is completely robust to injected delay, showing no degradation 同期方式は、遅延に比例して効率が落ちた。PIの解説ページは、マッチを擦る、イーサネットケーブルを挿すといった精密な作業を、300msを超える遅延で完了したとも書いている。 この実験の遅延の内訳も公開されている。モデルは別のRTX 4090マシンで動かし、ロボットとはLANでつないでいる。 | 内訳 | 移動ロボット | 据え置き | | --- | --- | --- | | モデル（RTC込み） | 96.89ms | 97.43ms | | 通信（LAN） | 21.20ms | 6.89ms | | 画像の縮小（ロボット側のCPU） | 11.22ms | 1.44ms | | その他 | 9.67ms | 3.00ms | | 合計 | 138.98ms | 108.76ms | 移動ロボット側の計算機はIntel NUC、据え置き側はRyzen 9 7950Xのワークステーションだ。同じ建物の中でも、ロボット側の通信と計算機の差だけで約30ms（138.98 − 108.76 ＝ 30.22ms）開く。 新しい方式は「遅延の上限」を学習の時点で決める 2025年12月、PIは学習時RTCを出した。推論のたびに穴埋めの計算（逆伝播を含む）をする代わりに、学習中に遅延を模擬し、先頭の行動を条件として与えておく。推論時の追加の計算はなくなる。 ただし、模擬する遅延の幅は学習の時点で決める。論文の実機実験の設定はこうだ。 > We sample delays uniformly between 0 and 10 during training, which supports a maximum latency of 200ms on a 50Hz robot. 論文は、この方式の弱点もはっきり書いている。 > However, training-time RTC is fundamentally less flexible than inference-time RTC | 項目 | 推論時RTC（2025年6月） | 学習時RTC（2025年12月） | | --- | --- | --- | | 仕組み | 推論中に先頭dステップを固定し、残りを描き直す | 学習中に遅延を模擬し、先頭の行動を条件に与える | | 推論時の追加計算 | あり（π0.5で76ms→97ms） | なし | | 想定する遅延 | 理論上は d ≤ 25（H＝50）。+200ms（d≈16）まで劣化なし | 学習で模擬した0〜10ステップ（最大200ms） | | 実験のモデルと総遅延 | π0.5、LAN越しのRTX 4090で139ms／109ms | π0.6、遠隔のH100で108ms（d≈5） | | 想定より遅延が増えたら | 制約の範囲ならそのまま使える | 論文は挙動を報告していない | π0.6の実験では、推論時RTCも135ms（d≈7）で動かし、2つの方式の作業成績は同程度だった。差が出るとすれば、遅延が想定を超えたときだ。 新しい方式ほど、遅延の上限を最初に決め打ちする。 クラウドに脳を置くなら、ここが効いてくる。 GPUを海の向こうに置いたら何msか（試算） PIは、APIのGPUがどこにあるかを公表していない。ここからは仮定の計算だ。 材料は2つある。学習時RTCの論文で遠隔のH100を使ったときの実測108msと、Azureの公表値にあるリージョン間の往復時間（2026年7月30日までの30日間の中央値）だ。108msのうち通信が何msかは論文に書かれていないので、GPUが論文と同じくらい近い状態から、遠くなった分の往復をそのまま足す。 | GPUの場所（仮定） | 足す往復 | 合計 | d（50Hz） | 学習時RTCの想定（0〜10） | 推論時RTCの実験（+200msまで劣化なし） | | --- | --- | --- | --- | --- | --- | | 論文と同じ近さ | 0ms | 108ms | 5 | 内側 | 内側 | | 国内の別リージョン（東日本→西日本） | 11ms | 119ms | 5 | 内側 | 内側 | | 米国西部（東日本→米国西部2） | 100ms | 208ms | 10 | 境目 | 内側 | | 米国東部（東日本→米国東部） | 162ms | 270ms | 13 | 外側 | 内側 | 日米の往復を足すと208ms。論文が書く「最大200ms」を8ms超えるが、ステップに直すと208÷20＝10.4で、切り捨ててd＝10。学習で模擬した0〜10の、ちょうど端に乗る。 移動ロボットならさらに厳しい。前の表で、移動ロボットは据え置きより約30ms遅かった。同じ差が乗れば238msでd＝11、学習の範囲の外に出る計算だ。 一方、推論時RTCの実験は+200msまで劣化がなかったので、米国東部の往復162msでもその範囲に収まる。同じ往復時間でも、方式によって「余裕」と「想定の外」に分かれる。 APIでどちらが使われているかは、公表されていない。 ロボットの中に置く側の数字もある。NVIDIAのモデルカードによると、GR00T N1.7の推論は、データセンター向けのH100で27.9ms、ロボット向けのJetson AGX Thorで93.8msだ（どちらもTensorRT、ノイズ除去4ステップ、カメラ1台）。差は65.9ms。このモデルなら、往復が約66msより長くなるクラウドのH100より、ロボットに載せたThorのほうが早く答えを返す（画像の転送時間などを含まない筆者の計算）。日米の往復100msは、この差を超えている。 正直に言えば、この試算はPIの研究用の構成の値とAzureのリージョン間の中央値を足しただけで、APIのモデル、チャンクの長さ、RTCの方式、GPUの場所はどれも公表されていないので、本当の値はAPIの上で測るしかない。 :::note info あなたのロボットからGPUまでの往復は、何msだった？ 構成（有線か無線か、どこのクラウドか）と一緒に、コメントで教えてほしい。 ::: APIを待つ間に試すべき3つのこと APIは順番待ちだ。だが、遅延の測り方とRTCの効き方は、今日、手元で確かめられる。 1. ロボットなしで、RTCが何をしているか見る LeRobotにはRTCが組み込まれていて、データセットだけで効き方を図にできる。ドキュメントにあるコマンドはこうだ（CUDAのGPUが要る）。 # LeRobotのリポジトリを取得したディレクトリで pip install -e ".[pi]" python examples/rtc/eval_dataset.py \ --policy.path=lerobot/pi0_libero_finetuned \ --dataset.repo_id=HuggingFaceVLA/libero \ --rtc.execution_horizon=10 \ --rtc.max_guidance_weight=10.0 \ --device=cuda 通常の生成とRTCの生成が左右に並び、新しいチャンクの先頭が、前のチャンクの終わりに沿わされていく様子が見える。execution_horizonは前のチャンクと合わせるステップ数で、ドキュメントの目安は8〜12だ。 2. openpiで遠隔推論を組み、自分の遅延を分布で測る openpiは、モデルを別のサーバーで動かし、WebSocketで行動を送る構成を最初から持っている。リモート推論の説明とsimple_clientを使えば、ロボットがなくても往復の時間を測れる。 # GPUマシン（openpiはUbuntu 22.04で検証。推論にはVRAM 8GB超） git clone --recurse-submodules git@github.com:Physical-Intelligence/openpi.git cd openpi GIT_LFS_SKIP_SMUDGE=1 uv sync GIT_LFS_SKIP_SMUDGE=1 uv pip install -e . uv run scripts/serve_policy.py --env DROID # 測る側のPC（同じリポジトリを用意する）。アドレスは例 uv run examples/simple_client/main.py --env DROID --host 192.168.0.10 simple_clientは、クライアント側で測った1回あたりの時間（client_infer_ms）を、平均、P50、P95、P99で表示する。見るべきは平均ではなくP99だ。 それを20msで割って切り捨てた数が、あなたの構成のdになる。 # 遅延予算の計算（筆者作成・未実行。値は本文の試算と同じ） from math import floor def delay_steps(latency_ms: float, hz: int = 50) -> int: """観測を送ってから次のチャンクが届くまでに進むステップ数 d""" return floor(latency_ms / (1000 / hz)) H = 50 # 1チャンクのステップ数（π0.5の論文設定） TRAINED_MAX = 10 # 学習時RTCが模擬した遅延の上限（論文の実機設定） for name, ms in [("論文と同じ", 108), ("国内", 119), ("米国西部", 208), ("米国東部", 270)]: d = delay_steps(ms) print(name, ms, "ms -> d =", d, "| 学習時RTCの範囲内:", d <= TRAINED_MAX, "| 理論上の天井内:", d <= H // 2) 手計算では、dは順に5、5、10、13になり、学習時RTCの範囲を外れるのは米国東部だけだ（米国西部は端のd＝10）。 3. SmolVLAを、別のマシンから非同期で動かす LeRobotの非同期推論は、推論サーバー（PolicyServer）とロボット側のクライアント（RobotClient）を分ける。ロボットは手持ちの行動を実行しながら、残りが閾値を切ったら次の観測を送る。ドキュメントのコマンドを、LAN内のGPUマシンにつなぐ形に書き換えるとこうなる（アドレス、ポート、ロボットとカメラの設定は例。筆者は実行していない）。 # GPUマシン側 python -m lerobot.async_inference.policy_server \ --host=192.168.0.10 \ --port=8080 # ロボット側（SO-100の例。カメラのキーはポリシーが期待する名前に合わせる） python -m lerobot.async_inference.robot_client \ --server_address=192.168.0.10:8080 \ --robot.type=so100_follower \ --robot.port=/dev/tty.usbmodem585A0076841 \ --robot.id=follower_so100 \ --robot.cameras="{ laptop: {type: opencv, index_or_path: 0, width: 1920, height: 1080, fps: 30}}" \ --task="dummy" \ --policy_type=smolvla \ --pretrained_name_or_path=lerobot/smolvla_base \ --policy_device=cuda \ --actions_per_chunk=50 \ --chunk_size_threshold=0.5 \ --aggregate_fn_name=weighted_average \ --debug_visualize_queue_size=True 効くパラメータは2つだ。 - actions_per_chunk：1回に出す行動の数。既定は50で、ドキュメントの典型値は10〜50 - chunk_size_threshold：手持ちの行動がこの割合を切ったら、次の観測を送る。0に近いと同期と同じになり、1に近いと毎ステップ送る。ドキュメントは0.5〜0.6がうまくいったとしている --debug_visualize_queue_size=Trueを付けると、手持ちの行動の残量の推移を描ける。残量が0に張り付くなら、推論の遅さに対して制御が速すぎる。 ドキュメントも、行動を使い切るならfpsを下げるよう勧めている。推論時のメモリは、π0の14GBに対してSmolVLAは約2GBで済む。 知らないと損する5つの罠 罠1：平均の遅延で予算を組む PIの測定でも、通信と画像の縮小はばらついていた（50回の推論の平均±標準偏差）。 Network 21.20 ± 3.12ms (mobile) Image resize 11.22 ± 5.00ms (mobile) Azureの往復時間も中央値で、半分の時間はそれより遅い。dは平均ではなく、遅いほうの裾で決まる。学習時RTCなら、dが10を一瞬でも超えた瞬間、モデルは学習で見ていない状況に置かれる。 対策：simple_clientのP99（少なくともP95）でdを計算すること。 罠2：非同期にすれば、つなぎ目の問題も消えると思う 非同期推論は、推論の間にロボットが止まる問題を消す。だが、新旧のチャンクの食い違いは別の問題だ。LeRobotのRTCのドキュメントは、非同期推論は「待ち時間」を、RTCは「チャンクの不連続」を解くものだと整理している。そして重なりを平均するだけの方式は、PIの実験で+100msの遅延のときに保護停止を起こした。 対策：遅延が大きい構成では、非同期推論にRTCを重ねること。LeRobotなら実機の実行に--inference.type=rtcを付ける。ドキュメントも両方を一緒に使うよう勧めている。 罠3：回線が切れたら、手持ちは最大1秒 1本のチャンクは50ステップ、1秒分しかない。通信が切れれば、ロボットは手持ちの行動を使い切った時点で指令を失う。PIの発表も、早期の展開でこんな問題があったと書いている。 including network connectivity challenges for some Googleは、7月に出したGemini Robotics On-Device 2をロボット側で動かす理由として、まさにこれを挙げている。 > Many robotic applications need to operate without network latency or internet connectivity. 対策：手持ちの行動が尽きたら安全に止まる処理を、必ずロボット側に置くこと。止められない作業なら、ロボットの中で動く代わりのモデルを用意する。 罠4：APIのモデルが、知らないうちに新しくなる PIは発表で、APIの裏のモデルを素早く更新し、最新のモデルを提供し続けると書いている。 Pi can update the models underpinning the API rapidly 版を固定できるのか、更新を知らせるのかは書かれていない。ロボットにとってモデルの更新は、動作が変わることそのものだ。昨日つかめた部品を、今日つかめなくなるかもしれない。 対策：代表的な作業を決めて、毎日同じ条件で成功率を測ること。結果は日時と一緒に残しておく。 罠5：上げたデータが、次のモデルの材料になるか分からない 申込フォームが聞くのは遠隔操作データと模擬データの時間数で、データの扱いには触れていない。一方、PIが2月に出したパートナーの記事には、パートナーのデータを事前学習に入れた場合と入れない場合を比べたグラフがある。 UPT = Ultra data included in pre-training (+UPT = included, -UPT = not included). パートナーのデータがPIの事前学習に入った例は、すでにあるということだ。APIで上げたデータがどう扱われるかは、公開されている情報からは分からない。 対策：規約でデータの使い道を確認できるまで、人の顔や社外秘の工程が映った映像は上げないこと。 推論の置き場所3択と、今すぐ試せる代替5選 置き場所3択 | 置き場所 | 足される通信 | 推論の速さの例（公表値） | 強み | 弱み | 今使える例 | | --- | --- | --- | --- | --- | --- | | クラウド | WANの往復（東日本→米国西部2で100ms） | 非公開 | 最新モデル。GPUと微調整を任せられる | 切断と揺れ。価格とデータの扱いが非公開 | Pi API β（順番待ち） | | 構内のGPU | LANで6.89〜21.20ms（PIの実測） | π0.5で97ms（RTX 4090、RTC込み） | 遅延が読める。データが外に出ない | GPUの運用。PIの公開重みはπ0.5まで | openpi、LeRobotの非同期推論 | | ロボットの中 | なし | GR00T N1.7で93.8ms（Jetson AGX Thor、TensorRT） | 切断に強い | 載せられるモデルの大きさと速さに限りがある | GR00T N1.7、SmolVLA、Gemini Robotics On-Device 2（早期アクセスのパートナー限定） | 代替5選（Hugging Faceのデータ、10月20日に取得） | モデル | 提供元 | パラメータ | ライセンス | 直近30日のDL | 名前を含むモデル数 | 推論の目安 | | --- | --- | --- | --- | --- | --- | --- | | SmolVLA（lerobot/smolvla_base） | Hugging Face | 0.45B | Apache-2.0 | 150,095 | 6,102 | 推論時メモリ約2GB | | GR00T N1.7（nvidia/GR00T-N1.7-3B） | NVIDIA | 3.1B | NVIDIA Open Model License | 108,706 | 443 | AGX Thorで93.8ms | | π0.5（lerobot/pi05_base） | PI（LeRobotが移植） | 3.6B | Gemma | 18,027 | 3,520 | VRAM 8GB超（openpi） | | MolmoAct2（allenai/MolmoAct2） | Ai2 | 5.4B | Apache-2.0（研究・教育向けと明記） | 14,951 | 308 | 設定によってはbf16で16GB未満 | | OpenVLA（openvla/openvla-7b） | OpenVLAの研究チーム | 7.5B | MIT | 152,456 | 80 | 1回の推論で1ステップ分 | パラメータはHugging FaceのAPIが返すsafetensorsの合計、直近30日のDLはモデルページの「Downloads last month」の値だ。名前を含むモデル数は、ロボティクス部門でsmolvla、groot、pi05、molmoact、openvlaを検索した件数で、派生モデルのおおまかな目安にすぎない。MolmoAct2のライセンスは、Hugging FaceのカードではなくGitHubの記載による。 表から読み取れること： - 直近30日のDL数1位は、2024年6月に論文が出たOpenVLA（15万2,456）。 ただし1回の推論で1ステップ分しか出さないので、遠隔推論とは相性が悪い（筆者の見立て）。 - 派生が最も多いのはSmolVLA（6,102）。 0.45Bで、推論時のメモリは約2GB。ロボットの中に置くなら筆頭だ。 - π0.5の重みは、ApacheではなくGemmaの条件で配られている。 openpiのコード自体はApache-2.0だ。商用で使う前に条件を確かめること。 - PIの最新モデルは、この表のどこにもない。 openpiの公開重みはπ0、π0-FAST、π0.5までで、Hugging FaceのPIの組織ページにあるモデルはFASTトークナイザー1つだけだった。最新世代を使う道は、今のところAPIしかない。 構内のGPUやロボットの中で、微調整したπ0.5やSmolVLAをRTC付きで動かすなら、LeRobotのドキュメントにある実行コマンドがそのまま土台になる（ロボット、ポート、カメラ、タスクは例。筆者は実行していない）。 lerobot-rollout \ --strategy.type=base \ --policy.path=${HF_USERNAME}/policy_repo_id \ --inference.type=rtc \ --inference.rtc.execution_horizon=10 \ --inference.rtc.max_guidance_weight=10.0 \ --robot.type=so100_follower \ --robot.port=/dev/tty.usbmodem58FA0834591 \ --robot.cameras="{ gripper: {type: opencv, index_or_path: 1, width: 640, height: 480, fps: 30}, front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}}" \ --task="Move green small object into the purple platform" \ --duration=120 \ --device=cuda 教訓：ロボットの脳は「速いGPU」より「近いGPU」に置く ✗ "クラウドのGPUのほうが速い" → GR00T N1.7はH100で27.9ms、Thorで93.8ms。差の66msは日米の往復100msで消える ✗ "同じ建物のLANなら遅延はほぼゼロ" → π0.5のLAN越し推論でも、移動ロボットで139ms ✗ "遅延は平均で見ればいい" → 学習時RTCが想定したのは最大200ms。裾が超えた瞬間は想定の外 ✗ "チャンクは平均でつなげばいい" → +100msで振動し、ロボットの保護停止が作動した ✗ "APIのモデルは昨日と同じ" → Pi can update the models underpinning the API rapidly PIのAPIは、ロボットの世界にLLMと同じ「頭脳をAPIで借りる」形を持ち込んだ。サンフランシスコのAlmondは、自社の双腕ロボットが自分の組み立てに使うスペーサーのバリ取りを、APIで4日試行錯誤したモデルにやらせている。自前のGPUなしで最新モデルを試せるのは、大きな前進だ。 ただ、チャットと違ってロボットは待ってくれない。50Hzの制御では20msが1ステップで、推論の遅れはそのまま「前の計画で動く時間」になる。今日、自分のロボットからGPUまでの往復をP99で測り、20msで割ってみてほしい。その数字が、申込フォームの答えになる。 参考リンク Frontier Models for Every Robot: Announcing our API beta - Physical Intelligence pi.website Pi API beta waitlist - Physical Intelligence pi.website Real-Time Action Chunking with Large Models - Physical Intelligence pi.website Real-Time Execution of Action Chunking Flow Policies - arXiv:2506.07339 arxiv.org Training-Time Action Conditioning for Efficient Real-Time Chunking - arXiv:2512.05964 arxiv.org The Physical Intelligence Layer - Physical Intelligence pi.website openpi - GitHub Physical-Intelligence/openpi Running openpi models remotely - openpi github.com/Physical-Intelligence/openpi/blob/main… simple_client - openpi github.com/Physical-Intelligence/openpi/tree/main… Real-Time Chunking (RTC) - LeRobot Docs huggingface.co Asynchronous Inference - LeRobot Docs huggingface.co Azure network round-trip latency statistics - Microsoft Learn learn.microsoft.com GR00T-N1.7-3B - Hugging Face huggingface.co Gemini Robotics 2 brings whole body intelligence to robots - Google DeepMind deepmind.google Gemini Robotics On-Device 2 - Model Card - Google DeepMind deepmind.google lerobot/smolvla_base - Hugging Face huggingface.co lerobot/pi05_base - Hugging Face huggingface.co MolmoAct2 - GitHub allenai/molmoact2 openvla/openvla-7b - Hugging Face huggingface.co Models: robotics - Hugging Face huggingface.co Physical Intelligence - Hugging Face huggingface.co --- この記事が役に立ったら、いいねと保存を！ PIのAPIに申し込もうとしているロボット屋の同僚にも、フォームを出す前にシェアしてあげてほしい。 コメントで教えてほしい。 - あなたのロボットの推論、Cloud？ Local？ Both？ - ロボットからGPUまでの往復、P99で何msだった？ - 最新モデルをクラウドで借りる？ それともSmolVLAやπ0.5を手元で回す？

I went through Physical-Intelligence/openpi line by line and created this. t.co

Choosing a Local LLM Through Fate (2026 Edition) --- canonical_url: 'akari-iku.github.io' --- > This post is an adapted write-up of a chat I had with a friend who asked, "So, what's actually going on in the AI and local LLM scene?" Table of Contents - [Introduction](#introduction) - [Local LLMs (Summoning Your Own)](#local-llms-summoning-your-own) - [What You Need to Run a Local LLM](#what-you-need-to-run-a-local-llm) - [Harnesses (Mystic Codes)](#harnesses-mystic-codes) - [How to Choose a Model to Run (Choosing a Heroic Spirit)](#how-to-choose-a-model-to-run-choosing-a-heroic-spirit) - [Wrapping Up](#wrapping-up) - [References](#references) Introduction Whenever local LLMs come up, I get hit with the same questions every single time: "Are local LLMs actually any good?", "What about the Chinese models?" and "What is each one good at?" Answering them over and over was getting tiresome, so I decided to write it all down in one place. Consider this post my answer. That said, new models come out practically every day, and their characteristics shift with each company's direction and with how its research goes. What I write here is only the trend as of October 2026. So rather than going model by model, I'll organise things by model family, such as the Qwen family and the Gemma family. Models in the same family often have similar personalities, even across generations. In Part 1, I laid out the AI industry as of October 2026 as a Holy Grail War from the Fate series. The AI labs (Masters) pit their models (Servants) against each other in pursuit of the Holy Grail that is AGI. NVIDIA (the Church Overseer) sells GPUs (mana) to every faction. The labs split into the big US players, which keep their weights to themselves, and the Chinese companies (the Eastern faction), which often release theirs. In this post, I'll walk you through how to join that Holy Grail War as an ordinary magus who belongs to no faction. A local LLM, a model you run on your own PC, is the Fate equivalent of summoning a Servant of your own. Even if you haven't read Part 1, the cheat sheet below will get you through. | Fate term | AI industry equivalent | | --- | --- | | Holy Grail | AGI | | Master | AI lab (and its CEO) | | Servant (Heroic Spirit) | Model | | Manifestation | A model up and running | | Spirit Origin | What's inside a model (the numerical precision of its weights) | | Noble Phantasm | A model's trump-card ability or mechanism | | Mana | Compute (GPUs) | | Command Spells | System prompts and safety constraints | | Church Overseer | NVIDIA | | Throne of Heroes | Training data | | Eastern faction | Chinese companies that often release their weights (DeepSeek, Qwen, GLM, Kimi and others) | Local LLMs (Summoning Your Own) Using an API Means Borrowing a Model (Someone Else's Servant) Using ChatGPT or Claude through an API or an app is like hiring a Servant that's already contracted to another Master. They're strong. Seriously strong. A Heroic Spirit with parameters in the trillions works for you for a monthly subscription in the app, or on pay-as-you-go pricing through the API. Here's the catch: the Command Spells are held by the AI lab that provides the model. What the model answers and what it refuses, when the specs change, how the information you hand over gets treated: the provider has the final say. If your Servant's personality has changed one day, you just have to accept it. Running Locally Means Using Your Own PC (Your Own Workshop) With a local LLM, you summon the Servant yourself, in your own workshop (your PC), with your own mana (your GPU). The Heroic Spirits you can summon at home are still a rank below the big factions' top tier. But that gap is shrinking year by year, and the models that run locally are no longer weaklings. And the Command Spells are in your own hands. | | Borrowing through an API | Summoning locally | | --- | --- | --- | | Strength | Top-tier Heroic Spirits | Whatever your own mana can summon | | Command Spells | Held by the other Master | Held by you | | Where your information goes | Through the other party's workshop | Never leaves your own workshop | | Running costs | Monthly fee for the app, pay-as-you-go for the API | Electricity and the upfront investment | | Spec changes | Change whenever it suits the other party | Don't change until you change them | Why Bother Running Models Locally? (Why Summon Your Own) If you compare peak performance alone, APIs are still ahead. Even so, there are four reasons to run models locally: - The information can't leave: Confidential documents, customers' personal data, data that a contract forbids you to take outside. Whether you can send these to someone else's workshop is a governance question before it's a technical one. If the data never takes a single step outside your own workshop (PC), the debate over whether it can be sent outside doesn't arise at all. - You can pin down the behaviour: A model built into your work can start giving different answers from yesterday's after an update on the provider's side. For the people running the system, that's a plain old incident. Weights on your own machine won't change by a single byte until you swap them out yourself. - Even legitimate work can get refused: Frontier models' safety measures can lead them to refuse perfectly legitimate work, in fields such as medicine or security. As long as the Command Spells are in the hands of the AI lab providing the model, the Servant may refuse to obey even when your order is legitimate. - It's just fun: The moment a Heroic Spirit starts talking on your own hardware never gets old. When people think of demand for local LLMs, I think many picture adult uses. But the people struggling with the third reason are doing perfectly legitimate work, in fields like medicine and security. Anyone reading this post will probably remember it well: in July 2026, Hugging Face was breached by an autonomous AI agent. Hugging Face tried to use commercial models to investigate the breach, but was turned away by the providers' safety measures. According to Hugging Face's report, those safeguards "cannot distinguish an incident responder from an attacker". So Hugging Face ran GLM-5.2, an open-weight model, on its own infrastructure and pressed on with analysing the logs. <huggingface.co> Ready-Made Machines for Local LLMs (Shops That Sell the Whole Workshop) One of Caster's class skills is Territory Creation: the ability to build a magecraft workshop that works in your favour. Stick with local LLMs long enough, and the gaming PC on your desk starts to fall short as a workshop, in VRAM and in memory. To meet that demand, between 2025 and 2026, several companies started selling workshops for local LLMs as off-the-shelf products. In October 2025, NVIDIA released DGX Spark, a compact machine with 128GB of unified memory. NVIDIA says it can run models of up to around 200 billion parameters right on your desk. As a higher-end machine, DGX Station, with 748GB of memory, is also sold through PC makers. Compact PCs and laptops with AMD's Ryzen AI Max+ 395 also advertise 128GB of unified memory. Apple offers the Mac Studio, which can be fitted with up to 512GB of memory. Unified memory is an architecture in which the CPU and the GPU share the same memory. Because most of the main memory can go to the model, you can summon Heroic Spirits that 8GB of VRAM can't. However, unified memory's read and write speed (bandwidth) is often slower than dedicated GPU VRAM. Your Heroic Spirit will show up, but it tends to move slowly. Workshops are getting pricier, too. Memory supply is tight in 2026, and in February 2026 NVIDIA raised DGX Spark's suggested retail price from $3,999 to $4,699. In October 2026, a version with the memory cut to 64GB goes on sale through OEMs, starting at $4,999. The Overseer has started selling workshops as well as mana, but the price tags on those workshops are set by the memory market. What You Need to Run a Local LLM A summoning needs three things: a catalyst, mana and a summoning circle. Add the Mystic Code (harness) that I'll cover in a later section, and the whole flow looks like this: !Image description Weights Files (Catalysts) To summon a Heroic Spirit, you need a relic tied to their legend. The catalyst you bring decides who answers the call. In the local LLM world, the model's weights file is your catalyst. Think of it as a Heroic Spirit called from the Throne of Heroes (the training data) and set into a form you can carry around. llama.cpp uses a format called GGUF, and Hugging Face has loads of these files. Some are published by the official developers; others are converted and uploaded by volunteers. File names usually include the model name, the parameter count and the quantisation type, something like model-name-9B-Q4_K_M.gguf. Once you can read these names, you're halfway through choosing a catalyst. Try Before You Buy in the Cloud (Support Servants) Fate/Grand Order has a system where you can borrow another Master's Servant as a Support and take them along on a quest. Before you summon that Servant yourself, you can see how they handle. You can do exactly the same with open-weight models. Services such as Cloudflare's Workers AI, Google's AI Studio (Gemma 4 at 31B and 26B) and Hugging Face's Inference Providers let you try open models running in the cloud. Before downloading several GB of weights, you can check how a model handles Japanese, how well it follows instructions and what quirks its tone has. In October, I did this myself with Clef, a decision model whose weights Cloudflare released, by calling it from Workers AI. <akari-iku.github.io> It has also become more common for inference services and inference engines to ship support on the same day a new model's weights are released: what's known as "Day-0 support". For example, the inference engine vLLM announced Day-0 support on 27 July 2026, the day Kimi K3's weights were released. New open models also show up early on OpenRouter, which lets you use models from multiple inference services in one place. So get a feel for a Servant's quirks as a Support, decide which Heroic Spirit to summon at home, and only then shut yourself away in your workshop. Just keep in mind that the version running in the cloud and the quantised version you use locally often differ in precision. A version quantised for local use (quantisation means lowering the precision of a model's numbers to make it smaller) can answer a little differently. VRAM (Mana Capacity) A Master's mana capacity decides the calibre of Heroic Spirit they can summon. In local LLMs, that's VRAM. My setup is an RTX 5060 with 8GB of VRAM (8151 MiB). As magi go, honestly, that's below average. Even so, there are Heroic Spirits I can summon. You can estimate the VRAM you need with some back-of-the-napkin maths: - The weights file size: goes into VRAM more or less as it is - The context: grows the longer you make the model remember the conversation - Headroom: your OS and browser drink from the same VRAM straw With 8GB, the realistic ceiling for a weights file is 5 to 6GB. A model whose file is exactly 8GB can't put all of its layers on the GPU of an 8GB card. Looking only at the file size and assuming the model will fit in VRAM is the classic rookie trap everyone falls into. The other trap is the "runs on 8GB" label from whoever distributes the model. When I tried Qwen3.8-27B in August, the figures in Unsloth's requirements table turned out to be the total of main memory and VRAM, not VRAM alone. If you read those figures as the answer to whether a model runs on a single 8GB card, you'll get it wrong. <akari-iku.github.io> A Master short on mana still has ways to make up for it. Rin Tohsaka stores mana in jewels ahead of time and uses it in battle. With local LLMs, too, you can add main memory and send the layers that don't fit over to the CPU, or add GPUs to get more VRAM. If that still isn't enough, you end up replacing the whole workshop. Quantisation (Manifesting on Too Little Mana) There's a way to force a high-calibre Heroic Spirit to manifest on very little mana: whittle down its Spirit Origin and settle for lower stats. Quantisation is a technique that lowers the precision of the weights' numbers to make the file smaller. Numbers originally held at 16 bits get rounded to 4 or 8 bits. The bit count is the size of the Spirit Origin; the fewer bits, the more you've whittled it down. Naturally, the model gets a little less clever. Picture a Servant with Strength A manifesting at B or C. | Quantisation | How far it ranks down (rough guide) | When to use it | | --- | --- | --- | | Q8 | Nearly its original stats | When you have mana to spare | | Q6 / Q5 | Hardly noticeable in practice | When you have a little to spare | | Q4_K_M | Drops slightly, but plenty usable | The standard. If in doubt, pick this | | Q3 | Degradation starts to show | When it just won't fit otherwise | | Q2 and below | Can't fire its Noble Phantasm | When getting it running is the goal in itself | There's one important rule of thumb: running a large model at Q4 is usually smarter than running a small model at high precision. A high-calibre Heroic Spirit has higher base stats, even after ranking down a bit. Go down to Q3 or below, though, and the story changes, so it's worth treating Q4_K_M as your floor. As a rough guide, a Q4_K_M file comes to about "parameter count (in billions) × 0.6GB". That's about 5GB for 8B and about 7GB for 12B. So what you can summon comfortably on 8GB of VRAM is roughly up to around 9B. llama.cpp (Summoning Circle) Last comes the summoning circle: the runtime for local LLMs. Ollama and LM Studio are pre-drawn ritual kits. They're easy, and good for summoning your very first Servant. As long as I use models whose layers all fit in VRAM, I've never had any trouble sticking with Ollama. My advice: start with Ollama, then move to llama.cpp when you want to run a model that won't fit in VRAM, or when you want finer control. llama.cpp gives you the chalk to draw the circle yourself. In return, you can decide in detail which parts of the model go on the GPU and which go to the CPU. When you run a model that won't fit in VRAM, that split shows up in the speed. In my August tests, running the same model at the same VRAM usage, Ollama's 6.30 tok/s became 9.35 tok/s on llama.cpp. The basic incantation is just this: llama-server -hf <username>/<repo>:Q4_K_M -ngl 99 -c 8192 - -hf: fetches the catalyst from Hugging Face. The download runs only the first time - -ngl: how many of the model's layers to put on the GPU. Writing 99 or all means "all of them" - -c: the context length. The bigger it is, the more VRAM it uses On launch, it brings up a chat screen you can open in a browser, plus an OpenAI-compatible API. In recent versions of llama.cpp, if you don't specify -ngl, it automatically decides how many layers to load so that they fit in VRAM (a feature called --fit, enabled by default). If you want to decide manually, lower the -ngl number. Layers that don't fit run on the CPU and main memory. In Fate terms, that's the Master making up the missing mana with their own life force: it runs, but it gets visibly slower. For the 8GB gang, the basic strategy is finding a setup that puts every layer on the GPU. Harnesses (Mystic Codes) A Harness Lets a Model Do More (What a Mystic Code Does) In Fate/Grand Order, Masters fight wearing a Mystic Code. Change your Mystic Code, and your Master Skills (the skills the Master can use) change, and so do the ways you can heal and buff your Servants. In local LLMs, the equivalent is the harness. A harness is a program that runs outside the model and hands it tools such as a shell, file editing and web search. When the model says to use a tool, the harness runs it and returns the result to the model. By repeating this back-and-forth, the model can move the work forward on its own instead of just returning answers. Claude Code, Codex CLI and OpenCode are all harnesses. llama-server brings up an OpenAI-compatible API, so with any harness that lets you change where it connects, you can put a Mystic Code on a Servant you summoned locally, too. !Image description DeepSeek Harness (A Mystic Code Handed Out by the Eastern Faction) On 13 August 2026, DeepSeek released DeepSeek Harness (command name dsh) under the MIT licence. It's modular to the bone: the model, the tools, the sandbox and even the work loop can all be swapped out as plugins. According to the official documentation, you can point it at external APIs or at your own server with an OpenAI-compatible API. The Eastern faction has started handing out not just its Heroic Spirits' weights, but the Mystic Codes to put on them as well. It's still a developer preview, though, and the README says in capitals: "THERE WILL BE COMPATIBILITY-BREAKING CHANGES." Matching the Harness to the Model (Choosing a Mystic Code) Just because good Mystic Codes come free doesn't mean you should put the same one on every Servant. Models differ in the tool-calling conventions they got used to in training, in the tasks they're good at and in how long they reason. When choosing a harness, too, you'd ideally get to know the traits of the Servant you summoned first, then pick a Mystic Code that plays to them. What gets talked about as a model's "strengths" is usually its raw performance, measured on the model alone without a harness. How it actually feels in use changes a lot with these three things: - Response preferences: whether it answers briefly and to the point, adds plenty of explanation or shows its thinking - How you instruct it for coding: how you write the prompt, whether you have it write tests first, whether you have it output changes as diffs - Harness design: the agent loop, how tools are called, how memory is kept, how it retries after a failure Especially for agent and coding work, differences in harness design tend to show up in the results more than differences between the models themselves. My impression is that Chinese models (Qwen, DeepSeek, GLM, Kimi) have more room to grow when you put them in a good harness than when you use them bare. Western models (Gemma, gpt-oss, Granite, Muse) and Mistral behave relatively predictably out of the box, but the right harness lifts them too. What decides the model you pick is often not "what is this model good at?" but "does it click with my harness?" Never Let an LLM Grade Its Own Homework (The Master's Job) There's one more reason to know your model's traits: LLMs tend to go easy on their own work. In a 2024 study by Panickssery, Bowman and Feng, LLMs could recognise text they had written themselves, and the better a model was at recognising it, the higher it scored its own text. A March 2026 study by Khullar et al. confirmed the same tendency in coding agents. When models were asked whether to approve a PR, they were more likely to miss problematic changes if they evaluated it as work they had done themselves in the same conversation. Once a coding agent has written your code, leave the review to a different model. If you stick with the same model, at least use a separate session. Combine that with checks that don't rely on a model's judgement, such as tests. Not taking your own Servant's report on trust alone, and bringing in another pair of eyes, is the Master's job. How to Choose a Model to Run (Choosing a Heroic Spirit) Deciding which model to summon gets easier if you look at memory, licence, use case, fit with your harness and the availability of quantised versions, in that order. Set the Ceiling by Memory (Mana Capacity) The first thing to look at is the size you can run with your memory. With the rule of thumb from the VRAM section (at Q4_K_M, parameter count (in billions) × 0.6GB), you can work out the size at which every layer fits in VRAM. Even if not every layer fits, the model will run with some layers in main memory, but it gets slower accordingly. MoE models are estimated a little differently. Take Qwen3.8-Flash-Next: it has 125B total parameters, and even its Q4 quantised version (Unsloth's UD-Q4_K_XL) is about 111GB. But only 6B of it is active per token. If you keep the expert parts in main memory and send them to the CPU with a llama.cpp option such as --n-cpu-moe, you may be able to run it even with little VRAM. In exchange, you need a lot of main memory. It's a summoning close to the way Rin Tohsaka fights in the VRAM section, making up the shortfall with mana stored in jewels. The tools for running models are multiplying, too. Strata, released in September 2026, for example, is an inference engine for running Qwen3.8-Flash-Next on a GPU with 12GB or more of VRAM and 32GB or more of main memory. As tools built for particular models appear, the range of Heroic Spirits you can summon widens little by little. You can also use tools to check what will run on your hardware. On GitHub, there's open-source software that inspects your PC's GPU and memory and suggests models you can run. llmfit gives rough figures for the memory needed and the speed for each quantisation type, and it supports MoE models too. whichllm ranks the models you can run based on recent benchmark results. Both give estimates, so in the end you run the model yourself and check. Check the Family and the Licence (True Name) Next, check the licence against what you'll use the model for. Whether you're trying it out personally, building it into your work or offering it as a service changes which conditions you need to check. Even within the same family, the licence can differ by model size. Here are the main families as of October 2026: | Family | Faction | Licence | Official | Hugging Face | | --- | --- | --- | --- | --- | | Qwen (Alibaba) | East | Apache 2.0 for the 27B and others. The largest, 2.4T, and Flash-Next use their own licence | qwen.ai | Qwen | | DeepSeek | East | MIT | deepseek.com | deepseek-ai | | GLM (Z.ai) | East | GLM-5.3 uses its own licence. GLM-5.3-Flash and 5.2 and earlier are MIT | z.ai | zai-org | | Kimi (Moonshot AI) | East | Kimi K3 uses its own licence (the K2 generation is modified MIT) | kimi.ai | moonshotai | | Gemma (Google) | West | Apache 2.0 (Gemma 4) | Google DeepMind | google | | gpt-oss (OpenAI) | West | Apache 2.0 | OpenAI | openai | | Granite (IBM) | West | Apache 2.0 | IBM | ibm-granite | | Muse (Meta) | West | Muse Glimmer is Apache 2.0 | Meta | meta-models | | Mistral | West (Europe) | Small 4 is Apache 2.0. Medium 3.5 is modified MIT (companies with consolidated monthly revenue above $20 million in the previous month can't use it) | mistral.ai | mistralai | Most of these custom licences put conditions on businesses that offer the model to third parties through an API or similar, or on products with many users or high revenue. Some, like Qwen3.8-Flash-Next, require a separate licence for offering the model through an API or building it into an AI assistant product, regardless of scale. If you're just running a model yourself on your own machine, it often isn't a problem, but check the model card and the licence file on Hugging Face before you use it. Decide by Use Case (The Heroic Spirit's Personality) Here's how each family tends to shape up. This is my own read as of October 2026, and it will change with each company's direction and research. | Family | Tendencies (strengths) | Sizes that run locally | | --- | --- | --- | | Qwen | An all-rounder that handles anything. Especially strong in Chinese and English, and also handles Japanese and Korean, its fellow CJK languages (Chinese, Japanese, Korean), to some extent. Also strong at coding, long contexts, tool use and agents | The previous generation has small models from 0.6B to 9B (for phones and light PCs). 27B to 32B is comfortable on a GPU with around 24GB. The latest Qwen3.8 starts at 27B | | DeepSeek | Good at reasoning. Good at thinking step by step, and strong at maths, logic and coding | For local use, the distilled versions of the previous-generation R1 (1.5B to 14B on 8 to 16GB, 32B on around 24GB). The latest V4.1 generation is several hundred B or more, server-class | | GLM | Strong at coding, agents, long-running tasks and software development. Efficient MoE | In the latest generation, even the small GLM-5.3-Flash is 320B (18B per token), server-class | | Kimi | Strong at agents, long contexts (up to 1 million tokens), long coding sessions and knowledge work. Extremely sparse MoE | The K2 generation (1T total, 32B per token) can be quantised and run if you have a large amount of memory. K3 (2.8T) needs a large cluster | | Gemma | Small to mid-sized, focused on efficiency. Handles images and audio, and is good at agents and reasoning. Suits laptops and phones | E2B and E4B (phones and laptops), 12B, 26B (4B per token), 31B | | gpt-oss | Reasoning, tool use, agents. Reasoning depth is adjustable. Also strong at maths and coding | 20b (16GB of memory), 120b (a single 80GB-class GPU). MoE, with only a small part active per token | | Granite | Enterprise-focused. Emphasises reasoning, agents, tool use and data governance. You can switch whether it shows its thinking | 3B and 8B (light to mid-range), 30B (high-end consumer GPUs) | | Muse | Aimed at agents running locally. Tool calling, always-on agents, coding, handling images | Muse Glimmer 30B (about 17 to 20GB at 4-bit; practical on a 24GB GPU or a Mac) | | Mistral | Efficient small models, plus multilingual and image handling. Also supports coding and agents. A strong flavour of European sovereignty | For local use, the previous-generation Ministral (3B, 8B, 14B). Small 4 (119B) and above need large resources | Check the Fit with Your Harness (Mystic Code) As I wrote in the harness section, the strengths listed above are about raw performance. How a model actually feels changes a lot with its response tendencies, how you instruct it and the harness design. Once you've narrowed it down to two or three candidates, the most reliable way to choose is to put them in your own harness, have them do the same task and compare the results. Are the Quantised Versions Out Yet? (Preparing the Spirit Origin) A freshly released model can't necessarily be run locally straight away. Day-0 support from inference services and inference engines means "it can be run". The quantised versions (GGUF) you can use locally usually arrive a bit later. If you run models with llama.cpp, the time to try is once the quantisation you want appears in the repository of a creator who distributes GGUFs, such as Unsloth. Examples That Run on 8GB of VRAM (Heroic Spirits You Can Summon) Here are the candidate Heroic Spirits I can summon on my 8GB VRAM setup. I assigned the classes on my own whim. The VRAM figures come from third-party roundup sites. | Class | Model | VRAM needed at Q4 (rough guide) | Traits | | --- | --- | --- | --- | | Saber | Qwen3.5 9B | Around 6GB | True to the "finest" class (the class considered the most outstanding in Fate), it has no quirks and handles anything. All layers fit in VRAM, with room to spare for context. If in doubt, start here | | Archer | Granite 4.2 8B (IBM) | About 4.6GB | True to its class skill Independent Action, it runs on little mana. You can put what you save towards a longer context. Apache 2.0 | | Berserker | Gemma 4 12B (Google) | About 7GB | Higher calibre, but over the 5 to 6GB ceiling for a weights file. You'll need to shorten the context or lower -ngl. If you want it to read images, this is the one | | Assassin | Bonsai 27B | About 3.9GB | Billed as a drastically shrunk Qwen3.6-27B. Takes its class skill Presence Concealment way too far. I haven't tested it, and it isn't one for your first summoning | Failures from My Own Tests (Summoning Mishaps) In Fate, summonings don't always go to plan. In Fate/stay night, Rin Tohsaka meant to summon Saber and got Archer instead. What's more, she made two slip-ups in that one summoning. The first was the time. Every clock in the Tohsaka mansion had been set an hour fast, so Rin thought she was summoning at 2 a.m., when her mana peaks, but she actually performed the ritual at 1 a.m. The second was where she stood. A Master is supposed to chant outside the summoning circle, but Rin was standing on it. A Heroic Spirit can't appear where a being of this world is already present, so Archer appeared somewhere other than the circle. I haven't tried Assassin's Bonsai 27B yet, but in August I did try a model built on the same idea of quantising a 27B extremely small: UD-IQ1_S (6.19GB), Unsloth's version of Qwen3.8-27B with its Spirit Origin whittled down to 1-bit class. Because it keeps the important layers at higher bit counts, its average, calculated from the file size, is about 1.8 bits. This summoning ended in a mishap: it lost outright to an 8B 4-bit quantised version (qwen3:8b) that I summoned on the same 8GB of VRAM. | | Qwen3.8-27B 1-bit class (6.19GB) | qwen3:8b 4-bit (4.9GB) | | --- | --- | --- | | Leap year function | Cut off at 700 tokens, incomplete | Completed in 43 tokens | | Binary search function | Cut off at 700 tokens, incomplete | Completed in 89 tokens | | Generation speed | 31.87 tok/s | 47 tok/s | | VRAM usage | 6974 MiB | 5722 MiB | The 1-bit-class 27B fit every layer in VRAM and ran at a decent clip. Even so, it couldn't finish writing the leap year check within the 700-token limit. Its thinking may simply have run too long, but it fell short of the 8B running under the same conditions. Carve away too much of the Spirit Origin, and its actual capability seems to vanish right along with its presence. There was also a mishap where Japanese answers looped forever: the model kept on writing about cherry blossoms blooming in both summer and autumn. The cause of that one, though, wasn't the whittled-down Spirit Origin; I had forgotten to set presence_penalty (a parameter that suppresses repeating the same words). When I set it to 1.5, Unsloth's recommended value, the loop stopped. Not every summoning mishap is the Spirit Origin's fault. Just like Rin standing on the summoning circle, sometimes the summoner gets the procedure wrong. <akari-iku.github.io> Whether Bonsai 27B can overturn this result is best left undecided until you've tried it with your own hands. Models That Won't Run on 8GB (Heroic Spirits You Can't Summon) The Eastern faction's top-tier models can't be summoned on 8GB, whatever you do. Even 27B, the small version of Qwen3.8, needs around 16GB at Q4. Whittle its Spirit Origin down to 1-bit class and it fits in 8GB, but in that state the 27B couldn't beat an 8B 4-bit quantised version. Western Heroic Spirits are no different: Muse Glimmer, whose weights Meta released in August, is 30B and needs around 18GB at Q4. That doesn't mean you can't spar with the Heroic Spirits you can't summon at home, though. As I wrote in the section on trying models in the cloud, several companies offer services that run open models in the cloud. Summon the top-tier models that won't fit at home in the cloud, and at home run the models your own mana can summon: that's plenty. Within what your setup can run, go ahead and run models and have fun with them. The models you can summon at home are getting better every year, too. Even within the same 9B class, they get visibly smarter with each generation. If you just sit tight, the Heroic Spirits shrink to fit our mana capacity. I think that's the best thing about this industry. {% details Sidebar Physical AI calls for a different way of choosing %} Everything so far about strengths, weaknesses and harnesses applies to the world of text and images. Once you get to Physical AI (AI with a body) that drives robots, the way you think about choosing a model changes quite a lot. - Required abilities: Rather than understanding long contexts or coding, what matters is the speed of looking at camera footage and turning it straight into action, physically correct movement, safe operation and low latency. - Data and evaluation: Instead of learning from huge amounts of web text, the focus is on robot operation data and on reinforcement learning and imitation learning in simulation. Evaluation, too, is measured by task success rates and safety, not coding benchmarks. - How the models are built: Rather than language models themselves, the focus is on VLA (Vision-Language-Action) models, which produce actions from video and language, and on world models, which predict how the world moves. Here are the models whose names come up often as of October 2026, split into the US and the West, and China. Versions are updated very quickly in this field, so read the model names on the assumption that they'll be out of date by the time you read this. | Faction | Model / family | Developer | Weights released? | Traits | | --- | --- | --- | --- | --- | | US / West | π0.7 (April 2026) | Physical Intelligence | Latest version not released. π0 to π0.5 released via openpi | Strong at dexterous manipulation and at combining skills it hasn't learnt | | US / West | Isaac GR00T N1.7 (generally available from July 2026) | NVIDIA | Released (NVIDIA Open Model License; commercial use allowed) | A 3B for humanoids. Designed for inference on NVIDIA's Jetson | | US / West | Gemini Robotics 2 (July 2026) | Google DeepMind | Not released. ER 2, which handles reasoning, is available through the Gemini API. The main model is limited to some partner companies | Whole-body control and planning, and coordination between multiple robots | | US / West | OpenVLA (June 2024) | Stanford, UC Berkeley and others | Released (MIT; built on Llama 2) | 7B. Often used as a baseline for comparison in research | | US / West | Helix 02 (January 2026) | Figure AI | Not released | Controls the whole body of the company's own humanoid with a single system | | US / West | SmolVLA, Octo | Hugging Face (LeRobot), UC Berkeley and others | Released (SmolVLA under Apache 2.0, Octo under MIT) | Small, and easy to run on consumer GPUs and edge devices | | China | Qwen-RobotManip / RobotNav / RobotWorld (June 2026) | Alibaba (Qwen team) | Not released (no plans to release) | A set covering object manipulation, navigation and a world model. The manipulation model is built on Qwen3.5-4B | | China | RynnBrain (February 2026) | Alibaba DAMO Academy | Released (Apache 2.0) | Built on Qwen3-VL, from 2B to 122B. A "brain" model that handles understanding and planning | | China | Hy-Embodied (June to July 2026) | Tencent | Released (VLA, RxBrain and VLM-1.0 under Apache 2.0) | Splits understanding the environment, reasoning and generating actions across three models | | China | Genie Envisioner 2.0, GO-2 (April 2026) | AgiBot | Genie Envisioner released (its original parts under CC BY-NC-SA 4.0). No GO-2 weights to be found (as of October 2026) | Large amounts of data collected on real robots, plus a world model and simulator | | China | UnifoLM-WLA-1.0 (September 2026) | Unitree | Released (Apache 2.0) | 6B. Handles 64 tasks on the company's own G1 robot. The code for further training is released too | | China | Xiaomi-Robotics-1 (August 2026) | Xiaomi | Released (Apache 2.0) | 5B. Built on Qwen3-VL | | China | InternVLA-A1.5 (July 2026) | Shanghai AI Lab | Released (both code and weights under CC BY-NC-SA 4.0) | Brings understanding, predicting the future and acting together in one model | | China | Seed GR-3 (July 2025) | ByteDance | Not released | 4B. Two-armed manipulation and handling soft objects such as cloth | Line them up, and the pattern looks awfully familiar from language models. In the US and the West, many companies keep their latest versions closed, and only some, such as NVIDIA, release their weights. In China, many companies release their weights, such as Tencent, Unitree and Xiaomi, though some, such as Alibaba's Qwen team and ByteDance, don't. Physical AI doesn't have settled standard benchmarks yet, so the performance rankings are fluid. The pecking order gets reshuffled here, too. A faction that's strong in language models isn't necessarily strong in robots as well. Having released weights that you can run locally also takes on a different meaning: running models on the small devices mounted on the robots themselves (edge devices). That said, many VLA models are built on models that are strong at understanding language and images, so this isn't completely unrelated to everything so far. In Physical AI, what data you collect, and how much, and what control system you put the model on carry even more weight than which base model you use. In Fate terms, it's a world where how you train the Heroic Spirit after the summoning, and how the Master calls the plays, count for more than the Heroic Spirit's calibre. {% enddetails %} Wrapping Up The Holy Grail War framing held up all the way down to local LLMs. There's just one place where reality parts ways with Fate, though. The Holy Grail War in Fate/stay night is a ritual for seven chosen Masters only. Ordinary people may get caught up in it, but they can't take part. Not so in the real world. The catalysts are public, how to draw the summoning circle is public, and even I, with a mere 8GB of mana, can summon Heroic Spirits. Even if the Holy Grail itself is out of reach, anyone can go as far as forming a contract with a Servant and commanding it with their own Command Spells. None of this is a given. Many in the Eastern faction keep releasing the weights of their top-tier models, volunteers keep quantising and distributing them, and summoning circles like llama.cpp are maintained free of charge. Pull just one of those Jenga blocks, and the Holy Grail War goes right back to being a spectator sport. The catalyst warehouse is about to change hands, too. In September 2026, NVIDIA agreed to acquire Hugging Face, and it plans to complete the acquisition in the first half of 2027, after reviews in various countries. NVIDIA has promised to keep the current setup, in which anyone can upload and download models and datasets, and to keep supporting other chipmakers. Whether that promise holds after the acquisition is something I think rogue magi in particular should keep a close eye on. If you compare peak performance alone, borrowed Servants are still stronger for now. Even so, only the Master who holds the Command Spells can decide what a model does and what it refuses, and which information goes outside. Before you choose a Servant on strength alone, check whose hands the Command Spells are in. I ask of you, are you that model's Master? References - NVIDIA Confirms $12.9B Hugging Face Acquisition (HPCwire) - Zhipu AI releases GLM-5.2 (DataNorth) - Security incident disclosure (Hugging Face, July 2026) - Alibaba releases Qwen3.8-27B (DataNorth) - Nvidia DGX (Wikipedia) - 2/23/2026 Price Change Announcement (NVIDIA Developer Forums) - DGX Spark 64GB (NVIDIA Blog) - DeepSeek Harness: Configure models (official documentation) - Best Local LLMs for 4GB, 6GB, and 8GB VRAM in 2026 (Mayhemcode) - Best LLM for 8 GB of VRAM in 2026 (bestllmfor) - The Qwen3.8 announcement and Bonsai 27B (GIGAZINE) - Kimi K3 Is Here: Efficient Day-0 Support on vLLM (vLLM) - NVIDIA Form 8-K (2 September 2026, the agreement to acquire Hugging Face) - Qwen3.8-Flash-Next (Hugging Face) - Strata (GitHub) - llmfit (GitHub) - whichllm (GitHub) - deepseek-ai/deepseek-harness (GitHub) - The Open-Sourcing of DeepSeek Harness (InfoQ) - LLM Evaluators Recognize and Favor Their Own Generations (arXiv) - Self-Attribution Bias: When AI Monitors Go Easy on Themselves (arXiv) - I Squeezed a 27B Into 1-bit on 8GB of VRAM. The 8B at 4-bit Won Anyway (my blog)

(4/4) Shoutout to collaborator @ziyaoshangguan.bsky.social and advisor Tesca Fitzgerald ! PR adding a no-delta config: openpi#1048

🚨 ATTENTION: Your fine-tuned π0-FAST should reach ~85% on LIBERO-10, not the commonly quoted ~60%. The ONLY change: no extra delta transform in fine-tuning. 💻 Code+results: Severus-Yang0/libero-delta-ablation 🤗 Checkpoints: huggingface.co 🐛 Issue: openpi#1047 (1/4)