Google Just Open-Sourced ML Drift, the GPU Engine Behind LiteRT. Here's What Changes for You
If you've ever tried to ship an ML model that runs fast on users' GPUs, you know the problem: you have no idea which GPU, driver or graphics API your app will meet. A Pixel, an iPhone, a browser tab and a Windows laptop all speak different dialects.
On 8 October, Google's AI Edge team open-sourced the engine it uses to deal with exactly that. It's called ML Drift, and if you use TensorFlow Lite or LiteRT, it affects you.
What ML Drift is
ML Drift is an open-source (Apache 2.0), cross-platform GPU compute engine built for on-device AI/ML inference. It hides the hardware and low-level API differences across OpenGL ES, OpenCL, Metal and WebGPU.
It's the core GPU acceleration engine inside LiteRT (Google's on-device runtime), and it's also available as a standalone library if you're building your own graphics or inference runtime.
It isn't a research toy. Google says it already runs in production on millions of devices daily, in Chrome, YouTube Shorts, Photos and Meet.
Why it matters
One shader codebase instead of three. The old TFLite GPU delegate needed separately maintained shaders for OpenGL, OpenCL and Metal, with tensors hardcoded to physical GPU objects like textures and buffers. ML Drift introduces tensor virtualization: a tensor's logical layout is separated from how it's physically stored on the GPU, so one set of shader templates works across backends, with minimal runtime overhead according to Google.
5D tensors, finally. The legacy delegate was hardcoded to 4D tensors. ML Drift supports 5D out of the box, which helps 3D convolutional and spatiotemporal models. Google names YOLO 11n, MobileViT v2 and Swin Transformer v2 as examples.
Built with on-device LLMs in mind. LLM inference has a compute-heavy prefill stage and a memory-bandwidth-heavy decode stage. ML Drift switches kernels and layouts depending on which stage it's in. In Google's Gemma benchmarks, it used up to 12% less memory than other frameworks.
Desktop, too (in preview). The WebGPU backend, compiled natively through Dawn (Chromium's WebGPU implementation), lets ML Drift run on Windows and Linux, alongside a native Metal backend on macOS. Mobile and edge remain the main focus.
Custom ops are easier. There's a custom op framework with direct registration APIs, plus a SKILL.md guide that helps coding agents write, register and check custom shaders.
The production numbers Google shares: up to 40% lower average frame latency for YouTube Shorts effects on Android and iOS, up to 30% faster on-device performance for some Adobe Lightroom and Photoshop features, and 30% lower model latency for some Snapchat lens effects.
If you use the TFLite GPU delegate
This is the part to act on. The legacy TFLite GPU delegate will no longer receive new feature updates. Google recommends moving to the LiteRT ML Drift GPU accelerator, which it says is fully backwards compatible with existing models. Google also says models already running on TFLite GPU should maintain or beat their previous benchmarks.
One caveat for Android developers using unbundled runtimes to keep app size down: ML Drift acceleration is available now in the standalone LiteRT packages, but it's still "coming soon" to LiteRT in Google Play Services.
How to get started
The announcement gives two paths:
- App developers: use the LiteRT ML Drift accelerator. You get model partitioning, tensor virtualization and LLM acceleration out of the box.
- GPU and systems engineers: build directly on ML Drift's standalone C++ APIs from the GitHub repo to build custom GpuModel compute graphs and write your own shaders. There are quickstarts for OpenCL on Android and WebGPU via Dawn.
Source code: google-ai-edge/ml-drift. Issues and feature requests go on GitHub, and questions go to the Google AI Edge Community Forum.
The takeaway
If your app relies on the TFLite GPU delegate, it won't break today, but it's now frozen. Put a migration to the LiteRT ML Drift accelerator on your roadmap, benchmark your models before and after, and check the Play Services caveat if you ship an unbundled runtime.
Source: ML Drift: Next-Gen GPU AI/ML Inference at the Edge (Google Developers Blog)
---
I write Signal Tech, a free weekly plain-English briefing on AI, chips and dev tools: dailysignaltech.beehiiv.com
Disclosure: this post was drafted with help from an AI assistant and reviewed and edited by me.
GitHub - google-ai-edge/ml-drift: GPU-Accelerated AI/ML Inference
Blog Post : ML Drift: Next-Gen GPU AI/ML Inference at the Edge - Google Developers Blog
>The Google AI Edge Team is excited to announce the open-source release of ML Drift, our high-performance, cross-platform, on-device GPU compute engine specifically built for on-device AI/ML inference, under the Apache 2.0 license. By abstracting hardware and low-level API complexities of on-device GPUs across OpenGL ES, OpenCL, Metal, and WebGPU, ML Drift empowers developers to build real-time, interactive ML experiences from advanced video effects to generative AI across multiple platforms. Serving as the core GPU acceleration engine within LiteRT, ML Drift is also available as a standalone library for custom graphics and inference runtimes providing a unified foundation to deliver peak performance everywhere.
136 points · 16 comments
Mentions
Google open-sourced ML Drift, the GPU engine behind LiteRT — Apache 2.0, one codebase for OpenCL, Metal, WebGPU and OpenGL ES. Cross-GPU on-device inference is the unglamorous problem that decides what actually ships. A big win for open source! google-ai-edge/ml-drift
Google Just Open-Sourced ML Drift, the GPU Engine Behind LiteRT. Here's What Changes for You If you've ever tried to ship an ML model that runs fast on users' GPUs, you know the problem: you have no idea which GPU, driver or graphics API your app will meet. A Pixel, an iPhone, a browser tab and a Windows laptop all speak different dialects. On 8 October, Google's AI Edge team open-sourced the engine it uses to deal with exactly that. It's called ML Drift, and if you use TensorFlow Lite or LiteRT, it affects you. What ML Drift is ML Drift is an open-source (Apache 2.0), cross-platform GPU compute engine built for on-device AI/ML inference. It hides the hardware and low-level API differences across OpenGL ES, OpenCL, Metal and WebGPU. It's the core GPU acceleration engine inside LiteRT (Google's on-device runtime), and it's also available as a standalone library if you're building your own graphics or inference runtime. It isn't a research toy. Google says it already runs in production on millions of devices daily, in Chrome, YouTube Shorts, Photos and Meet. Why it matters One shader codebase instead of three. The old TFLite GPU delegate needed separately maintained shaders for OpenGL, OpenCL and Metal, with tensors hardcoded to physical GPU objects like textures and buffers. ML Drift introduces tensor virtualization: a tensor's logical layout is separated from how it's physically stored on the GPU, so one set of shader templates works across backends, with minimal runtime overhead according to Google. 5D tensors, finally. The legacy delegate was hardcoded to 4D tensors. ML Drift supports 5D out of the box, which helps 3D convolutional and spatiotemporal models. Google names YOLO 11n, MobileViT v2 and Swin Transformer v2 as examples. Built with on-device LLMs in mind. LLM inference has a compute-heavy prefill stage and a memory-bandwidth-heavy decode stage. ML Drift switches kernels and layouts depending on which stage it's in. In Google's Gemma benchmarks, it used up to 12% less memory than other frameworks. Desktop, too (in preview). The WebGPU backend, compiled natively through Dawn (Chromium's WebGPU implementation), lets ML Drift run on Windows and Linux, alongside a native Metal backend on macOS. Mobile and edge remain the main focus. Custom ops are easier. There's a custom op framework with direct registration APIs, plus a SKILL.md guide that helps coding agents write, register and check custom shaders. The production numbers Google shares: up to 40% lower average frame latency for YouTube Shorts effects on Android and iOS, up to 30% faster on-device performance for some Adobe Lightroom and Photoshop features, and 30% lower model latency for some Snapchat lens effects. If you use the TFLite GPU delegate This is the part to act on. The legacy TFLite GPU delegate will no longer receive new feature updates. Google recommends moving to the LiteRT ML Drift GPU accelerator, which it says is fully backwards compatible with existing models. Google also says models already running on TFLite GPU should maintain or beat their previous benchmarks. One caveat for Android developers using unbundled runtimes to keep app size down: ML Drift acceleration is available now in the standalone LiteRT packages, but it's still "coming soon" to LiteRT in Google Play Services. How to get started The announcement gives two paths: - App developers: use the LiteRT ML Drift accelerator. You get model partitioning, tensor virtualization and LLM acceleration out of the box. - GPU and systems engineers: build directly on ML Drift's standalone C++ APIs from the GitHub repo to build custom GpuModel compute graphs and write your own shaders. There are quickstarts for OpenCL on Android and WebGPU via Dawn. Source code: google-ai-edge/ml-drift. Issues and feature requests go on GitHub, and questions go to the Google AI Edge Community Forum. The takeaway If your app relies on the TFLite GPU delegate, it won't break today, but it's now frozen. Put a migration to the LiteRT ML Drift accelerator on your roadmap, benchmark your models before and after, and check the Play Services caveat if you ship an unbundled runtime. Source: ML Drift: Next-Gen GPU AI/ML Inference at the Edge (Google Developers Blog) --- I write Signal Tech, a free weekly plain-English briefing on AI, chips and dev tools: dailysignaltech.beehiiv.com Disclosure: this post was drafted with help from an AI assistant and reviewed and edited by me.
Source (Google Developers Blog, 8 Oct 2026): developers.googleblog.com Code: google-ai-edge/ml-drift Worth knowing: Google says it is already in production across millions of devices (Chrome Built-In AI, Photos Unblur, Shorts effects; Adobe and Snap are named partners). Company latency claims (for example up to 40% lower frame latency on Shorts) are theirs. Standalone LiteRT packages have it today; Play Services support is still coming.
GitHub - google-ai-edge/ml-drift: GPU-Accelerated AI/ML Inference Blog Post : ML Drift: Next-Gen GPU AI/ML Inference at the Edge - Google Developers Blog >The Google AI Edge Team is excited to announce the open-source release of ML Drift, our high-performance, cross-platform, on-device GPU compute engine specifically built for on-device AI/ML inference, under the Apache 2.0 license. By abstracting hardware and low-level API complexities of on-device GPUs across OpenGL ES, OpenCL, Metal, and WebGPU, ML Drift empowers developers to build real-time, interactive ML experiences from advanced video effects to generative AI across multiple platforms. Serving as the core GPU acceleration engine within LiteRT, ML Drift is also available as a standalone library for custom graphics and inference runtimes providing a unified foundation to deliver peak performance everywhere.