midscene

Tapilot 开源了 📱 不连接电脑，直接打字或说出任务，让 AI 看屏幕操作安卓手机。 基于 Midscene，无需 ADB 或 root，执行过程可回放。也有 Skill／Codex 插件。 源码：tapilot-ai/tapilot 官网：tapilot.lifeos.md 终于开源了，拖延了好久 @dingyi 😄 t.co

I built a voice-controlled Android app—even WeChat works I built a voice-controlled Android app powered by Midscene—even WeChat works. No ADB, developer mode or USB cable. Wireless Codex control too. See it in action 👇 I plan to open-source the app. Follow for updates! Built with Midscene, the open-source UI automation framework. Give their repo a star: web-infra-dev/midscene

一句话操控安卓手机的 demo 底层是这个开源项目 Midscene，1.5 万 star 字节 web-infra-dev 在维护，MIT 协议 纯视觉操作界面，像人一样看屏幕 不用写元素选择器 Android、iOS、鸿蒙、桌面通吃 AndroidWorld 93.1% web-infra-dev/midscene

以前拿它跑测试，现在拿它给自己的产品做宣传。 现在 Midscene 还能用 ChatGPT 订阅额度，登录授权后就能接，不用另买 API 额度。 Midscene skills： web-infra-dev/midscene-skills 用得上，也帮 Midscene 点个 star 😃 主仓库： web-infra-dev/midscene

Testing a toast after it's already gone: a small Midscene example I work on Midscene. One awkward UI test: submit a form, then check the success toast. By the time the check runs, the toast may already be gone. Midscene has startObserving() for this. Start recording before the action, stop afterward, and run the assertion against the recorded frames: const observer = await agent.startObserving(); await agent.aiAct('submit the form'); const observation = await observer.stop(); await observation.aiAssert('a success toast appeared during the process'); await observation.dispose(); The same approach is useful for banners and page transitions. It samples the screen, so a very brief event can still be missed; this isn't a frame-perfect game recorder. Midscene is open source. Model calls have their own cost. Docs: midscenejs.com Source: web-infra-dev/midscene What transient UI would you want to test this way?