Open-AutoGLM

Open-AutoGLM is an open-source phone agent framework and model release from Zhipu AI (Z.AI), built on AutoGLM. It reads the phone screen with a vision-language model and carries out tasks through ADB or HDC, so a user can say something like "open Xiaohongshu and search for food" and the agent plans and executes the steps.

What It Is

Phone Agent is a Python framework that controls a phone the way a person would. Each step takes a screenshot, the vision-language model interprets the interface, and the agent outputs an action. It then executes that action on the device and repeats the loop until the task is done. The agent runs on the user's computer, while the model runs either as a remote API or on a self-hosted server.

The agent's action set includes Launch, Tap, Type, Swipe, Back, Home, Long Press, Double Tap, Wait and Take_over. Per the README, it supports 50+ popular Chinese Android apps and 60+ HarmonyOS apps.

Models and Deployment

Two models are published on Hugging Face and ModelScope: AutoGLM-Phone-9B, optimized for Chinese mobile apps, and AutoGLM-Phone-9B-Multilingual, which supports English and other-language content. The README documents two ways to get a model endpoint:

• Use hosted third-party services (Zhipu BigModel or ModelScope).
• Self-host with vLLM or SGLang, which exposes an OpenAI-compatible endpoint.

The repository includes a script that checks whether a deployment works. The README states the models are released under the MIT License and the code under Apache-2.0.

Using the Agent

The agent can be driven from a command-line interface, in interactive or single-task mode, or from a Python API. Configuration covers the model, the maximum number of steps, the device ID, the language and system prompts (Chinese or English), and verbose reasoning output. Android devices need Android 7.0+, developer mode, USB debugging and ADB Keyboard for text input. HarmonyOS devices use HDC, and iPhone setup goes through WebDriverAgent. Remote debugging over WiFi is supported, and multiple devices can be managed.

Sensitive operations trigger confirmation callbacks, and login or captcha situations can be handed to a human through a takeover callback. Midscene.js has adapted AutoGLM, which gives iOS and Android automation through its SDK.

Background and Intent

The launch blog post dated December 8, 2025 describes AutoGLM's development since April 2023 and says Z.AI open-sourced it so phone-use capability would not stay inside a few vendors. It also says private deployment lets enterprises and developers keep data, logs and permissions in their own environments. The README says the project is for research and learning only and prohibits illegal use.