HomeProjectsArticlesPK BoardAbout
Explore Projects

AppAgent

Stale
GitHub Python MIT

Description

AppAgent is an LLM-based multimodal agent framework designed to operate smartphone apps like a human, supporting touch interaction and autonomous exploration.

Key Features

  • LLM-based multimodal agent that operates smartphone apps via tapping and swiping
  • Supports both autonomous exploration and human demonstration learning modes
  • Simplified action space design operates across apps without system backend access
  • Integrates GPT-4V and Qwen-VL multimodal models for decision making
  • Grid overlay feature precisely locates unlabeled UI elements on screen
  • Works with Android Studio emulator — no physical device required

Use Cases

💡 Automate social media operations like following users and posting content
💡 Assist visually impaired users in navigating smartphone applications
💡 Batch automate repetitive mobile tasks at scale
💡 Research human-computer interaction and GUI automation

Strengths & Limitations

✅ Strengths

  • • High community interest (6.9k stars)
  • • Permissive open-source license (MIT)
  • • Established track record (2 years in production)

⚠️ Limitations

  • • No updates in over 17 months

Tags

multimodal smartphone gui-agent llm gpt4v agent-framework

Categories

🤖 Agent Framework 🌐 Browser Agent

Quick Start

1. Install Android Debug Bridge (adb) and connect your Android device via USB
2. Clone the repo and install dependencies: `pip install -r requirements.txt`
3. Configure your OpenAI API key and request interval in `config.yaml`
4. Start the exploration phase — choose autonomous exploration or human demonstration
5. Run the deployment phase to execute your target task
Visit GitHub Visit Website View Docs

Related Projects

Open-AutoGLM

26.1k · Python
Stale B

Open-AutoGLM is an open phone agent model and framework enabling AI to autonomously operate smartphone interfaces, unlocking the AI Phone experience for everyone.

phone-agentgui-agentagent-framework +2
  • · Multimodal screen understanding — perceive smartphone screen content via vision-language models (VLM) to understand current interface state
  • · ADB/HDC device control — control Android and HarmonyOS devices through Android Debug Bridge and HDC protocols
  • · Natural language task planning — user describes requirements in natural language, system auto-parses intent and plans operation flows
View full details → View on GitHub

AutoAgent

9.8k · Python
Stale B

Fully-automated and zero-code LLM agent framework that enables users to build and deploy custom AI agents through natural language without writing code.

agent-frameworkzero-codeautomation +2
  • · Fully-automated agent building through natural language — no coding required to create, customize, and deploy AI agents
  • · Zero-code framework democratizing AI development for users without technical backgrounds
  • · Self-managing workflow generation that dynamically creates and optimizes agent pipelines from high-level descriptions
View full details → View on GitHub

ART

10.7k · Python
Active A+

ART (Agent Reinforcement Trainer) trains multi-step agents for real-world tasks using GRPO reinforcement learning, enabling on-the-job training for models like Qwen, Llama, and more.

reinforcement-learningagent-trainingGRPO +3
  • · GRPO-based reinforcement learning for training multi-step agents on real-world tasks
  • · W&B Training serverless RL with 40% lower cost and 28% faster training via shared inference cluster
  • · Pre-built notebooks for email search, 2048, Tic Tac Toe, Codenames, and MCP tool mastery
View full details → View on GitHub