Alibaba, a Chinese tech giant, has recently introduced its first family of embodied AI models, known as the Qwen-Robot suite, developed by Alibaba’s Tongyi Lab. This innovative suite connects large language models with real-world robotic actions and is currently undergoing pilot testing with select Alibaba Cloud enterprise clients. The Qwen-Robot suite consists of three models tailored for navigation, manipulation, and world modeling to assist robots operating in physical spaces.
Alibaba highlights that these models empower machines to perceive, reason, and interact with the physical world, contributing to the global effort to advance embodied AI beyond conventional chatbot applications. The Qwen family AI models excel in understanding the physical world by recognizing objects, comprehending spatial relationships, following complex visual instructions, and reasoning about real-world environments. For instance, a model can interpret a command like, “Go to the kitchen, find the red cup, pick it up, and place it on the shelf.”
However, bridging the gap between understanding tasks and executing them poses a challenge. While a vision-language model (VLM) can articulate the steps necessary to complete a task, it cannot directly control a robot’s movements. Integrating human language and visual comprehension with the motor actions needed to interact with the physical world is complex due to the distinct nature of robot training data compared to internet data.
To address this challenge, Alibaba devised the Qwen-Robot Suite, comprising three specialized models. Qwen-RobotNav focuses on navigation and movement, aiding robots in following instructions, navigating locations, tracking targets, and supporting autonomous driving. Qwen-RobotManip, on the other hand, emphasizes physical interaction, enabling robots to grasp, move, and manipulate objects using a vast training dataset. Lastly, Qwen-RobotWorld serves as a world model, predicting environmental changes and assisting robots in understanding the potential outcomes of their actions.
Alibaba recently showcased the effectiveness of its Qwen-RobotNav model on a Unitree Go2 quadruped robot by guiding it through an unfamiliar apartment solely based on spoken instructions across multiple rooms. The company also introduced Qwen-RobotClaw, a robotic agent framework, enabling Qwen models to utilize the Qwen-Robot suite as tools in the physical world. Additionally, Alibaba made Chat2Robot, a browser-based platform for testing embodied AI interactions, available in open-source.
Alibaba’s venture into embodied AI with the launch of its Qwen-Robot models signifies a broader industry trend towards developing AI systems capable of understanding and interacting in the real world. As competition in physical AI intensifies globally, various companies and start-ups are investing in AI software for autonomous decision-making, including Google DeepMind, Nvidia, Physical Intelligence, Skild AI, and Figure AI. China, leveraging its manufacturing expertise with AI investments, is emerging as a key player in the development of embodied AI technologies, with companies like Alibaba, Tencent, and others actively pursuing advancements in this field.
About the Author
