Microsoft has announced its exploration into human-robot interactions using the ChatGPT artificial intelligence technology. Developed by OpenAI, ChatGPT is capable of generating and linking ideas, as well as remembering conversations. Over the past month, Microsoft has been incorporating this technology into its various services, starting with Azure OpenAI Service in mid-January, and subsequently integrating it into the Bing search engine and Edge browser. The latest development includes a preview of the new AI-based Bing in mobile apps and Edge browser, as well as the search engine version for Skype.
The tech giant revealed that it is currently testing this technology in robots to enable more natural interactions with humans and outlined its expectations for this integration. Microsoft’s aim with this research is to explore if ChatGPT can go beyond text and reason about the physical world to assist with robotics tasks. The company seeks to facilitate easier human-robot interactions without the need for complex programming languages or detailed knowledge of robotic systems.
Microsoft faces the challenge of teaching the chatbot to solve problems considering factors such as the operational environment’s context and how physical actions can impact the world’s state. While OpenAI’s ChatGPT can perform tasks independently, Microsoft has established design principles to guide language models towards solving robotic tasks. Microsoft envisions ChatGPT unlocking a new paradigm in robotics, streamlining the typically slow, costly, and inefficient process of rewriting robot behavior code.
To enhance the Large Language Model (LLM), Microsoft has outlined design principles to provide guidance for robotic tasks, including defining a set of Robotics Application Programming Interfaces (APIs). Developers write task descriptions for ChatGPT to interpret, specifying what the machine should or should not do, and how the chatbot should respond. They evaluate the AI’s responses for quality and safety using natural language and a simulator. Once the desired response is achieved and deemed viable, the final AI chat code is implemented in the robot.
Microsoft shared experiments demonstrating the actions robots can perform using OpenAI’s model. The chatbot successfully learned to control a real drone, write complex code structures, such as creating a zigzag pattern to organize shelves, and analyze user intention and detailed signals for precise drone control in the Microsoft AirSim simulator. In experimental scenarios, ChatGPT effectively completed tasks like stacking wooden blocks and finding objects using detection functions.
Microsoft’s innovative approach opens the door to more efficient and intuitive human-robot interactions, paving the way for advancements in robotics technology.
About the Author
