Advancements in large language models (LLMs) have expanded their utility across various applications, including content summarization and tackling intricate tasks. A cutting-edge frontier now involves integrating LLMs with robotics and physical hardware, bringing generative AI into the physical realm. To demonstrate this concept, we created an engaging game called RoboTic-Tac-Toe for the AWS re:Invent 2024 Builders Fair. This interactive game leverages Amazon Bedrock, Strands Agents, AWS IoT Core, AWS Lambda, and Amazon DynamoDB to showcase how LLMs can handle game strategies, complex tasks, and real-time robot control.
RoboTic-Tac-Toe features physical robots maneuvering on a tic-tac-toe board, guided by LLMs to coordinate both gameplay and robotic movements. Players interact with the game through natural language commands, dictating the robot’s actions on the board. The game emphasizes the seamless integration of AWS services, allowing for dynamic generation of real-time instructions instead of relying on pre-programmed sequences.
The game setup includes a mobile-friendly web portal where players access RoboTic-Tac-Toe by scanning a QR code. The robots respond to commands by executing precise movements to place markers on the board, all orchestrated by LLMs. This innovative approach enables AI-driven gameplay, where players can compete against an intelligent opponent controlled by LLMs.
A key component of the system is the integration of computer vision techniques, crucial for precise robot movements and accurate gameplay. Using AWS services and advanced computer vision methods, a Raspberry Pi camera positioned above the game board captures images that are processed through Principal Component Analysis (PCA) and OpenCV to track robot orientation and position effectively.
The AI architecture built on AWS IoT Core comprises Strands Agents, including the Supervisor Agent, Move Agent, and Game Agent. These agents automate tasks, manage interactions, and facilitate strategic decision-making, ensuring efficient gameplay and seamless user experience. The game progress is tracked and managed using AWS Lambda, Amazon DynamoDB, and Strands Agent, enabling AI-driven automation of moves and gameplay progress tracking.
RoboTic-Tac-Toe exemplifies the convergence of AI, robotics, and cloud computing in creating interactive and engaging experiences. This project underscores the potential of AWS IoT, machine learning, and generative AI not only in gaming but also in education and beyond. As AI-powered robotics advance, RoboTic-Tac-Toe offers a glimpse into a future of intelligent and interactive gaming experiences. Stay tuned for upcoming enhancements, expanded gameplay modes, and more captivating AI-powered interactions.
The team behind this innovative project includes Georges Hamieh, a Senior Technical Account Manager at Amazon Web Services, Mohamed Salah, a Senior Solutions Architect, and Saddam Hussain, also a Senior Solutions Architect. Each brings a distinct expertise in data, AI, and cloud solutions, reflecting a shared passion for innovation and technology. Dr. Omer Dawelbeit, a Principal Solutions Architect at AWS, contributes valuable insights and experience in designing scalable solutions for customers across various sectors.
About the Author
