Drones have emerged as a versatile tool to assist Blind and Low Vision (BLV) individuals. However, developing innovative interaction methods is essential to allow BLV users to effectively communicate with drones. In a recent project, an LLM-powered assistive drone was created specifically for BLV users. This drone utilizes an LLM to convert users’ high-level objectives into step-by-step commands for the drone and to interpret visual data from images.
A study involving 9 BLV participants was conducted to identify potential use cases and preferred ways of interaction. Through a collaborative and iterative process, a prototype was developed based on feedback from 3 BLV users and 5 experts in the field. Subsequently, a user evaluation involving 6 additional BLV participants was carried out to assess the refined prototype, which received positive feedback.
This research contributes to the expanding realm of studies that focus on harnessing LLM technology to foster inclusivity. By bridging the gap between BLV individuals and drones through advanced interaction techniques, this project aims to enhance the independence and usability of drones for this specific user group. This innovative approach not only showcases the potential of LLMs in creating a more inclusive environment but also highlights the importance of user engagement and feedback in the development process.
About the Author
