OpenAI has introduced real-time video analysis and screen sharing features to ChatGPT’s Advanced Voice Mode, marking a significant expansion of the AI chatbot’s capabilities. The new functions, announced during a livestream, allow ChatGPT Plus, Team, and Pro subscribers to interact with the AI through their phone cameras and share their device screens for real-time analysis and assistance.
The video feature enables users to point their phones at objects and receive immediate responses from ChatGPT, while the screen sharing function allows the AI to analyze and provide guidance on device settings, applications, and various on-screen content. According to OpenAI’s demonstration, the system can identify objects, assist with tasks, and provide feedback on user activities in real-time. The rollout begins immediately but excludes users in the EU, Switzerland, Iceland, Norway, and Liechtenstein, with Enterprise and Edu subscribers scheduled to receive access in January.
The development places OpenAI in direct competition with similar offerings from Google’s Project Astra and Microsoft’s Copilot Vision. The company acknowledges that while the system shows promise, it may occasionally make mistakes, as demonstrated during a CNN “60 Minutes” segment where it incorrectly solved a geometry problem.
Sources: TechCrunch, VentureBeat