Google’s efforts to use AI underpinned by LLMs to automate user tasks is very similar to the “computer use” ability released by Anthropic last week, which experts believe could revolutionize the automation market once rolled out as a finished product as a huge amount of work continues to be done over computers.
Anthropic’s “computer use” ability, in turn, enables developers to instruct Claude 3.5 Sonnet, through the Anthropic API, to read and interpret what’s on the display, type text, move the cursor, click buttons, and switch between windows or applications — much as today’s robotic process automation (RPA) tools can be instructed — much more laboriously — to do.
While Jarvis seems to be aimed at consumers, the technology could also be used across enterprises given that many development activities, workflow and automation management, CRM, ERP, etc are accessed over the browser via web-based clients or interfaces.