Copilot Actions: AI-powered browser automation that does the work for you!

Microsoft It is being done through a program called Copilot Actions 's experimental new feature gives its AI assistant a new capability: instead of just providing information and advice, it performs tasks for you directly on the web. This new feature, which was introduced in Copilot Labs The functionality tested in is designed to bring the Copilot From a chatbot to an AI Agent that automates the browser.

Users can now use simple natural language commands to make the Copilot Complete tasks that used to require multiple manual clicks. For example, you can just tell it to "Use the OpenTable.com Make me a reservation for two at a restaurant" or "Make a reservation for two at the 1800Flowers.com Order a bouquet of flowers on." Currently.Copilot Actions Individuals in the United States, the United Kingdom, Canada, Australia, India and New Zealand have been provided with Microsoft The test is open to users who are logged in to their accounts. It is worth noting that this feature is currently limited to two to three uses per day for free users.

How it works: a "digital laborer" in the cloud

Copilot Actions The implementation is quite innovative. Rather than going through a complex API integration, it's more straightforward to launch a standalone virtual machine in the cloud with the Edge Browser Example. [1, 4] Copilot will act like a human user by analyzing the visual elements of a web page and simulating actions such as mouse clicks, page scrolling, and keyboard input to complete the task. [1]

When the task progresses to the point where personal information needs to be entered (e.g., address, payment method) or a critical decision needs to be made, theCopilot will pause and request the user to intervene. The user can choose to provide information or simply take over the remote browser to complete the operation. Microsoft Emphasizing.Copilot Actions There is no access to passwords or personal data saved by the user's local browser because it operates in an isolated cloud environment.

Furthermore, in order to ensure transparency and control.Copilot Screenshots of the operated pages are captured during the task execution and saved together with the dialog history. Users can view them at any time Copilot You can also pause or completely cancel a task at any time.

Real "agents" or advanced "macros"?

Microsoft The move is in line with the tech industry's hottest trend of "AI agents," which are considered the next big breakthrough in AI after big language models, and are centered on giving AI the ability to understand, plan, and execute complex tasks on its own. [1, 3]

pass (a bill or inspection etc) Copilot Actions(math.) genusMicrosoft demonstrates the path to its realization: the Copilot Positioned as a unified portal to interact with various professional AI agents. [1] The services currently offered, such as travel and food ordering, are just initial attempts. In the future, users may be able to Copilot Studio low-code platforms like this one, create proprietary agents that handle specific business processes and Copilot Calling them in covers a wide range of scenarios from personal assistants to enterprise-level automated processes. [2, 3, 5]

The Security-Privacy Tradeoff

Giving AI access to web operations inevitably raises concerns about security.Microsoft The disclaimer admits that the tool is still in its early stages and could be affected by cyber attacks and other common security risks.

The nature of this model, which allows AI to read and manipulate web content, makes it theoretically exposed to the same security threats as human users. Although Microsoft Strong mitigations are built in and prohibit access to sites that contain offensive or harmful content, but users still need to be vigilant in their use, especially on sites that involve login credentials and sensitive personal information. It's a trade-off between convenience and risk.

Copilot Actions The launch of AI Assistant marks the evolution of mainstream AI assistants from "knowledge providers" to "task performers". While currently limited in functionality and scope, this is a clear sign of what's to come: your AI assistant will not only tell you what to do, it will do it for you.