Local automated operation realization process
The local operation feature of UI-TARS-desktop is its most basic and revolutionary feature. To automate a local computer, the user only needs to follow a simple workflow:
1. Initiation and set-up phase:
The application first needs to be installed correctly, by executing the .exe or .msi installer for Windows users, or via a .dmg file for Mac users. Upon startup, the system will automatically detect the local environment and complete the initialization setup.
2. Command input phase:
After selecting the "Local Operation" mode in the main interface, users can directly input natural language commands into the text box. Note that:
- Instructions should be as specific as possible (e.g., "open the project folder on the D drive" rather than "open that folder").
- Complex multi-step tasks are recommended to be broken down into a few simple instructions
- Can include preemptive actions to start the application (e.g. "Open Excel first")
3. Implementation monitoring phase:
After pressing the Execute button, the user will observe:
- Mouse pointer automatically moves and clicks on the target position according to the AI's understanding
- The system interface displays the current execution steps and progress in real time
- Pause and wait for user confirmation when encountering problems
Throughout the process, the AI will continue to understand the interface state changes through the screen shots and dynamically adjust the operation strategy to ensure that the task is completed accurately.
This answer comes from the articleUI-TARS Desktop: Desktop Intelligentsia Application for Computer Control Using Natural LanguageThe
English 
简体中文 
日本語 
Deutsch 
Português do Brasil