MIRIX: A Multi-Intelligent Personal Assistant for Intelligent Tracking of Screen Activity

MIRIX is an innovative multi-intelligent personal assistant focused on tracking user screen activity and providing intelligent Q&A services. By capturing real-time visual data, it transforms users' digital actions into structured memories to build a searchable timeline of their digital lives.MIRIX combines six memory types - Core Memory, Situational Memory, Semantic Memory, Program Memory, Resource Memory, and Knowledge Base - and is claimed to achieve a LOCOMO accuracy rate of 85.381 TP3T and an optimization of storage efficiency of 99.91 TP3T. It automatically extracts and organizes information from documents, web pages, and applications that users browse, helping them to quickly review and locate digital activities.Developed by a team from the University of California, San Diego, and New York University, and open-sourced under the Apache 2.0 license, MIRIX is suitable for users who need to efficiently manage their digital lives.

Function List

• Real-time screen activity tracking: Automatically records what users do on the computer, such as browsing the web, reading documents or using applications.
• Intelligent Q&A function: supports users to ask questions through natural language, for example, asking what they read yesterday or where a certain document is saved.
• Structured Memory Generation: Transform on-screen visual data into searchable structured memories to build a timeline of digital activity.
• Information Extraction and Organization: Automatically extracts key information from documents, web pages and applications and stores it categorized by topic.
• Workflow Learning: Record and optimize users' common operating procedures to simplify repetitive tasks.
• Multimodal Memory System: Combines six memory types (core, situational, semantic, procedural, resource, and knowledge base) to improve information retrieval accuracy.
• Open source support: based on the Apache 2.0 license, allowing users to view code, contribute to development, or make suggestions.

Using Help

Installation process

MIRIX provides desktop applications and backend services with a simple installation process for Windows, macOS and Linux. Below are the detailed steps:

1. Visit the official website: Open https://mirix.io/Click on the "Download" button and select the package that matches your operating system (Windows, macOS or Linux).
2. Download & InstallationWindows users may need to allow apps from unknown sources, and macOS users need to authorize them in System Preferences.
3. Initialization settings: When you start MIRIX for the first time, you will be guided through the process of setting up user accounts and permissions. Make sure to grant screen recording and file access permissions in order for MIRIX to capture screen activity.
4. Configuring the backend (optional): If you need to use advanced features (such as cloud backup or multi-device synchronization), follow the docs.mirix.io of the backend configuration guide to download and deploy the backend service. Run the following command to install the backend:

   git clone https://github.com/Mirix-AI/MIRIX.git
   cd MIRIX
   npm install
   npm start
   

5. Updates and Maintenance: MIRIX automatically checks for updates and it is recommended that you visit it regularly! docs.mirix.io Check for the latest version or fix patch.

Usage

At the heart of MIRIX is the ability to track screen activity and provide intelligent Q&A. Below is a guide to operating the main features:

1. Screen activity tracking

MIRIX runs in the background and automatically captures visual data from your screen. When launched, it records the web pages, documents, and applications you open. For example, if you view a web page about machine learning, MIRIX records the page title, URL, and key content. You don't need to save it manually, the system will automatically generate a timeline. Users can view the most recent activity in the main interface, and the timeline is sorted by date and application type.

2. Intelligent Question and Answer

MIRIX supports natural language questions. Open the main interface, click the "Ask MIRIX" button, and enter a question such as:昨天我读了什么关于机器学习的文章？ maybe 我把 PostgreSQL 文档保存在哪里？. The system analyzes the memory bank and returns relevant answers. The answers usually include the time, location of the file or a link to a web page. If the results are inaccurate, keywords can be adjusted using the "Refine Search" option.

3. Extraction and organization of information

MIRIX extracts key information from documents and web pages. For example, when reading a PDF, it automatically recognizes the title, summary and keywords and stores them as semantic memory. Users can view categorized information in the Memory Hub in categories such as documents, web pages and applications. Click on a record to view details or jump directly to the source file.

4. Workflow optimization

MIRIX learns from repeated user actions. For example, if you often edit a document in specific steps, MIRIX records these steps and generates shortcut templates. You can select a template in the "Workflows" panel and click "Apply" to automate the task. Templates can be customized for complex tasks such as batch file organization.

5. Memory management

MIRIX's Memory Hub supports searching, backup and deletion. Open "Memory Hub", enter keywords to search for memories, or use filters to find memories by time and type. For backup function, you need to configure the backend service, run the following command:

mirix backup --path /your/backup/directory

Regularly clearing useless memory can optimize performance, click "Settings > Clear Memory" to select a time frame for deletion.

6. Community contributions

MIRIX is an open source project and users can access the github.com/Mirix-AI/MIRIX Check out the code or submit a suggestion for improvement. If you find a problem, click on the "Issue" page in your GitHub repository to describe the problem and attach a log file. You can contact us at yuw164@ucsd.eduThe

Featured Function Operation

• multimodal memory: MIRIX's six memory types work together. For example, situational memory records the time and context when you open a web page, semantic memory extracts keywords from a web page, and procedural memory saves steps. Users can switch to different memory perspectives through "Memory Type".
• Real-time visual data processing: MIRIX uses visual data analysis technology to recognize text and images on the screen. The capture frequency (default once per second) can be adjusted in the settings to reduce the performance footprint.
• PrivacyMIRIX provides local storage option and data is not uploaded to the cloud by default. Users can enable encryption in "Settings > Security" to secure sensitive information.

More details of use can be found in docs.mirix.io A complete guide covering backups, performance optimization, and security settings. [](https://docs.mirix.io/)[](https://github.com/Mirix-AI/MIRIX)

application scenario

1. academic research
   Researchers can use MIRIX to track papers and web pages read and quickly find key points in a particular document. For example, type 上周我读了哪些关于深度学习的文章？MIRIX lists the relevant records to save searching time.
2. daily office hours
   Workers can use MIRIX to document workflows, such as organizing reports or managing emails, and MIRIX generates shortcut templates to simplify repetitive tasks and increase efficiency.
3. Personal knowledge management
   Students or self-learners can organize their learning resources with MIRIX. The system automatically extracts key information from notes, web pages and videos to build a personal knowledge base for easy review.
4. project management
   Team members can use MIRIX to record project-related documents and discussions. Inquiries 上个月的项目计划在哪里？You can quickly locate the file.

QA

1. How does MIRIX ensure data privacy?
   MIRIX stores data locally by default and requires users to manually enable cloud backup. AES-256 encryption can be enabled in the settings to protect sensitive information.
2. Does it support multi-device synchronization?
   Yes, after configuring the back-end service, MIRIX supports multi-device synchronization. You need to login to the same account on each device and enable the cloud service.
3. What are the system requirements for MIRIX?
   MIRIX supports Windows 10+, macOS 10.15+ and major Linux distributions. A minimum of 8GB of RAM and 2GB of storage is recommended.
4. How can I get involved in MIRIX development?
   interviews github.com/Mirix-AI/MIRIXIf you have any questions, please feel free to contact us, clone the codebase, submit a Pull Request, or send us feedback on the Issue page.