SQLBot: The Intelligent Bot That Converts Natural Language to SQL Queries

SQLBot is an intelligent data query system that combines Large Language Modeling (LLM) and Retrieval Augmentation Generation (RAG) technologies. As a functional module of the open source Business Intelligence (BI) tool DataEase, it mainly solves the problem of ordinary business people who are unfamiliar with the SQL language and find it difficult to get data directly from the database. Users can ask questions through everyday speech , SQLBot can understand the intent of the question , and automatically generate accurate SQL query code , and ultimately return query results or visual charts . The tool is supported out-of-the-box and can be launched with a simple configuration of the big model and data source. It is also easy to integrate and can be embedded into a variety of third-party business systems, empowering these applications with conversational data analytics. In addition, SQLBot provides workspace-based resource isolation and fine-grained data permission control to ensure the security of data query process.

Function List

• Natural Language to SQL: Core functionality that automatically converts a user's natural language question (Text) into a structured query language (SQL).
• RAG Enhancement: Integrate Retrieval Augmented Generation (RAG) techniques to improve the accuracy of SQL generation for large models in specific data environments.
• Data Source Support: Can connect and query a variety of mainstream databases, such as MySQL.
• model management: Supports access to and management of multiple large-scale language models that can be configured and switched by the user on demand.
• privilege control: Provide workspace-level resource isolation and fine-grained data rights management to safeguard data security.
• Easy to integrate: Supports rapid embedding into third-party business systems as plug-ins or modules, as well as integration by n8n, Dify, Coze and other AI application development platforms.
• out-of-the-box: Provides one-click installation scripts and Docker deployment methods to simplify the deployment and configuration process.

Using Help

The process of installing and using SQLBot is very straightforward and is divided into four main steps: environment preparation, service startup, initial configuration, and getting started.

1. Environment preparation and installation

Before installing SQLBot, your server needs to have theDockercap (a poem)Docker Compose. These two tools are the foundation of containerized deployment.

Once the preparation is complete, you can follow the following commands to perform a one-click installation:

1. Create an installation directory and go to
   Choose a suitable location on your Linux server, for example/opt/sqlbotand create the catalog.

   mkdir -p /opt/sqlbot
   cd /opt/sqlbot
   

2. Download Configuration File
   utilizationcurlCommand to download from the official GitHub repositorydocker-compose.yamlfile. This file defines the SQLBot service and its dependent environments.

   curl -o docker-compose.yaml https://raw.githubusercontent.com/dataease/SQLBot/main/docker-compose.yaml
   

3. Starting services
   Execute the following command and Docker Compose will automatically pull the image and start the SQLBot service based on the configuration file.-dparameter indicates running in the background.

   docker compose up -d
   

   You can also pass the1PanelApp Store for visual one-click deployment.

2. Visits and logins

Once the service has started, access in your browser the IP address of the deployment server and the8000Ports:http://<你的服务器IP>:8000/The

The default system administrator account information is as follows:

• user ID: admin
• cryptographic: SQLBot@123456

It is recommended to change the default password after the first login.

3. Core configuration

After successfully logging in, two core configurations are required to start using the Smart Ask Count feature:Configuring the Big Modelcap (a poem)Adding a Data SourceThe

1. Configuring the Big Model
   • After entering the system, find the menu related to "Model Management".
   • Click "New" or "Add Model" and select the model you want to use (e.g., GPT, Wenxin Yiyin, Tongyi Qianqian, or locally deployed models).
   • Follow the interface prompts to fill in the model's API Key, Base URL, and other authentication information.
   • Once saved, the model is added to the system. You can add multiple models and set one to be used as the default model for queries.
2. Adding a Data Source
   • Switch to the "Data Sources" menu and click "New Data Source".
   • Select your data source type from the list of supported database types, e.g. "MySQL".
   • Fill in the connection information for this data source, including:
     • name (of a thing): Give this data source an easily recognizable name, such as "Production Environment Sales Database".
     • Hostname/IP address: The address of the database server.
     • ports: Database service port.
     • database name: The name of the specific database to which you need to connect.
     • Username/Password: The account and password used for database authentication.
   • After filling in the form, you can click the "Test" button to test whether the connection is successful or not. After passing the test, click "Save".

4. Starting smart counting

After completing the above configuration, you can start using it.

• Go to the Question and Answer screen (often called "Conversation" or "Q&A").
• In the input box, describe the data you want to query in natural language. For example, you might enter, "Help me find out what the top five highest selling products were last month."
• SQLBot sends your problem to a configured large model for processing, generates SQL statements, and automatically executes the query.
• Query results are displayed directly on the interface in the form of tables, cards or charts, allowing you to get quick access to data insights.

application scenario

1. Self-service data analysis for business people
   For employees in marketing, operations, sales, etc., who usually do not have the ability to write SQL, SQLBot allows them to ask questions directly to the database through natural language, such as "Query the growth trend of new users in the last 30 days" or "Statistics on the percentage of sales of various types of commodities in East China", thus realizing self-service query and analysis of data and greatly improving the efficiency of data-driven decision-making. SQLBot allows them to ask questions directly to the database through natural language, such as "query the growth trend of new users in the last 30 days" or "count the sales volume of various products in East China", thus realizing self-service query and analysis of data and greatly improving the efficiency of data-driven decision-making.
2. Rapid Data Exploration
   Data analysts need to frequently write query statements with different dimensions when performing exploratory data analysis (EDA). With SQLBot, analysts can quickly validate data hypotheses through conversations such as "Is there a difference in average unit price between male and female users?" , thus putting more effort into data insights rather than code writing.
3. Embedded Analytics
   It is possible to integrate SQLBot's Q&A capabilities into an enterprise's existing business systems (e.g. CRM, ERP). When using these systems, users do not need to jump to a dedicated BI platform, they can query relevant data directly through the dialog window, realizing the seamless integration of data analysis capabilities and business processes.

QA

1. What types of large models does SQLBot support?
   SQLBot is open by design and supports access to the mainstream large language models on the market, including OpenAI's GPT series, Baidu's Wenxin Yiyin, Ali's Tongyi Qianqian, and so on. In addition, it also supports access to open source models deployed locally through tools such as Ollama.
2. Do I have to install DataEase first to use SQLBot?
   SQLBot can be considered part of the DataEase ecosystem and is best experienced when used in conjunction with it. But it is also a system in its own right that can be deployed and integrated independently, and you can integrate it into your other business platforms.
3. How does it ensure the accuracy of the generated SQL?
   SQLBot utilizes RAG (Retrieval Augmented Generation) technology. Before sending a user problem to the big model, the system retrieves the most relevant information about the problem from the database metadata (e.g., table structure, field names, comments, etc.) and provides this information to the big model along with it as a Context. This greatly improves the model's ability to understand data structures and thus generate more accurate SQL.
4. How is data security ensured?
   SQLBot provides a workspace-level resource isolation mechanism, where data sources and Q&A history of different spaces are isolated from each other. At the same time, administrators can perform fine-grained data permission control to ensure that users can only query data within their authority to avoid data leakage.