Dataherald is an enterprise-level natural language to SQL AI conversion engine that allows non-technical users to directly query the database through conversation without writing SQL statements.

Dataherald

Core parameters and statistics of Dataherald

Dataherald is officially positioned as an enterprise-level natural language to SQL conversion engine. Its core value is to allow non-technical users to directly query relational databases through daily conversations, without the need for the data team to write SQL statements. The fundamental difference between it and traditional BI tools is that it does not rely on preset dashboards or fixed report templates, but analyzes user intentions in real time and dynamically generates query statements, and supports multiple rounds of dialogue to gradually refine requirements.

Projects	Public Information
Official positioning	Enterprise-grade Natural Language to SQL engine
Core capabilities	NL2SQL, multi-round dialogue context, complex SQL generation (JOIN/subquery/aggregation/window function)
Supported databases	PostgreSQL, MySQL, BigQuery, Snowflake, Databricks, MS SQL Server, ClickHouse, MariaDB, Redshift
Vector Storage	Pinecone, Astra, Chroma
Deployment method	Self-hosting (Docker Compose), cloud hosting (Enterprise Edition)
Input method	Natural language (mainly English)
Output format	SQL statement + query result + CSV export
Open Source License	Apache-2.0
GitHub Stars	~3,600
GitHub Forks	~264
Code Contributors	19
Latest version	v1.0.3 (2024-04-30, GitHub Releases)
Total releases	9 versions
Core languages	Python (58.5%), TypeScript (39.3%)
System Components	Engine, Enterprise API, Admin Console, Slackbot

Deployment architecture: Dataherald adopts a microservice architecture, including four independent components: Engine (core NL2SQL engine), Enterprise (user/organization/authentication management), Admin Console (management interface) and Slackbot (Slack integration). Each component is orchestrated uniformly through Docker Compose, supporting split deployment on demand.

Breadth of database coverage: From v0.0.1 to v1.0.3, Dataherald has gradually integrated 9 types of relational databases and 3 types of vector storage, covering mainstream OLTP (MySQL, PostgreSQL, SQL Server), OLAP (ClickHouse, Redshift) and cloud data warehouses (BigQuery, Snowflake, Databricks). This is its key differentiator from the NL2SQL solution that only supports a single type of database.

Iteration rhythm: The first public release v0.0.1 (2023-08), v1.0.0 (2024-01), v1.0.3 (2024-04), after which the GitHub commit frequency dropped significantly and is currently in a maintenance period. When selecting, you need to evaluate community activity and long-term support risks.

Dataherald’s users and market recognition

Dataherald’s market recognition is mainly reflected in open source community feedback and enterprise PoC verification. The official has not disclosed specific revenue data, number of paying customers or SLA commitment details.

GitHub community popularity: 3,600+ stars and 264 forks, which is at an upper-middle level in the NL2SQL open source track. Comparison of similar projects: SQLChat has about 4k stars, Vanna has about 12k stars, and DB-GPT has about 14k stars. Dataherald is characterized by providing a complete set of four components (Engine + Enterprise + Admin Console + Slackbot), which is closer to an enterprise-level delivery form than most projects that only provide a core inference engine.

Enterprise verification scenario: Typical use cases highlighted in the official documentation and GitHub README include embedded Q&A capabilities within SaaS, natural language counting robots based on Slack, and self-service counting portals for business teams. These use cases are aimed at medium and large enterprises that have invested in data warehouses but lack analytical manpower, rather than small and micro teams.

Ecological cooperation: The project integrates LangSmith for observability, supports Pinecone/Astra/Chroma three vector databases as Schema context storage, and can be connected with mainstream LLM services (OpenAI GPT series Anthropic Claude, self-hosted models). This shows that it is designed to remain model-agnostic and not tied to a single AI vendor.

Adoption Prerequisites: The real value release of Dataherald requires that the enterprise already has ① structured relational data assets, ② clear Schema documents or golden query samples (Golden SQLs), and ③ IT teams are willing to maintain additional self-hosted infrastructure. Without any one of them, the landing effect will be significantly reduced.

Cost Advantages of Dataherald

Dataherald's cost structure needs to be examined separately from the three levels of "C-side/individual users", "API/developer integration", and "enterprise/privatized deployment".

C client/individual users:

• Explicit Cost: The open source community edition is completely free, and the Apache-2.0 license allows arbitrary use, modification and redistribution. Individual developers only need to bear the operating costs of their own servers (running 4 containers in Docker Compose mode, and the estimated minimum configuration is 4 cores and 8G memory).
• Hidden Cost: You need to configure the LLM API Key (such as OpenAI, Anthropic) by yourself, and the LLM call fee is billed by token. A typical query including Schema scanning + SQL generation consumes about 2,000-8,000 tokens, which increases with the query complexity. In scenarios with frequent calls, LLM API overhead can quickly exceed infrastructure costs.

API/Developer Integration:

• REST API layer: The open source versions of Engine and Enterprise components provide a complete RESTful API (including prompts, sql-generations, nl-generations, finetuning and other endpoints), which developers can integrate into their own applications for free.
• Tuning Cost: Dataherald supports fine-tuning (Finetuning) based on Golden SQLs, but the fine-tuning process consumes OpenAI training credits and requires the preparation of high-quality Question-SQL paired samples. The recommended number of samples is 50-200, and the cost of a single fine-tuning is on the order of tens of dollars.
• Implicit integration costs: You need to manually configure the Schema description for each database connection (or run an automatic scan), maintain the Golden SQLs sample library, and deal with the hidden logic that the LLM-generated SQL does not meet expectations. These engineering efforts are often greater than the cost of the API calls themselves.

Enterprise/Private Deployment:

• Enterprise Edition Pricing: The official price of the Enterprise Edition has not been disclosed. According to industry practice, it is speculated that a subscription system is adopted. The billing dimensions usually include: number of database connections, API call quota, and user seat SLA level. Enterprise Edition added value includes SSO integration, audit logging, and dedicated SLA support.
• Infrastructure Cost: Private deployment requires the enterprise to manage Docker to run the bounded MongoDB database, vector database and network configuration. At a moderate load of 1,000 queries per day, the monthly infrastructure cost is estimated to be US$100-500 (cloud host + vector storage + bandwidth).
• Human operation and maintenance cost: At least one development or operation and maintenance personnel familiar with Docker and LLM calls is required to be responsible for system maintenance, Golden SQLs management and query quality monitoring. This hidden cost is typically 3-5 times the cost of infrastructure.

Cost comparison: NL2SQL open source solution

Solution	Open Source License	Deployment Complexity	Database Coverage	Fine-tuning Support	Enterprise Features	Community Activity
Dataherald	Apache-2.0	Medium (4 components Docker)	9 types of DB + 3 types of vector storage	✅ Built-in Finetuning API	✅ Admin Console + Slackbot + Enterprise	Medium (3.6k stars)
Vanna	MIT	Low (Python library)	Mainly supports SQLite/PG	✅ Trained via DDL documents	❌ None	High (12k stars)
SQLChat	MIT	Low (Node library)	Mainly supports MySQL/PG	❌ None	❌ None	Medium (4k stars)
DB-GPT	Apache-2.0	High (Multiple components)	Multiple DBs	✅ Support	✅ Full enterprise features	High (14k stars)

Cost Difference Highlights:

• Dataherald provides the most complete enterprise-level package in the open source NL2SQL solution (Admin console Slack integration, multi-tenancy, audit log ready), suitable for teams that need "out-of-the-box" rather than building from scratch.
• If you already have LLM API credits and have low tolerance for deployment complexity, Vanna (single Python library installation) or SQLChat (Node.js package) have lower initial startup costs.
• DB-GPT leads in terms of functional completeness and community scale, but its deployment complexity is also higher, and it is mainly optimized for Chinese scenarios, which is different from Dataherald's English-first positioning.

Main functions of Dataherald

• Natural Language to SQL (NL2SQL): Enter "Sales Ranking by Region in the Last Quarter", the engine automatically recognizes the aggregation intent and generates SQL statements with GROUP BY and ORDER BY. The core mechanism is to map user questions to database schema through LLM, and then combine the conversation context to synthesize executable SQL. The difference from traditional BI tools is that there is no preset report structure and users can freely describe any query dimensions.

• Multiple rounds of dialogue context preservation: Continue to ask questions based on the first query result (such as "See only East China" or "Change to display by month"). The engine retains the filter conditions and aggregation logic of the pre-order SQL, and only incrementally modifies the WHERE clause or GROUP BY field. The actual value of this to business users is that there is no need to describe the complete requirements at once, and the scope of the query can be gradually narrowed down like a conversation with a human. The interactive rounds of a single analysis task are usually extended from 1 round to 3-5 rounds, but each round remains semantically coherent.

• Database Schema automatic awareness: The engine automatically scans the database table structure, field names, field types, primary and foreign key relationships, and automatically matches field aliases and associated keys when generating SQL. Schema scan results are stored in MongoDB and vector databases. When generating SQL, LLM only retrieves tables and fields related to user questions as context, avoiding the entire Schema being stuffed into Prompt and causing token expansion.

• Natural language explanation of query results (NL Generation): For the generated SQL statements and execution results, the engine automatically generates natural language explanations to explain "what filtering was done for this SQL, what dimensions were aggregated by, and what is the basis for sorting". The key value of this feature for non-technical users is that even if they cannot understand SQL, they can understand whether the query logic is correct, thereby establishing trust in the results generated by AI.

• Golden SQLs management and model fine-tuning: Supports storing verified "question-SQL" pairs into the Golden SQLs collection, and automatically fine-tuning (Finetuning) GPT series models based on these samples. The accuracy of the fine-tuned model on similar business queries is significantly improved. This is a "more accurate the more you use it" mechanism: in the initial stage, it relies on the knowledge of general LLM, and as the company accumulates proprietary query samples, the accuracy gradually converges to 90%+.

• Intermediate step visualization (Streaming): The Streaming endpoint introduced in v1.0.2 shows the intermediate reasoning steps of SQL generation - from Schema retrieval to SQL synthesis to result execution - allowing users and developers to see the AI ​​thinking chain, facilitating debugging and trust building.

• Slack integrated query robot: Through the Slackbot component, users can directly use natural language to ask questions to the database in the Slack channel, and the robot returns query results or CSV files. This is especially useful for operations, marketing, and sales teams without technical backgrounds: there is no need to open any BI tools to complete data acquisition in daily workflows.

• CSV export and file storage: Query results can be directly exported to CSV and stored in S3 (by configuring AWS credentials), which is suitable for subsequent secondary analysis in Excel or Google Sheets. Automatically use file storage when the number exceeds 50 lines to prevent the API response payload from being too large.

Dataherald’s model and version evolution

Dataherald’s version history clearly reflects the evolutionary path from prototype verification to enterprise functionality improvement to ecological expansion. The following is compiled based on GitHub Releases public information.

Mainline release

Version	Release Date	Core Changes	Milestones
v0.0.1	2023-08	Initial version, basic NL2SQL query function	Project approval, MVP verification
v0.0.2	2023-09-14	RESTful endpoint reconstruction, MongoDB collection name standardization, introduction of db_connection_id association	API structure finalization, moving from rapid prototyping to standardization
v0.0.3	2023-09-26	LLM Credentials supports SSH connection optimization and asynchronous Schema scanning	Enterprise connection capabilities are enhanced and scanning performance is optimized
v0.0.4	2023-10-07	Endpoint rename ObjectId foreign key NL generation split	API semantics clear, prepare for 1.0
v0.0.5	2023-10-26	llm_api_key field simplified S3 CSV storage, error code system	Simplified configuration and improved observability
v0.0.6	2023-11-14	CSV generation flag S3 certificate configurable	Improved data export capabilities
v1.0.0	2024-01-17	Finetuning API, Prompt/SQL-Generation/NL-Generation three-phase split Golden SQLs collection	Architecture Maturity Milestone
v1.0.1	2024-03-05	ClickHouse supports MariaDB official support, refresh endpoint, error code refinement	Database coverage expansion
v1.0.2	2024-04-04	MS SQL Server, Astra/Pinecone serverless support Streaming intermediate step LangSmith integration	Ecological connection and observability enhancement
v1.0.3	2024-04-30	Redshift support, multiple Schema support (PG/BigQuery/Snowflake/Databricks)	The latest version, perfect for enterprise multi-schema scenarios

Interpretation of evolutionary context

Phase 1: Prototype verification (v0.0.1-v0.0.2): The first two versions mainly completed the end-to-end process of "from natural language to SQL". The RESTful API refactoring in v0.0.2 lays the foundation for all subsequent enterprise features.

Phase 2: Enterprise connection capability building (v0.0.3-v0.0.6): Gradually complete SSH connections, multiple databases supporting CSV export to S3 storage, error code systems and other enterprise-necessary but non-core AI capabilities. This stage shows that the Dataherald team realizes that the obstacle to the implementation of NL2SQL in enterprises is not only AI accuracy, but also data connectivity and operation and maintenance observability.

Phase Three: 1.0 Architecture Maturity (v1.0.0): v1.0.0 is a major architectural change that splits the original single "question → answer" process into a three-stage pipeline - Prompt (problem understanding) → SQL Generation (SQL synthesis) → NL Generation (result interpretation), and introduces Finetuning API. The three-stage split allows each section to be independently optimized, cached, and audited independently, which is a key design decision for enterprise-level deployment.

Phase 4: Ecological expansion and maintenance (v1.0.1-v1.0.3): Focus on expanding database coverage (ClickHouse, MariaDB, SQL Server, Redshift) and vector storage options (Astra, Pinecone serverless), while improving observability through Streaming endpoints. After v1.0.3, the project entered a low-active maintenance period, and no new feature versions were released.

Candidate Verification and Community Contribution

In addition to mainline releases, Dataherald drives participation from 19 contributors through Pull Requests and Issues, covering bug fixes, documentation improvements, and minor feature enhancements. But overall, the core development of the project is led internally by the team, and community contributors mainly focus on documentation and marginal functions.

Version Strategy Evaluation: Dataherald's version naming follows the semantic version specification (SemVer), but it only took 8 months from v0.0.1 to v1.0.3, and then stagnated. When selecting, you need to evaluate: whether the current functions meet the needs, and whether you are willing to accept the risk of community fork or self-maintenance.

Dataherald’s technical advantages

Dataherald's technical advantage does not lie in the breakthrough of a single algorithm, but in the engineering architecture design - how to encapsulate LLM's NL2SQL capabilities into an enterprise-level system that can be implemented, observable, and iterable.

Three-stage pipeline architecture

Dataherald splits a natural language query processing into three independent stages:

User input → [Prompt] → [SQL Generation] → [NL Generation] → User output
                 ↓ ↓
           Schema vector retrieval Golden SQLs matching

• Prompt stage: Receive user natural language input, combine the conversation history (if any) and relevant Schema information retrieved from the vector database, and assemble it into an LLM-friendly prompt. The key optimization is: instead of injecting the entire database schema at once, only the tables and fields most relevant to the user's problem are selected through vector similarity retrieval, which greatly reduces token consumption and reduces the distraction of LLM.
• SQL Generation Phase: Send the assembled Prompt to LLM to generate SQL. If the Golden SQLs fine-tuning model is configured, use the fine-tuning model first to improve accuracy; otherwise, fall back to the general model. The Streaming endpoint introduced in v1.0.2 allows real-time viewing of intermediate reasoning steps of SQL generation - LLM's thought chain, field matching process JOIN condition selection - which is critical for debugging and trust building.
• NL Generation phase: Use natural language to explain to the user "what this query did" for the generated SQL and execution results. This is an underestimated but extremely valuable design: non-technical users usually cannot read SQL, but through natural language interpretation they can quickly judge whether the query logic is correct and decide whether to accept the results.

Collaboration of Schema awareness and vector retrieval

Dataherald's Schema handling mechanism is the core dividing line between it and the simple Prompt wrapper:

1. Automatic scanning: Scan the database through the asynchronous background task of the POST /api/v1/table-descriptions/sync-schemas endpoint to obtain table names, field names, field types, comments, and primary and foreign key relationships.
2. Schema vectorized storage: Vectorize the table and field description information (name + comment) using the Embedding model and store it in the Pinecone/Astra/Chroma vector database.
3. Runtime retrieval: When the user asks a question, first do Embedding on the question, retrieve the Top-K related tables and fields in the vector library, and only inject these contexts into LLM Prompt.
4. Incremental caching: Scan results are cached in MongoDB, supporting incremental updates rather than full reconstruction. The POST /api/v1/table-descriptions/refresh endpoint (introduced in v1.0.1) is designed to efficiently refresh the table list without rescanning all data.

The engineering significance of this mechanism is that enterprise databases often have hundreds of tables and thousands of fields. If all of them are injected into the LLM context, token consumption will be unacceptable and LLM will be seriously distracted. Vector retrieval + dynamic injection control the Schema context of each query within 3-8 tables, taking into account both accuracy and cost.

Golden SQLs closed with iteration

Dataherald's Finetuning mechanism constitutes a continuous optimization process:

Business query → SQL generation → Manual review → Store in Golden SQLs → Fine-tune the model → Improve accuracy
                                                      ↑
                                              Trigger Finetuning periodically

• Golden SQLs Collection: Stores validated "Natural Language Question ↔ Standard SQL" pairs. Each pair contains question, sql, db_connection_id and metadata.
• Fine-tuning process: Call POST /api/v1/finetuning to create a fine-tuning task, and the engine automatically formats Golden SQLs into the data set format required for OpenAI fine-tuning and submits it. After fine-tuning is completed, you can query the status through GET /api/v1/finetuning/{id}. If the status is SUCCEEDED, it can be used for SQL generation.
• Actual results: According to the official documentation, the fine-tuned model significantly improves the accuracy of SQL generation in proprietary business domains. Although the precise value is not disclosed, it is logically reasonable: the general model may understand that "sales" is SUM(amount), but does not understand the enterprise-specific "net sales = SUM(amount)-SUM(discount)-SUM(return)"; after fine-tuning, the model can learn these business rules.

Model independence and replaceability

Dataherald maintains the abstraction of the underlying LLM at the architectural level: the engine accesses different models through configuration interfaces and is not bound to a single OpenAI supplier. Official support includes GPT-4/GPT-3.5, Claude series, and self-hosted models (via local deployment compatible with the OpenAI API format). This design has practical value in enterprise procurement: you can use GPT-4 to do the upper limit of PoC verification accuracy, and switch to a self-hosted model after going online to reduce inference costs and control data sovereignty.

Multi-tenancy and permission isolation

The Enterprise component provides organization-wide user management, role permissions, and database connection isolation. Each database connection can be configured with an independent LLM API Key, supporting read-only mode (preventing UPDATE/DELETE/DDL statements from being generated) and data desensitization. These mechanisms are strictly necessary in multi-department or multi-customer scenarios - different departments can only query tables and data within the authorization scope.

How to use Dataherald

Dataherald provides multiple entrances and integration methods, covering different usage scenarios from developer API integration to business team Slack interaction.

Deployment entry comparison

Entrance	Applicable people	Startup method	Pre-requisites
Engine API (core engine)	Developer	Docker Compose runs Engine service	Docker, MongoDB, LLM API Key
Enterprise API (Full Featured)	Developer/IT Admin	Docker Compose runs all 4 services	Docker, MongoDB, Vector Database LLM API Key
Admin Console (administrative interface)	Data Analyst/Administrator	Launched with Enterprise, browser access	Enterprise API running
Slackbot	Business Teams	Launched with Enterprise, Slack app configuration	Enterprise API + Slack app permissions
REST API	Developer	Call Engine/Enterprise endpoint directly	Deployed API Base URL

Rapid deployment and startup (self-hosted)

Minimum configuration requirements (PoC level):

# 1. Clone the repository
git clone https://github.com/Dataherald/dataherald.git
cddataherald

# 2. Configure contextual variables (refer to .env.example in each service directory)
# At least configuration required: OPENAI_API_KEY, MONGODB_URI

# 3. Start all services with one click
./docker-run.sh

The above command will start Engine (port 80), Enterprise (port 81), Admin Console (port 3000), and Slackbot, and automatically create a Docker network. After startup, the management console can be accessed through http://localhost:3000.

API call example

Create database connection:

curl -X POST http://localhost:80/api/v1/database-connections \
  -H "Content-Type: application/json" \
  -d '{
    "alias": "production_db",
    "connection_uri": "postgresql://user:password@host:5432/mydb",
    "llm_api_key": "<YOUR_LLM_API_KEY>"
  }'

Synchronize Schema:

curl -X POST http://localhost:80/api/v1/table-descriptions/sync-schemas \
  -H "Content-Type: application/json" \
  -d '{"db_connection_id": "<connection_id>"}'

Initiate a natural language query:

curl -X POST http://localhost:80/api/v1/prompts/sql-generations \
  -H "Content-Type: application/json" \
  -d '{
    "db_connection_id": "<connection_id>",
    "question": "Sales ranking by region last quarter"
  }'

Fine-tuned model:

curl -X POST http://localhost:80/api/v1/finetuning \
  -H "Content-Type: application/json" \
  -d '{
    "db_connection_id": "<connection_id>",
    "golden_sql_ids": ["<id1>", "<id2>"]
  }'

Typical usage process

1. Initialization: Deploy service → Create database connection → Synchronize Schema → Confirm that the scan status is SYNCHRONIZED.
2. Verification: Submit several basic queries (simple SELECT, conditional filtering) to check the generation quality and execution correctness.
3. Accumulate samples: For high-frequency business queries, store the verified Question-SQL pairs into Golden SQLs.
4. Finetuning: Finetuning is triggered after accumulating 50+ samples to improve vertical domain accuracy.
5. Go online: Configure Admin Console role permissions → Open to business teams → Monitor query logs and error rates.
6. Iteration: Regularly review query logs, add new query patterns to Golden SQLs, and continue fine-tuning.

Preset Notes

• If the user's database table and field names are in non-English (such as Chinese), Dataherald's Schema scanning and LLM understanding will be significantly reduced - this is one of the main language limitations of the current version.
• For production users, it is recommended to enable read-only mode first, and then relax permissions after confirming that SQL generation will not cause unexpected UPDATE/DELETE operations.
• Schema automatic scanning may take several minutes for large libraries. The /refresh endpoint introduced in v1.0.1 can significantly shorten the incremental update time.

Product Pricing for Dataherald

Dataherald's pricing system is divided into two paths: open source community version and enterprise commercial version. The official price of the enterprise version has not been disclosed.

Open Source Community Edition (Apache-2.0):

• Fee: Completely free, no limit on the number of users, query volume or database connections.
• Content included: All source code of Engine (core engine) + Enterprise (multi-tenant API) + Admin Console (management interface) + Slackbot (Slack integration).
• Applicable conditions: You need your own server or cloud host to run Docker Compose, and configure MongoDB, vector database and LLM API Key by yourself.
• Commercial Use Restrictions: The Apache-2.0 license allows free use and modification, but the product is not allowed to be redistributed directly as a SaaS service (subject to the license terms).

Enterprise Edition (undisclosed pricing):

• Estimated to include: SSO (SAML/OIDC) integration, audit logs, dedicated SLA support, priority technical support, enterprise-grade deployment guide.
• Billing dimension speculation: A combined subscription model based on the number of database connections + monthly API calls + number of user seats. Referring to similar open source commercialization projects (such as N8n, Appsmith), the annual fee for the enterprise version may be in the range of $5,000-$50,000, but this is only an industry inference, and the official quotation shall prevail.
• Acquisition method: You need to contact the official sales team to obtain a quotation and trial. The official website does not provide a self-service purchase entrance.

LLM call charges (separate from Dataherald product charges):

• This is an additional cost for using Dataherald and is directly dependent on the user's choice of LLM provider and the volume of calls.
• A typical NL2SQL query of GPT-4 consumes approximately 2,000-5,000 tokens (input Schema + question), and is priced at approximately $0.01-0.03/time based on GPT-4. The monthly LLM fee for high-frequency scenarios (average 10,000 times per day) is $3,000-$9,000.
• Using GPT-3.5-Turbo or a self-hosted model can reduce this cost by 10-30x, at the possible expense of build accuracy.
• It is recommended that the cost of LLM calls be reserved in the budget model, which usually exceeds the cost of Dataherald's own infrastructure.

Application scenarios of Dataherald

Scenario 1: The business team collects data by themselves

Task Description: Non-technical teams such as market operations, sales management, and financial analysis need to frequently obtain reports from the data warehouse. The traditional process requires ① Applying for reports in the BI tool → ② Waiting for the data warehouse team to schedule → ③ Repeatedly communicating on demand details → ④ Obtaining static reports. Dataherald simplifies it as follows: users ask questions in natural language directly in Slack or the Admin Console and get query results instantly.

Actual income:

• The single query cycle is shortened from an average of 4-6 hours to 1-3 minutes (deduction).
• The data team is freed from the repetitive "write SQL-modify SQL" and focus on data modeling and governance.
• Business teams can freely explore data without waiting for scheduling, and decision-making response speed is improved.

Focus of implementation verification: Whether business users are willing to change the habit of "waiting for reports" and actively ask questions in natural language; and whether the first generation accuracy rate of common queries reaches 70%+ (below this value will cause users to give up).

Scenario 2: SaaS product embedded data Q&A capabilities

Task Description: Data-intensive SaaS products such as CRM, ERP, and project management hope to allow end users to query in-product data through natural language instead of groping through complex filtering interfaces. Dataherald's Engine API can be embedded as a product's "data analysis assistant" feature.

Actual income:

• Reduce user learning costs - no need to learn filter syntax, you can get data by asking questions in your native language.
• Reduce the development and maintenance work of preset reports in the product - dynamic generation replaces fixed reports.
• Improve user stickiness and data activity, turning passive viewing into active and passive exploration.

Focus of implementation verification: Whether multi-tenant data isolation can be accurately achieved (users of tenant A cannot see the data of tenant B through SQL injection); and whether the response performance of the Engine under high concurrency (such as peak hours of SaaS) is within an acceptable range.

Scenario 3: Data analysis acceleration - complex query skeleton generation

Task Description: When professional data analysts face complex analysis requirements that require multi-table JOIN, window functions, and subqueries, they use Dataherald to quickly generate SQL skeletons, and then fine-tune and optimize on this basis.

Actual income:

• SQL writing efficiency is estimated to increase by 2-3 times, especially for unfamiliar table structures (no need to manually check the Schema definition).
• Reduce low-level syntax errors - JOIN condition errors, GROUP BY omissions, misuse of aggregate functions, etc. can be avoided through AI generation.
• Analysts can focus more on data analysis and business interpretation rather than SQL syntax debugging.

Key points of implementation verification: Dataherald’s generation quality for complex queries (more than 4 tables JOIN, recursive CTE, dynamic PIVOT). The current version has limited stability on very complex queries, and analysts need to have SQL capabilities to review and correct, rather than fully trusting the generated results.

Scenario 4: Slack embedded data operation

Task Description: Enterprises embed database query capabilities into daily work communication channels through Slackbot components. The management directly asked "number of new customers this week" in the channel, and the robot replied with the data; the operation staff asked "distribution by region", and the context remained consistent.

Actual income:

• Zero friction in data acquisition—the entire process of counting, analyzing, and sharing can be completed without leaving Slack.
• Query results and conversation records are naturally saved in Slack channels, forming a traceable data discussion history.
• Reduce the "data island" effect within the enterprise - non-technical roles see data conversations in public channels and subtly learn and imitate data query behaviors.

Key points of implementation verification: Slackbot's ability to maintain long conversation context; and the compliance risks of sensitive data displayed in Slack channels (whether PII fields need to be filtered).

Applicable groups of Dataherald

Core Adaptation Crowd

• Data Analyst: You can use Dataherald to quickly generate SQL skeletons, reduce duplication of work, and devote more time to data insights. The prerequisite for adaptation is that analysts have SQL audit capabilities and can correct imperfect SQL generated by AI. Unsuitable scenarios: Production scenarios that have strict quality requirements for complex queries and cannot tolerate any SQL errors.

• Business Operations/Marketing/Sales Team: Obtain data reports directly through natural language, getting rid of dependence on the data team. The prerequisite for adaptation is that the enterprise's data model is relatively standardized (field names are clear and annotated), and the query requirements are mainly based on aggregated reports (sums, counts, rankings, trends) rather than complex multi-step analysis. Unsuitable scenarios: situations that require extremely high data accuracy (such as financial reconciliation), or where the query language is Chinese and the table/field names are also in Chinese.

• IT/Data Engineer: Responsible for the deployment, maintenance and Golden SQLs sample management of Dataherald system. The prerequisite for adaptation is that the team has Docker operation and maintenance capabilities and is willing to invest time in continuous optimization of Schema configuration, sample accumulation, and model fine-tuning. Inapplicable scenarios: Teams that do not have full-time operation and maintenance personnel, or the database is an old system (field names are meaningless and uncommented), or the data update frequency is extremely high (level per minute) and requires real-time Schema awareness.

• SaaS Product Manager/Technical Lead: Evaluate embedding Dataherald into own products to provide data analysis capabilities. The premise of adaptation is that the data scenario of the product is mainly for query-oriented users in the American/European English market. Unapplicable scenarios: Products targeting the Chinese market (accuracy drops significantly when data table names and field names are in Chinese), or products that require advanced permission auditing and multi-layer data isolation.

Unsuitable boundaries and preconditions

• Language Limitation: Dataherald's Schema scanning and NL generation use English as the main working language. Although the underlying LLM can handle Chinese input, the schema field descriptions, error messages, and management interfaces of the entire system are designed for English. In scenarios with Chinese table names and field names, it is recommended to prioritize Chinese optimization solutions such as DB-GPT.
• Database fragmentation: If an enterprise's database is distributed on 50+ independent instances, each of which requires separate configuration of connections and Schema scans, and maintenance costs increase linearly. It is recommended that only the core data warehouse be accessed, and edge databases still require traditional methods.
• Model dependency risk: Dataherald does not bind a single model, but the quality of SQL generation is highly dependent on the capabilities of the chosen LLM. If you choose low-cost models such as GPT-3.5-Turbo, the accuracy on complex queries may not meet production requirements; if you choose GPT-4, the inference cost may become a budget burden. It is recommended that multiple model combinations be tested simultaneously during the PoC phase.
• Data Security Audit: Although the Enterprise component provides basic authentication and multi-tenant support, the official does not disclose compliance certification information such as SOC2/GDPR. Strong compliance industries such as finance and medical need to confirm compliance terms with the sales team before purchasing.

Summary and Outlook

Dataherald provides a highly engineered open source solution in the enterprise-level NL2SQL track. Its core value lies in the three-stage pipeline architecture Schema-aware vector retrieval and Golden SQLs closed fine-tuning mechanism. These designs distinguish it from a simple LLM Prompt wrapper.

Current Advantages:

• Wide database coverage (9 types of relational databases + 3 types of vector storage), leading the open source NL2SQL solution.
• Complete components (Engine + Enterprise + Admin Console + Slackbot), close to enterprise-level delivery form.
• Model-independent design, not tied to a single LLM supplier, allowing switching by cost and scenario.
• The "the more you use it, the more accurate it becomes" relationship formed by Golden SQLs + Finetuning, which continuously improves accuracy in proprietary business domains.

Current Limitations:

• The project has entered a low-active maintenance period since v1.0.3. There are no new feature versions within half a year, and community contributions mainly remain at marginal repairs. Long-term maintenance risks need to be assessed when selecting a model, and a fork self-maintenance plan should be retained if necessary.
• Insufficient Chinese natural language support - Schema scanning and field description NL Generation are mainly in English, and the scenario applicability of Chinese table names/field names is limited.
• The generation quality of ultra-complex SQL (JOIN with more than 5 tables, recursive CTE, dynamic PIVOT, multi-level subquery nesting) is not stable enough, requiring secondary review by analysts, and cannot be fully automated.
• Key business information such as enterprise version pricing, SLA commitment, and compliance certification (SOC2/GDPR) have not been officially disclosed. Enterprises need to confirm one by one through sales channels before purchasing.

Industry Outlook: NL2SQL is moving from "usable" to "easy to use", and the engineering route represented by Dataherald (Schema awareness + fine-tuning closure + multi-component collaboration) is the right direction. As the basic capabilities of LLM continue to improve (such as the progress of new models such as DeepSeek-V4 and GPT-5 in SQL generation), the accuracy bottleneck of NL2SQL will gradually be alleviated. By then, middle-tier platforms like Dataherald that have sufficient engineering preparations will directly benefit. But the premise is that the project can resume its maintenance rhythm, otherwise it will be overtaken in functionality and ecology by alternatives with higher community activity (such as DB-GPT, Vanna).

Procurement/Adoption Risk Assessment: It is recommended that Dataherald be introduced as a "non-critical path data query accelerator" rather than a "core data infrastructure". First use the open source version to do a PoC in a small area (1-2 business teams and 5-10 core tables) for 2-4 weeks, focusing on verifying ① whether the accuracy of English queries reaches 70%+, ② the performance of Schema scanning and vector retrieval on the own database, and ③ the effect improvement after fine-tuning. After the PoC is passed, we will evaluate whether it can be extended to a wider range of business scenarios and whether enterprise version support services are needed. If the project does not return to active maintenance for a long period of time, it is recommended to prioritize the evaluation of alternatives with a more active community as migration targets.

Related tools: github-copilot, Cursor

Version Info

• Stable Release ：The first stable version supports multi-round dialogue context, complex JOIN, and subquery generation. There is no official precise date yet. （~2025-06）
• Beta ：Beta test version, supports basic Text-to-SQL query, no official precise date yet. （~2025-03）