AI Verify is an open source AI governance testing framework and software toolset led by the AI ​​Verify Foundation under IMDA in Singapore. It includes the AI ​​Verify Testing Framework (11 international AI governance principles assessment framework), AI Verify Toolkit (traditional AI model testing tool) and Project Moonshot (LLM evaluation and red team testing toolkit), covering compliance testing needs from traditional ML to generative AI.

AIVerify

Core parameters and statistics

AI Verify is an open source AI governance testing framework and software toolset led by the AI Verify Foundation under the Singapore Infocomm Media Development Authority (IMDA). It is not a single product, but consists of three layers of capabilities: AI Verify Testing Framework (governance framework), AI Verify Toolkit (traditional AI testing software) and Project Moonshot (LLM evaluation toolkit).

Projects	Public Information
Official positioning	AI governance testing framework and software toolkit
Nature of organization	IMDA wholly owned not-for-profit subsidiary
Open Source License	Apache 2.0
Governance Principles	11 International AI Governance Principles
Toolkit latest version	2.2.1 (2026-03-23, GitHub Releases)
Moonshot latest version	0.7.6 (2026-02-05, GitHub Releases)
GitHub Stars (Toolkit)	79 stars, 24 forks, 20 contributors
GitHub Stars (Moonshot)	337 stars, 65 forks, 20 contributors
Framework first released	2022-05-25 (MVP)
GenAI Framework Update	2025-05-29
Support Platform	Web, API, Desktop

Organizational Background: The AI Verify Foundation is a wholly-owned subsidiary of the Singapore government agency IMDA, which means that its governance framework is naturally consistent with Singapore’s AI governance policy (as well as the US and European frameworks that benchmark it), and is suitable for multinational enterprises with cross-border compliance needs.

Dual tool strategy: AI Verify Toolkit covers traditional supervised learning models (classification/regression, tabular/image data), and Project Moonshot covers LLM evaluation and red team testing. The two are complementary but do not overlap. Enterprises need to choose the corresponding tool chain based on their own AI asset types.

Community scale: The two core warehouses total about 416 stars. The scale is small but most of the contributors come from the IMDA official team and government-related institutions. It is a government-led open source rather than a spontaneous growth type of the community.

User and market recognition

AI Verify's market recognition is mainly reflected in the two levels of intergovernmental cooperation and participation of multinational enterprises, rather than the spontaneous enthusiasm of the community.

Intergovernmental Benchmarking: The AI ​​Verify Testing Framework has completed cross-mapping (Crosswalk) with international frameworks such as the US NIST AI RMF, the EU AI Act, OECD, and the G7 Hiroshima Process. In May 2025, a new version of Crosswalk with the NIST AI RMF Generative AI Profile was released. This means businesses using AI Verify can reduce the cost of duplicative assessments when dealing with multi-jurisdictional compliance requirements.

Enterprise participation: Premier members include AWS, Dell, Google, IBM, Microsoft, Salesforce, Red Hat, Resaro, etc. General members include Ant Group Temus, ACCA, etc., covering multiple industries such as technology, finance, and auditing. These members not only provide financial support, but also directly participate in tool development and testing standard setting.

Pilot Verification: The Global AI Assurance Pilot launched in February 2025 attracted 100+ participants from 30+ organizations, covering actual testing of GenAI applications in multiple industries from finance to medical care. It will be upgraded to Global AI Assurance Sandbox in July 2025 to continue to expand the testing ecosystem.

User Reviews: X0PA (AI recruitment platform) rates the Toolkit as "providing a robust framework for assessing and improving the fairness of AI systems", Dell says it "provides a unified framework for organizations to achieve honest, fair and equitable systems", and Singapore Airlines believes it is "an excellent starting point for self-assessment of AI systems".

Cost advantage

AI Verify’s cost structure is directly related to its government context, making it significantly different from commercial AI governance tools.

C-side/Individual: The PDF and Excel versions of AI Verify Testing Framework can be downloaded and used for free. Project Moonshot and AI Verify Toolkit are both Apache 2.0 open source without any licensing fees. Individual developers or small teams can clone and use it directly from GitHub.

API/Developers: Project Moonshot provides Python libraries (pip install aiverify-moonshot[all]), Web API and CLI, completely free. Developers need to bear the cost of LLM API calls (the inference cost incurred by connecting the target model during evaluation) and infrastructure costs (such as GPU servers). AI Verify Toolkit needs to be deployed and run locally or within the enterprise.

Enterprise/Private: All tools support self-hosted deployment, no software license fees. The actual costs for enterprises are concentrated in: deployment and operation and maintenance manpower, test data preparation AI model API call fees, and application costs of the compliance team learning framework. For companies that require third-party testing and certification, the AI ​​Tester Accreditation Program (launched in May 2026) provides a list of officially recognized testing service providers. The relevant audit fees are subject to the service provider's quotation.

The free truth: It is true that open source has no license fees, but the hidden costs involved in fully implementing the AI ​​Verify framework, such as personnel training, test context construction, and continuous compliance monitoring, cannot be ignored. Compared with commercial AI governance platforms (such as IBM AI Governance, Google Vertex AI Governance), AI Verify has lower explicit costs, but requires greater internal technical capabilities and governance maturity.

Main functions

AI Verify's core capabilities are organized into a three-layer structure, with each layer facing different usage roles:

• AI Verify Testing Framework (governance framework layer): Provides a structured evaluation method for 11 international AI governance principles. Each principle contains a four-layer progressive structure of Principle→Outcome→Process→Evidence. The framework can be used independently by downloading PDF/Excel forms, or combined with Toolkit or Moonshot to generate compliance reports. Ideal for compliance teams and auditors.

• AI Verify Toolkit (traditional AI test layer): For supervised learning classification and regression models, it provides technical tests such as fairness (bias detection), robustness (image corruption/adversarial samples), interpretability (SHAP value), and security (data leakage testing). Supports table data and image data, and customized reports can be generated after the test is completed. The current version 2.2.1 also integrates the Veritas Financial Compliance Test Suite to meet the security baseline requirements of the MAS.

• Project Moonshot (LLM evaluation layer): Provides 100+ open source benchmark data sets, covering multi-dimensional evaluations such as performance (accuracy BLEU), trusted security (bias, toxicity, hallucination), etc. Supports automated red team attacks (based on research-driven algorithmic strategies combined with generative LLM to generate adversarial cues) to perform testing against multiple LLM endpoints simultaneously. Provides HTML interactive reports and JSON raw data export, supporting CI/CD pipeline integration. Current version 0.7.6.

Hidden linkage: The three layers do not operate independently - the evaluation results of the framework layer can be directly mapped to the technical test items of Toolkit/Moonshot, and the test report can reversely verify the compliance statement of the framework layer. For example, the compliance team first uses the framework to complete principle-level self-examination, and then hands it to the engineering team to use Moonshot to perform specific red team testing. The final test data automatically backfills the framework report, forming a link from management to technology.

Model and version evolution

The relevant information has not been made public, please refer to the official real-time page.

Technical advantages

AI Verify's technical advantage comes from the combination of "government-level framework consistency + open source tool chain coverage" rather than a single performance parameter.

Framework Alignment: The AI ​​Verify Testing Framework has been cross-mapped to the top principles of NIST AI RMF, ISO 42001, and EU AI Act. This means that when an enterprise uses AI Verify to conduct compliance assessments, its output can be partially reused to handle compliance reviews in other jurisdictions, reducing the cost of repeated assessments. The mechanism is that Crosswalk documents clearly mark the conceptual equivalence between frameworks, rather than simply declaring alignment.

Dual tool chain covers all modes: Toolkit covers the table/image model of traditional ML, and Moonshot covers the text/dialogue model of LLM. The two are unified in testing methodology but independent in technology implementation. This segmentation avoids applying traditional ML testing methods to LLM scenarios and prevents a single tool chain from being overly expanded.

Modular architecture (2.0+): Toolkit 2.0 is refactored into a five-component architecture of apigw (API gateway), portal (front-end portal), test-engine (test engine), test-engine-worker (test work node), and shared-library (shared library). Test algorithms run as independent Python modules that can be developed, deployed, and updated independently without blocking the overall release cycle.

Automated red team capabilities: Moonshot partially automates red team testing that traditionally relies on manual labor, and uses algorithmic strategies (such as gradient/rule-based adversarial generation) and generative LLM-assisted test case generation to achieve parallel attack testing of multiple LLM endpoints. This is Moonshot’s key differentiating capability from pure benchmarking tools.

Applicable scenario reminder: The flexibility brought by the high alignment of the framework also means that it is not as deep as specialized solutions. Detecting specific types of AI bias or security vulnerabilities may need to be supplemented by specialized tools such as Anthropic’s jailbreak assessment framework Meta’s Purple Llama.

How to use

The usage entrance of AI Verify is divided into three paths based on different roles and test goals:

How to use	Suitable for the crowd	Core operations	Cost
Testing Framework Documentation	Compliance teams, auditors	Download PDF/Excel, complete self-examination according to 11 principles, and generate alignment report	Free
AI Verify Toolkit	Data Scientist ML Engineer	Deploy local context, load models and data sets, run fairness/robustness tests, and export reports	Free and open source, need to prepare your own infrastructure
Project Moonshot	LLM developer, security team	pip install aiverify-moonshot[all], connect to LLM endpoints, run benchmarks or red team attacks, generate HTML reports	Free and open source, LLM API call fees are required

Get started quickly with Project Moonshot:

# Install Moonshot
pip install "aiverify-moonshot[all]"

# Clone test assets and Web UI
python -m moonshot -i moonshot-data -i moonshot-ui

# Start Web UI
python -m moonshot web
# Open the browser http://localhost:3000/

# Or start the interactive CLI
python -m moonshot cli interactive

Typical implementation path: The compliance team first downloads the Framework PDF and completes a principle-level self-examination → identifies the AI system that requires technical verification → the ML team uses Toolkit to run the fairness/robustness test of the traditional model → the LLM team uses Moonshot to perform benchmarking and red team testing → backfills the framework report with test results → generates a complete AI Verify compliance report.

Product Pricing

All core tools of AI Verify are open source under the Apache 2.0 license, and the public pricing information is as follows:

• C-side/individual developers: Framework documents are free to download, and Toolkit and Moonshot are completely open source and free. For personal use, the actual cost is only the LLM API call fee (at the time of Moonshot review) and computer infrastructure.
• Team/Enterprise Self-Hosting: There is zero license fee for the software, and the cost is concentrated on deployment and operation (Docker/K8s), storage and computing resources. Toolkit deployment requires a server with at least 4 cores and 16GB of RAM.
• Third Party Testing and Certification: AI Tester Accreditation Program (launched in May 2026) provides officially recognized third-party testing service providers. The relevant audit and certification fees are subject to the service provider's quotation. The official reference price is not disclosed.
• Foundation Member: Companies can obtain strategic influence and governance committee seats by joining the AI ​​Verify Foundation. Membership fees are subject to official business communications, and the standard price is not disclosed.

Comparison of commercial solutions: Compared with commercial platforms such as IBM AI Governance (billed by asset volume) and Google Vertex AI Governance (billed by number of evaluations), the software cost of AI Verify is zero, but it requires enterprises to have stronger self-deployment and self-operation capabilities. Total cost of ownership (TCO) depends on the trade-off between in-house labor versus commercial platform subscription fees.

Application scenarios

Typical implementation scenarios for AI Verify are concentrated in regulated industries that need to meet AI compliance and security requirements:

• AI compliance in the financial industry: Banks, insurance and other financial institutions regulated by MAS can use the Veritas suite in the AI ​​Verify Toolkit to verify the fairness and security of credit scoring, anti-fraud and other models to meet the MAS's AI governance guidelines. Combined with the Framework, complete the full-link documentation from principles to evidence to respond to regulatory inspections.

• LLM Security Benchmarking and Red Teaming: Enterprises use Project Moonshot to perform benchmarking (bias, toxicity, hallucinations) and automated red teaming attacks on base and fine-tuned models before deploying generative AI applications. Moonshot’s HTML report can directly serve as the basis for the AI ​​Safety Committee’s risk review. The real benefit is in moving human red teams from "on-demand execution" to "continuous integration".

• Cross-border AI governance alignment: Singaporean companies with operations in both the United States and China can use the cross-mapping of the AI ​​Verify Framework with NIST AI RMF and ISO 42001. A set of assessment outputs can simultaneously meet compliance requirements in multiple places and reduce duplication of parallel assessment work. In practice, additional review is still needed to address differences in local laws and regulations.

• Government and Public Sector AI Procurement Acceptance: When government agencies purchase AI systems, they can list the AI ​​Verify compliance report as one of the supplier's deliverables as a quantifiable basis for acceptance. The Singapore government is already promoting this practice.

Applicable people

AI Verify's multi-layered design enables it to serve three types of core roles, each of which focuses on different levels of capabilities:

• Compliance and Risk Control Team: Need to build an AI governance framework and respond to regulatory inspections. Mainly use the Testing Framework's 11-principle self-examination process and Crosswalk mapping documents to produce compliance reports. The prerequisite is to have basic knowledge of AI governance and understand regulatory requirements (such as EU AI Act, MAS Guidelines).

• AI/ML Engineers and Data Scientists: Need to verify the technical safety of the model. Traditional ML scenarios use AI Verify Toolkit for fairness and robustness testing; LLM scenarios use Project Moonshot for benchmarking and red team testing. The prerequisite is familiarity with the model deployment environment and the Python/Pip tool chain.

• Third-party audit and testing agency: Provides independent AI audit services. You can use the AI ​​Tester Accreditation Program to obtain officially recognized qualifications, and use the AI ​​Verify tool chain to perform standardized testing and issue certification reports for customers.

Not suitable for boundaries: AI Verify is not suitable for teams that require deeply customized original security strategies (the framework provides general guidelines rather than specialized solutions); it is not suitable for organizations that have no awareness of AI governance and internal processes (the implementation of the tool requires the cooperation of organizational maturity); nor is it suitable for rapid detection scenarios that only require a single dimension (such as only testing LLM illusion, community-specific tools are more lightweight).

Summary and Outlook

The core value of AI Verify is that it is a rare AI governance testing system that is led by government agencies and is fully open source. It is based on 11 internationally aligned governance principles as the top layer, and covers a complete evaluation cycle from principles to technology through two tool chains: Toolkit (traditional AI) and Moonshot (LLM). For regulated companies that need to deal with cross-border compliance requirements, it provides a "one set of assessments, reuse in multiple places" path.

Current limitations: The community size of the tool chain is much smaller than that of commercial competitors (the two core warehouses total about 416 stars), documentation and technical support rely on the official IMDA team, and the speed of problem response and function iteration is not as fast as that of the rapidly evolving commercial AI governance platform. Moonshot is still in beta and stability needs to be verified before large-scale deployment in a production environment. Framework’s self-assessment model naturally relies on organizational honesty and lacks independent verification mechanisms (this gap is being filled by the AI ​​Tester Accreditation Programme).

Implementation Suggestions: It is recommended to use Framework PDF to complete a round of quick principle-level self-examination internally (about 1-2 weeks) to identify the priorities of the AI ​​systems that really require technical verification, and then select the corresponding tool chain (Toolkit or Moonshot) to complete technical testing on 1-2 pilot systems. Before purchasing Moonshot/Toolkit, enterprises need to focus on verifying: the match between the test data and their own scenarios, the red team module's ability to cover business-specific risks, and the degree of acceptance of the Framework report in the target regulatory jurisdiction.

Related tools: originality-ai, gptzero

Version evolution of AI Verify

AI Verify's version evolution follows two product lines, plus iterations of the framework itself.

AI Verify Toolkit Main Line

• 2.2.1 (2026-03-23): The latest stable version, including bug fixes and plug-in updates, is recommended for production use.
• 2.0.0 (~2025): Architecture reconstruction version, split from a single application into a modular architecture (apigw, portal, test-engine, test-engine-worker, shared-library), introducing an independent test algorithm running mechanism and Veritas financial compliance integration.
• 1.0 (2022-05-25): MVP released, supporting basic fairness and robustness testing of traditional ML models.

Project Moonshot Main Line

• 0.7.6 (2026-02-05): The latest version supports IMDA Starter Kit integrated Web UI improvements and red team module enhancements.
• 0.1.0 (2024-05-31): First public release, positioned as "one of the world's first LLM evaluation toolkits".
• The project is still in the Beta stage, with an iteration frequency of about 1-2 versions per month, and there are 26 Releases on GitHub.

Framework Milestones

• 2022-05-25: AI Verify testing framework MVP released for international pilots.
• 2023-06-07: The framework and toolkit are fully open source, and the AI ​​Verify Foundation is established.
• 2023-10-13: Cross-mapping release with US NIST AI RMF.
• 2024-05-31: Project Moonshot released, Model AI Governance Framework for GenAI released.
• 2025-05-29: The enhanced version of the framework is released, officially covering GenAI assessment, and adding cross-mapping with NIST GenAI Profile, Hiroshima Process CoC, and ISO 42001.
• 2026-05-18: AI Tester Accreditation Program was launched to establish a third-party AI testing and certification system.

Version Info

• AI Verify Toolkit 2.2.1 ：The latest stable version of AI Verify Toolkit, rewritten based on a modular architecture, supports independent test algorithm module operation, a redesigned back-end test engine, a new version of front-end workflow Veritas financial compliance integration, and enhanced computer vision test support. （2026-03-23）
• AI Verify Toolkit 2.0.0 ：Major architecture upgrade, adopting modular design, introducing independent testing algorithm engine, new front-end workflow, and integrating Veritas financial compliance testing suite. There is no official precise date yet. （~2025）
• Project Moonshot 0.7.6 ：The latest version of Project Moonshot (LLM evaluation toolkit) supports 100+ benchmark data sets, automated red team attack module Web UI and CLI dual operation interface CI/CD integrated report generation. （2026-02-05）
• AI Verify MVP ：The AI ​​Verify testing framework and software toolkit were released for the first time in the form of MVP, soliciting pilots and feedback from the international community, and were jointly launched by IMDA and PDPC. （2022-05-25）