Independent AI testing and evaluation for government.
Ayla tests AI systems before agencies rely on them. We check vendor claims, red-team models and agents, and stand up open-weight models on your own infrastructure, with US-citizen evaluators and tests built on Ayla’s own government data.
- Test
- Red teaming, tool use
- Finding
- An instruction hidden in a pasted document made the agent email a draft to an address outside the agency.
- Severity
- High
- Reproduce
- Steps, inputs and the exact output, in the report
- Filed under
- NIST AI RMF, Measure
- We build no AI models
- We resell no AI models
- US-citizen evaluators
- Tests built on government data
What we test.
Independent verification and validationIV&V
We rerun a vendor’s performance claims on your own tasks, then test the edge cases your mission depends on, before the system goes live.
You get: Each claim reproduced or not, with the test behind it.
Red teaming for models and agentsAdversarial testing
We attack the system the way a determined user would: prompt injection, data leaks, jailbreaks, and agents that use a tool or a permission they should not.
You get: Every finding with the steps to reproduce it and how severe it is.
Open-weight model onboardingOn-premises and offline deployment
We stand up an open-weight or fine-tuned model on your own infrastructure, including networks with no internet connection, check that it serves correctly and measure what compression costs in quality.
You get: A model that runs where your data lives, with its measured trade-offs.
Training data and supply chain reviewProvenance audit
We trace where a model’s weights, training data and software dependencies came from, and flag any part whose origin is unknown or comes from a source you do not accept.
You get: A list of what was traced, what was not, and the risk in each gap.
Tested on government work, not generic benchmarks.
A public benchmark tests what its authors picked. Ayla builds test cases from its own government data and tools: job postings from 1,100+ official sources, OPM federal workforce records, and SAM.gov and USAspending contract data, so a model is tested on the documents your office handles.
The same data is public on Ayla’s hiring data pages and by API and MCP server.
- Read a job announcement and pull out its qualifications.
- Place a posting in its OPM occupational series.
- Answer a hiring-rule question and cite the rule.
- Summarize a contract notice and name what it asks for.
How an evaluation runs
- Tell us the system and the decision it supports.
- We agree in writing what we test, with which data, and where.
- Our evaluators run the tests in your environment or in ours.
- You get the report: the method, every test, every finding and what we could not test.
The rules it keeps
- No stake in the result
- Ayla builds no AI model and resells none, so no test is of our own product.
- US citizens
- Every evaluator is a US citizen who lives and works in the United States.
- Clearances
- Some evaluators hold active clearances, and cleared work goes only to them. Your security officer verifies each one.
- Your data
- Testing runs inside your environment when your data cannot leave it, and nothing from it trains any model.
- Conflicts of interest
- No evaluation of a system built by a recruiting client in the last 12 months. The four commitments
Conflicts of interest
Ayla also recruits for contractors and publishes contract data. These four commitments keep that work out of every evaluation. They follow FAR subpart 9.5 on organizational conflicts of interest.
In force since
Ayla does not bid on work it helped define.
Every government engagement gets a written disclosure and a mitigation plan.
Ayla does not evaluate any system built by a recruiting client in the last 12 months.
Recruiting records are kept out of all evaluation work.
The AI contracts list uses public SAM.gov and USAspending data only. Nonpublic information from any engagement never reaches it.
Reports your reviewers can file.
NIST AI Risk Management Framework
Findings filed under its four functions: Govern, Map, Measure and Manage.
NIST Generative AI Profile
The risks it names for generative AI, tested where they apply.
OMB memo M-25-21
The practices it asks of high-impact AI, testing before deployment among them.
What each one asks of an agency, with its source: how agencies test AI before they rely on it.
Questions AI program teams ask.
- What is independent verification and validation for AI?
- What is AI red teaming?
- Do federal agencies have to test AI before they deploy it?
- Can Ayla evaluate AI on a classified or offline network?
- Does Ayla build or sell AI models?
- How do agencies buy AI evaluation from Ayla?
Buying through a contracting officer? Ask for a quote, a W-9 or a security review
Testing an AI system before it goes live?
Tell us the system and the decision it supports. We reply within two business days.
Tell us the system. We reply in two business days. Hiring evaluators instead? Send us a role