AI procurement for mid-market companies: questions buyers should ask before signing

Mid-market companies are often large enough to care deeply about privacy, security and operational risk, but small enough that procurement cannot become a six-month enterprise exercise for every new tool. AI products make that balance harder because the visible application may depend on external models, retrieval systems, subprocessors and changing provider behavior that are not obvious from the demo.

NIST's Generative AI Profile, UK Government AI procurement guidance and the ICO's AI audit framework all point in the same direction: define the intended use and risk, investigate third-party dependencies, establish acceptable performance, make responsibilities explicit in contracts and keep monitoring after purchase. The practical challenge is turning those principles into questions a buyer can actually ask before signing.

1. Define the job before evaluating the model

Procurement should start with the business task, not with a vendor shortlist. NIST's AI risk-management approach is context-specific, and UK procurement guidance similarly emphasizes defining the problem and desired outcomes before selecting technology. A system summarizing internal documents has a different risk profile from one making customer-facing recommendations or taking actions in business systems.

• What exact task is the AI expected to perform?
• Who will use it, and who may be affected by its output?
• What company, customer or regulated data will enter the system?
• What happens if the system is wrong?
• Which decisions or actions still require a human?

2. Ask what is actually inside the product

The vendor selling the product may not operate every component. NIST notes that generative-AI value chains can include procured models, datasets and software that are difficult for downstream users to inspect. Buyers should therefore ask which model provider, retrieval systems and third-party tools sit behind the product and which of them receive customer data.

• Which underlying model or model providers are used?
• Can the provider change the model without customer notice?
• Which subprocessors receive prompts, files, metadata or logs?
• Does the product call external tools or services on the customer's behalf?
• Which responsibilities remain with the buyer?

3. Ask exactly what happens to your data

Enterprise AI products often make commitments about business-data handling, but those commitments vary by vendor, product and plan. OpenAI's current enterprise privacy and business-data documentation is one example of the evidence a buyer should look for: it describes business-data ownership/control, default non-training on business data, encryption and enterprise privacy controls. That is illustrative, not a market-wide assumption.

• Are prompts, files and outputs used to train or improve models by default?
• How long are prompts, outputs and logs retained?
• Where is the data stored and processed?
• Can vendor personnel access customer content, and under what controls?
• Which subprocessors receive the data?
• What is deleted at contract termination, and on what timeline?

4. Define acceptable quality before signing

The ICO's AI audit guidance recommends establishing an acceptable level of accuracy before procurement and seeking evidence from suppliers about source and model quality. For generative AI, one generic accuracy number may not be meaningful. Buyers should define the failure types that matter in the actual workflow.

• Does the system complete representative tasks correctly?
• Can it distinguish supported from unsupported factual claims?
• Does it cite or expose sources where the workflow requires traceability?
• How does it behave when information is missing or conflicting?
• When should it abstain, escalate or ask for human review?

5. Test the product with the real workflow

A polished demo shows the vendor's preferred scenario. A useful pilot tests the buyer's own workflow with representative inputs, access controls and integrations. That includes edge cases, adversarial inputs where relevant, user permissions and the manual steps that still exist around the AI feature.

The pilot should also test the total process. If employees spend significant time checking every output or correcting data before the AI can use it, that operating cost belongs in the procurement decision even if the model itself performs well.

6. Review security beyond the trust page

Security certifications and trust pages can reduce due-diligence effort when they are current and relevant, but they do not answer every operational question. Buyers should verify identity controls, tenant isolation, administrator privileges, audit logs, integration permissions and incident-response processes for the product they will actually deploy.

• SSO, MFA and role-based access.
• Tenant and data isolation model.
• Administrative access and support access controls.
• Audit logs available to the customer.
• Secrets and integration-token handling.
• Incident response and customer-notification process.

7. Clarify responsibility when something goes wrong

ICO guidance emphasizes clearly defining controller and processor roles and the details of third-party processing. The same discipline should extend to operational responsibility: who investigates a data incident, who supports a failed integration, who owns an incorrect automated action and what escalation path exists when the AI system behaves unexpectedly?

Liability, indemnity and regulatory obligations require qualified legal review, but procurement can still make the underlying operational responsibilities explicit before lawyers negotiate the final wording.

8. Ask how the product changes after purchase

AI products can change materially without a traditional software release. The vendor may update the model, switch providers, change retrieval architecture, alter system prompts or add subprocessors. UK procurement guidance recognizes that AI contracts need to accommodate technology evolution rather than assuming the purchased system stays static.

• What model or provider changes require customer notice?
• Can the buyer remain on a previous model/version for a period of time?
• How are material subprocessor changes communicated?
• What regression or safety testing occurs before major changes?
• Can a customer opt out of a material change that alters risk or compliance posture?

9. Understand economics beyond the seat price

The contract price may be only part of the operating cost. Usage limits, overages, implementation, integration work, human review, premium support and change-management effort can materially affect the economics of adoption. Buyers should model those costs from the actual workflow rather than relying on generic AI productivity claims.

10. Plan the exit before dependence grows

ICO third-party guidance includes reviewing outsourced services and retaining the ability to switch providers when continued use is no longer appropriate. For AI tools, that means understanding what can be exported and what disappears when the contract ends.

• Can conversations, uploaded files and generated outputs be exported?
• Can prompts, workflow templates or knowledge-base configuration be migrated?
• Are audit logs exportable?
• What deletion commitments apply after termination?
• What dependencies will have to be rebuilt if the vendor is replaced?

A practical AI procurement checklist

• Use case, users and prohibited uses documented.
• Underlying model/provider dependencies understood.
• Data use, training, retention and subprocessors documented in writing.
• Task-specific evaluation completed with representative examples.
• Security, access and audit controls reviewed.
• Privacy/security responsibilities reflected in the contract process.
• Material model and provider changes addressed.
• Usage and total operating cost understood.
• Export, deletion and provider-exit path documented.

Buy the operating system around the model, not just the demo

The model matters, but procurement risk usually sits in the surrounding system: data flows, permissions, integrations, quality controls, vendor dependencies, contracts and change management. Mid-market buyers do not need to recreate enterprise procurement bureaucracy. They do need enough evidence to know what they are buying, what happens when it changes and how to leave if it stops meeting the business's requirements.

What to check before you commit

1. Define the workflow and failure consequences before comparing vendors.
2. Verify model, subprocessor and data-handling boundaries in writing.
3. Evaluate quality against the buyer's real tasks rather than relying on generic benchmarks.
4. Plan for model/provider changes as part of contract and operational review.
5. Document export and deletion before the organization becomes dependent on the product.

A note on measurement

Teams that treat AI procurement as an engineering project usually measure the wrong thing. Instrument the business outcome first — cycle time, cost per transaction, resolution rate, revenue retention — then work backwards to the technical metrics that move it.