Skip to main content

If your product makes decisions, your customers will ask who answers for them

Software companies are building AI into products their customers use to decide about people. Enterprise buyers, insurers and regulators now ask the vendor for evidence before they sign, and internal IT runs AI over access and security too.

AIC.Karoo Software (Demo)

Customer AI review: Enterprise Bank (Demo)

Sample data

  • Do you train on our data?Drafted
  • Can we override a single decision?Drafted
  • Tested for unfair outcomes this year?Needs your input
  • Will you tell us before the model changes?Drafted

For

  • Software and SaaS companies
  • IT service providers
  • CIOs and CISOs
  • Product and engineering leads

Where AI decides, and what goes wrong

  • Decides about: Your customers' customers

    AI features that decide for your customers

    You become the supplier your customer must review, and cannot answer their questions.

  • Decides about: Users

    Suspending accounts and moderating content

    People locked out or silenced with no route to a person.

  • Decides about: Employees

    Granting and removing access

    Access changed by an automated rule nobody reviews, in both directions.

  • Decides about: Systems and the data in them

    AI writing and changing code

    Unreviewed changes reach production because the assistant was usually right.

An enterprise customer sends its AI review

The procurement question that now arrives before the contract does.

  1. 1

    The questions arrive

    The customer asks whether you train on their data, whether they can override a decision, whether you can explain one, whether you test for unfair outcomes, and whether you will warn them before the model changes.

  2. 2

    Answers come from the record

    Answers are drafted from your register, decision log, checks and policies, and anything the record cannot answer is flagged rather than guessed.

  3. 3

    A person approves

    The person who answers for the product feature approves each answer before it goes.

  4. 4

    The customer can check

    Your trust page shows what AIC observed and what you declared, and a sealed pack backs up the answers.

Rules that already apply

A summary, not legal advice about your organisation.

POPIA sections 19 and 21

Section 19 requires appropriate security safeguards. Section 21 requires a written agreement when you process personal information for a customer as their operator.

What it asks of youSecurity you can evidence, and contract terms your customers will ask you to prove.

POPIA section 71

Applies to your customers' automated decisions, and to yours when your product decides about people directly. Read the guide

What it asks of youFeatures that let a person intervene and let the affected person be heard.

EU AI Act

Places obligations on providers of AI systems used in the EU, including transparency when people interact with AI, and full provider duties for high-risk systems from 2 December 2027. Read the guide

What it asks of youDocumentation, logging and human oversight built into the product for customers who use it in the EU.

ISO/IEC 42001 and SOC 2

The management system and assurance standards enterprise buyers most often ask software suppliers about.

What it asks of youEvidence collected once that answers each framework's requirements.

Questions

Our customers make the decisions, not us. Why does this matter to us?

Because your customers answer for decisions your product makes for them, and they will ask you how it behaves. A supplier that can answer clearly, with evidence, is easier to buy.

Can the decision log run inside our product?

Yes. Your product can send each consequential decision to AIC with an API key and hold it for a named person to review. Overrides are recorded by the signed-in person who made them.

We already have SOC 2. What does AIC add?

SOC 2 shows your security controls work. It does not show who answers for each decision your AI makes about people. AIC records that, and evidence collected once counts towards both.