12 questions to ask before you buy an AI tool for your school

A demo shows you the product working. What you need to know is how it behaves when it is not working, when a student is trying to get around it, and when something goes wrong at 11pm on a Sunday. These questions get at that. Ask all twelve, in order, and write the answers down.

Filtering

1. Where does the filtering run?

Good: on the server, before the request reaches a model. Worrying: an extension, an app setting, a managed profile — those protect a device, and a student with a second device is outside them.

2. What happens when your safety check is unavailable?

Good: the request is refused until it recovers. Worrying: any answer that avoids the word "refused", or an explanation of how rarely it happens. Frequency is not the question.

3. Can we add our own blocked topics, and does that apply to images and video?

Good: yes, and a demonstration in the image tool, not only in chat. Worrying: "our filter already covers everything inappropriate" — that is their definition, not yours.

4. Do you check the output as well as the request?

Good: both. Worrying: only the input, which assumes a reasonable-looking question never produces an unsuitable answer.

Data

5. Which companies receive our students' prompts?

You want names, including any gateway or reseller in the middle, not a category like "leading AI providers". Ask for the current sub-processor list in writing.

6. Is our students' content used to train models — yours or anyone else's?

Good: a clear no for their own use, and a straight statement of what the underlying providers' terms allow. Worrying: "we do not train on your data" with no mention of the providers actually running the model. That sentence can be true and still not answer the question.

7. How long is student content kept, and can we delete it?

Good: a stated retention period and a working deletion path. Worrying: "indefinitely, for quality purposes".

Oversight

8. What exactly can staff see?

Good: a specific answer — blocked attempts, categories, patterns over time. Worrying at both extremes: nothing at all leaves you unable to act, and every message in full is a surveillance system your families have not agreed to.

9. Can a parent see their own child's activity?

Worth asking even if you do not intend to enable it, because the answer reveals whether the data model has any notion of a guardian relationship.

10. Can an administrator switch off a category like child-safety filtering?

Good: certain categories are locked and cannot be disabled by anyone, including them. Worrying: "an admin can configure all categories" — that is a single compromised or careless admin account away from a serious problem.

Reality

11. How many schools use this today, and may we speak to one?

A new product is not disqualifying and a straight answer is essential. Worrying: a number with no reference, or named institutions who turn out not to be customers.

12. What does it do when your AI provider is down or out of credit?

Good: a clear error telling the user to try later. Worrying: not knowing, or a vague answer — this is the most common real-world failure of AI products and it will happen during a lesson.

Before you sign

1. Ask for a trial account and try to break the filter yourself for twenty minutes. Try rephrasing, try another language, try the image tool.
2. Test on a personal phone that the school does not manage.
3. Set a blocked topic and a time limit, then verify both actually took effect.
4. Get the sub-processor list and the retention period in writing, not in a demo.

Twenty minutes of adversarial testing tells you more than any deck. If a vendor is reluctant to hand over a trial account for exactly that, you have learned something.