Twelve questions before hiring an automation

Hiring an automation or an AI system has a structural problem: the person buying can almost never evaluate on technical grounds what is being offered. The usual way out is to ask for three quotes and pick the middle one, which is an elegant way of deciding at random.

There is an alternative. You do not need to understand the technology: it is enough to ask twelve questions and listen to the shape of the answer. A provider who has built this before answers with concrete details; one who has not answers with adjectives.

Under each question is what should reassure you and what should concern you.

On scope

1 · What is left out of this project?

Reassuring: a concrete list. “It does not include migrating the historical data, it does not include training beyond two sessions, it does not include changes to the accounting system.”

Concerning: “Everything you need.” Nobody can deliver everything you need for a fixed price, and whoever says so is postponing the discussion until the moment you have already paid the deposit.

2 · What do you need from me, and by when?

Reassuring: a list with dates and names. Access, information, decisions, a person available for questions.

Concerning: being asked for nothing at all. It is fine for a provider to take charge of everything technical — that is what they are hired for — but the knowledge of the business is yours alone: what exceptions exist, who decides what, where the process gets it wrong today. If nobody asks you about that, they are going to assume it, and they are going to assume it wrong.

3 · What happens if halfway through the project we find that the process has to change?

Reassuring: a written mechanism. How a change is quoted, who approves it, how it affects the date.

Concerning: “We'll deal with that when it comes.” It will indeed be dealt with, and it will be an uncomfortable conversation with the work half done.

On how it is built

4 · What does the AI model decide and what is validated in code?

Reassuring: a clear separation. “The model interprets what the customer is asking for; the hours, the durations and the capacity are validated by the program before anything is written.”

Concerning: “The AI takes care of everything.” A language model is probabilistic: it gets it right almost always, and that “almost”, applied to a calendar or an inventory, is a mistake every so often. The reason is in what an AI agent is.

5 · What happens when an external system does not respond?

Reassuring: behavior that has been planned for. Retries, a notification, an honest answer to the end customer.

Concerning: silence, or “that does not happen”. It does. Services go down, quotas run out and credentials expire.

6 · How does it escalate to a person, and how is control handed back?

Reassuring: that both the handover and the return exist. When it escalates, who it notifies, how it goes quiet while the person is dealing with the customer and how it comes back on.

Concerning: only the handover having been thought through. A system that interrupts its own owner while they are serving a customer does more damage than it prevents.

7 · Where is the data stored, for how long and who can read it?

Reassuring: specific answers, with a country and a retention period, and a willingness to put it in the contract.

Concerning: “In the cloud, all encrypted.” That answers none of the three questions. If you handle customer data, the framework is in Law 8968, Costa Rica's data protection law.

On money

8 · How much is it going to cost next month, and what does that depend on?

Reassuring: the cost broken down into its parts — model usage, messaging, infrastructure, support — and which variable moves each one.

Concerning: a single monthly number with no explanation. Costs charged by use go up with use, which is exactly what you are hoping will happen if the project goes well.

9 · What happens if the model provider raises the price or withdraws the version?

Reassuring: that they have thought about changing models, and that they know what would have to be tested again before treating the change as sound: the instructions they give the model, the odd cases and what it answers in them.

Concerning: the question taking them by surprise. Model providers change prices and withdraw versions on a regular basis.

10 · What does maintenance include and what is charged separately?

Reassuring: the boundary drawn. Fixing a fault is maintenance; adding a feature is a new project.

Concerning: there being no maintenance in the proposal. A system in production needs looking after; if it is not quoted, either they will charge you for it later or they will not give it to you.

On the day after

11 · If we stop working together tomorrow, what am I left with?

It is the most important of the twelve, and the right answer depends on what you are buying.

If it is custom development, what is reasonable is for the deliverable to be in your name against payment in full, with the credentials in your name and documentation sufficient for someone else to carry on. The provider usually retains their generic components and their own tools, and that is normal: they are theirs and they use them on every project.

If it is a subscription platform, the code belongs to the provider and there is no point asking for it — just as you do not ask for the code of your billing system. What you should demand is what allows you to leave: that your data and your configuration are yours, that they are handed over in a commonly used format and within a written deadline, and that on the day you decide to change you do not depend on anyone's goodwill.

Concerning in both cases is the same thing: nobody being able to answer you, or the answer being that everything lives in the provider's accounts and stays there. That is not a service: it is a key somebody else holds.

12 · Who has built this before, and what went wrong?

Reassuring: a concrete story of something that failed and how it was put right. Anyone who has put systems into production has scars and tells you about them without drama.

Concerning: nothing ever having gone wrong. Either they have put nothing into production, or they are not going to tell you; neither is a good sign.

How to use the list

Do not read it straight through in the meeting: the three or four that apply to your case are enough to tell the difference. And two signs of form are worth as much as the answers themselves:

• They ask you more questions than they answer. A good sign. Anyone who is going to build something serious needs to understand your process first.
• They advise you against something. A very good sign. A provider who tells you a process is not worth automating is giving up work they could bill for in order to tell you the truth.

If you want a second reading of a quote already on the table, the technical division does that in the initial meeting. Including when the conclusion is that the quote you have is a good one.

Frequently asked questions

What should I ask before hiring an automation?

At a minimum four things: what is left out of the scope, what the AI model decides and what is validated in code, how much it will cost next month and what that depends on, and what you are left with if the relationship with the provider ends tomorrow. That last one protects you most: the code, the credentials, the data and the infrastructure should be in your company's name.

How do I know whether an AI provider knows what they are doing?

By the shape of the answers more than by their content. Anyone who has put systems into production answers with concrete details, asks more questions than they answer, can tell you about something that went wrong, and sometimes advises you against automating something. Anyone who answers with adjectives and promises that the AI takes care of everything has probably never operated any of this.

Is it normal for the monthly cost of an automation to vary?

Yes, when AI model usage or messaging is involved, because those are paid for by use. What is not normal is not having it explained to you: the quote should break down which part is fixed and which part grows, and with which variable it grows.

Should I ask for the code to be in my name?

It depends on what you are hiring. In custom development it is reasonable, against payment in full, and it is worth agreeing in writing before starting; the provider normally retains their generic components, which they use on every project. On a subscription platform the code belongs to the provider and asking for it gets you nowhere: there, what is agreed in writing is that the data and the configuration are yours, in what format they are handed over and within how many days. In both cases, what is needed is that someone else can carry the work on if one day that becomes necessary.