Will I receive an exorbitant bill for my AI?

“Tokens or license” is the first decision to make when contracting artificial intelligence. AI models can be consumed in two ways: through tokens, with no cap, where you pay for what you use and the limit is set by the organization's governance; or through a license, where there is a cap and, when it is reached, the service is discontinued.

A license prevents bill surprises, although it can interrupt service. Tokens offer more freedom, but require governance and observability.

In this video we discuss the different risks involved in the two contracting methods, how limits per user, cache and guardrails reduce consumption, how to measure whether the business case closes with a defined maximum monthly expenditure, and aspects that make observability and flexibility key conditions.

How to limit consumption

There are several techniques. One is to set limits per user: in a banking chatbot for checking balances, for example, each user's spending is limited so that questions unrelated to the service don't cause excessive charges.

Another method is answer caching: if several users ask the same question using different words, an intermediary layer detects that it's the same question and responds with the already stored answer, without consulting the model again. It's efficient, although technically complex.

And then there are the guardrails, which put limits on the cost, the type of consumption, and the type of responses.

The business case rules

Before starting, it's advisable to define how much you can spend per month to ensure the project is profitable and estimate consumption per user. If the expected volume of spending exceeds that limit, the project shouldn't proceed. And not every business case requires a new model: prioritizing is the first line of defense for the budget.

Why can't it be projected accurately?

Prices, versions, and supplier policies change during the project. Initial calculations may not be valid at the end. However, it is possible to project based on actual consumption., observe it continuously and not depend on a single model, so you can switch when a better or cheaper alternative appears.