Measuring what an AI agent brings you: the honest method

You'll find promises everywhere with figures on what AI saves. None of them concern your workshop, your practice or your restaurant. The only measurement that matters is the one you do yourself, on your work, with your numbers. It requires neither software nor complicated spreadsheet: a notebook, a baseline week and a bit of rigor are enough.

IWhat you can actually measure

An agent delivers in four ways, and each is measured differently. It returns time: the minutes you no longer spend on a task. It shortens deadlines: a quote sent the same day instead of three days later. It makes things happen that were not happening: the follow-up you kept putting off. It gets results: a payment, an appointment, a signed quote.

Do not track all four. Take three indicators at most : one in time, one in deadline or volume, one in result. A plumber who entrusts his overdue payment follow-ups will for example track the minutes spent following up, the number of follow-ups sent, and the number of days between the due date and payment.

Beware of indicators that flatter without saying anything. The number of emails prepared is worthless if half go to the trash. The number of tokens consumed measures an expense, not a gain. Always ask yourself the same question: if this figure doubles, will my business be better off? If the answer is uncertain, change your indicator.

IIMeasure before launching the agent

This is the step that almost everyone skips, and without it no comparison is possible. During a normal week, before launching the agent, note your three indicators. For time, note the start and end time each time you do the task. For deadlines, use files from the previous month: dates appear in your emails and invoices.

Choose a representative week. A week of vacation or rush skews everything. If your activity is very seasonal, take figures from the same period last year, when you have them.

IIITrack for four weeks

Launch the agent, then record the same metrics each week in the same way. Four weeks is a reasonable timeframe: the first is often used for adjustment during the trial run and initial validations, and the following three show normal operation. Do not judge based on the first week, which is almost always the weakest.

The line on sends approved without revision is precious. If it rises week after week, your adjustments are bearing fruit: each clarification added to the task or documents avoids a revision. If it stalls, the problem often comes from the instruction or a missing document, not from the agent itself.

IVCompare against the real cost

The cost of an agent is not limited to the subscription. It also includes your time reviewing it, especially at the beginning, and the time spent setting up the mission. At ABYS, the subscription ranges from 19 euros excl. VAT per month for 100,000 tokens to 349 euros excl. VAT for 5 million, and the account displays consumption. Your dashboard tells you whether the tank you chose is sufficient or if it is too large.

To give value to the time returned, do not use a theoretical figure. Ask yourself a simple question: what did you do with those hours? A carpenter who spends two additional hours in the workshop can quantify them in billable work. A manager who spends them resting gains real benefit, but one that cannot be quantified in euros. Both count; just say which one you are counting.

VDecide: continue, adjust or stop

After four weeks, place your baseline figures next to those of the last week. The decision follows simple logic, which you can set in advance to avoid fooling yourself.

Stopping is not a failure. A poorly chosen task teaches you which task to choose next. Continuing, however, commits you to keeping the tracking once a month: an activity changes, and an agent that was profitable may stop being so.