Methodia by Solicare

Track record

A method is judged on what it ruled out.

This framework was not written to be published. It took shape while evaluating very different technologies, inside an IT department of 1,500 people, with the business teams concerned. Here are four of those evaluations, and what each one taught the method.

Four evaluations

Unrelated subjects, one and the same protocol.

That is precisely what makes the method transferable: it rests on no particular domain knowledge, it rests on how the question is put.

Phase 1, steps 3 and 4

Large language models for claims handling support

Five successive campaigns, run on a single set of 129 questions built with the domain experts. The set stayed unchanged from one campaign to the next: what changed were the model versions.

What the setup showed

On a technology moving at that pace, the gap between two campaigns a few months apart often exceeds the gap between two competitors at a given moment. Deciding on a single campaign amounts to deciding on a snapshot taken at random.

Phase 1, step 3

Nine speech recognition solutions

Nine candidates put head to head on an identical protocol: same recordings, same criteria, same number of runs, scored against a reference transcript established beforehand.

What the setup showed

The ranking implied by the commercial demos and the one observed on real recordings do not overlap. Degraded conditions, accent, background noise, speech rate, overlapping voices, are what genuinely separates the solutions.

Phase 1, step 1

A sign language technology

Placed on the TRL scale before any head to head, to establish what was an available product and what was still a research subject.

What the setup showed

The textbook case for step 1. The need was strong and so was the appetite, but the maturity did not yet support a commitment. Saying so early, and saying it with a shared scale rather than an intuition, avoids a project started too soon and abandoned later.

Phase 1, steps 3 and 5

Four low code platforms

Compared before commitment, on representative development cases rather than on the vendors' demonstration examples.

What the setup showed

With this kind of tool the commitment lasts years, and the decisive criterion is not how easy the first screen is: it is what leaving costs. That evaluation is what brought exit cost into the scorecard.

The outcome

Rule out, then secure.

Above all, this work served to rule out solutions that were compelling in a demo, and to secure those that did make it into production. Both matter, and the first is often worth more than the second.

The cost ratio is the only argument that counts here. A properly run evaluation represents a few weeks of work. A deployment committed to the wrong solution represents years of contract, a data migration, change management to redo, and a team with no appetite left for the subject.

36

innovation experiments led, five of which were industrialised

1,500

people in the IT department where these evaluations were run

Who stands behind the method

Yann Daudin

Founder of Solicare. A consultant since 2007, first in firms and then on my own account: strategy and organisation consulting, information systems transformation, innovation.

From 2020 to 2026, head of the Innovation department within the IT division of France Travail, the French national employment agency. That is where this method took shape, from having to answer the same question in different guises: does this technology deliver what it promises, on our own cases, and at what price.

I now work independently on these evaluations, alone and under my own name. No team you have not met will turn up on the engagement.

Full background on LinkedIn

Verifiable markers

  • Cigref 2025 report

    Co-lead of the working group on technological innovation and value for information systems, and author of the report published in January 2025, with forty experts from twenty-nine organisations. Cigref is the association of large French companies and public bodies for digital matters. See it on cigref.fr

  • TradEmploi

    Real time translation into one hundred and thirty languages for the reception of non French speaking jobseekers, a project I led end to end, deployed across the whole agency network. Awarded Gold and the jury's special prize at the 2023 Grand Prix Syntec Conseil, the French consulting industry awards. Open source under AGPL v3. See the repository

  • Open Source Program Office

    Founded and ran the OSPO: publication doctrine, licensing, review process. Around forty public repositories and more than fifteen products released. See the organisation

Public demonstrators

Solicare publishes its own tools, so you can judge on evidence rather than on claims. In French.

Your case looks like none of these. That is normal.

The method assumes no prior knowledge of your domain. It assumes access to your experts and to your real cases. Thirty minutes are enough to check that those conditions are met.

Book a call How an engagement runs