When their new AI model, Mythos found a software flaw and was then told to build working attack code for it, it succeeded on the first try 83% of the time. That’s a stunning engineering result . . . right?
However, not so stunning in our context. We do background investigations for a living. Let me tell you what 83% looks like in our world. A model that’s confidently wrong 17% of the time will quietly clear bad people through your hiring funnel, your vendor onboarding, and your M&A pipeline.
When 17% of your confident answers are wrong, “the model said he was fine” is not going to play well in front of a board. Or a regulator. Or, a jury.
I say all this as someone who uses AI every single day and would not go back. The tools are extraordinary. But extraordinary is not the same as finished.
HUMINT first. AI amplifies.