Why AI Still Makes Things Up

AI models make things up because they are prediction engines. There is no database to reference. They predict the best thing to say. They do that because we train them to. And consequently, to AI models, the right answer and the wrong answer are indistinguishable.

And it doesn't stop there. Models are trained from broad data including the web, and this can result in a blog rant and a peer-reviewed study holding the same weight and validity. They are also trained to please us. So it's not really a bug. It's a combination of design, data, and human nature.

A recent study confirmed that certain patterns from training become deeply ingrained, causing AI models to use the most common pattern instead of, perhaps, the correct one.

Key Insight: Because AI responses are always confident-sounding, you have to verify the content. The saying is: "Number and names... Verify the claims."