September 12, 2026

Artificially Intelligent, Actually Useful.

An AI Update So You Don't Have To Wing It

Calibrate Your Trust

Welcome to Issue 2 of our fundamentals series. Last quarter we opened the hood on AI. This quarter we look at why the engine misfires, and why there doesn't appear to be a fix in sight.

The thing that makes AI so risky isn't that it's sometimes wrong. It's that it's sometimes convincingly wrong. Many AI professionals believe that hallucinations are baked in because of how the models are trained and graded, and ultimately stem from the fact that models are prediction engines and not database engines. Consequently, it's not a phase that AI will "outgrow".

This issue wasn't written to make you cynical. Instead, it was written to help you understand where you can lean on AI, and where you need to do your due diligence.

Why AI Still Makes Things Up

AI models make things up because they are prediction engines. There is no database to reference. They predict the best thing to say. They do that because we train them to. And consequently, to AI models, the right answer and the wrong answer are indistinguishable.

And it doesn't stop there. Models are trained from broad data including the web, and this can result in a blog rant and a peer-reviewed study holding the same weight and validity. They are also trained to please us. So it's not really a bug. It's a combination of design, data, and human nature.

A recent study confirmed that certain patterns from training become deeply ingrained, causing AI models to use the most common pattern instead of, perhaps, the correct one.

Key Insight: Because AI responses are always confident-sounding, you have to verify the content. The saying is: "Number and names... Verify the claims."

Read the Breakdown →

When AI Lies About Your Business

A U.S. company lost $390,000 due to canceled contracts after Google's AI overview invented a lawsuit against it. A court held Google liable. Two weeks prior, a clinic was held responsible for false advertising when its website chatbot said two doctors had titles they did not have.

Why It Matters: You can't stop AI models from talking about your company. So, audit what the tools are saying about your brand on a recurring basis, and correct any mistakes you find.

Read the Forbes Piece →

Running the Business on a Black Box

A black box is a system with known inputs and outputs, but unknown internal mechanisms. Generative AI is a black box. Nobody, not even the leading engineers, can clearly describe what it absorbs in training, or how an input becomes an output. Some see this as a technical problem. Others believe it's the nature of large language models.

The Practical Angle: Since you cannot guarantee AI responses, build guardrails around them: include clear sources of truth and a human in the loop for important or high-stakes output.

Read the Berkeley Analysis →

Quick Hits

.Foundations

Managing AI Trust

Ashley Shedlock argues that AI verification shouldn't be a choice between complete trust or pure doubt; it should match what's at stake. A quick brainstorm session can be unsupervised, but a response that involves real money deserves real scrutiny.

.Video

A Five-Minute Explainer on AI Hallucinations

This excellent non-technical video explains why AI models give wrong answers with total confidence. It offers the idea that if you ask AI for sources, or tell the AI up front that “It's okay if you don't know,” you may get slightly better results.

.Deep Dive

When Benchmark Scores Get Gamed

In one audit, the automated graders issued incorrect pass/fail verdicts on roughly one-third of the tests they reviewed. In another case, a model being tested found the answers to certain questions in the question data itself; it didn't solve those questions at all.

Industry Developments

Nvidia Buys Hugging Face for $13 Billion

Nvidia is buying Hugging Face, the company that hosts millions of open-source AI models. This purchase is at the center of the debate over whether advanced AI models should be open or closed. With this, Nvidia is looking to support the open side.

Resect Raises $25M to Catch AI Hallucinations

A startup named Resect came out of stealth mode to deliver an accountability layer for AI. According to Resect, this layer watches for hallucinations and corrects them before they reach users. The company is setting its sights at high-stakes industries like finance and healthcare.

Alright, that's a wrap!

This issue covers hallucinations which cost one company $390,000 in canceled contracts. It also points out that language models are a black box that AI engineers can't fully explain, and the benchmarks we use to rank the models are inaccurate and can be gamed.

I agree that the fix won't come from advancements related to large language models. Accuracy may keep improving, but hallucinations are an intrinsic part of the technology itself. So, build a clear source of truth around AI response, and keep a human in the loop when the stakes are high.

Thoughts?

Artificially Intelligent, Actually Useful. is brought to you by iS2 Digital
Your Software at the Speed of Change

Click here to subscribe, if you want to receive our newsletter directly.

Explore More Newsletters

August 29, 2026

Got a project in mind?
Tell us about it.

Let’s Talk

Artificially Intelligent, Actually Useful.

Why AI Still Makes Things Up

When AI Lies About Your Business

Running the Business on a Black Box

Quick Hits

Industry Developments

Explore More Newsletters

Write the Finish Line

Names, Numbers and SMB Outcomes

Check the Expiration Date

Got a project in mind?
Tell us about it.

Artificially Intelligent, Actually Useful.

Why AI Still Makes Things Up

When AI Lies About Your Business

Running the Business on a Black Box

Quick Hits

Industry Developments

Never miss a post! Share it!

Better Business, Not Better Buzzwords

Explore More Newsletters

Write the Finish Line

Names, Numbers and SMB Outcomes

Check the Expiration Date

Got a project in mind?Tell us about it.

Got a project in mind?
Tell us about it.