|
| AI Under the Hood I've been having a lot of conversations with people at very different points with AI. And what I keep coming back to is that understanding how these models actually work is what makes everything click. The tools, the strategy, the decisions about where to use AI and where not to. It all rests on that. This holds whether you've been into AI for three years or three days; whether you're running agentic workflows or just figuring out where AI could offer strategic advantages. We're starting a quarterly series on AI fundamentals to give both groups the same starting point: what is a language model, how is it built, and where does it break down. No computer science required. Think of it as an owner’s manual: a good starting point, and a refresher worth coming back to. | Strip away the marketing and an LLM is two things: a huge set of numerical weights, and code that calculates using them. The model takes your prompt, breaks it into tokens that are converted to numbers, then generates one token at a time by predicting what comes next. All math. That mechanism explains the limits. The model itself has no real-time knowledge, no persistent memory between calls, and no ability to act on its own. It can hallucinate confident answers because it isn’t retrieving facts. It’s predicting plausible text from statistical patterns identified across massive training data. Key Insight: Every limitation you’ve heard about, including hallucinations and forgetting context, comes from the same source. Knowing that helps you scope what AI can and can’t reliably do for your business. |
The foundational steps in teaching AI how to communicate are pretraining, fine-tuning, and RLHF (Reinforcement Learning from Human Feedback). Pretraining exposes the model to terabytes of text (essentially the whole internet, plus books and code); think of it like a toddler absorbing language, picking up patterns without being taught rules. Fine-tuning is when a teacher steps in, giving the AI examples of correct and incorrect responses. RLHF is where humans step in, ranking competing responses. The Practical Angle: A model isn't a chatbot; it's the trained artifact. ChatGPT, Claude, and the others are a model plus system instructions, retrieval, and continuous feedback layered on top. Most of the value of AI chatbots and other AI tools lives above the model itself. See the Breakdown → There's a classic logic puzzle with five houses, five nationalities, five pets, where each clue narrows the field until one answer remains. When AI is asked to solve it, GPT-4 reportedly failed every time. LLMs work by prediction; when tasks involve complex composition, they often fail. Bigger models raise the bar on what counts as complex, but don’t remove the problem. Breaking one hard problem into smaller ones (chain-of-thought) helps, but the issue doesn’t disappear. The Bottom Line: Reasoning models and chain-of-thought have made AI better at multistep problems, but the underlying constraint hasn't changed. Confident wrong answers still surface on novel questions. Pattern completion is its strength. Judgment is still human. Read the Analysis → | Quick Hits.Foundations The Five Years Nobody Noticed ChatGPT didn’t appear out of nowhere. It was the result of five years of compounding research. From the 2017 Transformer paper through AI's “iPhone moment,” this piece traces how we got there. The featured article explains how LLMs work; this one explains how they arrived. | .Video The Full LLM Explainer Andrej Karpathy, OpenAI co-founder, former head of AI at Tesla, and now on Anthropic's pretraining team, built what may be the ultimate AI intro video. At 3.5 hours it’s a commitment, but each chapter runs 5–20 minutes. Go deeper on concepts from this issue: pretraining, hallucinations, or jagged intelligence. | .Deep Dive Inside the Transformer Architecture The featured article covers parameters and training. This one goes deeper into the Transformer itself: the attention mechanism that lets AI models relate every word to every other word at once. It's a deep but accessible read, paired with a companion video. |
|
| Industry DevelopmentsThe Church Weighs In on AI Pope Leo XIV published “Magnifica Humanitas” on May 25, the first papal encyclical on AI, centered on the protection of human dignity. Anthropic co-founder Christopher Olah stood alongside the pope at the launch. That pairing signals where serious ethical conversations around AI are heading. | California Plans for the AI Workforce Shock Governor Gavin Newsom signed an executive order directing state agencies to prepare workers and small businesses for AI-driven workforce disruption. The order targets severance standards, retraining programs, and revisions to the state’s WARN Act. California set the pace before with SB 53, and other states followed. Expect this order to spread the same way. |
|
| |
|
