A long, long time ago in what feels like a different world–so 3 years ago–Ethan Mollick said,

He was discussing GPT-4 and it was certainly true at that point. GPT-4 was the first model that was somewhat useful at doing basic tasks. There was a stark difference between the free “bad” AI and GPT-4. From my memory, GPT-4 was the first model to consistently get high-school level questions right.

Models have continued to improve since GPT-4 and we now have an entirely different beast, reasoning models hooked up to agents. I’m shocked at how few people outside of software engineering, realize that AI models Can Just Do Things™. It’s a regular occurrence in my life to have an idea for something that would be interesting, hand the task off to an AI agent like Claude Code, have it do it, and then come back to a finished product that after a few tweaks might get deployed to a server or run somewhere. These capabilities have existed for nearly a year at this point, yet my friends are still shocked when I show them basic demonstrations of these capabilities.

This gap means that people’s mental models of what “AI” is and can do are often wildly different. One of the most frustrating things in an argument is when you’re 30 minutes in and you realize your argument about whether or not X action qualifies as Y category has just been going in circles because you disagree about what the definition of Y category is. This happens constantly in conversations about AI because, again, most people are using derpy chatbots and are coming to the correct conclusion that a derpy chatbot is not that useful. Agents that get real work done, on the other hand, paint a very different picture of what AI is capable of.

The big labs are trying to get their frontier capabilities in front of regular users with the latest spate of always-on personal assistant type products like Muse, Dot, and Duet. Whether this is the final form factor of helpful agents for non-power users remains to be seen.