I imagine you’ve already read about, or tried, ChatGPT. I did too. And yes, the experience of using it is quite surprising. But I don’t want to write about it, I want to elevate from it. Here are 3 thoughts on the progress in Artificial Intelligence to elevate.

From ELIZA to ChatGPT: The Evolution of Conversational AI

Although ChatGPT brings to bear an enormous amount of knowledge in its conversation with you, most people don’t recognize that it comes from a long lineage of conversation AIs which started with ELIZA.

ELIZA was created in the 1960s by Joe Weizenbaum at MIT (my alma mater!). And it was pretty cool with lots of copycat implementations. I believe I had to code an experience like it in one of my classes and when I tried it out, I remember ascribing a lot of meaning to my interactions with it. I had coded it, and yet the interaction with it was still surprising. One can also take away the sense of progress in AI, from a very rudimentary beginning. Persistence and focus pays off with progress in Artificial Intelligence.

The Unfolding Potential of Large Language Models

ChatGPT uses what are called large language models. The underlying model for it is GPT 3.5 (this is actually a series of models), and there is a reinforcement layer that was developed in order to have humans train it on what is expected in a human conversation. I think the highlight here is that large language models are somewhat measured on the billions of parameters they have. When you go 10x the number of parameters you start getting a somewhat different behavior. Think of it like emergent phenomena. And these types of models, often referred to as LLMs have been scaling up at roughly 10x each year for the last few years. The velocity of this improvement in this type of AI is quite a feat. This puts into question what can we expect in 3 more years in this area. At this breakneck speed of development in a few short years even more developments will surprise you. We’re certainly not at the cusp of what LLMs can do.

AI Benchmarks: Beyond Average, Yet Short of Expertise

ChatGPT is a popular instance of a phenomenon that is becoming more common these days. That AI products are starting to beat humans in many general intelligence benchmarks, like image recognition, question / answering, or summarization. The field of AI has a good number of benchmarks in which teams can compete to showcase an improvement in quality of AI research. What is surprising is that many of these benchmarks, which also have an average human achievement score measure, are now being beaten by AI models. Notice that I said Average. These benchmarks also often have an expert human score, and the AIs aren’t beating that score. So in the past we could tell when a computer was performing a task as it was very clumsy. Now we’re at a stage when they appear very competent at certain tasks (better than average) but often fail to beat experts at these tasks. So pay attention to AI benchmarks and the progress that is happening when compared to human performance. It helps you see the forest before the trees.

What do you think about the progress in AI?