A man holding a post-it note with AI on it

What is ChatGPT?

Print Friendly and PDF

Posted: May 29, 2023 | Categories: Artificial Intelligence

Several friends asked me lately what I think of ChatGPT and the other chat systems we hear a lot about lately on television and other media. The world is absolutely freaking out about this tech with doomsayers announcing the end of humanity and pundits digging up pictures of the Terminator for their posts.

ChatGPT isn’t really artificial intelligence (AI), it feels like it is, especially since the ‘conversations’ you have with it seem so… conversational. Its how intelligent feels that scares people today. At the end of the day, it’s just a next word predictor that uses a machine learning (ML) model to do the prediction, nothing more.

Now, people working in the AI field are going to take offense to my statement about it not being AI, so let me explain.

ChatGPT isn’t intelligent; it’s the application of a very sophisticated machine learning model that predicts the next word in a sentence. That’s it! That’s all it does (as I understand it). It appears to be intelligent because of what it does, but it’s really not. It’s just the result of a very smart mathematical algorithm and it will NOT take over the world and destroy humanity. It can’t, because, in the immortal words of Terry Gilliam in Monty Python and the Holy Grail, “it’s only a model.”

It uses a probability calculation to predict what the next word will be in a sentence or paragraph, based on previous words in the sentence or paragraph and the data set used to train the model.

If you give it a small training set, say for example just the Gettysburg Address, ChatGPT has almost no chance of predicting anything. However, if you give it a very large data set to train from, say the contents of the Internet or a complete library of books, or even a subset of those, the probability that ChatGPT can guess the next word increases dramatically. With the whole internet at its back, any algorithm will seem intelligent.

When you train a model with a very large (huge) data set and pass a sentence or paragraph repeatedly through the model (recursively more or less), adding each prediction then running it through again, you get a system that’s able to write sentences and paragraphs for you.

The power is in how the algorithm categories and stores analysis results, how much data the model has to work from, and how well it can find and use the data it has.

So, how does it work? I don’t know, I’m not an AI scientist. But I believe it’s something like this:

  1. The user gives it some input like a question or a command
  2. It uses Natural Language Processing of your question or command to build a semantic model of the input. At this point, ChatGPT knows what it thinks you said or asked for.
  3. It applies its prediction model to the semantic model from your input and predicts the first word in a response.
  4. It repeatedly feeds the semantic model and the generated output through the model again until the probability of it being able to predict the next word falls below a certain threshold.

That’s it, as far as I can tell - no way for that to take over the world and destroy humanity.

Does it understand previous queries and can it use that data to help improve the quality of responses for subsequent queries? Yes, absolutely. But it likely doesn’t keep anything but the current conversation in context. It could track the requests and responses from all queries worldwide and use that to improve prediction results, perhaps by users rating the results of every query, but the data set already consumes a tremendous amount of storage and the engine consumes a tremendous amount of computing power (compute) to process the queries.

Andreessen Horowitz published an article that discusses the compute limitations of AI in Navigating the high Cost of AI Compute.

As I mentioned this briefly in Islands of AI Everywhere, stand-alone AI models are useful and interesting, but until you connect them together, and by this I mean millions of models, we’re not anywhere close to having true intelligence in AI.

With ChatGPT, there isn’t a huge bank of compute and storage keeping track of everything and making decisions. The only decision this algorithm makes is calculating the probability of a word being the right word for the next word in a sentence. Nothing more.

Will we reach a time where we do have millions and more ML models feeding into each other to accomplish great things. Yeah, I think so. Will this happen in my lifetime? No, I doubt it. Will this produce sentience? Real intelligence? It can, I think, but the compute and storage requirements for this are beyond comprehension today. So many technological advances are required coupled with everything in our world being connected. It is possible, even likely, but I don’t see it happening within 1,000 years or more (and I read a lot of Science Fiction).

OK, what is going to happen next?

Well, people are going to lose jobs as IBM has already announced Job cuts may intensify as IBM plans to use AI to replace 30% of back office jobs because there are things that ML models (not AI) can do faster, more efficiently, and even more accurately than humans can.

Next, and this is the scary part, companies will rush to implement ML-based solutions in their business processes and products. In this rush to implement ML, these companies will cut corners to get to market first, beating their competitors, and won’t implement properly tested models or implement solutions with the right guardrails.

Companies will be so excited about the efficiency improvements or cost reductions that they’ll start applying ML models to things that they shouldn’t. This is why several computer and software industries approached congress years ago about government controls on the use of AI. Here are some articles regarding Microsoft’s view on this.

In this rush to implement ML solutions, people will get hurt or killed, there’s no doubt about it. Again, companies will exaggerate the quality of ML models and push them into products before they’re ready and when you trust something that’s not fully tested or can’t handle every scenario, it is going to fail and people will suffer. I’m not only talking about physical harm, but emotional harm as well.

Imagine how quickly someone will rush a stock portfolio management model to market and trusting individuals invest their entire savings in the program. When the algorithm fails to deal with an unexpected action in the market, those savings plans will disappear.

Imagine too what happens when companies apply ML to their accounting systems and customer service processes. How hard it will be to resolve issues when no one anymore understands these systems or has access to make changes manually. A great example of this was what happened when FedEx eliminated call centers when they believed customers could do everything online but their online systems couldn’t handle the load during the holidays.

Untangling the mess created by application of faulty ML models to anything will be a nightmare that we have to live through.

We live in very interesting times and I’m excited to see this, but it’s going to be a rocky road unless Governments figure out how to create guardrails or, in absence of that, companies self-regulate themselves (which doesn’t seem likely).

Photo by Hitesh Choudhary on Unsplash.

Next Post: Algirithms Making Stuff Up

Previous Post: Islands of AI Everywhere