This February, OpenAI’s ChatGPT passed a theory-of-mind test where scientists assessed the unobservable mental state — a form of self-awareness, and the ability to comprehend other’s thoughts.
The discovery was made when Michal Kosinski, a computational psychologist and professor at Stanford University, ran tests to see if ChatGPT could ascertain unobservable mental states, like desires and beliefs, to others. If the tests were successful, it would suggest that ChatGPT does in fact possess theory of mind.
Guess what he found?
Amazingly, even though ChatGPT wasn’t actually trained or even built to perform theory of mind tasks, it had performance comparable with that of a nine-year-old.
Michal said: “An alternative explanation is that ToM-like ability is spontaneously emerging in language models as they are becoming more complex and better at generating and interpreting human-like language. This would herald a watershed moment in AI’s development.”
Since ChatGPT isn’t actually trained on ToM, it would suggest the ability emerged spontaneously. 🤯
Kosinski stresses in his paper that the “results should be interpreted with caution.” However, he suggests it’s possible that ChatGPT’s ability to pass these tasks was “a byproduct” of its mounting language ability. Alternatively, he poses that it might just be using its incredible flair for language to give the superficial impression it’s engaging in theory of mind thinking.
ChatGPT even outperformed an Alphabet (Google)-owned company, DeepMind Technologies–a British AI research lab founded in 2010, acquired by Google in 2014. DeepMind was developed just for theory of mind tasks, but it only compared to a 4-year-old in ability.
Since I now work full-time within an AI company (Content at Scale), and am close-up watching and working with an incredible AI Justin McGill built using three different language models and proprietary software, capable of writing all kinds of incredible long-form content for a myriad of clients and industries that actually rivals content writers, I’ve been increasingly obsessed with the idea of artificial superintelligence.