It has been claimed that Chat-GPT tests in the top 1% when it comes to creativity, using the Torrance Test of Creativity. I used to work at the Torrance Center and I am familiar with the test and was trained in scoring it. I am highly skeptical of the claim the researcher made that Chat-GPT has “creative abilities.”
In a sea of articles proclaiming the wonders of Chat-GPT’s creativity given its test results without looking deep into it, I found one article in which the author, Eric Holloway, took the time and patience to apply critical thinking to looking at the results. He is not even against AI. He confirmed the ways I have been skeptical:
- Chat-GPT had scored in top 1% for originality. The problem is that all the ideas that Chat-GPT drew from is from its internet database. This is equivalent to an open-book test. None of the human test-takers whom Chat-GPT were compared against were allowed to access the internet or open a book. Would you say reading ideas from the internet or book is creative? Those ideas still come from humans.
- Chat-GPT had scored in top 1% in fluency (the ability to come up with ideas that are coherent). The problem is that the researchers selected responses of Chat-GPT and submitted it to the test. It is humans who are deciding which ideas are coherent. This must mean while incoherent responses given by Chat-GPT were left out of the test, those made by human subjects were left in the test. Is that a fair comparison?
- The author expressed how Chat-GPT does not show the ability to be insightful like a human creative. Its answers all come from chance and he writes “it just gets lucky.” After he tested it long enough, its creative responses become repetitive.
Despite the researcher of the Chat-GPT study, Dr. Erik Guzik, having stated he is “careful…not to interpret the data very much” he also said, “AI seems to be developing creative ability on par with or even exceeding human ability.” This is an unusually bold and uncareful conclusion for a researcher to make and present to the public. It encourages the public to read the test results in an uncritical way. Chat-GPT is very far from being proven to have creative abilities, even according to the results of the test.
You can see by the way this research is being hyped up on the internet and in the news, how this misinformation hurts creative professionals and creative people in general. It gives the impression of devaluing creative people, and businesses will eat the presentation of this research up and use it as an excuse to make them jobless.
The misrepresentation of this research can be disempowering and discouraging for creative people to hear, and I know it has upset many of them. Creative children who may think twice about developing their creative skills if a machine is able to be “in the top 1% in creativity.” Creatives are a population that is neurodivergent, meaning they have strengths and weaknesses different from most of society. Their strengths are often already undervalued, and society keenly sees their weaknesses. People in society may find excuses to devalue them more.
The nature of human creativity is very much a mystery. Chat-GPT scored in the top 1% on a creativity test, but it still may not even be creative at all. Comparing the processes Chat-GPT and humans used to come up with the results of the test is comparing apples and oranges. When humans take the test, they do not have access to a database. In order to come up with answers to a question like “How many different ways can you use a baseball bat?” Chat-GPT can read off a database of answers, but humans need to use their imagination. Even if the answers already existed before, they are not freshly prepared in the human mind when they are given this question. Human creativity involves feelings and consciousness, which Chat-GPT admits it does not have. Not least to say, a test cannot capture the full nature of creativity. I think comparing Chat-GPT and humans paradoxically shows how ineffable and intricate the human creative process is.
