Choose your language:

Hong Kong
New Zealand
United Kingdom
United States
robot with headset waving

Tech Throwback Thursday: A Turing Test retrospective

January 02, 2015
By Alexander Lucas

As the 2014-2015 awards season kicks off, a lot of people will pay attention to "The Imitation Game," a film about Alan Turing, a brilliant mathematician who is well-known for his work in the concept of artificial intelligence. This attention is likely to spark interest in the programs that tried to pass the Turing Test and what artificial intelligence really means. The promises and fears of practical artificial intelligence are often bandied about hypothetically, but it is fascinating to see what AI has surfaced since the early 1950s.

"I propose to consider the question, 'Can machines think?'" 

In 1950 Alan Turing described a party game in which a judge could not see two players and had to determine the genders of each. One player would try to assist the judge, while the other would try to trick the judge.  

The idea of the Turing Test in computers is similar. A computer would replace one of those unseen players and the judge would try to determine which player was human and which was AI. There are several variations on the implementation of this test and differing bars of success, but simplistically, a first goal of AI is to "pass" the Turing Test.

Has any program passed the Turing Test? The answer depends on your viewpoint.

Eugene Goostman

Scanning the tech headlines in June 2014, a reader may have felt a rise of pride (or panic) at news of the first computer to pass the Turing Test. A program known as Eugene Goostman had convinced 33 percent of the assembled judges that it was human and not a chatting script.

Reading only the headlines probably made this feat seem more impressive than it really was. Web visitors could try out the program themselves and attempt to talk with Eugene. If nothing else, it was a neat way to draw attention to the 60th anniversary of Turing's death.

However, there were a couple of things wrong with this competition and Eugene's claim. 

  • The premise behind this program was that it was a 13-year-old Ukrainian boy. This was a deliberate attempt to not meet the full test of imitating an adult human, but rather to disguise potentially strange answers as a language barrier, teenage sass or youthful ignorance. Though not against the rules of the competition, this handicapping certainly skirted the spirit of the test.
  • The 33 percent success rate was much lower than with previous chat scripts, so claims of being the first would be wrong in any context. Older attempts like Jabberwacky and A.L.I.C.E. had much better success rates.

Out of curiosity, I thought I would also chat with Eugene, although of course I already knew he was a bot in advance. I was underwhelmed. Far from answering questions, most of the programs’ responses were deflections and requests for information about me. Eugene is a neat concept that begs the question of where this all began.

ELIZA and the rise of the chatterbots

A chatterbot is simply a computer program with natural language processing meant to provide seemingly normal conversation to a user's inputs. In theory, the concept is you could have a never-ending conversation with a chatterbot that would flow naturally from topic to topic.

You might call ELIZA, the first program of its kind, the Eve of chatterbots. Joseph Weizenbaum, while at MIT, created ELIZA to simulate a conversation with a psychotherapist. It was programmed with canned phrases in response to keywords. So for example, if you were to mention your daughter, ELIZA might ask about your family.

Despite the simplicity of ELIZA's logic programming, it had the ability to convince some users that it was human. As a first step, ELIZA was impressive, but the program’s limited responses kept it from passing the Turing Test.  

Modern chatterbots

Today you can still find chatterbots in existence, either as phishing scams on messaging services or as legitimate helping tools on some major websites to provide assistance in finding the right things. Turing Tests like the Loebner Prize are held each year to reward the best-performing chatterbot.

Interested in trying one out for yourself? Eugene Goostman is no longer available for chat, but these successful bots would be more than happy to talk with you: 

Do you have any experience with chatterbots? What is the most interesting thing they have said to you?  Let us know in the comments below. And be sure to check out our last Tech Throwback Thursday: The Nintendo Power Glove.

A self-styled storyteller, Alexander Lucas loves to share his vast knowledge of tech, innovation and design trivia. TEKsystems’ resident video designer is also an avid history buff and writes about technology innovation through time.

Blog Archive