Turing test beaten by Russian chatterbot

Sunday, June 8, 2014

Yesterday, the Russian chatterbot “Eugene Goostman” by a team of Russian and Ukrainian developers became the first machine to pass a Turing test, under the academic event organizers’ interpretation of the test as originally described by British mathematician Alan Turing. The competition was held at the Royal Society in London, England, and was organized by Kevin Warwick and Huma Shah of the University of Reading to mark the 60th anniversary of Turing’s death on June 7, 1954.

The Turing test is a test of artificial intelligence aiming to fulfil the suggestion of Alan Turing in his 1950 paper “Computing Machinery and Intelligence”, which stated that within fifty years, an “average interrogator” would, following a five-minute long conversation, “not have more than 70 per cent chance” of correctly predicting whether they are speaking to a human, or a machine — which would be able to, as such, fool at least 30% of human judges into thinking it is human.

In the contest, where Eugene Goostman and four other bots competed, the bot successfully tricked 33% of the participating judges, which included television actor Robert Llewellyn of the BBC television series Red Dwarf, and John Sharkey, Baron Sharkey, a sponsor of Turing’s 2013 posthumous pardon. To give the bot a “believable personality”, Goostman is portrayed as being a thirteen year-old boy of the Ukraine; the bot’s head developer Vladimir Veselov stated that this made Goostman “not too old to know everything and not too young to know nothing”.

The bot had previously come close to beating the Turing test on several occasions; it has been a three-time runner-up for the Loebner Prize, and it won a Turing contest at Bletchley Park in 2012, held to mark the 100th anniversary of Turing’s birth. In the 2012 Bletchley Park competition Goostman was, notably, only one percent away from the target of 30%.

Speaking about the achievement, Warwick stated: “Some will claim that the Test has already been passed. The words Turing Test have been applied to similar competitions around the world. However this event involved the most simultaneous comparison tests than ever before, was independently verified and, crucially, the conversations were unrestricted. A true Turing Test does not set the questions or topics prior to the conversations. We are therefore proud to declare that Alan Turing’s Test was passed for the first time on Saturday.”

Veselov felt that the achievement was “remarkable”, and suggested that it could help increase interest in artificial intelligence and chatterbot technology.