Menu
These graduate students had no idea their teaching assistant was a robot

These graduate students had no idea their teaching assistant was a robot

'Jill Watson' now answers students' questions with 97 percent certainty

On the Internet, "nobody knows you're a dog," as the old meme goes, and today, the same can increasingly be said of robots.

There are already scheduling robots that are virtually indistinguishable from humans, and recently students at the Georgia Institute of Technology learned that "Jill Watson" -- a teaching assistant they had relied upon all semester -- was in fact artificially intelligent.

“The world is full of online classes, and they’re plagued with low retention rates,” said Ashok Goel, a Georgia Tech professor who teaches a class entitled Knowledge-Based Artificial Intelligence. “One of the main reasons many students drop out is because they don’t receive enough teaching support. We created Jill as a way to provide faster answers and feedback.”

The class is a core requirement of Georgia Tech’s online master’s program in computer science, and it tends to draw a lot of questions from students. It's offered every semester, and each time, the 300 or so students enrolled post roughly 10,000 messages in the course's online forums, Goel estimates.

That volume has often overwhelmed Goel and his eight teaching assistants, so this time, he added a ninth: Jill.

Jill is a virtual TA, and -- as her name suggests -- she's implemented on IBM’s Watson platform. Goel and his team of graduate students started to build Jill last year, including training her on the roughly 40,000 questions that had been asked in the KBAI class since it was first offered in fall 2014.

“One of the secrets of online classes is that the number of questions increases if you have more students, but the number of different questions doesn’t really go up,” Goel said. “Students tend to ask the same questions over and over again.”

Jill started work as a TA in January, but she wasn’t very good for the first few weeks, often giving odd and irrelevant answers. Her responses were posted in a forum that wasn’t visible to students.

“Initially her answers weren't good enough because she would get stuck on keywords,” said Lalith Polepeddi, one of the graduate students who co-developed the virtual TA.

For example, on one occasion, a student asked about organizing a meet-up to go over video lessons with others, and Jill gave an answer referencing a textbook that could supplement the video lessons: same keywords, but different context.

Further tweaking ensued, and today Jill can answer questions with 97 percent certainty. Initially, the human TAs would upload her successful responses to the students, but by the end of March, Jill didn’t need any assistance: She wrote to the class directly if she was 97 percent positive her answer was correct.

The big reveal didn't happen until late April, when Goel informed his AI students that they had actually been interacting with a bot all semester. One reportedly said her mind was "blown." Since then some students have organized an alumni forum to learn about new developments with Jill after the class ends; another group has launched an open-source project to replicate her.

Next semester, Jill will return with a new name and a new goal: to answer 40 percent of all class questions by the end of the year.


Follow Us

Join the newsletter!

Or

Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

Featured

Slideshows

Channel gathers to celebrate the Reseller News Innovation Awards 2020 winners

Channel gathers to celebrate the Reseller News Innovation Awards 2020 winners

More than 500 channel leaders gathered in Auckland on 21 October at the ​Reseller News Innovation Awards ​2020 to celebrate the achievements of the New Zealand technology industry's top partners, start-ups, vendors, distributors and individuals.

Channel gathers to celebrate the Reseller News Innovation Awards 2020 winners
Meet the winners of the 2020 Reseller News Innovation Awards

Meet the winners of the 2020 Reseller News Innovation Awards

Reseller News honoured the standout players of the New Zealand channel in front of more than 500 technology leaders in Auckland on 21 October, recognising the achievements of top partners, start-ups, vendors, distributors and individuals.

Meet the winners of the 2020 Reseller News Innovation Awards
Meet the Reseller News 30 Under 30 Tech Awards 2020 winners

Meet the Reseller News 30 Under 30 Tech Awards 2020 winners

This year’s Reseller News 30 Under 30 Tech Awards were held as an integral part of the first entirely virtual Emerging Leaders​ forum, an annual event dedicated to identifying, educating and showcasing the New Zealand technology market’s rising stars. The 30 Under 30 Tech Awards 2020 recognised the outstanding achievements and business excellence of 30 talented individuals​, across both young leaders and those just starting out. In this slideshow, Reseller News honours this year's winners and captures their thoughts about how their ideas of leadership have changed over time.​

Meet the Reseller News 30 Under 30 Tech Awards 2020 winners
Show Comments