For the third semester, Ashok Goel, Ph.D., will use AI teaching assistants for his course on Knowledge-Based Artificial Intelligence in the online Master in Computer Sciences program.
The TAs, which are implemented on IBM’s Watson platform, are responsible for assisting through the class’ online forum, where they answer questions, post class updates and even engage in casual conversation with students.
The virtual TA led its first class in Spring 2016, under the alias Jill Watson. Goel did not reveal that Jill was an artificial intelligence bot until after final exams, resulting in an overwhelmingly positive response from the students.
“One student wanted to nominate Jill for the outstanding TA award; another wanted to take Jill out for dinner!” Goel said.
The project received national press coverage, and got picked up by The Washington Post, The Wall Street Journal and other major news outlets and, thus, brought widespread attention to the potential uses of AI in the classroom and beyond.
Without human guidance, Jill was able to answer a number of common questions from students. The course instructors generally receive over 10,000 questions a semester; these were fed to Jill in order to help her learn.
Jill utilizes the technology from IBM’s Watson, a system designed to answer questions posed in natural language. Originally designed to answer “Jeopardy!” questions, Watson is now being applied in a variety of fields, including finance and health care.
Goel and his team spent several weeks testing Jill on a separate, hidden forum. Initially, this process was a struggle for the AI bot; while the system was designed to look for keywords, it would sometimes misinterpret their context, causing her to confuse questions that were worded similarly but had different meanings.
Goel and his team modified the software to improve Jill’s decision-making ability. Afterward, Jill was communicating directly with students whenever she was 97 percent confident that her answers were correct.
For the most part, very few of the students noticed anything unusual about their TA, who was fielding questions on topics ranging from the specifics of certain assignments to general questions about the subject of the class.
To give the outstanding TA a helping hand, Goel introduced a second virtual TA this past fall, hidden amongst the 15 TAs for the course.
At the beginning of the semester, Goel informed the students not all of their TAs were necessarily human.
Students were eager to hunt down the AIs, who were using the aliases “Stacy Sisko” and “Ian Braun”, respectively.
Ian, a bot essentially identical to the original Jill Watson used during the spring semester, was identified by about a sixth of students as being a bot, while Stacy, a slightly upgraded bot who was capable of more complex actions, like creating weekly wrap-ups of class content that referenced previous conversations on the forum, was chosen by fifty percent of students in the section as a bot.
Goel predicted that some students may have attempted to outsmart the AI TAs by asking questions in unique ways, allowing them to identify which TAs were human and which were bots.
Students also developed their own chatbots, based on Jill, that conversed about the course. Forty bots were developed over the course of the fall semester.
Analysis of preliminary results seems to suggests that the use of chatbots in the class resulted in more student engagement, with student comments going from 32 per semester to 38 per semester, on average.
This semester, Goel and his team are introducing new avatars of Jill in the class but are staying mum about the details in order to allow students to have an opportunity to solve the mystery.
The avatars use newly developed tech, and Goel’s team is optimistic about their performance.
They are also developing an AI Tutor designed to substitute for many roles a professor typically performs in the classroom; instead of merely answering commonly asked questions, this tutor will be capable of “grading assignments, examinations and programming projects, cognitive tutoring on the course materials, as well as metacognitive tutoring on open-ended projects,” Goel said.
His team plans to submit the project to the IBM Watson AI XPRIZE, a $5 million competition that is designed to encourage the development and implementation of world-changing AI technologies in scalable, effective ways that can help to solve the challenges that face our
collective and global society.
“Instead of a single, universal goal for all teams, this competition invites each team to define its own grand challenge,” Goel said. “It is a privilege and an honor to represent Georgia Tech in this competition.”
Goel anticipates that the use of AI in education will contribute to a style of learning that is “interactive and personal as well as immersive and social.”
While artificial intelligence software is not yet capable of accomplishing this goal, the rapid pace of development in the field and growing interest in AI from the general public are reflective of the promise that the field holds for the future.
In addition to the AI TA project, the lab Goel runs is also involved in other initiatives for
students’ learning.
“[We are] working on developing new technologies that will provide future students easy access to scientific literature relevant to their questions, problems and goals,” Goel said.
“This is so much fun because we get to live in future worlds of our own imagination.”