Can ChatBots Increase Student Engagement And Learning? [Case Study]

Posted on by

SIRI

 Our ability to speak and have conversations to pass on knowledge is one of our most cherished human qualities.   As a young father, I have enjoyed seeing my little ones learn new words and learn how to communicate their needs.  Learning language is one of our first big lessons of life.

For me, it has been exciting to see the increasing capacity of computers emulating humans.   In chess programs, computers show remarkable strengths in solving planning problems.   Researchers have built cars that can drive without human guidance over challenging desert.  [2] Programs like WATSON can now win game shows like Jeopardy. [3] This is amazing science!

Emulating human conversation, however, is still a frontier to be conquered.    While programs exist that emulate human conversation (i.e. chatbots), these programs still feel like computer programs.   We can do better!  Researchers from the center for Psychology at Athabasca University performed a study to answer the question if chatbots can emulate a historical figure and serve as a teaching tool in their distance learning programs.   For complete details on their study, please visit the following link.

Results from study:

A sample of 53 students from Athabasca University psychology students were asked to chat with FreudBot for 10 minutes, a chatbot programmed with domain knowledge of Sigmund Freud.   The students were asked focused questions regarding the quality of the activity and how they would prioritize improvements to the Freudbot.  Students replied to questions using a scale from 1 to 5 (1=negative, 5=very positive).  [1]

Let M = {average response from students}

  • Would you recommend this activity be expanded to include other topics? Yes (M=4)
  • How engaging did you find this activity? Yes (M=3.08)
  • How memorable do you think this activity will be for learning about Freud? (M=3.04)
  • How useful did you find this activity for learning about Freud? (M=2.96)

The study observed that students had no issues on staying on task.

Opportunities for improvement for FreudBot:

  • The students desired to see improvements in the quality of chat responses.
  • The students also desired to see improvements in the audio response of the system.

Personal reflections:

I believe we can achieve greater educational impact by exploring the following questions and observations:

  • ChatScript: This study used a technology known as AIML.  In particular, I would be interested in seeing if ChatScript can produce measured improvements in the quality of chat responses.   AIML pattern matches by string or text matching.  Let’s say you wanted to make your chatbot vegetarian.   In AIML, you would need to manually encode rules for every form of meat. (i.e. turkey, bacon, ham, steak, etc.)   Using ChatScript’s concept syntax (note the ~), you could express the rule elegantly in one line of code. [4]?: (Do you like ~meat?) No.  I am vegetarian.To learn more about the advantages of ChatScript from the creator of the system, please visit a .   It should be noted that Bruce’s work won the Loebner Prize in 2010 and 2011.
  • Less is more: Is it easier to improve the educational impact of the chatbot when fewer rules need to be drafted and maintained?
  • Student modified chatbots: Can we make changing chatbots accessible to students? What would be the benefits of students correcting the chatbot?
  • Video: Let’s say we’re talking with a chatbot of Albert Einstein.   If the student asks “what is your theory of relativity,” then the system can provide a brief answer with a link to a YouTube video.
  • Link to virtual worlds: Can the learning outcomes and the level of engagement improve if the students used SecondLife or Minecraft to encounter the historical character?   In an American history class, it would be cool to encounter Benjamin Franklin in his print shop or library in a virtual world.

 

As always, we love to hear your comments and ideas.    Have a great week!

 

Please consider supporting our Kickstarter project.  

[1] Heller, B., Proctor, M., Mah, D., Jewell, L. & Cheung, B. (2005). Freudbot: An Investigation of Chatbot Technology in Distance Education. In P. Kommers & G. Richards (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2005 (pp. 3913-3918). Chesapeake, VA: AACE.

[2] http://en.wikipedia.org/wiki/Stanley_(vehicle)

[3] https://en.wikipedia.org/wiki/Watson_(computer)

[4] http://sourceforge.net/projects/chatscript/