Artificial Intelligence Done Right

At VitalSource®, we have been at it a long time with AI technology. Our efforts around AI date back to the work we did at Carnegie Mellon University, our Doer Effect research, and our Bookshelf CoachMe automatic question generation. We must protect student learners and we believe being grounded in learning science and taking a research-backed approach is the first step. We incorporate AI when appropriate and leverage the strengths of that technology to deliver positive learning experiences rooted in learning science. Critically, our question generation technique does not rely on creating text from scratch using generative AI, so our questions are guaranteed to be right and not introduce factual errors.

There are real risks for students when we provide tools that aren’t rigorous. Right now, ChatGPT is the tool that is making the most waves. But here’s what we need to remember about this:

• There is no guarantee that the information that ChatGPT gives students is correct. In fact, it is quite easy to get ChatGPT to say incorrect things. 
• ChatGPT is not based on learning science.  
• A good reason to use ChatGPT is when the user is the subject matter expert. The SME will know if the response is correct or incorrect. Students are not experts. They do not always know how to distinguish fact from fiction, and ChatGPT’s output often does not achieve education-level reliability, despite appearing as coherent and well-written as a textbook.
  

As Ethan Mollick says in a recent post, "The world is changing fast. But one thing that is not changing is the best way for people to learn." Generative AI like ChatGPT has potential for enhancing educational applications. However, it is still in its infancy stages. While we are exploring ChatGPT for expanding our learning science-based experiences, much of our work is about developing ways to ensure those experiences are sound, reliable, and effective. This includes things like validating ChatGPT's output before we include it in an activity, as we already do with our existing AI-generated questions.

There’s still a lot of research to be done to understand how to use generative AI in ways that are effective and safe, especially for student users. EdTech companies owe their student users the protection that they will do that research and make sure to use these tools in the right way, even if that means going slower in order to be more rigorous.

In 2021, VitalSource launched Bookshelf CoachMe, a new feature within Bookshelf, powered by AI, that is designed to boost student engagement by providing embedded practice questions while they read their textbook. This Learn-by-Doing approach, based on learning science, improves the overall study experience by helping students discover what they know and what they don’t, and where to focus next. As the name suggests, it’s like a free, built-in study coach available to students whenever they need it.

VitalSource’s team of learning scientists studied the efficacy of questions generated through AI in an award-winning 2021 paper, proving that these questions are equivalent to human-authored questions in several different ways.  All questions are AI-generated, aligned to the core texts and delivered through VitalSource’s Bookshelf platform. These AI-generated practice questions appear alongside the etext to give students a chance to pause and practice key concepts while studying without being disruptive. Questions are also available at the end of each chapter to boost student confidence and provide insight into individual knowledge gaps.

Since its launch, this AI tool has created more than 2 million unique study questions, all backed by learning science and based on approved and assigned texts. And so far, more than 300,000 students have engaged.

There is no shortcut for teaching. You may be able to outsource finding quick answers to a tool like ChatGPT, but not teaching. As we come to market with these tools, we must look to instructors to be partners. 

We have done a great deal of research that shows the instructor has the greatest influence over how students use their learning resources—and how students use their resources impacts their outcomes. For example, a recent study showcases how an instructor applied implementation practices that incentivized student engagement, which increased student engagement dramatically and drove up exam scores, across multiple courses and semesters. The first challenge is to create research-based learning tools and the second is to work with instructors to apply them in such a way as to maximize student use of those tools.   

An interesting finding from a recent study by Tyton Partners shows that among faculty who have not tried generative AI tools firsthand, they firmly believe they will have a negative effect on student learning. Among faculty who have tried them, they believe they will have a positive effect. 

As we think about how to work with instructors, we can drive positive experiences and results as long as students understand how to use these technologies thoughtfully while acknowledging the limitations. Our research proves instructors are an invaluable partner in that effort.