Takako Aikawa, in red, poses with Digital Humanities Lab students, from left: Shara Bhuiyan ’24, Jackie Lin ’24, Yifan Wang ’24, and Peihua Huang ’24.
PHOTO: M. SCOTT BRAUER
Consider a student of English trying out a new app for language learning. As the student begins to a type a story— say about a duck swimming in a river—images pop up on her screen, illustrating her original composition in real time. “The app picks up a duck image from the web, then a river image, so you can basically create your own picture book,” says Takako Aikawa, senior lecturer in Japanese at MIT Global Languages in the School of Humanities, Arts, and Social Sciences (SHASS). “As you type, it can help you visualize what you just wrote.”
That app, dubbed Story Maker, is just one of a number of innovative new learning applications Aikawa has helped students create as part of an interdisciplinary collaboration fostered by the Programs in Digital Humanities, informally known as the Digital Humanities Lab (DH Lab). Launched in 2019 with a $1.3 million grant from the Andrew W. Mellon Foundation and based in SHASS, the DH Lab integrates digital and humanities education, teaching, and research.
Since its creation, the DH Lab has launched 23 diverse projects in collaboration with 10 disciplines in SHASS and the School of Architecture and Planning, exploring efforts such as analyzing the formal content of historical photographs through machine learning to creating tools for teaching and learning about the spread of democracy in Africa through game elements. With approximately 30 participating students each semester, the lab is now the largest Undergraduate Research Opportunities Program (UROP) host on campus.
Students working with Aikawa through UROP experimented with technologies such as visual recognition, text-to- speech, and natural language processing to push the bounds of language learning software far beyond point-and-click flash cards. “The beauty of this project is it was really the students’ creativity that led the way in utilizing existing technologies to design new tools,” Aikawa says.
Aikawa led the project as a digital humanities faculty fellow embedded in the DH Lab for a semester. “Our fellows reap the rewards of the amazing work that MIT undergrads can do developing computational methods and tools,” says Stephanie Ann Frampton, faculty director of the DH Lab and associate professor of literature. “In turn, work in the lab provides a unique opportunity for students to have a crash course in a humanities or social sciences field.”
Sharing knowledge
Before coming to MIT to teach Japanese in 2013, Aikawa worked at Microsoft Research as a computational linguist. At MIT, she has worked to create language-learning apps using natural language processing and other computational tools. For this project, she started by teaching students about linguistics and language pedagogy, helping them think like language instructors about how best to engage students. “I hope that they get at least a little taste of the complexity of human languages,” she says.
Meanwhile, DH Lab faculty and staff provided the scaffolding to help students design and program the apps using a variety of popular technologies. “It was really about creating a space in which both the humanities and computing sides were able to communicate with each other and speak each other’s language—no pun intended,” says Ryaan Ahmed, the lab’s technical director, who also develops music software for the Cambridge-based company Artusi. Before even starting to code, students consulted with Aikawa, Ahmed, and lead developer Michael Jean to design interfaces that would be both easy to use and aesthetically pleasing, wireframing apps with design software.
“It’s different than what students are usually doing in computer science classes, in which they are trying to solve problems by writing code,” Ahmed says. “This may be their first time really thinking about the end-user experience.”
Among the apps they created was one that automatically generates crossword puzzles and hangman games from a list of vocabulary and definitions. Another creates flash cards from images. “It uses object detection to find all of the objects in a scene and then constructs these rich flash cards where students have to identify the objects,” says Ahmed. The app helps instructors generate content quickly while teaching students words in context.
Another application allows an instructor to upload text in the form of a dialogue, then prompts students to complete half the conversation using multiple choice. Students had to come up with techniques for the app to automatically generate wrong answers. “Instead of an instructor creating a multiple-choice quiz with one right answer and three wrong answers, the system created the dialogue,” says Ahmed. “Students had to think about what it means to create a plausible wrong answer.”
Creating so many different kinds of modules for language learning allowed students to learn in a variety of ways, using speech, text, and visual prompts. “I learned a lot more about full-stack development and how computer science is used outside of the school setting,” says Peihua Huang ’24, herself a non-native English speaker who joined the project out of a desire to help others learn English in a fun and meaningful way. Another student, Shara Bhuiyan ’24, appreciated the freedom students had to bring their visions to fruition. “We were given creative control over the projects,” she says. “Senior lecturer Aikawa and Ryaan were both very supportive in the process, providing us with guidance in whatever direction we decided to take the project.”
Future collaboration
The apps were designed as asynchronous tools that could supplement the remote learning environment of the Covid-19 pandemic. However, the DH Lab and Global Languages—which share space on the sixth floor of Building 16—are now discussing ways they can further collaborate, including by expanding to support languages beyond English. Frampton, who also teaches Latin in MIT’s Ancient and Medieval Studies program, says, “I’m already looking forward to using the hangman and quiz-generator tools.”
The DH Lab, Frampton says, fulfilled students’ interest in learning about new coding software, such as NLTK—the Natural Language Tool Kit used for processing human languages in Python. But it also sparked their interest in the more human side of language learning. “We saw that they were equally motivated by the pedagogical mission of Takako’s project—how do we make teaching and learning easier, more fun, and more natural for faculty and students at MIT and beyond?”
In that regard, says Aikawa, the student projects exceeded her expectations, creating applications that are both technically complex and addictively enjoyable to use. “The bottom line is that these applications are fun,” she says. “Learners love to keep using them, and that enables them to learn a foreign language much more quickly.”