Have you ever been advised to “walk a mile in someone else’s shoes?” Thinking from another person’s perspective can be a difficult endeavor – but acknowledging our own mistakes and biases is key to building understanding across communities. By challenging our preconceived notions, we confront prejudice, such as racism and xenophobia, and perhaps develop a more inclusive perspective on other people.
To aid perspective adoption, MIT researchers developed “On the Plane,” a virtual reality role-playing game (VR RPG) that simulates discrimination. In this case, the game depicts xenophobia directed against a Malaysian-American woman, but this approach can be generalized. On a plane, players can take on the role of characters from different backgrounds, engaging in dialogue with others while making in-game choices for a series of prompts. In turn, the players’ decisions control the outcome of a tense conversation between characters about cultural differences.
As a VR RPG, “On the Plane” encourages players to take on new roles that may be outside of their personal experiences in a first-person view, allowing them to confront in-group/out-group bias by incorporating new perspectives into their understanding of different cultures. Players interact with three characters: Sarah, a first-generation Muslim American of Malaysian descent who wears a hijab; Marian, a white woman from the Midwest with little exposure to other cultures and customs; or flight attendant. Sarah represents the outgroup, Marian is a member of the group, and the flight attendant is a bystander who witnesses an exchange between the passengers.
“This project is part of our efforts to harness the power of virtual reality and artificial intelligence to address social ills, such as discrimination and xenophobia,” says Caglar Yıldırım, a research scientist at the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL), co-author and game designer involved in the project. By taking turns between the two passengers, players experience how one’s xenophobia manifests itself and how it affects the other passenger. The simulation engages players in critical thinking and seeks to foster empathy for the “other” passenger since her clothing is not a “prototype” of what an American should look like. .
Yıldırım worked alongside the project’s principal investigator, D. D., director of the MIT Center for Advanced Virtual Reality. “It’s not possible for a simulation to give one person the experiences of someone else’s life, but while you can’t ‘walk in someone else’s shoes’ in that sense, a system like this can help people recognize and understand social patterns at work,” says Harrell, also an author. Participant and designer of this project, “Like Prejudice.” “An engaging and immersive interactive narrative can also affect people emotionally, opening the door to changing and broadening users’ perspectives.”
This simulation also uses an interactive narrative engine that creates many choices for responses to in-game interactions based on a model of how people are socially categorized. The tool gives players a chance to change their standing in the simulation with their response choices for each prompt, affecting their affinity towards the other two characters. For example, if you’re playing the flight attendant, you can respond to Marian’s anti-alien expressions and attitudes toward Sarah, and switch both of your affiliations. The engine will then provide you with a different set of narrative events based on your changes in standing with others.
To animate each avatar, “On the Plane” includes AI knowledge representation techniques that are controlled by finite-likelihood state machines, a tool commonly used in machine learning systems for pattern recognition. With the help of these machines, the characters’ body language and gestures can be customized: if you play Marianne, the game will customize her behavior towards Sarah based on the user’s input, affecting how comfortable she is to appear in front of a perceived group member. . Likewise, players can do the same from Sarah’s point of view or from the hostess’ point of view.
In a 2018 paper based on work done in collaboration between MIT CSAIL and the Qatar Computing Research Institute, Harel and co-author Sercan Şengün call for designers of virtual systems to be more inclusive of identities and habits in the Middle East. They claimed that if the designers allowed users to customize virtual avatars that were more representative of their background, this could enable players to engage in a more supportive experience. Four years later, “On the Plane” achieves a similar goal, which is to incorporate a Muslim perspective into an immersive setting.
“Many default identity systems, such as avatars, accounts, profiles, and player personas, are not designed to meet the needs of people across diverse cultures. We have used statistical and AI methods combined with qualitative methods to see where the gaps are,” they point out. “Our project is helping to bring about a shift in perspective so that people treat each other with respect and promote understanding through representations of diverse cultural avatars.”
Harel Wilderim’s work is part of MIT’s IDSS initiative to combat systemic racism (ICSR). Harrell is a member of the initiative’s steering committee and is the leader of the newly formed Anti-Racism, Games and Immersive Media Unit, which studies behavior, cognition, social phenomena, and computational systems related to race and racism in video games and immersive experiences.
The researchers’ latest project is part of the ICSR’s broader goal to launch and coordinate interdisciplinary research that addresses the processes of racial discrimination across American institutions. Using big data, members of the research initiative develop and use computing tools that advance racial equality. Yildirim and Harel achieve this goal by depicting a problematic recurring scenario that shows how prejudice infiltrates our daily lives.
“In the post-9/11 world, Muslims often suffer from racial profiling at American airports. On the Plane builds on this,” says MIT Professor Fotini Christia, director and co-director of the Social Systems Research Center (SSRC) or IDSS. type of collective favoritism, which is a well-established finding in psychology.” This game also takes a novel approach to analyzing fixed bias by using virtual reality rather than field experiments to simulate bias. Excitingly, this research shows that VR can be used as a tool to help us better measure prejudice, and combat systemic racism and other forms of discrimination.”
On the Plane was developed on the Unity game engine using the XR interaction toolkit and Harrell’s Chimeria platform to create interactive novels incorporating social rating. The game will be published for research later this year on both desktop computers and the standalone Meta Quest Wireless headset. A paper on the work was presented in December at the 2022 IEEE International Conference on Artificial Intelligence and Virtual Reality.