Eye Measurements as Objective Measures of Flow Experience

Author: Salma Elnagar
Mentors: Shinsuke Shimojo and Mohammad Shehata
Editor: Jagath Vytheeswaran

Abstract

Flow experience is achieved when a person is said to be “in the zone” as they achieve a fit between skill-and-challenge level in a certain activity. Many of these activities, such as video games, music, or athletic competitions, involve the participation or company of other people. Thus, interpersonal communication could be related to reaching the mental state of flow. Previous research has linked brain areas like the inferior frontal gyrus to individual flow. However, no experiments have yet been conducted on interpersonal flow. The large-scale purpose of this project is to fill this gap by identifying the neural correlations of interpersonal flow. In order to do so, objective measurements of the inherently subjective flow experience must first be identified. In this study, we evaluated eye measurements such as pupil diameter, blink rate and eye fixations as objective measures for individual flow. Two versions of our experiment using a musical notes game were conducted on ten and six subjects respectively with eye measures being collected using Eyelink 1000. While there was a significant difference in gaze direction among various gaming levels, there appeared to be no significant change in blink rates or pupil size. This suggests that gaze direction could be an objective measurement of flow experience.

Introduction

figure 1
Figure 1: Flow as a function of skill and challenge levels. Flow is a balance between overload and boredom.

Consider a gamer, oblivious to the outside world as they click and play for hours, thinking it has only been a few minutes since they started – or a soloist, absorbed by their music while performing Paganini’s complex caprices on stage, forgetting about the hundreds of people who gathered around to watch them play. These scenarios, along with many more everyday ones such as enjoying a gripping book or fully concentrating on an interesting math problem, expose one of the many marvels of the human brain: its ability to get in “the zone”. In such state, a person is fully and satisfactorily engaged in an activity to the extent that their perceptions of themselves, others, and time are dampened. In neuroscience terms, we dub this mental state “flow.” In a more concrete sense, flow is the skill-challenge balance between overload, in which a person perceives high challenge and low skill, and boredom, in which a person perceives high skill but low challenge. Learning more about flow is the first step towards creating interventions for clinical disorders of attention, like Attention Deficit Hyperactivity Disorder (ADHD).

To create and scientifically study this very subjective experience is not an easy mission. In order to create flow experience in the laboratory, we used an open-source music game where participants were asked to hit notes when they appeared on screen (Figure 2). Three conditions were created based on the difficulty of the game (defined by note frequency) according to each participant: boredom, flow, and overload conditions. Various eye measurements were then gathered and compared to answers in post-experimental surveys to determine the validity of said measurements as objective indicators of flow.

figure 2
Figure 2. Screenshots of the open-source music game. Participants were asked to hit the notes using a keyboard when they reached the red judgment line. The number of notes were adjusted for each condition (boredom, flow and overload). On the left is an example of boredom condition trial and on the right is an example of a normal condition trial.

The broader objective of the ongoing research was to investigate the neural correlates of interpersonal flow. Namely, we were interested in the brain areas responsible for how two or more people reach flow state when in each other’s vicinity. In order to do so, there was a need to objectively confirm that a person is “in the zone”. One possible method that has been used in studies investigating attention was to use eye measurements such as blink rate, pupil diameter and gaze direction.1–3 Eye measurements are considered objective measures as they provide estimates of intensity of mental activity and of changes in mental states and cognitive mechanisms in a spontaneous and involuntary manner.4 Past attention studies have suggested that blink rate increases in boredom condition and decreases in overload condition; pupil diameter decreases in boredom condition and increases in overload condition; and gaze direction can predict whether a person is in optimal performance (flow) or not (Table 1).

Table 1: Summary of the Literature Review. Predictions from attention research indicate some significant differences should be found between the three conditions in our three measurements.

Pupil Diameter Blink Rate Gaze Direction
Literature Review Small pupil diameter in low arousal, medium pupil diameter in mid-arousal (/flow), large pupil diameter in high arousal.2 Eyeblinks were significantly reduced during the stimulus-processing period of high attention and were transiently suppressed during the response period.3 There is a significant statistical correlation between subjects’ first fixation and their pattern of choices (System 1 or 2).1
Predictions in Flow Small pupil diameter in boredom condition, medium pupil diameter in normal/ flow condition, and  large pupil diameter in overload condition. High blink rate in boredom condition, medium blink rate in normal/ flow condition, and low blink rate in overload condition. More fixations outside of the field of vision/attention and on the visual distractors during bored condition.

The opposite during flow.

Methods

Experimental Paradigm

An open-source music game was used where participants hit the notes when they reached the judgment line as seen in Figure 2. Before the actual experiment, participants were asked to attend a ten-minute session where they chose a song that appealed to them and a note frequency that was at the correct difficulty level. We determined the proper difficulty level by asking each participant to try various levels of the game and having them choose the one song they found neither too difficult nor too easy. This was the “flow” or normal condition. A boredom condition of each participant’s chosen song was created by either decreasing the number of notes and making them repetitive in the first version of our experiment or maintaining the same number of notes but making them repetitive, and then reversing and shuffling the song in the second version. Similarly, the overload condition was created by increasing the note frequency (for both versions). Finally, a 30 second baseline period, where a fixation point and no notes appeared on screen, preceded each trial, to help in data analysis later. Furthermore, visual and auditory distractors were introduced to the periphery of the screen every few seconds for a second while the game was on. Their purpose was to determine whether or not participants were paying full attention as an indication of flow. The two experimental paradigms used are summarized in Table 2.

Table 2: Descriptions of the Paradigms for the Two Experiments.

Paradigm Number of Participants Number of Trials Boredom condition
Experiment 1 Game + Eye measurements

+ Performance

10 (and 2 excluded) 3 for each condition (Boredom, Normal, Overload) Same song as overload and normal conditions
+
Repetitive, less number of notes as other conditions
Experiment 2 Game + Eye measurements+

Performance+

Questionnaire +

EEG (on 3 participants)

6 3 for each condition (Boredom, Normal, Overload) Same song but reversed and shuffled

+

Repetitive, same number of notes as other conditions

Eye Measurements

figure 3
Figure 3: Eyelink. A computer screen showing how the Eye link detects the eye pupil in order to measure the pupil diameter as well as blink rate and gaze direction

Eye measurements were collected using the Eye link 1000 device (Figure 3). The Eye link 100 is a device that detects the eye pupil by having the participant look in the direction of a camera in the device. The device is placed at the bottom of the screen where the computer game is played by the participant.

Figure 4
Figure 4: Questionnaire. The questionnaire is used as a subjective measure of flow experience and is compared with the eyelink results.

The data collected from Eye link was normalized to the baseline on MATLAB. The device was optimal for the task as it was not attached to participants, and it collected measurements for blink rate, pupil diameter and gaze direction simultaneously. It is also relatively easily connected to Matlab and has been used in others’ experiments.

Behavioural Measurements

Likert scale questions from 1-7 were used to avoid biases. The questionnaire in Figure 4 was created for and used in other experiments on interpersonal flow in the Shimojo Lab.

Data Analysis

Code in Matlab 2017 was written to analyse each participant’s eye-measurement results and average out the results for all participants for each trial and for each condition.

Results

Our results suggest that there is no significant relationship between blink rate and flow, or pupil size and flow (Table 3). However, data analysis on a region of interest in the used computer game’s screen shows a promising, significant correlation between gaze direction and flow experience. Specifically, it shows that we can measure whether a participant is in flow, boredom or overload condition based on where their gaze is directed. These results match behavioural results from a questionnaire filled in by participants after each trial, as well as those from participants’ performances in each trial. These findings are promising for solving the problem of determining, objectively, whether experimental subjects are in flow. However, more investigation is needed in order to determine whether gaze direction actually measures flow or merely difficulty level.

Table 3: Summary of Findings for the Three Measurements. Only gaze direction yielded significant results.

Pupil Diameter Blink Rate Gaze Direction
Summary of Findings No significant difference in pupil diameter between conditions No significant difference in blink rate between conditions Significant difference in gaze direction (ROI) between conditions

Pupil Size

There was no significant difference in pupil size among different conditions outside of the first ninety seconds of gameplay (Figures 5 and 6). Additionally, compared to the baseline, the pupil size increased in the boredom condition and stayed the same in the overload condition. The results thus strongly suggest that pupil size on its own is not a measure for flow experience. Further analysis of the pupil size data could be done in order to assess whether other pupil size measurements like pupil size at a certain time or period of time are possible predictors of flow.

figure 5
Figure 5: Normalized pupil size for Experiment 1. There were no differences between the three conditions.
figure 6
Figure 6: Normalized pupil size for Experiment 2. As in Experiment 1, there were no differences between the three conditions.

Blink Rate

Results from blink rate in Experiment 1 show a significant difference between the boredom condition and the normal and overload conditions but not much significance in difference between normal and overload conditions (Figure 7). This seems promising at first as it may indicate that blink rate increases in boredom condition and decreases in overload condition as predicted from previous research in attention. However, replicating the experiment with more notes in the boredom condition shows that this difference is merely a result of having too few notes. Namely, in the second experiment, there was no significant difference in blink rate between the conditions (Figure 8). Thus, just like pupil diameter, averaged blink rate does not seem to be an indication for flow, and further data analysis could be done.

figure 7
Figure 7: Normalized blink rate for Experiment 1. The boredom condition has an average higher blink rate.
figure 8
Figure 8: Normalized blink rate for Experiment 2. Notice that there are no significant differences between the three conditions

Gaze Direction

Results from gaze direction were the most promising thus far. The highest fixations were on the centre of the fixation points in the baseline period of all trials in both experiments. In the boredom condition of Experiment 1, fixations were far up the screen due to the very low number of notes (Figure 9), while in Experiment 2, they are very similar to the fixations in the overload condition (Figure 10). However, an interesting finding can be observed in the normal/ flow condition in both experiments: a green shadow to the top of the screen (Figures 8 and 9). This region of interest (Figure 11) has been analysed by mean gaze per pixel and the results show that there are indeed significant differences between the three conditions (Figure 12). These significant findings denote that gaze direction could serve as an objective measure of flow experience as the flow condition is significantly different than the boredom and overload conditions in this particular region of interest.

figure 9
Figure 9: Heat map of gaze direction in Experiment 1. Note the green region in the Normal condition in comparison with Boredom and overload.
figure 10
Figure 10: Heat map of gaze direction in Experiment 2. Once again, note the green region (ROI in the next figure).
figure 11
Figure 11: The region of interest.
figure 12
Figure 12: Mean gaze per pixel for the three conditions: Note the significant difference among the three conditions.

Behavioural Measurements

Finally, the results from the questionnaire show a significant difference between all three conditions –boredom, normal and overload—in all questions asked (Figure 13). This is a replication of previous work in Shimojo’s lab which affirms that, subjectively, people feel the difference between each game mode.

figure 13
Figure 13: Questionnaire results. There are significant differences between the three conditions in the questionnaire questions

Future experiments can explore other behavioural measurements to objectively identify flow in an experimental setting that involves the presence of another person. Behavioural methods used in previous experiments include heart rate and heart rate variability as well as blood pressure, which were found to correlate with individual flow.5 Utilizing these methods to study interpersonal flow and combining them with neuroimaging data can help identify the underlying brain mechanisms of flow experience.

Conclusion

In conclusion, although not all eye measurements are an accurate, objective measure of flow, some of them – namely, gaze direction – are. Our experiment opens the door for more exploration of whether the eye can objectively tell us about the flow state and, more generally, mental state. In the long term, we hope that our research can allow for a better understanding and treatment of disorders like ADHD. More research could be done on possible flow signals, like blood pressure and heart rate, in tandem with neuroimaging data to determine the neural basis of flow. Furthermore, additional investigation of pupil diameter and blink rate in relation to flow is needed to establish it more securely as a viable indicator of flow.

Acknowledgments

I would like to thank my mentor and co-mentor Dr. Shinsuke Shimojo and Dr. Mohammad Shehata for giving me the opportunity to work on what I found to be a very interesting and beneficial project. I would also particularly like to thank Dr. Shehata for helping me greatly especially in the data analysis and MATLAB code. Additionally, special thanks for my lab mates Naomi, Mia, Miao, Shota and Dung-Hun and my friends Matt and Eoin for being patient participants. Finally, I would like to thank my college, Lucy Cavendish College of the University of Cambridge, for giving me a travel grant.

References

1- Innocenti, A., Rufa, A. & Semmoloni, J. Overconfident behavior in informational cascades: An eye-tracking study. Journal of Neuroscience, Psychology, and Economics 3, 74-82 (2010).

2- McGinley, M., David, S. & McCormick, D. Cortical Membrane Potential Signature of Optimal States for Sensory Signal Detection. Neuron 87, 179-192 (2015).

3- Oh, J., Jeong, S. & Jeong, J. The timing and temporal patterns of eye blinking are dynamically modulated by attention. Human Movement Science 31, 1353-1365 (2012).

4- Harmat, L., Ørsted Andersen, F., Ullén, F., Wright, J. & Sadlo, G. Flow Experience: Empirical Research and Applications.

5- de Manzano, Ö., Theorell, T., Harmat, L., & Ullén, F. The psychophysiology of flow during piano playing. Emotion, 10, 301–311 (2010).

Interview with Professor Marianne Bronner

By Alycia Lee, CURJ Associate Editor

Inline image 1

1. What motivated you to study neural crest cells?

I first became interested in developmental biology when I was a graduate student at Johns Hopkins. As an undergrad, I had taken a curriculum that was similar to the curriculum of physics students here at Caltech, and so I hadn’t had much biology. As a consequence, when I was in graduate school, I had to take some undergraduate courses, one of which was developmental biology. I was absolutely blown away by the concepts presented in the course, and decided that was what I wanted to study for my thesis research. In fact, the questions I starting examining then in developmental biology are what I’ve been working on ever since. Learning about the neural crest—the cell population that we investigate—was an epiphany. I knew that was what I wanted to investigate and I’ve been doing it ever since.

I became interested in evolutionary studies only much later in my career. For many years, I taught in a course for graduate students and postdocs at the Marine Biological Laboratory in Woods Hole and became the course director in 1997. I organized the course, invited all these incredible researchers, and for six weeks, sat through their lectures and participated in the lab. The course went over several topics, starting with the development of creepy crawly organisms that live in the sea, and moving up the evolutionary tree all the way to highest vertebrates. I’d been working on the neural crest, which is a vertebrate-specific cell population, and I started wondering, “where (evolutionarily speaking) did these cells come from and why?” That’s how I got interested in evolution. I came to Caltech in 1996, and shortly after arriving here, decided I wanted to work on the evolutionary aspects of neural crest development.

2. What are the importance and applications of neural crest cells?

The neural crest is a cell population that is implicated in many types of birth defects and cancers. These cells are prone to birth defects, because they give rise to a large portion of the facial skeleton so defects in their migration or proliferation can result in cleft palate, heart defects, and other abnormalities. We primarily focus on basic science issues to help understand the mechanisms underlying neural crest development, specifically examining how they form, migrate, and differentiate. The long-term applications of this research are far-reaching: perhaps leading to early diagnosis and recognition of therapeutic targets. By understanding the role of genes involved in neural crest development, we may understand how mutations in these genes result in some of the most common human birth defects.

Regarding cancer: Neural crest derived cells are very prone to metastasis. For example, all the pigment cells in the human body come from neural crest cells, and abnormalities in these cells can result in melanoma. We think that some of the events occurring during metastasis recapitulate the programs that were active during embryonic development. So by understanding the normal function of these programs, we may understand why cells become cancerous.

3. Where is developmental biology headed?

We live in a very exciting time. The tools that we have now are amazing—for example, our ability to perform gene sequencing of cell types and even whole organisms at the population and single-cell level. Now, one can do a gene knockout in any organism, because of CRISPR/Cas9. Because of these advances, we can answer questions that have been around for a long time, but previously were too difficult to tackle. While the questions haven’t changed, the depth with which we can understand them has steadily increased with time. My goal is to revisit classical questions, and get answers to them in a way we never could before. The fundamental questions have not changed, but the technology has advanced to make it possible to answer those questions.

4. Which of your experimental results have surprised you the most?

That’s a hard question. Everything I work on always surprises me. Usually, if I formulate a hypothesis, and then have one of my students or postdocs test it, we find that the original hypothesis was wrong but the new answer leads us in an equally exciting direction. That’s what I love about science—it doesn’t work the way you think it would but new discoveries make it ever more intriguing. When you try to figure out how things work, you formulate a hypothesis, but you can’t be wed to it. You have to try to understand and change it as you go along.

In our studies of evolution, we study a basal vertebrate called lamprey. These animals look like eels, but are jawless vertebrates (whereas eels have very good jaws!). Lampreys are the most primitive animal that has a neural crest, but they do not have all neural crest derived cell types present in humans. In a paper that we published last year, we were looking for a population of neural crest cells in lamprey that contributes to the nerves that control movement in the gut, called the “enteric nervous system.”

During embryonic development, after the formation of the central nervous system (CNS), neural crest cells are initially contained within the CNS, but then leave as single cells that migrate into the periphery. In fact, neural crest cells that arise in the hindbrain undergo the longest migrations of any embryonic cell type. They migrate from the neck region into the foregut (esophagus), then continue down the entire length of the gut and contribute to neurons that innervate the gut. These are important because they cause peristalsis, and without them, an organism would die.

We decided to look for this population of neural crest cells in lamprey. Lampreys have neural crest cells that contribute to the face, and structures in the trunk. However, when we looked for these “neck” neural crest cells, we couldn’t find them—there were no neural crest cells that invaded the gut and migrated down its length. My hypothesis was that lamprey lacks an enteric nervous system. Much to my disappointment, the postdoc working on this project found that lamprey have perfectly good enteric neurons, but we couldn’t find where they came from. Eventually, we found that they came from a different source than the neck neural crest. It was very exciting for us, because the neurons are there, but this enteric population of neural crest cells have yet to exist in primitive vertebrates, and also are only found in jawed vertebrates. It’s as if the neural crest “took over” new cell types during the course of vertebrate evolution.

5. What projects that your lab is currently working on are you most excited about?

I’m always excited about whatever we’re doing. One project is again an evolutionary project using the lamprey. In the 1980s, there was a hypothesis formulated that suggested that the invention of the neural crest is what made vertebrates so successful, because it enabled the head to elaborate and for jaws to form, making vertebrates excellent predators. This was referred to as the “New Head” of vertebrates. We have been testing whether this new head already existed in primitive vertebrates like lamprey. We’ve done a lot of genomic work and what we found is that lamprey don’t actually seem to have that new of a head. Still, they are perfectly good vertebrates. So we think that the new head might not have arisen at the base of vertebrates, but rather with jawed vertebrates. This is surprising, because this hypothesis had been around for forty years, but now we are not sure that it is correct.

The other project is more medically relevant. We’re interested in a population of neural crest cells that migrates into the heart. In the heart of lamprey and other fish, there are two chambers whereas, in birds and mammals, there are four chambers. We’ve been studying neural crest cells that migrate to the heart. By labeling these cells with fluorescent markers, one of my graduate students has found that some of these neural crest cells form part of the muscle of the heart—the myocardium—which was not known before. The reason I’m excited is that these cells have stem cell properties. Whereas the hearts of salamanders and fish can regenerate, the hearts of higher vertebrates fail to do so. So we wonder if this neural crest population may be involved in helping damaged heart tissue regenerate.

6. How is computation used in your research?

All the transcriptome analysis we do involves computation. For example, one of my graduate students is profiling cardiac neural crest cells by single-cell RNA sequencing (RNA-seq), which is a new, cool technique. We’re getting interesting and exciting results. An issue is that we get a huge amount of data from these single-cell RNA seq experiments as well as from whole population transcriptome analysis. We are often not sure what to make of this “embarrassment of riches.” Computation enables us to sort through the data, pick the wheat from the chaff, and try to understand what differences there are between cell populations and what the differences mean.

In the single-cell RNA seq, we have hundreds of single cells. We are focusing on the ones we are most interested in, like these cardiac neural crest cells. One of the questions I would like to address is if there is something special about these cells, and what genes are important for giving this population of neural crest the ability to go to the right places and contribute to the right cell types in the heart. That’s where computation will be really critical.

7. What aspects of Caltech do you treasure the most?

I love that it’s small, and filled with brilliant students and colleagues. As a consequence of the small size, I know many of the other faculty. Therefore, there are no barriers to collaborating with people. Collaborations sometimes emerge via interactions between my postdocs and other postdocs, and sometimes between me and other faculty. These interactions are ongoing and really fun, and take us in directions that we wouldn’t otherwise go. That’s wonderful—being around so many bright people here makes it a very exciting place to be.

The other thing is that at Caltech, you’re allowed to think out of the box and do things that are crazy. When I decided I wanted to work on lamprey—that was pretty crazy. I went to my division chair and said, “Can I have a room to set up some tanks?” And he said, “Sure.” He gave me a room and we set up some large fish tanks, and we got animals sent from the Great Lakes. Then, we started growing them in the lab. Now people come from all over the world to our lamprey facility to work on these beasts in the summertime. We’ve become—in a way—the lamprey center of the world. Being at Caltech enabled that. I don’t know if I could have done it at any other place.

8. What do you enjoy doing in your spare time?

I like to swim, I like to hike—I’m an exercise freak. I play with my dog, I like to cook. I go to the theater a lot. I have a subscription to the Pantages theatre, because I saw Hamilton and really enjoyed the show. I think having other things to do outside of science is really important, because it allows you to step back and think about things and reflect.

9. What advice would you give to yourself as a college student?

The advice I would give myself would be to take life less seriously and have more fun. I think I was much too studious. It’s really important to have balance in life. And you are a better scientist if you step back every once in a while and view the “big picture.” I also think it’s really important to take more writing and literature classes. As a scientist, I spend so much of my time writing. I would tell my younger self to take more courses to hone my writing skills. I didn’t spend enough time writing when I was a student. I see a lot of scientists struggle when it comes to writing a paper or grants.

10. What advice would you give to young female students aspiring for a successful career in STEM?

Have confidence in yourself. One of the worst problems women (and some men) have, and I had this in spades, is a complete lack of confidence. We all have imposter syndrome. We all think we shouldn’t be here—that we’re not smart or good enough. And that continues forever. If you can develop that mindset that you can do it and you do belong in STEM, it’s really important and takes you a long way.

Also, women worry so much about how to balance family and career to the point that it can be debilitating. I always knew I wanted to have a family. If I had to choose between having a family and having a career, I probably would have chosen having a family. But I didn’t have to choose. And I think that having children has been enormously helpful in my career. It sounds counterintuitive, but as a mother and a scientist, you become super organized. I found that once I had kids, I became very focused. In eight hours, I could get as much done as somebody who was spending fourteen.

To be a successful scientist, it takes a very complicated set of skills. We always focus on—in biology, at least—being a good bench scientist. As graduate students and postdocs, people are always at the bench. But there are other skills you need to master if you want to be successful in this career—skills that higher education does not teach you. When you transit from being in someone else’s lab to a faculty position, you are often totally unprepared for the next step. All of a sudden, you have to go from being a bench scientist to a manager. I think people skills are more important than some realize. Learning how to get along with and manage people is an incredibly important skill.

This applies to all young people: Find your passion. The most important thing is to find something that you want to do. Do what you really love. If you can find that special passion that inspires you, it can keep you going for a lifetime. In my case, I feel I get paid to do a job that I would do for free! It’s so much fun. That’s what makes life enjoyable.

 

Development of a Virtual Reality Platform for the Study of Human Behavior

Author: Natalia Brody, Emory University, Class of 2019
Project Partner: Luca Donini, University of Cambridge
Mentors: Ralph Adolphs, Ph.D. and Juri Minxha, Caltech

Abstract
In the past, neuroscientists have used a single experimental paradigm to
study the way we interact with and process the world around us: a subject is shown a stimulus while some form of brain activity (ex. EEG, fMRI, single-units) is recorded. However, the stimulus is typically something simple (i.e., an image, a sound, or a video) and does not reflect the richness of the real world. This project attempts to address this problem by creating a more engaging platform for neuroscience research via virtual reality. Using the game engine Unity, several virtual environments were designed with threat and reward tasks meant to incentivize spatial exploration. Ultimately, virtual reality is shown to be a valid platform for neuroscience research: it elicits realistic responses, gives access to abundant and rich data, and can be applied to niches difficult to study in real-life scenarios (i.e., fear).

Introduction
Conventional ways of understanding the brain largely depend on 2D stimuli such as pictures and videos. However, brain functions such as emotion and social cognition require richer stimuli and a more trackable user experience in order to be adequately studied. For example, to begin addressing how PTSD affects the brain it would be necessary to 1) elicit a realistic emotional response that is expected to be altered by PTSD and 2) closely monitor the behavioral and psychophysiological responses associated with this response. This example is applicable to more than PTSD—Autism Spectrum Disorders and common disorders such as ADHD, phobias, and anxiety also require rich stimuli and the ability to closely monitor an individual’s experience in order to be best studied. Our project proposes that virtual reality is a fitting tool for studying these types of conditions. In addition to the ability to better evaluate human behavior, data collected in virtual reality (VR) could be compiled to create diagnostic models for neurological disorders. These models could be created by comparing the individual’s experience of a certain virtual environment with the experience of someone suspected to have a disorder, or comparing the experience of someone known to have a disorder with someone suspected to have it.

The main goal of this project was to create a more engaging and immersive platform for stimuli via virtual reality that could be used as an alternative to the traditional approach. To achieve this goal, a library of virtual environments and 360 ̊videos were created and presented to subjects in order to examine aspects of VR people respond to and to investigate whether certain stimuli (such as visual or auditory cues) affect decision making in this context. The project was conducted in two parts. The first part of the project was a pilot program in which numerous draft virtual environments were built and used to evaluate various aspects of VR. The second part of the project, “Experimental Study” consisted of a finalized game environment that was presented to subjects in a formal experimental settings.

Methods
Preliminary Game Design
Unity game engine is a platform used to develop video games for PC, consoles, mobile devices, and websites. Over the course of this project, Unity was used to build a library of virtual environments that could then be displayed in an Oculus Rift virtual reality headset. To begin, several drafts of potential environments for use in our study were
built. Audio, object, and animation packages were purchased from the Unity asset store (an online store through which users can package and distribute their Unity creations to other users) to include in the scenes. Because there are so many possibilities and factors of interest available for manipulation in VR, multiple draft environments were created. Across the various virtual environments, a theme of fear was maintained that would serve as a metric for measuring whether VR was realistic enough and its ability to elicit a strong emotional response (in this case, fear) from a subject. For example, a subject could encounter a frightening monster and, if they had an adverse reaction to this encounter, it could help demonstrate VR as an engaging form of stimuli that evokes reactions parallel to those that would occur in the real world.

The environments created for our library all have similar structure as exploratory spaces that contain both threats and rewards. Threats vary from wolves to zombies and other monsters, while the reward, collectible gold coins, remains the same across environments. This threat and reward system was included in order to incentivize spatial exploration (i.e. wanting to wander the space in order to obtain rewards) and to demonstrate that although virtual, components of virtual reality can influence decision making and the way one explores his environment. Additionally, 360 ̊ videos were included in the project library. These videos were sent to the Oculus Rift through Vrideo, an application that distributes immersive videos for use in virtual reality. The purpose of these videos was to give a broader understanding of how subjects interact with and respond to certain aspects of VR.

Pilot Program
After the project library was completed, a pilot program was created so that members of my lab could explore the environments and videos and give feedback. To collect feedback on the library environments, questionnaires were created using Qualtrics. Qualtrics is a tool that creates surveys for research purposes and offers a wide range of features for creating survey questions and analyzing response data. There were a total of three Qualtrics questionnaires: one to review an environment, one to review a video, and one to make comparisons about all of the environments and videos explored. Through the feedback collected, which features of VR are most and least effective were identified. This information was then used to refine current virtual environments and, in the future, can be used as a guide for research motivated VR game design.

Secondary Game Design
After the pilot program, a more complex virtual environment was created for the second phase of the project, a formal experimental study. Thus, a partnership with White Door Games game developer was started and permission to use their VR video game, Dreadhalls, for the project was obtained. The terms of this partnership allowed for access to and modification to the game’s source-code in order to meet the project’s needs.
Modifications included adding a timer, a coin counter, a map with customized visibility, and variability to maze structure. Unlike the free-roaming and spacious environments built for the first pilot program, Dreadhalls restricted user mobility. The game had a maze-like structure that forced the subject to travel down narrow hallways and make frequent decisions as they traverse from one side to the other. This format allowed for the investigation of decision-making in the presence of threat OR of threats in the envi-
ronment, and to understand how certain conditioned stimuli (such as a seemingly innocuous sound which becomes predictive of threat) can affect how subjects explore the maze. Overall, this data driven approach aimed to extrapolate trends regarding strategy and decision making in order to further verify that VR engages the user and provides insightful data on behavior.

Experimental Study
A second trial was created, through which Dreadhalls would again be used to further evaluate the suitability of VR as a platform for neuroscience research. Like the pilot program, participants would experience the environment(s) being tested and then provide feedback on a Qualtrics questionnaire. For this pilot program, a new set of questionnaires specific to Dreadhalls were provided. Subjects were recruited by sending emails to the Caltech “houses” (dormitories).

Upon arrival, subjects were administered game instructions and game controller instructions. Each subject was given 45 minutes to attempt the game as many times as they wanted. Each participant received a minimum of $10 for participation, but could increase their award based on how many coins they collected (COINS bonus) and
whether they reached the maze exit within the game (EXIT bonus). After the participant finished, they completed the questionnaire and received their monetary reward.