Eye Measurements as Objective Measures of Flow Experience

Author: Salma Elnagar
Mentors: Shinsuke Shimojo and Mohammad Shehata
Editor: Jagath Vytheeswaran

Abstract

Flow experience is achieved when a person is said to be “in the zone” as they achieve a fit between skill-and-challenge level in a certain activity. Many of these activities, such as video games, music, or athletic competitions, involve the participation or company of other people. Thus, interpersonal communication could be related to reaching the mental state of flow. Previous research has linked brain areas like the inferior frontal gyrus to individual flow. However, no experiments have yet been conducted on interpersonal flow. The large-scale purpose of this project is to fill this gap by identifying the neural correlations of interpersonal flow. In order to do so, objective measurements of the inherently subjective flow experience must first be identified. In this study, we evaluated eye measurements such as pupil diameter, blink rate and eye fixations as objective measures for individual flow. Two versions of our experiment using a musical notes game were conducted on ten and six subjects respectively with eye measures being collected using Eyelink 1000. While there was a significant difference in gaze direction among various gaming levels, there appeared to be no significant change in blink rates or pupil size. This suggests that gaze direction could be an objective measurement of flow experience.

Introduction

figure 1

Figure 1: Flow as a function of skill and challenge levels. Flow is a balance between overload and boredom.

Consider a gamer, oblivious to the outside world as they click and play for hours, thinking it has only been a few minutes since they started – or a soloist, absorbed by their music while performing Paganini’s complex caprices on stage, forgetting about the hundreds of people who gathered around to watch them play. These scenarios, along with many more everyday ones such as enjoying a gripping book or fully concentrating on an interesting math problem, expose one of the many marvels of the human brain: its ability to get in “the zone”. In such state, a person is fully and satisfactorily engaged in an activity to the extent that their perceptions of themselves, others, and time are dampened. In neuroscience terms, we dub this mental state “flow.” In a more concrete sense, flow is the skill-challenge balance between overload, in which a person perceives high challenge and low skill, and boredom, in which a person perceives high skill but low challenge. Learning more about flow is the first step towards creating interventions for clinical disorders of attention, like Attention Deficit Hyperactivity Disorder (ADHD).

To create and scientifically study this very subjective experience is not an easy mission. In order to create flow experience in the laboratory, we used an open-source music game where participants were asked to hit notes when they appeared on screen (Figure 2). Three conditions were created based on the difficulty of the game (defined by note frequency) according to each participant: boredom, flow, and overload conditions. Various eye measurements were then gathered and compared to answers in post-experimental surveys to determine the validity of said measurements as objective indicators of flow.

figure 2

Figure 2. Screenshots of the open-source music game. Participants were asked to hit the notes using a keyboard when they reached the red judgment line. The number of notes were adjusted for each condition (boredom, flow and overload). On the left is an example of boredom condition trial and on the right is an example of a normal condition trial.

The broader objective of the ongoing research was to investigate the neural correlates of interpersonal flow. Namely, we were interested in the brain areas responsible for how two or more people reach flow state when in each other’s vicinity. In order to do so, there was a need to objectively confirm that a person is “in the zone”. One possible method that has been used in studies investigating attention was to use eye measurements such as blink rate, pupil diameter and gaze direction.1–3 Eye measurements are considered objective measures as they provide estimates of intensity of mental activity and of changes in mental states and cognitive mechanisms in a spontaneous and involuntary manner.4 Past attention studies have suggested that blink rate increases in boredom condition and decreases in overload condition; pupil diameter decreases in boredom condition and increases in overload condition; and gaze direction can predict whether a person is in optimal performance (flow) or not (Table 1).

Table 1: Summary of the Literature Review. Predictions from attention research indicate some significant differences should be found between the three conditions in our three measurements.

Pupil Diameter Blink Rate Gaze Direction
Literature Review Small pupil diameter in low arousal, medium pupil diameter in mid-arousal (/flow), large pupil diameter in high arousal.2 Eyeblinks were significantly reduced during the stimulus-processing period of high attention and were transiently suppressed during the response period.3 There is a significant statistical correlation between subjects’ first fixation and their pattern of choices (System 1 or 2).1
Predictions in Flow Small pupil diameter in boredom condition, medium pupil diameter in normal/ flow condition, and  large pupil diameter in overload condition. High blink rate in boredom condition, medium blink rate in normal/ flow condition, and low blink rate in overload condition. More fixations outside of the field of vision/attention and on the visual distractors during bored condition.

The opposite during flow.

Methods

Experimental Paradigm

An open-source music game was used where participants hit the notes when they reached the judgment line as seen in Figure 2. Before the actual experiment, participants were asked to attend a ten-minute session where they chose a song that appealed to them and a note frequency that was at the correct difficulty level. We determined the proper difficulty level by asking each participant to try various levels of the game and having them choose the one song they found neither too difficult nor too easy. This was the “flow” or normal condition. A boredom condition of each participant’s chosen song was created by either decreasing the number of notes and making them repetitive in the first version of our experiment or maintaining the same number of notes but making them repetitive, and then reversing and shuffling the song in the second version. Similarly, the overload condition was created by increasing the note frequency (for both versions). Finally, a 30 second baseline period, where a fixation point and no notes appeared on screen, preceded each trial, to help in data analysis later. Furthermore, visual and auditory distractors were introduced to the periphery of the screen every few seconds for a second while the game was on. Their purpose was to determine whether or not participants were paying full attention as an indication of flow. The two experimental paradigms used are summarized in Table 2.

Table 2: Descriptions of the Paradigms for the Two Experiments.

Paradigm Number of Participants Number of Trials Boredom condition
Experiment 1 Game + Eye measurements

+ Performance

10 (and 2 excluded) 3 for each condition (Boredom, Normal, Overload) Same song as overload and normal conditions
+
Repetitive, less number of notes as other conditions
Experiment 2 Game + Eye measurements+

Performance+

Questionnaire +

EEG (on 3 participants)

6 3 for each condition (Boredom, Normal, Overload) Same song but reversed and shuffled

+

Repetitive, same number of notes as other conditions

Eye Measurements

figure 3

Figure 3: Eyelink. A computer screen showing how the Eye link detects the eye pupil in order to measure the pupil diameter as well as blink rate and gaze direction

Eye measurements were collected using the Eye link 1000 device (Figure 3). The Eye link 100 is a device that detects the eye pupil by having the participant look in the direction of a camera in the device. The device is placed at the bottom of the screen where the computer game is played by the participant.

Figure 4

Figure 4: Questionnaire. The questionnaire is used as a subjective measure of flow experience and is compared with the eyelink results.

The data collected from Eye link was normalized to the baseline on MATLAB. The device was optimal for the task as it was not attached to participants, and it collected measurements for blink rate, pupil diameter and gaze direction simultaneously. It is also relatively easily connected to Matlab and has been used in others’ experiments.

Behavioural Measurements

Likert scale questions from 1-7 were used to avoid biases. The questionnaire in Figure 4 was created for and used in other experiments on interpersonal flow in the Shimojo Lab.

Data Analysis

Code in Matlab 2017 was written to analyse each participant’s eye-measurement results and average out the results for all participants for each trial and for each condition.

Results

Our results suggest that there is no significant relationship between blink rate and flow, or pupil size and flow (Table 3). However, data analysis on a region of interest in the used computer game’s screen shows a promising, significant correlation between gaze direction and flow experience. Specifically, it shows that we can measure whether a participant is in flow, boredom or overload condition based on where their gaze is directed. These results match behavioural results from a questionnaire filled in by participants after each trial, as well as those from participants’ performances in each trial. These findings are promising for solving the problem of determining, objectively, whether experimental subjects are in flow. However, more investigation is needed in order to determine whether gaze direction actually measures flow or merely difficulty level.

Table 3: Summary of Findings for the Three Measurements. Only gaze direction yielded significant results.

Pupil Diameter Blink Rate Gaze Direction
Summary of Findings No significant difference in pupil diameter between conditions No significant difference in blink rate between conditions Significant difference in gaze direction (ROI) between conditions

Pupil Size

There was no significant difference in pupil size among different conditions outside of the first ninety seconds of gameplay (Figures 5 and 6). Additionally, compared to the baseline, the pupil size increased in the boredom condition and stayed the same in the overload condition. The results thus strongly suggest that pupil size on its own is not a measure for flow experience. Further analysis of the pupil size data could be done in order to assess whether other pupil size measurements like pupil size at a certain time or period of time are possible predictors of flow.

figure 5

Figure 5: Normalized pupil size for Experiment 1. There were no differences between the three conditions.

figure 6

Figure 6: Normalized pupil size for Experiment 2. As in Experiment 1, there were no differences between the three conditions.

Blink Rate

Results from blink rate in Experiment 1 show a significant difference between the boredom condition and the normal and overload conditions but not much significance in difference between normal and overload conditions (Figure 7). This seems promising at first as it may indicate that blink rate increases in boredom condition and decreases in overload condition as predicted from previous research in attention. However, replicating the experiment with more notes in the boredom condition shows that this difference is merely a result of having too few notes. Namely, in the second experiment, there was no significant difference in blink rate between the conditions (Figure 8). Thus, just like pupil diameter, averaged blink rate does not seem to be an indication for flow, and further data analysis could be done.

figure 7

Figure 7: Normalized blink rate for Experiment 1. The boredom condition has an average higher blink rate.

figure 8

Figure 8: Normalized blink rate for Experiment 2. Notice that there are no significant differences between the three conditions

Gaze Direction

Results from gaze direction were the most promising thus far. The highest fixations were on the centre of the fixation points in the baseline period of all trials in both experiments. In the boredom condition of Experiment 1, fixations were far up the screen due to the very low number of notes (Figure 9), while in Experiment 2, they are very similar to the fixations in the overload condition (Figure 10). However, an interesting finding can be observed in the normal/ flow condition in both experiments: a green shadow to the top of the screen (Figures 8 and 9). This region of interest (Figure 11) has been analysed by mean gaze per pixel and the results show that there are indeed significant differences between the three conditions (Figure 12). These significant findings denote that gaze direction could serve as an objective measure of flow experience as the flow condition is significantly different than the boredom and overload conditions in this particular region of interest.

figure 9

Figure 9: Heat map of gaze direction in Experiment 1. Note the green region in the Normal condition in comparison with Boredom and overload.

figure 10

Figure 10: Heat map of gaze direction in Experiment 2. Once again, note the green region (ROI in the next figure).

figure 11

Figure 11: The region of interest.

figure 12

Figure 12: Mean gaze per pixel for the three conditions: Note the significant difference among the three conditions.

Behavioural Measurements

Finally, the results from the questionnaire show a significant difference between all three conditions –boredom, normal and overload—in all questions asked (Figure 13). This is a replication of previous work in Shimojo’s lab which affirms that, subjectively, people feel the difference between each game mode.

figure 13

Figure 13: Questionnaire results. There are significant differences between the three conditions in the questionnaire questions

Future experiments can explore other behavioural measurements to objectively identify flow in an experimental setting that involves the presence of another person. Behavioural methods used in previous experiments include heart rate and heart rate variability as well as blood pressure, which were found to correlate with individual flow.5 Utilizing these methods to study interpersonal flow and combining them with neuroimaging data can help identify the underlying brain mechanisms of flow experience.

Conclusion

In conclusion, although not all eye measurements are an accurate, objective measure of flow, some of them – namely, gaze direction – are. Our experiment opens the door for more exploration of whether the eye can objectively tell us about the flow state and, more generally, mental state. In the long term, we hope that our research can allow for a better understanding and treatment of disorders like ADHD. More research could be done on possible flow signals, like blood pressure and heart rate, in tandem with neuroimaging data to determine the neural basis of flow. Furthermore, additional investigation of pupil diameter and blink rate in relation to flow is needed to establish it more securely as a viable indicator of flow.

Acknowledgments

I would like to thank my mentor and co-mentor Dr. Shinsuke Shimojo and Dr. Mohammad Shehata for giving me the opportunity to work on what I found to be a very interesting and beneficial project. I would also particularly like to thank Dr. Shehata for helping me greatly especially in the data analysis and MATLAB code. Additionally, special thanks for my lab mates Naomi, Mia, Miao, Shota and Dung-Hun and my friends Matt and Eoin for being patient participants. Finally, I would like to thank my college, Lucy Cavendish College of the University of Cambridge, for giving me a travel grant.

References

1- Innocenti, A., Rufa, A. & Semmoloni, J. Overconfident behavior in informational cascades: An eye-tracking study. Journal of Neuroscience, Psychology, and Economics 3, 74-82 (2010).

2- McGinley, M., David, S. & McCormick, D. Cortical Membrane Potential Signature of Optimal States for Sensory Signal Detection. Neuron 87, 179-192 (2015).

3- Oh, J., Jeong, S. & Jeong, J. The timing and temporal patterns of eye blinking are dynamically modulated by attention. Human Movement Science 31, 1353-1365 (2012).

4- Harmat, L., Ørsted Andersen, F., Ullén, F., Wright, J. & Sadlo, G. Flow Experience: Empirical Research and Applications.

5- de Manzano, Ö., Theorell, T., Harmat, L., & Ullén, F. The psychophysiology of flow during piano playing. Emotion, 10, 301–311 (2010).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s