Eye Movement Strategies in Facial Expression Recognition

Menshikova, G.Ya. Lomonosov Moscow State University

Klumova, S.B. Lomonosov Moscow State University

References

Abbruzzese, L., Magnani, N., Robertson, I.H., Mancuso, M. (2019). Age and gender differences in emotion recognition. Frontiers in Psychology, 10, 2371. URL: https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2019.02371/full (accessed: 03.12.2025).

Ananyeva, K.I., Barabanshchikov, V.A., Kharitonov, V.N. (2010). Isostatic eye movement patterns when perceiving a human face. In: V.A. Barabanschikov, (ed.). Experimental Psychology in Russia: Traditions and Prospects. (pp. 195–200). Moscow: Institute of Psychology of the RAE Publ. (In Russ.)

Antoniades, C.A., Spering, M. (2024). Eye movements in Parkinson’s disease: from neurophysiological mechanisms to diagnostic tools. Trends in Neurosciences, 47(1), 71–83. https://doi.org/10.1016/j.tins.2023.11.001

Arizpe, J., Kravitz, D.J., Yovel, G., Baker, C.I. (2012). Start position strongly influences fixation patterns during face processing: difficulties with eye movements as a measure of information use. PLOS One, 7(2), e31106. URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0031106 (accessed: 03.11.2025).

Barabanshchikov, V.A. (2012). Facial expressions and their perception. Moscow: Institute of Psychology of the RAS Publ. (In Russ.)

Barabanschikov, V.A., Ananyeva, K.I., Kharitonov, V.N. (2009). The organization of eye movements in the perception of facial images. Eksperimental’naya psihologiya = Experimental Psychology, 2(2), 31–60. (In Russ.)

Barabanschikov, V.A., Besprozvannaya, I.I. (2019). Tracks of personality on the schematic image of the face. Eksperimental’naya psihologiya = Experimental Psychology, 12(2), 16–34. (In Russ.). https://doi.org/10.17759/exppsy.2019120202

Barabanschikov, V.A., Zhegallo, A.V. (2013). Recognition of facial expressions in the proximal periphery of the visual field. Eksperimental’naya psihologiya = Experimental Psychology, 6(2), 58–83. (In Russ.)

Barabanschikov, V.A., Zhegallo, A.V. (2018). Okulomotor activity at the perception of dynamic and static expressions of the face. Eksperimental’naya psihologiya = Experimental Psychology, 11(1), 5–34. (In Russ.). https://doi.org/10.17759/exppsy.2018110101

Basiul, I.A., Demidov, A.A., Diveev, D.A. (2018). Isostatic oculomotor patterns in trust perception by facial expressions. Eksperimental’naya psihologiya = Experimental Psychology, 11(4), 70–78. (In Russ.). https://doi.org/10.17759/exppsy.2018110406

Belopolsky, V.I. (2007). Human gaze: its nature and functions. Vestnik Gosudarstvennogo universiteta prosveshcheniya. Seriya: Psihologicheskie nauki = Bulletin of the State University of Education. Series: Psychology, (4), 13–20. (In Russ.)

Dickie, E.W., Armony, J.L. (2008). Amygdala responses to unattended fearful faces: interaction between sex and trait anxiety. Psychiatry Research: Neuroimaging, 162(1), 51–57. https://doi.org/10.1016/j.pscychresns.2007.08.002

Durán, J.I., Fernández-Dols, J.-M. (2021). Do emotions result in their predicted facial expressions? A meta-analysis of studies on the co-occurrence of expression and emotion. Emotion, 21(7), 1550–1569. https://doi.org/10.1037/emo0001015

Ekman, P., Friesen, W.V., Simons, R.C. (1985). Is the startle reaction an emotion? Journal of Personality and Social Psychology, 49(5), 1416–1426. https://doi.org/10.1037/0022-3514.49.5.1416

Ekundayo, O.S., Viriri, S. (2021). Facial expression recognition: A review of trends and techniques. IEEE Access, 9, 136944–136973. https://doi.org/10.1109/ACCESS.2021.3113464

Foulsham, T., Kingstone, A. (2013). Fixation-dependent memory for natural scenes: An experimental test of scanpath theory. Journal of Experimental Psychology: General, 142(1), 41–56. https://doi.org/10.1037/a0028227

Freedman, E.G., Foxe, J.J. (2018). Eye movements, sensorimotor adaptation and cerebellar-dependent learning in autism: toward potential biomarkers and subphenotypes. European Journal of Neuroscience, 47(6), 549–555. https://doi.org/10.1111/ejn.13625

Ge, H., Zhu, Z., Dai, Y., Wang, B., Wu, X. (2022). Facial expression recognition based on deep learning. Computer Methods and Programs in Biomedicine, 215, 106621. URL: https://www.sciencedirect.com/science/article/abs/pii/S0169260722000062?via%3Dihub (accessed: 03.11.2025).

Geraets, C.N.W., Klein Tuente, S., Lestestuiver, B.P., van Beilen, M., Nijman, S.A., Marsman, J.B.C., Veling, W. (2021). Virtual reality facial emotion recognition in social environments: An eye-tracking study. Internet Interventions, 25, 100432. URL: https://www.sciencedirect.com/science/article/pii/S2214782921000725?via%3Dihub (accessed: 03.11.2025).

Hall, J.K., Hutton, S.B., Morgan, M.J. (2010). Sex differences in scanning faces: Does attention to the eyes explain female superiority in facial expression recognition? Cognition and Emotion, 24(4), 629–637. https://doi.org/10.1080/02699930902906882

Henderson, J.M., Williams, C.C., Falk, R.J. (2005). Eye movements are functional during face learning. Memory and Cognition, 33(1), 98–106. https://doi.org/10.3758/BF03195300

Hoppe, S., Loetscher, T., Morey, S.A., Bulling, A. (2018). Eye movements during everyday behavior predict personality traits. Frontiers in Human Neuroscience, 12, 1–8. https://doi.org/10.3389/fnhum.2018.00105

Hsiao, J.H., Cottrell, G. (2008). Two fixations suffice in face recognition. Psychological Science, 19(10), 998–1006. https://doi.org/10.1111/j.1467-9280.2008.02191.x

Hsiao, J.H., Lan, H., Zheng, Y., Chan, A.B. (2021). Eye movement analysis with hidden Markov models (EMHMM) with co-clustering. Behavior Research Methods, 53(6), 2473–2486. https://doi.org/10.3758/s13428-021-01541-5

Hunter, L., Roland, L., Ferozpuri, A. (2020). Emotional expression processing and depressive symptomatology: Eye-tracking reveals differential importance of lower and middle facial areas of interest. Depression Research and Treatment, (1), 1049851. URL: https://onlinelibrary.wiley.com/doi/10.1155/2020/1049851 (accessed: 03.11.2025).

Kleinke, C.L. (1986). Gaze and eye contact: A research review. Psychological Bulletin, 100(1), 78–100. https://doi.org/10.1037/0033-2909.100.1.78

Lim, J.Z., Mountstephens, J., Teo, J. (2020). Emotion recognition using eye-tracking: taxonomy, review and current challenges. Sensors, 20(8), 2384. URL: https://www.mdpi.com/1424-8220/20/8/2384 (accessed: 03.11.2025).

Liu, M., Zhan, J., Wang, L. (2024). Specified functions of the first two fixations in face recognition: Sampling the general-to-specific facial information. iScience, 27(9), 110686. https://doi.org/10.1016/j.isci.2024.110686

Luniakova, E.G., Gani-zada, J. (2019). Eye movement strategies in facial expression recognition are not related to the strength of inversion and thatcherization effects. The Russian Journal of Cognitive Science, 6(3), 6–13.

Menshikova, G.Y., Pichugina, A.O. (2020). Holistic face processing: the eye tracking data. Eksperimental’naya psihologiya = Experimental Psychology, 13(4), 72–87. (In Russ.). https://doi.org/10.17759/exppsy.2020130405

Noton, D., Stark, L. (1971). Scanpaths in eye movements during pattern perception. Science, 171(3968), 308–311. https://doi.org/10.1126/science.171.3968.308

Olszanowski, M., Pochwatko, G., Kuklinski, K., Scibor-Rylski, M., Lewinski, P., Ohme, R.K. (2015). Warsaw Set of Emotional Facial Expression Pictures: A validation study of facial display photographs. Frontiers in Psychology, 5, 1516. URL: https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2014.01516/full (accessed: 03.11.2025).

Papinutto, M., Lao, J., Ramon, M., Caldara, R., Miellet, S. (2017). The facespan — the perceptual span for face recognition. Journal of Vision, 17(5), 16. URL: https://psycnet.apa.org/record/2017-24418-001 (accessed: 03.12.2025).

Peterson, M.F., Eckstein, M.P. (2013). Individual differences in eye movements during face identification reflect observer-specific optimal points of fixation. Psychological Science, 24(7), 1216–1225. https://doi.org/10.1177/0956797612471684

Rafiee, Y., Schacht, A. (2023). Sex differences in emotion recognition: Investigating the moderating effects of stimulus features. Cognition and Emotion, 37(5), 863–873. https://doi.org/10.1080/02699931.2023.2222579

Rawal, N., Stock-Homburg, R.M. (2022). Facial emotion expressions in human–robot interaction: A survey. International Journal of Social Robotics, 14(7), 1583–1604. https://doi.org/10.1007/s12369-022-00867-0

Roslan, N.S., Izhar, L.I., Faye, I., Amin, H.U., Mohamad Saad, M.N. et al. (2019). Neural correlates of eye contact in face-to-face verbal interaction: An EEG-based study of the extraversion personality trait. PLOS One, 14(7), e0219839. URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0219839 (accessed: 03.12.2025).

Rubo, M., Huestegge, L., Gamer, M. (2020). Social anxiety modulates visual exploration in real life – but not in the laboratory. British Journal of Psychology, 111(2), 233–245. https://doi.org/10.1111/bjop.12396

Schurgin, M.W., Nelson, J., Iida, S., Ohira, H., Chiao, J.Y., Franconeri, S.L. (2014). Eye movements during emotion recognition in faces. Journal of Vision, 14(13), 14. URL: https://jov.arvojournals.org/article.aspx?articleid=2213025 (accessed: 03.11.2025).

Shin, W.-G., Park, H., Kim, S.-P., Sul, S. (2023). Individual differences in gaze-cuing effect are associated with facial emotion recognition and social conformity. Frontiers in Psychology, 14, 1219488. URL: https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2023.1219488/full (accessed: 03.11.2025).

Thompson, A.E., Voyer, D. (2014). Sex differences in the ability to recognise non-verbal displays of emotion: A meta-analysis. Cognition and Emotion, 28(7), 1164–1195. https://doi.org/10.1080/02699931.2013.875889

Velichkovsky, B.M., Joos, M., Helmert, J.R., Pannasch, S. (2005). Two visual systems and their eye movements: Evidence from static and dynamic scene perception. In: B.G. Bara, L. Barsalou, M. Bucciarelli, (eds.). Proceedings of the XXVII annual conference of the Cognitive Science Society (July 21–23, 2005). (pp. 2283–2288). Mahwah: Lawrence Erlbaum Publ.

Walker-Smith, G.J., Gale, A.G., Findlay, J.M. (2013). Eye movement strategies involved in face perception. Perception, 42(11), 1120–1133. https://doi.org/10.1068/p060313n

White, D., Burton, A.M. (2022). Individual differences and the multidimensional nature of face perception. Nature Reviews Psychology, 1(5), 287–300. https://doi.org/10.1038/s44159-022-00041-3

Yarbus, A.L. (1965). The role of eye movements in the visual process. Moscow: Nauka Publ. (In Russ.)

Zhegallo, A.V., Basiul, I.A. (2023). Predictors of nonverbal recognition of emotional facial expressions. Eksperimental’naya psihologiya = Experimental Psychology, 16(3), 53–68. (In Russ.). https://doi.org/10.17759/exppsy.2023160304

Zhong, N., Hsiao, J., Zhou, G., Hayward, W. (2023). Association of idiosyncratic eye-movement patterns with holistic processing of faces as measured by the composite face effect and the face inversion effect. Visual Cognition, 31(7), 485–500. https://doi.org/10.1080/13506285.2024.2315783

Zinchenko, Yu.P., Pervichko, E.I. (2016). Clinical and psychological study of emotion regulation: a cultural-activity approach. Voprosy Psikhologii, (3), 41–57. (In Russ.)