Background. Recently, scientific and technological progress allows the widespread use of high-tech electronic means to create security systems. The advantages of identifying people who are high on drugs or alcohol with video surveillance systems on pupillograms are indisputable. However, those who bear aggressive intentions stay in the shade. The standard method of identifying emotions aimed at recording facial expressions is sufficient enough, but it is difficult to recognize negative intentions in a person if they keep control of themselves. To solve this problem, we propose to switch from passive safety systems to active ones. Therefore, studies of the pupillary response to the stimuli presented are relevant today.
The Objective of the research is to identify patterns of pupillograms that can be used to control pupillary reactions to the stimuli significant for an individual. Simultaneously, the following tasks were solved: checking the possibility of interpreting the pupillogram by synchronizing them with the tracks of the attention focus and searching for the sites of the pupillograms allegedly resulting from emotions in response to the presented stimuli.
Design. At the first stage, the images used as stimuli presented to the subjects of the research were selected. Incentives were thematic in nature and contributed to identifying the unstable psychophysical state of a person or their susceptibility to aggression. At the second stage, the calibration of the optoelectronic system used to record the pupillograms and oculograms, as well as stabilizing factors that affect the size of the pupils, was carried out. Pupilograms were obtained using groups of two age categories (16–25 years old and 45–50 years old) of 10 and 5 subjects accordingly (both males and females). The subjects selected for the research did not have any eye diseases; their eye sight was normal or adjusted.
Results.The interdependence of the size of the pupils and the displacement of the center of attention were identified. The verification of the pupillogram rank correlation was obtained when different subjects viewed identical sequences of visual stimuli showed that in general the p significance level did not exceed the critical value alpha = 0.05. The reliability of the correlation confirms the pupillograms depend on the shape of the objects viewed and the patterns that unite the pupillograms. The microsaccades in pupillograms are well explained by moving and focusing the gaze on the details of the image, which makes it possible to interpret them as waves of attention. Synchronizing the pupillograms and oculograms allows distinguishing areas that are presumably explained by the emotional reaction of the individual to a weak external stimulus. The Fourier analysis of the pupillograms revealed a change in the observed frequency spectrum, depending on the presence or absence of an emotional reaction, the speed of the shift in the focus of attention.
Findings.The observed set of frequencies suggests a connection between the diameters of the eye pupils and the brain potentials. The practical significance of the results is to expand the possibilities of using biometric security systems, including prevention of suicide in adolescents.
PDF: Download
Keywords: attention; focus of attention; brain evoked potential; security systems; pupillogram; microcascades; oculogram;
Available Online 30.01.2019
Fig. 2. Pupil response to simple stimuli (a-d); Typical reaction to a specific (Boronenko et al., 2019) test object (e), bottom-up: pupillogram, center of view X coordinate, center of attention center Y, illumination factor k.
Fig. 2. Pupil response to simple stimuli (a-d); Typical reaction to a specific (Boronenko et al., 2019) test object (e), bottom-up: pupillogram, center of view X coordinate, center of attention center Y, illumination factor k.
Table 1. Rank Correlation
VAR |
Rho |
t |
p |
Тау |
Инверс |
Z |
p |
Гамма |
R Пирсона |
1S \ 2S |
-,0.1561 |
-,3.9165 |
9.9938E-5 |
-,0.1074 |
-40,686. |
,3.9874 |
6.6789E-5 |
-,0.1074 |
-,0.2603 |
1S \ 3S |
-,0.0949 |
-,2.3629 |
,0.0184 |
-,0.0646 |
-24,468. |
,2.398 |
,0.0165 |
-,0.0646 |
,0.0853 |
1S \ 4S |
-,0.1355 |
-,3.3896 |
,0.0007 |
-,0.0896 |
-33,946. |
,3.327 |
,0.0009 |
-,0.0896 |
-,0.1303 |
1S \ 4S |
,0.6725 |
,22.5143 |
,0. |
,0.4975 |
188,448. |
,18.4701 |
,0. |
,0.4976 |
,0.6693 |
2S \ 3S |
-,0.3651 |
-,10.2789 |
,0. |
-,0.2385 |
-113,026. |
,9.3648 |
,0. |
-,0.2385 |
-,0.2772 |
2S \ 4S |
,0.0194 |
,0.5284 |
,0.5974 |
,0.0154 |
8,586. |
,0.6302 |
,0.5285 |
,0.0154 |
,0.1319 |
2S \ 4S |
-,0.1212 |
-,3.3564 |
,0.0008 |
-,0.085 |
-48,748. |
,3.5007 |
,0.0005 |
-,0.085 |
-,0.2976 |
3S \ 4S |
,0.4626 |
,13.6761 |
,0. |
,0.3058 |
144,926. |
,12.0082 |
,0. |
,0.3058 |
,0.4568 |
3S \ 4S |
-,0.1325 |
-,3.505 |
,0.0005 |
-,0.0953 |
-45,178. |
,3.7434 |
,0.0002 |
-,0.0953 |
-,0.2054 |
4S \ 4S |
-,0.3154 |
-,9.0705 |
,0. |
-,0.2014 |
-112,198. |
,8.236 |
,0. |
-,0.2014 |
-,0.4566 |
Table 2. Descriptive statistics
|
N total |
Mean |
Standard Deviation |
Sum |
Minimum |
Median |
Maximum |
1S |
364 |
0.99875 |
0.07096 |
363.54354 |
0.86444 |
0.992 |
1.33609 |
2S |
758 |
0.98118 |
0.11338 |
743.73589 |
0.61018 |
1 |
1.26994 |
3S |
689 |
0.99248 |
0.12762 |
683.82037 |
0.63156 |
1 |
1.43725 |
4S |
747 |
0.98347 |
0.14308 |
734.65451 |
0.64147 |
1 |
1.30845 |
Table 3. Value parameters of the mathematical model
Gaussian |
|
Value |
Standard Error |
t-Value |
Prob>|t| |
Dependency |
Peak1 |
y0 |
0 |
0 |
-- |
-- |
0 |
Peak1 |
xc |
0.50834 |
0.0074 |
68.6652 |
1.61E-71 |
0.6891 |
Peak1 |
A |
0.00535 |
0.00336 |
1.59155 |
0.11553 |
0.98495 |
Peak1 |
w |
0.08917 |
0.02257 |
3.95112 |
1.70E-04 |
0.87623 |
Peak2 |
y0 |
0 |
0 |
-- |
-- |
0 |
Peak2 |
xc |
0.61801 |
0.03141 |
19.6738 |
5.65E-32 |
0.95839 |
Peak2 |
A |
0.00985 |
0.00485 |
2.03177 |
0.04558 |
0.98541 |
Peak2 |
w |
0.17945 |
0.06334 |
2.83308 |
0.00587 |
0.96216 |
Peak3 |
y0 |
0 |
0 |
-- |
-- |
0 |
Peak3 |
xc |
0.98483 |
0.01371 |
71.8127 |
5.14E-73 |
0.78469 |
Peak3 |
A |
0.05751 |
0.00313 |
18.3937 |
4.29E-30 |
0.88686 |
Peak3 |
w |
0.57918 |
0.03467 |
16.7059 |
1.74E-27 |
0.87518 |
Peak4 |
y0 |
0 |
0 |
-- |
-- |
0 |
Peak4 |
xc |
1.71418 |
0.05683 |
30.1653 |
9.31E-45 |
0.91466 |
Peak4 |
A |
-0.0074 |
0.00352 |
-2.105 |
0.03851 |
0.9568 |
Peak4 |
w |
0.27994 |
0.10754 |
2.60306 |
0.01106 |
0.91192 |
Peak5 |
y0 |
0 |
0 |
-- |
-- |
0 |
Peak5 |
xc |
1.98185 |
0.01743 |
113.679 |
2.00E-88 |
0.9043 |
Peak5 |
A |
-0.0179 |
0.0033 |
-5.4389 |
5.96E-07 |
0.9581 |
Peak5 |
w |
0.23862 |
0.02776 |
8.59698 |
6.54E-13 |
0.86043 |
Peak6 |
y0 |
0 |
0 |
-- |
-- |
0 |
Peak6 |
xc |
2.57665 |
0.00437 |
589.804 |
0.00745 |
|
Peak6 |
A |
-0.0374 |
9.92E-04 |
-37.735 |
7.38E-52 |
0.34456 |
Peak6 |
w |
0.33779 |
0.01043 |
32.3847 |
5.49E-47 |
0.35634 |
Formula 1
Formula 2
Arakelov G.G. (1998). Galvanic skin reaction as a manifestation of emotional, orienting and motor components of stress. In G.G. Arakelov, & E.K. Schott [Psikhologicheskiy zhurnal], 19(4),70–79.
Beatty J. (2000). The pupillary system. In J. Beatty, B. Lucero-Wagoner. Handbook of psychophysiology.New York, 142–162.
Belov D. R. (2016). Saccades and Predict Potentials when Playing Tetris. In D.R. Belov, E.A. Milyutina, & S.F. Kolodyazhny[Rossiyskiy fiziologicheskiy zhurnal imeni I.M. Sechenova],102(10), 1233–1245.
Borisyuk G.N. et al. (2002). Models of the dynamics of neural activity in brain processing of information - the results of the “decade”. [Uspekhi fizicheskikh nauk], 172(10), 1189–1214. doi:10.3367/UFNr.0172.200210d.1189
Boronenko M., Boronenko Y., Zelenskiy V., & Kiseleva E. (2020). Use of Active Test Objects in Security Systems. In: Ayaz H. (eds.) Advances in Neuroergonomics and Cognitive Engineering. AHFE 2019. Advances in Intelligent Systems and Computing, Vol. 953. Springer, Cham.doi:10.1007/978-3-030-20473-0_43
Cacioppo, J. T. (2004). Feelings and emotions: role for electrophysioogical markers. In J. T. Cacioppo. Biological Psychology, 67, 235–243. doi:10.1016/j.biopsycho.2004.03.009
Costela F.M. [et al.] (2017). Changes in visibility as a function of spatial frequency and microsaccade occurrence. European Journal of Neuroscience,45(3), 433–439.doi:10.1111/ejn.13487
Ershova R.V. (2018). High sensitivity and its connection with the parameters of pupillary reaction and personal characteristics. In R.V. Ershova, E.V. Yarmots [Vestnik VyatGU],4, 130–138.
Ershova R.V. (2014). On psychophysiological predictors of personal properties. In R.V. Ershova, N.N. Varchenko, K.A. Gankin [Chelovecheskiy Kapital],7,52–55.
Ganin I.P., Kosichenko E.A., & Kaplan A.Ya. (2018). Recognition of the subjective focus of interest in emotionally significant visual stimuli in the brain-computer interface on the P300 wave. [Vestnik Moskovskogo universiteta]. Series 14. Psychology, 1, 3–20. doi:10.11621/vsp.2018.01.03
Kiran [et al.] R. (2018). Real-Time Eye-Tracking System to Evaluate and Enhance Situation Awareness and Process Safety in Drilling Operations. IADC/SPE Drilling Conference and Exhibition, 6–8 March, Fort Worth, Texas.[USA].doi:10.2118/189678-MS.
Kolesnikov V.V. et al. (2004). Features of the pupillary reflex in drug addicts during the period of acute withdrawal. [Voprosy narkologii], 4, 39–46.
Kruchinina A.P. (2018). Mathematical model of optimal saccadic eye movement realized by a pair of muscles. In А.P. Kruchinina, A.G. Yakushev [Biofizika], 63(2),334–341. doi:10.1134/S0006350918020161
Kutsalo A.L. et al.(2018). Dynamic pupillometry as a screening method for diagnosing poisoning with industrial toxicants. In [Meditsina ekstremalnykh situatsiy], 20(3),487–493.
Kutsalo Anatoly Leonidovich (2004). Pupillometriya as a method of express diagnostics of drug intoxication: Ph.D. in Medicine, thesis. St. Petersburg, 118.
Lowet E. (2018). Microsaccade-rhythmic modulation of neural synchronization and coding within and across cortical areas V1 and V2. In E. Lowet [et al.] PLoS biology, 16(5).Art. e2004132. doi: 10.1371/journal.pbio.2004132
Maddess T. [et al.] (2009). Multifocal pupillographic visual field testing in glaucoma. Clinical & experimental ophthalmology,37(7), 678–686.doi:10.1111/j.1442-9071.2009.02107.x
Menshikov G.Ya., & Kovalev A.I. (2018). The role of nystagmus eye movements in developing the illusion of body movement. [Vestnik Moskovskogo universiteta]. Series 14. Psychology, 4, 135–148. doi:10.11621/vsp.2018.04.135
Mesin, L. (2018). Estimation of complexity of sampled biomedical continuous time signals using approximate entropy In L. Mesin, Frontiers in Physiology, 9(710). doi: 10.3389/fphys.2018.00710
Meyberg S. [et al.] (2015). Microsaccade-related brain potentials signal the focus of visuospatial attention. NeuroImage, 104, 79–88.doi:10.1016/j.neuroimage.2014.09.065
Novikov N. A. (2018). The role of beta and gamma rhythms in the implementation of working memory functions. In N.A. Novikov, & B.S. Gutkin [Psikhologiya. Zhurnal Vysshey shkoly ekonomiki],15(1), 174–182.
Ohayon, Jacques. "Pupil Distortion Measurement and Psychiatric Diagnosis Method." U.S. Patent Application No. 15/853, 899.
Ohayon J. Pupil distortion measurement and psychiatric diagnosis method: Patent US 10,182,755; applicant J. Ohayon. 15/853,899; filed 25.12.17. 10.
Otero-Millan J., Macknik S. L., & Martinez-Conde S. System and method for using microsaccade dynamics to measure attentional response to a stimulus: Patent US 9,854,966; assignee D. Health. 14/359,235; filed 23.11.12; pub. date 30.05.13, pub. WO2013/078462. 10.
Otero-Millan J. System and method for using microsaccade dynamics to measure attentional response to a stimulus : Patent US 9,854,966. In J. Otero-Millan, S. L. Macknik, S. Martinez-Conde; assignee: Dignity Health.14/359,235; filed 23.11.12; pub. date 30.05.13, pub. WO2013/078462. – 10.
Pashkov A.A. (2017). Electroencephalographic biomarkers of experimentally induced stress. In A.A. Pashkov, I.S. Dakhtin, N.S. Kharisova [Vestnik YUzhno-Uralskogo gosudarstvennogo universiteta], Series Psychology, 10(4). doi:10.14529/psy170407
Perotti L. [et al.] (2019). Discrete structure of the brain rhythms. Scientific reports, 9(1),1105.doi:10.1038/s41598-018-37196-0
Potter M.C. [et al.] (2014). Detecting meaning in RSVP at 13 ms per picture. Attention, Perception, & Psychophysics,76(2),270–279. doi:10.3758/s13414-013-0605-z
Pritchard W. S. (1981). Psychophysiology of P300. Psychological Bulletin, 89,506–540.doi:10.1037/0033-2909.89.3.506
Pronina A.S., Grigoryan R.K., & Kaplan A.Ya. (2018). Human eye movements when typing in the brain-computer interface based on the P300 potential: the effect of stimulus size and distance between stimuli. [Vestnik Moskovskogo universiteta]. Series 14. Psychology, 4, 120–134. doi:10.11621/vsp.2018.04.120
Sapountzis, P. (2018). Neural signatures of attention: insights from decoding population activity patterns. In P. Sapountzis, & G. G. Gregoriou, Frontiers in Bioscience,23, 221–246. doi: 10.2741/4588
Scherberger, H. (2005). Cortical local field potential encodes movement intentions in the posterior parietal cortex. In H. Scherberger, M.R. Jarvis, & R.A. Andersen, Neuron, 46, 347–354. doi: 10.1016/j.neuron.2005.03.004
Stern R. M. [et al.] (2001). Psychophysiological recording. Oxford [England]; New York: Oxford University Press, 282.
Stern, R.M., W.J. Ray, & K.S. Quigley (2001). "Eyes: Pupillography and electrooculography." Psychophysiological recording.Oxford University Press, New York, 125–141. doi:10.1093/acprof:oso/9780195113594.003.0009
Stern R.M. et al. (2001). Psychophysiological recording. Oxford University Press, USA. doi: 10.1093/acprof:oso/9780195113594.003.0003
Strauch, C. (2018.) Pupil dilation but not microsaccade rate robustly reveals decision formation. In C. Strauch, L. Greiter, A. Huckauf, Scientific reports.8. Art. 13165. 10.1038/s41598-018-31551-x
Tseng V.W.S. [et al.] (2018). AlertnessScanner: what do your pupils tell about your alertness. Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services. New York, 41.doi:10.1145/3229434.3229456
Watanabe M. [et al.] (2019). Ocular drift reflects volitional action preparation– Electronic text data. European Journal of Neuroscience.Retrieved from: https://onlinelibrary.wiley.com/doi/10.1111/ejn.14365 (accessed 18.06.2019). doi: 10.1111/ejn.14365
Marina P. Boronenko, Vladimir I. Zelensky, Elizaveta S. Kiseleva. (2019). Using waves of attention as a marker of hidden intentions. National Psychological Journal. 2, 88-98