Using waves of attention as a marker of hidden intentions

PDF (Rus)

Recieved: 04/09/2019

Accepted: 06/10/2019

Published: 07/30/2019

p.: 88-98

DOI: 10.11621/npj.2019.0212

Keywords: attention; focus of attention; brain evoked potential ; security systems; pupillogram; microcascades; oculogram

Available online: 30.01.2019

To cite this article:

Boronenko, Marina P., Zelensky, Vladimir I., Kiseleva, Elizaveta S.. Using waves of attention as a marker of hidden intentions. // National Psychological Journal 2019. 2. p.88-98. doi: 10.11621/npj.2019.0212

Copied to Clipboard

Copy

Boronenko, Marina P. Yugra State University

Zelensky, Vladimir I. Yugra State University

Kiseleva, Elizaveta S. Yugra State University

Abstract

Background. Recently, scientific and technological progress allows the widespread use of high-tech electronic means to create security systems. The advantages of identifying people who are high on drugs or alcohol with video surveillance systems on pupillograms are indisputable. However, those who bear aggressive intentions stay in the shade. The standard method of identifying emotions aimed at recording facial expressions is sufficient enough, but it is difficult to recognize negative intentions in a person if they keep control of themselves. To solve this problem, we propose to switch from passive safety systems to active ones. Therefore, studies of the pupillary response to the stimuli presented are relevant today.

The Objective of the research is to identify patterns of pupillograms that can be used to control pupillary reactions to the stimuli significant for an individual. Simultaneously, the following tasks were solved: checking the possibility of interpreting the pupillogram by synchronizing them with the tracks of the attention focus and searching for the sites of the pupillograms allegedly resulting from emotions in response to the presented stimuli.

Design. At the first stage, the images used as stimuli presented to the subjects of the research were selected. Incentives were thematic in nature and contributed to identifying the unstable psychophysical state of a person or their susceptibility to aggression. At the second stage, the calibration of the optoelectronic system used to record the pupillograms and oculograms, as well as stabilizing factors that affect the size of the pupils, was carried out. Pupilograms were obtained using groups of two age categories (16–25 years old and 45–50 years old) of 10 and 5 subjects accordingly (both males and females). The subjects selected for the research did not have any eye diseases; their eye sight was normal or adjusted.

Results.The interdependence of the size of the pupils and the displacement of the center of attention were identified. The verification of the pupillogram rank correlation was obtained when different subjects viewed identical sequences of visual stimuli showed that in general the p significance level did not exceed the critical value alpha = 0.05. The reliability of the correlation confirms the pupillograms depend on the shape of the objects viewed and the patterns that unite the pupillograms. The microsaccades in pupillograms are well explained by moving and focusing the gaze on the details of the image, which makes it possible to interpret them as waves of attention. Synchronizing the pupillograms and oculograms allows distinguishing areas that are presumably explained by the emotional reaction of the individual to a weak external stimulus. The Fourier analysis of the pupillograms revealed a change in the observed frequency spectrum, depending on the presence or absence of an emotional reaction, the speed of the shift in the focus of attention.

Findings.The observed set of frequencies suggests a connection between the diameters of the eye pupils and the brain potentials. The practical significance of the results is to expand the possibilities of using biometric security systems, including prevention of suicide in adolescents.

Fig. 1. Pupilogram synchronized with oculogram. Marina P. Boronenko, Vladimir I. Zelensky, Elizaveta S. Kiseleva. (2019). Using waves of attention as a marker of hidden intentions. National Psychological Journal. 2, 88-98

Fig. 2. Pupil response to simple stimuli (a-d); Typical reaction to a specific (Boronenko et al., 2019) test object (e), bottom-up: pupillogram, center of view X coordinate, center of attention center Y, illumination factor k.

Table 1. Rank Correlation

VAR	Rho	t	p	Тау	Инверс	Z	p	Гамма	R Пирсона
1S \ 2S	-,0.1561	-,3.9165	9.9938E-5	-,0.1074	-40,686.	,3.9874	6.6789E-5	-,0.1074	-,0.2603
1S \ 3S	-,0.0949	-,2.3629	,0.0184	-,0.0646	-24,468.	,2.398	,0.0165	-,0.0646	,0.0853
1S \ 4S	-,0.1355	-,3.3896	,0.0007	-,0.0896	-33,946.	,3.327	,0.0009	-,0.0896	-,0.1303
1S \ 4S	,0.6725	,22.5143	,0.	,0.4975	188,448.	,18.4701	,0.	,0.4976	,0.6693
2S \ 3S	-,0.3651	-,10.2789	,0.	-,0.2385	-113,026.	,9.3648	,0.	-,0.2385	-,0.2772
2S \ 4S	,0.0194	,0.5284	,0.5974	,0.0154	8,586.	,0.6302	,0.5285	,0.0154	,0.1319
2S \ 4S	-,0.1212	-,3.3564	,0.0008	-,0.085	-48,748.	,3.5007	,0.0005	-,0.085	-,0.2976
3S \ 4S	,0.4626	,13.6761	,0.	,0.3058	144,926.	,12.0082	,0.	,0.3058	,0.4568
3S \ 4S	-,0.1325	-,3.505	,0.0005	-,0.0953	-45,178.	,3.7434	,0.0002	-,0.0953	-,0.2054
4S \ 4S	-,0.3154	-,9.0705	,0.	-,0.2014	-112,198.	,8.236	,0.	-,0.2014	-,0.4566

Table 2. Descriptive statistics

	N total	Mean	Standard Deviation	Sum	Minimum	Median	Maximum
1S	364	0.99875	0.07096	363.54354	0.86444	0.992	1.33609
2S	758	0.98118	0.11338	743.73589	0.61018	1	1.26994
3S	689	0.99248	0.12762	683.82037	0.63156	1	1.43725
4S	747	0.98347	0.14308	734.65451	0.64147	1	1.30845

Table 3. Value parameters of the mathematical model

Gaussian		Value	Standard Error	t-Value	Prob>\|t\|	Dependency
Peak1	y0	0	0	--	--	0
Peak1	xc	0.50834	0.0074	68.6652	1.61E-71	0.6891
Peak1	A	0.00535	0.00336	1.59155	0.11553	0.98495
Peak1	w	0.08917	0.02257	3.95112	1.70E-04	0.87623
Peak2	y0	0	0	--	--	0
Peak2	xc	0.61801	0.03141	19.6738	5.65E-32	0.95839
Peak2	A	0.00985	0.00485	2.03177	0.04558	0.98541
Peak2	w	0.17945	0.06334	2.83308	0.00587	0.96216
Peak3	y0	0	0	--	--	0
Peak3	xc	0.98483	0.01371	71.8127	5.14E-73	0.78469
Peak3	A	0.05751	0.00313	18.3937	4.29E-30	0.88686
Peak3	w	0.57918	0.03467	16.7059	1.74E-27	0.87518
Peak4	y0	0	0	--	--	0
Peak4	xc	1.71418	0.05683	30.1653	9.31E-45	0.91466
Peak4	A	-0.0074	0.00352	-2.105	0.03851	0.9568
Peak4	w	0.27994	0.10754	2.60306	0.01106	0.91192
Peak5	y0	0	0	--	--	0
Peak5	xc	1.98185	0.01743	113.679	2.00E-88	0.9043
Peak5	A	-0.0179	0.0033	-5.4389	5.96E-07	0.9581
Peak5	w	0.23862	0.02776	8.59698	6.54E-13	0.86043
Peak6	y0	0	0	--	--	0
Peak6	xc	2.57665	0.00437	589.804		0.00745
Peak6	A	-0.0374	9.92E-04	-37.735	7.38E-52	0.34456
Peak6	w	0.33779	0.01043	32.3847	5.49E-47	0.35634

Formula 1

Formula 2

References

Andersen R.A. (2009). Intention, action planning, and decision making in parietal-frontal circuits. In R.A. Andersen, & H.Cui, Neuron, 63(5), 568–583.doi: 10.1016/j.neuron.2009.08.028
Arakelov G.G. (1998). Galvanic skin reaction as a manifestation of emotional, orienting and motor components of stress. In G.G. Arakelov, & E.K. Schott [Psikhologicheskiy zhurnal], 19(4),70–79.
Beatty J. (2000). The pupillary system. In J. Beatty, B. Lucero-Wagoner. Handbook of psychophysiology.New York, 142–162.
Belov D. R. (2016). Saccades and Predict Potentials when Playing Tetris. In D.R. Belov, E.A. Milyutina, & S.F. Kolodyazhny[Rossiyskiy fiziologicheskiy zhurnal imeni I.M. Sechenova],102(10), 1233–1245.
Borisyuk G.N. et al. (2002). Models of the dynamics of neural activity in brain processing of information - the results of the “decade”. [Uspekhi fizicheskikh nauk], 172(10), 1189–1214. doi:10.3367/UFNr.0172.200210d.1189
Boronenko M., Boronenko Y., Zelenskiy V., & Kiseleva E. (2020). Use of Active Test Objects in Security Systems. In: Ayaz H. (eds.) Advances in Neuroergonomics and Cognitive Engineering. AHFE 2019. Advances in Intelligent Systems and Computing, Vol. 953. Springer, Cham.doi:10.1007/978-3-030-20473-0_43
Cacioppo, J. T. (2004). Feelings and emotions: role for electrophysioogical markers. In J. T. Cacioppo. Biological Psychology, 67, 235–243. doi:10.1016/j.biopsycho.2004.03.009
Costela F.M. [et al.] (2017). Changes in visibility as a function of spatial frequency and microsaccade occurrence. European Journal of Neuroscience,45(3), 433–439.doi:10.1111/ejn.13487
Ershova R.V. (2018). High sensitivity and its connection with the parameters of pupillary reaction and personal characteristics. In R.V. Ershova, E.V. Yarmots [Vestnik VyatGU],4, 130–138.
Ershova R.V. (2014). On psychophysiological predictors of personal properties. In R.V. Ershova, N.N. Varchenko, K.A. Gankin [Chelovecheskiy Kapital],7,52–55.
Ganin I.P., Kosichenko E.A., & Kaplan A.Ya. (2018). Recognition of the subjective focus of interest in emotionally significant visual stimuli in the brain-computer interface on the P300 wave. [Vestnik Moskovskogo universiteta]. Series 14. Psychology, 1, 3–20. doi:10.11621/vsp.2018.01.03
Kiran [et al.] R. (2018). Real-Time Eye-Tracking System to Evaluate and Enhance Situation Awareness and Process Safety in Drilling Operations. IADC/SPE Drilling Conference and Exhibition, 6–8 March, Fort Worth, Texas.[USA].doi:10.2118/189678-MS.
Kolesnikov V.V. et al. (2004). Features of the pupillary reflex in drug addicts during the period of acute withdrawal. [Voprosy narkologii], 4, 39–46.
Kruchinina A.P. (2018). Mathematical model of optimal saccadic eye movement realized by a pair of muscles. In А.P. Kruchinina, A.G. Yakushev [Biofizika], 63(2),334–341. doi:10.1134/S0006350918020161
Kutsalo A.L. et al.(2018). Dynamic pupillometry as a screening method for diagnosing poisoning with industrial toxicants. In [Meditsina ekstremalnykh situatsiy], 20(3),487–493.
Kutsalo Anatoly Leonidovich (2004). Pupillometriya as a method of express diagnostics of drug intoxication: Ph.D. in Medicine, thesis. St. Petersburg, 118.
Lowet E. (2018). Microsaccade-rhythmic modulation of neural synchronization and coding within and across cortical areas V1 and V2. In E. Lowet [et al.] PLoS biology, 16(5).Art. e2004132. doi: 10.1371/journal.pbio.2004132
Maddess T. [et al.] (2009). Multifocal pupillographic visual field testing in glaucoma. Clinical & experimental ophthalmology,37(7), 678–686.doi:10.1111/j.1442-9071.2009.02107.x
Menshikov G.Ya., & Kovalev A.I. (2018). The role of nystagmus eye movements in developing the illusion of body movement. [Vestnik Moskovskogo universiteta]. Series 14. Psychology, 4, 135–148. doi:10.11621/vsp.2018.04.135
Mesin, L. (2018). Estimation of complexity of sampled biomedical continuous time signals using approximate entropy In L. Mesin, Frontiers in Physiology, 9(710). doi: 10.3389/fphys.2018.00710
Meyberg S. [et al.] (2015). Microsaccade-related brain potentials signal the focus of visuospatial attention. NeuroImage, 104, 79–88.doi:10.1016/j.neuroimage.2014.09.065
Novikov N. A. (2018). The role of beta and gamma rhythms in the implementation of working memory functions. In N.A. Novikov, & B.S. Gutkin [Psikhologiya. Zhurnal Vysshey shkoly ekonomiki],15(1), 174–182.
Ohayon, Jacques. "Pupil Distortion Measurement and Psychiatric Diagnosis Method." U.S. Patent Application No. 15/853, 899.
Ohayon J. Pupil distortion measurement and psychiatric diagnosis method: Patent US 10,182,755; applicant J. Ohayon. 15/853,899; filed 25.12.17. 10.
Otero-Millan J., Macknik S. L., & Martinez-Conde S. System and method for using microsaccade dynamics to measure attentional response to a stimulus: Patent US 9,854,966; assignee D. Health. 14/359,235; filed 23.11.12; pub. date 30.05.13, pub. WO2013/078462. 10.
Otero-Millan J. System and method for using microsaccade dynamics to measure attentional response to a stimulus : Patent US 9,854,966. In J. Otero-Millan, S. L. Macknik, S. Martinez-Conde; assignee: Dignity Health.14/359,235; filed 23.11.12; pub. date 30.05.13, pub. WO2013/078462. – 10.
Pashkov A.A. (2017). Electroencephalographic biomarkers of experimentally induced stress. In A.A. Pashkov, I.S. Dakhtin, N.S. Kharisova [Vestnik YUzhno-Uralskogo gosudarstvennogo universiteta], Series Psychology, 10(4). doi:10.14529/psy170407
Perotti L. [et al.] (2019). Discrete structure of the brain rhythms. Scientific reports, 9(1),1105.doi:10.1038/s41598-018-37196-0
Potter M.C. [et al.] (2014). Detecting meaning in RSVP at 13 ms per picture. Attention, Perception, & Psychophysics,76(2),270–279. doi:10.3758/s13414-013-0605-z
Pritchard W. S. (1981). Psychophysiology of P300. Psychological Bulletin, 89,506–540.doi:10.1037/0033-2909.89.3.506
Pronina A.S., Grigoryan R.K., & Kaplan A.Ya. (2018). Human eye movements when typing in the brain-computer interface based on the P300 potential: the effect of stimulus size and distance between stimuli. [Vestnik Moskovskogo universiteta]. Series 14. Psychology, 4, 120–134. doi:10.11621/vsp.2018.04.120
Sapountzis, P. (2018). Neural signatures of attention: insights from decoding population activity patterns. In P. Sapountzis, & G. G. Gregoriou, Frontiers in Bioscience,23, 221–246. doi: 10.2741/4588
Scherberger, H. (2005). Cortical local field potential encodes movement intentions in the posterior parietal cortex. In H. Scherberger, M.R. Jarvis, & R.A. Andersen, Neuron, 46, 347–354. doi: 10.1016/j.neuron.2005.03.004
Stern R. M. [et al.] (2001). Psychophysiological recording. Oxford [England]; New York: Oxford University Press, 282.
Stern, R.M., W.J. Ray, & K.S. Quigley (2001). "Eyes: Pupillography and electrooculography." Psychophysiological recording.Oxford University Press, New York, 125–141. doi:10.1093/acprof:oso/9780195113594.003.0009
Stern R.M. et al. (2001). Psychophysiological recording. Oxford University Press, USA. doi: 10.1093/acprof:oso/9780195113594.003.0003
Strauch, C. (2018.) Pupil dilation but not microsaccade rate robustly reveals decision formation. In C. Strauch, L. Greiter, A. Huckauf, Scientific reports.8. Art. 13165. 10.1038/s41598-018-31551-x
Tseng V.W.S. [et al.] (2018). AlertnessScanner: what do your pupils tell about your alertness. Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services. New York, 41.doi:10.1145/3229434.3229456
Watanabe M. [et al.] (2019). Ocular drift reflects volitional action preparation– Electronic text data. European Journal of Neuroscience.Retrieved from: https://onlinelibrary.wiley.com/doi/10.1111/ejn.14365 (accessed 18.06.2019). doi: 10.1111/ejn.14365

To cite this article:

Copied to Clipboard

Copy