ISSN 2079-6617
eISSN 2309-9828
Using waves of attention as a marker of hidden intentions

Using waves of attention as a marker of hidden intentions

PDF (Rus)

Recieved: 04/09/2019

Accepted: 06/10/2019

Published: 07/30/2019

p.: 88-98

DOI: 10.11621/npj.2019.0212

Keywords: attention; focus of attention; brain evoked potential ; security systems; pupillogram; microcascades; oculogram

Available online: 30.01.2019

To cite this article:

Boronenko, Marina P., Zelensky, Vladimir I., Kiseleva, Elizaveta S.. Using waves of attention as a marker of hidden intentions. // National Psychological Journal 2019. 2. p.88-98. doi: 10.11621/npj.2019.0212

Copied to Clipboard

Copy
Issue 2, 2019

Boronenko, Marina P. Yugra State University

Zelensky, Vladimir I. Yugra State University

Kiseleva, Elizaveta S. Yugra State University

Abstract

Background. Recently, scientific and technological progress allows the widespread use of high-tech electronic means to create security systems. The advantages of identifying people who are high on drugs or alcohol with video surveillance systems on pupillograms are indisputable. However, those who bear aggressive intentions stay in the shade. The standard method of identifying emotions aimed at recording facial expressions is sufficient enough, but it is difficult to recognize negative intentions in a person if they keep control of themselves. To solve this problem, we propose to switch from passive safety systems to active ones. Therefore, studies of the pupillary response to the stimuli presented are relevant today.

The Objective of the research is to identify patterns of pupillograms that can be used to control pupillary reactions to the stimuli significant for an individual. Simultaneously, the following tasks were solved: checking the possibility of interpreting the pupillogram by synchronizing them with the tracks of the attention focus and searching for the sites of the pupillograms allegedly resulting from emotions in response to the presented stimuli.

Design. At the first stage, the images used as stimuli presented to the subjects of the research were selected. Incentives were thematic in nature and contributed to identifying the unstable psychophysical state of a person or their susceptibility to aggression. At the second stage, the calibration of the optoelectronic system used to record the pupillograms and oculograms, as well as stabilizing factors that affect the size of the pupils, was carried out. Pupilograms were obtained using groups of two age categories (16–25 years old and 45–50 years old) of 10 and 5 subjects accordingly (both males and females). The subjects selected for the research did not have any eye diseases; their eye sight was normal or adjusted.

Results.The interdependence of the size of the pupils and the displacement of the center of attention were identified. The verification of the pupillogram rank correlation was obtained when different subjects viewed identical sequences of visual stimuli showed that in general the p significance level did not exceed the critical value alpha = 0.05. The reliability of the correlation confirms the pupillograms depend on the shape of the objects viewed and the patterns that unite the pupillograms. The microsaccades in pupillograms are well explained by moving and focusing the gaze on the details of the image, which makes it possible to interpret them as waves of attention. Synchronizing the pupillograms and oculograms allows distinguishing areas that are presumably explained by the emotional reaction of the individual to a weak external stimulus. The Fourier analysis of the pupillograms revealed a change in the observed frequency spectrum, depending on the presence or absence of an emotional reaction, the speed of the shift in the focus of attention.

Findings.The observed set of frequencies suggests a connection between the diameters of the eye pupils and the brain potentials. The practical significance of the results is to expand the possibilities of using biometric security systems, including prevention of suicide in adolescents.

Fig. 1. Pupilogram synchronized with oculogram. Marina P. Boronenko, Vladimir I. Zelensky, Elizaveta S. Kiseleva. (2019). Using waves of attention as a marker of hidden intentions. National Psychological Journal. 2, 88-98

Fig. 2. Pupil response to simple stimuli (a-d); Typical reaction to a specific (Boronenko et al., 2019) test object (e), bottom-up: pupillogram, center of view X coordinate, center of attention center Y, illumination factor k.

Fig. 1. Pupilogram synchronized with oculogram. Marina P. Boronenko, Vladimir I. Zelensky, Elizaveta S. Kiseleva. (2019). Using waves of attention as a marker of hidden intentions. National Psychological Journal. 2, 88-98

Fig. 2. Pupil response to simple stimuli (a-d); Typical reaction to a specific (Boronenko et al., 2019) test object (e), bottom-up: pupillogram, center of view X coordinate, center of attention center Y, illumination factor k.

Table 1. Rank Correlation

VAR

Rho

t

p

Тау

Инверс

Z

p

Гамма

R Пирсона

1S \ 2S

-,0.1561

-,3.9165

9.9938E-5

-,0.1074

-40,686.

,3.9874

6.6789E-5

-,0.1074

-,0.2603

1S \ 3S

-,0.0949

-,2.3629

,0.0184

-,0.0646

-24,468.

,2.398

,0.0165

-,0.0646

,0.0853

1S \ 4S

-,0.1355

-,3.3896

,0.0007

-,0.0896

-33,946.

,3.327

,0.0009

-,0.0896

-,0.1303

1S \ 4S

,0.6725

,22.5143

,0.

,0.4975

188,448.

,18.4701

,0.

,0.4976

,0.6693

2S \ 3S

-,0.3651

-,10.2789

,0.

-,0.2385

-113,026.

,9.3648

,0.

-,0.2385

-,0.2772

2S \ 4S

,0.0194

,0.5284

,0.5974

,0.0154

8,586.

,0.6302

,0.5285

,0.0154

,0.1319

2S \ 4S

-,0.1212

-,3.3564

,0.0008

-,0.085

-48,748.

,3.5007

,0.0005

-,0.085

-,0.2976

3S \ 4S

,0.4626

,13.6761

,0.

,0.3058

144,926.

,12.0082

,0.

,0.3058

,0.4568

3S \ 4S

-,0.1325

-,3.505

,0.0005

-,0.0953

-45,178.

,3.7434

,0.0002

-,0.0953

-,0.2054

4S \ 4S

-,0.3154

-,9.0705

,0.

-,0.2014

-112,198.

,8.236

,0.

-,0.2014

-,0.4566

Table 2. Descriptive statistics

 

N total

Mean

Standard Deviation

Sum

Minimum

Median

Maximum

1S

364

0.99875

0.07096

363.54354

0.86444

0.992

1.33609

2S

758

0.98118

0.11338

743.73589

0.61018

1

1.26994

3S

689

0.99248

0.12762

683.82037

0.63156

1

1.43725

4S

747

0.98347

0.14308

734.65451

0.64147

1

1.30845

Table 3. Value parameters of the mathematical model

Gaussian

Value

Standard Error

t-Value

Prob>|t|

Dependency

Peak1

y0

0

0

--

--

0

Peak1

xc

0.50834

0.0074

68.6652

1.61E-71

0.6891

Peak1

A

0.00535

0.00336

1.59155

0.11553

0.98495

Peak1

w

0.08917

0.02257

3.95112

1.70E-04

0.87623

Peak2

y0

0

0

--

--

0

Peak2

xc

0.61801

0.03141

19.6738

5.65E-32

0.95839

Peak2

A

0.00985

0.00485

2.03177

0.04558

0.98541

Peak2

w

0.17945

0.06334

2.83308

0.00587

0.96216

Peak3

y0

0

0

--

--

0

Peak3

xc

0.98483

0.01371

71.8127

5.14E-73

0.78469

Peak3

A

0.05751

0.00313

18.3937

4.29E-30

0.88686

Peak3

w

0.57918

0.03467

16.7059

1.74E-27

0.87518

Peak4

y0

0

0

--

--

0

Peak4

xc

1.71418

0.05683

30.1653

9.31E-45

0.91466

Peak4

A

-0.0074

0.00352

-2.105

0.03851

0.9568

Peak4

w

0.27994

0.10754

2.60306

0.01106

0.91192

Peak5

y0

0

0

--

--

0

Peak5

xc

1.98185

0.01743

113.679

2.00E-88

0.9043

Peak5

A

-0.0179

0.0033

-5.4389

5.96E-07

0.9581

Peak5

w

0.23862

0.02776

8.59698

6.54E-13

0.86043

Peak6

y0

0

0

--

--

0

Peak6

xc

2.57665

0.00437

589.804

0.00745

Peak6

A

-0.0374

9.92E-04

-37.735

7.38E-52

0.34456

Peak6

w

0.33779

0.01043

32.3847

5.49E-47

0.35634

Formula 1. Marina P. Boronenko, Vladimir I. Zelensky, Elizaveta S. Kiseleva. (2019). Using waves of attention as a marker of hidden intentions. National Psychological Journal. 2, 88-98

Formula 1

Formula 2. Marina P. Boronenko, Vladimir I. Zelensky, Elizaveta S. Kiseleva. (2019). Using waves of attention as a marker of hidden intentions. National Psychological Journal. 2, 88-98

Formula 2

References

  1. Andersen R.A. (2009). Intention, action planning, and decision making in parietal-frontal circuits. In R.A. Andersen, & H.Cui, Neuron, 63(5), 568–583.doi: 10.1016/j.neuron.2009.08.028
  2. Arakelov G.G. (1998). Galvanic skin reaction as a manifestation of emotional, orienting and motor components of stress. In G.G. Arakelov, & E.K. Schott [Psikhologicheskiy zhurnal], 19(4),70–79.

  3. Beatty J. (2000). The pupillary system. In J. Beatty, B. Lucero-Wagoner.  Handbook of psychophysiology.New York, 142–162.

  4. Belov D. R. (2016). Saccades and Predict Potentials when Playing Tetris. In D.R. Belov, E.A. Milyutina, & S.F. Kolodyazhny[Rossiyskiy fiziologicheskiy zhurnal imeni I.M. Sechenova],102(10)1233–1245.

  5. Borisyuk G.N. et al. (2002). Models of the dynamics of neural activity in brain processing of information - the results of the “decade”. [Uspekhi fizicheskikh nauk],  172(10), 1189–1214. doi:10.3367/UFNr.0172.200210d.1189

  6. Boronenko M., Boronenko Y., Zelenskiy V., & Kiseleva E. (2020). Use of Active Test Objects in Security Systems. In: Ayaz H. (eds.) Advances in Neuroergonomics and Cognitive Engineering. AHFE 2019. Advances in Intelligent Systems and Computing, Vol. 953. Springer, Cham.doi:10.1007/978-3-030-20473-0_43

  7. Cacioppo, J. T. (2004). Feelings and emotions: role for electrophysioogical markers. In J. T. Cacioppo. Biological Psychology67, 235–243. doi:10.1016/j.biopsycho.2004.03.009

  8. Costela F.M. [et al.] (2017). Changes in visibility as a function of spatial frequency and microsaccade occurrence. European Journal of Neuroscience,45(3), 433–439.doi:10.1111/ejn.13487

  9. Ershova R.V. (2018). High sensitivity and its connection with the parameters of pupillary reaction and personal characteristics. In R.V. Ershova, E.V. Yarmots [Vestnik VyatGU],4, 130–138.

  10. Ershova R.V. (2014). On psychophysiological predictors of personal properties. In R.V. Ershova, N.N. Varchenko, K.A. Gankin  [Chelovecheskiy Kapital],7,52–55.

  11. Ganin I.P., Kosichenko E.A., & Kaplan A.Ya. (2018). Recognition of the subjective focus of interest in emotionally significant visual stimuli in the brain-computer interface on the P300 wave. [Vestnik Moskovskogo universiteta]. Series 14. Psychology, 1, 3–20. doi:10.11621/vsp.2018.01.03

  12. Kiran [et al.] R. (2018). Real-Time Eye-Tracking System to Evaluate and Enhance Situation Awareness and Process Safety in Drilling Operations. IADC/SPE Drilling Conference and Exhibition, 6–8 March, Fort Worth, Texas.[USA].doi:10.2118/189678-MS.

  13. Kolesnikov V.V. et al. (2004). Features of the pupillary reflex in drug addicts during the period of acute withdrawal. [Voprosy narkologii], 4, 39–46.

  14. Kruchinina A.P. (2018). Mathematical model of optimal saccadic eye movement realized by a pair of muscles. In А.P. Kruchinina, A.G. Yakushev [Biofizika], 63(2),334–341. doi:10.1134/S0006350918020161

  15. Kutsalo A.L. et al.(2018). Dynamic pupillometry as a screening method for diagnosing poisoning with industrial toxicants. In [Meditsina ekstremalnykh situatsiy]20(3),487–493.

  16. Kutsalo Anatoly Leonidovich (2004). Pupillometriya as a method of express diagnostics of drug intoxication: Ph.D. in Medicine, thesis. St. Petersburg, 118.

  17. Lowet E. (2018). Microsaccade-rhythmic modulation of neural synchronization and coding within and across cortical areas V1 and V2. In E. Lowet [et al.] PLoS biology,  16(5).Art. e2004132. doi:  10.1371/journal.pbio.2004132

  18. Maddess T. [et al.] (2009). Multifocal pupillographic visual field testing in glaucoma. Clinical & experimental ophthalmology,37(7), 678–686.doi:10.1111/j.1442-9071.2009.02107.x

  19. Menshikov G.Ya., & Kovalev A.I. (2018). The role of nystagmus eye movements in developing the illusion of body movement. [Vestnik Moskovskogo universiteta]. Series 14. Psychology, 4, 135–148. doi:10.11621/vsp.2018.04.135

  20. Mesin, L. (2018). Estimation of complexity of sampled biomedical continuous time signals using approximate entropy In L. Mesin, Frontiers in Physiology, 9(710). doi: 10.3389/fphys.2018.00710

  21. Meyberg S. [et al.] (2015). Microsaccade-related brain potentials signal the focus of visuospatial attention. NeuroImage104, 79–88.doi:10.1016/j.neuroimage.2014.09.065

  22. Novikov N. A. (2018). The role of beta and gamma rhythms in the implementation of working memory functions. In N.A. Novikov, & B.S. Gutkin [Psikhologiya. Zhurnal Vysshey shkoly ekonomiki],15(1), 174–182.

  23. Ohayon, Jacques. "Pupil Distortion Measurement and Psychiatric Diagnosis Method." U.S. Patent Application No. 15/853, 899.

  24. Ohayon J. Pupil distortion measurement and psychiatric diagnosis method: Patent US 10,182,755; applicant J. Ohayon. 15/853,899; filed 25.12.17. 10.

  25. Otero-Millan J., Macknik S. L., & Martinez-Conde S. System and method for using microsaccade dynamics to measure attentional response to a stimulus: Patent US  9,854,966; assignee D. Health. 14/359,235; filed 23.11.12; pub. date 30.05.13, pub. WO2013/078462. 10.

  26. Otero-Millan J. System and method for using microsaccade dynamics to measure attentional response to a stimulus : Patent US 9,854,966. In J. Otero-Millan, S. L. Macknik, S. Martinez-Conde; assignee: Dignity Health.14/359,235; filed 23.11.12; pub. date 30.05.13, pub. WO2013/078462. – 10.

  27. Pashkov A.A. (2017). Electroencephalographic biomarkers of experimentally induced stress. In A.A. Pashkov, I.S. Dakhtin, N.S. Kharisova [Vestnik YUzhno-Uralskogo gosudarstvennogo universiteta], Series Psychology10(4).  doi:10.14529/psy170407

  28. Perotti L. [et al.] (2019). Discrete structure of the brain rhythms. Scientific reports,  9(1),1105.doi:10.1038/s41598-018-37196-0

  29. Potter M.C. [et al.] (2014). Detecting meaning in RSVP at 13 ms per picture. Attention, Perception, & Psychophysics,76(2),270–279. doi:10.3758/s13414-013-0605-z

  30. Pritchard W. S. (1981). Psychophysiology of P300. Psychological Bulletin89,506–540.doi:10.1037/0033-2909.89.3.506

  31. Pronina A.S., Grigoryan R.K., & Kaplan A.Ya. (2018). Human eye movements when typing in the brain-computer interface based on the P300 potential: the effect of stimulus size and distance between stimuli. [Vestnik Moskovskogo universiteta]. Series 14. Psychology, 4, 120–134. doi:10.11621/vsp.2018.04.120

  32. Sapountzis, P. (2018). Neural signatures of attention: insights from decoding population activity patterns. In P. Sapountzis, & G. G. Gregoriou, Frontiers in Bioscience,23, 221–246. doi: 10.2741/4588

  33. Scherberger, H. (2005). Cortical local field potential encodes movement intentions in the posterior parietal cortex. In H. Scherberger, M.R. Jarvis, & R.A. Andersen, Neuron, 46, 347–354. doi: 10.1016/j.neuron.2005.03.004

  34. Stern R. M. [et al.] (2001). Psychophysiological recording. Oxford [England]; New York: Oxford University Press, 282.

  35. Stern, R.M., W.J. Ray, & K.S. Quigley (2001). "Eyes: Pupillography and electrooculography." Psychophysiological recording.Oxford University Press, New York, 125–141. doi:10.1093/acprof:oso/9780195113594.003.0009

  36. Stern R.M. et al. (2001). Psychophysiological recording. Oxford University Press, USA. doi: 10.1093/acprof:oso/9780195113594.003.0003

  37. Strauch, C. (2018.) Pupil dilation but not microsaccade rate robustly reveals decision formation. In C. Strauch, L. Greiter, A. Huckauf, Scientific reports.8. Art. 13165. 10.1038/s41598-018-31551-x

  38. Tseng V.W.S. [et al.] (2018). AlertnessScanner: what do your pupils tell about your alertness. Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services.  New York, 41.doi:10.1145/3229434.3229456

  39. Watanabe M. [et al.] (2019). Ocular drift reflects volitional action preparation– Electronic text data. European Journal of Neuroscience.Retrieved from: https://onlinelibrary.wiley.com/doi/10.1111/ejn.14365 (accessed 18.06.2019). doi: 10.1111/ejn.14365

To cite this article:

Boronenko, Marina P., Zelensky, Vladimir I., Kiseleva, Elizaveta S.. Using waves of attention as a marker of hidden intentions. // National Psychological Journal 2019. 2. p.88-98. doi: 10.11621/npj.2019.0212

Copied to Clipboard

Copy