Study on drivers' visual perception characteristics during the take-over of vehicle control in automated driving
Corresponding Author
Jianwei Niu
Department of Logistics Engineering, School of Mechanical Engineering, University of Science and Technology Beijing, Beijing, China
Correspondence Jianwei Niu, Department of Logistics Engineering, School of Mechanical Engineering, University of Science and Technology Beijing, Beijing 100081, China.
Email: niujw@ustb.edu.cn
Search for more papers by this authorHaixin Xu
Department of Logistics Engineering, School of Mechanical Engineering, University of Science and Technology Beijing, Beijing, China
Search for more papers by this authorYipin Sun
Department of Logistics Engineering, School of Mechanical Engineering, University of Science and Technology Beijing, Beijing, China
Search for more papers by this authorHua Qin
Department of Industrial Engineering, School of Mechanical-Electronic and Automobile Engineering, Beijing University of Civil Engineering and Architecture, Beijing, China
Search for more papers by this authorCorresponding Author
Jianwei Niu
Department of Logistics Engineering, School of Mechanical Engineering, University of Science and Technology Beijing, Beijing, China
Correspondence Jianwei Niu, Department of Logistics Engineering, School of Mechanical Engineering, University of Science and Technology Beijing, Beijing 100081, China.
Email: niujw@ustb.edu.cn
Search for more papers by this authorHaixin Xu
Department of Logistics Engineering, School of Mechanical Engineering, University of Science and Technology Beijing, Beijing, China
Search for more papers by this authorYipin Sun
Department of Logistics Engineering, School of Mechanical Engineering, University of Science and Technology Beijing, Beijing, China
Search for more papers by this authorHua Qin
Department of Industrial Engineering, School of Mechanical-Electronic and Automobile Engineering, Beijing University of Civil Engineering and Architecture, Beijing, China
Search for more papers by this authorAbstract
A high degree of automated driving distracts drivers more easily, resulting in slow recognition of critical events during driving and slow responses to emergencies. Automated driving and manual switching processes are also prone to erroneous decisions. We conducted a simulated automated driving experiment to study participants' visual perception characteristics during the take-over of vehicle control. The present study used dynamic videos to imitate the driving situations when drivers returned their gaze from the distractive source to the road. We collected the drivers' eye movement data to analyze the search strategy and physiological characteristics of the drivers after the take-over reminder. The results showed that the instant information search method of the drivers was scanning of the driving scene. When the degree of distraction deepened and the hazard level of scenes increased, the pupil diameter of the drivers increased and the fixation duration became longer. These findings can help to design take-over warnings and support more intelligent automated driving systems to judge whether measures should be taken to interfere with the driver's operation to avoid collisions. Furthermore, the drivers' fixation point distribution focused on the left side and the lower side of the scene. We suggest that the take-over warning is displayed in the head-up display. This study provides a better understanding of drivers' visual perception characteristics when drivers' eyesight returns from other distractors to the driving scene and a good theoretical basis for the design of hazard warning information for automated driving.
CONFLICT OF INTERESTS
The authors declare that there are no conflict of interests.
REFERENCES
- Anderson, J. M., Kalra, N., Stanley, K. D., Sorensen, P., Samaras, C., & Oluwatola, O. A. (2014). Autonomous Vehicle Technology: A Guide for Policymakers. Santa Monica, CA: RAND Corporation.
- van den Beukel, A. P., & van der Voort, M. C. (2013). The influence of time-criticality on Situation Awareness when retrieving human control after automated driving. IEEE International Conference on Intelligent Transportation Systems, 2000–2005.
- Biederman, I. (1972). Perceiving real-world scenes. Science, 177, 77–80.
- Brandenburg, S., & Chuang, L. (2019). Take-over requests during highly automated driving: How should they be presented and under what conditions? Transportation Research Part F: Traffic Psychology and Behaviour, 66, 214–255.
- Chee, D. Y. T., Lee, H. C. Y., Patomella, A.-H., & Falkmer, T. (2019). The visual search patterns of drivers with autism spectrum disorders in complex driving scenarios. Journal of Transport & Health, 14, 100597.
- Crundall, D., Chapman, P., Trawley, S., Collins, L., Loon, E. V., Andrews, B., & Underwood, G. (2012). Some hazards are more attractive than others: Drivers of varying experience respond differently to different types of hazard. Accident Analysis & Prevention, 45, 600–609.
- Davenport, J. L. (2007). Consistency effects between objects in scenes. Memory & Cognition, 35, 393–401.
- Desolneux, A., Moisan, L., & Morel, J. M. (2004). Gestalt theory and computer vision. Theory & Decision Library A, 38, 71–101.
10.1007/1-4020-2081-3_4 Google Scholar
- Friedman, A. (1979). Framing pictures: The role of knowledge in automatized encoding and memory for gist. Journal of Experimental Psychology. General, 108, 316–355.
- Gerhard, M., Christopher, C., & Joost, D. E. (2015). Review of eye-related measures of drivers' mental workload. Procedia Manufacturing, 3, 2854–2861.
10.1016/j.promfg.2015.07.783 Google Scholar
- Gold, C., Dambock, D., Lorenz, L., & Bengler, K. (2013). “Take over!” How long does it take to get the driver back into the loop? Proceedings of the Human Factors & Ergonomics Society Annual Meeting, 57, 1938–1942.
10.1177/1541931213571433 Google Scholar
- Gold, C., Körber, M., Lechner, D., & Bengler, K. (2014). Taking over control from highly automated vehicles in complex traffic situations: The role of traffic density. Human Factors, 58, 642–652.
- He, S. Y., Liang, B., Pan, G. B., Wang, F., & Cui, L. L. (2017). Influence of dynamic highway tunnel lighting environment on driving safety based on eye movement parameters of the driver. Tunnelling and Underground Space Technology, 67, 52–60.
- Henderson, J. (2005). Introduction to real-world scene perception. Visual Cognition, 12, 849–851.
- Hills, P. J., Thompson, C., & Pake, J. M. (2018). Detrimental effects of carryover of eye movement behaviour on hazard perception accuracy: Effects of driver experience, difficulty of task, and hazardousness of road. Transportation Research Part F: Traffic Psychology and Behaviour, 58, 906–916.
- Hollingworth, A. (2004). Constructing visual representations of natural scenes: The roles of short- and long-term visual memory. Journal of Experimental Psychology Human Perception & Performance, 30(3), 519–537.
- Iqbal, S. T., Zheng, X. S., & Bailey, B. P. (2004). Task-evoked pupillary response to mental workload in human-computer interaction. Paper presented at the Conference on Human Factors and Computing Systems.
- Konstantopoulos, P., Chapman, P., & Crundall, D. (2010). Driver's visual attention as a function of driving experience and visibility. Using a driving simulator to explore drivers' eye movements in day, night and rain driving. Accident Analysis and Prevention, 42(3), 827–834.
- Lehsing, C., Ruch, F., Kölsch, F. M., Dyszak, G. N., Haag, C., Feldstein, I. T., … Bowers, A. R. (2019). Effects of simulated mild vision loss on gaze, driving and interaction behaviors in pedestrian crossing situations. Accident Analysis and Prevention, 125, 138–151.
- Levy, N. K., Golkin, S. S., Borowsky, A., & Vakil, E. (2019). Facilitating hazard awareness skills among drivers regardless of age and experience through repetitive exposure to real-life short movies of hazardous driving situations. Transportation Research Part F: Traffic Psychology and Behaviour, 60, 353–365.
- Liu, H. W., Yang, R., Wang, L., & Liu, P. (2019). Evaluating initial public acceptance of highly and fully autonomous vehicles. International Journal of Human-Computer Interaction, 35, 919–931.
- Liu, P., Guo, Q. R., Ren, F., Wang, L., & Xu, Z. G. (2019). Willingness to pay for self-driving vehicles: Influences of demographic and psychological factors. Transportation Research Part C: Emerging Technologies, 100, 306–317.
- Liu, P., Ma, Y. J., & Zuo, Y. Q. (2019). Self-driving vehicles: Are people willing to trade risks for environmental benefits? Transportation Research, 125, 139–149.
- Liu, P., & Xu, Z. G. (2020). Public attitude toward self-driving vehicles on public roads: Direct experience changed ambivalent people to be more positive. Technological Forecasting & Social Change, 151, 119827.
- Liu, P., Yang, R., & Xu, Z. G. (2019). Public acceptance of fully automated driving: Effects of social trust and risk/benefit perceptions. Risk Analysis, 39(2), 326–341.
- Ma, R., & Kaber, D. B. (2005). Situation awareness and workload in driving while using adaptive cruise control and a cell phone. International Journal of Industrial Ergonomics, 35(10), 939–953.
- Merat, N., Jamson, A. H., Lai, F. C., & Carsten, O. (2012). Highly automated driving, secondary task performance, and driver state. Human Factors, 54(5), 762–771.
- Niezgoda, M., Tarnowski, A., Kruszewski, M., & Kamiński, T. (2015). Towards testing auditory–vocal interfaces and detecting distraction while driving: A comparison of eye-movement measures in the assessment of cognitive workload. Transportation Research Part F, 32, 23–34.
- Oliva, A. (2005). Gist of the scene. Neurobiology of Attention, 696, 251–256.
10.1016/B978-012375731-9/50045-8 Google Scholar
- Palinko, O., & Kun, A. (2012). Exploring the influence of light and cognitive load on pupil diameter in driving simulator studies. Proceedings of the Symposium on Eye Tracking Research and Applications.
- Petermann-Stock, I., Hackenberg, L., Muhr, T., & Mergl, C. (2013). How long does the driver need? An analysis of takeover times from various ancillary activities during a highly automated traffic jam. Presented at the Sixth Conference Driver Assistance System, Munich, Germany, 1–26.
- Recarte, M. A., & Nunes, L. (2000). Effects of verbal and spatial-imagery task on eye fixations while driving. Journal of Experimental Psychology: Applied, 6, 31–43.
- Recarte, M. A., Pérez, E., Conchillo, A., & Nunes, L. M. (2008). Mental workload and visual impairment: Differences between pupil, blink, and subjective rating. Spanish Journal of Psychology, 11, 374–85.
- Regan, M. A., Lee, J. D., & Young, K. L. (2008). Driver distraction: Theory, effects and mitigation. Boca Raton, FL, USA: CRC Press.
10.1201/9781420007497 Google Scholar
- Scialfa, C. T., Borkenhagen, D., Lyon, J., Deschênes, M., Horswill, M., & Wetton, M. (2012). The effects of driving experience on responses to a static hazard perception test. Accident Analysis and Prevention, 45, 547–553.
- Smith, B. W. (2013). SAE levels of driving automation. Retrieved from http://cyberlaw.stanford.edu/blog/2013/12/sae-levels-driving-automation
- Tsai, Y., Viirre, E., Strychacz, C., Chase, B., & Jung, T. (2007). Task performance and eye activity: Predicting behavior relating to cognitive workload aviation. Space, and Environmental Medicine, 78, 176–185.
- Underwood, S. E. (2014). Automated vehicles forecast vehicle symposium opinion survey. Paper presented at the Automated Vehicles Symposium, San Francisco, CA.
- Ventsislavova, P., Gugliotta, A., Peña-Suarez, E., Garcia-Fernandez, P., Eisman, E., Crundall, D., & Castrob, C. (2016). What happens when drivers face hazards on the road? Accident Analysis and Prevention, 91, 43–54.
- Wadud, Z., MacKenzie, D., & Leiby, P. (2016). Help or hindrance? The travel, energy and carbon impacts of highly automated vehicles. Transportation Research Part A, 86, 1–18.
- Wang, Y., Bao, S., Du, W. J., Ye, Z. R., & James, R. S. (2017). Examining drivers' eye glance patterns during distracted driving: Insights from scanning randomness and glance transition matrix. Journal of Safety Research, 63, 149–155.
- Wickens, C. D., & Hollands, J. G. (2000). Engineering psychology and human performance ( 3rd ed.). Upper Saddle River, NJ: Prentice Hall.
- Xu, Y., Li, S. W., Tan, D. R., Guo, D., & Wang, Y. Q. (2018b). Recognition method of construction conflict based on driver's eye movement. Accident Analysis and Prevention, 113, 193–201.
- Xu, Z. G., Zhang, K. F., Min, H. G., Wang, Z., Zhao, X. M., & Liu, P. (2018a). What drives people to accept automated vehicles? Findings from a field experiment. Transportation Research Part C, 95, 320–334.
- Zeeb, K., Buchner, A., & Schrauf, M. (2015). What determines the take-over time? An integrated model approach of driver take-over after automated driving. Accident Analysis & Prevention, 78, 212–221.
- Zeeb, K., Buchner, A., & Schrauf, M. (2016). Is take-over time all that matters? The impact of visual-cognitive load on driver take-over quality after conditionally automated driving. Accident Analysis & Prevention, 92, 230–239.
- Zhang, X. G., & Gao, J. P. (2019). Research on the fixation transition behavior of drivers on expressway in foggy environment. Safety Science, 119, 70–75.