Abstract

In the research and development (R&D) and verification and validation (V&V) phases of autonomous driving decision-making and planning systems, it is necessary to integrate human factors to achieve decision-making and evaluation that align with human cognition. However, most existing datasets primarily focus on vehicle motion states and trajectories, neglecting human-related information. In addition, current naturalistic driving datasets lack sufficient safety-critical scenarios while simulated datasets suffer from low authenticity. To address these issues, this paper constructs the Risk-Informed Subjective Evaluation and Eye-tracking (RISEE) dataset which specially contains human subjective evaluations and eye-tracking data apart from regular naturalistic driving trajectories. By leveraging the complementary advantages of drone-based (high realism and extensive scenario coverage) and simulation-based (high safety and reproducibility) data collection methods, we first conduct drone-based traffic video recording at a highway ramp merging area. After that, the manually selected highly interactive scenarios are reconstructed in simulation software, and drivers’ first-person view (FPV) videos are generated, which are then viewed and evaluated by recruited participants. During the video viewing process, participants’ eye-tracking data is collected. After data processing and filtering, 3567 valid subjective risk ratings from 101 participants across 179 scenarios are retained, along with 2045 qualified eye-tracking data segments.

Data Collection Process

Examples of FPV Videos

Representive Scenarios

We present two representative highly interactive videos.

Typical Scenarios

Typical scenarios are used to calibrate participants’ risk expectations (i.e., all the viewed scenarios’ risk levels are bracketed between these two reference extremes).

• Typical Safe Scenario

• Typical Critical Scenario

Baseline Scenarios

Baseline videos are used to acclimate participants to the viewing environment.

• Baseline Scenaro with a Sedan’s Perspactive

• Baseline Scenaro with a Truck’s Perspactive

Citation

@article{risee2025,                                                 
   title={RISEE: A Highly Interactive Naturalistic Driving Trajectories Dataset with Human Subjective Risk Perception and Eye-tracking Information},                 
   author={Wu, Xinzheng and Chen, Junyi and Wang, Peiyi and Chen, Shunxiang and Meng, Haolan and Shen, Yong},       
   journal={arXiv preprint arXiv:2507.19490},                                   
   year={2025},                                            
   note={Preprint accepted by IEEE ITSC 2025}                        
}   

Contact

If you have any questions or suggestions, please feel free to open an issue on GitHub or contact us (wuxinzheng@tongji.edu.cn or chenjunyi@tongji.edu.cn).