Explainable Pedestrian-Vehicle Conflict Risk Prediction
Multimodal pedestrian–vehicle conflict-risk prediction from a single image: SegFormer + pose + scene features, SHAP-selected, XGBoost 95.48% on RSUD20K.
Same image, more context
Scroll to watch it unfold. Each beat is one documented step of the project.
- 01
Road segmentation
An improved SegFormer finds the road.
- 02
People and vehicles
YOLO detects pedestrians and vehicles.
- 03
Pose
MediaPipe estimates each pedestrian's pose.
- 04
Spatial relationships
Distances and positions between people and vehicles are measured.
- 05
Risk + explanation
An ensemble scores the risk and SHAP shows which features drove it. On RSUD20K, XGBoost reached 95.48% accuracy.
- 06
Outcome
One street photo gives a conflict-risk score with its reasons.
An illustration of how the system works, drawn from the write-up below. It is not recorded output.
Overview
Pedestrian–vehicle conflicts are a major urban-safety problem, and off-the-shelf detectors struggle to judge risk from a static image because they miss scene context. This project predicts conflict risk from a single street photo by fusing several signals — improved SegFormer road segmentation, person detection (YOLO) and MediaPipe pose, plus the spatial relationships between pedestrians and vehicles — for a holistic view rather than a single cue.
Approach
A custom feature-extraction strategy captures multi-scale spatial relationships, pedestrian pose dynamics and road-scene context. The top features — selected by SHAP — feed several models (FT-Transformer, XGBoost, CatBoost) and a stacked-ensemble meta-learner, giving explainable predictions with uncertainty estimates at low latency. It's evaluated on RSUD20K, a Bangladeshi road-scene dataset, so it reflects local traffic and pedestrian behaviour.
Results
XGBoost reached 95.48% accuracy, with the stacked ensemble close behind (95.15%), CatBoost (94.43%) and FT-Transformer (89.67%) — while keeping predictions interpretable via SHAP and fast enough for proactive, real-world pedestrian-safety systems.
By the numbers
Tech stack & key skills
Core tools, methods and skills demonstrated in this project: