SEED4D is a comprehensive synthetic dataset that offers large-scale multi-view dynamic 4D driving data. By combining complex, dynamic, and multi-view data, this dataset provides the necessary mixture to aid in the development of 3D and 4D reconstruction methods in the autonomous driving context. Our Synthetic Ego-Exo Dynamic 4D (SEED4D) dataset includes two large-scale multi-view synthetic urban scene datasets, one static (3D) dataset with 212K inward- and outward-facing vehicle images from 2K scenes and the other dynamic (4D) dataset with 16.8M images from 10K trajectories, each sampled at 100 points in time. It contains egocentric images, exocentric images, and LiDAR data, making it possible to reconstruct different perspectives of the scene. Moreover, we have also created a customizable and easy-to-use data generator for spatio-temporal multi-view data creation.
What is covered by our Synthetic Ego-Exo Dynamic 4D (SEED4D) dataset?
Two datasets in one
SEED4D includes two large-scale multi-view synthetic urban scene datasets:
The Static Ego-Exo Dataset is designed for few-view image reconstruction tasks in an autonomous driving setting and contains 2002 single-timestep complex outdoor driving scenes, each offering six plus one outward-facing vehicle images and 100 images from exocentric viewpoints on a bounding sphere for supervision. Only a single vehicle in the scene is equipped with this setup. We define ego views to be 900 x 1600 pixels to match the NuScenes dataset, while surround vehicle exo views are 600 x 800 pixels.
The Dynamic Ego-Exo Dataset is a temporal dataset that consists of 10.5K driving trajectories suitable for 4D forecasting, reconstruction, or video prediction tasks. Each trajectory is 100 steps long, corresponding to a driving length of 10 seconds. The 10.5k trajectories come from a total of 498 scenes across all towns. In each scene, there are 21 vehicles equipped with six plus one outward-facing vehicle camera and ten inward-facing surround vehicle exocentric images. The ego views have a size of 128 x 256 pixels, while exo views have a dimension of 98 x 128 pixels.
Benchmark results
- Multi-view Novel View Synthesis. We evaluate the performance of existing methods in reconstructing the scene using multiple spherical views. The 100 available views are split into training and test data using an 80/20 ratio.
- Monocular Metric Depth Estimation. Since our dataset contains ground truth depth maps, we evaluated two recent monocular metric depth estimation methods without further fine-tuning them on our data.
- Single-shot Few-Image Scene Reconstruction. In our few-image-to-3D reconstruction task targeted for an automotive use-case, we deviated from existing comparisons by evaluating method performance on egocentric outward-facing views. Furthermore, we supervised the resulting novel views using 360° exocentric spherical views.
How was our data generated?
Both datasets presented in this paper were generated using our data generator, which provides an easy-to-use front-end for the CARLA Simulator. With our data generator, users can easily define parameters including the town, the initial position of the vehicle, the weather, the number of traffic participants, as well as the quantity and type of sensors and their respective positions (both ego and exocentric views).
The dataset provides RGB, depth maps, semantic and instance segmentation for each image. Furthermore, the 3D bounding boxes of each vehicle in the scene is also available.
Our data is generated in virtual towns
The generated data is sourced from Towns 1 to 7 and 10HD. Towns 1, and 3 to 7, as well as 10HD, were utilized for training purposes while all 100 scenes from Town 2 were reserved for testing.
Here, you can view examples from the various towns.
BibTeX
Please cite as follows:
@InProceedings{Kastingschafer_2025_WACV,
author = {K\"astingsch\"afer, Marius and Gieruc, Th\'eo and Bernhard, Sebastian and Campbell, Dylan and Insafutdinov, Eldar and Najafli, Eyvaz and Brox, Thomas},
title = {SEED4D: A Synthetic Ego-Exo Dynamic 4D Data Generator Driving Dataset and Benchmark},
booktitle = {Proceedings of the Winter Conference on Applications of Computer Vision (WACV)},
month = {February},
year = {2025},
pages = {7741-7753} }
Do you want to know more?
*If the contact form does not load, please check the advanced cookie settings and activate the functional cookies for the purpose of contact management.