Camera
We used an off-the-shelf ZED 2i Stereo Camera (lens focal length 2.1 mm, no polariser), which has a built-in IMU and magnetometer.
On-road data collection setup
The camera was placed behind the windshield of a car and connected to a laptop installed with the ZED software development kit (SDK) (v4.0.6) and the ZED Robot Operating System (ROS) wrapper (v4.0.5). Considering the way the windshield inclines and that the mounting holes are located at the bottom of the camera but not at its top, it was mounted upside down for convenience and safety reasons.
We accordingly set the camera_flip parameter to true to allow the SDK to account for this upside-down setup to generate the correct stereo images.
The left frames of these images, whose poses are estimated by a SLAM algorithm in our method and to which the 3D positions of all landmarks are referenced, were actually acquired by the right camera because of this upside-down setup.
Therefore, our photometric calibration was later performed on the right camera.
We disabled the camera’s auto-white balance and auto-exposure functionalities. Instead, we used a fixed white balance, and manually adjusted the exposure time, in conjunction with the gain, according to the lighting condition of the scene. We took note of the combination of exposure time and gain used to collect each data sequence (eight different combinations in total). Later we performed a photometric calibration for each combination of these two parameters.
Additionally, a number of parameters which control various image processing algorithms applied to the RAW images by the onboard image signal processor are also configurable. We fixed these parameters throughout our on-road data collection and photometric calibration. The following table shows a full list of camera settings we used. For the description and the valid value range for some of the settings, please refer to the StereoLabs Docs.
| Setting | Value/Configuration | |
|---|---|---|
| On-road data collection | Photometric calibration | |
| Camera flip | True | False |
| Resolution | 1920 × 1080 | |
| Frame rate | 15 Hz | N/A |
| White balance | 4700 | |
| Gain | Sequence dependent | Same as the on-road data collection |
| Exposure | ||
| Brightness | 4 | |
| Contrast | 4 | |
| Hue | 0 | |
| Saturation | 4 | |
| Sharpness | 4 | |
| Gamma | 5 | |
| Image enhancement | Disabled | |
Data format
During the collection, the data, together with the timestamps, was logged to ROS bags, which were later parsed to generate the following files (see the Overview page for the folder structure).
- Rectified left and right images at a frame rate of 15 Hz saved as PNG files, each at a resolution of 1920×580.
- IMU data at a rate of 400 Hz saved as CSV files.
- Magnetometer data at a rate of 50 Hz saved as CSV files.
Note that
- The stereo rectification was performed by the ZED SDK using its factory calibrated stereo parameters.
- The original image size was 1920×1080. The images were later cropped by 290 pixels at the top (to remove the mostly-sky region) and by 210 pixels at the bottom (to remove the car’s bonnet and interior reflections caused by the windshield). See the image below for an example image of the left frame before and after the cropping (outlined in red). The final image size after cropping has an aspect ratio of 1920/580 ≈ 3.31, which is very close to the aspect ratio (1241/376 ≈ 3.30) of the images in the KITTI odometry benchmark.
Camera intrinsic parameters
The intrinsic parameters of the camera (after stereo rectification) are specified in our GitHub repo. Note that these parameters have already taken into account of the upside-down setup of our on-road data collection as well as our cropping, and therefore can be directly used for the images in SDIRF.