Improved accuracy for rolling shutter cameras

Images taken with rolling shutter cameras, for example, contain a vertical displacement when there is a relative movement between cameras and objects during image acquisition. Pix4Dmapper has introduced a model to correct such displacement, enabling professionals to obtain better accuracy with rolling shutter cameras.

Global shutter and rolling shutter

Many DSLR cameras, consumer drone cameras, and phone cameras use a rolling shutter in order to take better quality videos and to reduce motion blur. Major players in the drone market, DJI drones and GoPro cameras for example, have a rolling shutter embedded. The ease of use and reasonable price of these drones and cameras enable more and more people to use them for mapping work.

This animation illustrates how the two main types of shutters work. Global Shutter (left) cameras record the entire image frame at nearly the exact same time. This is commonly seen in CCD sensor cameras. Rolling Shutter (right) cameras record each image frame line by line, from the top to the bottom. This type of shutter now occupies a majority of the camera market.

Rolling Shutter Effect

Rolling shutter effect occurs when the camera-object relation has already changed during the line-by-line readout of the same image. When using these distorted images in photogrammetry software, the pixel displacement may cause problems for feature matching and thus inaccurate camera parameters. Inaccurately computed camera parameters will certainly result in poor quality mapping accuracies. Here we focus specifically on consumer drone cameras and action cameras because most DSLR cameras have a mechanical shutter implemented which enables users to set a short exposure time. In this case, the effect would be so little that can be neglected.

In Version 2.1, Pix4Dmapper Pro has implemented a rolling shutter correction model which corrects this displacement and thus better estimates camera parameters When to Correct Under what conditions should this correction be applied to achieve a higher mapping accuracy? When the vertical displacement is larger than 2 pixels, we recommend using our rolling shutter correction model. Displacement can be calculated using the simple formula below:

Where Δ (pixels) represents the vertical displacement, V (meters/second) represents drone speed, T (seconds) represents the readout time of the sensor, f (millimeters) the focal length, P (millimeters/pixel) the vertical pixel size, and h (meters) the flying height, or the distance between the sensor and the object.

For example, a DJI Phantom 2 Vision + with the readout time of 74 milliseconds, flying at a speed of 8 m/s (18 mph) and at 70 meters above ground will have a vertical displacement of 10 pixels due to the rolling shutter effect. The drone has moved almost 60 cm during the image readout.

We provide the readout time of some widely-used rolling shutter cameras below:

How the Solution Works

Most photogrammetry software process based on the assumption that there is only one camera position during single image readout. The rolling shutter correction model in Pix4Dmapper takes the movement of camera positions into account.

These different camera positions for each row are approximated by applying a linear interpolation between the two camera positions at the start and finish of the image readout. This linear approximation has been shown to work especially well with quadrotor gimbal drones that stabilize the camera to always face nadir. With Pix4Dmapper’s rolling shutter correction it is now possible to fly your drone at a faster speed without compromising the result accuracy. Since many consumer drones are equipped with rolling shutter cameras, this is a true enabler of professional mapping with consumer type drones.

Quality Assessment

After applying the rolling shutter correction model, the Phantom 4 and Inpire 1 X5 camera parameters as well as GCP locations were accurately computed. The following table shows a proved accuracy improvement by assessing 83 validation points from processed Phantom 4 and Inspire 1 – X5 image data sets, with and without rolling shutter correction.

Download the Quality Reports