The oceanic environment is the most mysterious for scientific research as well as industrial applications. In this article, we discuss a new approach for underwater photogrammetry, which is extremely difficult due to inconsistent lighting conditions, change of refractive index (different medium), and color loss from the limited penetration distance of certain wavelengths…
In recent years, photogrammetry has extended from aerial to terrestrial, and now to underwater. Using photogrammetry under water is still in an exploratory phase, but the need already exists for certain industries, such as environmental monitoring, archaeology, forensics, and infrastructure inspection.
With a measurable 3D model or 2D map of a large area, ship or plane wrecks can be documented for scene reconstruction; bridge piers can be inspected for maintenance and repair; ancient cultural heritage can be mapped and archived for further research; focused regions can be repetitively monitored for detecting environmental change; and more.
Distance-based Image Acquisition
Common in most underwater projects, images were triggered by time-lapse from a camera moving above or around the object of interest. However, this creates a lot of problems for post-processing. Unlike above-ground, moving speed underwater cannot be estimated precisely. If the images are time-based, the divers will take too many useless images while swimming against the current and not enough images when moving with it. Such inconsistent image overlap either requires a lot of manual work to pick out the unnecessary images or risks messing up the calibration due to short baselines or disconnection of adjacent images.
The new acquisition method, developed by Geolab, requires a team of professional divers. For their offshore projects, there were at least four professional divers (including one boat operator and one or two photographers) along with one project coordinator, each of the team members playing an important role in the image acquisition process.
It allows the divers to know at what position each image has to be taken instead of taking images randomly based on time-lapse. The divers manually trigger the camera at specific spots based on the distance to the object and to the previous images. This results in a much more regular image acquisition plans, easier and better 3D reconstruction while optimizing the precious time spent underwater.
However, mapping underwater is still a lot trickier than doing the same on land. The light does not travel in water the same way as in air, due to different light refraction rates, and small particles suspended in water can strongly affect the visibility under water. The divers need to know the basics of photogrammetry and to adapt the acquisition procedure underwater depending on the visibility conditions they witness on the spot.
For mapping professionals and surveyors, accuracy is a requirement. At the current moment, a majority of the underwater mapping projects are still in experimental phase, giving the acquired images a try to see if 3D could be reconstructed. Since an optimal way to capture images in the optimal plans for mapping has been developed, the next step is to ensure the 3D reconstruction was accurate and to be able to geolocate their projects in reference to a global coordinate system.
- Scales and Measurements
Once making sure Pix4D’s software could correctly calibrate the camera, it is important to scale the projects to their true dimensions. Based upon a common scaling method used in Pix4Dmapper’s rayCloud, rulers or customized scale bars were placed in the mapping area and their measurement was assigned in Pix4Dmapper to scale the entire projects.
These scaling tools should preferably be as long as possible and placed perpendicular to each other to decrease the relative length error and the possible error in one direction. To verify the 3D reconstruction is precise, extra scale bars can be placed on site. They can be used to assess the software-computed measurements after the scaling step, match their actual dimensions. This verification can ensure accuracy of any 2D or 3D measurement in the projects, which is essential for any mapping work. One can easily understand the need for accurate shipwreck size, area of coral reef coverage, length of cracks on a bridge pier, etc.
- Tags, GCPs, and Geolocation
Similar to aerial mapping practices, some metal tags were also placed on site, well distributed over the surveyed area. Yet not compulsory, they help the software in finding and matching identical keypoints between images and can be used as Ground Control Points (GCPs) for georeferencing the project with a specific underwater and on-board the boat procedure, not detailed here.
- Limits and Visual Aspects
In shallow water, say within 3 meters of depth, a common issue is the change of light patterns illuminating the mapped area between images. This can happen when the direct rays of sunlight hit the surface of water. Surface waves create numerous concave and convex lenses which reflect and refract the light in a very complex and unpredictable way. The content of images depicting the same object can be very different from one picture to another, resulting in difficult photogrammetry processing.
This explains why for underwater mapping projects, the ones in shallow water regions can be extremely difficult. However, going deeper does not necessarily mean getting away with light problems. When light travels in water, it is absorbed, refracted, or reflected, and each wavelength composing it behaves differently, leading to different traveling distance underwater.
Here are graphs showing the theoretical distance light penetrates underwater according to its wavelengths and the loss of color due to this physics fact. From the figure, we can see that the underwater environment gets darker proportional to the distance from the surface. Even though the short wavelengths possess larger energy and penetrate longer, they are still refracted and absorbed. Over the depth of 40 meters, only blue light is visible and the loss of light is rapid.
- Color Adjustment
For projects performed in the depth between 5 to 25 meters underwater, by setting higher ISO values and increasing the exposure time of their cameras when needed, they could capture sufficient light, but it resulted in sacrificed image quality. This is unfortunately very common for underwater mapping and more widely for underwater photography. To mitigate this matter or if the projects go beyond 30 to 40 meters, artificial lighting would be needed: but again, ensuring uniform lighting over the mapped area would represent another challenge to deal with.
As all the wavelengths of light do not penetrate equally in water, the visual impact on the captured images is strong. At medium depth, some specific filters can be placed in front of the camera to help capture more of specific wavelengths, but at higher depths some colors are no longer present and the only way to bring them back is again with artificial light.
The captured images can nevertheless be color corrected in an image editing software before the photogrammetry process. To some extent, this can help to restore more natural colors while keeping the continuity of color intensity. This is not a mandatory step, since the original images can also give accurate results using Pix4D’s software, however, it certainly gives a more visually-pleasant outcome, which can be very useful for some applications such as coral reef monitoring.
Thanks to the recent development of Pix4D’s photogrammetry software and Geolab’s data acquisition techniques, accurate underwater mapping is now possible, even though both companies are still advancing their solutions. In the near future, Geolab plans to upgrade their camera setup and ultimately adapt their acquisition method to remotely operated vehicles (ROVs), while Pix4D will improve its current workflow for processing underwater images.
With the pre-calibrated Pix4D camera database, Pix4Dmapper is capable of reconstructing 3D within a very short processing time. However, since the key camera interiors-focal length changes significantly under water, which breaks the assumptions for general aerial and terrestrial mapping cases, more adaptable processing options will be needed. Pix4D and Geolab will collaborate on future projects and assist each other’s progress, extending the advancement beneath the water surface.
Author: Lisa Chen (Pix4D), Romain Pinel (Geolab: www.geolab.re)