Merging aerial and terrestrial data with Pix4D

Photogrammetric surveying faces a major challenge when it comes to vegetation; it is perhaps the main challenge, according to Reto Weiss. Weiss is owner and managing director of pixmap, a small company near Bern, Switzerland specializing in drone services. He explains that traditional nadir images alone are often insufficient for capturing vertical surfaces like buildings and their surroundings effectively. To overcome this limitation, they are typically supplemented with oblique images, usually taken at angles of 45° or steeper, often with the use of drones. However, the presence of tall trees or extensive canopies nearby can obstruct the drones' line of sight when capturing images.

In these cases, additional imagery from the ground becomes crucial. That's where PIX4Dcatch comes into play. We had a conversation with Reto Weiss to understand how he used the PIX4Dcatch RTK workflow to merge terrestrial and aerial data, offering an efficient solution to the challenge posed by obstructive vegetation in photogrammetric surveying.

Can you describe the test project “Schützenhaus”?

The Schützenhaus test project was about testing the interaction of drone imagery and terrestrial footage captured with PIX4Dcatch. It’s near pixmap’s headquarters, so the ideal location to test the PIX4Dcatch RTK workflow. We wanted to see if we could achieve accurate results despite visibility challenges due to vegetation.

Aerial view of the Schützenhaus test project, with lime trees to the south

The building is part of a rifle range, about 30 meters long, with the south façade lined and almost hidden by four large lime trees. So capturing this building in 3D is not possible using just aerial photogrammetry and drones. What is needed is a combination of aerial and terrestrial imagery in order to fully capture the building and create a 3D model.

We decided to use a Mavic 3 Enterprise for aerial capture and an iPhone 14 Pro with PIX4Dcatch for terrestrial capture. Both were equipped with GNSS/RTK for exact geo-referencing. For photogrammetric processing, we used PIX4Dmatic, which explicitly supports the merging of aerial and terrestrial data. Since the test project worked well, we decided to use the same setup for a commercial project.

What is “Project Villa”?

We often survey buildings that are going to be renovated or developed. This involves capturing the building(s) and their surroundings in 3D to create a digital twin. From the 3D model (point cloud or mesh), individual building parts are often vectorized later. Often the 3D model is also made available in the cloud (PIX4Dcloud). For these sorts of projects, in addition to PIX4Dcatch and PIX4Dmatic, we also use PIX4Dcloud Advanced and PIX4Dsurvey (for vectorizing).

Project Villa is a large detached house near Bern that is to be heavily remodeled. It has a large garden with a sloping terrain in the south, and to the west and north of the house, it is surrounded by tall trees. We could immediately tell that it would be challenging to comprehensively capture this villa photogrammetrically.

Project Villa with trees lining the west and north

Because of the positive results from the test project Schützenhaus, we decided to use the Mavic 3 Enterprise (M3E). It was clear to us that we would also need to use the PIX4Dcatch RTK workflow, in order to capture otherwise obscured, important parts of the villa.

Did you run into any difficulties?

A big challenge was RTK reception. In the areas you cannot capture from the air, where PIX4Dcatch is used, the GPS reception is usually poor*. This was the case in both the test project and Project Villa. I would start scanning with PIX4Dcatch, and if there was a temporary interruption, I would then scan in the open and move until I had RTK reception again.

Another challenge was that the point clouds were not always aligned, as a consequence of the very different perspectives and RTK inaccuracies. Although this was not critical in the test project, it became more obvious in Project Villa. We realized it was essential to have common GCP or tie points that are visible from the air as well as the ground. With enough tie points and some manual effort, the point clouds can then be aligned quite well to increase accuracy and avoid visual transitions.

*(PIX4Dcatch has since been updated with the GeoFusion algorithm, which works to improve the accuracy of RTK data in areas with low GNSS signal or reception).

Project details

How did you merge the aerial and terrestrial data?

For the drone shots, we used a typical nadir and a 45° oblique flight plan, with high overlap, resulting in 750 images. The flight took just under a quarter of an hour. In addition, around 400 pictures were taken with a manually controlled drone, from 2-10 meters above ground, wherever, for example, canopies covered parts of the house or the terrain around it.

Upper cameras on one flight level = programmed mission. Lower cameras = manually flown trajectories

In addition, there were a total of four PIX4Dcatch scans which were recorded as individual projects.

PIX4Dcatch project #1: 411 images on a path around parts of the villa, back and forth

All in all, 1600 images were taken from all the missions and processed in a PIX4Dmatic project. In addition to the GCP targets, which were laid out in advance and measured using GNSS equipment, additional tie points had to be determined during processing to improve the alignment of the terrestrial images. Thanks to these tie points, we achieved a high-quality point cloud that, while not flawless, met the customer’s requirements and delivered an outcome (merging terrestrial and aerial data) that was not possible before!

I felt that the combination of data from the aerial and terrestrial acquisition was quite complex, and required some experience, as at least two, but mostly many, point clouds are created which must be precisely aligned in the end to obtain a consistent whole. It took the team 4 hours in the field and one day for processing.

What did you learn from this project?

Besides successfully completing the commercial "Project Villa”, following on from our test Schützenhaus, we learned a lot using the PIX4Dcatch RTK workflow:

• Automatic flight plans form a good basis for 3D-capturing;
• Manual drone surveys are often a necessary addition, however, automated waypoint flight would be desirable for this as well;
• PIX4Dcatch allows the filling of specific gaps and capturing of parts that are otherwise hardly visible;
• PIX4Dmatic works well with a mix of perspectives and cameras;
• Thorough planning, execution, and meticulous processing in such a project is of utmost importance.

With a decade of expertise in drone-based photogrammetry and access to state-of-the-art hardware and software, we demonstrated the capability to effectively execute these projects with the PIX4Dcatch RTK workflow on a commercial scale.

“The PIX4Dcatch RTK workflow and PIX4Dmatic make it possible to capture complex objects by combining drone captures with terrestrial imagery." - Reto Weiss, pixmap