Over the last few years, more and more high quality imaging sensors have become available at lower and lower prices. At the same time, people have access to software which can handle such sensors efficiently and automatically build 3D models purely from images.

It is realitivly simple for even a novice to reconstruct simple geometries in 3D. But more complex geometry still poses a challenge.

Mapping more efficiently

Efficient mapping is answering the question: “how many image locations are needed to model a complex structure (for instance a castle) with all its geometrically complex structures and inside all rooms.”

The answer is different when considering cameras and drones compared to traditional laser scanning.

With laser scanning, a single scan can measure a full circle of 306 degrees, as long as there are no obstructions.

However, mapping purely from images is a little more complicated.

Image based 3D reconstructions need at least two images. Based on two images, one can compute a 3D point only for the structures visible in both images. The amount of a 3D structure that can be modelled with two images also depends on the lens: a fisheye lens covering 180 degree of the scene can map a much larger area than perspective images that depict only a small area.

As we planned to map the entire castle in a single day, these questions were vitally important: how could we map most efficiently while still capturing a full reconstruction?

Capture the castle

The Chillon castle (also known as Chateau de Chillon) is an island castle located on the shore of Lake Geneva, next to city of Montreux. The castle consists of over 21 independent buildings that over the centuries were gradually connected to become Switzerland’s most visited historic monument.

The oldest parts of the castle have not been definitively dated, but the first written record of the castle is in 1160 or 1005. The rocky island on which the castle is built was both a natural protection and a strategic location to control the passage between northern and southern Europe. From the mid 12th century, the castle was home to the Counts of Savoy, and it was greatly expanded in the 13th century by Pietro II. The Castle was never taken in a siege, but did change hands through treaties.

It was made popular by Lord Byron, who wrote the poem “The Prisoner Of Chillon” in 1816 and who also carved his name on a pillar of the dungeon. The castle has inspired many artists and writers, from Jean-Jacques Rousseau, Victor Hugo and Henry James to Delacroix and Courbet.

In its current state, the Chillon Castle is the result of several centuries of constant building, adaptations, renovations and restorations.

Surveying equipment

Our challenge was to map the castle using only consumer-grade hardware. Because the data was obtained by several people in a single day, equipment also had to be light enough to carry comfortably. The following surveying equipment was used to capture the images.

GNSS Data	A Trimble R 10 GNSS receiver was used in RTK-mode using a virtual reference station for measuring 11 Ground Control Points (GCPs) twice in different satellite constellations. The mean differences between the two measurements were 1 cm in x and y and 2 cm in z. Some of the GCPs were visible in the image data and used in the 3D reconstruction and some of them were used to define the tachymetric network as described below.
Tachymeter data	For surveying the GCPs measurable in the terrestrial and aerial pictures, a tachymetric network was measured with a Trimble 5601 total station from the land side.
Canon 6D 8mm Sigma lens	The Canon 6D has a 20.2MP full frame CMOS sensor. The images in this experiment have been captured from various positions with the camera mounted on a tripod. The Sigma 8mm lens produces 180 degree imagry.
Sony alpha 7r with 8mm lens	The Sony alpha 7r has a full frame CMOS senosor. In conjunction with the 8mm Rokinon fisheye lens, it captures APS-C size images with 4800x3200 pixels. All images have been captured on a tripod mount.
GoPro Hero3+ Black Edition	GoPro designed this camera as an action camera. It is a wide-spread camera which is well suited for use on a UAV due to its light weight of just 100g. The GoPro captures images at a resolution of 4000x3000 pixyels and comes with a wide angle fisheye lens. Images can be triggered through a mobile phone connected to WiFi. There is a time lapse capture mode with various time intervals starting from 0.5 seconds. The images captured in this project used the hand held interval capturing mode of 0.5 seconds.
DJI Phantom 2 Vision	The DJI Phantom 2 Vision is a remotely controlled quadro-copter. It is fitted with a 4000x3000 pixel camera that can be triggered from a mobile phone or an interval mode starting from 1 image/second. Similar to the GoPro, it has a wide fisheye lens. The images in this project were captured using the Phantom Vision 2 in remote controlled flight mode with an image acquistion interval of 1 second.

Being a cultural heritage site, no surveying marks were allowed. We used distinctive natural points of the buildings for control points instead.

Most of the points were measured from different viewpoints in both instrument faces; some as a spatial intersection without distance measurement (for example, most of the spheroids on the top of the roofs) and some with reflector-less distance measurement. The least square adjustment took into account the errors of the GNSS-points, also refining them, and led internal to mean standard deviations of 8, 7 and 6 mm in x, y and z.

The images were captured with a very high overlap (80-90%), which is not difficult to achieve considering the use of many wide angle or even fisheye lenses. More care and overlap is needed to connect the outdoor terrestrial images with the indoor images of the same camera. Since we do not have ground control points for the indoor parts, these connections needed to be very strong - that is, have a very high overlap.

Capturing the images

To capture the images, the Pix4D team split into groups and were each assigned to a different part of the structure. More than 6,000 images were captured over the day.

1,885 images were captured from the castle courtyards and inside rooms. More overlap was needed when moving from the outside courtyards through the open doors towards the inside. Geo-referencing is possible from the tachymetric measurements of the towers that are visible from the individual courtyards.

The 1,933 images taken from a tripod connect the inside rooms with two of the courtyards. The reconstruction is geo-referenced by the GCPs visible in the courtyards.

725 nadir and oblique images have been captured from a custom made quadro-copter with a GoPro gimble mount.

712 images have been capture from the courtyards with the DJI Phantom 2 Vision.

469 images have been captured from the lake side (with the DJI operated from a boat) and from the front side of the castle.

Reconstructing the 3D model in Pix4Dmapper

The model was reconstruced in Pix4Dmapper software. Pix4Dmapper allows you to create projects by merging already computed subprojects. This has mainly two advantages:

• Speed: Efficient and fast processing for smaller sized sub projects
• Quality: Separate quality control

In the case of Chillon, the data of the various cameras and sources has been processed independently before combining projects. Each sub project has been geo-referenced with the same ground control points when visible and the quality and accuracy had been ensured.

After this step we merged the different sub projects into larger projects until – finally- all images have been put together. At each step of the merging process, a separate bundle adjustment was performed (“Re-optimize” option in rayCloud Editor). The following sections give more detail in some of the sub projects.

Ensuring accuracy: getting great results with a complex project

The final reconstruction including all images was obtained by creating a new project and merging with existing projects. When merging projects one can use the following cues to bring the individual projects into the same coordinate system by a rigid 7 parameter transformation:

• Overlapping images: if two projects contain at least three of the same images, they will be used to find the rigid transformation that merges both reconstructions together.
• Ground Control Points: if the individual reconstructions are geo-referenced by GCP’s there is no need to compute a rigid transformation as the merged project will be the combination of the individual projects.
• Manual tie points: if the individual projects contain manual tie points with the same name, then the corresponding triangulated 3D points are used to compute the rigid transformation that converts the individual reconstructions to the same coordinate system.

When merging projects, image features and matches are not recomputed in order to make the merging process efficient a shows the final accuracy of all Ground Control Points after Bundle Block Adjustment of the complete project.

After the initial processing (first step), the external and internal camera parameters of all images are known and will be used for the point cloud densification.

Chillon is captured

In four hours, the thousand year-old structure was mapped. More than 6,000 images were captured, and a 3D model was produced with an accuracy up to 5mm.
As well as proving the value of Pix4Dmapper for architectural projects, the map and model of Castle Chillon is an important artifact which can be explored by people all over the world.