Some places just deserve to be exported in a nice high-quality export so you can properly examine all the details of the map.

The City of Sanctuary from the community atlas is certainly one of these places. This lovely city by master mapper Christina Trani and resident artist Sue Daniel is a visual masterwork of a city map. So, I decided to try to make a proper high resolution export of this map. The result is a map 200,000 by 200,000 pixels, or 40 gigapixels in size. The .png file for this map is over 12GB in size, and even though I have a powerful computer, actually opening this source image in an image editor is almost impossible without it crashing.

Of course, providing such an image for download is useless, since nobody can actually use it, but a far better approach is to provide it as a zoomable image on the atlas website. The image viewer used for such a zoomable image loads different images depending on your zoom level, ensure that you only load what you need at any one time, making it reasonably easy to explore this gigantic export with reasonable loading times as you zoom or scroll. You can explore this zoomable image yourself by going to the City of Sanctuary map page, and then click the Zoomable Image button located over the map image. It should load reasonably quickly, but your physical distance from my server would affect loading times. I recommend you click the fullscreen button in the toolbar at the bottom of the zoomable image viewer for a much more impressive experience.

So, how did I accomplish this feat? Well, let me start by underlining that this was not a simple one-click task, it did require quite some work and a lot of computer time. Campaign Cartographer 3+ cannot export a map of this size directly, as you all probably know already. Once you start going past 10,000 by 10,000 pixels, you start to struggle. The exact max size depends on things like the complexity of the map, and I can promise you that this map scores very high on that complexity scale. Fortunately, CC3+ can export individual parts of the maps, something I took full advantage of when I wrote the scripts for the Large Exports annual issue.

This gargantuan export is mostly done following the steps in the aforementioned annual, but the sheer size of this export provided some unique challenges. I expect many people aren’t familiar with the annual either, so I’ll talk a bit about the steps as well.

Preparing the Map

To solve the problem about a limited export size in CC3+, we export the map in pieces using the Rectangular Section PNG export option which allows us to export the map in tiles, where each tile have the same individual export limitation that exporting the whole map has, so by exporting the map in tiles and stitching them together we can arrive at a much larger image than CC3+ can produce directly.

However, we need to consider the effects in the map. In CC3+, an effect will only appear on an exported tile if the entity creating that effect is withing the same tile. For example, a shadow from a tall column will only be visible in the same tile as the column, even if it is supposed to extend into the area of the neighboring tile. This may seem like a stupid limitation on first glance, but the problem here is available resources. If all effects from all entities outside the current til should be included, CC3+ would basically have to render the entire map to be able to compute all the effects to figure out what would have an impact on the tile, and if CC3+ was able to do that, we wouldn’t have needed to export in tiles in the first place. The effects mostly affected by this are effects that extends far beyond the border of the entities that create them. The main effects affected are shadows, glows and edge fades.

So, for this map, there where a few things that caused problems. The sea contours are very soft, created by using large edge fades, and there are some large dark shadows (also made by solid entities with a large edge fade) across parts of the map that serves the purpose of varying the light levels in the map. Both these are hugely problematic  for the tiled export, so I had to decrease the size of the edge fade for the sea contours, as well as remove the black shadows. While the map do look prettier with both these in their original state, the map still looks good with these changes, and it makes a huge change for my export. You can see the example image to the right how the sea contour looks if exporting without these modifications, I believe horrible is a good way of describing those harsh straight cutoffs.

Note that the smaller tiles you are making, the more such issues you are going to see. To be able to create this gigantic export, I actually had to divide the maps into 40 tiles in each direction, for a grand total of 1,600 tiles, all 5,000×5,000 pixels in size.

If you look closely on the zoomable image, you are going to see a few places where effects issues like this are visible, especially around the city wall, but it is mostly minor flaws.

Exporting the Tiles

As mentioned earlier, I used the scripts form the Large Exports annual issue to manage this. The scripts works quite nicely, but there are a few issues when you try to tell CC3+ to go ahead and export 1,600 images in one go. The chance of it crashing in the middle somewhere is actually quite high, and even if it didn’t crash, some tiles might have export errors. So, what I did was to slightly modify the script so that I could tell CC3+ how many tiles to skip, so I could start the process wherever it stopped.

Some of these crashes was for unknown reasons (possibly a small memory leak, or resource shortage), but other crashes where very consistent. For example, I found a building in the map what would always crash CC3+ when I zoomed to close to it (closer than anyone would normally ever zoom in the map), and exporting in really high resolution is for the internals of CC3+ much the same as zooming in really close. To resolve these crashes, I deleted the original building, and redrew it to look exactly like the original.

Other problems that occurred was that sometimes CC3+ failed to load one of the images in the tile, thus ending up with a red X in the tile, and I just had to export that tile again and everything would be fine. All in all, very few tiles where effected by this, but the ability to start the export at a given tile to re-export these was extremely important.

Exporting these tiles also took a long time. Exporting all 1600 tiles had my computer working continuously for over 2 days. Considering I made a few silly mistakes (like changing the effects midway through when I resumed from a crash) that caused me to start the entire process over several times my computer have been working more or less continuously on this for more than a week.

I also noticed an error in the final result that I didn’t notice earlier because I inspected the tiles one by one; some of the larger texts have been cut of (such as the map title itself). I’ll probably fix this some time, but it will actually involve re-doing a lot of the time consuming work, and since it don’t affect the actual map, I’ll just let it be for now.

Stitching the Tiles Together

Actually stitching the 1,600 tiles together is no easy tasks either. I use ImageMagick for this, and for smaller images, you can just give it the list of images and tell it how they are tiled (40 by 40 in my case), but with this monstrosity that wouldn’t do. The files themselves are pretty large, and the result even more so. ImageMagick really struggled when trying to combine all the 1,600 images in one go, so in the end I had to resort to first having it create 40 rows by combining all the 40 images on each row, and then combining all the rows to one image. But this also required several hundred gigabytes of available storage, both for storing the tiles and the rows, but also for temporary working space ImageMagick required when processing these images. Just this process alone also took quite a few hours.

The final image was a .png file (it had to be .png, as .jpg doesn’t support images with this high a resolution) of over 12 gigabytes in size. And not only was it large, but none of my image editors could even open it, windows explorer couldn’t even list out the dimensions of the image. The individual rows (200,000 by 5,000 pixels) can be worked with just fine, but I haven’t actually been able to open the final image (200,000 by 200,000 pixels).

The Zoomable Image

The final part of the puzzle was to create the zoomable image. For this, I use Zoomify. They have a free version of their product, and it comes with a converter that can automatically take a large image and break it down into a tiles which are suitable for a zoomable image. These tiles comes in different resolutions, so for example when you look at the entire map, you are served a lower resolution version of the whole thing which is quick to load, and as you zoom in, you are given higher resolution versions, but only of the tiles actually in your viewport, so you don’t have to load the full resolution of the entire image. Zoomify requires a javascript on the web page, but since this was for the community atlas, I’ve already added the scripts there a long time ago to support the other zoomable images up there.

Enjoy the result. I think it came out pretty nice, and well worth the effort (and power bill).