Subscribe to download
Mars

A photogrammetry scene of the Mars landscape constructed from photos taken by Curiosity Rover. These photos are publicly available on NASA's website: http://mars.nasa.gov/msl/multimedia/raw/ .

Some notes on the scene's production:

First I downloaded a load of raw MSL NAVCAM images, then cropped off the dark edges using an ImageMagick command-line tool. Next I added them to the standard version of Agisoft PhotoScan, manually masking out areas where bits of the rover appeared (tedious over ~1000 images, but only took an hour or so) – I imagine I could have automated this somehow using provided metadata about general camera direction etc.

For camera calibration, it was pretty much all automatic when aligning all the images, beyond having to manually assign images from the left and right cameras to different lenses in PhotoScan. (The two NAVCAM cameras used definitely aren’t the same – the ‘centre’ for each is quite different. I expect JPL to have gloriously accurate measurements of precisely how each camera responds on these instruments?) Things aligned much better after that.

I used the simple pair pre-selection option in PhotoScan to get it to align stuff much faster (there being stereographic pairs for everything helped enormously there) – but beyond that small hint that there might be such pairs present, PhotoScan had no idea about the initial positioning or angles of any of the images. Some didn’t align (usually shots taken during a drive without a corresponding full panorama) but the vast majority did. For the final version, It took the best part of a weekend of processing to align images then build the (ultra-quality) point cloud on my quad-core i7 laptop; the final arbitrary mesh needed a 64GB-memory machine here at Valve to build. There’s some extrapolation enabled to fill in holes in the mesh, but not much.

Colours were added in Photoshop (I generated UVs in PhotoScan, for ten 8k textures exported as PNG) – I used some colour images from the rover as a simple reference. There was a bit of cleanup in Photoshop on areas which were overly bright or dark (the contrast-stretched raw images aren’t ideal for PhotoScan’s exposure adjustment) but edits were only of a large-scale nature. Blurry and glitchy sections of the texture all stayed in.

Terrain in the middle distance was generated in PhotoScan from the same dataset, this time as a low-poly heightfield-style mesh with one texture. That got the middle chopped out and the edges trimmed – there’s an alpha-mask on that texture so I could paint out particularly bad sections around the edge. Using a proper HiRISE-derived terrain model could probably work much better – PhotoScan can project on to provided meshes, so the shape could have come from orbit while the pixels come from the ground.

The crater rim and Mount Sharp are all a bit artificial – sections of panorama from different cameras chopped up in Photoshop with alpha masks painted on, then stuck on to cylindrical chunks of billboard geometry. I used various full panoramas as reference for the size and positioning of the mountain. Very unscientific. The sky and clouds themselves are fully procedural in Modo, rendered to a skydome – artistically based on some shots of Martian clouds taken by Curiosity. It’s where the ‘game level designer’ aspect really comes in – convince the player into believing the lighting and the surroundings, and they can really feel like they’re somewhere else. An early version of the scene with black-and-white foreground geometry and a blank sky was interesting on an intellectual level, but nowhere near as visceral.

Scale for the rover was obtained by getting a number from Wikipedia for its height – and for the terrain by using the scale bar on some MAHLI terrain imagery, and matching physical features between that imagery and the model. I was really happy when the wheels of the rover model fitted perfectly into tracks through the Martian dirt – confirmation that I was somewhere in the right area. Oddly enough, I’m a bit unsure as to the precise direction of ‘up’ – I should probably use a HiRISE-derived terrain model to calibrate that.

So yes, definitely game-style content rather than a scientific product, but it was fun to build as a quick side-project – and it’s really opened our eyes as to some of the non-game possibilities of VR. Being able to visit a location that’s real but otherwise difficult or impossible to access is quite compelling – and being able to walk through that same location hugely helps both in the sense of place and in its understanding.