Before we build VPB from the source code and use it for terrain creation later, we should establish some prerequisites. One is to install OSG headers and libraries at a reachable location. Of course every reader of this book should be able to achieve this.

The other requirement is that you must have the GDAL library installed, which will be used heavily in VPB for reading and parsing original raster data. You may download the source code and build it from the source code too. But GDAL has already provided various downloadable binaries for different platforms and versions. Linux, Mac OS X, and Windows users please see the download link for details:

http://trac.osgeo.org/gdal/wiki/DownloadingGdalBinaries

And Debian and Ubuntu developers can also make use of the common apt-get command to obtain GDAL binaries and developer files:

# apt-get install gdal-bin
# apt-get install libgdal-dev

Let us start.

Most OSG-based projects, including VPB and some other projects introduced before (osgOcean and so on), use CMake as the building system. So it is important for them to find various OSG libraries as dependencies. CMake provides an automatic searching script which can look for OSG installations under /usr and /usr/bin directories, as well as the place indicated by the environment variable OSG_DIR. The CMake system will then try to find OSG's necessary header files in the include subdirectory of each folder specified in OSG_DIR, and library files in the lib subdirectory. If successful, it presets these locations as the default values before the user-configuration process. This, of course, brings convenience when there are too many options to set for the same dependency.

There is a similar solution for specifying GDAL options in CMake, but it uses another environment variable GDAL_DIR instead, which indicates where GDAL binaries and libraries are installed.

The Large Geometric Models Archive project at Georgia Tech has some excellent terrain data that can be used here to show how VPB works with original geographic data. The project site is managed by Greg Turk and Brendan Mullins. You can visit it at http://www.cc.gatech.edu/projects/large_models/.

And we are mainly interested in the Grand Canyon data, which can be found at http://www.cc.gatech.edu/projects/large_models/gcanyon.html.

Download the BMP format of the elevation and texture maps. We will use them along with the vpbmaster tool soon.

Let us start.

Have a look at the generated directory. It includes a large number of files and folders with the same name infix—L[a]_X[b]_Y[c]. Here a means the level number, and b and c are range identifiers. Level 0 is the rough level, and level 6 in this example is the most detailed. In the following screenshot, there is only one L0 subfolder and several L2 subfolders. So what do they mean here?

VPB will always try to split the input image to some square sections, for instance, 4096 x 2048 (L0) will be separated into two 2048 x 2048 tiles. And they are named L1_X0_Y0 and L1_X1_Y0, which lie in the out_root_L0_X0_Y0 folder. The L1 sections will then be split again using the quad-tree structure, that is, each level's tile is divided into four pieces of the next level. So we can see in the output folder four L2 subfolders (X0_Y0 to X1_Y1) for L1_X0_Y0, and the other four (X2_Y0 to X3_Y1) for L1_X1_Y0. All child levels will be placed in these L2 subfolders.

Thanks to OSG's paged LOD (level-of-details) mechanism, the tile will only be replaced by its four sub-tiles when the viewer is near enough, and will render data of its range with a higher resolution. The basic structure of a quad-tree LOD in terrain rendering is shown in the following diagram:

And the node structure can be described as follows:

osg::Group* nextLvGroup;
...
nextLvGroup->addChild( pagedNextTile1 );
nextLvGroup->addChild( pagedNextTile2 );
nextLvGroup->addChild( pagedNextTile3 );
nextLvGroup->addChild( pagedNextTile4 );
osg::PagedLOD* pagedThisTile;
...
thisTile->addChild( dataOfThisLevel ); // The rough level
thisTile->addChild( nextLvGroup ); // The refined level

The nodes pagedNextTile1 to pagedNextTile4 are actually files with the prefix out_La'_Xb'_Yc'. In this case:

a' = a + 1
For tile1: b' = 2 * b, c' = 2 * c
For tile2: b' = 2 * b + 1, c' = 2 * c
For tile3: b' = 2 * b, c' = 2 * c + 1
For tile4: b' = 2 * b + 1, c' = 2 * c + 1

As .osgb is a binary native format, it is nearly impossible to quickly read and understand the contents of the generated database. We can slightly change the command-line arguments of vpbmaster to support writing to ASCII files (.osg or .osgt), as well as writing out tile images (using --image-ext to set a valid image extension) at the same time:

# vpbmaster -d data/gcanyon_height.png -t data/
gcanyon_color_4k2k.png -o output/out.osg --image-ext bmp

By default, each tile of VPB is formed by 64 x 64 vertices and mapped by a 256 x 256 texture. As the original elevation and texture size is 4096 x 2048, VPB must build at least seven levels to get to the highest data resolution. The entire generation time may be too long and the result may not be necessary in some situations. In this case, we can control the levels to generate manually by specifying the -l parameter:

# vpbmaster -l 3 -d data/gcanyon_height.png -t data/
gcanyon_color_4k2k.png -o output/out.osgb

Other two useful arguments are --terrain (default) and --polygonal. The option --terrain means to use the osgTerrain::TerrainTile class for generating grid geometries based on height fields, which is used by default. The option --polygonal will treat the data as triangle faces and must tessellate and simplify them while creating tiles, which is much slower and not good for further analyzing work. The two opinions can't co-exist in one terrain generation process.

You may either download the earth's imagery from the TrueMarble or BlueMarble website:

TrueMarble: http://www.unearthedoutdoors.net/global_data/true_marble/download

BlueMarble: http://earthobservatory.nasa.gov/Features/BlueMarble/

We are going to make use of the free data from TrueMarble. Here is their copyright information:

"Unearthedoutdoors.net contains graphics, information, data, reviews, and other content accessible by any Internet user. All Content is owned and/or copyrighted by Unearthed Outdoors, LLC (unless otherwise explicitly noted), and may be used only in accordance with this limited use license.

Unearthed Outdoors, LLC is protected by copyright pursuant to U.S. copyright laws, international conventions, and other copyright laws."

You can't use these data for commercial purpose without the permissions from Unearthed Outdoors, LLC.

Let us start.

# osgviewer output/out.osgb

OSG has a special node type osg::CoordinateSystemNode for the viewer system to convert data between XYZ and latitude/longitude/height, and it also builds a local transition matrix internally for node transformations in the scene graph. In polygonal mode (set with the option --polygonal), VPB will set it as the parent node of the entire terrain sub-graph to guide the creation of all sub-tiles on the earth. But in grid mode (--terrain), the osgTerrain::Terrain, which is the derivative class, will be used instead. Both classes store the earth dimension and coordinate information. The --geocentric option here will indicate VPB to use the center of the earth as the terrain's center and defines units in meters directly. It is actually defined as the ECEF coordinate system, which has an ellipsoid as the measurement of the earth shape, called World Geodetic System 1984 (WGS-84). Refer to the following link for details:

http://en.wikipedia.org/wiki/ECEF

We can make use of some other coordinate systems by specifying the --cs option. It uses a PROJ4 format string to declare new coordinate systems, for instance:

# vpbmaster -cs "+proj=latlong +datum=WGS84" ...

More information about the coordinate system string format can be found at the following PROJ4 project website:

http://trac.osgeo.org/proj/

We will continue working on the earth model and try to add some height fields at a certain longitude and latitude range. Thanks to the SRTM project, we can freely download and use the globa-elevation data along with the textures for non-commercial purposes. The download link is:

http://srtm.csi.cgiar.org/SELECTION/inputCoord.asp

Citation:

"Jarvis A., H.I. Reuter, A. Nelson, E. Guevara, 2008, Hole-filled seamless SRTM data V4, International Centre for Tropical Agriculture (CIAT), available from http://srtm.csi.cgiar.org."

You must read the disclaimer given in the following link before making use of SRTM's elevation data for any purposes:

http://srtm.csi.cgiar.org/SELECTION/SRT_disclaimer.htm

Let us start.

You can find in the working directory a series of new files and folders, which are created and managed by VPB. The build_master.source is an ASCII file wrapping up all the source data and build options. Open it with any text editor, and you will find that the file looks like a OSG native scene file (.osgt) and may even be loaded with the osgDB::readNodeFile() function. It has an osgTerrain::TerrainTile node to save build options (output name, extents, levels, and others) via the vpb::DatabaseBuilder object, and save child layers for different input data.

The build_master.tasks file records all the sub-tiles to be generated during the whole process. The status of each sub-tile task can be found in the tasks folder. A standard status file may be automatically written, as shown in the following code block:

application : osgdem --run-path /usr/local/bin -s
build_master.source --record-subtile-on-leaf-tiles -l 8 --
subtile 3 0 0 --task tasks/build_subtile_L3_X0_Y0.task --
log logs/build_subtile_L3_X0_Y0.log
date : [building time]
duration : [building duration]
fileListBaseName : outputout_subtile_L3_X0_Y0/
out_L3_X0_Y0.osgb.task.0
hostname : [host name]
pid : [pid]
source : build_master.source
status : [pending/completed]

A pending task indicates that the sub-tile is not created yet; and completed task means there is no need to work on the sub-tile unless the user needs a complete rebuild. If the building process is canceled, or crashes due to some system reason, you can make use of the task files' status and rerun vpbmaster with the --tasks argument:

# vpbmaster --tasks build_master.tasks

Tasks that are marked as completed will be skipped this time. Otherwise, when you execute vpbmaster again, the finished data will be overwritten instead of a resuming process.

To make use of the patching functionality, there are two prerequisites: first, you must have the new data; and second, you should keep the build_master.source file and all source files used to produce the old database, as they are needed for handling resolution and boundary problems. Sub-tiles that are not affected by the new patch will not be rebuilt anymore.

Let us start.

The world file is an ASCII parameter file used for geo-referencing raster map images. It was first introduced by the ESRI. This kind of files (with the extension .tfw, .tifw, or .wld) often describes the location, scale, and rotation of the map with six lines (each with a decimal number). When GDAL is going to read a TIFF file, it will automatically look for a world file with the same name and associate these two files together for gathering necessary information.

The meaning of the six-line parameter is as follows:

Because the newly-created patch file (sub_gcanyon_height.tif) doesn't contain any geographic information inside, we have to provide a world file to place it at an appropriate place and with suitable resolution. If you have GDAL installed, we will find an executable named gdalinfo. Let us use it to check the image with the associated .wld file:

# gdalinfo data/sub_gcanyon_height.tif

And you will get some report, as shown in the following code block:

Corner Coordinates:
Upper Left ( 999.750, 500.250)
Lower Left ( 999.750, 244.250)
Upper Right ( 1511.750, 500.250)
Lower Right ( 1511.750, 244.250)
Center ( 1255.750, 372.250)

Because the patch is 1024 x 512 but only covers a 500 x 250 area, the maximized level will increase to 7. And you could see some L7 files in part of the subfolders of output, maybe L2_X0_Y0 and L2_X1_Y0 in this recipe, as the new patch has intersections with them.

In fact it is common to obtain geospatial data in the GeoTIFF format instead, which already has such metadata embedded.

You can download the latest NVTT source code from the following link:

http://code.google.com/p/nvidia-texture-tools/downloads/list

Or you can use SVN to check it out:

# svn checkout http://nvidia-texture-tools.googlecode.com/
svn/branches/2.0/ nvtt

NVTT also provides CMake scripts for cross-platform building. But for Linux users, you can directly compile it by executing the following commands under the NVTT root directory:

# ./configure
# make
# make install

An important note for building NVTT with CMake: By default, CMake will generate makefiles for compiling static libraries, but in this way the results will not work for the corresponding OSG plugin. So you must add a definition NVTT_SHARED=1 to force generating shared libraries while running the cmake executable, that is:

# cmake /home/nvtt -DNVTT_SHARED=1

The cmake-gui tool can't be used here as it doesn't allow macros to be added as arguments.

Let us start.

As we know, OSG uses the osgDB::ReaderWriter class as the base interface of all reader/writer plugins. Every file format is parsed within a certain plugin and then returned an osg::Image or osg::Node pointer as the result (or save to specified filename while writing scene nodes and images). For instance, COLLADA 3D models are handled by osgdb_dae. You will see a ReaderWriterDAE class defined in the DAE plugin source code which implements the concrete data reading and conversion.

But in this recipe, we meet another kind of OSG plugins—the image processor plugin. It uses a base interface called osgDB::ImageProcessor and implements its derived classes in plugins. This class has two important virtual methods to override:

virtual void compress(osg::Image&,
osg::Texture::InternalFormatMode, bool, bool,
CompressionMethod, CompressionQuality);
virtual void generateMipMap(osg::Image&, bool,
CompressionMethod);

Re-implement them and then the processor will be able to compress images to specific formats and generate mipmaps for them. That is also what the osgdb_nvtt plugin does with the external NVTT library.

Let us start.

Do you remember the sub-tile task files in the tasks folder? Let us open some task files randomly this time and have a look at the hostname line:

fileListBaseName : outputout.osgb.task.0
hostname : [email protected]
...
fileListBaseName :
outputout_subtile_L3_X0_Y0/out_L3_X0_Y0.osgb.task.0
hostname : [email protected]
...
fileListBaseName :
outputout_subtile_L3_X0_Y1/out_L3_X0_Y1.osgb.task.0
hostname : [email protected]
...

You can find that VPB automatically divides the tasks and sends commands to different hosts within the local network. It's really good to see that a series of high-performance computers can cooperate so smoothly on a huge generation work. It really is a time-saving idea if you have such an environment!

To build a NFS system, you can try the GlusterFS at http://www.gluster.org/.

And for SSH implementations under different platforms, see the OpenSSH website for details:

http://www.openssh.com/

Let us start.

The file cache is checked and reused while the database pager is processing paged nodes in request. First, it determines if a new name is a remote filename (with the hostname at the beginning of the name string). If so, it will try to find the required hostname and filename in the cache folder. If it succeeds, the pager will mark the request as 'high latency' and directly read from the local cached files later.

You can also decide if a file should be cached or not by specifying an osgDB::FileLocationCallback object. It has two virtual methods to override:

virtual Location fileLocation(const std::string& filename,
const Options* options);
virtual bool useFileCache() const;

The first method will return if the file is local (LOCAL_FILE) or remote (REMOTE_FILE), and local files will not be added to the cache. The second method can quickly enable or disable the use of caching. You can at any time specify your own location callback by calling the setFileLocationCallback() method of the reading option object:

osg::ref_ptr<osgDB::Options> options = new osgDB::Options;
options->setFileLocationCallback( ownCallback );
pagedNode->setDatabaseOptions( options.get() );

OSG also supports a revision mechanism that can provide adding/removing/modifying information of the remote terrain database. The database pager will then decide if the scene graph must be updated due to changes on the remote server. This functionality is not enabled by default at the time of writing this book, but you can try the osgdatabaserevisions example in the OSG source code to see how it works.