TOPO Creator

Description:
Some MapSends has built-in topo database. The database is stored in the \MAP\BLX folder in the *.blx files. Format of the *.blx files is very complex (to my opinion this is the most complex of Mapsend's formats).

Topographic data composed in the series of data chunks, each chunk represents 128x128 matrix of heights. Chunks in turn, composed from 15 smaller matrices to build 128x128 matrix. Moreover, chunks are compressed with Magellan's custom compression algorithm.

All this was discovered by Henrik Johansson (see his messages #203 and #797 in Mapsend_Format yahoo group). He created Python program to extract/create BLX files (available at MapSend_Format yahoo group). Many thanks to Henrik for his outstanding work! In my program I'm using his algorithms for compression/decompression of chunks and algorithms to create matrices.

I choose the most explicit format of the input files: Terrain Matrix model compatible with free 3DEM viewer. In this format Height matrix is represented by matrix of signed integers, all georefference information is stored as a plain text in a separate file (*.hdr). Terrain Matrix can be created with 3DEM viewer from may other 3D formats. 3DEM viewer available for free at http://www.visualizationsoftware.com/3dem.html.

TOPO file structure

Not all MapSends has topo database. As far as I know, almost all Mapsends can show topo information on PC screen. Some of them can export topo to GPSR.

Some examples (MapSends/MMO that initially do not have TOPO):

a) MMO1.00, MMO1.10, MMO2.00: can show TOPO on PC screen and export TOPO to GPSR
b) Direct Route can show TOPO on PC and export TOPO to GPSR
c) Streets&Destination USA can show TOPO on PC, but cannot export to GPSR . But if S&D's mapsend.exe is replaced with MapSend Topo's mapsend.exe, S&D becomes fully compatible with TOPO (including export, all the other stay the same).
d) MapSend Streets Europe, MapSend S&D Canada, MapSend S&D Australia. Similar to S&D US. Can show TOPO on PC screen, but can not export to GPSR. But if mapsend.exe is replaced with mapsend.exe engine from MapSend Topo, they can export topo to GPSR.

To make MapSend/MMO compatible with TOPO the following is needed:

1) Create \MAP\BLX folder containing *.blx files with TOPO database. The *.blx files can be created with TOPO_creator from available 3D data.
2) topo3d.ini and topo3d.cfg files must be in the MAP folder. topo3d.ini file contain info how topo lines appear on PC screen, topo3d.cfg contain info how TOPO will be shown in GPSR
3) Topo3DInit=topo3d.ini,BLX0 string must be in the [PATH] section of mapsend.ini (or MMOffice.ini) files in the root directory.

For convenience I add the feature to do all above mentioned automatically with TOPO_creator, ("Do Topo") button.

Some Mapsends (Streets Europe, S&D Canada, S&D Australia) needs some more modifications to show TOPO on PC screen:

1) the following section must be inserted to the map.ini file (say, at the end)

[ISO_DRAW]
labels=15 170 175 80 15 s 255 255 255 255
lines_style=3 220 220 190 1 170 175 80 1 255 0 0 1

2) Show_Topo=0 should be replaced with Show_Topo=1 in the [OPTIONS] section of the mapsend.ini file (this also needed for S&D USA).

TOPO map will be shown if the map is in the range covered with your MapSend.

To have an idea what mean parameters in the topo3d.ini and topo3d.cfg please have a look at Ryan Walsh's Msg#1070 in the MapSend_format yahoo group.

Data preparation

3D data must follow the following rules:

1). The must not contain abrupt changes of height (usually at the borders)
2) They must be WGS84 Lat/Long
3) Latitude and Longitude increments (steps) can differ from each other, but not less than 0.0002 degree each (otherwise map will not be shown). Mapsend Topo USA uses 0.0008 degree grid, so custom maps can be as much as 4 times more detailed.
4) Number of rows and columns must be N*128, where N is a positive integer.

Example of data preparation

Many different utilities can be used for data preparation, in the example I will use the following free software:

1) 3DEM viewer : available at http://www.visualizationsoftware.com/3dem.html (viewer - format converter)
2) gdalwarp.exe : extremely powerful command-line utility for format/projection/resampling (Henrik, thanks again for recommendation). This utility is a part of open source GDAL project. I found this utility in the OpenEV Binary Kit for Windows (OpenEV_FW_174.zip file available at http://openev.sourceforge.net or www.remotesensing.org).

Step 1.

1660095.DEM.SDTS.TAR ( 10m UTM DEM) data set was downloaded from the Internet and converted to hunter.dem with sdts2dem.exe utility. File size is ~9Mb

Step 2.

Observe the hunter.dem with 3DEM viewer:



Step 3.

We see that the map contains zeros at border area, they must be removed. Select (write on a piece of paper) area we can trust (to be sure that zero borders are out of this area), say: -74.2496, -74.1273, 42.2491, 42.1259. Then, this map is 10 m UTM, so it must be converted to WGS84 Lat/Long and number of columns and rows must be N*128.

Let us do some calculations:

Longitude increment: -74.1273 - -74.2496 = 0.1223. With minimal allowed step of 0.0002 it gives: 0.1223/0.0002=611.5. We are allowed of N*128: 611.5/128=4.78. Minimal integer close to 4.78 is 4, so we must use 128*4=512 cols.

Latitude increment: 42.2491 - 42.1259 = 0.1232. With minimal allowed step of 0.0002 it gives: 0.1232/0.0002=616. We are allowed of N*128: 616/128=4.81. Minimal integer close to 4.81 is 4, so we must use 128*4=512 rows.

(Note: this calculations can be done automatically with TOPO_creator, but will take more steps to prepare the dataset, see below)

Summary of our task:

We must cut edges of our hunter.dem to selected area (-74.2496, -74.1273, 42.2491, 42.1259), convert data to WGS84 Lat/Long, resample to have 512 rows and 512 columns.

This is one step operation with gdalwarp utility (!):

gdalwarp.exe -t_srs WGS84 -rcs -ts 512 512 -te -74.2494 42.1263 -74.1259 42.2493 hunter.dem hunter.tif

see description of gdalwarp.exe for all set of parameters. Here we use:

-t_srs WGS84 (WGS84 Lat/Long projection)
-rcs (cubic spline resampling, slowest, but the best)
-ts 512 512 (width, height (cols/rows))
-te -74.2494 42.1263 -74.1259 42.2493 xmin ymin xmax ymax- selected region, to cut edges (zeros)
hunter.dem : our data
hunter.tif : output file in GeoTiff format (default for gdalwarp.exe)

Conversion takes several seconds. Here is the result, image is stretched horizontally compared to original data, this is due to UTM-Lat/Long conversion.



Convert this GeoTiff hunter.gif file to Terrain Matrix (Save Terrain Matrix-Binary (Signed integer). This will be hunter.bin file ready to *.blx creation.

TOPO_creator



1. If your MapSend/MMO does not contain TOPO file structure, press Do Topo, select root folder for your MapSend/MMO, press "OK". The following will be done:

a). empty BLX folder will be created
b) topo3d.ini and topo3d.cfg files will be created with the default content (ready to use), but you should replace "YourBLXfile.blx" in the topo3d.ini to the name of file created with TOPO_creator.
c)"Topo3DInit=topo3d.ini,BLX0" will be inserted to the mapsend.ini or MMOffce.ini file

Now your MMO/Mapsend ready to accept TOPO data. Note: if your MapSend already contain topo structure this step should be skipped. This button is needed only once and for the Mapsends/MMO without TOPO support..

2. Select source BIN matrix (hunter.bin in the above example)

3. Press "Check" to validate the data. If resampling is needed or step is less then 0.0002 you will see warning messages. If number of Rows/columns is not N*128, they will be truncated (with warning messages). If step is less than 0.0002 you will see warning message with recommended parameters for gdalwarp.exe (for lazy people, who prefer press buttons in favor of doing calculations).

4. Select target BLX file. Press "Create".

The above example takes 3-5 seconds (Pentium 4, 2Ghz, 256).

5. Replace "0=......" in the topo3d.ini to the name of file created, put *.blx file in the \MAP\BLX folder. Done.



Here is an example:

MMO: (with default settings topo map is visible at zoom higher than 1:500 000, (10 mi scale). For some reasons topo sometimes is not visible if so, uncheck and then check again Show Topo in the Options menu. In the example summit POIs (small triangles) are inserted over the TOPO map. They were extracted from MapSend TOPO US (POI_DB_extractor-POI_DB_creator).



For comparison below is the same area taken from MapSend TOPO US. We can see that our custom map is more accurate and detailed than MapSend Topo US map.



Apendix A. Format of Terrain Matrix (used by 3DEM)

There are 2 files, name is the same, but extesions are different:

1. Header file with georefference information (*.hdr). This is plain ASCII text. The following fields are used by TOPO_creator:

a) It checks data_format and map_projection (they must be int16 and Lat/Lon) and elev_m_unit (must be meters).
b) left_map_x; lower_map_y, upper_map_y , riht_map_x
c) number_of_rows; number_of_cols

All the others have no meaning for TOPO_creator, but must exists (with any value after =)

file_title = hunter
data_format = int16
map_projection = Lat/Lon
left_map_x = -74.249481
lower_map_y = 42.126022
right_map_x = -74.127419
upper_map_y = 42.248978
number_of_rows = 512
number_of_columns = 512
elev_m_unit = meters
elev_m_minimum = 413
elev_m_maximum = 1233
elev_m_missing_flag = -9999

2. Elevation data (2 bytes integers), meters (*.bin). Columns and rows as indicated: (MaxX, Min X: Longitude, MinY, MaxY:Latitude)

(MinX, MinY)................................................................(MaxX,MinY)
..........................................................................................................

.........................................................................................................

(MinX, MaxY)..............................................................(MaxX,MaxY)

Note: From Min to Max for both directions, so for Northern hemisphere North is at the botom row, and South is at the top.

Also you can create elevation matrix as a plain ASCII text of heights (following the above mentioned rules), then this text data can be converted to binary data with 3DEM. Every point will be a real number with 2 digits after decimal separator:

Here is a part from the beginning of the file:

693.00 686.00 666.00 655.00 648.00 633.00 612.00 609.00 618.00 615.00 603.00 596.00 591.00 584.00.....


Requirements:
* MS Windows

More information:
TOPO Creator

Download