User:MPC/MDR Subfile Format

From OpenStreetMap Wiki
Jump to navigation Jump to search

User:MPC

A reverse-engineering of Garmin's MDR Subfile format

MDR Header

Byte Offset Length (bytes) Description Notes
Header 0x000 2 MDR Header length. Various lengths have been seen. e.g:
MapSource-generated = 708 bytes (0x2C4) thus 44 subsections.
Garmin's own (NT) maps have used 568 bytes (0x238) thus 40 subsections.
length=0x11E has been seen too - thus only 19 subsections.
0x002 19 Garmin Common Header The ID-String will be "GARMIN MDR"
0x015 2 Codepage ? 0xe4 0x04 --> 1252 spotted // ( 0xe404 codepage probably --Liosha 15:45, 1 September 2009 (UTC) )
0x017 2 Flags ? 0x01 0x00, 0x07 0x00 or 0x12 0x00 spotted
0x019 4 Flags ? Spotted:
"0x01 0x00 0x11 0x00"
"0x02 0x00 0x17 0x00"
"0x01 0x00 0x0e 0x00"
"0x01 0x00 0x09 0x00"
MDR1
Subtile List 		0x01D 4 Offset. The section MDR 1 contains a list of maps and for each map an offset to a reverse index for that map.

List of subtile files referenced in many other MDR blocks. Required

0x021 4 Length
0x025 2 Record size. 0x04 (minimum), 0x08 if bit 0 of 'flags' is set
0x027 4 Flags. If bit 0 is set, there's an offset present in the records.
If bit 1 is set then the 'tile' value is a hex references to a .GMP subtile file.)
MDR2 0x02B 4 Offset In MapSource-generated file same as in MDR4
0x02F 4 Length ? In MapSource-generated file 0
0x033 2 Record size. 0x02
0x035 4 ? 0x00
MDR3 0x039 4 Offset. In MapSource-generated file same as in MDR4
0x03D 4 Length ? In MapSource-generated file 0
0x041 2 Record size. 0x02
0x043 4 ? 0x00
MDR4
POI List 		0x047 4 Offset. ? A list of POI types used in the map.
0x04B 4 Length.
0x04F 2 Record size. Always seems to be 0x03
0x051 4 ? Have only spotted 0x00 so far
MDR5
City and ZIP 		0x055 4 Offset. Extended Gazetteer list. Required
0x059 4 Length
0x05D 2 Record size. Variable record size. 5 bytes and more
0x05F 4 Flags. These bits seem to specify the lengths of various optional and mandatory fields within the records.
MDR6
Shortform Gazetteer 		0x063 4 Offset. Shortform Gazetteer - seems to be an alternative to the Extended Gazetteer (MDR5) above.
0x067 4 Length
0x06B 2 Record size. Variable record size. 3 bytes known from Garmin files, 4 bytes believed possible.
0x06D 4 ? 0x01 seen in Garmin file.
MDR7
Sorted Roads 		0x071 4 Offset.
0x075 4 Length
0x079 2 Record size. 8 bytes known from Garmin files, 5 spotted by Elrond
0x07B 4 ? 0x018E2 seen in Garmin file.
MDR8 0x07F 4 Offset.
0x083 4 Length
0x087 2 Record size.
0x089 4 ? 0x00 seen in Garmin file.
MDR9 0x08D 4 Offset.
0x091 4 Length
0x095 2 Record size. 3 or 4 depending on count of POIs.
0x097 4 ? 0x00 seen in Garmin file.
MDR10 0x09B 4 Offset. POI cross-index
0x09F 4 Length. MDR10 contains the same number of entries as MDR11 so the ratio of this value to MDR11's length tells us the size of each MDR10 record.
0x0A3 4 ? 0x00 seen in Garmin file.
MDR11 0x0A7 4 Offset. POI strings table
0x0AB 4 Length. See comment above about the integer relationship between this value and MDR10's length.
0x0AF 2 Record size. 12 has been seen in Garmin's own files.
9 has been seen in MapSource-generated files.
0x0B1 4 ? 0x575 seen in Garmin's own files.
0x71 in MapSource-generated files.
MDR12 0x0B5 4 Offset.
0x0B9 4 Length.
0x0BD 2 Record size.
0x0BF 4 ? 0x00 seen in Garmin file.
MDR13 0x0C3 4 Offset.
0x0C7 4 Length.
0x0CB 2 Record size.
0x0CD 4 ? 0x00 seen in Garmin file.
MDR14 0x0D1 4 Offset.
0x0D5 4 Length.
0x0D9 2 Record size.
0x0DB 4 ? 0x00 seen in Garmin file.
MDR15
Sorted Strings 		0x0DF 4 Offset.
0x0E3 4 Length.
0x0E7 1 ? 0x00 seen in Garmin file. Just one byte here, so it seems.
MDR16 0x0E8 4 Offset.
0x0EC 4 Length.
0x0F0 2 Record size.
0x0F2 4 ? 0x00 seen in Garmin file.
MDR17
String Compression Tables 		0x0F6 4 Offset. MapSource may not generate this table, but Garmin's own maps have it.
0x0FA 4 Length.
0x0FE 4 ? 0x00 seen in Garmin file.
MDR18 0x102 4 Offset.
0x106 4 Length.
0x10A 2 Record size.
0x10C 4 ? 0x00 seen in Garmin file.
MDR19 0x110 4 Offset.
0x114 4 Length.
0x118 2 Record size. 3 was seen by Elrond
0x11A 4 ? 0x00 seen in Garmin file.
MDR20 0x11E 4 Offset.
0x122 4 Length.
0x126 2 Record size.
0x128 4 ? 0x60000 seen in Garmin file.
MDR21 0x12C 4 Offset.
0x130 4 Length.
0x134 2 Record size.
0x136 4 ? 0x00 seen in Garmin file.
MDR22 0x13A 4 Offset.
0x13E 4 Length.
0x142 2 Record size.
0x144 4 ? 0x231E seen in Garmin file.
MDR23 0x148 4 Offset.
0x14C 4 Length.
0x150 2 Record size.
0x152 4 ? 0x00 seen in Garmin file.
MDR24 0x156 4 Offset.
0x15A 4 Length.
0x15E 2 Record size.
0x160 4 ? 0x00 seen in Garmin file.
MDR25 0x164 4 Offset.
0x168 4 Length.
0x16C 2 Record size.
0x16E 4 ? 0x00 seen in Garmin file.
MDR26 0x172 4 Offset.
0x176 4 Length.
0x17A 2 Record size.
0x17C 4 ? 0x00 seen in Garmin file.
MDR27 0x180 4 Offset.
0x184 4 Length.
0x188 2 Record size.
0x18A 4 ? 0x00 seen in Garmin file.
MDR28 0x18E 4 Offset.
0x192 4 Length.
0x196 2 Record size.
0x198 4 ? 0x00 seen in Garmin file.
MDR29 0x19C 4 Offset.
0x1A0 4 Length.
0x1A4 2 Record size.
0x1A6 4 ? 0x01F6 seen in Garmin file.
MDR30
Index to Geographical words table 		0x1AA 4 Offset.
0x1AE 4 Length.
0x1B2 2 Record size Only 0x04 spotted so far.
0x1B4 4 ? 0x00 seen in Garmin file.
MDR31
Geographical words table 		0x1B8 4 Offset.
0x1BC 4 Length.
MDR32
Index to StreetAddress words table 		0x1C0 4 Offset.
0x1C4 4 Length.
0x1C8 2 Record size only 0x04 spotted so far.
0x1CA 4 ? 0x00 seen in Garmin file.
MDR33
StreetAddress words table 		0x1CE 4 Offset.
0x1D2 4 Length.
MDR34 0x1D6 4 Offset.
0x1DA 4 Length.
0x1DE 2 Record size.
0x1E0 4 ? 0x00 seen in Garmin file.
MDR35 0x1E4 4 Offset.
0x1E8 4 Length.
0x1EC 2 Record size.
0x1EE 4 ? 0x00 seen in Garmin file.
MDR36 0x1F2 4 Offset.
0x1F6 4 Length.
0x1F4 2 Record size.
0x1FC 4 ? 0x00 seen in Garmin file.
MDR37 0x200 4 Offset.
0x204 4 Length.
0x208 2 Record size.
0x20A 4 ? 0x00 seen in Garmin file.
MDR38 0x20E 4 Offset.
0x212 4 Length.
0x216 2 Record size.
0x218 4 ? 0x33 seen in Garmin file.
MDR39 0x21C 4 Offset.
0x220 4 Length.
0x224 2 Record size.
0x226 4 ? 0x00 seen in Garmin file.
??? 0x22A 4 Offset Suspicious: points past the end of the .MDR file in the Garmin maps I have here - SH.
0x22E 4 Length.
0x232 2 Record size.
0x234 4 ? 0x00 seen in Garmin file.

MDR1 Subtile List

This section lists all subtiles that are referenced by this MDR file. The index numbers (counting from 1) of the subtiles listed in this section are used in various other MDR sections to specify the subtile in which a given city (or other item) might be found.

The records look like this:

Offset (bytes) Length (bytes) Description
0x00 4 Subtile file name coded into a long. If bit 1 of the 'flags' field was set in the MDR1 header, then a 'printf' format of "I%07X.GMP" will create the subtile's name. Otherwise the subtile is held in the set of files "%08ld.{NOD,LBL,etc}".
0x04 4 Optional - only appears if bit 0 of the 'flags' field was set in the MDR1 header. Seems to be an absolute pointer (within the MDR file) to tile-specific data.


MDR4 POI List

This section lists all the POI types that will appear in the map and (presumably) is used to build the menu structure in the GUI of the GPS units and MapSource.

The records look like this:

Offset (bytes) Length (bytes) Description
0x00 1 POI major type.
0x01 1 Often seems to be 0x00 (apparently is some sort of index, used by MDR8/9/10/11??).
0x02 1 POI subtype.

POI Types are listed in OSM_Map_On_Garmin/POI_Types

MDR5 City and ZIP List

The MDR5 section contains a sorted list of all cities. The contents of each record in this section are consistent, but exactly what is present (and how many bytes are used by each element) is guided by the 'flags' field in the MDR5 header.

According to Bernhard Heibler, beside the cities there are also examples where this list contains ZIPs.

According to Steve Hosgood, the entries in the MDR5 table should be thought of as forming an "Extended Gazetteer" (see the similar MDR6 table forming a "Shortform Gazetteer - see below). Each entry of this Extended Gazetteer seem to be as follows:


First byte(s): a mandatory reference to the tile in which the city lies:

Offset (bytes) Length (bytes) Description
0x00 1 or 2 (depends on length of the MDR1 section) 1-based index to the subtile that provides this entry (from MDR1).


Next byte(s): a mandatory reference to that subtile's LBL file:

Offset (bytes) Length (bytes) Description
0x00 1 - 4 (given by bits [1:0] of the MDR5 'flags' plus one) Ignoring the most significant 2 bits gives us a city number in its subtile's ".LBL" file.

.The most significant bit is set if this entry has a matching entry in the Shortform Gazetteer (MDR6).
Not seen the next most significant bit set.


Next 3 bytes: a mandatory offset to this city's entry in MDR7:

Offset (bytes) Length (bytes) Description
0x00 3 Offset to the street list in MDR7. Ignore the most significant bit though - this seems to be a flag indicating the length of the item being pointed-at (8 or 9 bytes so it seems).


The rest is as follows:

Offset (bytes) Length (bytes) Description Notes
0x00 3 Offset When Flags Byte #3 bit 7 is set, this is a zero-based offset into MDR20.
0x03 1 Flags byte #2 When Flags Byte #3 bit 7 is set, seems to indicate the structure of the MDR20 data being pointed-to.
0x04 1 Flags byte #3 Bit 7 seems to be set only if the MDR20 offset is valid.

Very strong evidence is building up to indicate that when Flags Byte 3 bit 7 is set, then the 3-byte number at offset 0x00 points into MDR20, and that the Flags Byte #2 at offset 0x04 is indicating the structure of what to expect there.

For instance, if Flags Byte #2 is 0x17 or 0x27, then the MDR20 byte at [offset+0x06] almost always has its upper nybble set to '0'. The probability of that happening by chance is very low!

Additionally, other patterns can be spotted in the blocks of MDR20 data when Flags Byte #2 has a given value. Based on observations in other parts of Garmin's file formats, it could well be that Flags Byte #2 is a bitfield indicating what to expect in the data block. Obviously, patterns 0x27 and 0x17 have three bits set in common - probably explaining the "[offset+0x06]" observation above being shared between them.

On all records (so far) with Flag Byte #2 set to 0x27, the byte at [offset+0x03] of the MDR20 block has its low nybble set to '0'. I (Steve Hosgood) have not yet written an automated tester for these observations - pattern-spotting tends to be something humans are good at, so the work is being done by hand to start with.

MDR6 ShortForm Gazetteer

The MDR6 section is a global sorted list of all cities. This is much the same as with the Extended Gazetteer (above), but there are no duplicate entries, it's just an array containing a reference to the given city's subtile file and the number of the city within that tile.

Each element of the array is as follows:

Offset (bytes) Length (bytes) Description Notes
0x00 1 Subtile number. This is the "1-based" index to a subtile record in the MDR1 array.
0x01 2 City number (In that subtile's LBL subfile??)

It may be that a given .IMG file is composed of more than 256 subtiles, in which case the following alternative may exist, with the "Record Length" field of the MDR6 header set to "4". This has not been confirmed yet.

Offset (bytes) Length (bytes) Description Notes
0x00 2 Subtile number. This is the "1-based" index to a subtile record in the MDR1 array.
0x02 2 City number (In that subtile's LBL subfile??)

MDR7 Sorted Roads List

This is a global sorted list of Road Names.

Offset (into that element) Length (bytes) Description
0x00 1 or 2 Subtile index from MDR1.
0x01 or 0x02 3 Road pointer offset into the subtile's NET file. The top bit, 0x800000, is a flag
0x04 or 0x05 3 (Optional) Pointer to a label in MDR15. If this entry is missing, then the table at subsection #1 of MDR17 will contain indexes to the elements of this table instead.

MDR9 POI Chapter-list

MDR9 contains a one byte index (starting at 1 and counting up) followed by a 2 or 3 byte reference into MDR10. The reference is to the first record of that 'chapter' in MDR10.

For simple maps not requiring a chapter-list, then just the bytes "01 01 00 00" (i.e. index 0x01, reference 0x000001) seems to be enough, which treats MDR10 as one single chapter.

MDR10 POI Category-based Pick-lists

It currently looks like MDR10 might have the same record length as MDR9.

MDR10 contains a one byte POI subtype followed by a 2 or 3 byte reference to a POI record-number in MDR11. Because the POI records in MDR11 are sorted alphabetically, the POI record-numbers in any given chapter of MDR10 will be in ascending order to preserve the alphabetical sorting of items in that chapter.

The top bit of the record-number is a flag. If it is set, then the POI name is unique, if not then the POI name is the same as the previous one.

MDR11 Sorted POI Master List

MDR11 references all the POIs in a map in alphabetical order with no regard of POI type. Functionality for filtering them on a type basis is provided by MDR9/MDR10 and MDR18/MDR19.

Each entry in MDR11 has the following format:

Offset (into that element) Length (bytes) Description
0x00 1 or 2 Subtile index from MDR1.
0x01 or 0x02 1 (probably possible for this to be two bytes, even three) POI number in that subtile.
0x02 or 0x03 2 Subsection for that POI in its subtile (or maybe city-number within the subtile??)
0x04 or 0x05 3 Offset to this POI in that subtile's .LBL file
0x07 or 0x08 2 Zeroes??
0x09 or 0x0A 3 (Optional) Pointer to this POI's name in the MDR15 string-list.

If there is no reference to MDR15, then the third subsection of MDR17 will have a table pointing to the relevant entries here in MDR11 instead.

MDR15 Sorted Strings List

MDR15 consists merely of a number of concatenated null-terminated strings, sorted alphabetically. These are (optionally) referenced from the MDR5, MDR7 and MDR11 sections.

MDR17 String Compression-Lists

The MDR17 block is slightly more complex than some of the others because it is broken into a number of subsections. The first few bytes of a subsection is a length indicator (strangely encoded, see below). This is followed by a short subsection header, typically of 2 or 4 bytes (presumably indicating the nature of the list(s) that follow). The body of a subsection is then filled by one or more lists of items.

So far, it seems that MDR17 contains three subsections:

When subsection #1 contains just a single list, its indexes refer to items in MDR7. The two- or four-character strings in the list are the first characters of the streetnames referred-to by the MDR7 table. Each two- or four-character string only appears once, its index points to the first street in MDR7 with that starting sequence.

When subsection #2 contains just a single list, its indexes refer to items in MDR5. The two- or four-character strings in the list are the first characters of the city names referred-to by the MDR5 table. Each two- or four-character string only appears once, its index points to the first city in MDR5 with that starting sequence.

When subsection #3 contains just a single list, its indexes refer to items in MDR11. The two- or four-character strings in the list are the first characters of the POI names referred-to by the MDR11 table. Each two- or four-character string only appears once, its index points to the first POI in MDR11 with that starting sequence.

The MDR17 subsection length encoding

Look at the first byte, find the lowest bit n that is set. For example: byte = 0x51: n = 0, byte = 0x84, n = 2. n is the size of the length field itself, including the first byte. Read the first n+1 bytes in little endian and assign to integer i. The length of the following header + list is i >> (n+1).

Example: 

Bytes: f4 11 03
bits of first byte: 1111 0100
lowest set bit n: 2
Interpret the first n+1 bytes as a little endian integer: 0x311f4
Length of the header plus the body that follow: i >> n+1: 0x623e

The MDR17 subsection header

The subsection header seems to consist of two bytes if there's only one list in that subsection:

Two bytes from subsection header Meaning
09 01 A 2-chars and 1-byte-index list follows
13 01 A 2-chars and 2-bytes-index list follows
1D 01 A 2-chars and 3-bytes-index list follows
?? ?? A 4-chars and 1-byte-index list follows
23 03 A 4-chars and 2-bytes-index list follows
2D 03 A 4-chars and 3-bytes-index list follows

The patterns would seem to indicate that "19 03" would herald a 4-chars and 1-byte-index list. This has not been seen yet.

However, as mentioned above, a subsection can contain many lists, one after the other, with no apparent breaks in between! An example is the Garmin 2005 "UK and Ireland" map, where the first subsection's header uses 3 bytes to indicate that it will take 242510 (0x3B34E) bytes, then there appear to be four bytes in the header: "0x00 0x80 0x01 0x00" followed by a body containing:

  • A 2 char and 1 byte index list with just one entry
  • A 2 char and 2 byte index list with 0xA4 entries
  • A 4 char and 3 byte index list with 0x44DD entries
  • A 4 char and 3 byte index list with 0x3F45 entries
  • A 2 char and 2 byte index list with 0xFC entries
  • A 2 char and 2 byte index list with 0x9C entries
  • A 2 char and 1 byte index list with 0x7 entries
  • A 2 char and 1 byte index list with 0x3 entries
  • A 2 char and 2 byte index list with 0x100 entries
  • A 2 char and 2 byte index list with 0x122 entries
  • A 2 char and 2 byte index list with 0x130 entries

These lists differ from the "normal" lists in that their index values are zero-based rather than one-based.


The MDR17 subsection body

The body consists of one or more lists. There don't seem to be any indicators within MDR17 to say where one list stops and the next starts, nor any info as to what sort of list is where when there are more than one list in the subsection. Presumably this indexing information is elsewhere...

The lists spotted in Garmin's files so far have the following patterns (per element):

Offset (into that element) Length (bytes) Description
0x00 2 Two characters in Latin-1 encoding.
0x02 1 A single-byte number


Offset (into that element) Length (bytes) Description
0x00 2 Two characters in Latin-1 encoding.
0x02 2 A two byte number (little-endian).


Offset (into that element) Length (bytes) Description
0x00 2 Two characters in Latin-1 encoding.
0x02 3 A three byte number (little-endian).


Offset (into that element) Length (bytes) Description
0x00 4 Four characters in Latin-1 encoding.
0x04 2 A two byte number (little-endian).


Offset (into that element) Length (bytes) Description
0x00 4 Four characters in Latin-1 encoding.
0x04 3 A three byte number (little-endian).

A list just consists of a sequence of any one of these elements (all elements being of the same type within any given list). Within any list, the elements' strings are sorted in alphabetical order, and the elements' numbers form an ascending sequence (but with many gaps).

In the case of a subsection with multiple lists: when one list finishes, the next list just starts on the next available byte.

The four-character strings can clearly be seen to be parts of place names, along with parts of postcodes and maybe even street addresses.

The strings can be padded with NULL bytes on the right, meaning that three-character strings can appear as members of the four character lists, and one-character strings as members of the two-character lists. Extra complication is found in that two-character strings padded with two NULLs can appear in the four-character lists, as can single-character strings padded with three NULLs!

MDR19 (unknown)

MDR19 seems to have a record length of 3 and the same number of records as MDR11. The entries are 3byte unique indexes (probably into MDR11). This table can be seen as a permutation table for the MDR11 entries. The highest bit (23) seems to be a flag.

MDR30/31 Geographical Words Table

The MDR30 section contains an array of index-items. Each index-item is 4 bytes long as follows:

Offset (into that element) Length (bytes) Description
0x00 2 An offset into the MDR31 strings-table.
0x02 2 Flags?? Almost always zero, but one or two entries of "0x111" have been seen.

Each 'offset' is the zero-based byte offset of a string in the MDR31 table. To get the length of string 'N', subtract the offset of string 'N+1' from the offset of string 'N'. Notice that there are no NULL characters terminating the strings in the MDR31 string-table, so you have to add your own.

The MDR31 section contains a large area of variable-length strings, concatenated end-to-end. (Notice that the MDR31 header lacks a field for element-length). There are strings in here for all the languages that the unit supports. All the words seem to be geographical in nature - "way", "junction", "river", "wanderweg" etc. The words are sorted into alphabetical order with no regard for which language they belong to.

Strangely, there seem to be no accented Latin-1 characters in these words.

MDR32/33 StreetAddress Words Table

The MDR32 section contains an array of index-items. Each index-item is 4 bytes long as follows:

Offset (into that element) Length (bytes) Description
0x00 2 An offset into the MDR33 strings-table.
0x02 2 0x0000 most of the time. But lots of other values have been seen (Garmin NT map). If the values are not 0x00 they are incresing as far as possible. Maybe this is an offset to another MDR table (??).

Each 'offset' is the zero-based byte offset of a string in the MDR33 table. To get the length of string 'N', subtract the offset of string 'N+1' from the offset of string 'N'. Notice that there are no NULL characters terminating the strings in the string-table, so you have to add your own.

The MDR33 section contains a large area of variable-length strings, concatenated end-to-end. (Notice that the MDR33 header lacks a field for element-length). There are strings in here for all the languages that the unit supports. All the words seem to be parts of streetnames - "avenue", "road", "street", "boulevard" etc. The words are sorted into alphabetical order with no regard for which language they belong to.

Strangely, there seem to be no accented Latin-1 characters in these words.

Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=User:MPC/MDR_Subfile_Format&oldid=730493"