This document describes the Mobile Application Link (MAL) protocol. MAL is a standard protocol used for communications between a mobile device and a server process existing at a remote location. The current implementation of MAL is layered on top of the HTTP protocol. MAL can be made to run on top of other reliable protocols as well.
NOTE: Certain issues regarding the MAL protocol have yet to be decided. These items are noted in this specification in red and preceded with the word "PENDING". In some cases a data value may be given as "TBD," meaning its format is to be determined.
The purpose of MAL is to provide a standard protocol to be used when synchronizing a mobile device with a remote server. Currently, only mobile devices running Microsoft® Windows® CE or 3Com® Palm OSTM are supported; however, MAL is designed to run on all mobile devices, such as cell phones and pagers, in addition to Windows CE and Palm OS devices.
When a mobile device is synchronized with a server, the databases on the device are updated with information from the server and the data on the server is updated with information from the device. A database on the device is analogous to a file in many operating systems. On Windows CE devices, databases are often called objects. This specification uses "database" to mean any collection of data on a mobile device. It uses the term "record" to refer to an individual unit of data within a database. These terms shouldn't be confused with the other definitions of database and record.
During synchronization, the server reads device-side database records (sent to the server by the mobile device) and makes corresponding changes to a server-side data store. This server-side data store might be any file or (traditional) database accessible to the server. The data store might reside on the same computer as the server or a computer networked to the server's computer. The server is responsible for reading the device-side database records, translating them into a format that the server-side data store accepts, determining changes in the server-side data store, translating those changes into a format that the mobile device accepts, and sending those changes to the device.
The goal of MAL is to allow any MAL-enabled mobile device to be synchronized with any MAL-enabled server. (The initial MAL source release includes support for Palm OS and Windows CE devices and Windows NT, Linux, and FreeBSD servers.)
The MAL protocol specifies only how information is sent between the mobile device and the server. It doesn't specify what the device should do with the information it receives from the server, nor does it specify what the server should do with the information it receives from the device. In particular, it doesn't specify how the server should translate device-side database records into server-side data or vice versa.
A major design goal of MAL was to keep it simple and small, making it easy to adopt. As a vendor, you implement the MAL protocol and architecture using free source code provided and produce a MAL-compliant server.
The MAL protocol is a layer on top of the HTTP protocol. That is, the device and server use HTTP to communicate. The body of the HTTP message is a MAL message.
Note that to communicate with the server, the mobile device must be connected to a network. This may be through the use of a modem connected directly to the device or by serially connecting the mobile device to a desktop computer which in turn can connect to the server. (The initial MAL source release includes support for Windows 95, Windows 98, and Windows NT as the desktop computer's operating system.) The term "client" is used to mean the mobile device connected to a network, regardless of how that connection takes place.
A MAL message consists of one or more commands sent in one direction (either from the device to the server or from the server to the device).
All communications begin with a message from the device to the server. This is called the "request message." The server sends back a "response message." A "session" consists of one request message from the device and one response message from the server. A full synchronization might be performed in one MAL session, or it might take several MAL sessions.
This section describes the encoding and format of MAL messages. The next section describes how the device and server use MAL to communicate.
The data that the MAL protocol interprets consists of integers and strings. All integers are encoded in network byte order (most significant byte first).
MAL uses the following data types:
cint (see below)
uint32 (unsigned 32-bit integer)
uint16 (unsigned 16-bit integer)
uint8 (unsigned 8-bit integer)
string (a cint length followed by that many
bytes. No NULL terminator. A length of 0 denotes the empty string.)
bytes (an uninterpreted sequence of bytes)
All integers in MAL are CINT or compact integers. The CINT is more compact for the smaller values, which are the values that occur most frequently in MAL messages.
Range Length Interpretation
0 <= i < 254 1 byte The value of i is the value in the byte.
254 <= i < 2^16 3 bytes The first byte is 0xFE. The next two bytes are the value of i.
2^16 <= i < 2^32 5 bytes The first byte is 0xFF. The next four bytes are the value of i.
MAL messages are sent as HTTP requests and responses. A MAL HTTP request uses the minimal valid HTTP/1.1 header:
POST uri HTTP/1.1(CRLF)
Content-Length: MALMsgLength(CRLF)
(CRLF)
MALMsg
The uri after the POST is determined by the string sent from the
server in a SERVERCONFIG command. The default value
is /sync.
A MAL HTTP response is also minimal:
HTTP/1.1 200 OK(CRLF)
MALMsg
(server closes socket)
A MAL client may also need to generate additional headers if connecting to the MAL server through a proxy or other HTTP gateway. There are many parts of a MAL message which could be represented in HTTP headers, but MAL chooses to use a minimal subset of HTTP in favor of the more compact MAL protocol.
The reason for using HTTP is so that the client can communicate with the server through a firewall. Many users can only connect to the internet through a proxy server firewall.
PENDING: There may be additional HTTP headers sent as we move forward.
A MAL message has the format:
The header field is four bytes long and identifies the message as a MAL message. Following the header is a variable-length series of MAL commands. The last command in a MAL message is always END.
The MAL message header has the format:
The first two bytes contain the magic number 0xDA7E. The magic number indicates the beginning of the MAL message.
The next two bytes contain the MAL protocol version number. The first byte is the major version, and the second byte is the minor version. The major version is incremented on major, incompatible changes. The minor version is incremented if a compatible change is made. That is, if the clients and servers using older versions of the protocol can understand messages using this newer version, only the minor number is incremented.
An individual MAL command has the format:
where:
The MAL protocol identifies commands by the command tag integer. For convenience, this specification assigns a name to each command in addition to the integer tag.
Individual MAL commands are listed and described in the section "MAL Commands."
In a request message (a message from the device to the server), the first command must be HELLO, which initiates a session and contains information about the user to be synchronized. The last command in the request message must be END. These two commands are required. Other possible commands in a request message might send the server information about the device or might send the server records from a database.
A response message may begin with any command allowed from the server to the device. However, the response message must end with a GOODBYE command followed by an END command. The GOODBYE command contains a data field that specifies if the server expects the device to call back with more information or if the synchronization is complete. If the device does not receive a message that ends with GOODBYE/END combo, it displays an error to the user.
This section describes how the MAL protocol is used. Each command is described individually in the next section.
At the heart of a typical session, the device sends to the server database records that the server should use to modify data on the server side, and the server responds with records that the device should use to modify the database on the device side. Both the device and the server use the same commands to send records to each other.
To begin sending records, the device or server sends the command OPENDATABASE to specify which database the records come from. Following this command are a series of RECORD commands. The RECORD command contains an individual record and specifies if it has been changed, added, deleted, or unchanged since the last synchronization. The CLOSEDATABASE command immediately follows the last record. If the CLOSEDATABASE command is missing, a synchronization error occurs.
For example:
OPENDATABASE
RECORD
RECORD
.
.
.
RECORD
CLOSEDATABASE
If a server sends an OPENDATABASE command to the device and that database does not exist on the device, it will be created from the information supplied in the DATABASECONFIG command.
Before the device can send database records to the server, the server must tell the device which databases it is going to synchronize. This is done in an initial session. The initial session occurs when a new device connects with the server for the first time or when a hard reset has been performed on the device since the last synchronization.
In the initial session, the device begins with a request message. This message contains only the HELLO command followed by the END command. The server responds with a message containing one or more DATABASECONFIG commands. A DATABASECONFIG command contains the name of a database that the server will synchronize and the synchronization mode. The synchronization mode specifies whether the server wants to receive all of the records in that database or only those records modified since the last synchronization. This response message ends with a GOODBYE command that specifies that the device should call again.
Upon receiving the response to the initial request, the device sends a new request message that contains the databases the server requested in its response. This message uses the OPENDATABASE, RECORD, and CLOSEDATABASE commands described previously. The server knows whether all of the records have been sent or only the modified records have been sent based on which DATABASECONFIG settings it sent the device. The device might send no commands for a particular database if that database doesn't exist, if it's empty, or if only modifications have been requested and it contains no modifications. See the DATABASECONFIG command description for more specific information.
Sending clear text passwords over networks is not usually a good idea. A password that is ecrypted in some manner and sent is useless as well, a person with malicious intent could simply send the encypted password to trick the server into believing it was another user. A better, but by no means foolproof method, is to change the bytes representing the authorization over the network every sync, but still be certian that the user's password was valid. A way to do this is to concatonate the username, password, and a server specified value, called a nonce. The concatonated result is sent though a one way hash function (for MAL we will use MD5) and the hashed value is sent to the server. The server does the same thing, concatonates the username, password and nonce, hashing the result. If the hashed values match, the password is valid. For this seneriao to be secure both the server and client know the password, but never commumicated it openly via the MAL protocol. How the password is set up initially is outside the scope of this document.
The server may choose to send the device a cookie using the COOKIE command. A cookie is an opaque sequence of bytes. When the device receives a cookie, it retains the cookie and returns it to the server in the next HELLO command.
The server may use the cookie for anything that it likes. It might be used to determine the success or failure of the previous synchronization, or it might be used to uniquely identify each device with which the server synchronizes, or anything else that would be appropriate.
You can extend the MAL protocol using the EXPANSION command.
The EXPANSION command is command tag 1. It denotes a user-defined command. User defined commands are passed through the MAL layer untouched. As with all MAL commands, the EXPANSION command's tag is followed by a CINT specifying the command data's length which is followed by that many bytes of data. Typically, you'd use the first byte of data to identify a unique command and all of the remaining data is data passed to that command.
PENDING: There may or may not be sub-structure to the EXPANSION command. There will also be an API on the client to allow applications to handle expansion commands.
PENDING: This example is completely wrong.
Here is an example of what the MAL messages would look like for a simple synchronization. This example is intended only to give a general idea of what a session looks like. For convenience, it uses command names instead of tags and skips the HTTP wrappers and the data length value that should follow each command tag.
The first time a device synchronizes with a server, the following session occurs:
request message:
HELLO "joe" "16 byte hashed password" 325000 0
END
response message:
SERVERCONFIG "http://myserver.com/user_home_page" "message about the server"
server_URI
SENDDEVICEINFO true
DATABASECONFIG db0 SENDMODS false false
DATABASECONFIG db1 SENDALL false false
DATABASECONFIG db2 DONTSEND false false
COOKIE cookieData
NONCE nonce string
GOODBYE CALLAGAIN_STATUS
END
This session begins with the device sending the server a HELLO command. The HELLO message specifies the name and password of the user to be synchronized, how many free bytes are available on the device, and a cookie length of 0. Because this is the first message between the device and server, the cookie field of the HELLO command is not provided.
The server responds with a message that tells the device what databases it wants to synchronize. The first command, SERVERCONFIG, tells the device which server it is communicating with. SERVERCONFIG is followed by a SENDDEVICEINFO command, which requests device-specific information, and then three commands indicating that the server wants the modified records from db0, all of the records from db1, and none of the records from db2. Then, the server sends the device a cookie, and nonce value and a GOODBYE command indicating that it expects the device to call back.
This request and response complete the first session. The next session occurs immediately following the initial session and might look like this:
request message:
HELLO "joe" "16 byte hashed password" 325000 len cookieData
DEVICEINFO "WinCE" "2.0" 8 160 160 "106v12e..." "English" "ISOLatin1" 0
OPENDATABASE db0
RECORD rec1 ADDED ....
RECORD rec2 ADDED ....
...
RECORD recN ADDED ...
CLOSEDATABASE
OPENDATABASE db1
RECORD rec0 ADDED...
RECORD rec1 ADDED...
...
RECORD recN ADDED...
CLOSEDATABASE
END
response message:
OPENDATABASE db0
RECORD rec0 ADDED ...
RECORD rec10 DELETED ...
RECORD rec15 MODIFIED ...
CLEARMODS
CLOSEDATABASE
OPENDATABASE db1
RECORD rec100 ADDED ...
RECORD rec111 ADDED ...
...
CLEARMODS
CLOSEDATABASE
COOKIE newCookieData
NONCE new Nonce string
GOODBYE DONE_STATUS "Synchronization is complete"
END
This session again begins with a HELLO command. This time, the device sends back the cookieData that it received from the server in the last session. The HELLO command is followed by a DEVICEINFO command that tells the server necessary information about the device. This includes the OS name and version number, the color depth, the screen's dimensions, the device's serial number, and the language and character encoding the device supports. The server may use this information to determine how to format data for the device and to determine what types of data to send. Next, the device sends the modified records for db0 followed by all the records in the database db1. As shown above, each RECORD command contains a field specifying whether the record was modified, added, deleted, or unchanged since the last synchronization. Because this is the first synchronization, all records are listed as ADDED.
The server's response message begins by specifying that db0 should be opened for writing. What follows are a series of commands specifying records that should be added, deleted, or modified in db0. When the synchronization of db0 is complete, the server sends a CLEARMODS command to request that the client clear its change log or dirty bits for the records in db0 followed by the CLOSEDATABASE command. Next, the server opens db1 for writing, specifies the changed records in db1 and then clears the modifications and closes that database. Finally, the server sends a new cookie to the device and a GOODBYE message indicating the synchronization has been completed and generated no errors.
Note that if the same device synchronizes with the same server at a later date, the initial session does not have to be repeated. The device keeps track of which databases it synchronized previously and thus might immediately send the appropriate database records in the first request message. The server can then immediately perform all synchronization steps and respond with the appropriate commands.
This section describes the individual MAL commands. The heading of each command description below contains both the command tag as used by the protocol and the command name as used in this specification. This heading is followed by a list of the data the command takes. Each data item is shown as a typed value.
All data fields shown for each command are required unless the field is preceded by a field that specifies its length. For example, the HELLO command specifies a cookie field, which may or may not exist. The cookie field is preceded by a cookieLength field. If the cookie does not exist, you set the cookieLength to 0 and skip the cookie field. If the field is not preceded by a length field, you must specify that field.
The following commands can be sent only from the client (the mobile device) to the server.
string username
uint8[16] digestAuth
uint8[16] nonce
cint availableBytes
cint cookieLength
bytes cookieBegins a request message. This must be the first command in each request message, or the server returns an error.
- username
- The name the server uses to identify the user.
- digestAuth
- The server password and nonce with the md5 hash function run over them.
- nonce
- The nonce sent down by the server that was used to create the digestAuth.
- availableBytes
- Amount of storage available on the device.
- cookieLength
- Length in bytes of the cookie data. If there is no cookie, specify 0 for the cookieLength and skip the cookie field.
- cookie
- A series of bytes that the device previously received from the server in a COOKIE command.
PENDING: In the next revision clients will authenticate using a mechanism which is very similar to HTTP/1.1 Digest Authentication. See ftp://ftp.ietf.org/internet-drafts/draft-ietf-http-authentication-03.txt for more information.
string osName
string osVersion
cint colorDepth
cint screenWidth
cint screenHeight
string serialNumber
string language
string charset
cint platformDataLength
bytes platformDataContains information about the device.
- osName
- The name of the operating system used on the device. Possible values are
PalmOSorWinCE.- osVersion
- The version number of the operating system used on the device.
- colorDepth
- The device's color depth in bits.
- screenWidth
- The device's screen width in pixels.
- screenHeight
- The device's screen height in pixels.
- serialNumber
- A platform-specific string that uniquely identifies the device.
- language
- Describes the language that the device uses (i.e., English, Japanese).
- charset
- Describes the character set that can be displayed on the device. (See the MIME specification for valid values.)
- platformDataLength
- The length in bytes of platformData.
- platformData
- An opaque bag of bytes to be sent from the device to the server.
The DEVICEINFO command is sent in response to a SENDDEVICEINFO command from the server.
string dbName
Informs the server that no database with that name exists on the client. The client generates this command in response to a DATABASECONFIG command which specifies a database which is unknown to the client.
- dbName
- The name of the database which does not exist.
cint count
int32 temporaryId
int32 newId
...
PENDING: The NEWIDS command is a cint count followed by an array of int32s of count size. How should this be represented in the spec??
Reports the results of sending a RECORD down with a mod of NEW_TEMPORARY_UID. Sends the old temporaryId and the corresponding newId assigned by the device. The server must always reference this record by the newId that is reported here. The temporaryId is not valid when referencing RECORDs on the device.
- count
- The number of mappings being reported. There will be count number of integers in the command. This number should always be a multiple of two as there should be one temporaryId and one newId.
- temporaryId
- This is the UID the server sent down on a RECORD with a status of NEW_TEMPORARY_UID.
- newId
- This is the UID the device assigned to the RECORD.
The following commands can only be sent from the server to the client (device).
uint8 sendRequests that the device should send the DEVICEINFO command with information about the device.
- send
- Specify true to request device information; false otherwise.
string dbname
cint config
uint8 sendRecordPlatformData
cint platformDataLength
bytes platformData
Specifies which databases the client should send to the server during the next synchronization session.
- dbName
- The name of the device database.
- config
- Specifies which records to send:
SENDALL- The client should send all records in the database.
SENDMODS- The client should send only records since the last synchronization.
DONTSEND- The client should not send any records for this database.
- sendRecordPlatformData
- A Boolean value that indicates whether the device should send platform-dependent data when it sends records to the server.
- platformDataLength
- Length in bytes of platformData.
- platformData
- Platform dependant data needed to create the database if it doesn't exist.
PENDING: The format of the platform data needs to be documented for each platform
| Status of Device Database | config=SENDALL | config=SENDMOD |
|---|---|---|
| Doesn't exist | Send UNKNOWNDATABASE | Send UNKNOWNDATABASE |
| Contains no records | Send nothing | Send nothing |
| Contains modified records | Send all records | Send modified records |
| Contains no modified records | Send all records | Send nothing |
The first time a client communicates with the server, the server sends the device one or more DATABASECONFIG commands to specify which databases it is going to synchronize. This command might also be sent if the device or server have changed; that is, if the device has been through a hard reset or if the server has been updated.
When DATABASECONFIG commands are sent, they are typically the only contents of the message. That is, you wouldn't synchronize databases 1 through 5 and then send a DATABASECONFIG command for database 6 in the same message.
string userURL
string message
string serverURIContains information about the server.
- userURL
- The page the user should be sent to if they want to administer the server.
- message
- A string to display to the user when they are looking at the server information.
- serverURI
- A string to send in the HTTP request.
string nonce string
cint cookieLength
bytes cookie
- cookieLength
- The length of the cookie data.
- cookie
- The cookie.
For more information, see the section "Cookies."
string currentTask
- currentTask
- The task currently being performed.
Each server determines what a task is and which tasks to inform the client of. For example, the server may decide that the synchronization of each database is a separate task, and currentTask may be the name of the database currently being synchronized. The client might then display a dialog with the message "Synchronizing dbName."
It's common for such a dialog to display a progress bar indicating how much of the task has been completed. This is accomplished with the ITEM command.
PENDING: How language and character encodings are handled.
cint currentItemNumber
cint totalItemCount
string currentItem
Informs the client what item of the current task the server is performing. You can think of an item as a subtask.
- currentItemNumber
- This subtask's place in the list of subtasks. This value should be one more than the number of subtasks already completed.
- totalItemCount
- The total number of subtasks to this task.
- currentItem
- A string identifying the current subtask.
Each server determines what a task is and what subtasks it contains. For example, a task may be "synchronizing database1" and the subtask might be the updating of each individual record in the database. Suppose a database has 35 modified records. In this case, totalItemCount would be 35, currentItemNumber is the number of the record being updated, and currentItem could be a string describing the record.
The ITEM command relies on the TASK command to identify the task being performed and to prompt the client to display a dialog. If a TASK command doesn't precede the ITEM command, the client displays an error message and synchronization fails.
string dbName
Deletes the named database from the client.
- dbName
- The database to be deleted.
Tells the client that all modifications for the named database have been performed on the server side, so the client may clear any dirty bits or change log it has for the records of that database. This command is only valid inside an OPENDATABASE command and always has a length of zero.
cint syncStatus
cint errorCode
string errorMessage
Sent by the server to end the session.
- syncStatus
- Indicates whether synchronization is complete. This can be one of the following:
DONE_STATUS- Synchronization is complete.
CALLAGAIN_STATUS- Synchronization is not complete. The device should contact the server again to complete the synchronization.
- errorCode
- TBD
- errorMessage
- A message that should be displayed to the user after the synchronization is complete.
GOODBYE does not end the message itself. The GOODBYE command must be immediately followed by an END command to end the message. GOODBYE must be the last command before END.
The commands listed in this section can be sent in either direction.
Marks the end of a sequence of commands. The END command is not followed by a length field of 0.
Denotes a user-defined command. User-defined commands are passed through the system untouched. The next value after command tag 1 is taken to be the length of the command's data as usual. The following length number of bytes is taken to be the data for the EXPANSION command.
PENDING: There may or may not be sub-structure to the EXPANSION command.
string dbName
Opens the database on the device.
- dbName
- The database to be opened.
This command is followed by a series of RECORD commands. The RECORD command contains the information as to whether the RECORD is to be modified, added, or deleted. The last record command is followed by a CLOSEDATABASE command. If this command is sent from a server to a client and the database does not exist, the client will use the information from a previous DATABASECONFIG command to create the database.
Closes the last database opened by a OPENDATABASE command. The length of this command is always 0. If an OPENDATABASE command isn't eventually followed by a CLOSEDATABASE command, a synchronization error occurs.
uint32 uid
cint mod
cint recordDataLength
bytes recordData
cint platformDataLength
bytes platformData
Information about a record.
- uid
- A 32-bit unique identifier for the record.
- mod
- Specifies the modification status of the record, which can be one of the following:
UNMODIFIED- The record has not been changed since the last synchronization.
MODIFIED- Some or all of the record's data has been changed since the last synchronization.
NEW- The record has been added since the last synchronization.
DELETED- The record has been deleted since the last synchronization.
NEW_TEMPORARY_UID- The record has been added since the last synchronization and the record's UID is a temporary identifier. The device should report the actual UID assigned by the device using the NEWIDS command.
- recordDataLength
- The length in bytes of recordData.
- recordData
- The record's contents.
- platformDataLength
- The length in bytes of platformData.
- platformData
- An opaque bag of bytes to be sent along with the record data.
A RECORD command can be sent from client to server or from server to client. RECORD commands often appear in succession. A series of RECORD commands must be preceded with OPENDATABASE and ended with CLOSEDATABASE.