By John Efstathiades, Principal Engineer, Pebble Bay Consulting

Many embedded applications require a database, for example to store configuration data or the results of calculations or measurements. Using an embedded database allows data to be stored in a standard format that can be easily read and manipulated. In particular, databases based on the SQL relational language have become popular, and there are many tools available to retrieve and analyse data in SQL-compatible formats.

For embedded applications, simplicity and compactness are key criteria. SQLite is an open source relational database engine with a SQL-compatible query mechanism that meets these requirements, yet provides the power needed for sophisticated applications. This makes it ideal for the embedded world, which has led SQLite to become the one of the most widely-deployed SQL database engines. It is used in the Apple Mac, the Symbian operating system, the Firefox web browser, and software by Adobe, Google, McAfee, Skype and many others.

One popular real-time operating system (RTOS) for embedded systems is Green Hills Software’s INTEGRITY®. Pebble Bay undertook a project to port SQLite to this operating system; this article describes the way this was implemented, and in particular the development of effective test routines. To the best of our knowledge, this is the first commercial implementation of SQLite running on INTEGRITY.

Porting SQLite

INTEGRITY is a good candidate to run SQLite because it provides a POSIX API (application programming interface). SQLite comes with a POSIX support layer as standard, which means the porting process is simplified by re-using this existing support layer and linking it with INTEGRITY’s POSIX support. Porting SQLite involved creating a number of OS-dependent support routines, and fixing a number of linkage issues related to the way INTEGRITY virtual address space applications are built. An intimate understanding of INTEGRITY’s approach to memory management proved vital in solving these problems.

On the target system, the SQLite library is linked into an application process running in its own virtual address space container. The application process can access the database files by making direct calls to the SQLite C-language API or by using C++ methods that provide a wrapper around the C API functions and objects. In addition, it is possible to use the standard command line interface to SQLite (via a serial or telnet connection to the target) to access the database to store and retrieve data, and perform database maintenance such as a backup and restore.

Pebble Bay ported the latest version of SQLite to INTEGRITY running on an embedded board based on an ARM9 processor. The SQLite amalgamated source file was used as the starting point. Pebble Bay then created Green Hills MULTI® projects to build the SQLite library as a static archive and also as a POSIX DLL. The MULTI debugger was also used extensively to identify and correct problems found during the porting and testing process.

Testing and verification

Once the port was complete, it was necessary to show that it was working correctly, and to provide a test framework for embedded database applications on INTEGRITY.

To test the basic operation of the SQLite port, Pebble Bay ran a demonstration application which used the core API of SQLite to access, store and retrieve data from a database. Beyond this initial confidence check, further rigorous testing was required to ensure the SQLite port was functioning correctly.

SQLite comes with an extensive regression test suite that exercises the programming APIs and performs simulations of unexpected events such as file system errors, memory exhaustion or power failure. It includes many thousands of tests, providing a very thorough validation of the SQLite implementation.

The freely available SQLite test suite is written in the Tcl scripting language and requires a Tcl library and interpreter on the target system, which is typically available on Windows and Linux workstations. This test suite is used by the developers as they maintain and update the SQLite code base.

Another test harness is TH3, which consists of proprietary tests written in C and uses only the published SQLite interfaces. TH3 was developed by SQLite architect and maintainer Richard Hipp to run on resource-constrained embedded targets that do not provide typical workstation services and tools. For this project the TH3 test harness was considered the best choice to test and verify the operation of the ported SQLite software.

TH3 takes a series of test description files, based in part on the tests from the Tcl test suite, and generates C test programs that can be complied and downloaded to an embedded system to verify that SQLite is working correctly. This also aims to give sufficient test coverage to enable SQLite to be used in safety-critical systems such as in medical or military applications. Code coverage provided by TH3 is 100% MC/DC (Modified Condition/Decision Coverage) over the SQLite core component and approaching this for the non-core components. Using TH3 does require a license to be purchased (which is available from Richard Hipp at www.sqlite.org/th3.html).

Pebble Bay used TH3 to generate over thirty specific test programs that ran through many tens of thousands of tests, and which took approximately fifteen hours to complete on the target system. The generated test code exercised various aspects of the SQLite port, including handling of standard SQL transactions but also out-of-memory errors, disk I/O errors, and simulation of power loss while committing transactions. In testing on the target system, TH3 found a number of configuration problems as well as operating-system and compiler specific issues such as handling of floating point NaN (Not-a-Number) conditions, which needed be corrected in the port.

Conclusions

INTEGRITY’s POSIX support makes it a relatively simple target for porting SQLite. The real challenge was in producing comprehensive and thorough test and validation software. In our work on this project, Pebble Bay was able to develop a robust and thoroughly tested port of SQLite that can be used for embedded applications running on INTEGRITY, enabling them to benefit from SQLite’s simplicity and small size. Our knowledge of INTEGRITY’s operation and our experience in using the MULTI development tools enabled us to complete the port to a high quality, on time and within budget.

“Porting SQLite to the INTEGRITY operating system will further broaden its appeal as a database in embedded applications,” said Richard Hipp, creator of SQLite. “Pebble Bay did an excellent job, demonstrating their understanding of embedded databases and their ability to work with middleware.”

More information about SQLite is available at www.sqlite.org.

More information about the SQLite port to INTEGRITY can be obtained by contacting Pebble Bay Consulting.

Green Hills Software, MULTI and INTEGRITY are trademarks or registered trademarks of Green Hills Software, Inc. in the U.S. and/or internationally. All other trademarks are the property of their respective owners.