There is a couple of advantage of supporting CORK system with BDB. First, stability and manageability will be increased. Because, instead of creating many storage files for each shared object (this approach has been used for the current CORK storage system), BDB builds one big database file, therefore, the system will be easier to handle and more stable for large datasets. Second, the access speed will be increased. The implemented database uses the Btree data structure. For a large data set, Btree can retrieve datas faster than hash structure that has been used for current CORK, theoretically; O(log n) vs.O(n). Moreover, by connecting with BDB, we prove the portability of CORK system. The CORK system will be easily connected with many other different databases in back-end by the similar approach of BDB and CORK connection.

There are some other issues regarding CORK and BDB.


Q: Thread safety: Are there any threading issues with the underlying DB? The server currently performs all writes in a single thread, but reads of different objects can occur in different threads. This could be restricted if necessary, but if the DB is thread-safe it would be better to allow this kind of concurrency.

A: BDB fully supports multithreaded applications and Locking. The Locking system provides multiple reader or single writer access to objects. The Locking system is used extensively by the BDB Access Methods to acquire the right to read or write database pages. Threading is always assumed in the BDB Java API, so no special flags are required, and underlying BDB functions will always behave as if the DB_THREAD flag was specified; that is usable by multiple threads within a single address space.

Q: Fault tolerance: One big DB file is great -- unless it gets corrupted. I know the BDB has recovery capabilities, but do we have to do anything to make that work? Is there a way that we can test this? (Even some unscientific testing would be good: start up a server, load a bunch of objects, and terminate the server with Ctrl-C or the "kill" command. Repeat numerous times).

A: Regarding fault tolerance, it requires using Transactions and Logging for BDB data structures in order to support recoverability from system or application failure. The Transaction and Logging systems provide recoverability and atomicity for multiple database operations. The Transaction system ensures that database operations may be undone or redone in the case of applications or system failure.

Although Transaction and Logging system is not fully available, the current DB implementation supports the fault tolerance for system or application failure to some degree. By using Write-Through Cache protocol, the database is able to ensure, when the system or application fails (terminate the server with Ctrl-C or the "kill" command), previously saved data is available the next time the application runs.

In future, we need to develop Transaction subsystem in order to make operations fully atomic, consistent, isolated, and durable in the face of system and application failures.

Q: Class evolution: If a class definition changes, does anything need to be done to make the objects stored under the old class definition accessible?

A: Using JSX package is the solution for this problem and this work has been done to support the evolution of the classes for CORK system.

Q: Speed: If you haven't done so already, could you put together a simple test driver that creates and populates a large number of objects, sleeps for a bit (until the server has flushed everything from memory) and then re-accesses the objects? This could be used for timing both the current file-based implementation and your new DB-based implementation. Hopefully we aren't actually O(n) for accesses now, but it would be nice to be able to quantify how much of a speed improvement the new DB gives us.

A: It will be available to get the access speed benchmark test results by building a test driver on the machine.

Q: Platform dependence: Are the database files platform independent? If not, are there any mechanisms for converting from one platform to another?

A: There are some minor differences for building BDB on Windows and Unix. However, once installed, database will be accessible platform-independently. You can refer to the section, "How to build BDB", for building on both Windows and Unix machine.



Back to Berkeley DB (Sleepycat) Storage

/public/projects/bridge/design/Some issues and future work regarding BDB supports for CORK Login | Web Editor | Full Editor
Last modified 9/11/03 1:11 PM by isenhour (history)
Site contents