Berkeley DB Programmer's Reference Guide

Legal Notice

This documentation is distributed under an open source license. You may review the terms of this license at:

Oracle, Berkeley DB, and Sleepycat are trademarks or registered trademarks of Oracle. All rights to these marks are reserved. No third-party use is permitted without the express prior written consent of Oracle.

Other names may be trademarks of their respective owners.

To obtain a copy of this document's original source code, please submit a request to the Oracle Technology Network forum at:


Table of Contents

Conventions Used in this Book
For More Information
Contact Us
1. Introduction
An introduction to data management
Mapping the terrain: theory and practice
Data access and data management
Relational databases
Object-oriented databases
Network databases
Clients and servers
What is Berkeley DB?
Data Access Services
Data management services
What Berkeley DB is not
Berkeley DB is not a relational database
Berkeley DB is not an object-oriented database
Berkeley DB is not a network database
Berkeley DB is not a database server
Do you need Berkeley DB?
What other services does Berkeley DB provide?
What does the Berkeley DB distribution include?
Where does Berkeley DB run?
The Berkeley DB products
Berkeley DB Data Store
Berkeley DB Concurrent Data Store
Berkeley DB Transactional Data Store
Berkeley DB High Availability
2. Access Method Configuration
What are the available access methods?
Selecting an access method
Btree or Heap?
Hash or Btree?
Queue or Recno?
Logical record numbers
General access method configuration
Selecting a page size
Selecting a cache size
Selecting a byte order
Duplicate data items
Non-local memory allocation
Btree access method specific configuration
Btree comparison
Btree prefix comparison
Minimum keys per page
Retrieving Btree records by logical record number
Hash access method specific configuration
Page fill factor
Specifying a database hash
Hash table size
Heap access method specific configuration
Queue and Recno access method specific configuration
Managing record-based databases
Selecting a Queue extent size
Flat-text backing files
Logically renumbering records
3. Access Method Operations
Database open
Opening multiple databases in a single file
Configuring databases sharing a file
Caching databases sharing a file
Locking in databases based on sharing a file
Partitioning databases
Specifying partition keys
Partitioning callback
Placing partition files
Retrieving records
Storing records
Deleting records
Database statistics
Database truncation
Database upgrade
Database verification and salvage
Flushing the database cache
Database close
Secondary indexes
Error Handling With Secondary Indexes
Foreign key indexes
Cursor operations
Retrieving records with a cursor
Storing records with a cursor
Deleting records with a cursor
Duplicating a cursor
Equality Join
Data item count
Cursor close
4. Access Method Wrapup
Data alignment
Retrieving and updating records in bulk
Bulk retrieval
Bulk updates
Bulk deletes
Partial record storage and retrieval
Storing C/C++ structures/objects
Retrieved key/data permanence for C/C++
Error support
Cursor stability
Database limits
Disk space requirements
Specifying a Berkeley DB schema using SQL DDL
Access method tuning
Access method FAQ
5. Java API
Java configuration
Java programming notes
Java FAQ
6. C# API
7. Standard Template Library API
Dbstl introduction
Standards compatible
Performance overhead
Dbstl typical use cases
Dbstl examples
Berkeley DB configuration
Registering database and environment handles
Truncate requirements
Auto commit support
Database and environment identity checks
Products, constructors and configurations
Using advanced Berkeley DB features with dbstl
Using bulk retrieval iterators
Using the DB_RMW flag
Using secondary index database and secondary containers
Using transactions in dbstl
Using dbstl in multithreaded applications
Working with primitive types
Storing strings
Store and Retrieve data or objects of complex types
Storing varying length objects
Storing arbitrary sequences
Dbstl persistence
Direct database get
Change persistence
Object life time and persistence
Dbstl container specific notes
db_vector specific notes
Associative container specific notes
Using dbstl efficiently
Using iterators efficiently
Using containers efficiently
Dbstl memory management
Freeing memory
Type specific notes
Dbstl miscellaneous notes
Special notes about trivial methods
Using correct container and iterator public types
Dbstl known issues
8. Berkeley DB Architecture
The big picture
Programming model
Programmatic APIs
Dbm/Ndbm, Hsearch
Scripting languages
Supporting utilities
9. The Berkeley DB Environment
Database environment introduction
Creating a database environment
Sizing a database environment
Opening databases within the environment
Error support
DB_CONFIG configuration file
File naming
Specifying file naming to Berkeley DB
Filename resolution in Berkeley DB
Shared memory regions
Remote filesystems
Environment FAQ
10. Berkeley DB Concurrent Data Store Applications
Concurrent Data Store introduction
Handling failure in Data Store and Concurrent Data Store applications
Architecting Data Store and Concurrent Data Store applications
11. Berkeley DB Transactional Data Store Applications
Transactional Data Store introduction
Why transactions?
Handling failure in Transactional Data Store applications
Architecting Transactional Data Store applications
Opening the environment
Opening the databases
Recoverability and deadlock handling
Degrees of isolation
Snapshot Isolation
Transactional cursors
Nested transactions
Environment infrastructure
Deadlock detection
Database and log file archival
Log file removal
Recovery procedures
Hot failover
Using Recovery on Journaling Filesystems
Recovery and filesystem operations
Berkeley DB recoverability
Transaction tuning
Transaction throughput
Transaction FAQ
12. Berkeley DB Replication
Replication introduction
Replication environment IDs
Replication environment priorities
Building replicated applications
Replication Manager methods
Base API Methods
Building the communications infrastructure
Connecting to a new site
Managing Replication Manager Group Membership
Adding Sites to a Replication Group
Removing Sites from a Replication Group
Primordial Startups
Upgrading Groups
Managing Replication Files
Running Replication Manager in multiple processes
One replication process and multiple subordinate processes
Persistence of local site network address configuration
Programming considerations
Handling failure
Other miscellaneous rules
Running Replication using the db_replicate Utility
One Replication Process and Multiple Subordinate Processes
Common Use Case
Avoiding Rollback
When to Consider an Integrated HA Application
Choosing a Replication Manager Ack Policy
Synchronizing with a master
Delaying client synchronization
Client-to-client synchronization
Blocked client operations
Clients too far out-of-date to synchronize
Initializing a new site
Bulk transfer
Transactional guarantees
Master Leases
Changing Group Size
Read your writes consistency
Getting a token
Token handling
Using a token to check or wait for a transaction
Clock Skew
Using Replication Manager message channels
Sending messages over a message channel
Receiving messages
Special considerations for two-site replication groups
Network partitions
Replication FAQ
Ex_rep: a replication example
Ex_rep_base: a TCP/IP based communication infrastructure
Ex_rep_base: putting it all together
Ex_rep_chan: a Replication Manager channel example
13. Distributed Transactions
Berkeley DB XA Implementation
Building a Global Transaction Manager
Communicating with multiple Berkeley DB environments
Recovering from GTM failure
Managing the Global Transaction ID (GID) name space
Maintaining state for each distributed transaction.
Recovering from the failure of a single environment
Recovering from GTM failure
XA Introduction
Configuring Berkeley DB with the Tuxedo System
Update the Resource Manager File in Tuxedo
Build the Transaction Manager Server
Update the UBBCONFIG File
Restrictions on XA Transactions
XA: Frequently Asked Questions
14. Application Specific Logging and Recovery
Introduction to application specific logging and recovery
Defining application-specific log records
Automatically generated functions
Application configuration
15. Programmer Notes
Signal handling
Error returns to applications
Environment variables
Multithreaded applications
Berkeley DB handles
Name spaces
C Language Name Space
Filesystem Name Space
Memory-only or Flash configurations
Disk drive caches
Copying or moving databases
Compatibility with historic UNIX interfaces
Run-time configuration
Performance Event Monitoring
Using the DTrace Provider
Using SystemTap
Example Scripts
Performance Events Reference
Programmer notes FAQ
16. The Locking Subsystem
Introduction to the locking subsystem
Configuring locking
Configuring locking: sizing the system
Standard lock modes
Deadlock detection
Deadlock detection using timers
Deadlock debugging
Locking granularity
Locking without transactions
Locking with transactions: two-phase locking
Berkeley DB Concurrent Data Store locking conventions
Berkeley DB Transactional Data Store locking conventions
Locking and non-Berkeley DB applications
17. The Logging Subsystem
Introduction to the logging subsystem
Configuring logging
Log file limits
18. The Memory Pool Subsystem
Introduction to the memory pool subsystem
Configuring the memory pool
Warming the memory pool
The warm_cache() function
19. The Transaction Subsystem
Introduction to the transaction subsystem
Configuring transactions
Transaction limits
Transaction IDs
Multiple Threads of Control
20. Sequences
21. Berkeley DB Extensions: Tcl
Loading Berkeley DB with Tcl
Installing as a Tcl Package
Loading Berkeley DB with Tcl
Using Berkeley DB with Tcl
Tcl API programming notes
Tcl error handling
22. Berkeley DB Extensions
Using Berkeley DB with Apache
Using Berkeley DB with Perl
Using Berkeley DB with PHP
23. Dumping and Reloading Databases
The db_dump and db_load utilities
Dump output formats
Loading text into databases
24. Additional References
Additional references
Technical Papers on Berkeley DB
Background on Berkeley DB Features
Database Systems Theory