Package com.sleepycat.bind.tuple

package com.sleepycat.bind.tuple
Bindings that use sequences of primitive fields, or tuples.

Tuple Formats

The serialization format for tuple bindings are designed for compactness, serialization speed and proper default sorting.

When a format is used for database keys, it is important to use default sorting for best performance. Although a custom comparator may be specified for a database or invalid input: '{@link <a href="{@docRoot'}/com/sleepycat/persist/model/KeyField.html#comparable">entity index}, custom comparators often reduce performance because comparators are called very frequently during Btree operations.

For proper default sorting, the byte array of the stored format must be designed so that a byte-by-byte unsigned comparison results in the natural sort order, as defined by the Comparable.compareTo(T) method of the data type. For example, the natural sort order for integers is the standard mathematical definition, and is implemented by Integer.compareTo, Long.compareTo, etc. This is called default natural sorting.

Although most tuple formats provide default natural sorting, not all of them do. Certain formats do not provide default natural sorting for historical reasons (see the discussion of packed integer and float formats below.) Other formats sacrifice default natural sorting for other performance factors (see the discussion of BigDecimal formats below.)

String Formats

All String formats support default natural sorting.

Strings are stored as a byte array of UTF encoded characters, either where the length must be known by the application, or the byte array is zero-terminated. The UTF encoding is described below.

  • Null strings are UTF encoded as { 0xFF }, which is not allowed in a standard UTF encoding. This allows null strings, as distinct from empty or zero length strings, to be represented. Using default sorting, null strings will be ordered last.
  • Zero (0x0000) character values are UTF encoded as non-zero values, and therefore embedded zeros in the string are supported. The sequence { 0xC0, 0x80 } is used to encode a zero character. This UTF encoding is the same one used by the native Java UTF libraries and is called Modified UTF-8. However, this encoding of zero does impact the lexicographical ordering, and zeros will not be sorted first (the natural order) or last.
  • For all character values other than zero, the standard UTF encoding is used, and the default sorting is the same as the Unicode lexicographical character ordering.

Binding classes and methods are provided for zero-terminated and known-length String values.

Integer Formats

Fixed Size Integer Formats

All fixed size integer formats support default natural sorting.

The size of the stored value depends on the type, and ranges (as one would expect) from 1 byte for type byte and class Byte, to 8 bytes for type long and class Long.

Signed numbers are stored in the buffer in MSB (most significant byte first) order with their sign bit (high-order bit) inverted to cause negative numbers to be sorted first when comparing values as unsigned byte arrays, as done in a database.

Unsigned numbers, including characters, are stored in MSB order with no change to their sign bit. Arrays of characters and unsigned bytes may also be stored and may be treated as String values. For booleans, true is stored as the unsigned byte value one and false as the unsigned byte value zero.

Packed Integer Formats

The packed integer format stores integers with small absolute values in a single byte. The size increases as the absolute value increases, up to a maximum of 5 bytes for int values and 9 bytes for long values.

The packed integer format can be used for integer values between Long.MIN_VALUE and Long.MAX_VALUE. However, different bindings and methods are provided for type int and long, to avoid unsafe casting from long to int when int values are used.

Because the same packed format is used for int and long values, stored int values may be expanded to long values without introducing a format incompatibility. In other words, you can treat previously stored packed int values as packed long values.

Packed integer formats come in two varieties: those that support default natural sorting and those that don't. The formats of the two varieties are incompatible. For new applications, the format that supports default natural sorting should normally be used. There is no performance advantage to using the unsorted format.

The format with support for default natural sorting stores values in the inclusive range [-119,120] in a single byte.

The unsorted packed integer format is an older, legacy format that is used internally and supported for compatibility. It stores values in the inclusive range [-119,119] in a single byte. Because default natural sorting is not supported, this format should not be used for keys. However, it so happens that packed integers in the inclusive range [0,630] are sorted correctly by default, and this may be useful for some applications.

BigInteger Formats

All BigInteger formats support default natural sorting.

BigInteger values are variable length and are stored as signed values with a preceding byte length. The length has the same sign as the value, in order to support default natural sorting.

The length is stored as a 2-byte (short), fixed size, signed integer. Supported values are therefore limited to those with a byte array (BigInteger.toByteArray()) representation with a size of 0x7fff bytes or less. The maximum BigInteger value is (20x3fff7 - 1) and the minimum value is (-20x3fff7).

Floating Point Formats

Floats and doubles are stored in a fixed size, 4 and 8 byte format, respectively. Floats and doubles are stored using two different representations: a representation with default natural sorting, and an unsorted, integer-bit (IEEE 754) representation. For new applications, the format that supports default natural sorting should normally be used. There is no performance advantage to using the unsorted format.

For float values, Float.floatToIntBits and the following bit manipulations are used to convert the signed float value to a representation that is sorted correctly by default.

 int intVal = Float.floatToIntBits(val);
 intVal ^= (intVal < 0) ? 0xffffffff : 0x80000000;

For double values, Float.doubleToLongBits and the following bit manipulations are used to convert the signed double value to a representation that is sorted correctly by default.

 long longVal = Double.doubleToLongBits(val);
 longVal ^= (longVal < 0) ? 0xffffffffffffffffL : 0x8000000000000000L;

In both cases, the resulting int or long value is stored as an unsigned value.

The unsorted floating point format is an older, legacy format that is supported for compatibility. With this format, only zero and positive values have default natural sorting; negative values do not.

BigDecimal Formats

BigDecimal values are stored using two different, variable length representations: a representation that supports default natural sorting, and an unsorted representation. Differences between the two formats are:

  • The BigDecimal format with default natural sorting should normally be used for database keys.
    • Default natural sorting is supported.
    • The stored value is around 3 bytes larger than the unsorted format, more or less, and is a minimum of 8 bytes.
    • More computation is required for serialization than the unsorted format.
    • Trailing zeros after the decimal place are stripped, meaning that precision is not preserved.
  • The unsorted BigDecimal format should normally be used for non-key values.
    • Default natural sorting is not supported.
    • The stored value is around 3 bytes smaller than the sorted format, more or less, and is a minimum of 3 bytes.
    • Less computation is required for serialization than the sorted format.
    • Trailing zeros after the decimal place are preserved, meaning that precision is preserved.

Both formats store the scale or exponent separately from the unscaled value, and the stored size does not increase proportionally as the absolute value of the scale or exponent increases.

  • Class
    Description
    BigDecimalBinding
    A concrete TupleBinding for an unsorted BigDecimal value.
    BigIntegerBinding
    A concrete TupleBinding for a BigInteger value.
    BooleanBinding
    A concrete TupleBinding for a Boolean primitive wrapper or a boolean primitive.
    ByteBinding
    A concrete TupleBinding for a Byte primitive wrapper or a byte primitive.
    CharacterBinding
    A concrete TupleBinding for a Character primitive wrapper or a char primitive.
    DoubleBinding
    A concrete TupleBinding for an unsorted Double primitive wrapper or an unsorted double primitive.
    FloatBinding
    A concrete TupleBinding for an unsorted Float primitive wrapper or an unsorted float primitive.
    IntegerBinding
    A concrete TupleBinding for a Integer primitive wrapper or an int primitive.
    LongBinding
    A concrete TupleBinding for a Long primitive wrapper or a long primitive.
    MarshalledTupleEntry
    A marshalling interface implemented by key, data or entity classes that are represented as tuples.
    MarshalledTupleKeyEntity
    A marshalling interface implemented by entity classes that represent keys as tuples.
    PackedIntegerBinding
    A concrete TupleBinding for an unsorted Integer primitive wrapper or an unsorted int primitive, that stores the value in the smallest number of bytes possible.
    PackedLongBinding
    A concrete TupleBinding for an unsorted Long primitive wrapper or an unsorted long primitive, that stores the value in the smallest number of bytes possible.
    ShortBinding
    A concrete TupleBinding for a Short primitive wrapper or a short primitive.
    SortedBigDecimalBinding
    A concrete TupleBinding for a sorted BigDecimal value.
    SortedDoubleBinding
    A concrete TupleBinding for a sorted Double primitive wrapper or a sorted double primitive.
    SortedFloatBinding
    A concrete TupleBinding for a sorted Float primitive wrapper or sorted a float primitive.
    SortedPackedIntegerBinding
    A concrete TupleBinding for a sorted Integer primitive wrapper or a sorted int primitive, that stores the value in the smallest number of bytes possible.
    SortedPackedLongBinding
    A concrete TupleBinding for a sorted Long primitive wrapper or a sorted long primitive, that stores the value in the smallest number of bytes possible.
    StringBinding
    A concrete TupleBinding for a simple String value.
    TupleBase<E>
    A base class for tuple bindings and tuple key creators that provides control over the allocation of the output buffer.
    TupleBinding<E>
    An abstract EntryBinding that treats a key or data entry as a tuple; it includes predefined bindings for Java primitive types.
    TupleInput
    An InputStream with DataInput-like methods for reading tuple fields.
    TupleInputBinding
    A concrete EntryBinding that uses the TupleInput object as the key or data object.
    TupleMarshalledBinding<E extends MarshalledTupleEntry>
    A concrete TupleBinding that delegates to the MarshalledTupleEntry interface of the data or key object.
    TupleOutput
    An OutputStream with DataOutput-like methods for writing tuple fields.
    TupleTupleBinding<E>
    An abstract EntityBinding that treats an entity's key entry and data entry as tuples.
    TupleTupleKeyCreator<E>
    An abstract key creator that uses a tuple key and a tuple data entry.
    TupleTupleMarshalledBinding<E extends MarshalledTupleEntry & MarshalledTupleKeyEntity>
    A concrete TupleTupleBinding that delegates to the MarshalledTupleEntry and MarshalledTupleKeyEntity interfaces of the entity class.
    TupleTupleMarshalledKeyCreator<E extends MarshalledTupleEntry & MarshalledTupleKeyEntity>
    A concrete key creator that works in conjunction with a TupleTupleMarshalledBinding.