-
- All Implemented Interfaces:
-
java.io.Closeable,java.lang.AutoCloseable
public final class Buffer.UnsafeCursor implements Closeable
A handle to the underlying data in a buffer. This handle is unsafe because it does not enforceits own invariants. Instead, it assumes a careful user who has studied MobonOkio's implementationdetails and their consequences.
Buffer InternalsMost code should use
{@code Buffer}as a black box: a class that holds 0 or more bytes ofdata with efficient APIs to append data to the end and to consume data from the front. Usuallythis is also the most efficient way to use buffers because it allows MobonOkio to employ severaloptimizations, including:- Fast Allocation: Buffers use a shared pool of memory that is notzero-filled before use.
- Fast Resize: A buffer's capacity can change without copying itscontents.
- Fast Move: Memory ownership can be reassigned from one buffer toanother.
- Fast Copy: Multiple buffers can share the same underlying memory.
- Fast Encoding and Decoding: Common operations like UTF-8 encoding anddecimal decoding do not require intermediate objects to be allocated.
These optimizations all leverage the way MobonOkio stores data internally. MobonOkio Buffers areimplemented using a doubly-linked list of segments. Each segment is a contiguous range within a8 KiB
{@code byte[]}. Each segment has two indexes,{@code start}, the offset of the firstbyte of the array containing application data, and{@code end}, the offset of the first bytebeyond{@code start}whose data is undefined.New buffers are empty and have no segments:
We append 7 bytes of data to the end of our empty buffer. Internally, the buffer allocates asegment and writes its new data there. The lone segment has an 8 KiB byte array but only 7bytes of data:{@code * * Buffer buffer = new Buffer(); * }
When we read 4 bytes of data from the buffer, it finds its first segment and returns that datato us. As bytes are read the data is consumed. The segment tracks this by adjusting itsinternal indices.{@code * * buffer.writeUtf8("sealion"); * * // [ 's', 'e', 'a', 'l', 'i', 'o', 'n', '?', '?', '?', ...] * // ^ ^ * // start = 0 end = 7 * }
As we write data into a buffer we fill up its internal segments. When a write doesn't fit intoa buffer's last segment, additional segments are allocated and appended to the linked list ofsegments. Each segment has its own start and end indexes tracking where the user's data beginsand ends.{@code * * buffer.readUtf8(4); // "seal" * * // [ 's', 'e', 'a', 'l', 'i', 'o', 'n', '?', '?', '?', ...] * // ^ ^ * // start = 4 end = 7 * }
The start index is always inclusive and the end index is alwaysexclusive. The data preceding the start index is undefined, and the dataat and following the end index is undefined.{@code * * Buffer xoxo = new Buffer(); * xoxo.writeUtf8(Strings.repeat("xo", 5_000)); * * // [ 'x', 'o', 'x', 'o', 'x', 'o', 'x', 'o', ..., 'x', 'o', 'x', 'o'] * // ^ ^ * // start = 0 end = 8192 * // * // [ 'x', 'o', 'x', 'o', ..., 'x', 'o', 'x', 'o', '?', '?', '?', ...] * // ^ ^ * // start = 0 end = 1808 * }After the last byte of a segment has been read, that segment may be returned to an internalsegment pool. In addition to reducing the need to do garbage collection, segment pooling alsosaves the JVM from needing to zero-fill byte arrays. MobonOkio doesn't need to zero-fill its arraysbecause it always writes memory before it reads it. But if you look at a segment in a debuggeryou may see its effects. In this example, one of the "xoxo" segments above is reused in anunrelated buffer:
There is an optimization in{@code * * Buffer abc = new Buffer(); * abc.writeUtf8("abc"); * * // [ 'a', 'b', 'c', 'o', 'x', 'o', 'x', 'o', ...] * // ^ ^ * // start = 0 end = 3 * }{@code Buffer.clone()}and other methods that allows two segmentsto share the same underlying byte array. Clones can't write to the shared byte array; insteadthey allocate a new (private) segment early.
Segments are not shared when the shared region is small (ie. less than 1 KiB). This is intendedto prevent fragmentation in sharing-heavy use cases.Unsafe Cursor API{@code * * Buffer nana = new Buffer(); * nana.writeUtf8(Strings.repeat("na", 2_500)); * nana.readUtf8(2); // "na" * * // [ 'n', 'a', 'n', 'a', ..., 'n', 'a', 'n', 'a', '?', '?', '?', ...] * // ^ ^ * // start = 0 end = 5000 * * nana2 = nana.clone(); * nana2.writeUtf8("batman"); * * // [ 'n', 'a', 'n', 'a', ..., 'n', 'a', 'n', 'a', '?', '?', '?', ...] * // ^ ^ * // start = 0 end = 5000 * // * // [ 'b', 'a', 't', 'm', 'a', 'n', '?', '?', '?', ...] * // ^ ^ * // start = 0 end = 7 * }This class exposes privileged access to the internal byte arrays of a buffer. A cursoreither references the data of a single segment, it is before the first segment (
{@code * offset == -1}), or it is after the last segment ({@code offset == buffer.size}).Call seek to move the cursor to the segment that contains a specified offset. Afterseeking, data references the segment's internal byte array, start is thesegment's start and end is its end.
Call next to advance the cursor to the next segment. This returns -1 if there areno further segments in the buffer.
Use readUnsafe to create a cursor to read buffer data and to create a cursor to read and write buffer data. In either case,always call close when done with a cursor. This is convenient with Java 7'stry-with-resources syntax. In this example we read all of the bytes in a buffer into a bytearray:
{@code * * byte[] bufferBytes = new byte[(int) buffer.size()]; * * try (UnsafeCursor cursor = buffer.readUnsafe()) { * while (cursor.next() != -1) { * System.arraycopy(cursor.data, cursor.start, * bufferBytes, (int) cursor.offset, cursor.end - cursor.start); * } * } * }Change the capacity of a buffer with resizeBuffer. This is only permitted forread+write cursors. The buffer's size always changes from the end: shrinking it removes bytesfrom the end; growing it adds capacity to the end.
WarningsMost application developers should avoid this API. Those that must use this API shouldrespect these warnings.
Don't mutate a cursor. This class has public, non-final fields because thatis convenient for low-level I/O frameworks. Never assign values to these fields; instead usethe cursor API to adjust these.
Never mutate
{@code data}unless you have read+write access. You are on thehonor system to never write the buffer in read-only mode. Read-only mode may be more efficientthan read+write mode because it does not need to make private copies of shared segments.Only access data in
{@code [start..end)}. Other data in the byte arrayis undefined! It may contain private or sensitive data from other parts of your process.Always fill the new capacity when you grow a buffer. New capacity is notzero-filled and may contain data from other parts of your process. Avoid leaking thisinformation by always writing something to the newly-allocated capacity. Do not assume thatnew capacity will be filled with
{@code 0}; it will not be.Do not access a buffer while is being accessed by a cursor. Even simpleread-only operations like clone are unsafe because they mark segments as shared.
Do not hard-code the segment size in your application. It is possible thatsegment sizes will change with advances in hardware. Future versions of MobonOkio may even haveheterogeneous segment sizes.
These warnings are intended to help you to use this API safely. It's here for developersthat need absolutely the most throughput. Since that's you, here's one final performance tip.You can reuse instances of this class if you like. Use the overloads of readUnsafe and readAndWriteUnsafe that take a cursor and close it after use.
-
-
Method Summary
Modifier and Type Method Description final intnext()Seeks to the next range of bytes, advancing the offset by {@code end - start}.final intseek(long offset)Reposition the cursor so that the data at {@code offset}is readable at{@code data[start]}.final longresizeBuffer(long newSize)Change the size of the buffer so that it equals {@code newSize}by either adding newcapacity at the end or truncating the buffer at the end.final longexpandBuffer(int minByteCount)Grow the buffer by adding a contiguous range of capacity in a singlesegment. voidclose()-
-
Method Detail
-
next
final int next()
Seeks to the next range of bytes, advancing the offset by
{@code end - start}. Returns thesize of the readable range (at least 1), or -1 if we have reached the end of the buffer andthere are no more bytes to read.
-
seek
final int seek(long offset)
Reposition the cursor so that the data at
{@code offset}is readable at{@code data[start]}.Returns the number of bytes readable in{@code data}(at least 1), or -1 if there are no datato read.
-
resizeBuffer
final long resizeBuffer(long newSize)
Change the size of the buffer so that it equals
{@code newSize}by either adding newcapacity at the end or truncating the buffer at the end. Newly added capacity may spanmultiple segments.As a side-effect this cursor will seek. If the buffer is being enlarged itwill move offset to the first byte of newly-added capacity. This is the size of thebuffer prior to the
{@code resizeBuffer()}call. If the buffer is being shrunk it will move offset to the end of the buffer.Warning: it is the caller’s responsibility to write new data to every byte of thenewly-allocated capacity. Failure to do so may cause serious security problems as the datain the returned buffers is not zero filled. Buffers may contain dirty pooled segments thathold very sensitive data from other parts of the current process.
-
expandBuffer
final long expandBuffer(int minByteCount)
Grow the buffer by adding a contiguous range of capacity in a singlesegment. This adds at least
{@code minByteCount}bytes but may add up to a full segment ofadditional capacity.As a side-effect this cursor will seek. It will move offset to thefirst byte of newly-added capacity. This is the size of the buffer prior to the
{@code * expandBuffer()}call.If
{@code minByteCount}bytes are available in the buffer's current tail segment that willbe used; otherwise another segment will be allocated and appended. In either case thisreturns the number of bytes of capacity added to this buffer.Warning: it is the caller’s responsibility to either write new data to every byte of thenewly-allocated capacity, or to shrink the buffer to the data written.Failure to do so may cause serious security problems as the data in the returned buffers isnot zero filled. Buffers may contain dirty pooled segments that hold very sensitive data fromother parts of the current process.
- Parameters:
minByteCount- the size of the contiguous capacity.
-
close
void close()
-
-
-
-