A buffer goes beyond just storing a list of elements. It has internal state that keeps track of the current position when reading data from or writing data to the buffer, as well as the end of valid data for reading, etc. To do this, each buffer maintains four indices into its list of elements; they are shown in Table 5.1. (We’ll see shortly how the indices are modified by the various buffer methods.)

The distance between the position and limit tells us the number of bytes available for getting/putting. Java provides two convenience methods for evaluating this distance.

boolean hasRemaining()
int remaining()

The following relationships among these variables are maintained as an invariant:

0 ≤ mark ≤ position ≤ limit ≤ capacity

The mark value “remembers” a position so you can come back to it later; the reset() method returns the position to the value it had when mark() was last called (unless doing so would violate the above invariant).

Typically, we create buffers either by allocation or by wrapping an array of primitives. The static factory methods for creating a ByteBuffer are shown in Table 5.2, along with the initial values of capacity, position, and limit for the returned instance. The initial value of mark is undefined for all newly created Buffer instances; attempts to reset() the position before calling mark() result in an InvalidMarkException.

To allocate a fresh instance, we simply call the static allocate() method for the type of buffer we want, specifying the number of elements:

   ByteBuffer byteBuf = ByteBuffer.allocate(20);
   DoubleBuffer dblBuf = DoubleBuffer.allocate(5);

Here byteBuf holds 20 bytes, and dblBuf holds 5 Java doubles. These buffers are fixed-size so they can never be expanded or contracted. If you find that the buffer you just allocated is too short, your only option is to allocate a new, correctly sized buffer.

We can also create a buffer from an existing array by calling the static wrap() method and passing the array to be wrapped

   byteArray[] = new byte[BUFFERSIZE];
   // ...Fill array...
   ByteBuffer byteWrap = ByteBuffer.wrap(byteArray);
   ByteBuffer subByteWrap = ByteBuffer.wrap(byteArray, 3, 3);

A buffer created by wrapping contains the data from the wrapped array. In fact, wrap() simply creates a buffer with a reference to the wrapped array, called the backing array. Any change to the data in the backing array changes the data in the buffer and vice versa. If we specify an offset and length to wrap(), the buffer is backed by the entire array with position and limit initially set to offset and offset + length. The elements preceding the offset and following the length are still accessible via the buffer.

Creation of a buffer by allocation isn’t really so different from wrapping. The only real difference is that allocate() creates its own backing array. You can get a reference to this backing array by calling array() on the buffer. You can even get the offset into the backing array of the first element used by the buffer by calling arrayOffset(). A buffer created with wrap() with a nonzero offset still has an array offset of 0.

So far, all of our buffers store data in Java-allocated backing arrays. Typically, the underlying platform (operating system) cannot use these buffers to perform I/O. Instead the OS must use its own buffers for I/O and copy the results to/from the buffer’s backing array. Such copying can get expensive, especially if there are many reads and writes requiring copying. Java NIO provides direct buffers as a way around this problem. With a direct buffer, Java allocates the backing store of the buffer from storage that the platform can use for I/O directly, so copying is unnecessary. Such low-level, native I/O generally operates at the byte level, so only ByteBuffers can be directly allocated.

   ByteBuffer byteBufDirect = ByteBuffer.allocateDirect(BUFFERSIZE);

You can test whether a buffer is direct by calling isDirect(). Since a direct buffer does not have a backing array, calling array() or arrayOffset() on a direct buffer will throw an UnsupportedOperationException. There are a few caveats to remember when considering whether to use direct buffers. Calling allocateDirect() doesn’t guarantee you are allocated a direct buffer—your platform or JVM may not support this operation, so you have to call isDirect() after attempting to allocate. Also, allocation and deallocation of a direct buffer is typically more expensive than for nondirect buffers, because the backing store of a direct buffer typically lives outside the JVM, requiring interaction with the operating system for management. Consequently, you should only allocate direct buffers when they will be used for a long time, over many I/O operations. In fact, it is a good idea to use direct buffers only if they provide a measurable increase in performance over nondirect buffers.

Once you have a buffer, it’s time to use it to hold data. As “containers” for data, buffers are used for both input and output; this is different from streams, which transfer data in only one direction. We place data into a buffer using put(), and retrieve data from a buffer using get(). A channel read() implicitly calls put(), and a channel write() implicitly calls get() on the given buffer. Below we present the get() and put() methods for ByteBuffer; however, the other buffer types have similar methods.

byte get()
ByteBuffer get(byte[] dst)
ByteBuffer get(byte[] dst, int offset, int length)
ByteBuffer put(byte b)
ByteBuffer put(byte[] src)
ByteBuffer put(byte[] src, int offset, int length)
ByteBuffer put(ByteBuffer src)

byte get(int index)
ByteBuffer put(int index, byte b)

The class ByteBuffer provides additional methods for relative and absolute get/put of other types besides bytes; in this way, it’s like a DataOutputStream.

〈type〉 get〈Type〉()
〈type〉 get〈Type〉(int index)
ByteBuffer put 〈Type〉(〈type〉 value)
ByteBuffer put〈Type〉(int index,〈type〉 value)
where "〈Type〉" stands for one of Char, Double, Int, Long, Short
and "〈type〉" stands for one of char, double, int, long, short

myBuffer.putInt(1).putInt(2);

Recall from Chapter 3 that multibyte values have a byte order, namely big- or little-endian. By default Java uses big-endian. You can get and set the order in which multibyte values are written to a byte buffer, using the built-in instances ByteOrder.BIG_ENDIAN and ByteOrder.LITTLE_ENDIAN.

ByteOrder order()
ByteBuffer order(ByteOrder order)

Let’s look at an example using byte order:

   ByteBuffer buffer = ByteBuffer.allocate(4);
   buffer.putShort((short) 1);
   buffer.order(ByteOrder.LITTLE_ENDIAN);
   buffer.putShort((short) 1);
   // Predict the byte values for buffer and test your prediction

With all of this talk about byte ordering, you may be wondering about the byte order of your processor. ByteOrder defines a method to answer your question:

static final ByteOrder BIG_ENDIAN
static final ByteOrder LITTLE_ENDIAN
static ByteOrder nativeOrder()

Before using a buffer for input or output, we need to make sure the buffer is correctly prepared with position and limit set to the proper values. Consider a CharBuffer created with capacity seven, which has been populated by successive calls to put() or read():

If we now want to use this buffer to do a channel write, since write() will start getting data at position, and stop at limit, we need to set limit to the current value of position, and set position to 0.

We could handle this ourselves, but fortunately Java provides some convenience methods to do the work for us; they are shown in Table 5.3.

Note that these methods do not change the buffer’s data, only its indices. The clear() method prepares the buffer to accept new data from a buffer put or channel read by setting position to zero and limit to capacity. Continuing the example above, after clear() the situation looks like this:

Subsequent calls to put()/read() fill the buffer, starting from the first element and filling up to the limit, which is set to the capacity.

   // Start with buffer in unknown state
   buffer.clear();        // Prepare buffer for input, ignoring existing state
   channel.read(buffer);  // Read new data into buffer, starting at first element

Despite its name, clear() doesn’t actually change the buffer’s data; it simply resets the buffer’s main index values. Consider a buffer recently populated with data (say, 3 characters) from put() and/or read(). The position value indicates the first element that does not contain valid data:

The flip() method prepares for data transfer out of the buffer, by setting limit to the current position and position to zero:

Subsequent calls to get()/write() retrieve data from the buffer starting from the first element and proceeding up to the limit. Here is an example of flip()’s usage:

   // ... put data in buffer with put() or read() ...
   buffer.flip();                // Set position to 0, limit to old position
   while (buffer.hasRemaining()) // Write buffer data from the first element up to limit
     channel.write(buffer);

Suppose you’ve written some or all of the data from a buffer and you’d like to go back to the beginning of the buffer to write the same information again (for example, you want to send it on another channel). The rewind() method sets position to zero and invalidates the mark. It’s similar to flip() except limit remains unchanged. When might you use this? Well, you might want to write everything you send over the network to a logger:

   // Start with buffer ready for writing
   while (buffer.hasRemaining())  // Write all data to network
     networkChannel.write(buffer);
   buffer.rewind();               // Reset buffer to write again
   while (buffer.hasRemaining())  // Write all data to logger
     loggerChannel.write(buffer);

The compact() operation copies the elements between position and limit to the start of the buffer, to make room for subsequent put()/read() calls. The value of position is set to the length of the copied data, the value of limit is set to the capacity, and mark becomes undefined. Consider the following buffer state before compact() is called:

Here is the situation after compact():

Why use this operation? Suppose you have a buffer for writing data. Recall that a nonblocking call to write() only uses the data it can send without blocking; therefore, write() will not necessarily send all elements of the buffer. Now you need to read() new data into the buffer, after the unwritten data. One way to handle this is to simply set position = limit and limit = capacity. Of course, you’ll need to reset these values later, after reading but before you call write() again. The problem is that eventually the buffer will run out of space; in the figures above, there would only be room for one more byte. Moreover, any space at the beginning of the buffer is wasted. This is exactly the problem compact() is designed to solve. By calling compact() after the write() but before the read() that will add more data, we move all the “left over” data to the start of the buffer, freeing up the maximum space for new data.

   // Start with buffer ready for reading
   while (channel.read(buffer) != -1) {
     buffer.flip();
     channel.write(buffer);
     buffer.compact();
   }
   while (buffer.hasRemaining())
     channel.write(buffer);

Note, however, that as we mentioned at the beginning of the chapter, copying data is a rather expensive operation, so compact() should be used sparingly.

NIO provides several ways of creating a new buffer that shares content with a given buffer, but differs on the processing of the elements. Basically, the new buffer has its own independent state variables (position, limit, capacity, and mark) but shares the backing storage with the original buffer. Any changes to the new buffer are shared with the original. Think of this as an alternate perspective on the same data. Table 5.4 lists the relevant methods.

The duplicate() method creates a new buffer that shares the content of the original buffer. The new buffer’s position, limit, mark, and capacity initially match the original buffer’s index values; however, the values are independent. Since the content is shared, changes to the original buffer or any duplicates will be visible to all. Let’s return to our example above where you want to write everything you send over the network to a logger.

   // Start with buffer ready for writing
   ByteBuffer logBuffer = buffer.duplicate();
   while (buffer.hasRemaining())     // Write all data to network
     networkChannel.write(buffer);
   while (logBuffer.hasRemaining())     // Write all data to logger
     loggerChannel.write(buffer);

Note that with buffer duplication, writing to the network and log could be done in parallel using different threads.

The slice() method creates a new buffer that shares some subsequence of the original buffer. The new buffer’s position is zero, and its limit and capacity are both equal to the difference between the limit and position of the original buffer. slice() sets the new buffer’s array offset to the original buffer’s position; however, calling array() on the new buffer still returns the entire array.

Channel reads and writes take only ByteBuffers; however, we may be interested in communicating using other primitive types. A ByteBuffer can create a separate “view buffer” that interprets its contents as some other primitive type (e.g., CharBuffer). Data of the new type can then be read from (and written to, although that is an optional operation) this buffer. The new buffer shares the backing storage of the original ByteBuffer; therefore, changes to either buffer are seen in both new and original buffers. A newly created view buffer has its position set to zero, and its contents start at the original buffer’s position. This is very similar to the slice() operation; however, since the view buffer operates over multibyte elements, the capacity and limit of the new buffer is the remaining number of bytes divided by the number of bytes in the corresponding primitive type (e.g., divide by 8 when creating a DoubleBuffer).

Let’s look at an example. Suppose you have received (via some Channel) a message that consists of a single byte, followed by a number of two-byte integers (i.e., shorts), in big-endian order. Because the message arrives over a Channel, it’s in a ByteBuffer, buf. The first byte of the message contains the number of two-byte integers that make up the rest of the message. You might call buf.getShort() the number of times indicated by the first byte. Or you can get all the integers at once, like this:

   // ...get message by calling channel.read(buf) ...
   int numShorts = (int)buf.get();
   if (numShorts < 0) {
      throw new SomeException()
   } else {
      short[] shortArray = new short[numShorts];
      ShortBuffer sbuf = buf.asShortBuffer();
      sbuf.get(shortArray); // note: will throw if header was incorrect!
   }

The asReadOnlyBuffer() method works just like duplicate() except that all mutator methods on the new buffer will always throw a ReadOnlyBufferException. This includes all forms of put(), compact(), etc. Even calls to array() and arrayOffset() for a nondirect buffer throw this exception. Of course, changes to the non-read-only buffer that generated this read-only buffer will still be shared. Like a buffer created with duplicate(), read-only buffers have independent buffer state variables. You can use the isReadOnly() method to test if a buffer is read-only. If a buffer is already read-only, calling duplicate() or slice() will create a read-only buffer.

Recall from Chapter 3 that characters are encoded as sequences of bytes, and that there are various mappings (called charsets) between sets of characters and byte sequences. Another use of NIO buffers is to convert among various charsets. To use this facility, you need to know about two additional classes in the java.nio.charset package (we have already encountered Charset in Chapter 3): CharsetEncoder and CharsetDecoder.

To encode, use a Charset instance to create an encoder and call encode:

   Charset charSet = Charset.forName("US-ASCII");
   CharsetEncoder encoder = charSet.newEncoder();
   ByteBuffer buffer = encoder.encode(CharBuffer.wrap("Hi mom"));

To decode, use the Charset instance to create a decoder and call decode:

   CharsetDecoder decoder = charSet.newDecoder();
   CharBuffer cBuf = decoder.decode(buffer);

While this approach certainly works, it can be inefficient when coding multiple times. For example, each call to encode/decode creates a new Byte/CharBuffer. Other inefficiencies crop up relating to coder creation and operation.

   encoder.reset();
   if (encoder.encode(CharBuffer.wrap("Hi "),buffer,false) == CoderResult.OVERFLOW) {
      // ... deal with lack of space in buffer ...
   }
   if (encoder.encode(CharBuffer.wrap("Mom"),buffer,true) == CoderResult.OVERFLOW) {
      // ... ditto ...
   }
   encoder.flush(buffer);

The encode() method converts the given CharBuffer into a byte sequence and writes the bytes to the given buffer. If the buffer is too small, encode() returns a CoderResult.OVERFLOW. If the input is completely consumed and the encoder is ready for more, CoderResult.UNDERFLOW is returned; otherwise the input is malformed in some way, and a CoderResult object is returned that indicates the nature and location of the problem. We set the final boolean parameter to TRUE only when we have reached the end of input to the encoder. flush() pushes any buffered encoder state to the buffer. Note that it is not strictly necessary to call reset(), which sets up the encoder’s internal state so it can encode again, on a freshly created encoder.

As we have seen, each selector has an associated set of channels which it monitors for specific I/O “operations of interest” to that channel. The association between a Selector and a Channel is represented by an instance of SelectionKey. (Note that a Channel instance can register more than one Selector instance, and so can have more than one associated instance of SelectionKey.) The SelectionKey maintains information about the kinds of operations that are of interest for a channel in a bitmap, which is just an int in which individual bits have assigned meanings.

The possible operations of interest are defined by constants of the SelectionKey class; each such constant is a bitmask (see Section 3.1.3) with exactly one bit set.

static int OP_ ACCEPT
static int OP_CONNECT
static int OP_READ
static int OP_WRITE
int interestOps()
SelectionKey interestOps(int ops)

SelectionKey register(Selector sel, int ops)
SelectionKey register(Selector sel, int ops, Object attachment)
int validOps()
boolean isRegistered()
SelectionKey keyFor(Selector sel)

The following code registers a channel for both reading and writing:

SelectionKey key = clientChannel.register(selector,
                    SelectionKey.OP_READ | SelectionKey.OP_WRITE);

Figure 5.1 shows a selector with a key set containing keys representing seven registered channels: two server channels on ports 4000 and 4001, and five client channels created from the server channels.

Figure 5.1. Selector with associated key sets.

Selector selector()
SelectableChannel channel()
void cancel()

With our channels registered with the selector and the associated keys specifying the set of I/O operations of interest, we just need to sit back and wait for I/O. We do this using the selector.

int select()
int select(long timeout)
int selectNow()
Selector wakeup()

After selection, we need to know which channels have ready I/O of interest. Each selector maintains a selected-key set containing the keys from the key set whose associated channels have impending I/O of interest. We access the selected-key set by calling the selectedKeys() method of the selector, which returns a set of SelectionKeys. We can then iterate over this set of keys, handling pending I/O for each:

   Iterator<SelectionKey> keyIter = selector.selectedKeys().iterator();
   while (keyIter.hasNext()) {
     SelectionKey key = keyIter.next();
     // ...Handle I/O for key's channel...
     keyIter.remove();
   }

The selector in Figure 5.1 has two keys in its selected-key set: K2 and K5.

Set<SelectionKey> keys()
Set<SelectionKey> selectedKeys()

The selected-key set tells us which channels have available I/O. For each of these channels, we need to know the specific ready I/O operations. In addition to the interest set, each key also maintains a set of pending I/O operations called its ready set.

int readyOps()
boolean isAcceptable()
boolean isConnectable()
boolean isReadable()
boolean isValid()
boolean isWritable()

For example, to see if the channel associated with a key has a read pending we can either use:

(key.readyOps() & SelectionKey.OP_READ) != 0

or

key.isReadable()

The keys in a selector’s selected-key set and the operations in each key’s ready set are determined by select(). Over time, this information can become stale. Some other thread may handle the ready I/O. Also, keys don’t live forever. A key becomes invalid when its associated channel or selector is closed. A key may be explicitly invalidated by calling its cancel() method. You can test the validity of a key by calling its isValid() method. Invalid keys are added to the cancelled-key set of the selector and removed from its key set at the next invocation of any form of select(), or close(). (Of course, removing a key from the key set means that its associated channel will no longer be monitored.)

When a channel is ready for I/O, we often need additional information to process the request. For example, with our Echo protocol, when a client channel is ready to write, we need data. Of course, the data we need to write was collected earlier by reading it from the same channel, but where do we store it until it can be written? Another example is the framing procedure from Chapter 3. If a message arrives a few bytes at a time, we may need to store the parts received so far until we have the complete message. In both cases, we need to associate state information with each channel. Well, we’re in luck! SelectionKeys make storing per-channel state easy with attachments.

Object attach(Object ob)
Object attachment()

To summarize, here are the steps in using a Selector:

If selectors tell you when I/O is ready, do you still need nonblocking I/O? Yes. A channel’s key in the selected-keys set doesn’t guarantee nonblocking I/O because key set information can become stale after select(). In addition, a blocking write blocks until all bytes are written; however, an OP_WRITE in the ready set only indicates that at least one byte can be written. In fact, you cannot register a channel with a selector unless it is in nonblocking mode: the register() method of SelectableChannel throws an IllegalBlockingModeException if invoked when the channel is in blocking mode.