Package org.apache.cassandra.utils.memory

Class BufferPool

java.lang.Object
org.apache.cassandra.utils.memory.BufferPool
public class BufferPool extends Object
A pool of ByteBuffers that can be recycled to reduce system direct memory fragmentation and improve buffer allocation performance.

Each BufferPool instance has one BufferPool.GlobalPool which allocates two kinds of chunks:

  • Macro Chunk
    • A memory slab that has size of MACRO_CHUNK_SIZE which is 64 * NORMAL_CHUNK_SIZE
    • Used to allocate normal chunk with size of NORMAL_CHUNK_SIZE
  • Normal Chunk
    • Used by BufferPool.LocalPool to serve buffer allocation
    • Minimum allocation unit is NORMAL_CHUNK_SIZE / 64
BufferPool.GlobalPool maintains two kinds of freed chunks, fully freed chunks where all buffers are released, and partially freed chunks where some buffers are not released, eg. held by ChunkCache. Partially freed chunks are used to improve cache utilization and have lower priority compared to fully freed chunks.

BufferPool.LocalPool is a thread local pool to serve buffer allocation requests. There are two kinds of local pool:

  • Normal Pool:
    • used to serve allocation size that is larger than half of NORMAL_ALLOCATION_UNIT but less than NORMAL_CHUNK_SIZE
    • when there is insufficient space in the local queue, it will request global pool for more normal chunks
    • when normal chunk is recycled either fully or partially, it will be passed to global pool to be used by other pools
  • Tiny Pool:
    • used to serve allocation size that is less than NORMAL_ALLOCATION_UNIT
    • when there is insufficient space in the local queue, it will request parent normal pool for more tiny chunks
    • when tiny chunk is fully freed, it will be passed to paretn normal pool and corresponding buffer in the parent normal chunk is freed
Note: even though partially freed chunks improves cache utilization when chunk cache holds outstanding buffer for arbitrary period, there is still fragmentation in the partially freed chunk because of non-uniform allocation size.

The lifecycle of a normal Chunk: 

    new                      acquire                      release                    recycle
 ────────→ in GlobalPool ──────────────→ in LocalPool ──────────────→ EVICTED  ──────────────────┐
           owner = null                  owner = LocalPool            owner = null               │
           status = IN_USE               status = IN_USE              status = EVICTED           │
              ready                      serves get / free            serves free only           │
                ↑                                                                                │
                └────────────────────────────────────────────────────────────────────────────────┘

  • Field Details

  • Constructor Details

    • BufferPool

      public BufferPool(String name, long memoryUsageThreshold, boolean recyclePartially)

  • Method Details

    • create

      public BufferPool.LocalPool create()
      Returns:
      a local pool instance and caller is responsible to release the pool

    • get

      public ByteBuffer get(int size, BufferType bufferType)

    • getAtLeast

      public ByteBuffer getAtLeast(int size, BufferType bufferType)

    • tryGet

      public ByteBuffer tryGet(int size)
      Unlike the get methods, this will return null if the pool is exhausted

    • tryGetAtLeast

      public ByteBuffer tryGetAtLeast(int size)

    • put

      public void put(ByteBuffer buffer)

    • putUnusedPortion

      public void putUnusedPortion(ByteBuffer buffer)

    • setRecycleWhenFreeForCurrentThread

      public void setRecycleWhenFreeForCurrentThread(boolean recycleWhenFree)

    • sizeInBytes

      public long sizeInBytes()
      Returns:
      buffer size being allocated, including pooled buffers and unpooled buffers

    • usedSizeInBytes

      public long usedSizeInBytes()
      Returns:
      buffer size being used, including used pooled buffers and unpooled buffers

    • overflowMemoryInBytes

      public long overflowMemoryInBytes()
      Returns:
      unpooled buffer size being allocated outside of buffer pool.

    • memoryUsageThreshold

      public long memoryUsageThreshold()
      Returns:
      maximum pooled buffer size in bytes

    • globalPool

      public org.apache.cassandra.utils.memory.BufferPool.GlobalPool globalPool()

    • releaseLocal

      public void releaseLocal()
      Forces to recycle free local chunks back to the global pool. This is needed because if buffers were freed by a different thread than the one that allocated them, recycling might not have happened and the local pool may still own some fully empty chunks.

    • debug

      public void debug(org.apache.cassandra.utils.memory.BufferPool.Debug newDebug, BufferPool.DebugLeaks newDebugLeaks)

    • roundUp

      public static int roundUp(int size)

    • roundUp

      public static int roundUp(int size, int unit)

    • shutdownLocalCleaner

      public void shutdownLocalCleaner(long timeout, TimeUnit unit) throws InterruptedException, TimeoutException
      Throws:
      InterruptedException
      TimeoutException

    • metrics

      public BufferPoolMetrics metrics()

    • unsafeReset

      public void unsafeReset()
      This is not thread safe and should only be used for unit testing.

    • unwrapBufferPoolManagedBuffer

      public ByteBuffer unwrapBufferPoolManagedBuffer(ByteBuffer originalBuffer)
      Returns:
      the inner buffer if it has a BufferPool.Chunk attached and originalBuffer in other cases