Package org.apache.cassandra.index.sai.disk.v1.postings

Class PostingsWriter

java.lang.Object
org.apache.cassandra.index.sai.disk.v1.postings.PostingsWriter
All Implemented Interfaces:
Closeable, AutoCloseable
@NotThreadSafe public class PostingsWriter extends Object implements Closeable
Encodes, compresses and writes postings lists to disk.

All postings in the posting list are delta encoded, then deltas are divided into blocks for compression. The deltas are based on the final value of the previous block. For the first block in the posting list the first value in the block is written as a VLong prior to block delta encodings.

In packed blocks, longs are encoded with the same bit width (FoR compression). The block size (i.e. number of longs inside block) is fixed (currently 128). Additionally blocks that are all the same value are encoded in an optimized way.

In VLong blocks, longs are compressed with DataOutput.writeVLong(long). The block size is variable.

Packed blocks are favoured, meaning when the postings are long enough, PostingsWriter will try to encode most data as a packed block. Take a term with 259 postings as an example, the first 256 postings are encoded as two packed blocks, while the remaining 3 are encoded as one VLong block.

Each posting list ends with a block summary containing metadata and a skip table, written right after all postings blocks. Skip interval is the same as block size, and each skip entry points to the end of each block. Skip table consist of block offsets and last values of each block, compressed as two FoR blocks.

Visual representation of the disk format: 

 +========+========================+=====+==============+===============+===============+=====+========================+========+
 | HEADER | POSTINGS LIST (TERM 1)                                                      | ... | POSTINGS LIST (TERM N) | FOOTER |
 +========+========================+=====+==============+===============+===============+=====+========================+========+
          | FIRST VALUE| FOR BLOCK (1)| ... | FOR BLOCK (N)| BLOCK SUMMARY              |
          +---------------------------+-----+--------------+---------------+------------+
                                                           | BLOCK SIZE    |            |
                                                           | LIST SIZE     | SKIP TABLE |
                                                           +---------------+------------+
                                                                           | BLOCKS POS.|
                                                                           | MAX VALUES |
                                                                           +------------+

  • Constructor Details

  • Method Details

    • getFilePointer

      public long getFilePointer()
      Returns:
      current file pointer

    • getStartOffset

      public long getStartOffset()
      Returns:
      file pointer where index structure begins (before header)

    • complete

      public void complete() throws IOException
      write footer to the postings
      Throws:
      IOException

    • close

      public void close() throws IOException
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Throws:
      IOException

    • write

      public long write(PostingList postings) throws IOException
      Encodes, compresses and flushes given posting list to disk.
      Parameters:
      postings - posting list to write to disk
      Returns:
      file offset to the summary block of this posting list
      Throws:
      IOException

    • getTotalPostings

      public long getTotalPostings()