Package org.apache.cassandra.index.sai.disk.v1

Class V1OnDiskFormat

java.lang.Object

org.apache.cassandra.index.sai.disk.v1.V1OnDiskFormat

All Implemented Interfaces:

OnDiskFormat

public class V1OnDiskFormat
extends Object
implements OnDiskFormat

Field Summary

Fields

Modifier and Type	Field	Description
static final V1OnDiskFormat	instance
static final Set<IndexComponent>	LITERAL_COMPONENTS
static final Set<IndexComponent>	NUMERIC_COMPONENTS
static final long	SEGMENT_BUILD_MEMORY_LIMIT	Global limit on heap consumed by all index segment building that occurs outside the context of Memtable flush.
static final NamedMemoryLimiter	SEGMENT_BUILD_MEMORY_LIMITER
static final Set<IndexComponent>	SKINNY_PER_SSTABLE_COMPONENTS
static final Set<IndexComponent>	VECTOR_COMPONENTS
static final Set<IndexComponent>	WIDE_PER_SSTABLE_COMPONENTS

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	V1OnDiskFormat()

Method Summary

Modifier and Type	Method	Description
protected boolean	isNotBuildCompletionMarker(IndexComponent indexComponent)
boolean	isPerColumnIndexBuildComplete(IndexDescriptor indexDescriptor, IndexIdentifier indexIdentifier)	Returns true if the per-column index components have been built and are valid.
boolean	isPerSSTableIndexBuildComplete(IndexDescriptor indexDescriptor)	Returns true if the per-sstable index components have been built and are valid.
PerColumnIndexWriter	newPerColumnIndexWriter(StorageAttachedIndex index, IndexDescriptor indexDescriptor, LifecycleNewTracker tracker, RowMapping rowMapping)	Create a new PerColumnIndexWriter to write the per-column on-disk components of an index.
PerSSTableIndexWriter	newPerSSTableIndexWriter(IndexDescriptor indexDescriptor)	Create a new PerSSTableIndexWriter to write the per-SSTable on-disk components of an index.
PrimaryKeyMap.Factory	newPrimaryKeyMapFactory(IndexDescriptor indexDescriptor, SSTableReader sstable)	Returns a PrimaryKeyMap.Factory for the SSTable
SSTableIndex	newSSTableIndex(SSTableContext sstableContext, StorageAttachedIndex index)	Create a new SSTableIndex for an on-disk index.
int	openFilesPerColumnIndex()	Return the number of open per-column index files that can be open during a query.
int	openFilesPerSSTableIndex(boolean hasClustering)	Return the number of open per-SSTable files that can be open during a query.
Set<IndexComponent>	perColumnIndexComponents(IndexTermType indexTermType)	Returns the set of IndexComponent for the per-column part of an index.
Set<IndexComponent>	perSSTableIndexComponents(boolean hasClustering)	Returns the set of IndexComponent for the per-SSTable part of an index.
void	validatePerColumnIndexComponents(IndexDescriptor indexDescriptor, IndexTermType indexTermType, IndexIdentifier indexIdentifier, boolean checksum)	Validate all the per-column on-disk components and throw if a component is not valid
void	validatePerSSTableIndexComponents(IndexDescriptor indexDescriptor, boolean checksum)	Validate all the per-SSTable on-disk components and throw if a component is not valid

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

SKINNY_PER_SSTABLE_COMPONENTS

public static final Set<IndexComponent> SKINNY_PER_SSTABLE_COMPONENTS

WIDE_PER_SSTABLE_COMPONENTS

public static final Set<IndexComponent> WIDE_PER_SSTABLE_COMPONENTS

LITERAL_COMPONENTS

public static final Set<IndexComponent> LITERAL_COMPONENTS

NUMERIC_COMPONENTS

public static final Set<IndexComponent> NUMERIC_COMPONENTS

VECTOR_COMPONENTS

public static final Set<IndexComponent> VECTOR_COMPONENTS

SEGMENT_BUILD_MEMORY_LIMIT

public static final long SEGMENT_BUILD_MEMORY_LIMIT

Global limit on heap consumed by all index segment building that occurs outside the context of Memtable flush.

Note that to avoid flushing small index segments, a segment is only flushed when both the global size of all building segments has breached the limit and the size of the segment in question reaches (segment_write_buffer_space_mb / # currently building column indexes).

ex. If there is only one column index building, it can buffer up to segment_write_buffer_space_mb.

ex. If there is one column index building per table across 8 compactors, each index will be eligible to flush once it reaches (segment_write_buffer_space_mb / 8) MBs.

SEGMENT_BUILD_MEMORY_LIMITER

public static final NamedMemoryLimiter SEGMENT_BUILD_MEMORY_LIMITER

instance

public static final V1OnDiskFormat instance

Constructor Details

V1OnDiskFormat

protected V1OnDiskFormat()

Method Details

newPrimaryKeyMapFactory

public PrimaryKeyMap.Factory newPrimaryKeyMapFactory(IndexDescriptor indexDescriptor,
 SSTableReader sstable)

Description copied from interface: OnDiskFormat

Returns a PrimaryKeyMap.Factory for the SSTable

Specified by:

newPrimaryKeyMapFactory in interface OnDiskFormat

Parameters:

indexDescriptor - The IndexDescriptor for the SSTable

sstable - The SSTableReader associated with the IndexDescriptor

newSSTableIndex

public SSTableIndex newSSTableIndex(SSTableContext sstableContext,
 StorageAttachedIndex index)

Description copied from interface: OnDiskFormat

Create a new SSTableIndex for an on-disk index.

Specified by:

newSSTableIndex in interface OnDiskFormat

Parameters:

sstableContext - The SSTableContext holding the per-SSTable information for the index

index - The StorageAttachedIndex

Returns:

the new SSTableIndex for the on-disk index

newPerSSTableIndexWriter

public PerSSTableIndexWriter newPerSSTableIndexWriter(IndexDescriptor indexDescriptor)
                                               throws IOException

Description copied from interface: OnDiskFormat

Create a new PerSSTableIndexWriter to write the per-SSTable on-disk components of an index.

Specified by:

newPerSSTableIndexWriter in interface OnDiskFormat

Parameters:

indexDescriptor - The IndexDescriptor for the SSTable

Throws:

IOException - if the writer couldn't be created

newPerColumnIndexWriter

public PerColumnIndexWriter newPerColumnIndexWriter(StorageAttachedIndex index,
 IndexDescriptor indexDescriptor,
 LifecycleNewTracker tracker,
 RowMapping rowMapping)

Description copied from interface: OnDiskFormat

Create a new PerColumnIndexWriter to write the per-column on-disk components of an index. The LifecycleNewTracker is used to determine the type of index write about to happen this will either be an OperationType.FLUSH indicating that we are about to flush a MemtableIndex or one of the other operation types indicating that we will be writing from an existing SSTable

Specified by:

newPerColumnIndexWriter in interface OnDiskFormat

Parameters:

index - The StorageAttachedIndex holding the current index build status

indexDescriptor - The IndexDescriptor for the SSTable

tracker - The LifecycleNewTracker for index build operation.

rowMapping - The RowMapping that is used to map rowID to PrimaryKey during the write operation

isPerSSTableIndexBuildComplete

public boolean isPerSSTableIndexBuildComplete(IndexDescriptor indexDescriptor)

Description copied from interface: OnDiskFormat

Returns true if the per-sstable index components have been built and are valid.

Specified by:

isPerSSTableIndexBuildComplete in interface OnDiskFormat

Parameters:

indexDescriptor - The IndexDescriptor for the SSTable SAI index

isPerColumnIndexBuildComplete

public boolean isPerColumnIndexBuildComplete(IndexDescriptor indexDescriptor,
 IndexIdentifier indexIdentifier)

Description copied from interface: OnDiskFormat

Returns true if the per-column index components have been built and are valid.

Specified by:

isPerColumnIndexBuildComplete in interface OnDiskFormat

Parameters:

indexDescriptor - The IndexDescriptor for the SSTable SAI index

indexIdentifier - The IndexIdentifier for the index

validatePerSSTableIndexComponents

public void validatePerSSTableIndexComponents(IndexDescriptor indexDescriptor,
 boolean checksum)

Description copied from interface: OnDiskFormat

Validate all the per-SSTable on-disk components and throw if a component is not valid

Specified by:

validatePerSSTableIndexComponents in interface OnDiskFormat

Parameters:

indexDescriptor - The IndexDescriptor for the SSTable SAI index

checksum - true if the checksum should be tested as part of the validation

validatePerColumnIndexComponents

public void validatePerColumnIndexComponents(IndexDescriptor indexDescriptor,
 IndexTermType indexTermType,
 IndexIdentifier indexIdentifier,
 boolean checksum)

Description copied from interface: OnDiskFormat

Validate all the per-column on-disk components and throw if a component is not valid

Specified by:

validatePerColumnIndexComponents in interface OnDiskFormat

Parameters:

indexDescriptor - The IndexDescriptor for the SSTable SAI index

indexTermType - The IndexTermType of the index

indexIdentifier - The IndexIdentifier for the index

checksum - true if the checksum should be tested as part of the validation

perSSTableIndexComponents

public Set<IndexComponent> perSSTableIndexComponents(boolean hasClustering)

Description copied from interface: OnDiskFormat

Returns the set of IndexComponent for the per-SSTable part of an index. This is a complete set of components that could exist on-disk. It does not imply that the components currently exist on-disk.

Specified by:

perSSTableIndexComponents in interface OnDiskFormat

Parameters:

hasClustering - true if the SSTable forms part of a table using clustering columns

perColumnIndexComponents

public Set<IndexComponent> perColumnIndexComponents(IndexTermType indexTermType)

Description copied from interface: OnDiskFormat

Returns the set of IndexComponent for the per-column part of an index. This is a complete set of components that could exist on-disk. It does not imply that the components currently exist on-disk.

Specified by:

perColumnIndexComponents in interface OnDiskFormat

Parameters:

indexTermType - the IndexTermType of the index

openFilesPerSSTableIndex

public int openFilesPerSSTableIndex(boolean hasClustering)

Description copied from interface: OnDiskFormat

Return the number of open per-SSTable files that can be open during a query. This is a static indication of the files that can be held open by an index for queries. It is not a dynamic calculation.

Specified by:

openFilesPerSSTableIndex in interface OnDiskFormat

Parameters:

hasClustering - true if the SSTable forms part of a table using clustering columns

openFilesPerColumnIndex

public int openFilesPerColumnIndex()

Description copied from interface: OnDiskFormat

Return the number of open per-column index files that can be open during a query. This is a static indication of the files that can be help open by an index for queries. It is not a dynamic calculation.

Specified by:

openFilesPerColumnIndex in interface OnDiskFormat

isNotBuildCompletionMarker

protected boolean isNotBuildCompletionMarker(IndexComponent indexComponent)