Package org.apache.cassandra.index.sai

Class StorageAttachedIndex

java.lang.Object
org.apache.cassandra.index.sai.StorageAttachedIndex
All Implemented Interfaces:
Index
public class StorageAttachedIndex extends Object implements Index

  • Field Details

  • Constructor Details

  • Method Details

    • validateOptions

      public static Map<String,String> validateOptions(Map<String,String> options, TableMetadata metadata)
      Used via reflection in IndexMetadata

    • register

      public void register(IndexRegistry registry)
      Description copied from interface: Index
      An index must be registered in order to be able to either subscribe to update events on the base table and/or to provide Searcher functionality for reads. The double dispatch involved here, where the Index actually performs its own registration by calling back to the supplied IndexRegistry's own registerIndex method, is to make the decision as to whether or not to register an index belong to the implementation, not the manager.
      Specified by:
      register in interface Index
      Parameters:
      registry - the index registry to register the instance with

    • unregister

      public void unregister(IndexRegistry registry)
      Description copied from interface: Index
      Unregister current index when it's removed from system
      Specified by:
      unregister in interface Index
      Parameters:
      registry - the index registry to register the instance with

    • getIndexMetadata

      public IndexMetadata getIndexMetadata()
      Description copied from interface: Index
      Returns the IndexMetadata which configures and defines the index instance. This should be the same object passed as the argument to setIndexMetadata.
      Specified by:
      getIndexMetadata in interface Index
      Returns:
      the index's metadata

    • getInitializationTask

      public Callable<?> getInitializationTask()
      Description copied from interface: Index
      Return a task to perform any initialization work when a new index instance is created. This may involve costly operations such as (re)building the index, and is performed asynchronously by SecondaryIndexManager
      Specified by:
      getInitializationTask in interface Index
      Returns:
      a task to perform any necessary initialization work

    • getMetadataReloadTask

      public Callable<?> getMetadataReloadTask(IndexMetadata indexMetadata)
      Description copied from interface: Index
      Return a task to reload the internal metadata of an index. Called when the base table metadata is modified or when the configuration of the Index is updated Implementations should return a task which performs any necessary work to be done due to updating the configuration(s) such as (re)building etc. This task is performed asynchronously by SecondaryIndexManager
      Specified by:
      getMetadataReloadTask in interface Index
      Returns:
      task to be executed by the index manager during a reload

    • getBlockingFlushTask

      public Callable<?> getBlockingFlushTask()
      Description copied from interface: Index
      Return a task which performs a blocking flush of any in-memory index data to persistent storage, independent of any flush of the base table. Note that this method is only invoked outside of normal flushes: if there is no in-memory storage for this index, and it only extracts data on flush from the base table's Memtable, then it is safe to perform no work.
      Specified by:
      getBlockingFlushTask in interface Index
      Returns:
      task to be executed by the index manager to perform the flush.

    • getInvalidateTask

      public Callable<?> getInvalidateTask()
      Description copied from interface: Index
      Return a task which invalidates the index, indicating it should no longer be considered usable. This should include an clean up and releasing of resources required when dropping an index.
      Specified by:
      getInvalidateTask in interface Index
      Returns:
      task to be executed by the index manager to invalidate the index.

    • getPreJoinTask

      public Callable<?> getPreJoinTask(boolean hadBootstrap)
      Description copied from interface: Index
      Return a task to be executed before the node enters NORMAL state and finally joins the ring.
      Specified by:
      getPreJoinTask in interface Index
      Parameters:
      hadBootstrap - If the node had bootstrap before joining.
      Returns:
      task to be executed by the index manager before joining the ring.

    • getTruncateTask

      public Callable<?> getTruncateTask(long truncatedAt)
      Description copied from interface: Index
      Return a task to truncate the index with the specified truncation timestamp. Called when the base table is truncated.
      Specified by:
      getTruncateTask in interface Index
      Parameters:
      truncatedAt - timestamp of the truncation operation. This will be the same timestamp used in the truncation of the base table.
      Returns:
      task to be executed by the index manager when the base table is truncated.

    • shouldBuildBlocking

      public boolean shouldBuildBlocking()
      Description copied from interface: Index
      Return true if this index can be built or rebuilt when the index manager determines it is necessary. Returning false enables the index implementation (or some other component) to control if and when SSTable data is incorporated into the index.

      This is called by SecondaryIndexManager in buildIndexBlocking, buildAllIndexesBlocking and rebuildIndexesBlocking where a return value of false causes the index to be exluded from the set of those which will process the SSTable data.

      Specified by:
      shouldBuildBlocking in interface Index
      Returns:
      if the index should be included in the set which processes SSTable data, false otherwise.

    • isSSTableAttached

      public boolean isSSTableAttached()
      Description copied from interface: Index
      For an index to qualify as SSTable-attached, it must do two things:

      1.) It must use SSTableFlushObserver to incrementally build indexes as SSTables are written. This ensures that non-entire file streaming builds them correctly before the streaming transaction finishes.

      2.) Its implementation of SecondaryIndexBuilder must support incremental building by SSTable.

      Specified by:
      isSSTableAttached in interface Index
      Returns:
      true if the index builds SSTable-attached on-disk components

    • getBackingTable

      public Optional<ColumnFamilyStore> getBackingTable()
      Description copied from interface: Index
      If the index implementation uses a local table to store its index data, this method should return a handle to it. If not, an empty Optional should be returned. This exists to support legacy implementations, and should always be empty for indexes not belonging to a SingletonIndexGroup.
      Specified by:
      getBackingTable in interface Index
      Returns:
      an Optional referencing the Index's backing storage table if it has one, or Optional.empty() if not.

    • dependsOn

      public boolean dependsOn(ColumnMetadata column)
      Description copied from interface: Index
      Called to determine whether this index targets a specific column. Used during schema operations such as when dropping or renaming a column, to check if the index will be affected by the change. Typically, if an index answers that it does depend upon a column, then schema operations on that column are not permitted until the index is dropped or altered.
      Specified by:
      dependsOn in interface Index
      Parameters:
      column - the column definition to check
      Returns:
      true if the index depends on the supplied column being present; false if the column may be safely dropped or modified without adversely affecting the index

    • supportsExpression

      public boolean supportsExpression(ColumnMetadata column, Operator operator)
      Description copied from interface: Index
      Called to determine whether this index can provide a searcher to execute a query on the supplied column using the specified operator. This forms part of the query validation done before a CQL select statement is executed.
      Specified by:
      supportsExpression in interface Index
      Parameters:
      column - the target column of a search query predicate
      operator - the operator of a search query predicate
      Returns:
      true if this index is capable of supporting such expressions, false otherwise

    • filtersMultipleContains

      public boolean filtersMultipleContains()
      Description copied from interface: Index
      Returns whether this index does any kind of filtering when the query has multiple contains expressions, assuming that each of those expressions are supported as defined by Index.supportsExpression(ColumnMetadata, Operator).
      Specified by:
      filtersMultipleContains in interface Index
      Returns:
      true if this index uses filtering on multiple contains expressions, false otherwise

    • customExpressionValueType

      public AbstractType<?> customExpressionValueType()
      Description copied from interface: Index
      If the index supports custom search expressions using the SELECT * FROM table WHERE expr(index_name, expression) syntax, this method should return the expected type of the expression argument. For example, if the index supports custom expressions as Strings, calls to this method should return UTF8Type.instance. If the index implementation does not support custom expressions, then it should return null.
      Specified by:
      customExpressionValueType in interface Index
      Returns:
      an the type of custom index expressions supported by this index, or an null if custom expressions are not supported.

    • getPostIndexQueryFilter

      public RowFilter getPostIndexQueryFilter(RowFilter filter)
      Description copied from interface: Index
      Transform an initial RowFilter into the filter that will still need to applied to a set of Rows after the index has performed it's initial scan. Used in ReadCommand#executeLocal to reduce the amount of filtering performed on the results of the index query.
      Specified by:
      getPostIndexQueryFilter in interface Index
      Parameters:
      filter - the intial filter belonging to a ReadCommand
      Returns:
      the (hopefully) reduced filter that would still need to be applied after the index was used to narrow the initial result set

    • getPostQueryOrdering

      public Comparator<ByteBuffer> getPostQueryOrdering(Restriction restriction, QueryOptions options)
      Description copied from interface: Index
      Return a comparator that reorders query result before sending to client
      Specified by:
      getPostQueryOrdering in interface Index
      Parameters:
      restriction - restriction that requires current index
      options - query options
      Returns:
      a comparator for post-query ordering; or null if not supported

    • validate

      public void validate(ReadCommand command) throws InvalidRequestException
      Description copied from interface: Index
      Used to validate the various parameters of a supplied ReadCommand, this is called prior to execution. In theory, any command instance may be checked by any Index instance, but in practice the index will be the one returned by a call to the getIndex(ColumnFamilyStore cfs) method on the supplied command. Custom index implementations should perform any validation of query expressions here and throw a meaningful InvalidRequestException when any expression or other parameter is invalid.
      Specified by:
      validate in interface Index
      Parameters:
      command - a ReadCommand whose parameters are to be verified
      Throws:
      InvalidRequestException - if the details of the command fail to meet the index's validation rules

    • getEstimatedResultRows

      public long getEstimatedResultRows()
      Description copied from interface: Index
      Return an estimate of the number of results this index is expected to return for any given query that it can be used to answer. Used in conjunction with indexes() and supportsExpression() to determine the most selective index for a given ReadCommand. Additionally, this is also used by StorageProxy.estimateResultsPerRange to calculate the initial concurrency factor for range requests
      Specified by:
      getEstimatedResultRows in interface Index
      Returns:
      the estimated average number of results a Searcher may return for any given query

    • isQueryable

      public boolean isQueryable(Index.Status status)
      Description copied from interface: Index
      Check if current index is queryable based on the index status.
      Specified by:
      isQueryable in interface Index
      Parameters:
      status - current status of the index
      Returns:
      true if index should be queryable, false if index should be non-queryable

    • validate

      public void validate(PartitionUpdate update, ClientState state) throws InvalidRequestException
      Description copied from interface: Index
      Called at write time to ensure that values present in the update are valid according to the rules of all registered indexes which will process it. The partition key as well as the clustering and cell values for each row in the update may be checked by index implementations
      Specified by:
      validate in interface Index
      Parameters:
      update - PartitionUpdate containing the values to be validated by registered Index implementations
      state - state related to the client connection
      Throws:
      InvalidRequestException

    • searcherFor

      public Index.Searcher searcherFor(ReadCommand command) throws InvalidRequestException
      Description copied from interface: Index
      Factory method for query time search helper.
      Specified by:
      searcherFor in interface Index
      Parameters:
      command - the read command being executed
      Returns:
      an Searcher with which to perform the supplied command
      Throws:
      InvalidRequestException

    • getFlushObserver

      public SSTableFlushObserver getFlushObserver(Descriptor descriptor, LifecycleNewTracker tracker)
      Description copied from interface: Index
      Get flush observer to observe partition/cell events generated by flushing SSTable (memtable flush or compaction).
      Specified by:
      getFlushObserver in interface Index
      Parameters:
      descriptor - The descriptor of the sstable observer is requested for.
      tracker - The LifecycleNewTracker associated with the SSTable being written
      Returns:
      SSTable flush observer.

    • getComponents

      public Set<Component> getComponents()
      Description copied from interface: Index
      Returns the SSTable-attached Components created by this index.
      Specified by:
      getComponents in interface Index
      Returns:
      the SSTable components created by this index

    • notifyIndexerAboutRowsInFullyExpiredSSTables

      public boolean notifyIndexerAboutRowsInFullyExpiredSSTables()
      Description copied from interface: Index
      When a secondary index is created on a column for a table with e.g. TWCS strategy, when this table contains SSTables which are evaluated as fully expired upon compaction, they are by default filtered out as they can be dropped in their entirety. However, once dropped like that, the index implementation is not notified about this fact via IndexGCTransaction as compaction on non-fully expired tables would do. This in turn means that custom index will never know that some data have been removed hence data custom index implementation is responsible for will grow beyond any limit. Override this method and return false in index implementation only in case you do not want to be notified about dropped fully-expired data. This will eventually mean that Index.Indexer.removeRow(Row) will not be called for rows contained in fully expired table. Return true if you do want to be notified about that fact. This method returns true by default.
      Specified by:
      notifyIndexerAboutRowsInFullyExpiredSSTables in interface Index
      Returns:
      true when fully expired tables should be included in compaction process, false otherwise.

    • indexerFor

      public Index.Indexer indexerFor(DecoratedKey key, RegularAndStaticColumns columns, long nowInSec, WriteContext writeContext, IndexTransaction.Type transactionType, Memtable memtable)
      Description copied from interface: Index
      Creates an new Indexer object for updates to a given partition.
      Specified by:
      indexerFor in interface Index
      Parameters:
      key - key of the partition being modified
      columns - the regular and static columns the created indexer will have to deal with. This can be empty as an update might only contain partition, range and row deletions, but the indexer is guaranteed to not get any cells for a column that is not part of columns.
      nowInSec - current time of the update operation
      writeContext - WriteContext spanning the update operation
      transactionType - indicates what kind of update is being performed on the base data i.e. a write time insert/update/delete or the result of compaction
      memtable - current memtable that the write goes into. It's to make sure memtable and index memtable are in sync.
      Returns:
      the newly created indexer or null if the index is not interested by the update (this could be because the index doesn't care about that particular partition, doesn't care about that type of transaction, ...).

    • getBuildTaskSupport

      public Index.IndexBuildingSupport getBuildTaskSupport()
      Description copied from interface: Index
      Get an instance of a helper to provide tasks for building the index from a set of SSTable data. When processing a number of indexes to be rebuilt, SecondaryIndexManager.buildIndexesBlocking groups those with the same IndexBuildingSupport instance, allowing multiple indexes to be built with a single pass through the data. The singleton instance returned from the default method implementation builds indexes using a ReducingKeyIterator to provide a collated view of the SSTable data.
      Specified by:
      getBuildTaskSupport in interface Index
      Returns:
      an instance of the index build task helper. Index implementations which return the same instance will be built using a single task.

    • groupBySize

      public static List<List<SSTableReader>> groupBySize(List<SSTableReader> toRebuild, int parallelism)
      Splits SSTables into groups of similar overall size.
      Parameters:
      toRebuild - a list of SSTables to split (Note that this list will be sorted in place!)
      parallelism - an upper bound on the number of groups
      Returns:
      a List of SSTable groups, each represented as a List of SSTableReader

    • onSSTableChanged

      public Collection<SSTableContext> onSSTableChanged(Collection<SSTableReader> oldSSTables, Collection<SSTableContext> newSSTables, IndexValidation validation)
      Returns:
      A set of SSTables which have attached to them invalid index components.

    • drop

      public void drop(Collection<SSTableReader> sstablesToRebuild)

    • memtableIndexManager

      public MemtableIndexManager memtableIndexManager()

    • view

      public View view()

    • termType

      public IndexTermType termType()

    • identifier

      public IndexIdentifier identifier()

    • keyFactory

      public PrimaryKey.Factory keyFactory()

    • baseCfs

      public ColumnFamilyStore baseCfs()

    • indexWriterConfig

      public IndexWriterConfig indexWriterConfig()

    • hasAnalyzer

      public boolean hasAnalyzer()

    • analyzer

      public AbstractAnalyzer analyzer()
      Returns an AbstractAnalyzer for use by write and query paths to transform literal values.

    • indexMetrics

      public IndexMetrics indexMetrics()

    • columnQueryMetrics

      public ColumnQueryMetrics columnQueryMetrics()

    • isInitBuildStarted

      public boolean isInitBuildStarted()

    • isIndexValid

      public BooleanSupplier isIndexValid()

    • supportsL0Shards

      public boolean supportsL0Shards()
      Vector indexes do not supporrt L0 shards due to the cost associated with resharding at flush time.
      Returns:
      true iff the index supports sharding at L0.

    • hasClustering

      public boolean hasClustering()

    • cellCount

      public long cellCount()
      Returns:
      the number of indexed rows in this index (aka. a pair of term and rowId)

    • openPerColumnIndexFiles

      public int openPerColumnIndexFiles()
      Returns:
      total number of per-index open files

    • diskUsage

      public long diskUsage()
      Returns:
      the total size (in bytes) of per-column index components

    • indexFileCacheSize

      public long indexFileCacheSize()
      Returns:
      the total memory usage (in bytes) of per-column index on-disk data structure

    • makeIndexNonQueryable

      public void makeIndexNonQueryable()
      Removes this index from the SecondaryIndexManager's set of queryable indexes.

    • validateTermSizeForRow

      public void validateTermSizeForRow(DecoratedKey key, Row row, boolean isClientMutation, ClientState state)
      Validate maximum term size for given row

    • validateTermSize

      public boolean validateTermSize(DecoratedKey key, ByteBuffer term, boolean isClientMutation, ClientState state)
      Returns:
      true if the size of the given term is below the maximum term size, false otherwise
      Throws:
      GuardrailViolatedException - if a client mutation contains a term that breaches the failure threshold

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • equals

      public boolean equals(Object obj)
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object