Package org.apache.cassandra.db.compaction

Class CompactionStrategyManager

java.lang.Object
org.apache.cassandra.db.compaction.CompactionStrategyManager
All Implemented Interfaces:
INotificationConsumer
public class CompactionStrategyManager extends Object implements INotificationConsumer
Manages the compaction strategies. SSTables are isolated from each other based on their incremental repair status (repaired, unrepaired, or pending repair) and directory (determined by their starting token). This class handles the routing between AbstractStrategyHolder instances based on repair status, and the AbstractStrategyHolder instances have separate compaction strategies for each directory, which it routes sstables to. Note that PendingRepairHolder also divides sstables on their pending repair id. Operations on this class are guarded by a ReentrantReadWriteLock. This lock performs mutual exclusion on reads and writes to the following variables: {@link this#repaired}, {@link this#unrepaired}, {@link this#isActive}, {@link this#params}, {@link this#currentBoundaries}. Whenever performing reads on these variables, the {@link this#readLock} should be acquired. Likewise, updates to these variables should be guarded by {@link this#writeLock}. Whenever the DiskBoundaries change, the compaction strategies must be reloaded, so in order to ensure the compaction strategy placement reflect most up-to-date disk boundaries, call {@link this#maybeReloadDiskBoundaries()} before acquiring the read lock to acess the strategies.

  • Field Details

    • compactionLogger

      public final CompactionLogger compactionLogger

    • TWCS_BUCKET_COUNT_MAX

      public static int TWCS_BUCKET_COUNT_MAX

  • Constructor Details

  • Method Details

    • getNextBackgroundTask

      public AbstractCompactionTask getNextBackgroundTask(long gcBefore)
      Return the next background task Returns a task for the compaction strategy that needs it the most (most estimated remaining tasks)

    • isEnabled

      public boolean isEnabled()

    • isActive

      public boolean isActive()

    • resume

      public void resume()

    • pause

      public void pause()
      pause compaction while we cancel all ongoing compactions Separate call from enable/disable to not have to save the enabled-state externally

    • getCompactionStrategyFor

      public AbstractCompactionStrategy getCompactionStrategyFor(SSTableReader sstable)
      return the compaction strategy for the given sstable returns differently based on the repaired status and which vnode the compaction strategy belongs to
      Parameters:
      sstable -
      Returns:

    • hasDataForPendingRepair

      public boolean hasDataForPendingRepair(TimeUUID sessionID)

    • hasPendingRepairSSTable

      public boolean hasPendingRepairSSTable(TimeUUID sessionID, SSTableReader sstable)

    • shutdown

      public void shutdown()

    • maybeReloadParamsFromSchema

      public void maybeReloadParamsFromSchema(CompactionParams params)
      Maybe reload the compaction strategies. Called after changing configuration.

    • maybeReloadDiskBoundaries

      protected void maybeReloadDiskBoundaries()
      Checks if the disk boundaries changed and reloads the compaction strategies to reflect the most up-to-date disk boundaries.

      This is typically called before acquiring the {@link this#readLock} to ensure the most up-to-date disk locations and boundaries are used.

      This should *never* be called inside by a thread holding the {@link this#readLock}, since it will potentially acquire the {@link this#writeLock} to update the compaction strategies what can cause a deadlock.

      TODO: improve this to reload after receiving a notification rather than trying to reload on every operation

    • getUnleveledSSTables

      public int getUnleveledSSTables()

    • getLevelFanoutSize

      public int getLevelFanoutSize()

    • getSSTableCountPerLevel

      public int[] getSSTableCountPerLevel()

    • getPerLevelSizeBytes

      public long[] getPerLevelSizeBytes()

    • isLeveledCompaction

      public boolean isLeveledCompaction()

    • getSSTableCountPerTWCSBucket

      public int[] getSSTableCountPerTWCSBucket()

    • groupSSTables

      public List<AbstractStrategyHolder.GroupedSSTableContainer> groupSSTables(Iterable<SSTableReader> sstables)
      Split sstables into a list of grouped sstable containers, the list index an sstable lives in matches the list index of the holder that's responsible for it

    • handleNotification

      public void handleNotification(INotification notification, Object sender)
      Specified by:
      handleNotification in interface INotificationConsumer

    • enable

      public void enable()

    • disable

      public void disable()

    • maybeGetScanners

      public AbstractCompactionStrategy.ScannerList maybeGetScanners(Collection<SSTableReader> sstables, Collection<Range<Token>> ranges)
      Create ISSTableScanners from the given sstables Delegates the call to the compaction strategies to allow LCS to create a scanner
      Parameters:
      sstables -
      ranges -
      Returns:

    • getScanners

      public AbstractCompactionStrategy.ScannerList getScanners(Collection<SSTableReader> sstables, Collection<Range<Token>> ranges)

    • getScanners

      public AbstractCompactionStrategy.ScannerList getScanners(Collection<SSTableReader> sstables)

    • groupSSTablesForAntiCompaction

      public Collection<Collection<SSTableReader>> groupSSTablesForAntiCompaction(Collection<SSTableReader> sstablesToGroup)

    • getMaxSSTableBytes

      public long getMaxSSTableBytes()

    • getCompactionTask

      public AbstractCompactionTask getCompactionTask(LifecycleTransaction txn, long gcBefore, long maxSSTableBytes)

    • getMaximalTasks

      public CompactionTasks getMaximalTasks(long gcBefore, boolean splitOutput, OperationType operationType)

    • getUserDefinedTasks

      public CompactionTasks getUserDefinedTasks(Collection<SSTableReader> sstables, long gcBefore)
      Return a list of compaction tasks corresponding to the sstables requested. Split the sstables according to whether they are repaired or not, and by disk location. Return a task per disk location and repair status group.
      Parameters:
      sstables - the sstables to compact
      gcBefore - gc grace period, throw away tombstones older than this
      Returns:
      a list of compaction tasks corresponding to the sstables requested

    • getEstimatedRemainingTasks

      public int getEstimatedRemainingTasks()

    • getEstimatedRemainingTasks

      public int getEstimatedRemainingTasks(int additionalSSTables, long additionalBytes, boolean isIncremental)

    • shouldBeEnabled

      public boolean shouldBeEnabled()

    • getName

      public String getName()

    • getStrategies

      public List<List<AbstractCompactionStrategy>> getStrategies()

    • overrideLocalParams

      public void overrideLocalParams(CompactionParams params)

    • getCompactionParams

      public CompactionParams getCompactionParams()

    • onlyPurgeRepairedTombstones

      public boolean onlyPurgeRepairedTombstones()

    • createSSTableMultiWriter

      public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, TimeUUID pendingRepair, boolean isTransient, IntervalSet<CommitLogPosition> commitLogPositions, int sstableLevel, SerializationHeader header, Collection<Index.Group> indexGroups, LifecycleNewTracker lifecycleNewTracker)

    • isRepaired

      public boolean isRepaired(AbstractCompactionStrategy strategy)

    • getStrategyFolders

      public List<String> getStrategyFolders(AbstractCompactionStrategy strategy)

    • supportsEarlyOpen

      public boolean supportsEarlyOpen()

    • mutateRepaired

      public void mutateRepaired(Collection<SSTableReader> sstables, long repairedAt, TimeUUID pendingRepair, boolean isTransient) throws IOException
      Mutates sstable repairedAt times and notifies listeners of the change with the writeLock held. Prevents races with other processes between when the metadata is changed and when sstables are moved between strategies.
      Throws:
      IOException

    • releaseRepairData

      public CleanupSummary releaseRepairData(Collection<TimeUUID> sessions)