Package org.apache.cassandra.db.compaction.writers

Class CompactionAwareWriter

java.lang.Object

org.apache.cassandra.utils.concurrent.Transactional.AbstractTransactional

org.apache.cassandra.db.compaction.writers.CompactionAwareWriter

All Implemented Interfaces:

AutoCloseable, Transactional

Direct Known Subclasses:

DefaultCompactionWriter, MajorLeveledCompactionWriter, MaxSSTableSizeWriter, ShardedCompactionWriter, SplittingSizeTieredCompactionWriter

public abstract class CompactionAwareWriter
extends Transactional.AbstractTransactional
implements Transactional

Class that abstracts away the actual writing of files to make it possible to use CompactionTask for more use cases.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.cassandra.utils.concurrent.Transactional.AbstractTransactional

Transactional.AbstractTransactional.State

Nested classes/interfaces inherited from interface org.apache.cassandra.utils.concurrent.Transactional

Transactional.AbstractTransactional

Field Summary

Fields

Modifier and Type	Field	Description
protected final ColumnFamilyStore	cfs
protected Directories.DataDirectory	currentDirectory
protected final Directories	directories
protected final long	estimatedTotalKeys
protected final boolean	isTransient
protected static final org.slf4j.Logger	logger
protected final long	maxAge
protected final long	minRepairedAt
protected final Set<SSTableReader>	nonExpiredSSTables
protected final TimeUUID	pendingRepair
protected final SSTableRewriter	sstableWriter
protected final LifecycleTransaction	txn

Constructor Summary

Constructors

Constructor	Description
CompactionAwareWriter(ColumnFamilyStore cfs, Directories directories, LifecycleTransaction txn, Set<SSTableReader> nonExpiredSSTables, boolean keepOriginals)

Method Summary

Modifier and Type	Method	Description
final boolean	append(UnfilteredRowIterator partition)	Writes a partition in an implementation specific way
protected Throwable	doAbort(Throwable accumulate)
protected Throwable	doCommit(Throwable accumulate)
protected Throwable	doPostCleanup(Throwable accumulate)	perform an exception-safe post-abort cleanup
protected void	doPrepare()	Do any preparatory work prior to commit.
long	estimatedKeys()	estimated number of keys we should write
Collection<SSTableReader>	finish()	we are done, return the finished sstables so that the caller can mark the old ones as compacted
Directories	getDirectories()	The directories we can write to
protected long	getExpectedWriteSize()
final File	getSStableDirectory()
Directories.DataDirectory	getWriteDirectory(Iterable<SSTableReader> sstables, long estimatedWriteSize)	Return a directory where we can expect expectedWriteSize to fit.
protected boolean	maybeSwitchLocation(DecoratedKey key)	Switches the file location and writer and returns true if the new key should be placed in a different data directory.
protected void	maybeSwitchWriter(DecoratedKey key)	Switches the writer if necessary, i.e.
protected SSTableWriter.Builder<?,?>	newWriterBuilder(Descriptor descriptor)	It is up to the caller to set the following fields: - SSTableWriter.Builder.setKeyCount(long), - SSTableWriter.Builder.setSerializationHeader(SerializationHeader) and, - SSTableWriter.Builder.setMetadataCollector(MetadataCollector)
protected boolean	realAppend(UnfilteredRowIterator partition)
CompactionAwareWriter	setRepairedAt(long repairedAt)
protected abstract boolean	shouldSwitchWriterInCurrentLocation(DecoratedKey key)	Returns true if the writer should be switched for reasons other than switching to a new data directory (e.g.
protected abstract long	sstableKeyCount()	Returns the key count with which created sstables should be set up.
protected int	sstableLevel()	Returns the level that should be used when creating sstables.
protected SSTableWriter	sstableWriter(Directories.DataDirectory directory, DecoratedKey nextKey)
protected void	switchCompactionWriter(Directories.DataDirectory directory, DecoratedKey nextKey)	Implementations of this method should finish the current sstable writer and start writing to this directory.

Methods inherited from class org.apache.cassandra.utils.concurrent.Transactional.AbstractTransactional

abort, abort, close, commit, commit, doPreCleanup, prepareToCommit, state

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.cassandra.utils.concurrent.Transactional

abort, close, commit, prepareToCommit

Field Details

logger

protected static final org.slf4j.Logger logger

cfs

protected final ColumnFamilyStore cfs

directories

protected final Directories directories

nonExpiredSSTables

protected final Set<SSTableReader> nonExpiredSSTables

estimatedTotalKeys

protected final long estimatedTotalKeys

maxAge

protected final long maxAge

minRepairedAt

protected final long minRepairedAt

pendingRepair

protected final TimeUUID pendingRepair

isTransient

protected final boolean isTransient

sstableWriter

protected final SSTableRewriter sstableWriter

txn

protected final LifecycleTransaction txn

currentDirectory

protected Directories.DataDirectory currentDirectory

Constructor Details

CompactionAwareWriter

public CompactionAwareWriter(ColumnFamilyStore cfs,
 Directories directories,
 LifecycleTransaction txn,
 Set<SSTableReader> nonExpiredSSTables,
 boolean keepOriginals)

Method Details

doAbort

protected Throwable doAbort(Throwable accumulate)

Specified by:

doAbort in class Transactional.AbstractTransactional

doCommit

protected Throwable doCommit(Throwable accumulate)

Specified by:

doCommit in class Transactional.AbstractTransactional

doPrepare

protected void doPrepare()

Description copied from class: Transactional.AbstractTransactional

Do any preparatory work prior to commit. This method should throw any exceptions that can be encountered during the finalization of the behaviour.

Specified by:

doPrepare in class Transactional.AbstractTransactional

finish

public Collection<SSTableReader> finish()

we are done, return the finished sstables so that the caller can mark the old ones as compacted

Overrides:

finish in class Transactional.AbstractTransactional

Returns:

all the written sstables sstables

estimatedKeys

public long estimatedKeys()

estimated number of keys we should write

append

public final boolean append(UnfilteredRowIterator partition)

Writes a partition in an implementation specific way

Parameters:

partition - the partition to append

Returns:

true if the partition was written, false otherwise

getSStableDirectory

public final File getSStableDirectory()
                               throws IOException

Throws:

IOException

doPostCleanup

protected Throwable doPostCleanup(Throwable accumulate)

Description copied from class: Transactional.AbstractTransactional

perform an exception-safe post-abort cleanup

Overrides:

doPostCleanup in class Transactional.AbstractTransactional

realAppend

protected boolean realAppend(UnfilteredRowIterator partition)

maybeSwitchWriter

protected void maybeSwitchWriter(DecoratedKey key)

Switches the writer if necessary, i.e. if the new key should be placed in a different data directory, or if the specific strategy has decided a new sstable is needed. Guaranteed to be called before the first call to realAppend.

maybeSwitchLocation

protected boolean maybeSwitchLocation(DecoratedKey key)

Switches the file location and writer and returns true if the new key should be placed in a different data directory.

shouldSwitchWriterInCurrentLocation

protected abstract boolean shouldSwitchWriterInCurrentLocation(DecoratedKey key)

Returns true if the writer should be switched for reasons other than switching to a new data directory (e.g. because an sstable size limit has been reached).

switchCompactionWriter

protected void switchCompactionWriter(Directories.DataDirectory directory,
 DecoratedKey nextKey)

Implementations of this method should finish the current sstable writer and start writing to this directory.

Called once before starting to append and then whenever we see a need to start writing to another directory.

Parameters:

directory -

nextKey -

sstableWriter

protected SSTableWriter sstableWriter(Directories.DataDirectory directory,
 DecoratedKey nextKey)

sstableLevel

protected int sstableLevel()

Returns the level that should be used when creating sstables.

sstableKeyCount

protected abstract long sstableKeyCount()

Returns the key count with which created sstables should be set up.

getDirectories

public Directories getDirectories()

The directories we can write to

getWriteDirectory

public Directories.DataDirectory getWriteDirectory(Iterable<SSTableReader> sstables,
 long estimatedWriteSize)

Return a directory where we can expect expectedWriteSize to fit.

Parameters:

sstables - the sstables to compact

setRepairedAt

public CompactionAwareWriter setRepairedAt(long repairedAt)

getExpectedWriteSize

protected long getExpectedWriteSize()

newWriterBuilder

protected SSTableWriter.Builder<?,?> newWriterBuilder(Descriptor descriptor)

It is up to the caller to set the following fields: - SSTableWriter.Builder.setKeyCount(long), - SSTableWriter.Builder.setSerializationHeader(SerializationHeader) and, - SSTableWriter.Builder.setMetadataCollector(MetadataCollector)