Package org.apache.cassandra.db

Class Directories

java.lang.Object
org.apache.cassandra.db.Directories
public class Directories extends Object
Encapsulate handling of paths to the data files. 
 
   /<path_to_data_dir>/ks/<cf dir>/ks-cf1-jb-1-Data.db
                         /<cf dir>/la-2-Data.db
                         /<cf dir>/.<index name>/ks-cf1.idx-jb-1-Data.db
                         /<cf dir>/.<index name>/la-1-Data.db
                         ...
Until v2.0, <cf dir> is just column family name. Since v2.1, <cf dir> has column family ID(tableId) added to its end. SSTables from secondary indexes were put in the same directory as their parent. Since v2.2, they have their own directory under the parent directory whose name is index name. Upon startup, those secondary index files are moved to new directory when upgrading. For backward compatibility, Directories can use directory without tableId if exists. In addition, more that one 'root' data directory can be specified so that <path_to_data_dir> potentially represents multiple locations. Note that in the case of multiple locations, the manifest for the leveled compaction is only in one of the location. Snapshots (resp. backups) are always created along the sstables there are snapshotted (resp. backuped) but inside a subdirectory named 'snapshots' (resp. backups) (and snapshots are further inside a subdirectory of the name of the snapshot). For secondary indexes, snapshots (backups) are not created in their own directory, but are in their parent's snapshot (backup) directory. This class abstracts all those details from the rest of the code.

  • Field Details

  • Constructor Details

  • Method Details

    • verifyFullPermissions

      public static boolean verifyFullPermissions(File dir, String dataDir)
      Checks whether Cassandra has RWX permissions to the specified directory. Logs an error with the details if it does not.
      Parameters:
      dir - File object of the directory.
      dataDir - String representation of the directory's location
      Returns:
      status representing Cassandra's RWX permissions to the supplied folder location.

    • getLocationForDisk

      public File getLocationForDisk(Directories.DataDirectory dataDirectory)
      Returns SSTable location which is inside given data directory.
      Parameters:
      dataDirectory -
      Returns:
      SSTable location

    • getDataDirectoryForFile

      public Directories.DataDirectory getDataDirectoryForFile(Descriptor descriptor)

    • find

      public Descriptor find(String filename)

    • getDirectoryForNewSSTables

      public File getDirectoryForNewSSTables()
      Basically the same as calling getWriteableLocationAsFile(long) with an unknown size (-1L), which may return any allowed directory - even a data directory that has no usable space. Do not use this method in production code.
      Throws:
      FSWriteError - if all directories are disallowed.

    • getWriteableLocationAsFile

      public File getWriteableLocationAsFile(long writeSize)
      Returns an allowed directory that _currently_ has writeSize bytes as usable space.
      Throws:
      FSWriteError - if all directories are disallowed.

    • getWriteableLocationToLoadFile

      public File getWriteableLocationToLoadFile(File sourceFile)
      Returns a data directory to load the file sourceFile. If the sourceFile is on same disk partition as any data directory then use that one as data directory otherwise use getWriteableLocationAsFile(long) to find suitable data directory. Also makes sure returned directory is not disallowed.
      Throws:
      FSWriteError - if all directories are disallowed.

    • getTemporaryWriteableDirectoryAsFile

      public File getTemporaryWriteableDirectoryAsFile(long writeSize)
      Returns a temporary subdirectory on allowed data directory that _currently_ has writeSize bytes as usable space. This method does not create the temporary directory.
      Throws:
      IOError - if all directories are disallowed.

    • removeTemporaryDirectories

      public void removeTemporaryDirectories()

    • getWriteableLocation

      public Directories.DataDirectory getWriteableLocation(long writeSize)
      Returns an allowed data directory that _currently_ has writeSize bytes as usable space.
      Throws:
      FSWriteError - if all directories are disallowed.

    • hasDiskSpaceForCompactionsAndStreams

      public boolean hasDiskSpaceForCompactionsAndStreams(Map<File,Long> expectedNewWriteSizes, Map<File,Long> totalCompactionWriteRemaining)
      Sums up the space required for ongoing streams + compactions + expected new write size per FileStore and checks if there is enough space available.
      Parameters:
      expectedNewWriteSizes - where we expect to write the new compactions
      totalCompactionWriteRemaining - approximate amount of data current compactions are writing - keyed by the file store they are writing to (or, reading from actually, but since CASSANDRA-6696 we expect compactions to read and written from the same dir)
      Returns:
      true if we expect to be able to write expectedNewWriteSizes to the available file stores

    • hasDiskSpaceForCompactionsAndStreams

      public static boolean hasDiskSpaceForCompactionsAndStreams(Map<File,Long> expectedNewWriteSizes, Map<File,Long> totalCompactionWriteRemaining, Function<File,FileStore> filestoreMapper)

    • hasDiskSpaceForCompactionsAndStreams

      public static boolean hasDiskSpaceForCompactionsAndStreams(Map<FileStore,Long> totalToWrite)
      Checks if there is enough space on all file stores to write the given amount of data. The data to write should be the total amount, ongoing writes + new writes.

    • getAvailableSpaceForCompactions

      public static long getAvailableSpaceForCompactions(FileStore fileStore)

    • perFileStore

      public static Map<FileStore,Long> perFileStore(Map<File,Long> perDirectory, Function<File,FileStore> filestoreMapper)

    • allFileStores

      public Set<FileStore> allFileStores(Function<File,FileStore> filestoreMapper)

    • getFileStore

      public static FileStore getFileStore(File directory)
      Gets the filestore for the actual directory where the sstables are stored. Handles the fact that an operator can symlink a table directory to a different filestore.

    • getWriteableLocations

      public Directories.DataDirectory[] getWriteableLocations()

    • getSnapshotDirectory

      public static File getSnapshotDirectory(Descriptor desc, String snapshotName)

    • getSnapshotDirectory

      public static File getSnapshotDirectory(File location, String snapshotName)
      Returns directory to write snapshot. If directory does not exist, then one is created. If given location indicates secondary index, this will return <cf dir>/snapshots/<snapshot name>/.<index name>. Otherwise, this will return <cf dir>/snapshots/<snapshot name>.
      Parameters:
      location - base directory
      snapshotName - snapshot name
      Returns:
      directory to write snapshot

    • getSnapshotDirectoryIfExists

      public static Optional<File> getSnapshotDirectoryIfExists(File location, String snapshotName)
      Returns directory to write a snapshot to. If directory does not exist, then it is NOT created. If given location indicates secondary index, this will return <cf dir>/snapshots/<snapshot name>/.<index name>. Otherwise, this will return <cf dir>/snapshots/<snapshot name>.
      Parameters:
      location - base directory
      snapshotName - snapshot name
      Returns:
      directory to write a snapshot

    • getSnapshotManifestFile

      public File getSnapshotManifestFile(String snapshotName)

    • getSnapshotManifestFile

      public static File getSnapshotManifestFile(File snapshotDir)

    • getSnapshotSchemaFile

      public File getSnapshotSchemaFile(String snapshotName)

    • getSnapshotSchemaFile

      public static File getSnapshotSchemaFile(File snapshotDir)

    • getBackupsDirectory

      public static File getBackupsDirectory(Descriptor desc)

    • getBackupsDirectory

      public static File getBackupsDirectory(File location)
      Returns directory to write a backup to. If directory does not exist, then one is created. If given location indicates secondary index, this will return <cf dir>/backups/.<index name>. Otherwise, this will return <cf dir>/backups/.
      Parameters:
      location - base directory
      Returns:
      directory to write a backup

    • getBackupsDirectoryIfExists

      public static Optional<File> getBackupsDirectoryIfExists(File location)
      Returns directory to write a backup to. If directory does not exist, then it is NOT created. If given location indicates secondary index, this will return <cf dir>/backups/.<index name>. Otherwise, this will return <cf dir>/backups/.
      Parameters:
      location - base directory
      Returns:
      directory to write a backup

    • isStoredInLocalSystemKeyspacesDataLocation

      public static boolean isStoredInLocalSystemKeyspacesDataLocation(String keyspace, String table)
      Checks if the specified table should be stored with local system data.

      To minimize the risk of failures, SSTables for local system keyspaces must be stored in a single data directory. The only exception to this are some of the system table as the server can continue operating even if those tables loose some data.

      Parameters:
      keyspace - the keyspace name
      table - the table name
      Returns:
      true if the specified table should be stored with local system data, false otherwise.

    • sstableLister

      public Directories.SSTableLister sstableLister(Directories.OnTxnErr onTxnErr)

    • sstableLister

      public Directories.SSTableLister sstableLister(File directory, Directories.OnTxnErr onTxnErr)

    • listSnapshots

      public Map<String,TableSnapshot> listSnapshots()

    • maybeLoadManifest

      protected static SnapshotManifest maybeLoadManifest(String keyspace, String table, String tag, Set<File> snapshotDirs)

    • listSnapshotDirsByTag

      protected Map<String,Set<File>> listSnapshotDirsByTag()

    • snapshotExists

      public boolean snapshotExists(String snapshotName)

    • clearSnapshot

      public static void clearSnapshot(String snapshotName, List<File> tableDirectories, com.google.common.util.concurrent.RateLimiter snapshotRateLimiter)

    • removeSnapshotDirectory

      public static void removeSnapshotDirectory(com.google.common.util.concurrent.RateLimiter snapshotRateLimiter, File snapshotDir)

    • trueSnapshotsSize

      public long trueSnapshotsSize()
      Returns:
      total snapshot size in byte for all snapshots.

    • getRawDiretoriesSize

      public long getRawDiretoriesSize()
      Returns:
      Raw size on disk for all directories

    • getTrueAllocatedSizeIn

      public long getTrueAllocatedSizeIn(File snapshotDir)

    • getKSChildDirectories

      public static List<File> getKSChildDirectories(String ksName)

    • isSecondaryIndexFolder

      public static boolean isSecondaryIndexFolder(File dir)

    • isSecondaryIndexFolder

      public static boolean isSecondaryIndexFolder(Path dir)

    • getCFDirectories

      public List<File> getCFDirectories()

    • getUIDGenerator

      public <T extends SSTableId> Supplier<T> getUIDGenerator(SSTableId.Builder<T> builder)
      Initializes the sstable unique identifier generator using a provided builder for this instance of directories. If the id builder needs that, sstables in these directories are listed to provide the existing identifiers to the builder. The listing is done lazily so if the builder does not require that, listing is skipped.

    • get

      public static Optional<File> get(File base, String... subdirs)