Package org.apache.cassandra.metrics

Class DecayingEstimatedHistogramReservoir

java.lang.Object
org.apache.cassandra.metrics.DecayingEstimatedHistogramReservoir
All Implemented Interfaces:
com.codahale.metrics.Reservoir, SnapshottingReservoir
public class DecayingEstimatedHistogramReservoir extends Object implements SnapshottingReservoir
A decaying histogram reservoir where values collected during each minute will be twice as significant as the values collected in the previous minute. Measured values are collected in variable sized buckets, using small buckets in the lower range and larger buckets in the upper range. Use this histogram when you want to know if the distribution of the underlying data stream has changed recently and you want high resolution on values in the lower range.

The histogram use forward decay [1] to make recent values more significant. The forward decay factor will be doubled every minute (half-life time set to 60 seconds) [2]. The forward decay landmark is reset every 30 minutes (or at first read/update after 30 minutes). The 30 minute rescale interval is used based on the assumption that in an extreme case we would have to collect a metric 1M times for a single bucket each second. By the end of the 30:th minute all collected values will roughly add up to 1.000.000 * 60 * pow(2, 30) which can be represented with 56 bits giving us some head room in a signed 64 bit long.

Internally two reservoirs are maintained, one with decay and one without decay. All public getters in a Snapshot will expose the decay functionality with the exception of the Snapshot.getValues() which will return values from the reservoir without decay. This makes it possible for the caller to maintain precise deltas in an interval of its choice.

The bucket size starts at 1 and grows by 1.2 each time (rounding and removing duplicates). It goes from 1 to around 18T by default (creating 164+1 buckets), which will give a timing resolution from microseconds to roughly 210 days, with less precision as the numbers get larger.

The series of values to which the counts in `decayingBuckets` correspond: 1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 14, 17, 20, 24, 29, 35, 42, 50, 60, 72 etc. Thus, a `decayingBuckets` of [0, 0, 1, 10] would mean we had seen 1 value of 3 and 10 values of 4.

Each bucket represents values from (previous bucket offset, current offset].

To reduce contention each logical bucket is striped accross a configurable number of stripes (default: 2). Threads are assigned to specific stripes. In addition, logical buckets are distributed across the physical storage to reduce conention when logically adjacent buckets are updated. See CASSANDRA-15213.

  • [1]: http://dimacs.rutgers.edu/~graham/pubs/papers/fwddecay.pdf
  • [2]: https://en.wikipedia.org/wiki/Half-life
  • [3]: https://github.com/dropwizard/metrics/blob/v3.1.2/metrics-core/src/main/java/com/codahale/metrics/ExponentiallyDecayingReservoir.java
See Also:
  • ExponentiallyDecayingReservoir

  • Field Details

    • DEFAULT_BUCKET_COUNT

      public static final int DEFAULT_BUCKET_COUNT
      The default number of decayingBuckets. Use this bucket count to reduce memory allocation for bucket offsets.
      See Also:

    • LOW_BUCKET_COUNT

      public static final int LOW_BUCKET_COUNT
      See Also:

    • DEFAULT_STRIPE_COUNT

      public static final int DEFAULT_STRIPE_COUNT

    • MAX_BUCKET_COUNT

      public static final int MAX_BUCKET_COUNT
      See Also:

    • DEFAULT_ZERO_CONSIDERATION

      public static final boolean DEFAULT_ZERO_CONSIDERATION
      See Also:

    • DEFAULT_WITHOUT_ZERO_BUCKET_OFFSETS

      public static final long[] DEFAULT_WITHOUT_ZERO_BUCKET_OFFSETS

    • DEFAULT_WITH_ZERO_BUCKET_OFFSETS

      public static final long[] DEFAULT_WITH_ZERO_BUCKET_OFFSETS

    • HALF_TIME_IN_S

      public static final long HALF_TIME_IN_S
      See Also:

    • MEAN_LIFETIME_IN_S

      public static final double MEAN_LIFETIME_IN_S

    • LANDMARK_RESET_INTERVAL_IN_NS

      public static final long LANDMARK_RESET_INTERVAL_IN_NS

  • Constructor Details

    • DecayingEstimatedHistogramReservoir

      public DecayingEstimatedHistogramReservoir()
      Construct a decaying histogram with default number of buckets and without considering zeroes.

    • DecayingEstimatedHistogramReservoir

      public DecayingEstimatedHistogramReservoir(boolean considerZeroes)
      Construct a decaying histogram with default number of buckets.
      Parameters:
      considerZeroes - when true, 0-value measurements in a separate bucket, otherwise they will be collected in same bucket as 1-value measurements

    • DecayingEstimatedHistogramReservoir

      public DecayingEstimatedHistogramReservoir(boolean considerZeroes, int bucketCount, int stripes)
      Construct a decaying histogram.
      Parameters:
      considerZeroes - when true, 0-value measurements in a separate bucket, otherwise they will be collected in same bucket as 1-value measurements
      bucketCount - number of buckets used to collect measured values

    • DecayingEstimatedHistogramReservoir

      public DecayingEstimatedHistogramReservoir(MonotonicClock clock)

    • DecayingEstimatedHistogramReservoir

      public DecayingEstimatedHistogramReservoir(boolean considerZeroes, int bucketCount, int stripes, MonotonicClock clock, long landmarkResetIntervalInNs)

  • Method Details

    • fastLog12

      public static float fastLog12(long v)

    • update

      public void update(long value)
      Increments the count of the bucket closest to n, rounding UP.
      Specified by:
      update in interface com.codahale.metrics.Reservoir
      Parameters:
      value - the data point to add to the histogram

    • updateBucket

      public void updateBucket(AtomicLongArray buckets, int index, long value)

    • stripedIndex

      public int stripedIndex(int offsetIndex, int stripe)

    • findIndex

      public static int findIndex(long[] bucketOffsets, long value)

    • size

      public int size()
      Returns the logical number of buckets where recorded values are stored. The actual number of physical buckets is size() * stripeCount() This method does not return the number of recorded values as suggested by the Reservoir interface.
      Specified by:
      size in interface com.codahale.metrics.Reservoir
      Returns:
      the number of buckets
      See Also:

    • stripeCount

      public int stripeCount()

    • getSnapshot

      public com.codahale.metrics.Snapshot getSnapshot()
      Returns a snapshot of the decaying values in this reservoir. Non-decaying reservoir will not be included in the snapshot.
      Specified by:
      getSnapshot in interface com.codahale.metrics.Reservoir
      Returns:
      the snapshot

    • getPercentileSnapshot

      public com.codahale.metrics.Snapshot getPercentileSnapshot()
      Specified by:
      getPercentileSnapshot in interface SnapshottingReservoir

    • clear

      public void clear()