Package io.narayana.perf

Class PerformanceProfileStore

java.lang.Object
io.narayana.perf.PerformanceProfileStore
public class PerformanceProfileStore extends Object
Maintain performance data and check for regressions. Performance data and configuration is stored in a directory named by the system property BASE_DIRECTORY_PROPERTY: - PerformanceProfileStore.last holds the best performance run keyed by the name of the test - PerformanceProfileStore.variance contains the variance for a test (keyed by test name) or, if not present, then the default variance (eg 1.1 indicates a variance of lest than 10%) - PerformanceProfileStore.args contains any arguments required by a test keyed by test name with value a comma separated string (the configured values for arguments can be overridden by setting a system property called "testname.args" to the new value) To disable regression checks set the boolean property FAIL_ON_PERF_REGRESSION_PROP To reset performance data for a test set the boolean property RESET_NETRICS_PROP

  • Field Details

  • Constructor Details

    • PerformanceProfileStore

      public PerformanceProfileStore()

  • Method Details

    • isResetMetrics

      public static boolean isResetMetrics()

    • isFailOnRegression

      public static boolean isFailOnRegression()

    • getVariance

      public static float getVariance()

    • getVariance

      public static float getVariance(String metricName)

    • getMetric

      public static Float getMetric(String metricName)

    • checkPerformance

      public static boolean checkPerformance(String metricName, float metricValue)

    • checkPerformance

      public static boolean checkPerformance(String metricName, float metricValue, boolean largerIsBetter)

    • checkPerformance

      public static boolean checkPerformance(StringBuilder info, String metricName, float metricValue, boolean largerIsBetter)

    • regressionCheck

      public static <T> Measurement<T> regressionCheck(WorkerWorkload<T> workload, String metricName, boolean useConfigArgs, int warmUpCount, int numberOfCalls, int threadCount, int batchSize)
      Measure the performance of a workload. The returned Measurement object contains the results of the measurement.
      Type Parameters:
      T - caller specific context data
      Parameters:
      workload - the actual workload being measured
      metricName - the name of the test used as a key into performance data PERFDATAFILENAME
      useConfigArgs - if true read test arguments from a file (getTestArgs(String)
      warmUpCount - Number of iterations of the workload to run before starting the measurement
      numberOfCalls - Number of workload iterations (workload is called in batchSize batches until numberOfCalls is reached)
      threadCount - Number of threads used to complete the workload
      batchSize - The workload is responsible for running batchSize iterations on each call
      Returns:
      the result of the measurement

    • regressionCheck

      public static <T> Measurement<T> regressionCheck(WorkerWorkload<T> workload, String metricName, boolean useConfigArgs, long maxTestTime, int warmUpCount, int numberOfCalls, int threadCount, int batchSize)
      Measure the performance of a workload. The returned Measurement object contains the results of the measurement.
      Type Parameters:
      T - caller specific context data
      Parameters:
      workload - the actual workload being measured
      metricName - the name of the test used as a key into performance data PERFDATAFILENAME
      useConfigArgs - if true read test arguments from a file (getTestArgs(String)
      maxTestTime - Abort the measurement if this time (in msecs) is exceeded TODO
      warmUpCount - Number of iterations of the workload to run before starting the measurement
      numberOfCalls - Number of workload iterations (workload is called in batchSize batches until numberOfCalls is reached)
      threadCount - Number of threads used to complete the workload
      batchSize - The workload is responsible for running batchSize iterations on each call
      Returns:
      the result of the measurement

    • regressionCheck

      public static <T> Measurement<T> regressionCheck(WorkerLifecycle lifecycle, WorkerWorkload<T> workload, String metricName, boolean useConfigArgs, long maxTestTime, int warmUpCount, int numberOfCalls, int threadCount, int batchSize)
      Measure the performance of a workload. The returned Measurement object contains the results of the measurement.
      Type Parameters:
      T - caller specific context data
      Parameters:
      lifecycle - lifecycle calls during the measurement
      workload - the actual workload being measured
      metricName - the name of the test used as a key into performance data PERFDATAFILENAME
      useConfigArgs - if true read test arguments from a file (getTestArgs(String)
      maxTestTime - Abort the measurement if this time (in msecs) is exceeded TODO
      warmUpCount - Number of iterations of the workload to run before starting the measurement
      numberOfCalls - Number of workload iterations (workload is called in batchSize batches until numberOfCalls is reached)
      threadCount - Number of threads used to complete the workload
      batchSize - The workload is responsible for running batchSize iterations on each call
      Returns:
      the result of the measurement

    • getArg

      public static <T> T getArg(String metricName, String[] args, int index, T defaultValue, Class<T> argClass)
      Convert a String to another type
      Type Parameters:
      T - the type which args[index] should be converted to
      Parameters:
      metricName - the name of the test to use if there was a data format error
      args - arguments returned from a prior call to getTestArgs(String)
      index - index into the args array to the value to be converted
      defaultValue - default value if the index is out of range
      argClass - class type to convert the value to (which must have a constructor that takes a String value)
      Returns:
      the converted value

    • getTestArgs

      public static String[] getTestArgs(String metricName)
      Lookup any configured test arguments PERFARGSFILENAME
      Parameters:
      metricName - the name of the test
      Returns:
      any test arguments or an empty array if there are none

    • getMatchingMetrics

      public static Map<String,Float> getMatchingMetrics(String pattern)
      get metrics matching a particular pattern
      Parameters:
      pattern - the regex used as the pattern
      Returns:
      a map of matching metric names to the current value of the metric