Class AbstractBenchmarkRunner

java.lang.Object
it.univr.di.cstnu.util.AbstractBenchmarkRunner
Direct Known Subclasses:
DispatchabilityBenchmarkRunner, MDEBenchmarkRunner
abstract class AbstractBenchmarkRunner extends Object
Base class for the benchmark runners that measure the average execution time (and standard deviation) of STNU algorithms on a given set of instances.

It provides the command-line options common to all runners (input files, output file, number of repetitions, timeout, save flag, version), the management of the output stream, and the infrastructure to execute an algorithm several times on fresh instances, collecting the execution-time statistics (repeatMeasure(String, String, Supplier, AlgorithmExecutor)).

Author:
posenato

  • Field Details

    • CSVSep

      static final String CSVSep
      CSV separator
      See Also:

    • currentEdgeImplClass

      final Class<STNUEdgeInt> currentEdgeImplClass
      Class for representing an edge.

    • inputFiles

      String[] inputFiles
      The input file names. Each file has to contain an STNU graph in GraphML format.

    • instances

      List<File> instances
      Input files as File objects.

    • nDCRepetition

      int nDCRepetition
      Parameter for asking how many times to execute each algorithm on each STNU.

    • output

      PrintStream output
      Output stream to outputFile

    • outputFile

      File outputFile
      Output file where to write the determined experimental execution times in CSV format.

    • save

      boolean save

    • timeOut

      int timeOut
      Parameter for asking timeout in sec.

    • versionReq

      boolean versionReq
      Software Version.

  • Constructor Details

    • AbstractBenchmarkRunner

      AbstractBenchmarkRunner()

  • Method Details

    • getNow

      static String getNow()
      Returns:
      current time in dateFormatter format

    • nanoSeconds2Seconds

      static double nanoSeconds2Seconds(double value)
      Parameters:
      value - value in nanoseconds
      Returns:
      the value in seconds

    • getOrCreateStats

      static org.apache.commons.math3.stat.descriptive.SummaryStatistics getOrCreateStats(@Nonnull it.unimi.dsi.fastutil.objects.Object2ObjectMap<AbstractBenchmarkRunner.GlobalStatisticsKey, org.apache.commons.math3.stat.descriptive.SummaryStatistics> map, @Nonnull AbstractBenchmarkRunner.GlobalStatisticsKey key)
      Returns the summary statistics associated with key in map, creating and registering it if absent.
      Parameters:
      map - one of the global-statistics maps of the runner.
      key - key of the statistics.
      Returns:
      the (possibly new) summary statistics for key.

    • getVersionAndDate

      abstract String getVersionAndDate()
      Returns:
      the version-and-date string of the concrete runner.

    • validateAlgorithmOptions

      @Nullable abstract String validateAlgorithmOptions()
      Checks the consistency of the algorithm-selection options of the concrete runner.
      Returns:
      null if the options are consistent; the error message to show otherwise.

    • makeSTNUInstance

      final STNU makeSTNUInstance(DenseTCGraph<STNUEdge> g)
      Returns an STNU object filled with the given graph and timeOut set as the parameter timeOut.
      Parameters:
      g - input graph
      Returns:
      an STNU instance

    • manageParameters

      final boolean manageParameters(String[] args)
      Simple method to manage command line parameters using args4j library.
      Parameters:
      args - input arguments
      Returns:
      false if a parameter is missing or wrong. True if every parameter is given in the right format.

    • repeatMeasure

      final AbstractBenchmarkRunner.RepetitionOutcome repeatMeasure(@Nonnull String fileName, @Nonnull String algLogName, @Nonnull Supplier<STNU> freshInstance, @Nonnull AbstractBenchmarkRunner.AlgorithmExecutor algorithm)
      Executes algorithm up to nDCRepetition times, each time on the fresh instance provided by freshInstance, collecting the execution time of each successful repetition. It stops earlier if a repetition times out or fails.
      Parameters:
      fileName - name of the network under test (used only for logging).
      algLogName - name of the algorithm under test (used only for logging).
      freshInstance - provider of the STNU instance to use in each repetition.
      algorithm - algorithm to execute and measure.
      Returns:
      the outcome of the repetitions.