Class TestEvoLudo

Object
TestEvoLudo
All Implemented Interfaces:
RunListener
public class TestEvoLudo extends Object implements RunListener
TestEvoLudo is a test suite for EvoLudo. It generates test cases from a generator file or directory and compares the results with reference files. The test suite can be run in two modes: (1) generating test cases from a generator file or directory, and (2) testing the generated or existing test cases. The test suite accepts the following command line options:
  • --generate <filename>: read option sets from file and generate test cases
  • --tests <directory>: directory for storing/retrieving test cases (defaults to references)
  • --references <directory>: directory for storing/retrieving test cases
  • --reports <directory>: directory for storing failed test reports
  • --compress: compress generated test files
  • --minor: dump differences for minor failures
  • --verb: verbose mode
  • --help, -h or no arguments: this help screen
Author:
Christoph Hauert

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    (package private) static enum 
    TestEvoLudo.ConsoleColors
    The control codes for changing the style of the console output.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    (package private) boolean
    dumpMinor
    The flag to indicate whether to dump differences for minor failures.
    (package private) EvoLudoJRE
    engine
    Pointer to engine.
    (package private) File
    generator
    Directory with generator scripts for tests.
    (package private) boolean
    isRunning
    The flag to indicate whether tests are running.
    (package private) Logger
    logger
    Logger for keeping track of and reporting events and issues.
    (package private) int
    nTestFailures
    The number of failed tests.
    (package private) int
    nTestMinor
    The number of tests failing with minor errors.
    (package private) int
    nTests
    The total number of tests.
    (package private) int
    nTestWarnings
    The number of tests with warnings.
    (package private) PrintStream
    output
    The output stream for logging messages.
    (package private) boolean
    performTest
    The flag to indicate whether to perform tests or generate test cases.
    (package private) File
    referencesDir
    Directory with reference results.
    (package private) File
    reportsDir
    Directory for storing reports of failed tests.
    (package private) static final Collection<String>
    SHA_EXCLUDE
    List of keys to exclude from test case comparisons.
    (package private) static final String
    SHA_PREFIX
    The prefix to mark the beginning of the SHA hash in the filename.
    (package private) boolean
    skipSHA
    The flag to indicate whether to skip SHA checks.
    (package private) File
    testsDir
    Directory containing tests or for storing generated tests.
    (package private) boolean
    useCompression
    The flag to indicate whether to use compression for the generated test files.
    (package private) boolean
    verbose
    The flag to indicate whether to run in verbose mode.

  • Constructor Summary

    Constructors
    Constructor
    Description
    TestEvoLudo()
    Constructor for TestEvoLudo.

  • Method Summary

    Modifier and Type
    Method
    Description
    private File
    checkReference(File references, Plist result, String refbasename)
    Search for reference file with the name base refbasename (no SHA, no extensions) in directory references.
    private boolean
    compareRuns(File refname, Plist reference, Plist replicate)
    Compare reference output reference with the output of the test replicate and generate a report if differences are found.
    void
    generate()
    Generate test cases from generator file or directory.
    protected void
    generate(File clo)
    Generate test cases from generator file clo.
    private String
    generateBasename(String clo, int idx)
    Generate the base name for tests from command line options clo and index idx.
    static String
    hashSHA256(Object obj)
    Compute SHA-256 hash of serializable object obj.
    void
    help()
    Print help screen.
    void
    logError(String msg)
    Log error message in red color to console or logger.
    void
    logMessage(String msg)
    Log message to console or logger.
    void
    logOk(String msg)
    Log message in green color to console or logger.
    void
    logTitle(String msg)
    Log bold message to console or logger.
    void
    logWarning(String msg)
    Log warning message in yellow color to console or logger.
    static void
    main(String[] args)
    Entry point for test routines of EvoLudo.
    void
    modelStopped()
    Called after a running EvoLudo model stopped because the model converged (or reached an absorbing state).
    void
    parse(String[] args)
    Parse the command line options.
    boolean
    run()
    Generate or run all tests.
    private boolean
    runModule(String task, String clo)
    Load and run module with parameters clo.
    private static File
    search(File file, String search)
    Recursively search for a file whose name starts with search in directory file.
    static String
    sha256(Plist plist, Collection<String> exclude)
    Compute the SHA-256 hash of the plist, excluding the keys in exclude.
    private String
    stripExport(String clo)
    Strip export option from command line options clo.
    void
    test(File dir)
    Test all files in directory dir.

    Methods inherited from class Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface RunListener

    modelDidInit, modelDidReset, modelRelaxed, modelRunning, modelSettings

  • Field Details

    • SHA_EXCLUDE

      static final Collection<String> SHA_EXCLUDE
      List of keys to exclude from test case comparisons.

    • SHA_PREFIX

      static final String SHA_PREFIX
      The prefix to mark the beginning of the SHA hash in the filename.
      See Also:

    • engine

      EvoLudoJRE engine
      Pointer to engine. Engine has EvoLudoJRE class but do not rely on JRE specifics.

    • logger

      Logger logger
      Logger for keeping track of and reporting events and issues.

    • output

      PrintStream output
      The output stream for logging messages.

    • testsDir

      File testsDir
      Directory containing tests or for storing generated tests.

    • reportsDir

      File reportsDir
      Directory for storing reports of failed tests.

    • referencesDir

      File referencesDir
      Directory with reference results.

    • generator

      File generator
      Directory with generator scripts for tests.

    • performTest

      boolean performTest
      The flag to indicate whether to perform tests or generate test cases.

    • isRunning

      boolean isRunning
      The flag to indicate whether tests are running.

    • useCompression

      boolean useCompression
      The flag to indicate whether to use compression for the generated test files.

    • dumpMinor

      boolean dumpMinor
      The flag to indicate whether to dump differences for minor failures.

    • verbose

      boolean verbose
      The flag to indicate whether to run in verbose mode.

    • skipSHA

      boolean skipSHA
      The flag to indicate whether to skip SHA checks.

    • nTests

      int nTests
      The total number of tests.

    • nTestFailures

      int nTestFailures
      The number of failed tests.

    • nTestMinor

      int nTestMinor
      The number of tests failing with minor errors.

    • nTestWarnings

      int nTestWarnings
      The number of tests with warnings.

  • Constructor Details

    • TestEvoLudo

      public TestEvoLudo()
      Constructor for TestEvoLudo.

  • Method Details

    • generate

      public void generate()
      Generate test cases from generator file or directory.

    • generate

      protected void generate(File clo)
      Generate test cases from generator file clo.
      Parameters:
      clo - the generator file

    • test

      public void test(File dir)
      Test all files in directory dir. This directory can either contain test files or files for generating them. In either case the test output is compared to the reference files.
      Parameters:
      dir - the directory with test files

    • sha256

      public static String sha256(Plist plist, Collection<String> exclude)
      Compute the SHA-256 hash of the plist, excluding the keys in exclude.
      Parameters:
      plist - the property list to hash
      exclude - the keys to exclude from the hash
      Returns:
      the SHA-256 hash as a hexadecimal string

    • hashSHA256

      public static String hashSHA256(Object obj) throws IOException, NoSuchAlgorithmException
      Compute SHA-256 hash of serializable object obj.
      Parameters:
      obj - the object to hash
      Returns:
      the SHA-256 hash as a hexadecimal string
      Throws:
      IOException - if an I/O error occurs
      NoSuchAlgorithmException - if SHA-256 is not supported

    • generateBasename

      private String generateBasename(String clo, int idx)
      Generate the base name for tests from command line options clo and index idx.
      Parameters:
      clo - the command line options
      idx - the index of the test
      Returns:
      the base filename

    • stripExport

      private String stripExport(String clo)
      Strip export option from command line options clo.
      Parameters:
      clo - the command line options
      Returns:
      the stripped command line options

    • runModule

      private boolean runModule(String task, String clo)
      Load and run module with parameters clo. The options --seed 0 is prepended and --delay 0 is appended to clo. Because for options that are specified multiple times the latter takes precedence, this ensures that the results are reproducible with potentially custom seeds and also run at full speed. Returns false if the module didn't run and hence no export was generated, even if requested with the --export option. This happens for example with --timestop 0.
      Parameters:
      task - the task that is running (generating or testing)
      clo - the command line options for running the module
      Returns:
      true if the module ran successfully

    • compareRuns

      private boolean compareRuns(File refname, Plist reference, Plist replicate)
      Compare reference output reference with the output of the test replicate and generate a report if differences are found. The reference is stored in refname. The method returns true if the test passed.
      Parameters:
      refname - the name of the reference file
      reference - the reference Plist
      replicate - the replicate Plist
      Returns:
      true if the test passed

    • checkReference

      private File checkReference(File references, Plist result, String refbasename)
      Search for reference file with the name base refbasename (no SHA, no extensions) in directory references. If refbasename is found verify the results in the plist result.
      Parameters:
      references - the directory with reference files
      result - the Plist with the results
      refbasename - the base name of the reference file (if it exists)
      Returns:
      if verification successful return File pointing to reference; if unsuccesssful return null; and if reference file not found return references

    • search

      private static File search(File file, String search)
      Recursively search for a file whose name starts with search in directory file.
      Parameters:
      file - the directory to search
      search - the name of the file to search for
      Returns:
      the file if found, otherwise null

    • modelStopped

      public void modelStopped()
      Description copied from interface: RunListener
      Called after a running EvoLudo model stopped because the model converged (or reached an absorbing state).
      Specified by:
      modelStopped in interface RunListener

    • run

      public boolean run()
      Generate or run all tests.
      Returns:
      true if successful

    • parse

      public void parse(String[] args)
      Parse the command line options.
      Parameters:
      args - the command line options

    • help

      public void help()
      Print help screen.

    • main

      public static void main(String[] args)
      Entry point for test routines of EvoLudo.
      Parameters:
      args - the string of command line arguments

    • logMessage

      public void logMessage(String msg)
      Log message to console or logger.
      Parameters:
      msg - the message to log

    • logTitle

      public void logTitle(String msg)
      Log bold message to console or logger.
      Parameters:
      msg - the message to log

    • logOk

      public void logOk(String msg)
      Log message in green color to console or logger.
      Parameters:
      msg - the message to log

    • logWarning

      public void logWarning(String msg)
      Log warning message in yellow color to console or logger.
      Parameters:
      msg - the warning to log

    • logError

      public void logError(String msg)
      Log error message in red color to console or logger.
      Parameters:
      msg - the error to log