PureTest - Scenario Runner

Scenario Runner
PureTest 4.0 June 2009	http://www.minq.se support@minq.se

Documentation Index

Scenario Runner
- Usage
Ant Task
Scenario Runner Task Reference
- Parameters
- Examples

Scenario Runner

Typically a product life cycle includes several releases. This could be bug releases and releases where new functions has been added. Every release includes time consuming and costly re-test activities. The concept of automatic regression test aims to reduce the cost and time for these tests.

PureTest include a command line utility, Scenario Runner, supporting this by executing scenarios in a PureLoad Configuration (PLC) file. Also included is a a Jakarta Ant task to support integration of regression tests into a build process.

Usage

The Scenario Runner is started from the command prompt as follows:

% install-home/bin/puretest runner <scenario file>

where install-home is the directory where PureTest is installed.

There are a number of options available to control the output from the runner. Starting the runner without arguments shows a help text with the available options:

Usage:
runner [-v   OFF | ERROR | INFO | DEBUG]   (runner verbose level)
       [-log OFF | ERROR | INFO | DEBUG]   (task log level)
       [-se]                               (stop execution within scnario on error)
       <scenario file> ...                 (scenario files to execute)

Default is '-v INFO -log ERROR'.

Options:

-v

controls the verbose level of the runner itself. Default is INFO.

OFF
No information displayed. Only exit code indicates success or failure.
ERROR
Show only error information and final summary per executed scenarios.
INFO
Show execution information and execution time information as well as final summary per executed scenarios.
DEBUG
Show more detailed information about scenarios executed and complete summay for scenarios and tasks executed.

-log: controls the log level used for all log output from the tasks being executed. Default is ERROR to only display info when tasks fail.
-se: a flag to control the behavior when an error is encountered. If set, the runner will stop executing if any task fails. The default behavior is to continue execution in case of error.

The default verbose and log levels (-v INFO and -log ERROR) will show execution time for each scenario, information on errors if such occurs and a final should summary if scenario(s) failed or executed ok. Set both -v and -log to OFF to run in completely silent mode.

The rest of the arguments are treated as PLC files.

When the runner exits, it will return an exit status code. Exit status 0 means success, 1 means one or more execution errors and 2 means invalid input or failure to load scenario file.

Note: The runner will skip the scenarios in the file that have any kind of distribution associated with them (if defined by using PureLoad).

Ant Task

The Jakarta Project refers to the build tool Ant as the "make without make's wrinkles". Ant is becoming the de facto standard in the Java and open-source world.

In the examples/ant directory provided with PureTest you a simple example of and build file and PLC files.

The following section assumes general Ant knowledge, but should be (together with the examples) enough to get started using Ant and PureTest

Scenario Runner Task Reference

The Scenario Runner Ant task is a simple ant task for invoking the scenario-runner provided with PureTest to execute scenarios defined in PLC file(s).

A set of PLC files to be executed can be defined using the dir attribute in combination with includes ad excludes attributes. If only one PLC file is to be executed use the file attribute.

Parameters

Attribute	Description	Required
dir	Where to fin the PC files.	Yes, one of file or dir
file	Name of a single PLC file.	Yes, one of file or dir
includes	Comma- or space-separated list of patterns of files in dir that should be included. All files are included when omitted.	No
excludes	Comma- or space-separated list of patterns of files in dir that should be excluded. No files are excluded when omitted.	No
verbose	Controls the verbose level of the runner itself. Default is INFO.	No
tasklog	Controls the log level used for all log output from the tasks being executed. Default is ERROR.	No
stoponerror	Stop execution of tasks in executing scenario on errors.	No
failonerror	Stop the buildprocess if execution of scenario fails.	No
classpath	The classpath to use to execute scenario runner.	No
classpathref	the classpath to use to execute scenario runner, given as reference to a path defined elsewhere.	No

Examples

Set Classpath

Used class path must include the jar-files in the lib directory of the PureTest distribution. The recommended way is to use a path reference as follows:

<property name="pureload.home" location="/home/janne/puretest-4.0"/>


<path id="pureload.path">
    <fileset dir="${pureload.home}/lib">
      <include name="**/*.jar"/>
    </fileset>
</path>

where the pureload.home property should be set to where your installation is located.

Define Task

To define the scenario-runner task, you can now do:

<taskdef name="scenario-runner" classpathref="pureload.path"
classname="com.pureload.tools.ant.ScenarioRunnerAntTask">
</taskdef>

Execute a PLC file

Now you can define targets to execute PLC files as follows:

<target name="test1">
      <scenario-runner classpathref="pureload.path"
                       file="plc/test.plc"
                       verbose="INFO" tasklog="ERROR"/>
   </target>

   
   <target name="test2">
      <scenario-runner classpathref="pureload.path"
                       dir="plc" includes="**/*.plc"
                       verbose="INFO" tasklog="ERROR"/>
   </target>