Glossary

Active Agent

An agent that initiates a connection to the JavaTest harness.

Active agents allow you to run tests in parallel using many agents at once and to specify the test machines at the time you run the tests. Use the Agent Monitor window to view the list of registered active agents and synchronize active agents with the JavaTest harness before running tests.

Agent

A lightweight Java application that receives tests from the test harness, runs them on the implementation being tested, and reports the results back to the test harness. Normally test agents are only used when the TCK and implementation being tested are running on different platforms. When running tests on a platform other than the one running the JavaTest harness, you must use an agent. The JavaTest harness uses three types of agents:

Agent Monitor

The JavaTest window used to synchronize active agents and to monitor agent activity. The Agent Monitor window displays the agent pool and the agents currently in use. To open the Agent Monitor window, choose Tasks > Monitor Agent Activity from the menu bar.

Agent Pool

A list in the Agent Monitor of the active agents that are connected with the JavaTest harness and available to run tests. Agents are removed from the agent pool when they are running tests.

All Tests

A test tree view filter that displays all folders and tests in the test suite.

Audit

The JavaTest harness includes an audit tool that you can use to analyze the test results in a work directory. The audit tool verifies that all tests in a test suite ran correctly and identifies any audit categories of a test run that had errors.

You can audit a test run in GUI mode or in batch mode.

Batch Mode

The mode in which the JavaTest harness is run from the command line without using a GUI. Running in batch mode allows you to include the JavaTest harness as part of a product build process.

Configuration

Information about the computing environment, required to execute a test suite. The JavaTest harness uses the configuration editor to collect two types of data in an interview file:

Use the configuration editor to collect or modify configuration information or to load an existing configuration. See Configuration Editor.

Configuration Editor

The JavaTest Configuration Editor provides two views, the All Values View and the Standard Values view. Use the configuration editor's All Values view to create a configuration file for the test run and to search interview titles, questions, and answers for character strings. Use the configuration editor's Standard Values view to modify specific runtime values.

To open the Configuration Editor, choose Configure > Configuration Editor from the Test Manager menu bar.

Current Configuration Filter

The configuration containing the test environment and standard values currently loaded in the test manager or specified in batch mode for use in running tests and displaying test status.

Custom Filter

A filter that can be edited in the Filter Editor to include or exclude the results of specific tests or folders either in views of the test tree or in test reports.

Desktop

The configuration and layout of the windows used by the JavaTest harness.

The desktop is saved when you exit from the harness and is automatically restored in your next session.

The JavaTest desktop is displayed in three user selectable styles:

Environment

See Test Environment.

Environment Files

Contain one or more test environments used by test suites prior to the JavaTest 3.0 harness. Environment files are identified by the .jte extension in the file name.

Environment Variables

Name/value pairs used by a test environment to provide information about how to run tests of a test suite on a particular platform.

Exclude List

Exclude list files (*.jtx), supply a list of invalid tests to be filtered out of a test run by the test harness. The exclude list provides a level playing field for all implementors by ensuring that when a test is determined to be invalid, then no implementation is required to pass it. Exclude lists are maintained by the technology specification Maintenance Lead and are made available to all technology licensees.

Use the configuration editor to add or remove exclude lists from a test run.

To view the contents of an exclude list, choose Configure > Show Exclude List from the Test Manager menu bar. Exclude lists can only be edited or modified by the test suite Maintenance Lead.

Filtered Out

Folders and their tests that have been excluded from the test run by one or more test run filters.

Filtered out folders and tests are identified in the test tree by grey Gray folder icon folder and Gray test icon test icons.

Filters

A facility in the JavaTest harness that accepts or rejects tests based on a set of criteria. Used to configure the Test Manager view and to create test reports. Use the Configuration Editor or the filter editor to configure the filters.

HTTP Server

Software included in the JavaTest harness that services HTTP requests used to monitor a test run from a remote work station.

Interview Files

Contains all of the information collected by the configuration editor about the test platform.

The JavaTest harness uses the interview file (.jti) to derive the environment variables required execute the test suite.

When changes to the environment variables are required, use the configuration editor to modify the .jti file.

.jtb Files

An ASCII file containing a lengthy series of batch commands and their arguments used in the batch mode command line. Using a batch command file allows you to repeatedly use a batch configuration without retyping the commands each time a test run is performed.

It is recommended that a descriptive name and the extension .jtb are used to help identify the function of each batch file.

.jte Files

See Environment Files.

.jti Files

See Interview Files.

.jtr Files

See Test Result Files.

.jtx Files

See Exclude List.

Keywords

Special values in a test description that describe how the test is executed.

Keywords are provided by the test suite for use in the Configuration Editor as a filter to exclude or include tests in a test run.

MDI

See Multiple Document Interface.

Multiple Document Interface

A window style in which the JavaTest desktop is a single top-level window that contains all JavaTest windows opened to perform a task.

Use the JavaTest Preferences dialog box to select the MDI window style. See JavaTest Preferences.

Observer

An optional class instantiated from the command line to observe a test run. The class implements a specific observer interface.

Parameters

Values used to configure an agent. The agent parameters can be set at the time the agent is started or, if the agent GUI is used, from the Parameters tab after the agent GUI has started.

Passive Agent

Agents that must wait for a request from the JavaTest harness before they can run tests.

The JavaTest harness initiates connections to passive agents as needed. Passive agents are simpler, but less flexible than active agents because you must specify the test machine as part of the test configuration, not at the time you run the tests. Passive agents do not allow you to run tests in parallel.

Port Number

A number assigned to the JavaTest harness that is used to link specific incoming data to an appropriate service.

Prior Status

A filter used to restrict the set of tests in a test run based on the last test result information stored in the test result files (.jtr).

Use the configuration editor to enable the Prior Status filter for a test run.

Progress Monitor

A dialog box that displays detailed information about the current configuration of a test run. Information displayed in the Progress Monitor is not altered by view filter settings.

To display the Progress Monitor, do one of the following:

Report Directory

The directory in which the JavaTest harness writes test reports.

The location of the report directory is set in the GUI or from the command line by the user when generating test reports.

SDI

See Single Document Interface.

Serial Agent

Use serial mode (serial agent) when you want the agent to use an RS-232 serial connection. Serial agents wait for the JavaTest harness to initiate the connection. Infrared, parallel, USB, and firewire connections can also be added through the JavaTest API by modeling the existing serial system.

Single Document Interface

A window style in which the JavaTest harness opens a console window and individual tool windows as separate top-level windows on an unbounded desktop. Use the JavaTest Preferences dialog box to select the SDI window style. See JavaTest Preferences.

Standard Values

Values in a configuration that govern how the JavaTest harness runs the tests of a test suite. Standard values are a part of the configuration interview but can change from test run to test run. You can use the Standard Values view of the Configuration Editor to edit the standard values in the current configuration. See Current Configuration.

System Properties

Contains environment variables from your system that are required to run the tests of a test suite.

Because the JavaTest harness cannot directly access environment variables, you must use command-line options to copy them into the JavaTest harness system properties.

Tabbed

A window style in which the JavaTest desktop is a single top-level window that displays all JavaTest windows as tabbed panes.

Use the JavaTest Preferences dialog box to select the Tabbed window style. See JavaTest Preferences.

Test Description

Machine readable information that describes a test to the JavaTest harness so that it can correctly process and run the related test. The actual form and type of test description depends on the attributes of the test suite. When using the JavaTest harness, the test description is a set of test-suite-specific name/values pairs in either HTML tables or Javadoc-style tags

Each test in a test suite has a corresponding test description that is typically contained in an HTML file.

Test Environment

A collection of values that provide information about how to run tests on a particular platform.

When a test in a test suite is run, the JavaTest harness gives the script a test environment containing environment variables that are derived from configuration data collected by the configuration editor. See configuration.

Prior to the JavaTest 3.0 harness, the environment variables were read from an environment file. Use of environment files is deprecated, however, the JavaTest harness provides support for those test suites that use environment files. See environment file.

Test Manager

The JavaTest window used to configure, run, monitor, and manage tests from its panels, menus, and controls.

The Test Manager window is divided into two panes. It displays the folders and tests of a test suite in the tree pane on the left and provides information about the selected test or folder in the information panes on the right. A new Test Manager window is used for each test suite that is opened.

Test Result Files

Contains all of the information gathered by the JavaTest harness during a test run.

The test result files (.jtr) are stored in a cache in the work directory associated with the test suite.

You can view the test result files in a web browser configured to use the JavaTest ResultBrowser servlet.

Test Run Filters

Include or exclude tests in a test run:

Test run filters are set using the configuration editor or the parameter editor.

Test Script

A script used by the JavaTest harness, responsible for running the tests and returning the status (pass, fail, error) to the harness. The test script must understand how to interpret the test description information returned to it by the test finder. The test script is a plug-in provided by the test suite. The Test Manager Properties dialog box lists the plug-ins that are provided by the test suite.

Test Suite

A collection of tests, used in conjunction with the JavaTest harness to verify compliance of the licensee's implementation of the technology specifications.

A test suite must be associated with a work directory before the JavaTest harness can run its tests.

Test Tree

The hierarchical representation of the folders and tests in a test suite.

The test tree is displayed in the Test Manager window and uses colored status icons to indicate the test status of the folders and tests. Use view filters to specify the folders and tests whose test status are displayed in the test tree.

Tests to Run

Test folders and/or tests specified as a starting point in the test tree for running tests.

Initial files are used to limit the test run to a specific branch of the test tree or to a specific test. The JavaTest harness walks the test tree starting at the folder and/or test and runs all tests (unless otherwise filtered out) that it finds.

Use the configuration editor or the parameter editor to modify the initial files listed in the test run configuration.

View Filters

Include or exclude the results of tests in views of the test tree. The following filters are provided in the combo box at the bottom of the test tree:

View filter settings do not include or exclude tests from a test run. Use the test run filter settings to include or exclude tests from a test run.

Work Directory

A directory associated with a specific test suite and used by the JavaTest harness to store files containing information about the test suite and its tests.

Until a test suite is associated with a work directory, the JavaTest harness cannot run tests.