WebCGM 2.1 Conformance Test Suite
Guidelines for Reviewers
Revision: 1.01
Date: November 7, 2008
Author: Robert Orosz
Document Versions
Rev
|
Date
|
Description of Change
|
1.00
|
Nov. 6, 2008
|
Initial release.
|
1.01
|
Nov. 7, 2008
|
Changed from XHTML to HTML. This works around an issue in Internet
Explorer where the XML source is displayed if the file is viewed
directly from the FTP site.
|
1.02
|
Feb. 12, 2008
|
Clarified the required fields that must be present in the standard
comment header.
|
Preface
This document provides instructions to reviewers of new tests submitted to the WebCGM 2.1 test suite. A careful, thoughtful review is essential to the quality of the test suite and therefore to the WebCGM 2.1 specification itself. The purpose of this document is to describe the necessary items to consider when doing a review.
Test Review Guidelines
Overall Topic(Chapter) Content
Consider a chapter and its associated test cases, and consider:
- Is the coverage complete? Are there features in the specification which are not tested?
Individual Test Content
Look at an individual test case and assess the following:
- Is the "look and feel" or "style" consistent with previously established, i.e. existing tests? New tests should not invent new styles without good cause.
- Names of files, etc. must use camel case.
- Is the test appropriately named?
- Does the test include the standard comment header (available in header.txt on the FTP site)? The required fields are Test, Creator, Description, Source, and Revision History. The Version field is reserved for future use and should be left as it is in the template. The Source field should include the specific subsection of WebCGM or CGM99 that is being tested. For example, a test that tested methods of the WebCGMNode interface would use "Source: WebCGM 2.1 5.7.4" in its comment header.
- Is the test traceable; i.e. does it identify what specific part of the WebCGM 2.1 specification is being tested?
- Is the CGM valid?
- Is the (X)HTML valid?
- Is the ECMAScript valid?
- The reference image must be in PNG format with 8-bit color. Does it correctly depict the graphical result of the test?
- The tests or their subtests should be atomic and well focussed.
- "Complexity density" of the test. An individual test should not be too complex (this is somewhat subjective).
- Minimal prerequisites. The test should not rely other parts of WebCGM beyond what is being tested.
- The test should be self-documenting, i.e. the displayed result of the test should contain sufficient information to judge pass or fail.
- In addition to the standard header, the ECMAScript, companion XML files, etc. should be well commented.