From 78b8e13e6fed5ef693a342bd673bd8392f30b800 Mon Sep 17 00:00:00 2001 From: Steve Barnes Date: May 31 2011 13:05:58 +0000 Subject: A few last minute additions to WritingTests --- diff --git a/WritingTests b/WritingTests index bcac256..5128084 100644 --- a/WritingTests +++ b/WritingTests @@ -1,16 +1,20 @@ -This file provides guidance on things to bear in mind when writing test scripts for the QA process: +This file provides some basic guidance on things to bear in mind when writing test scripts for the QA process: -(a) scripts should exit with either zero to signal success, or non-zero to signal a failure. A failure exit code causes the entire test script execution process to stop (which is fine - we need to see what failed) +(a) scripts should exit with either zero to signal success, or a non-zero value to signal failure. A failure exit code causes the entire test script execution process to stop (which is fine - we need to see what failed) -(b) several helper functions are available in test/0_lib/* that should be used in preference to directly calling the commands they implement. Some of the reasons for doing this are consistency in debug output and avoiding timing related test failures. Please review the functions so you're familiar with their existence/operation. +(b) several helper functions are available in test/0_lib/*. Please use these in preference to directly calling any of the commands they implement. You don't have to source the file in your test scripts, just call/use the functions as required. The helper functions are there (for example) to promote consistency in debugging output and help avoid timing related test failures eg, daemons not being given sufficient time to start before testing for their existence. Please review the contents of 0_lib/ so you're familiar with what's on offer. -(c) all test scripts should first make a call to t_Log, passing in $0 and a description of what the test is doing, something like: +(c) if you're using Bash, the first thing you should ideally do is make a call to t_Log, passing in $0 and a description of what the test is doing, something like: t_Log "Running $0 - Postfix SMTP test." -(d) test scripts are processed in alphabetical order, so it's best to install any required packages in a 0-install-blah.sh script. Anything starting with a _ is ignored, and so are files named `readme` (case insensitive) +(d) test scripts are processed in alphabetical order, so it's sensible to install any required packages in a 0-install-blah.sh script. -(e) all test scripts must be chmod +x in order to be executed. Equally, removing execute permissions from a script will prevent it from being run (or prefixing it with an _, both approaches work) +(e) anything starting with a _ is ignored, and so are files named `readme` (case insensitive). If you need a file to store config values or any kind of metadata that's used in your test script, it probably makes sense to put it in a file starting with _ and then sourcing/including it in your test script. +(f) all test scripts should be chmod +x in order to be executed. Equally, removing execute permissions from a script will prevent it from being run (or prefixing it with an _, both approaches work) +(g) please include a suitable #Author comment line in your test scripts, so we know who to contact in the event of questions/changes/issues. + +(h) try and keep stdout/debugging messages generated by your test scripts to a minimum.