The TestKit is a collection of scripts that leverages the ShUnit unit - testing environment. TestKit provides a pattern for creating test - suites using a simple configuration file approach. Full reporting on - test runs is provided in a convenient tabular format.
+ testing environment. The TestKit provides a pattern for creating + test suites using a simple configuration file approach. Full + reporting on test runs is provided in a convenient tabular format.Generally, writing a test script using the TestKit is a matter of minutes. A blank test is provided as a template, and that can be - expanded with whatever test steps are needed.
+ expanded with whatever test steps are needed. (See + examples/blank_test.sh)TestKit (and ShUnit) are implemented in the GNU Bash script language, but
a TestKit test script can invoke external applications, written in
whatever programming language or scripting tool is desired, using the
- standard POSIX interfaces.
-
The testkit is provided under the Apache License, version 2.0 (the + "License"). The license is available at: http://www.apache.org/licenses/LICENSE-2.0
Follow these steps to download and install the Feisty Meow TestKit:
+
Follow these steps to download and install a new "vanilla" version of the
+ TestKit:
sudo
+margin-left:0in;background:#DDD9C3">sudo
mkdir /opt/feistymeow.org
sudo chown -R $USER /opt/feistymeow.org
cd /opt/feistymeow.org
- git clone git://feistymeow.org/feisty_meow
The above steps may have been used to kick-start the local version of the + TestKit. It is perfectly valid to download the testkit and then copy + it into one's own source code for use; this is enabled under the Apache + License.
+It is also possible to check out the TestKit within one's own code base + (by adding the Feisty Meow® Codebase that was retrieved above). Then + one can retrieve an updated Feisty Meow® TestKit by running "git pull" on + the "feisty_meow" folder. This will get the latest version of + TestKit without disturbing whatever project's revision control repository + contains the TestKit for testing.
Linux is the easiest environment for running the TestKit, given that the
tests were built using the bash shell within a Linux environment. If
@@ -61,83 +80,81 @@ margin-left:0in;background:#DDD9C3">sudo
bc The apt-cyg program brings the convenience of the Debian and Ubuntu
installer application (apt-get) to Cygwin. This program does require
a couple of additional setup steps. This material is drawn from the
apt-cyg home page: https://github.com/transcode-open/apt-cyg 1.
- Install the basic Cygwin packages with setup.exe (rather than the long
- list above), but add these two packages which are not selected by default: 1. Install the basic Cygwin packages with setup.exe (rather than
+ the long list above), but add these two packages which are not selected by
+ default: 2.
- Download and install the apt-cyg program from within a Cygwin bash prompt: 2. Download and install the apt-cyg program from within a
+ Cygwin bash prompt: lynx -source
- rawgit.com/transcode-open/apt-cyg/master/apt-cyg > apt-cyg 3.
- Install the packages required for the TestKit: 3. Install the packages required for
+ the TestKit: apt-cyg install bc
- crypt cygutils emacs email expect gcc-g++ git gitk gvim \ 4.
- The installation will run for a while but then should conclude with all
- required packages installed. 4. The installation will run for a
+ while but then should conclude with all required packages installed. Tunning tests in TestKit uses a configuration file called
+ Running tests in TestKit uses a configuration file called
âtestkit.configâ to define the environment and, optionally, which test
scripts to run. This file is the main switchboard that defines where
- the tests will find users, home directories, queues, containers, and so
- forth. The configuration file can be specified via the environment
- variable âTESTKIT_CFG_FILEâ. This variable can be set to any
- location, enabling the configuration file to reside in a directory other
- than the toolkit folder. If the variable is not defined, then the
- testing config file defaults to â$TESTKIT_ROOT/testkit.configâ.
- crypt
- cygutils
- emacs
- email
- expect
- gcc-g++
- git
- gitk
- gvim
- inetutils
- less
- make
- mutt
- ncftp
- openssh
- perl
- procps
- python
- sharutils
- shutdown
- subversion
- time
- unzip
- util-linux
- vim
- wget
- xinit
- xterm
- zip
crypt
+ cygutils
+ emacs
+ email
+ expect
+ gcc-g++
+ git
+ gitk
+ gvim
+ inetutils
+ less
+ make
+ mutt
+ ncftp
+ openssh
+ perl
+ procps
+ python
+ sharutils
+ shutdown
+ subversion
+ time
+ unzip
+ util-linux
+ vim
+ wget
+ xinit
+ xterm
+ zip
Apt-cyg Installation Process
-
- install apt-cyg /bin
+ install apt-cyg /bin
- inetutils less make mutt ncftp openssh perl procps python
- sharutils \
- shutdown time unzip util-linux vim xinit xterm zip
+ inetutils less make mutt ncftp openssh perl procps python
+ sharutils \
+ shutdown time unzip util-linux vim xinit xterm zip
Setting up a Test Suite
-
The configuration file can be specified via the environment variable + âTESTKIT_CFG_FILEâ. This variable can be set to any location, + enabling the configuration file to reside in a directory other than the + toolkit folder. If the variable is not defined, then the testing + config file defaults to â$TESTKIT_ROOT/testkit.configâ.
The TESTKIT_ROOT variable is frequently referred to in command examples. It is set up automatically by the prepare_tools script (see below). @@ -148,8 +165,9 @@ margin-left:0in;margin-bottom:.0001pt;background:#DDD9C3">apt-cyg install bcbash {TESTKIT_FOLDER}/test_driver.sh -
Where the {TESTKIT_FOLDER} should be replaced with whatever path the TestKit is stored in.
Alternatively, if the TESTKIT_ROOT folder is already established, the @@ -157,7 +175,8 @@ margin-left:0in;background:#DDD9C3"> bash {TESTKIT_FOLDER}/test_driv
bash "$TESTKIT_ROOT/test_driver.sh"
+margin-left:0in;background:#DDD9C3"> bash + "$TESTKIT_ROOT/test_driver.sh"The above shows the "summary" view, which does not allow the individual tests to output to the console. If the "summary" flag @@ -220,14 +240,16 @@ background:#DDD9C3;margin-left:.1in;margin-right:.1in">$ cd
01: FAIL â
- AckPfft_Tests/Gorp_Tests/deslagToaster.sh
- 02: OKAY â AckPfft_Tests/Gorp_Tests/spumeMerchantry.sh
- 03: OKAY â AckPfft_Tests/Gorp_Tests/octopusLauncher.sh
- â¦
- 22: OKAY -- Snargle_Tests/scramTests/scramForPetunias.sh
FAILURE: 1 Tests Failed out of 22 Tests.
+margin-left:0in;background:#DDD9C3">FAILURE: + 1 Tests Failed out of 22 Tests.A failed test will also return a non-zero value from the test execution, enabling the run of a test suite to be tested for success when launched @@ -239,22 +261,39 @@ margin-left:0in;background:#DDD9C3">FAILURE: 1 Tests Failed out of 22 Tests.
cd {TESTKIT_FOLDER} # replace
- with actual location of TestKit.
- source prepare_tools.sh prepare_tools.sh
- source $TESTKIT_ROOT/library/process_configuration.sh
- define_and_export_variables
After loading the TestKit environment, one can execute a specific test and see its results, for example:
cd examples
- bash blank_test.sh
The test will run and output its results to the console (that is, output is sent to standard out and standard error, to be more precise).
+The ShUnit test environment provides several functions that can be used + to evaluate whether a test was successful or whether a result was the + expected value.
+The blank test (in examples/blank_test.sh) shows off every method that + exists in our version of ShUnit and describes how the functions can be + used for testing. Please refer to that for a good set of + examples. This test is also semi-canonical, in that it implements + every phase of testing, including setup and tear down methods.
+For more details on ShUnit in general or to get a later version, this is + the official website: https://github.com/kward/shunit2
+Note however that we have made some customizations in reporting in the + version stored with the testkit, so some features may be missed if a newer + version is placed in the testkit's "shunit" folder.