From: Chris Koeritz Date: Fri, 14 Aug 2020 13:43:21 +0000 (-0400) Subject: enabled branding of the testkit docs X-Git-Tag: 2.140.134~13^2~7 X-Git-Url: https://feistymeow.org/gitweb/?a=commitdiff_plain;h=37bec1594a45b2f0ecc35f9752ad1bebab4a8a98;p=feisty_meow.git enabled branding of the testkit docs --- diff --git a/testkit/doc/makefile b/testkit/doc/makefile new file mode 100644 index 00000000..7bc9877d --- /dev/null +++ b/testkit/doc/makefile @@ -0,0 +1,5 @@ + +all: testkit_reference.html + +testkit_reference.html: testkit_reference-source.html + @bash rebrand.sh diff --git a/testkit/doc/rebrand.sh b/testkit/doc/rebrand.sh new file mode 100644 index 00000000..093a1f1f --- /dev/null +++ b/testkit/doc/rebrand.sh @@ -0,0 +1,17 @@ +#!/bin/bash + +# makes a new copy of the testkit reference with branding info installed. + +echo "Rebranding the TestKit reference for: $TESTKIT_BRANDING" + +# pull in our testkit config. +source ../prepare_tools.sh ../prepare_tools.sh +source $TESTKIT_ROOT/library/process_configuration.sh +define_and_export_variables + +# just a search and replace. the source doc had better still have "$BRANDING" tags in it. +sed -e "s/\$BRANDING/$TESTKIT_BRANDING/g" \ + < testkit_reference-source.html \ + > testkit_reference.html + + diff --git a/testkit/doc/testkit_reference-source.html b/testkit/doc/testkit_reference-source.html new file mode 100644 index 00000000..d5570aa5 --- /dev/null +++ b/testkit/doc/testkit_reference-source.html @@ -0,0 +1,271 @@ + + + + TestKit Reference Manual + + +

$BRANDING TestKit Reference Manual

+

Author: Chris Koeritz

+
+ Version 1.0 ― Updated August 14 2020
+

The $BRANDING TestKit

+

The TestKit is a collection of scripts that leverages the ShUnit unit + testing environment.  TestKit provides a pattern for creating test + suites using a simple configuration file approach.  Full reporting on + test runs is provided in a convenient tabular format.

+

Generally, writing a test script using the TestKit is a matter of + minutes.  A blank test is provided as a template, and that can be + expanded with whatever test steps are needed.

+

TestKit (and ShUnit) are implemented in the GNU Bash script language, but + a TestKit test script can invoke external applications, written in + whatever programming language or scripting tool is desired, using the + standard POSIX interfaces.
+

+

Getting the TestKit

+

Follow these steps to download and install a new "vanilla" version of the + TestKit:
+

+
+

sudo + mkdir /opt/feistymeow.org
+ sudo chown -R $USER /opt/feistymeow.org
+ cd /opt/feistymeow.org
+ git clone git://feistymeow.org/feisty_meow

+
+

This is the code used to create the $BRANDING version of the TestKit.

+

It is possible to check out the TestKit within one's own code base, and + then it is possible to retrieve an updated Feisty Meow® TestKit by running + "git pull" on the "testkit" folder.  This will get the latest version + from the Feisty Meow® Codebase without disturbing whatever project's + revision control repository contains the TestKit for testing.

+

Preparing the TestKit on Linux

+

Linux is the easiest environment for running the TestKit, given that the + tests were built using the bash shell within a Linux environment.  If + some of the packages used in the tests are missing (such as expect and gnu + awk), these may need to be installed from the appropriate repository for + your Linux distribution.  Most distributions include these packages + automatically however.

+

Preparing the TestKit on Mac OS X

+

The test suite runs well on modern Macs with Intel CPUs.  Due to + some differences in the versions of a few applications on the Mac, some + GNU tools may need to be installed to run the TestKit.  These are + available via the Brew installer tool.
+

+

+

Preparing the TestKit on MS-Windows

+

The Cygwin Unix emulation system is required to run the TestKit on + Windows.  This package is available at: http://cygwin.com/install.html

+

The default packages selected by Cygwin are the starting point of the + install.  In addition to those packages, the following packages are + also required (see list below).  Rather than using the cygwin setup + program for this task, the next section describes how to install Cygwin + with the apt-cyg tool.  Apt-cyg is the preferred method, since it + involves less interaction with the somewhat clunky Cygwin installer.  + If necessary, it is possible to install all the packages without apt-cyg + just by using the Cygwin setup program.  To find each of these + packages more easily, try switching the “View” button on the Cygwin setup + program to “Full” to get an alphabetized list.

+
+

bc
+ crypt
+ cygutils
+ emacs
+ email
+ expect
+ gcc-g++
+ git
+ gitk
+ gvim
+ inetutils
+ less
+ make
+ mutt
+ ncftp
+ openssh
+ perl
+ procps
+ python
+ sharutils
+ shutdown
+ subversion
+ time
+ unzip
+ util-linux
+ vim
+ wget
+ xinit
+ xterm
+ zip

+
+

 

+

Apt-cyg Installation Process

+

The apt-cyg program brings the convenience of the Debian and Ubuntu + installer application (apt-get) to Cygwin.  This program does require + a couple of additional setup steps.  This material is drawn from the + apt-cyg home page: https://github.com/transcode-open/apt-cyg

+

1.  + Install the basic Cygwin packages with setup.exe (rather than the long + list above), but add these two packages which are not selected by default:

+ +

2.   + Download and install the apt-cyg program from within a Cygwin bash prompt:

+
+

lynx -source + rawgit.com/transcode-open/apt-cyg/master/apt-cyg > apt-cyg
+ install apt-cyg /bin

+
+

3.       + Install the packages required for the TestKit:

+
+

apt-cyg install bc + crypt cygutils emacs email expect gcc-g++ git gitk gvim \
+   inetutils less make mutt ncftp openssh perl procps python + sharutils \
+   shutdown time unzip util-linux vim xinit xterm zip

+
+

4.       + The installation will run for a while but then should conclude with all + required packages installed.

+

Setting up a Test Suite

+

Tunning tests in TestKit uses a configuration file called + “testkit.config” to define the environment and, optionally, which test + scripts to run.  This file is the main switchboard that defines where + the tests will find users, home directories, queues, containers, and so + forth.  The configuration file can be specified via the environment + variable “TESTKIT_CFG_FILE”.  This variable can be set to any + location, enabling the configuration file to reside in a directory other + than the toolkit folder.  If the variable is not defined, then the + testing config file defaults to “$TESTKIT_ROOT/testkit.config”.

+ The TESTKIT_ROOT variable is frequently referred to in command + examples.  It is set up automatically by the prepare_tools script (see + below). +

Running a Test Suite

+

Once the TestKit configuration file has been established, running a whole + test suite can be accomplished with this command:
+

+
+

 bash {TESTKIT_FOLDER}/test_driver.sh +

+

Where the {TESTKIT_FOLDER} should be replaced with whatever path + the TestKit is stored in.

+

Alternatively, if the TESTKIT_ROOT folder is already established, the + tests can be run with:

+
+

 bash "$TESTKIT_ROOT/test_driver.sh"

+
+

+

What to Expect From the Test Run

+

The test_driver.sh script will output a few informative lines of text + before printing a table of the tests that it intends to run.

+

After the test plan is shown, all of the tests listed will be executed in + the order they are listed in, and they will each produce output.  + Each individual test (usually a separate bash script) produces a summary + at the end of its run with a count of tests and a report of the tests + success or failure.

+

At the end of all the tests in the suite, the table of tests is printed + again with the results for each test.  For example, this is a test + run that had no errors in any test (that's good, since it is our super + simple example test):

+
$ cd + $FEISTY_MEOW_APEX/testkit
+ $ ./test_driver.sh summary
+ ===========================================================
+ Testkit environment loaded.
+ TESTKIT_ROOT=/opt/feistymeow.org/feisty_meow/testkit
+ TESTKIT_CFG_FILE=/opt/feistymeow.org/feisty_meow/testkit/testkit.config
+ TMP=/Users/fred/.tmp
+ TEST_TEMP=/Users/fred/.tmp/testkit_logs_fred
+ ===========================================================
+ +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+ TestKit running from: /opt/feistymeow.org/feisty_meow/testkit
+ TestKit config file: + /opt/feistymeow.org/feisty_meow/testkit/testkit.config
+ +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Full set of tests:
+ 1: /opt/feistymeow.org/feisty_meow/testkit/examples/blank_test.sh
+
+ ======================================================================
+ Wed Aug 12 14:11:00 EDT 2020: Now running test 1: + /opt/feistymeow.org/feisty_meow/testkit/examples/blank_test.sh
+   Test output file: + /Users/fred/.tmp/testkit_logs_fred/run_2020_08_12/test_log.vKf7J3
+ OK: successful test run for test + /opt/feistymeow.org/feisty_meow/testkit/examples/blank_test.sh
+
+
+ Results table for this test run:
+
+ 01: OKAY -- /opt/feistymeow.org/feisty_meow/testkit/examples/blank_test.sh
+
+ Total testing duration: 00:00 hh:mm (1 seconds total)
+ OK: All 1 Tests Ran Successfully.
+
+

The above shows the "summary" view, which does not allow + the individual tests to output to the console.  If the "summary" flag + is not passed, then the output from all the test runs is also shown.

+

Even when the summary view is used, all output files can + be examined after the run.  For example, in the above, the mentioned + output file "test_log.vKf7J3" can be checked to see exactly what happened + during the test.

+

A test with a failure in it will have “FAIL” next to the + test that failed, and the final output line will start with + “FAILURE”.  For example:

+
+

01: OKAY – + AckPfft_Tests/Gorp_Tests/deslagToaster.sh
+ 02: FAIL – AckPfft_Tests/Gorp_Tests/spumeMerchantry.sh
+ 03: OKAY – AckPfft_Tests/Gorp_Tests/octopusLauncher.sh
+ …
+ 22: OKAY -- Snargle_Tests/scramTests/scramForPetunias.sh

+

FAILURE: 1 Tests Failed out of 22 Tests.

+
+

A failed test will also return a non-zero value from the test execution, + enabling the run of a test suite to be tested for success when launched + externally, such as from a continuous integration system.

+

Loading the TestKit Environment

+

If one wishes to run individual tests within the test suite, rather than + the entire suite, this is done by loading the TestKit variables into the + current shell environment, like so:

+
+

cd {TESTKIT_FOLDER}  # replace + with actual location of TestKit.
+ source prepare_tools.sh prepare_tools.sh
+ source $TESTKIT_ROOT/library/process_configuration.sh
+ define_and_export_variables

+ # Show the important variables.
+ var $TESTKIT_ROOT $TESTKIT_CFG_FILE
+

After loading the TestKit environment, one can execute a specific test + and see its results, for example:

+
+

cd examples
+ bash blank_test.sh

+
+

The test will run and output its results to the console (that is, output + is sent to standard out and standard error, to be more precise).

+


+

+

+ + diff --git a/testkit/doc/testkit_reference.html b/testkit/doc/testkit_reference.html index d6fbbd10..3fedb723 100644 --- a/testkit/doc/testkit_reference.html +++ b/testkit/doc/testkit_reference.html @@ -5,7 +5,7 @@

Feisty Meow® TestKit Reference Manual

-
Version 1.0 ― Updated August 12 2020
+
Version 1.0 ― Updated August 14 2020

The Feisty Meow® TestKit

The TestKit is a collection of scripts that leverages the ShUnit unit testing environment.  TestKit provides a pattern for creating test @@ -20,7 +20,8 @@ standard POSIX interfaces.

Getting the TestKit

-

Follow these steps to download and install the Feisty Meow TestKit:
+

Follow these steps to download and install a new "vanilla" version of the + TestKit:

@@ -31,6 +32,12 @@ margin-left:0in;background:#DDD9C3">sudo cd /opt/feistymeow.org
git clone git://feistymeow.org/feisty_meow

+

This is the code used to create the Feisty Meow® version of the TestKit.

+

It is possible to check out the TestKit within one's own code base, and + then it is possible to retrieve an updated Feisty Meow® TestKit by running + "git pull" on the "testkit" folder.  This will get the latest version + from the Feisty Meow® Codebase without disturbing whatever project's + revision control repository contains the TestKit for testing.

Preparing the TestKit on Linux

Linux is the easiest environment for running the TestKit, given that the tests were built using the bash shell within a Linux environment.  If @@ -220,9 +227,9 @@ background:#DDD9C3;margin-left:.1in;margin-right:.1in">$ cd

01: FAIL – +margin-left:0in;background:#DDD9C3">01: OKAY – AckPfft_Tests/Gorp_Tests/deslagToaster.sh
- 02: OKAY – AckPfft_Tests/Gorp_Tests/spumeMerchantry.sh
+ 02: FAIL – AckPfft_Tests/Gorp_Tests/spumeMerchantry.sh
03: OKAY – AckPfft_Tests/Gorp_Tests/octopusLauncher.sh
…
22: OKAY -- Snargle_Tests/scramTests/scramForPetunias.sh

diff --git a/testkit/makefile b/testkit/makefile index 9ecda0fd..3fff396e 100644 --- a/testkit/makefile +++ b/testkit/makefile @@ -3,3 +3,7 @@ all: @bash test_driver.sh +doc: + cd doc + $(MAKE) + diff --git a/testkit/testkit.config b/testkit/testkit.config index 0d26181a..f0396bc6 100644 --- a/testkit/testkit.config +++ b/testkit/testkit.config @@ -18,6 +18,9 @@ BASE_USER=${USER} # Used for windows testing; provides the path to the binaries directory of cygwin. #CYGWIN_BIN_PATH=c:/cygwin/bin +# Used to insert branded bits into documentation and other places potentially. +TESTKIT_BRANDING="Feisty Meow®" + ############## # define the tests to run. this is the most convenient place to put this.