From: Chris Koeritz Date: Fri, 14 Aug 2020 13:43:21 +0000 (-0400) Subject: enabled branding of the testkit docs X-Git-Tag: 2.140.134~13^2~7 X-Git-Url: https://feistymeow.org/gitweb/?a=commitdiff_plain;h=37bec1594a45b2f0ecc35f9752ad1bebab4a8a98;p=feisty_meow.git enabled branding of the testkit docs --- diff --git a/testkit/doc/makefile b/testkit/doc/makefile new file mode 100644 index 00000000..7bc9877d --- /dev/null +++ b/testkit/doc/makefile @@ -0,0 +1,5 @@ + +all: testkit_reference.html + +testkit_reference.html: testkit_reference-source.html + @bash rebrand.sh diff --git a/testkit/doc/rebrand.sh b/testkit/doc/rebrand.sh new file mode 100644 index 00000000..093a1f1f --- /dev/null +++ b/testkit/doc/rebrand.sh @@ -0,0 +1,17 @@ +#!/bin/bash + +# makes a new copy of the testkit reference with branding info installed. + +echo "Rebranding the TestKit reference for: $TESTKIT_BRANDING" + +# pull in our testkit config. +source ../prepare_tools.sh ../prepare_tools.sh +source $TESTKIT_ROOT/library/process_configuration.sh +define_and_export_variables + +# just a search and replace. the source doc had better still have "$BRANDING" tags in it. +sed -e "s/\$BRANDING/$TESTKIT_BRANDING/g" \ + < testkit_reference-source.html \ + > testkit_reference.html + + diff --git a/testkit/doc/testkit_reference-source.html b/testkit/doc/testkit_reference-source.html new file mode 100644 index 00000000..d5570aa5 --- /dev/null +++ b/testkit/doc/testkit_reference-source.html @@ -0,0 +1,271 @@ + + + + TestKit Reference Manual + + +

$BRANDING TestKit Reference Manual

Author: Chris Koeritz

+ Version 1.0 â Updated August 14 2020

The $BRANDING TestKit

The TestKit is a collection of scripts that leverages the ShUnit unit + testing environment. TestKit provides a pattern for creating test + suites using a simple configuration file approach. Full reporting on + test runs is provided in a convenient tabular format.

Generally, writing a test script using the TestKit is a matter of + minutes. A blank test is provided as a template, and that can be + expanded with whatever test steps are needed.

TestKit (and ShUnit) are implemented in the GNU Bash script language, but + a TestKit test script can invoke external applications, written in + whatever programming language or scripting tool is desired, using the + standard POSIX interfaces.
+

Getting the TestKit

Follow these steps to download and install a new "vanilla" version of the + TestKit:
+

sudo + mkdir /opt/feistymeow.org
+ sudo chown -R $USER /opt/feistymeow.org
+ cd /opt/feistymeow.org
+ git clone git://feistymeow.org/feisty_meow

This is the code used to create the $BRANDING version of the TestKit.

It is possible to check out the TestKit within one's own code base, and + then it is possible to retrieve an updated Feisty MeowÂ® TestKit by running + "git pull" on the "testkit" folder. This will get the latest version + from the Feisty MeowÂ® Codebase without disturbing whatever project's + revision control repository contains the TestKit for testing.

Preparing the TestKit on Linux

Linux is the easiest environment for running the TestKit, given that the + tests were built using the bash shell within a Linux environment. If + some of the packages used in the tests are missing (such as expect and gnu + awk), these may need to be installed from the appropriate repository for + your Linux distribution. Most distributions include these packages + automatically however.

Preparing the TestKit on Mac OS X

The test suite runs well on modern Macs with Intel CPUs. Due to + some differences in the versions of a few applications on the Mac, some + GNU tools may need to be installed to run the TestKit. These are + available via the Brew installer tool.
+

Preparing the TestKit on MS-Windows

The Cygwin Unix emulation system is required to run the TestKit on + Windows. This package is available at: http://cygwin.com/install.html

The default packages selected by Cygwin are the starting point of the + install. In addition to those packages, the following packages are + also required (see list below). Rather than using the cygwin setup + program for this task, the next section describes how to install Cygwin + with the apt-cyg tool. Apt-cyg is the preferred method, since it + involves less interaction with the somewhat clunky Cygwin installer. + If necessary, it is possible to install all the packages without apt-cyg + just by using the Cygwin setup program. To find each of these + packages more easily, try switching the âViewâ button on the Cygwin setup + program to âFullâ to get an alphabetized list.

bc
+ crypt
+ cygutils
+ emacs
+ email
+ expect
+ gcc-g++
+ git
+ gitk
+ gvim
+ inetutils
+ less
+ make
+ mutt
+ ncftp
+ openssh
+ perl
+ procps
+ python
+ sharutils
+ shutdown
+ subversion
+ time
+ unzip
+ util-linux
+ vim
+ wget
+ xinit
+ xterm
+ zip

Apt-cyg Installation Process

The apt-cyg program brings the convenience of the Debian and Ubuntu + installer application (apt-get) to Cygwin. This program does require + a couple of additional setup steps. This material is drawn from the + apt-cyg home page: https://github.com/transcode-open/apt-cyg

1. + Install the basic Cygwin packages with setup.exe (rather than the long + list above), but add these two packages which are not selected by default:

subversion
wget

2. + Download and install the apt-cyg program from within a Cygwin bash prompt:

lynx -source + rawgit.com/transcode-open/apt-cyg/master/apt-cyg > apt-cyg
+ install apt-cyg /bin

3. + Install the packages required for the TestKit:

apt-cyg install bc + crypt cygutils emacs email expect gcc-g++ git gitk gvim \
+ inetutils less make mutt ncftp openssh perl procps python + sharutils \
+ shutdown time unzip util-linux vim xinit xterm zip

4. + The installation will run for a while but then should conclude with all + required packages installed.

Setting up a Test Suite

Tunning tests in TestKit uses a configuration file called + âtestkit.configâ to define the environment and, optionally, which test + scripts to run. This file is the main switchboard that defines where + the tests will find users, home directories, queues, containers, and so + forth. The configuration file can be specified via the environment + variable âTESTKIT_CFG_FILEâ. This variable can be set to any + location, enabling the configuration file to reside in a directory other + than the toolkit folder. If the variable is not defined, then the + testing config file defaults to â$TESTKIT_ROOT/testkit.configâ.

+ The TESTKIT_ROOT variable is frequently referred to in command + examples. It is set up automatically by the prepare_tools script (see + below). +

Running a Test Suite

Once the TestKit configuration file has been established, running a whole + test suite can be accomplished with this command:
+

bash {TESTKIT_FOLDER}/test_driver.sh +

Where the {TESTKIT_FOLDER} should be replaced with whatever path + the TestKit is stored in.

Alternatively, if the TESTKIT_ROOT folder is already established, the + tests can be run with:

bash "$TESTKIT_ROOT/test_driver.sh"

What to Expect From the Test Run

The test_driver.sh script will output a few informative lines of text + before printing a table of the tests that it intends to run.

After the test plan is shown, all of the tests listed will be executed in + the order they are listed in, and they will each produce output. + Each individual test (usually a separate bash script) produces a summary + at the end of its run with a count of tests and a report of the tests + success or failure.

At the end of all the tests in the suite, the table of tests is printed + again with the results for each test. For example, this is a test + run that had no errors in any test (that's good, since it is our super + simple example test):

$ cd + $FEISTY_MEOW_APEX/testkit
+ $ ./test_driver.sh summary
+ ===========================================================
+ Testkit environment loaded.
+ TESTKIT_ROOT=/opt/feistymeow.org/feisty_meow/testkit
+ TESTKIT_CFG_FILE=/opt/feistymeow.org/feisty_meow/testkit/testkit.config
+ TMP=/Users/fred/.tmp
+ TEST_TEMP=/Users/fred/.tmp/testkit_logs_fred
+ ===========================================================
+ +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+ TestKit running from: /opt/feistymeow.org/feisty_meow/testkit
+ TestKit config file: + /opt/feistymeow.org/feisty_meow/testkit/testkit.config
+ +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Full set of tests:
+ 1: /opt/feistymeow.org/feisty_meow/testkit/examples/blank_test.sh
+
+ ======================================================================
+ Wed Aug 12 14:11:00 EDT 2020: Now running test 1: + /opt/feistymeow.org/feisty_meow/testkit/examples/blank_test.sh
+ Test output file: + /Users/fred/.tmp/testkit_logs_fred/run_2020_08_12/test_log.vKf7J3
+ OK: successful test run for test + /opt/feistymeow.org/feisty_meow/testkit/examples/blank_test.sh
+
+
+ Results table for this test run:
+
+ 01: OKAY -- /opt/feistymeow.org/feisty_meow/testkit/examples/blank_test.sh
+
+ Total testing duration: 00:00 hh:mm (1 seconds total)
+ OK: All 1 Tests Ran Successfully.
+

The above shows the "summary" view, which does not allow + the individual tests to output to the console. If the "summary" flag + is not passed, then the output from all the test runs is also shown.

Even when the summary view is used, all output files can + be examined after the run. For example, in the above, the mentioned + output file "test_log.vKf7J3" can be checked to see exactly what happened + during the test.

A test with a failure in it will have âFAILâ next to the + test that failed, and the final output line will start with + âFAILUREâ. For example:

01: OKAY â + AckPfft_Tests/Gorp_Tests/deslagToaster.sh
+ 02: FAIL â AckPfft_Tests/Gorp_Tests/spumeMerchantry.sh
+ 03: OKAY â AckPfft_Tests/Gorp_Tests/octopusLauncher.sh
+ â¦
+ 22: OKAY -- Snargle_Tests/scramTests/scramForPetunias.sh

FAILURE: 1 Tests Failed out of 22 Tests.

A failed test will also return a non-zero value from the test execution, + enabling the run of a test suite to be tested for success when launched + externally, such as from a continuous integration system.

Loading the TestKit Environment

If one wishes to run individual tests within the test suite, rather than + the entire suite, this is done by loading the TestKit variables into the + current shell environment, like so:

cd {TESTKIT_FOLDER} # replace + with actual location of TestKit.
+ source prepare_tools.sh prepare_tools.sh
+ source $TESTKIT_ROOT/library/process_configuration.sh
+ define_and_export_variables

+ # Show the important variables.
+ var $TESTKIT_ROOT $TESTKIT_CFG_FILE

After loading the TestKit environment, one can execute a specific test + and see its results, for example:

cd examples
+ bash blank_test.sh

The test will run and output its results to the console (that is, output + is sent to standard out and standard error, to be more precise).

+ + diff --git a/testkit/doc/testkit_reference.html b/testkit/doc/testkit_reference.html index d6fbbd10..3fedb723 100644 --- a/testkit/doc/testkit_reference.html +++ b/testkit/doc/testkit_reference.html @@ -5,7 +5,7 @@

Feisty MeowÂ® TestKit Reference Manual

Version 1.0 â Updated August 12 2020

Version 1.0 â Updated August 14 2020

The Feisty MeowÂ® TestKit

The TestKit is a collection of scripts that leverages the ShUnit unit testing environment. TestKit provides a pattern for creating test @@ -20,7 +20,8 @@ standard POSIX interfaces.

Getting the TestKit

Follow these steps to download and install the Feisty Meow TestKit:
+

Follow these steps to download and install a new "vanilla" version of the + TestKit:

@@ -31,6 +32,12 @@ margin-left:0in;background:#DDD9C3">sudo cd /opt/feistymeow.org
git clone git://feistymeow.org/feisty_meow

This is the code used to create the Feisty MeowÂ® version of the TestKit.

Preparing the TestKit on Linux

Linux is the easiest environment for running the TestKit, given that the tests were built using the bash shell within a Linux environment. If @@ -220,9 +227,9 @@ background:#DDD9C3;margin-left:.1in;margin-right:.1in">$ cd

01: FAIL â +margin-left:0in;background:#DDD9C3">01: OKAY â AckPfft_Tests/Gorp_Tests/deslagToaster.sh
- 02: OKAY â AckPfft_Tests/Gorp_Tests/spumeMerchantry.sh
+ 02: FAIL â AckPfft_Tests/Gorp_Tests/spumeMerchantry.sh
03: OKAY â AckPfft_Tests/Gorp_Tests/octopusLauncher.sh
â¦
22: OKAY -- Snargle_Tests/scramTests/scramForPetunias.sh

diff --git a/testkit/makefile b/testkit/makefile index 9ecda0fd..3fff396e 100644 --- a/testkit/makefile +++ b/testkit/makefile @@ -3,3 +3,7 @@ all: @bash test_driver.sh +doc: + cd doc + $(MAKE) + diff --git a/testkit/testkit.config b/testkit/testkit.config index 0d26181a..f0396bc6 100644 --- a/testkit/testkit.config +++ b/testkit/testkit.config @@ -18,6 +18,9 @@ BASE_USER=${USER} # Used for windows testing; provides the path to the binaries directory of cygwin. #CYGWIN_BIN_PATH=c:/cygwin/bin +# Used to insert branded bits into documentation and other places potentially. +TESTKIT_BRANDING="Feisty MeowÂ®" + ############## # define the tests to run. this is the most convenient place to put this.