gunittest: example of a bash script to run test scripts as a cron job, document new CLI

git-svn-id: https://svn.osgeo.org/grass/grass/trunk@61498 15284696-431f-4ddb-bdfa-cd5b030d7da7

lib/python/docs/src/gunittest_running_tests.rst

@@ -0,0 +1,79 @@
+Running the tests of GRASS GIS
+==============================
+
+This is an advanced guide to running tests of GRASS GIS using GRASS
+testing framework (`gunittest`).
+
+Example Bash script to run be used as a cron job
+------------------------------------------------
+
+.. code-block:: bash
+
+    #!/bin/bash
+
+    set -e  # fail fast
+
+    REPORTS=".../testreports"
+    GRASSSRC=".../grass-src"
+    # here we suppose default compilation settings of GRASS and no make install
+    GRASSBIN="$GRASSSRC/bin.../grass71"
+    GRASSDIST="$GRASSSRC/dist..."
+    
+    # necessary hardcoded GRASS paths
+    GRASSDIST_PYTHON="$GRASSDIST/etc/python"
+    GRASS_MULTI_RUNNER="$GRASSSRC/lib/python/gunittest/multirunner.py"
+    GRASS_MULTI_REPORTER="$GRASSSRC/lib/python/gunittest/multireport.py"
+
+    DATE_FLAGS="--utc +%Y-%m-%d-%H-%M"
+    NOW=$(date $DATE_FLAGS)
+
+    # contains last executed command stdout and stderr
+    # here were rely on reports being absolute
+    OUTPUT_LOGFILE="$REPORTS/output-$NOW.txt"
+
+    # these are relative to REPORTS
+    CURRENT_REPORT_BASENAME="reports_for_date-"
+    FINAL_REPORT_DIR="summary_report"
+    CURRENT_REPORTS_DIR="$CURRENT_REPORT_BASENAME$NOW"
+    LOGFILE="$REPORTS/runs.log"
+
+    GRASSDATA="/grassdata/tests-grassdata"
+
+    echo "Nightly GRASS GIS test started: $NOW" >> $LOGFILE
+
+    # compile current source code from scratch
+    cd $GRASSSRC
+    make distclean -j4
+    svn up
+    ./configure ...  # or a script containing all the flags
+    make -j4
+
+    # run tests for the current source code
+    cd $REPORTS
+    mkdir $CURRENT_REPORTS_DIR
+    cd $CURRENT_REPORTS_DIR
+    python $GRASS_MULTI_RUNNER \
+        --grassbin $GRASSBIN \
+        --grasssrc $GRASSSRC \
+        --grassdata $GRASSDATA \
+        --location nc_spm_08_grass7 --location-type nc \
+        --location other_location --location-type other_type
+
+    # create overall report of all so far executed tests
+    # the script depends on GRASS but just Python part is enough
+    export PYTHONPATH="$GRASSDIST_PYTHON:$PYTHONPATH"
+    python $GRASS_MULTI_REPORTER --output $FINAL_REPORT_DIR \
+        $CURRENT_REPORT_BASENAME*/*
+
+    # although we cannot be sure the tests were executed was successfully
+    # so publish or archive results
+    rsync -rtvu --delete $REPORTS/ "/var/www/html/grassgistestreports"
+
+    echo "Nightly ($NOW) GRASS GIS test finished: $(date $DATE_FLAGS)" >> $LOGFILE
+
+A script similar to this one can be used as a cron job, on most Linux systems
+using ``crontab -e`` and adding the following line::
+
+    0 4 * * 1 /home/vpetras/grasstests/test_grass_gis.sh
+
+Which will perform the tests every Monday at 4 in the morning.

lib/python/docs/src/gunittest_testing.rst

@@ -46,16 +46,17 @@ Python ``unittest`` package, so the ways to build tests are very similar.
 Each test file should be able to run by itself and accept certain set of command
 line parameters. This is ensured using `gunittest.test()`.
 
-To run (invoke) all tests in the source tree run::
+To run (invoke) all tests in the source tree, you have to be in the source code
+directory, inside GRASS session and use command similar to this one::
 
-    python -m grass.gunittest.main [gisdbase] location test_data_category
+    python -m grass.gunittest.main --location nc_spm_grass7 --location-type nc
 
 All test files in all ``testsuite`` directories will be executed and
 a report will be created in a newly created ``testreport`` directory.
 Open the file ``testreport/index.html`` to browse though the results.
-You need to be in GRASS session to run the tests.
+Note that you need to be in GRASS session to run the tests in this way.
 
-The test_data_category parameter serves to filter tests accoring to data
+The ``--location-type`` parameter serves to filter tests according to data
 they can run successfully with. It is ignored for tests which does not have
 this specified.
 
@@ -353,6 +354,7 @@ Further reading
    :maxdepth: 2
 
    gunittest
+   gunittest_running_tests
 
 
 .. _unittest: https://docs.python.org/2/library/unittest.html