Merge pull request #5200 from AttilaVamos/HPCC-10522-improvement

HPCC-10522 Regression Suite --timeout and --pq (parallel query) parameters don't documented well

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>

testing/regress/README.rst

@@ -7,7 +7,7 @@ Parameters of Regression Suite:
 
 Command:
  
- ./regress --help
+    ./regress <-h|--help>
 
 Result:
 
@@ -28,15 +28,77 @@ Result:
 |       optional arguments:
 |            -h, --help            show this help message and exit
 |            --version, -v         show program's version number and exit
-|            --config [CONFIG]     Config file to use.
+|            --config [CONFIG]     config file to use. Default: regress.json.
 |            --loglevel [{info,debug}]
-|                                  Set the log level.
+|                                  set the log level. Use debug for more detailed logfile.
 |            --suiteDir [SUITEDIR], -s [SUITEDIR]
-|                               suiteDir to use. Default value is the current directory and it can handle relative path.
+|                                  suiteDir to use. Default value is the current directory and it can handle relative path.
 |            --timeout [TIMEOUT], -t [TIMEOUT]
-|                               timeout for query execution.
+|                                  timeout for query execution in sec. Use -1 to disable timeout. Default value defined in regress.json config file (see: 7.)
+
+
+Parameters of Regression Suite list sub-command:
+------------------------------------------------
+
+Command:
+
+    ./regress list <-h|--help>
+
+Result:
+
+|
+|       usage: regress list [-h]
+|
+|       positional arguments:
+|         clusters    Print clusters from config (regress.json by default).
+|
+|       optional arguments:
+|         -h, --help  show this help message and exit
+|
+
+Parameters of Regression Suite run sub-command:
+-----------------------------------------------
+
+Command:
+
+    ./regress run <-h|--help>
+
+Result:
+
+|
+|       usage: regress run [-h] [--pq threadNumber] [cluster]
+|
+|       positional arguments:
+|         cluster            Run the cluster suite. Default value is setup.
+|
+|       optional arguments:
+|         -h, --help         show this help message and exit
+|         --pq threadNumber  Parallel query execution with threadNumber threads. (If threadNumber is '-1' on a single node system then threadNumer = numberOfLocalCore * 2)
+|
+
+
+Parameters of Regression Suite query sub-command:
+-------------------------------------------------
+
+Command:
+
+    ./regress query <-h|--help>
+
+Result:
+
+|
+|       usage: regress query [-h] [--publish] [ECL query] [target cluster | all]
+|
+|       positional arguments:
+|         ECL query             Name of a single ECL query (mandatory).
+|         target cluster | all  Cluster for single query run. If cluster = 'all' then run single query on all clusters. Default value is thor.
+|
+|       optional arguments:
+|         -h, --help            show this help message and exit
+|         --publish             Publish compiled query instead of run.
+
+
 
-	
 Steps to run Regression Suite
 =============================
 
@@ -95,7 +157,7 @@ The result:
 |            `-------------------------------------------------`
 
 
-	    
+
 
 4. To run Regression Suite on a selected cluster (e.g. Thor):
 -------------------------------------------------------------
@@ -126,7 +188,7 @@ The first and last couple of lines look like this:
 |        [Pass]   2. Pass W20131119-173527 (1 sec)
 |        [Pass]   2. URL http://127.0.0.1:8010/WsWorkunits/WUInfo?Wuid=W20131119-173527
 |        [Action]   3. Test: aggsq1.ecl
-
+|
 |        .
 |        .
 |        .
@@ -136,7 +198,7 @@ The first and last couple of lines look like this:
 |        [Action] 257. Test: xmlparse.ecl
 |        [Pass] Pass W20131119-182537 (1 sec)
 |        [Pass] URL http://127.0.0.1:8010/WsWorkunits/WUInfo?Wuid=W20131119-182537
-|        
+|
 |         Results
 |         `-------------------------------------------------`
 |         Passing: 257
@@ -260,20 +322,33 @@ The format of result is same as above:
 6. Tags used in testcases:
 --------------------------
 
-To exclude testcase from cluster or clusters, the tag is:
+    To exclude testcase from cluster or clusters, the tag is:
 //no<cluster_name>
 
-To skip (similar to exclusion)
+    To skip (similar to exclusion)
 //skip type==<cluster> <reason>
 
-To build and publish testcase (e.g.:for libraries)
+    To build and publish testcase (e.g.:for libraries)
 //publish
 
-To set individual timeout for test case
-//timeout <timeout value in sec>
+    To set individual timeout for test case
+//timeout <timeout_value_in_sec>
+
+7. Key file generation:
+------------------------------
 
-7. Configuration setting in regress.json file:
-----------------------------------------------
+The regression suite stores every test case output into ~/HPCCSystems-regression/result directory. This is the latest version of result. (The previous version can be found in ~/HPCCSystems-regression/archives directory.) When a test case execution finished Regression Suite compares this output file with the relevant key file to verify the result.
+
+So if you have a new test case and it works well on all clusters (or some of them and excluded from all others by //no<cluster> tag inside it See: 6. ) then you can get key file in 2 steps:
+
+1. Run test case with ./regress [suitedir] query <testcase.ecl> <cluster> .
+
+2. Copy the output (testcase.xml) file from ~/HPCCSystems-regression/result to the relevant key file directory.
+
+(To check everything is fine, repeat the step 1 and the query should now pass. )
+
+8. Configuration setting in regress.json file:
+-------------------------------------------------------------
 
         "ip": "127.0.0.1",                              - ECl server address
         "username": "regress",                          - Regression Suite dedicated username and pasword

testing/regress/regress

@@ -39,32 +39,32 @@ if __name__ == "__main__":
     parser = argparse.ArgumentParser(prog=prog, description=description)
     parser.add_argument('--version', '-v', action='version',
                         version='%(prog)s 0.0.4')
-    parser.add_argument('--config', help="Config file to use.",
+    parser.add_argument('--config', help="config file to use. Default: regress.json",
                         nargs='?', default="regress.json")
-    parser.add_argument('--loglevel', help="Set the log level.",
+    parser.add_argument('--loglevel', help="set the log level. Use debug for more detailed logfile.",
                         nargs='?', default="info",
                         choices=['info', 'debug'])
-    parser.add_argument('--suiteDir', '-s', help="suiteDir to use.",
+    parser.add_argument('--suiteDir', '-s', help="suiteDir to use. Default value is the current directory and it can handle relative path.",
                         nargs='?', default=".")
-    parser.add_argument('--timeout', '-t', help="timeout for query execution.",
+    parser.add_argument('--timeout', '-t', help="timeout for query execution in sec. Use -1 to disable timeout. Default value defined in regress.json config file.",
                         nargs='?', default="-1")
 
     subparsers = parser.add_subparsers(help='sub-command help')
     parser_list = subparsers.add_parser('list', help='list help')
-    parser_list.add_argument('clusters', help="Print clusters from config.",
+    parser_list.add_argument('clusters', help="Print clusters from config (regress.json by default).",
                              action='store_true')
 
     parser_run = subparsers.add_parser('run', help='run help')
-    parser_run.add_argument('cluster', help="Run the cluster suite.",
+    parser_run.add_argument('cluster', help="Run the cluster suite. Default value is setup.",
                             nargs='?', type=str,  default='setup')
-    parser_run.add_argument('--pq', help="Parallel query execution with threadNumber threads. ('-1' can be use to calculate usable thread count on a single node system)",
+    parser_run.add_argument('--pq', help="Parallel query execution with threadNumber threads. (If threadNumber is '-1' on a single node system then threadNumer = numberOfLocalCore * 2 )",
                             type=checkPqParam,  default = 0,   metavar="threadNumber")
 
     parser_query = subparsers.add_parser('query', help='query help')
     parser_query.add_argument('query', help="Name of a single ECL query (mandatory).",
                               nargs='?', metavar="ECL query")
-    parser_query.add_argument('cluster', help="Cluster for single query run. If cluster = 'all' then run single query on all clusters.",
-                            nargs='?', default='thor', metavar="target cluster | all")
+    parser_query.add_argument('cluster', help="Cluster for single query run. If cluster = 'all' then run single query on all clusters. Default value is thor.",
+                            nargs='?', default='thor', metavar="target_cluster | all")
     parser_query.add_argument('--publish', help="Publish compiled query instead of run.",
                             action='store_true')