Merge remote-tracking branch 'origin/candidate-6.4.x' into candidate-6.4.4

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>

docs/ECLStandardLibraryReference/SLR-Mods/RemoteDirectory.xml

@@ -77,9 +77,14 @@ END;</programlisting>
   contains the relative path to the file from the specified
   <emphasis>directory</emphasis>.</para>
 
+  <para>The mask argument is a string that can include wildcard characters.
+  Valid wildcard characters are '*' (to match zero or more characters) and '?'
+  (to match exactly one character). Non-wild characters are matched exactly
+  and are case-sensitive. </para>
+
   <para>Example:</para>
 
   <programlisting format="linespecific">OUTPUT(STD.File.RemoteDirectory('edata12','\in','*.d00'));
 OUTPUT(STD.File.RemoteDirectory('10.150.254.6',
-      '/c$/training',,TRUE));</programlisting>
+      '/var/lib/HPCCSystems/hpcc-data/thor/','acc*',TRUE));</programlisting>
 </sect1>

docs/ECLWatch/ECLWa_mods/ECLWatchSrc.xml

@@ -1671,7 +1671,16 @@
         <title>Upload files</title>
 
         <para>You can upload files to your landing zone from the Landing Zone
-        page.</para>
+        page. <figure>
+            <title>Landing Zone Upload</title>
+
+            <mediaobject>
+              <imageobject>
+                <imagedata fileref="../../images/ECLWA440a.jpg"
+                           vendor="eclwatchSS" />
+              </imageobject>
+            </mediaobject>
+          </figure></para>
 
         <orderedlist>
           <listitem>
@@ -1704,8 +1713,9 @@
           </listitem>
 
           <listitem>
-            <para>Verify the file and dropzone are correct in the <emphasis
-            role="bold">Info Dialog</emphasis> that displays. <figure>
+            <para>Verify the file, folder, and dropzone information are
+            correct in the <emphasis role="bold">File Uploader</emphasis>
+            dialog that displays. <figure>
                 <title>Info Dialog</title>
 
                 <mediaobject>
@@ -1721,8 +1731,8 @@
             <para>Press the <emphasis role="bold">Start</emphasis> button to
             begin the upload.</para>
 
-            <para>The file progress indicator shows as the file uploads. When
-            upload is complete, the window closes.</para>
+            <para>The Upload action button displays the progress as the file
+            uploads. </para>
           </listitem>
         </orderedlist>
 

docs/ECLWatch/TheECLWatchMan.xml

@@ -1,5 +1,5 @@
 <?xml version="1.0" encoding="UTF-8"?>
-<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
 "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">
 <book xml:base="../">
   <title>Using ECL Watch</title>

docs/HPCCDataHandling/DH-Mods/DH-Mod1.xml

@@ -85,9 +85,12 @@
         </listitem>
 
         <listitem>
-          <para>From ECL Watch page, click on the <emphasis
-          role="bold">Upload/download File </emphasis> link in the menu on the
-          left side.</para>
+          <para>From the ECL Watch home page, press the <emphasis
+          role="bold">Files</emphasis> icon link at the top, then press the
+          <emphasis role="bold">Landing Zones</emphasis> button from the
+          sub-menu, and then click on the <emphasis
+          role="bold">Upload</emphasis> action button as illustrated in the
+          following image.</para>
 
           <para><figure>
               <title>Upload/download</title>
@@ -98,19 +101,16 @@
                              vendor="eclwatchSS" />
                 </imageobject>
               </mediaobject>
-            </figure></para>
-
-          <para><phrase> </phrase>Once you click on the Upload/download file
-          link, it will take you to the dropzones and files page, where you
-          can choose to <emphasis role="bold">Browse</emphasis> your machine
-          for a file to upload:</para>
+            </figure>A system file dialog opens. Browse and select the file to
+          upload and then press the <emphasis role="bold">Open</emphasis>
+          button.</para>
 
           <para><figure>
               <title>Dropzones</title>
 
               <mediaobject>
                 <imageobject>
-                  <imagedata fileref="../../images/LZimg04.jpg"
+                  <imagedata fileref="../../images/ECLWA440b.jpg"
                              vendor="eclwatchSS" />
                 </imageobject>
               </mediaobject>
@@ -118,16 +118,14 @@
         </listitem>
 
         <listitem>
-          <para>Press the <emphasis role="bold">Browse</emphasis> button to
-          browse the files on your local machine, select the file to upload
-          and then click <emphasis role="bold">Open</emphasis> button.</para>
-
-          <para>The file you selected should appear in the <emphasis
-          role="bold">Select a file to upload</emphasis> field.</para>
+          <para>Once you choose the file to upload, it will take you to the
+          <emphasis role="bold">File Uploader</emphasis> dialog, where you can
+          choose the landing zone, machine, and/or folder for the file to
+          upload.</para>
         </listitem>
 
         <listitem>
-          <para>Press on <emphasis role="bold">Upload Now</emphasis> to
+          <para>Press the <emphasis role="bold">Start</emphasis> button to
           complete the file upload.</para>
         </listitem>
       </orderedlist>
@@ -359,7 +357,7 @@
                   <row>
                     <entry><emphasis role="bold">Queue</emphasis></entry>
 
-                    <entry>Select the queue. </entry>
+                    <entry>Select the queue.</entry>
                   </row>
 
                   <row>

docs/Installing_and_RunningTheHPCCPlatform/Installing_and_RunningTheHPCCPlatform.xml

@@ -1497,10 +1497,18 @@ OUTPUT(L);</programlisting></para>
           </listitem>
 
           <listitem>
-            <para>A dialog opens. <emphasis role="bold">Browse</emphasis> your
-            local machine select the file to upload and then press the
-            <emphasis role="bold">Open</emphasis> button. <figure>
-                <title>File Uploader</title>
+            <para>A dialog opens. <emphasis role="bold">Browse</emphasis> to
+            and select the file to upload and then press the <emphasis
+            role="bold">Open</emphasis> button. </para>
+
+            <para>The file you selected should appear in the <emphasis
+            role="bold">File Name</emphasis> field. The data file is named:
+            <emphasis role="bold">2of12.txt</emphasis>.</para>
+          </listitem>
+
+          <listitem>
+            <para><figure>
+                <title>Start File Upload</title>
 
                 <mediaobject>
                   <imageobject>
@@ -1510,14 +1518,8 @@ OUTPUT(L);</programlisting></para>
                 </mediaobject>
               </figure></para>
 
-            <para>The file you selected should appear in the <emphasis
-            role="bold">File Name</emphasis> field. The data file is named:
-            <emphasis role="bold">2of12.txt</emphasis>.</para>
-          </listitem>
-
-          <listitem>
             <para>Press the <emphasis role="bold">Start</emphasis> button to
-            complete the file upload.</para>
+            complete the file upload. </para>
           </listitem>
         </orderedlist>
       </sect2>

docs/InstantCloud/InstantCloud.xml

@@ -51,15 +51,18 @@
       <para><emphasis role="redbold"></emphasis></para>
     </legalnotice>
 
-    <xi:include href="common/Version.xml" xpointer="xpointer(//*[@id='FooterInfo'])"
+    <xi:include href="common/Version.xml"
+                xpointer="xpointer(//*[@id='FooterInfo'])"
                 xmlns:xi="http://www.w3.org/2001/XInclude" />
 
-    <xi:include href="common/Version.xml" xpointer="xpointer(//*[@id='DateVer'])"
+    <xi:include href="common/Version.xml"
+                xpointer="xpointer(//*[@id='DateVer'])"
                 xmlns:xi="http://www.w3.org/2001/XInclude" />
 
     <corpname>HPCC Systems</corpname>
 
-    <xi:include href="common/Version.xml" xpointer="xpointer(//*[@id='Copyright'])"
+    <xi:include href="common/Version.xml"
+                xpointer="xpointer(//*[@id='Copyright'])"
                 xmlns:xi="http://www.w3.org/2001/XInclude" />
 
     <mediaobject role="logo">
@@ -1112,18 +1115,9 @@ OUTPUT(L);</programlisting></para>
           </listitem>
 
           <listitem>
-            <para>A dialog opens. <emphasis role="bold">Browse</emphasis> your
-            local machine select the file to upload and then press the
-            <emphasis role="bold">Open</emphasis> button. <figure>
-                <title>File Uploader</title>
-
-                <mediaobject>
-                  <imageobject>
-                    <imagedata fileref="images/LZimg04a.jpg"
-                               vendor="eclwatchSS" />
-                  </imageobject>
-                </mediaobject>
-              </figure></para>
+            <para>A dialog opens. <emphasis role="bold">Browse</emphasis> to
+            and select the file to upload and then press the <emphasis
+            role="bold">Open</emphasis> button. </para>
 
             <para>The file you selected should appear in the <emphasis
             role="bold">File Name</emphasis> field. The data file is named:

docs/One-Click_HPCCinAWS/One-Click_RuningHPCCinAmazonWebServicesEC2.xml

@@ -1091,9 +1091,11 @@ OUTPUT(L);</programlisting></para>
           <listitem>
             <?dbfo keep-together="always"?>
 
-            <para>From ECL Watch page, click on the <emphasis
-            role="bluebold">Upload/download File </emphasis> link in the menu
-            on the left side.</para>
+            <para>From the ECL Watch page, click on the <emphasis
+            role="bold">Files</emphasis> link icon then the <emphasis
+            role="bold">Landing Zones</emphasis> button to get to the Landing
+            Zones tab. On the Landing Zones tab press the <emphasis
+            role="bold">Upload</emphasis> action button.</para>
 
             <para><figure>
                 <title>Upload/download</title>
@@ -1106,39 +1108,30 @@ OUTPUT(L);</programlisting></para>
                 </mediaobject>
               </figure></para>
 
-            <para>Once you click on the Upload/download file link, it will
-            take you to the <emphasis role="bold">Dropzones and
-            Files</emphasis> page, where you can choose to <emphasis
-            role="bold">Browse</emphasis> your machine for a file to
-            upload:</para>
-
-            <para><figure>
-                <title>Dropzones and Files</title>
-
-                <mediaobject>
-                  <imageobject>
-                    <imagedata fileref="images/LZimg04.jpg"
-                               vendor="eclwatchSS" />
-                  </imageobject>
-                </mediaobject>
-              </figure></para>
+            <para>Once you click on the Upload action button, it will open a
+            dialog where you can choose to <emphasis
+            role="bold">Browse</emphasis> for a file to upload:</para>
           </listitem>
 
           <listitem>
             <?dbfo keep-together="always"?>
 
-            <para>Press the <emphasis role="bold">Browse</emphasis> button to
-            browse the files on your local machine, select the file to upload
-            and then press the <emphasis role="bold">Open</emphasis>
-            button.</para>
-
-            <para>The file you selected should appear in the <emphasis
-            role="bold">Select a file to upload:</emphasis> field. The data
-            file is named: <emphasis role="bold">2of12.txt</emphasis>.</para>
+            <para>Browse the files on your local machine, select the file
+            <emphasis role="bold">2of12.txt</emphasis> to upload and then
+            press the <emphasis role="bold">Open</emphasis> button.</para>
           </listitem>
 
           <listitem>
-            <para>Press on <emphasis role="bold">Upload Now</emphasis> to
+            <para><figure>
+                <title>File Uploader</title>
+
+                <mediaobject>
+                  <imageobject>
+                    <imagedata fileref="images/LZimg04a.jpg"
+                               vendor="eclwatchSS" />
+                  </imageobject>
+                </mediaobject>
+              </figure>Press <emphasis role="bold">Start</emphasis> to
             complete the file upload.</para>
           </listitem>
         </orderedlist>

docs/RuningHPCCinAmazonWebServicesEC2/RuningHPCCinAmazonWebServicesEC2.xml

@@ -2569,18 +2569,16 @@ OUTPUT(L);</programlisting></para>
                 </mediaobject>
               </figure></para>
 
-            <para>Once you click on the Upload/download file link, it will
-            take you to the <emphasis role="bold">Dropzones and
-            Files</emphasis> page, where you can choose to <emphasis
-            role="bold">Browse</emphasis> your machine for a file to
+            <para>Once you click on the Upload action button, you can
+            <emphasis role="bold">Browse</emphasis> for a file to
             upload:</para>
 
             <para><figure>
-                <title>Dropzones and Files</title>
+                <title>File Uploader</title>
 
                 <mediaobject>
                   <imageobject>
-                    <imagedata fileref="images/LZimg04.jpg"
+                    <imagedata fileref="images/LZimg04a.jpg"
                                vendor="eclwatchSS" />
                   </imageobject>
                 </mediaobject>
@@ -2590,19 +2588,13 @@ OUTPUT(L);</programlisting></para>
           <listitem>
             <?dbfo keep-together="always"?>
 
-            <para>Press the <emphasis role="bold">Browse</emphasis> button to
-            browse the files on your local machine, select the file to upload
-            and then press the <emphasis role="bold">Open</emphasis>
-            button.</para>
-
-            <para>The file you selected should appear in the <emphasis
-            role="bold">Select a file to upload:</emphasis> field. The data
-            file is named: <emphasis role="bold">2of12.txt</emphasis>.</para>
-          </listitem>
+            <para>Select the file to upload. The file you selected should
+            appear in the <emphasis role="bold">File Name</emphasis> field.
+            The data file is named: <emphasis
+            role="bold">2of12.txt</emphasis>.</para>
 
-          <listitem>
-            <para>Press on <emphasis role="bold">Upload Now</emphasis> to
-            complete the file upload.</para>
+            <para>Verify the file, folders, landing zone information then
+            press the <emphasis role="bold">Start</emphasis> button.</para>
           </listitem>
         </orderedlist>
       </sect2>

docs/RunningHPCCinaVirtualMachine/RunningHPCCinaVirtualMachine.xml

@@ -42,15 +42,18 @@
       <para></para>
     </legalnotice>
 
-    <xi:include href="common/Version.xml" xpointer="xpointer(//*[@id='FooterInfo'])"
+    <xi:include href="common/Version.xml"
+                xpointer="xpointer(//*[@id='FooterInfo'])"
                 xmlns:xi="http://www.w3.org/2001/XInclude" />
 
-    <xi:include href="common/Version.xml" xpointer="xpointer(//*[@id='DateVer'])"
+    <xi:include href="common/Version.xml"
+                xpointer="xpointer(//*[@id='DateVer'])"
                 xmlns:xi="http://www.w3.org/2001/XInclude" />
 
     <corpname>HPCC Systems<superscript>®</superscript></corpname>
 
-    <xi:include href="common/Version.xml" xpointer="xpointer(//*[@id='Copyright'])"
+    <xi:include href="common/Version.xml"
+                xpointer="xpointer(//*[@id='Copyright'])"
                 xmlns:xi="http://www.w3.org/2001/XInclude" />
 
     <mediaobject role="logo">
@@ -541,8 +544,8 @@
     <para><orderedlist>
         <listitem>
           <para>In your browser, go to the <emphasis role="bold">ECL
-          Watch</emphasis> URL displayed (circled in red) in Figure 1,
-          <emphasis>VM Welcome Screen</emphasis>. For example,
+          Watch</emphasis> URL displayed (circled in red) in the Figure above,
+          <emphasis>VirtualBox Welcome Screen</emphasis>. For example,
           http://nnn.nnn.nnn.nnn:8010, where nnn.nnn.nnn.nnn is your Virtual
           Machine's IP address.</para>
 
@@ -1124,8 +1127,8 @@ OUTPUT(L);</programlisting></para>
 
             <listitem>
               <para>A dialog opens. <emphasis role="bold">Browse</emphasis>
-              your local machine select the file to upload and then press the
-              <emphasis role="bold">Open</emphasis> button. <figure>
+              and select the file to upload and then press the <emphasis
+              role="bold">Open</emphasis> button. <figure>
                   <title>File Uploader</title>
 
                   <mediaobject>
@@ -1142,8 +1145,8 @@ OUTPUT(L);</programlisting></para>
             </listitem>
 
             <listitem>
-              <para>Press on <emphasis role="bold">Upload Now</emphasis> to
-              complete the file upload.</para>
+              <para>Press <emphasis role="bold">Start</emphasis> to complete
+              the file upload.</para>
             </listitem>
           </orderedlist>
         </sect3>

ecl/ecl-bundle/BUNDLES.rst

@@ -49,10 +49,56 @@ A bundle can specify dependencies on zero or more other modules, using the Depen
 After the name of the bundle, a version or version range can be specified to indicate that a particular
 version of the bundle is required.
 
+Enabling a Self-Test in a Bundle
+=================================
+To enable support for automatic testing by the regression suite and the smoketest process, follow these steps:
+
+1. Create a top-level "ecl" folder in the bundle.
+2. Add one or more ECL attributes to execute
+3. Create a top level "key" folder.
+4. Add ECL file(s) to the ecl folder and corresponding XML files to the key folder. Each XML file should contain the expected result of the corresponding ECL file's output.
+
+For example, if the "ecl" folder has MyTest1.ecl and MyTest2.ecl attribute definitions, the "key" folder should have MyTest1.xml and MyTest2.xml files.
+The XML files contain the result dataset(s) from running the same named attribute.
+The test is declared successful if the contents of the XML file exactly match the contents of the workunit output(s).
+
+**To generate a key file, use this command:**
+
+   ``ecl run --target <target> <path_to_your_ecl_file>``
+
+
+then remove the **<Result>** and **</Result>** tags and save as XML.
+
+Note: Adding a self-test does not automatically ensure testing by the regression suite or smoketest. 
+
+Running a Self-Test
+===================
+1. Install and start an HPCC platofrm or VM image installed then hpcc started
+2. Install the bundle installed with the ecl command line tool. 
+
+3. Run the Regression Test Engine from the bundle main directory like this:
+
+  ``<TEST_ENGINE_HOME>/ecl-test run -t <TARGET_PLATFORM>``
+
+where
+
+TEST_ENGINE_HOME: /testing/regress
+
+TARGET_PLATFORM: hthor | thor | roxie | all
+
+For example:
+
+from the ML bundle directory: 
+
+``/mnt/disk1/home/hpccdemo/.HPCCSystems/bundles/_versions/PBblas/V3_0_2/PBblas``
+
+use
+  ``/mnt/disk1/home/hpccdemo/build/CE/platform/HPCCPlatform/testing/regress/ecl-test run -t thor``
+
 Installing a bundle
 ===================
 To install a bundle to your development machine, use the ecl command line tool: 
 
-   ecl bundle install <bundlefile>.ecl 
+   ``ecl bundle install <bundlefile>.ecl`` 
 
 For complete details, see the Client Tools Manual, available in the download section of hpccsystems.com .