Merge pull request #4447 from g-pan/H9418-CertiFix

 HPCC-9418 Refactor Certify

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>

docs/HPCCCertify/Cert-Mods/CertPreflight.xml

@@ -0,0 +1,334 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
+"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">
+<chapter id="Cert_Prelight">
+  <title>Preflight</title>
+
+  <para>The first step in certifying that the platform is installed and
+  configured properly is to run a preflight check on the components. This
+  ensures that all machines are operating and have the proper executables
+  running. This also confirms there is adequate disk space, available memory,
+  and acceptable available CPU % values.</para>
+
+  <itemizedlist mark="bullet">
+    <listitem>
+      <para>Open ECL Watch in your browser using the following URL:</para>
+
+      <para><emphasis role="bold">http://nnn.nnn.nnn.nnn:pppp
+      </emphasis><emphasis role="bold">(where nnn.nnn.nnn.nnn is your ESP
+      Server’s IP Address and pppp is the port. The default port is
+      8010)</emphasis></para>
+
+      <informaltable colsep="1" frame="all" rowsep="1">
+        <?dbfo keep-together="always"?>
+
+        <tgroup cols="2">
+          <colspec colwidth="49.50pt" />
+
+          <colspec />
+
+          <tbody>
+            <row>
+              <entry><inlinegraphic
+              fileref="../../images/caution.png" /></entry>
+
+              <entry><emphasis role="bold">Note:</emphasis> That your IP
+              address could be different from the ones provided in these
+              figures. Please use the IP address provided by your
+              installation.</entry>
+            </row>
+          </tbody>
+        </tgroup>
+      </informaltable>
+    </listitem>
+  </itemizedlist>
+
+  <sect1 id="Preflight_system_servers">
+    <title>Preflight System Servers</title>
+
+    <orderedlist>
+      <listitem>
+        <?dbfo keep-together="always"?>
+
+        <para>Click on the <emphasis role="bold">System Servers</emphasis>
+        link under Topology.</para>
+
+        <para><figure>
+            <title>System Servers link</title>
+
+            <mediaobject>
+              <imageobject>
+                <imagedata fileref="../../images/CSimg01.jpg"
+                           vendor="eclwatchSS" />
+              </imageobject>
+            </mediaobject>
+          </figure></para>
+
+        <para>A screen similar to the following displays.</para>
+
+        <para><figure>
+            <title>System Servers page</title>
+
+            <mediaobject>
+              <imageobject>
+                <imagedata fileref="../../images/CSimg02.jpg"
+                           vendor="eclwatchSS" />
+              </imageobject>
+            </mediaobject>
+          </figure></para>
+      </listitem>
+
+      <listitem>
+        <?dbfo keep-together="always"?>
+
+        <para>Press the <emphasis role="bold">Submit</emphasis> button at the
+        bottom of this page to start preflight.</para>
+
+        <para><figure>
+            <title>Submit</title>
+
+            <mediaobject>
+              <imageobject>
+                <imagedata fileref="../../images/CSimg03.jpg"
+                           vendor="eclwatchSS" />
+              </imageobject>
+            </mediaobject>
+          </figure></para>
+      </listitem>
+    </orderedlist>
+
+    <sect2 id="Systems_Expected_results">
+      <title>EXPECTED RESULTS:</title>
+
+      <para>After pressing Submit, a screen similar to the following
+      displays.</para>
+
+      <para><figure>
+          <title>System Component Inforamtion</title>
+
+          <mediaobject>
+            <imageobject>
+              <imagedata fileref="../../images/CSimg04.jpg"
+                         vendor="eclwatchSS" />
+            </imageobject>
+          </mediaobject>
+        </figure>This screen displays information on several system
+      components. This information indicates whether several components are
+      actually up and running appropriately. The resulting page shows useful
+      information about each component. The component name, the condition, the
+      component state, how long the component has been up and running, the
+      amount of disk usage, memory usage and other information is available at
+      a glance.</para>
+
+      <?hard-pagebreak ?>
+
+      <para>If there are any failed components, they are highlighted in
+      orange, indicating they are not ready.</para>
+
+      <figure>
+        <title>Failed Component</title>
+
+        <mediaobject>
+          <imageobject>
+            <imagedata fileref="../../images/CS-sysfail.jpg"
+                       vendor="eclwatchSS" />
+          </imageobject>
+        </mediaobject>
+      </figure>
+    </sect2>
+  </sect1>
+
+  <sect1 id="Preflight_Data_Refinery">
+    <title>Preflight Thor</title>
+
+    <orderedlist>
+      <listitem>
+        <para>Click on the <emphasis role="bold">Cluster Processes</emphasis>
+        link under Topology.</para>
+
+        <para><figure>
+            <title>Cluster Processes Link</title>
+
+            <mediaobject>
+              <imageobject>
+                <imagedata fileref="../../images/CSimg05.jpg"
+                           vendor="eclwatchSS, topology" />
+              </imageobject>
+            </mediaobject>
+          </figure></para>
+      </listitem>
+
+      <listitem>
+        <para>Click on the <emphasis role="bold">mythor</emphasis>
+        link.</para>
+
+        <para><figure>
+            <title>mythor link</title>
+
+            <mediaobject>
+              <imageobject>
+                <imagedata fileref="../../images/CSimg06.jpg"
+                           vendor="eclwatchSS" />
+              </imageobject>
+            </mediaobject>
+          </figure></para>
+      </listitem>
+
+      <listitem>
+        <para>Check the <emphasis role="bold">Select All</emphasis> checkbox
+        (if necessary).</para>
+      </listitem>
+
+      <listitem>
+        <para>Press the <emphasis role="bold">Submit</emphasis> button to
+        start preflight.</para>
+
+        <para><figure>
+            <title>Submit</title>
+
+            <mediaobject>
+              <imageobject>
+                <imagedata fileref="../../images/CSimg03.jpg" />
+              </imageobject>
+            </mediaobject>
+          </figure></para>
+      </listitem>
+    </orderedlist>
+
+    <sect2 id="Expected_Results_Data_refinery">
+      <title>EXPECTED RESULTS:</title>
+
+      <para>After pressing Submit, a screen similar to the following should
+      display.</para>
+
+      <para><figure>
+          <title>ESP mythor system component information</title>
+
+          <mediaobject>
+            <imageobject>
+              <imagedata fileref="../../images/CSimg07.jpg" />
+            </imageobject>
+          </mediaobject>
+        </figure>This screen displays information on Thor components. This
+      information indicates whether the components are actually up and running
+      appropriately. The resulting page shows useful information about each
+      component. The component name, the condition, the component state, how
+      long the component has been up and running, the amount of disk usage,
+      memory usage and other information is available at a glance.</para>
+
+      <para><informaltable colsep="1" frame="all" rowsep="1">
+          <?dbfo keep-together="always"?>
+
+          <tgroup cols="2">
+            <colspec colwidth="49.50pt" />
+
+            <colspec />
+
+            <tbody>
+              <row>
+                <entry><inlinegraphic
+                fileref="../../images/tip.jpg" /></entry>
+
+                <entry>If your system has more than 1 Thor cluster, repeat
+                these steps for each cluster.</entry>
+              </row>
+            </tbody>
+          </tgroup>
+        </informaltable></para>
+
+      <?hard-pagebreak ?>
+
+      <para>If there are any failed components, they are highlighted in
+      orange, indicating they are not ready.</para>
+
+      <figure>
+        <title>Failed Component</title>
+
+        <mediaobject>
+          <imageobject>
+            <imagedata fileref="../../images/CS-1912.jpg" vendor="eclwatchSS" />
+          </imageobject>
+        </mediaobject>
+      </figure>
+    </sect2>
+  </sect1>
+
+  <sect1 id="Preflight_Roxie">
+    <title>Preflight the Roxie Cluster</title>
+
+    <orderedlist>
+      <listitem>
+        <para>Click on the <emphasis role="bold">Cluster Processes</emphasis>
+        link under Topology.</para>
+
+        <para><figure>
+            <title>Cluster Processes Link</title>
+
+            <mediaobject>
+              <imageobject>
+                <imagedata fileref="../../images/CSimg05.jpg" />
+              </imageobject>
+            </mediaobject>
+          </figure></para>
+      </listitem>
+
+      <listitem>
+        <para>Click on the <emphasis role="bold">myroxie</emphasis> link.
+        <figure>
+            <title>myroxie link</title>
+
+            <mediaobject>
+              <imageobject>
+                <imagedata fileref="../../images/CSimg08.jpg" />
+              </imageobject>
+            </mediaobject>
+          </figure></para>
+      </listitem>
+
+      <listitem>
+        <para>Press the <emphasis role="bold">Submit</emphasis> button to
+        start preflight.</para>
+      </listitem>
+    </orderedlist>
+
+    <sect2 id="Expected_results_roxie">
+      <title>EXPECTED RESULTS</title>
+
+      <para>After pressing Submit, a screen similar to the following should
+      display.</para>
+
+      <para><figure>
+          <title>Roxie system information</title>
+
+          <mediaobject>
+            <imageobject>
+              <imagedata fileref="../../images/CSimg09.jpg" />
+            </imageobject>
+          </mediaobject>
+        </figure></para>
+
+      <para>This indicates whether the Roxie nodes are up and running, and
+      some information about them.</para>
+
+      <informaltable colsep="1" frame="all" rowsep="1">
+        <?dbfo keep-together="always"?>
+
+        <tgroup cols="2">
+          <colspec colwidth="49.50pt" />
+
+          <colspec />
+
+          <tbody>
+            <row>
+              <entry><inlinegraphic
+              fileref="../../images/OSSgr3.png" /></entry>
+
+              <entry>If your system has more than 1 Roxie cluster, repeat
+              these steps for each cluster.</entry>
+            </row>
+          </tbody>
+        </tgroup>
+      </informaltable>
+    </sect2>
+  </sect1>
+</chapter>

docs/HPCCCertify/Cert-Mods/CertSprayDespray.xml

@@ -0,0 +1,379 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
+"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">
+<chapter id="Cert_Spray_Despray">
+  <title>Spray and Despray Data</title>
+
+  <para>This section verifies the systems ability to Spray and Despray
+  data.</para>
+
+  <para>Spraying takes a file and distributes pieces of it across the nodes.
+  Despray is the opposite--the system combines the data from the multiple
+  nodes into a single file.</para>
+
+  <sect1 id="Move_data_to_Landing_Zone">
+    <title>Certify Despray</title>
+
+    <para>The next step to verify that your system is working properly is to
+    test the Despray capabilities. Despray is when the system combines the
+    data from the multiple clusters into a singular file that can be moved to
+    the Landing Zone from the Data Refinery.</para>
+
+    <sect2>
+      <title>Despray from ECL Watch</title>
+
+      <!--The following cromulent information can also be found in the DataTutorial.***-->
+
+      <para>Despray is the opposite of spraying, is a good way to certify that
+      piece is working properly.</para>
+
+      <orderedlist>
+        <listitem>
+          <para>To despray, go to ECL Watch in a browser window.</para>
+
+          <para>Open ECL Watch in your browser using the following URL:</para>
+
+          <para><emphasis role="bold">http://nnn.nnn.nnn.nnn:pppp
+          </emphasis><emphasis role="bold">(where nnn.nnn.nnn.nnn is your ESP
+          Server’s IP Address and pppp is the port. The default port is
+          8010)</emphasis></para>
+
+          <para><informaltable colsep="1" frame="all" rowsep="1">
+              <?dbfo keep-together="always"?>
+
+              <tgroup cols="2">
+                <colspec colwidth="49.50pt" />
+
+                <colspec />
+
+                <tbody>
+                  <row>
+                    <entry><inlinegraphic
+                    fileref="../../images/tip.jpg" /></entry>
+
+                    <entry><para><emphasis role="bold">NOTE</emphasis>: To
+                    copy a URL to the clipboard, click the <inlinegraphic
+                    fileref="../../images/CS-1421.jpg" /> icon from the row of
+                    icons along the top of the ECL Watch page.
+                    </para><para>This opens a window with the full URL. Select
+                    the entire URL and you can copy it to the clipboard to
+                    paste elsewhere. </para><para> </para></entry>
+                  </row>
+                </tbody>
+              </tgroup>
+            </informaltable></para>
+        </listitem>
+
+        <listitem>
+          <para>Click the <emphasis>Browse Logical Files </emphasis>link under
+          DFU Files.</para>
+
+          <para><figure>
+              <title>Browse Files</title>
+
+              <mediaobject>
+                <imageobject>
+                  <imagedata fileref="../../images/CSimg14.jpg" />
+                </imageobject>
+              </mediaobject>
+            </figure></para>
+        </listitem>
+
+        <listitem>
+          <para>Click on the <inlinegraphic
+          fileref="../../images/RM-19.jpg" /> button next to the file listing
+          for <emphasis
+          role="bold">certification::full_test_distributed</emphasis>, then
+          select <emphasis role="bold">Despray.</emphasis></para>
+        </listitem>
+      </orderedlist>
+
+      <para><graphic fileref="../../images/CSimg15.jpg" /></para>
+
+      <sect3 id="Desprays">
+        <title>Desprays</title>
+
+        <para>After you select Despray, the <emphasis role="bold">Despray
+        File</emphasis> page opens.</para>
+
+        <!--***Do we need an image of the Despray File Page?***-->
+
+        <itemizedlist>
+          <listitem>
+            <para>Check the <emphasis role="bold">Source</emphasis>
+            information that is already filled in, this is the name of the
+            file you selected.</para>
+          </listitem>
+
+          <listitem>
+            <para>Provide <emphasis role="bold">Destination</emphasis>
+            information.</para>
+
+            <para><informaltable colsep="0" frame="none" rowsep="0">
+                <tgroup cols="2">
+                  <colspec align="left" colwidth="122.40pt" />
+
+                  <colspec colwidth="333.00pt" />
+
+                  <tbody>
+                    <row>
+                      <entry><emphasis role="bold">Destination
+                      Machine/dropzone</emphasis></entry>
+
+                      <entry>Use the drop list to select the machine to
+                      despray to. The items in the list are landing zones
+                      defined in the system’s confguration. Your system may
+                      have only one.</entry>
+                    </row>
+
+                    <row>
+                      <entry><emphasis role="bold">Destination IP
+                      Address</emphasis></entry>
+
+                      <entry>This is prefilled based upon the selected
+                      machine.</entry>
+                    </row>
+
+                    <row>
+                      <entry><emphasis role="bold">Destination Local
+                      Path</emphasis></entry>
+
+                      <entry>Provide the complete file path of the destination
+                      including file name and extention.</entry>
+                    </row>
+
+                    <row>
+                      <entry><emphasis role="bold">Destination<emphasis
+                      role="bold"> Network Path</emphasis></emphasis></entry>
+
+                      <entry>The complete network path of the destination
+                      including file name and extension. (read only)</entry>
+                    </row>
+
+                    <row>
+                      <entry><emphasis
+                      role="bold">Overwrite</emphasis></entry>
+
+                      <entry>Check this box to overwrite a file with the same
+                      name if it exists.</entry>
+                    </row>
+                  </tbody>
+                </tgroup>
+              </informaltable></para>
+          </listitem>
+
+          <listitem>
+            <?dbfo keep-together="always"?>
+
+            <para>Press the <emphasis role="bold">Submit</emphasis>
+            button.</para>
+
+            <para>The <emphasis role="bold">DFU Workunit</emphasis>
+            displays.<graphic fileref="../../images/CS-1911.jpg"
+            vendor="eclwatchSS" /></para>
+          </listitem>
+
+          <listitem>
+            <?dbfo keep-together="always"?>
+
+            <para>Press the <emphasis role="bold">Refresh</emphasis> button
+            periodically until the status of your request indicates it is
+            <emphasis role="bold">Finished</emphasis> or click on the
+            <emphasis role="bold">View Progress</emphasis> hyperlink to see a
+            progress indicator.</para>
+
+            <para><graphic fileref="../../images/DHMan-5.jpg"
+            vendor="eclwatchSS" />The Progress window shows a green progress
+            bar indicating the percentage of completion, as well as other
+            information related to the operation.</para>
+
+            <para>If a job fails, information related to the cause of the
+            failure also displays.</para>
+          </listitem>
+        </itemizedlist>
+      </sect3>
+
+      <sect3>
+        <title id="LandingZone_Expected_Results">EXPECTED RESULTS:</title>
+
+        <para>Upon completion of the despray operation you will have a single
+        file. You can then retrieve the file from the landing zone. This will
+        certify that the despray operation is working correctly.</para>
+      </sect3>
+    </sect2>
+  </sect1>
+
+  <sect1>
+    <title>Certify Spray</title>
+
+    <para>The file will be sprayed from the Landing Zone to the Data Refinery,
+    this will certify that data can be moved from Landing Zone to the Data
+    Refinery.</para>
+
+    <!--The following is a comment***
+
+-->
+
+    <sect2 id="Spray_the_Data_File_to_your_DR-THOR_Cluster">
+      <title>Spray a Data File to your THOR Cluster</title>
+
+      <para>To use a data file in our HPCC cluster, we must first “spray” it
+      to a Thor cluster. A <emphasis>spray</emphasis> or
+      <emphasis>import</emphasis> is the relocation of a data file from one
+      location to a Thor cluster. The term spray was adopted due to the nature
+      of the file movement – the file is partitioned across all nodes within a
+      cluster.</para>
+
+      <para>For this example, we will spray the full_test_distributed file
+      that we just put out on our landing zone.</para>
+
+      <para>We are going to spray the file to our Thor cluster and give it a
+      logical name of <emphasis
+      role="bold">certification::full_test_distributed</emphasis>. The
+      Distrubuted File Utility maintains a list of logical files and their
+      corresponding physical file locations.</para>
+
+      <orderedlist>
+        <listitem>
+          <para>Open ECL Watch in your browser using the following URL:</para>
+
+          <para><emphasis role="bold">http://nnn.nnn.nnn.nnn:pppp
+          </emphasis><emphasis role="bold">(where nnn.nnn.nnn.nnn is your ESP
+          Server’s IP Address and pppp is the port. The default port is
+          8010)</emphasis></para>
+        </listitem>
+
+        <listitem>
+          <para>Click on the <emphasis role="bold">Spray Fixed</emphasis>
+          hyperlink under the DFU Files menu on the left.</para>
+
+          <para>The <emphasis role="bold">Spray Fixed</emphasis> page
+          displays.</para>
+        </listitem>
+
+        <listitem>
+          <para>Using the Source <emphasis
+          role="bold">Machine/dropzone</emphasis> drop list, select the
+          Landing Zone where the file was placed.</para>
+
+          <para>Your system may have only one Landing Zone.</para>
+
+          <para>The IP Address is automatically filled and the Local Path is
+          partially filled with the default folder on your landing
+          zone.</para>
+        </listitem>
+
+        <listitem>
+          <para>Complete the <emphasis role="bold">Local Path</emphasis> to
+          include the complete file name or use the <emphasis
+          role="bold">Choose File</emphasis> button to select the file from a
+          list of files in the folder. (The file to choose is
+          <emphasis>full_test_distributed</emphasis>)</para>
+        </listitem>
+
+        <listitem>
+          <para>Fill in the <emphasis role="bold">Record Length</emphasis>
+          (47).</para>
+        </listitem>
+
+        <listitem>
+          <para>Fill in the <emphasis role="bold">Label</emphasis> using the
+          naming convention described earlier: <emphasis
+          role="bold">certification::</emphasis><emphasis
+          role="bold">full_test_distributed</emphasis>.</para>
+        </listitem>
+
+        <listitem>
+          <?dbfo keep-together="always"?>
+
+          <para>Make sure the <emphasis
+          role="bold">Replicate</emphasis><emphasis role="bold">
+          </emphasis>tab is checked.</para>
+
+          <para><emphasis role="bold">Note:</emphasis> If replication is
+          disabled in your Thor settings, this checkbox does not
+          appear.</para>
+
+          <para><figure>
+              <title>Dropzones and Files</title>
+
+              <mediaobject>
+                <imageobject>
+                  <imagedata fileref="../../images/CS-1909.jpg"
+                             vendor="eclwatchSS" />
+                </imageobject>
+              </mediaobject>
+            </figure></para>
+        </listitem>
+
+        <listitem>
+          <para>Press the <emphasis role="bold">Submit<emphasis role="bold">
+          </emphasis></emphasis>button.</para>
+        </listitem>
+
+        <listitem>
+          <?dbfo keep-together="always"?>
+
+          <para>Click on the <emphasis role="bold">View Progress</emphasis>
+          hyperlink</para>
+
+          <para><figure>
+              <title>View Progress</title>
+
+              <mediaobject>
+                <imageobject>
+                  <imagedata fileref="../../images/DTimg02.jpg"
+                             vendor="eclwatchSS" />
+                </imageobject>
+              </mediaobject>
+            </figure>The Workunit progress page displays.</para>
+
+          <para><figure>
+              <title>Spray Complete</title>
+
+              <mediaobject>
+                <imageobject>
+                  <imagedata fileref="../../images/DTimg03.jpg"
+                             vendor="eclwatchSS" />
+                </imageobject>
+              </mediaobject>
+            </figure></para>
+        </listitem>
+      </orderedlist>
+    </sect2>
+
+    <sect2 id="SPRAY_RESULTS" role="brk">
+      <title>EXPECTED RESULTS</title>
+
+      <orderedlist>
+        <listitem>
+          <para>To verify the spray, in ECL Watch, select the Browse Logical
+          Files link under DFU Files.</para>
+
+          <para><figure>
+              <title>Browse Logical Files</title>
+
+              <mediaobject>
+                <imageobject>
+                  <imagedata fileref="../../images/CS-1910.jpg" />
+                </imageobject>
+              </mediaobject>
+            </figure></para>
+        </listitem>
+
+        <listitem>
+          <para>Click on the <inlinegraphic
+          fileref="../../images/RM-19.jpg" /> button next to the file listing
+          for <emphasis
+          role="bold">certification::full_test_distributed</emphasis>, then
+          select <emphasis role="bold">Details</emphasis> to open the logical
+          file statistics.</para>
+        </listitem>
+
+        <listitem>
+          <para>View the details.</para>
+        </listitem>
+      </orderedlist>
+    </sect2>
+  </sect1>
+</chapter>

docs/HPCCCertify/Cert-Mods/CertThorRox.xml

@@ -0,0 +1,862 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
+"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">
+<chapter id="Generate_Certification_data_file">
+  <title>Certify Thor &amp; Roxie</title>
+
+  <para>The following sections will help you to Certify that the Thor, hThor,
+  and Roxie components of your system are all working correctly.</para>
+
+  <sect1 role="nobrk">
+    <title>Build Data on Thor</title>
+
+    <para><orderedlist>
+        <listitem>
+          <para>Open <emphasis role="bold">the ECL IDE</emphasis></para>
+
+          <para><emphasis role="bold">Enter the <emphasis role="bold">Login
+          ID</emphasis> and <emphasis role="bold">Password</emphasis>
+          provided.</emphasis><informaltable colsep="1" rowsep="1">
+              <tgroup cols="2">
+                <colspec colwidth="80pt" />
+
+                <colspec colwidth="100pt" />
+
+                <tbody>
+                  <row>
+                    <entry>Login ID</entry>
+
+                    <entry>hpccdemo</entry>
+                  </row>
+
+                  <row>
+                    <entry>Password</entry>
+
+                    <entry>hpccdemo</entry>
+                  </row>
+                </tbody>
+              </tgroup>
+            </informaltable></para>
+        </listitem>
+
+        <listitem>
+          <para>Open the <emphasis
+          role="bold">_Certification.BuildDataFiles</emphasis> file.
+          <itemizedlist>
+              <listitem>
+                <para>In the lower right corner of the ECL IDE you will see a
+                section labeled <emphasis
+                role="bluebold">Repository</emphasis>, containing a few
+                folders. These folders contain the ECL files. Click the + sign
+                next to <emphasis role="bluebold">Samples</emphasis>, open the
+                folder.</para>
+              </listitem>
+
+              <listitem>
+                <para>Navigate to the <emphasis
+                role="bluebold">_Certification</emphasis> folder and click the
+                + sign next to it to. Open it and view the contents.</para>
+
+                <para><figure>
+                    <title>ECL Files in _Certification</title>
+
+                    <mediaobject>
+                      <imageobject>
+                        <imagedata fileref="../../images/CSimg10-1.jpg" />
+                      </imageobject>
+                    </mediaobject>
+                  </figure></para>
+              </listitem>
+
+              <listitem>
+                <para>Double-click on the <emphasis
+                role="bold">BuildDataFiles</emphasis> file to open it.</para>
+
+                <para>In the BuildDataFiles file, you will see some ECL code
+                in the file as follows:</para>
+
+                <programlisting>  Base := Cert.Setup.NodeMult1; //max = 20
+  Mult := Cert.Setup.NodeMult2; //max = 20
+</programlisting>
+
+                <para>These two multipliers, <emphasis
+                role="bold">NodeMult1</emphasis> and <emphasis
+                role="bold">NodeMult2</emphasis> define the total number of
+                millions of records. The values as configured in the
+                _Certification sample generate 2,000,000 records. Typically
+                you would want to generate 1 million records per node, up to
+                400 nodes. The maximum data set size is 18,800,000,000 bytes
+                (47 * 400 million). The code used in this example is designed
+                to generate a maximum of 400 million records. A larger number
+                of nodes will result in fewer records per node, however the
+                code will still work as intended for this exercise.</para>
+
+                <?hard-pagebreak ?>
+              </listitem>
+            </itemizedlist></para>
+        </listitem>
+
+        <listitem>
+          <para>Select <emphasis role="bold">thor</emphasis> as Target from
+          the drop menu on the right side.</para>
+
+          <para><figure>
+              <title>Target thor</title>
+
+              <mediaobject>
+                <imageobject>
+                  <imagedata fileref="../../images/CSimg10-2.jpg" />
+                </imageobject>
+              </mediaobject>
+            </figure></para>
+        </listitem>
+
+        <listitem>
+          <para>Press the <emphasis role="bold">Submit</emphasis>
+          button.</para>
+        </listitem>
+      </orderedlist></para>
+
+    <para><figure>
+        <title>Submit button</title>
+
+        <mediaobject>
+          <imageobject>
+            <imagedata fileref="../../images/CSimg10.jpg" />
+          </imageobject>
+        </mediaobject>
+      </figure></para>
+
+    <para><emphasis role="bold">Note:</emphasis> This code generates a data
+    file and writes it to disk.</para>
+
+    <para></para>
+
+    <sect2 id="Expected_result_Generate_Data_file">
+      <title>EXPECTED RESULT:</title>
+
+      <para>Look for the green checkmark indicating successful
+      completion</para>
+
+      <para><figure>
+          <title>Green Checkmark</title>
+
+          <mediaobject>
+            <imageobject>
+              <imagedata fileref="../../images/CS-img11-2.jpg" />
+            </imageobject>
+          </mediaobject>
+        </figure></para>
+
+      <orderedlist>
+        <listitem>
+          <?dbfo keep-together="always"?>
+
+          <para><emphasis role="bold">Open ECLWatch </emphasis>and look at the
+          Workunit details page. This illustrates the steps of the query in
+          real-time.</para>
+
+          <para>This process generates 2,000,000 - 47-byte records in a file
+          with the logical name of <emphasis
+          role="bold">certification::full_test_distributed</emphasis></para>
+
+          <para><emphasis role="bold"><!--***OK, I EXPECTED LESS ROWS***--></emphasis></para>
+
+          <para><emphasis role="bold">Note</emphasis>: The filename and other
+          variables are defined in the <emphasis
+          role="bold">_Certification.Setup </emphasis>file.</para>
+
+          <para><figure>
+              <title>Workunit details page</title>
+
+              <mediaobject>
+                <imageobject>
+                  <imagedata fileref="../../images/CSimg12.jpg" />
+                </imageobject>
+              </mediaobject>
+            </figure></para>
+        </listitem>
+
+        <listitem>
+          <?dbfo keep-together="always"?>
+
+          <para>View the result by selecting the <emphasis
+          role="bold">Result1</emphasis> tab (should be similar to the
+          following):</para>
+
+          <para><figure>
+              <title>View the Result</title>
+
+              <mediaobject>
+                <imageobject>
+                  <imagedata fileref="../../images/CSimg13.jpg" />
+                </imageobject>
+              </mediaobject>
+            </figure></para>
+        </listitem>
+      </orderedlist>
+    </sect2>
+  </sect1>
+
+  <sect1 id="Certify_Data_Refinery_functionality">
+    <title>Certify Thor Functionality</title>
+
+    <para>This section certifies:</para>
+
+    <itemizedlist mark="bullet">
+      <listitem>
+        <para>Certify Data Refinery full sort capabilities</para>
+      </listitem>
+
+      <listitem>
+        <para>Certify Data Refinery local sort capabilities</para>
+      </listitem>
+
+      <listitem>
+        <para>Certify Data Refinery local dedup capabilities</para>
+      </listitem>
+
+      <listitem>
+        <para>Certify Data Refinery hash dedup capabilities</para>
+      </listitem>
+
+      <listitem>
+        <para>Certify Data Refinery compress I/O capabilities</para>
+      </listitem>
+
+      <listitem>
+        <para>Certify Data Refinery string search capabilities</para>
+      </listitem>
+    </itemizedlist>
+
+    <para></para>
+
+    <sect2>
+      <title id="Certify_DR">Certify Thor</title>
+
+      <orderedlist>
+        <listitem>
+          <para>Open the <emphasis role="bold">ECL IDE</emphasis></para>
+
+          <para><emphasis role="bold">Enter the <emphasis role="bold">Login
+          ID</emphasis> and <emphasis
+          role="bold">Password</emphasis>.</emphasis><informaltable colsep="1"
+              rowsep="1">
+              <tgroup cols="2">
+                <colspec colwidth="80pt" />
+
+                <colspec colwidth="100pt" />
+
+                <tbody>
+                  <row>
+                    <entry>Login ID</entry>
+
+                    <entry>hpccdemo</entry>
+                  </row>
+
+                  <row>
+                    <entry>Password</entry>
+
+                    <entry>hpccdemo</entry>
+                  </row>
+                </tbody>
+              </tgroup>
+            </informaltable></para>
+        </listitem>
+
+        <listitem>
+          <para>Open the <emphasis
+          role="bold">_Certification.Certify_DR</emphasis> file.</para>
+
+          <para><itemizedlist>
+              <listitem>
+                <para>In the lower right corner of the ECL IDE you will see a
+                section labeled as Repository, containing a few folders. This
+                contains the ECL files. Click the + sign next to
+                <emphasis>Samples</emphasis>, to open the folder.</para>
+              </listitem>
+
+              <listitem>
+                <para>Navigate to the _Certification folder and click the +
+                sign next to it to open it and view the contents.</para>
+
+                <para><figure>
+                    <title>ECL Certify_DR File</title>
+
+                    <mediaobject>
+                      <imageobject>
+                        <imagedata fileref="../../images/CSimg10-3.jpg" />
+                      </imageobject>
+                    </mediaobject>
+                  </figure></para>
+              </listitem>
+
+              <listitem>
+                <para>Double-click on the <emphasis
+                role="bold">Certify_DR</emphasis> file to open it.</para>
+              </listitem>
+            </itemizedlist></para>
+        </listitem>
+
+        <listitem>
+          <para>Select <emphasis role="bold">thor</emphasis> as the Target
+          from the drop menu on the right side.</para>
+
+          <para><figure>
+              <title>Target: thor</title>
+
+              <mediaobject>
+                <imageobject>
+                  <imagedata fileref="../../images/CSimg10-2.jpg" />
+                </imageobject>
+              </mediaobject>
+            </figure></para>
+        </listitem>
+
+        <listitem>
+          <para>Press the <emphasis role="bold">Submit</emphasis>
+          button.</para>
+
+          <para><figure>
+              <title>Submit button</title>
+
+              <mediaobject>
+                <imageobject>
+                  <imagedata fileref="../../images/CSimg10-S.jpg" />
+                </imageobject>
+              </mediaobject>
+            </figure></para>
+        </listitem>
+      </orderedlist>
+
+      <para><emphasis role="bold">Note:</emphasis> This file uses the
+      previously generated data and tests a series of Thor capabilities. It
+      does not write data to disk. It will take a few minutes to complete,
+      depending on the size of your system.</para>
+
+      <para></para>
+    </sect2>
+
+    <sect2 id="CEertify_DR_Expected_Result">
+      <title>EXPECTED RESULT</title>
+
+      <para>Look for the Green checkmark indicating successful
+      completion.</para>
+
+      <para><figure>
+          <title>Green checkmark</title>
+
+          <mediaobject>
+            <imageobject>
+              <imagedata fileref="../../images/CSimg11.jpg" />
+            </imageobject>
+          </mediaobject>
+        </figure></para>
+
+      <para>The ECL Watch <emphasis role="bold">Results</emphasis> tab section
+      should be as follows:</para>
+
+      <blockquote>
+        <para>Result 1 Full Global Join - should = 2 million : 2000000</para>
+
+        <para>Result 2 Local Join - should = 2 million (local): 2000000</para>
+
+        <para>Result 3 Dedup - should = 2 million (joined): 2000000</para>
+
+        <para>Result 4 Complex I/O - should = 2 million: 2000000</para>
+
+        <para>Result 5 Hash Aggregate (Should be 2 records): 2</para>
+
+        <para>Result 6 Global Aggregate (Should be 2 records): 2</para>
+
+        <para>Result 7 Local Aggregate (Should be 2 records): 2</para>
+
+        <para>Result 8 Global Grouped Rollup (Should be 2 records): 2</para>
+
+        <para>Result 9 Local Rollup (Should be 2 records): 2</para>
+
+        <para>Result 10 Local Grouped Rollup (Should be 2 records): 2</para>
+
+        <para>Result 11 Global It/Srt/Ddp (Should be 2 records): 2</para>
+
+        <para>Result 12 Global Grouped It/Srt/Ddp (Should be 2 records):
+        2</para>
+
+        <para>Result 13 Local It/Srt/Ddp (Should be 2 records): 2</para>
+
+        <para>Result 14 Local Grouped It/Srt/Ddp (Should be 2 records):
+        2</para>
+
+        <para>Result 15 String Search Results: 100000</para>
+      </blockquote>
+    </sect2>
+  </sect1>
+
+  <sect1 id="Certify_DDE">
+    <title>Certify Key build capabilities</title>
+
+    <para>This section will certify that the system can perform its key build
+    capabilities.</para>
+
+    <orderedlist>
+      <listitem>
+        <para>Open the <emphasis role="bold">ECL IDE</emphasis></para>
+
+        <para><emphasis role="bold">Enter the <emphasis role="bold">Login
+        ID</emphasis> and <emphasis
+        role="bold">Password</emphasis>.</emphasis><informaltable colsep="1"
+            rowsep="1">
+            <tgroup cols="2">
+              <colspec colwidth="80pt" />
+
+              <colspec colwidth="100pt" />
+
+              <tbody>
+                <row>
+                  <entry>Login ID</entry>
+
+                  <entry>hpccdemo</entry>
+                </row>
+
+                <row>
+                  <entry>Password</entry>
+
+                  <entry>hpccdemo</entry>
+                </row>
+              </tbody>
+            </tgroup>
+          </informaltable></para>
+      </listitem>
+
+      <listitem>
+        <para>Open <emphasis role="bold">_Certification.build_index</emphasis>
+        file.</para>
+
+        <para><itemizedlist>
+            <listitem>
+              <para>In the lower right corner of the ECL IDE you will see a
+              section labeled as Repository, containing a couple of folders.
+              This contains the ECL files. Select the + sign next to it
+              Samples, open the folder.</para>
+            </listitem>
+
+            <listitem>
+              <para>Navigate to the <emphasis
+              role="bold">_Certification</emphasis> folder and select the +
+              sign next to it to open it and view the contents.</para>
+
+              <para><figure>
+                  <title>Expand the _Certification folder</title>
+
+                  <mediaobject>
+                    <imageobject>
+                      <imagedata fileref="../../images/CSimg10-4.jpg" />
+                    </imageobject>
+                  </mediaobject>
+                </figure></para>
+            </listitem>
+
+            <listitem>
+              <para>Double-click on the <emphasis role="bold">build_index
+              </emphasis>file to open it.</para>
+            </listitem>
+          </itemizedlist></para>
+      </listitem>
+
+      <listitem>
+        <?dbfo keep-together="always"?>
+
+        <para>Select <emphasis role="bold">thor</emphasis> as Target from the
+        drop menu on the right side.</para>
+
+        <para><figure>
+            <title>Target: thor</title>
+
+            <mediaobject>
+              <imageobject>
+                <imagedata fileref="../../images/CSimg10-2.jpg" />
+              </imageobject>
+            </mediaobject>
+          </figure></para>
+      </listitem>
+
+      <listitem>
+        <?dbfo keep-together="always"?>
+
+        <para>Press the <emphasis role="bold">Submit</emphasis> button.</para>
+
+        <para><figure>
+            <title>Submit button</title>
+
+            <mediaobject>
+              <imageobject>
+                <imagedata fileref="../../images/CSimg10-S.jpg" />
+              </imageobject>
+            </mediaobject>
+          </figure></para>
+      </listitem>
+    </orderedlist>
+
+    <para><emphasis role="bold">Note:</emphasis> This file uses the previously
+    generated data. It builds an index on one of those data files.</para>
+
+    <sect2 id="Expected_Result_Certify_DR">
+      <title>EXPECTED RESULT</title>
+
+      <para>Look for the green checkmark indicating successful
+      completion.</para>
+
+      <para><figure>
+          <title>Green checkmark</title>
+
+          <mediaobject>
+            <imageobject>
+              <imagedata fileref="../../images/CSimg11-1.jpg" />
+            </imageobject>
+          </mediaobject>
+        </figure></para>
+
+      <para>The file we created earlier is indexed by Last Name and the index
+      file, <emphasis
+      role="bold">thor::full_test_distributed_index</emphasis>, is written to
+      disk.</para>
+    </sect2>
+  </sect1>
+
+  <sect1>
+    <title>Verify the Index Build</title>
+
+    <orderedlist>
+      <listitem>
+        <para>Open ECL Watch in your browser using the following URL:</para>
+
+        <para><emphasis role="bold">http://nnn.nnn.nnn.nnn:pppp
+        </emphasis><emphasis role="bold">(where nnn.nnn.nnn.nnn is your ESP
+        Server’s IP Address and pppp is the port. The default port is
+        8010)</emphasis></para>
+      </listitem>
+
+      <listitem>
+        <?dbfo keep-together="always"?>
+
+        <para>Click on the <emphasis>Browse Logical Files</emphasis> link
+        under DFU Files.</para>
+
+        <para><figure>
+            <title>Browse Logical Files link</title>
+
+            <mediaobject>
+              <imageobject>
+                <imagedata fileref="../../images/CSimg14.jpg" />
+              </imageobject>
+            </mediaobject>
+          </figure></para>
+      </listitem>
+
+      <listitem>
+        <?dbfo keep-together="always"?>
+
+        <para>Click on the <inlinegraphic fileref="../../images/RM-19.jpg" />
+        button next to the file listing for <emphasis
+        role="bold">thor::full_test_distributed_index</emphasis> , then select
+        <emphasis role="bold">Details</emphasis> to open the logical file
+        details page.</para>
+      </listitem>
+    </orderedlist>
+
+    <para><figure>
+        <title>Details</title>
+
+        <mediaobject>
+          <imageobject>
+            <imagedata fileref="../../images/CSimg14-2.jpg" />
+          </imageobject>
+        </mediaobject>
+      </figure><!--***dont we need to add more content here as to what exact Details we should see??***--></para>
+  </sect1>
+
+  <sect1 id="Certify_Thor_access_to_index_data" role="brk">
+    <title>Certify Thor Access to Indexed Data</title>
+
+    <para>This section certifies Thor access to indexed data.</para>
+
+    <sect2>
+      <title>Certify Thor Access</title>
+
+      <orderedlist>
+        <listitem>
+          <para>Open the <emphasis
+          role="bold">_Certification.read_index</emphasis> file.</para>
+
+          <para><itemizedlist>
+              <listitem>
+                <para>In the lower right corner of the ECL IDE you will see a
+                section labeled as Repository, containing a couple of folders.
+                This contains the ECL files. Click the + sign next to
+                <emphasis>Samples</emphasis>, open the folder.</para>
+              </listitem>
+
+              <listitem>
+                <para>Navigate to the <emphasis
+                role="bold">_Certification</emphasis> folder and click the +
+                sign next to it to open it and view the contents.</para>
+
+                <para><figure>
+                    <title>Expand the _Certification folder</title>
+
+                    <mediaobject>
+                      <imageobject>
+                        <imagedata fileref="../../images/CSimg10-5.jpg" />
+                      </imageobject>
+                    </mediaobject>
+                  </figure></para>
+              </listitem>
+
+              <listitem>
+                <para>Double-click on the <emphasis role="bold">read_index
+                </emphasis>file to open it.</para>
+              </listitem>
+            </itemizedlist></para>
+        </listitem>
+
+        <listitem>
+          <?dbfo keep-together="always"?>
+
+          <para>Select <emphasis role="bold">thor</emphasis> as Target from
+          the drop menu on the right side.</para>
+
+          <para><figure>
+              <title>Target: thor</title>
+
+              <mediaobject>
+                <imageobject>
+                  <imagedata fileref="../../images/CSimg10-2.jpg" />
+                </imageobject>
+              </mediaobject>
+            </figure></para>
+        </listitem>
+
+        <listitem>
+          <?dbfo keep-together="always"?>
+
+          <para>Press the <emphasis role="bold">Submit</emphasis>
+          button.</para>
+
+          <para><figure>
+              <title>Submit button</title>
+
+              <mediaobject>
+                <imageobject>
+                  <imagedata fileref="../../images/CSimg10-S.jpg" />
+                </imageobject>
+              </mediaobject>
+            </figure></para>
+        </listitem>
+      </orderedlist>
+    </sect2>
+
+    <sect2>
+      <title>EXPECTED RESULT:</title>
+
+      <para>The first 100 records from the query display, looking similar to
+      the following (BRYANT in last name).</para>
+
+      <para><figure>
+          <title>Results page</title>
+
+          <mediaobject>
+            <imageobject>
+              <imagedata fileref="../../images/CSimg13-1.jpg" />
+            </imageobject>
+          </mediaobject>
+        </figure></para>
+
+      <para></para>
+    </sect2>
+  </sect1>
+
+  <sect1>
+    <title>Compile and Publish a Roxie Query</title>
+
+    <orderedlist>
+      <listitem>
+        <para>Open the <emphasis
+        role="bold">_Certification.ReadIndexService</emphasis> file, If you do
+        not have it open already.</para>
+
+        <para><itemizedlist>
+            <listitem>
+              <para>In the lower right corner of the ECL IDE you will see a
+              section labeled as Repository, containing a couple of folders.
+              This contains the ECL files. Click the + sign next to <emphasis
+              role="bold">Samples</emphasis>, open the folder.</para>
+            </listitem>
+
+            <listitem>
+              <para>Navigate to the <emphasis
+              role="bold">_Certification</emphasis> folder and click the +
+              sign next to it to open it and view the contents.</para>
+
+              <para><figure>
+                  <title>_Certification folder</title>
+
+                  <mediaobject>
+                    <imageobject>
+                      <imagedata fileref="../../images/CSimg10-6.jpg" />
+                    </imageobject>
+                  </mediaobject>
+                </figure></para>
+            </listitem>
+
+            <listitem>
+              <para>Double-click on the <emphasis role="bold">ReadIndexService
+              </emphasis>file to open it.</para>
+            </listitem>
+          </itemizedlist></para>
+      </listitem>
+
+      <listitem>
+        <para>Select <emphasis role="bold">roxie</emphasis> as Target from the
+        drop menu on the right side.</para>
+
+        <para><figure>
+            <title>Target roxie</title>
+
+            <mediaobject>
+              <imageobject>
+                <imagedata fileref="../../images/CSimg24.jpg" />
+              </imageobject>
+            </mediaobject>
+          </figure></para>
+      </listitem>
+
+      <listitem>
+        <para>In the upper left corner the <emphasis
+        role="bold">Submit</emphasis> button has an arrow next to it. Select
+        the arrow to expose the <emphasis role="bold">Compile</emphasis>
+        option.</para>
+
+        <para>Select <emphasis role="bold">Compile</emphasis> from the drop
+        list under the submit button.</para>
+
+        <para><figure>
+            <title>Compile</title>
+
+            <mediaobject>
+              <imageobject>
+                <imagedata fileref="../../images/CSimg14-3.jpg" />
+              </imageobject>
+            </mediaobject>
+          </figure></para>
+
+        <para>Look for a green circle indicating successful completion. Once
+        complete, select the Workunit next to the the green circle.</para>
+
+        <para><figure>
+            <title>Completed Workunit: Green circle</title>
+
+            <mediaobject>
+              <imageobject>
+                <imagedata fileref="../../images/CS-1102.jpg" />
+              </imageobject>
+            </mediaobject>
+          </figure></para>
+      </listitem>
+
+      <listitem>
+        <para>Select the <emphasis role="bold">ECL Watch</emphasis> button at
+        the lower left corner of the window.</para>
+
+        <para><figure>
+            <title>Select ECL Watch</title>
+
+            <mediaobject>
+              <imageobject>
+                <imagedata fileref="../../images/CS-1913.jpg" />
+              </imageobject>
+            </mediaobject>
+          </figure></para>
+      </listitem>
+
+      <listitem>
+        <para>Select the <emphasis role="bold">Publish</emphasis> button from
+        the ECL Watch tab that you just opened. (you may have to scroll down
+        in the main window)<figure>
+            <title>ECL Watch Publish button</title>
+
+            <mediaobject>
+              <imageobject>
+                <imagedata fileref="../../images/CS-1411.jpg" />
+              </imageobject>
+            </mediaobject>
+          </figure></para>
+      </listitem>
+
+      <listitem>
+        <para>Open the ESP page in your browser using the following
+        URL:</para>
+
+        <para><emphasis role="bold">http://nnn.nnn.nnn.nnn:pppp
+        </emphasis><emphasis role="bold">(where nnn.nnn.nnn.nnn is your ESP
+        Server’s IP Address and pppp is the port. The default port is
+        8002)</emphasis></para>
+
+        <para><figure>
+            <title>Roxie ESP</title>
+
+            <mediaobject>
+              <imageobject>
+                <imagedata fileref="../../images/CSimg18.jpg" />
+              </imageobject>
+            </mediaobject>
+          </figure></para>
+      </listitem>
+
+      <listitem>
+        <para>Click the + sign next to <emphasis role="bold">roxie</emphasis>,
+        to expand it</para>
+      </listitem>
+
+      <listitem>
+        <para>Click <emphasis role="bold">readindexservice</emphasis></para>
+      </listitem>
+
+      <listitem>
+        <para>Enter the name BRYANT in the <emphasis
+        role="bold">lname</emphasis> field.<!--(output xml || output tables... )***--></para>
+
+        <para><figure>
+            <title>Enter lnamein</title>
+
+            <mediaobject>
+              <imageobject>
+                <imagedata fileref="../../images/CS-1802.jpg" />
+              </imageobject>
+            </mediaobject>
+          </figure></para>
+      </listitem>
+
+      <listitem>
+        <para>Press the <emphasis role="bold">Submit</emphasis> button at the
+        bottom of the form.</para>
+      </listitem>
+    </orderedlist>
+
+    <sect2 id="The_Expected_Result">
+      <title>EXPECTED RESULT:</title>
+
+      <para>A list of 100 records should display, looking similar to the
+      following (BRYANT in last name).</para>
+
+      <para><figure>
+          <title>Result</title>
+
+          <mediaobject>
+            <imageobject>
+              <imagedata fileref="../../images/CSimg20.jpg" />
+            </imageobject>
+          </mediaobject>
+        </figure></para>
+    </sect2>
+  </sect1>
+</chapter>