Big-Data-HPC-Platform/docs/DataHandling.xml

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">
<book lang="en_US">
  <bookinfo>
    <title>HPCC Data Handling</title>

    <mediaobject>
      <imageobject>
        <imagedata fileref="images/redswooshWithLogo3.jpg" />
      </imageobject>
    </mediaobject>

    <author>
      <surname>Boca Raton Documentation Team</surname>
    </author>

    <legalnotice>
      <para>We welcome your comments and feedback about this document via
      email to <email>docfeedback@hpccsystems.com</email> Please include
      <emphasis role="bold">Documentation Feedback</emphasis> in the subject
      line and reference the document name, page numbers, and current Version
      Number in the text of the message.</para>

      <para>LexisNexis and the Knowledge Burst logo are registered trademarks
      of Reed Elsevier Properties Inc., used under license. Other products,
      logos, and services may be trademarks or registered trademarks of their
      respective companies. All names and example data used in this manual are
      fictitious. Any similarity to actual persons, living or dead, is purely
      coincidental.</para>

      <para></para>
    </legalnotice>

        <xi:include href="Version.xml" xpointer="FooterInfo"
                xmlns:xi="http://www.w3.org/2001/XInclude" />

    <xi:include href="Version.xml" xpointer="DateVer"
                xmlns:xi="http://www.w3.org/2001/XInclude" />

    <corpname>HPCC Systems</corpname>

    <xi:include href="Version.xml" xpointer="Copyright"
                xmlns:xi="http://www.w3.org/2001/XInclude" />

    <mediaobject role="logo">
      <imageobject>
        <imagedata fileref="images/LN_Rightjustified.jpg" />
      </imageobject>
    </mediaobject>
  </bookinfo>

  <chapter id="Data_Handling">
    <title><emphasis>HPCC Data Handling</emphasis></title>

    <sect1 id="Introduction" role="nobrk">
      <title>Introduction</title>

      <para>There are a number of different ways in which data may be
      transferred to, from, or within an HPCC system. For each of these data
      transfers, there are a few key parameters that must be known.</para>

      <sect2 id="Prerequisites-for-most-file-movements">
        <title><emphasis role="bold">Prerequisites for most file
        movements:</emphasis></title>

        <itemizedlist>
          <listitem>
            <para>Logical filename</para>
          </listitem>

          <listitem>
            <para>Physical filename</para>
          </listitem>

          <listitem>
            <para>Record size (fixed)</para>
          </listitem>

          <listitem>
            <para>Source directory</para>
          </listitem>

          <listitem>
            <para>Destination directory</para>
          </listitem>

          <listitem>
            <para>Dali IP address (source and/or destination)</para>
          </listitem>

          <listitem>
            <para>Landing Zone IP address</para>
          </listitem>
        </itemizedlist>

        <para>The above parameters are used for these major data handling
        methods:</para>

        <itemizedlist>
          <listitem>
            <para>Import - Spraying Data from the Landing Zone to Thor</para>
          </listitem>

          <listitem>
            <para>Export - Despraying Data from Thor to Landing Zone</para>
          </listitem>

          <listitem>
            <para>Copy - Replicating Data from Thor to Thor (within same Dali
            File System)</para>
          </listitem>

          <listitem>
            <para>Copying Data from Thor to Thor (between different Dali File
            Systems)</para>
          </listitem>
        </itemizedlist>

        <para></para>
      </sect2>
    </sect1>

    <sect1 id="Data_Handling_Terms">
      <title>Data Handling Terms</title>

      <para>A <emphasis>spray</emphasis> or <emphasis>import</emphasis> is the
      relocation of a data file from one location (such as a Landing Zone) to
      a Data Refinery cluster. The term spray was adopted due to the nature of
      the file movement – the file is partitioned across all nodes within a
      cluster.</para>

      <para>A <emphasis>despray</emphasis> or <emphasis>export</emphasis> is
      the relocation of a data file from a Data Refinery cluster to a single
      machine location (such as a Landing Zone). The term despray was adopted
      due to the nature of the file movement – the file is reassembled from
      its parts on all nodes in the cluster and placed in a single file on the
      destination.</para>

      <para>A <emphasis>copy </emphasis>is the replication of a data file from
      one Data Refinery cluster to another cluster within the same
      environment.</para>

      <para>A <emphasis>Remote copy </emphasis>is the replication of a data
      file from one Data Refinery cluster to another cluster in a different
      environment.</para>

      <para>A <emphasis>Landing Zone</emphasis> (or Drop Zone) is a physical
      storage location defined in your system's environment. There can be one
      or more of these locations defined. A daemon (DaFileSrv) must be running
      on that server to enable file sprays and desprays.</para>

      <beginpage />
    </sect1>

    <sect1 id="Working_with_a_data_file">
      <title>Working with data files</title>

      <para>Once you start working with your HPCC system, you will want to
      process some real data, this section shows you how to load data to your
      HPCC system.</para>

      <sect2 id="Cautions_and_Warnings">
        <title>Before you begin</title>

        <para>First, you should consider the size of the data and the capacity
        of your system. A typical production HPCC system would have much more
        data capacity than a development system. The size of the files you
        wish to work with is limited by the size of your system.</para>
      </sect2>

      <sect2 id="Uploading_a_file">
        <title>Uploading a file</title>

        <para>For smaller data files, maximum of 2GB, you can use the
        upload/download file utility in ECL Watch.</para>

        <orderedlist>
          <listitem>
            <para>In your browser, go to the <emphasis role="bold">ECL
            Watch</emphasis> URL displayed example,
            http://nnn.nnn.nnn.nnn:8010, where nnn.nnn.nnn.nnn is your ESP
            Server's IP address.</para>

            <para><informaltable colsep="1" frame="all" rowsep="1">
                <?dbfo keep-together="always"?>

                <tgroup cols="2">
                  <colspec colwidth="49.50pt" />

                  <colspec />

                  <tbody>
                    <row>
                      <entry><inlinegraphic
                      fileref="images/caution.png" /></entry>

                      <entry>Your IP address could be different from the ones
                      provided in the example images. Please use the IP
                      address provided by <emphasis
                      role="bold">your</emphasis> installation.</entry>
                    </row>
                  </tbody>
                </tgroup>
              </informaltable></para>
          </listitem>

          <listitem>
            <para>From ECL Watch page, click on the <emphasis
            role="bold">Upload/download File </emphasis> link in the menu on
            the left side.</para>

            <para><figure>
                <title>Upload/download</title>

                <mediaobject>
                  <imageobject>
                    <imagedata fileref="images/LZimg03-1.jpg" />
                  </imageobject>
                </mediaobject>
              </figure></para>

            <para><phrase> </phrase>Once you click on the Upload/download file
            link, it will take you to the dropzones and files page, where you
            can choose to <emphasis role="bold">Browse</emphasis> your machine
            for a file to upload:</para>

            <para><figure>
                <title>Dropzones</title>

                <mediaobject>
                  <imageobject>
                    <imagedata fileref="images/LZimg04.jpg" />
                  </imageobject>
                </mediaobject>
              </figure></para>
          </listitem>

          <listitem>
            <para>Press the <emphasis role="bold">Browse</emphasis> button to
            browse the files on your local machine, select the file to upload
            and then click <emphasis role="bold">Open</emphasis>
            button.</para>

            <para>The file you selected should appear in the <emphasis
            role="bold">Select a file to upload</emphasis> field.</para>
          </listitem>

          <listitem>
            <para>Press on <emphasis role="bold">Upload Now</emphasis> to
            complete the file upload.</para>
          </listitem>
        </orderedlist>
      </sect2>

      <sect2 id="Uploading_files_w_secure_client">
        <title>Uploading files with a Secure Copy Client</title>

        <para>To upload a large file for processing to your virtual machine,
        you will need a tool that supports the secure copy protocol. In this
        section, we discuss using WinSCP. There are other tools available, but
        the steps are similar.</para>

        <para><orderedlist>
            <listitem>
              <para>Open the WinSCP tool, and login to your Virtual Machine's
              IP address using the username and password given.</para>

              <para><informaltable colsep="1" rowsep="1">
                  <tgroup cols="2">
                    <colspec colwidth="80pt" />

                    <colspec colwidth="100pt" />

                    <tbody>
                      <row>
                        <entry>Login ID:</entry>

                        <entry>hpccdemo</entry>
                      </row>

                      <row>
                        <entry>Password:</entry>

                        <entry>hpccdemo</entry>
                      </row>
                    </tbody>
                  </tgroup>
                </informaltable></para>
            </listitem>

            <listitem>
              <para>Once logged in, it should, navigate automatically to the
              landing zone folder. (/var/lib/LexisNexis/mydropzone)</para>
            </listitem>

            <listitem>
              <?dbfo keep-together="always"?>

              <para>Navigate to where your local file is in the left part of
              the window.</para>

              <para><figure>
                  <title>WinSCP</title>

                  <mediaobject>
                    <imageobject>
                      <imagedata fileref="images/LZimg05.jpg" />
                    </imageobject>
                  </mediaobject>
                </figure></para>
            </listitem>

            <listitem>
              <para>Select the data file to send and copy it to the landing
              zone, using drag-and-drop.</para>
            </listitem>
          </orderedlist></para>
      </sect2>
    </sect1>

    <sect1 id="Data_Handling_Methods">
      <title>Data Handling Methods</title>

      <para>There are several ways to spray, despray, or copy data
      files:</para>

      <itemizedlist>
        <listitem>
          <para>The DFU interface in Ecl Watch</para>
        </listitem>

        <listitem>
          <para>The DFU Plus command line utility</para>

          <para>See the <emphasis>Client Tools</emphasis> manual for
          details</para>
        </listitem>

        <listitem>
          <para>Using ECL Code and FileServices library functions.</para>

          <para>See the <emphasis>ECL Language Reference</emphasis> for
          details.</para>
        </listitem>
      </itemizedlist>

      <sect2 id="Data_Handling_Using_ECL-Watch">
        <title>Data Handling Using ECL Watch</title>

        <itemizedlist>
          <listitem>
            <para>Login to ECL Watch for the environment.</para>

            <para>The URL is the IP address where the ESP Server is installed
            plus the port to which the wssmc service is bound. The default
            port is 8010. For example:<programlisting>http://&lt;ESPserverIP&gt;:8010/</programlisting></para>
          </listitem>

          <listitem>
            <?dbfo keep-together="always"?>

            <para>Click on the <emphasis role="bold">Browse Logical
            Files</emphasis> hyperlink below <emphasis role="bold">DFU Files
            </emphasis>in the menu on the left.</para>

            <para>The Logical Files page displays showing all files with
            logical entries in the Dali Server’s Distributed File
            System.</para>

            <para><graphic fileref="images/DHMan-3.jpg" scalefit="1" /></para>

            <para>From this page, you can despray or copy any file.</para>
          </listitem>
        </itemizedlist>

        <sect3 id="Desprays">
          <title>Desprays</title>

          <itemizedlist>
            <listitem>
              <para>Locate the file to despray in the list of files, then
              select the arrow graphic on the left hand side, then select
              <emphasis role="bold">Despray</emphasis> from the pop-up
              menu.</para>
            </listitem>

            <listitem>
              <para>Check the <emphasis role="bold">Source</emphasis>
              information that is already filled in.</para>
            </listitem>

            <listitem>
              <para> Provide <emphasis role="bold">Destination</emphasis>
              information.</para>

              <para><informaltable colsep="0" frame="none" rowsep="0">
                  <tgroup cols="2">
                    <colspec align="left" colwidth="122.40pt" />

                    <colspec colwidth="333.00pt" />

                    <tbody>
                      <row>
                        <entry><emphasis
                        role="bold">Destination-Machine</emphasis></entry>

                        <entry>Use the drop-list to select the machine to
                        despray to. The items in the list are landing zones
                        defined in the system’s confguration.</entry>
                      </row>

                      <row>
                        <entry><emphasis role="bold">Destination-IP
                        Address</emphasis></entry>

                        <entry>This is prefilled based upon the selected
                        machine.</entry>
                      </row>

                      <row>
                        <entry><emphasis role="bold">Destination-Local
                        Path</emphasis></entry>

                        <entry>Provide the complete file path of the
                        destination including file name and extention.</entry>
                      </row>

                      <row>
                        <entry><emphasis role="bold">Destination-<emphasis
                        role="bold">Network Path</emphasis></emphasis></entry>

                        <entry>The complete network path of the destination
                        including file name and extension. (read only)</entry>
                      </row>

                      <row>
                        <entry><emphasis
                        role="bold">Overwrite</emphasis></entry>

                        <entry>Check this box to overwrite a file with the
                        same name if it exists.</entry>
                      </row>
                    </tbody>
                  </tgroup>
                </informaltable></para>
            </listitem>

            <listitem>
              <?dbfo keep-together="always"?>

              <para>Press the <emphasis role="bold">Submit</emphasis>
              button.</para>

              <para>The <emphasis role="bold">DFU Workunit</emphasis>
              displays.<graphic fileref="images/DHMan-4.jpg"
              scalefit="1" /></para>
            </listitem>

            <listitem>
              <?dbfo keep-together="always"?>

              <para>Press the <emphasis role="bold">Refresh</emphasis> button
              periodically until the status of your request indicates it is
              <emphasis role="bold">Finished</emphasis> or click on the
              <emphasis role="bold">View Progress</emphasis> hyperlink to see
              a progress indicator.</para>

              <para><graphic fileref="images/DHMan-5.jpg" scalefit="1" />The
              Progress window shows a green progress bar indicating the
              percentage of completion, as well as other information related
              to the operation.</para>

              <para>If a job fails, information related to the cause of the
              failure also displays.</para>
            </listitem>
          </itemizedlist>
        </sect3>

        <sect3 id="Spray_Fixed">
          <title><emphasis role="bold">Spray Fixed</emphasis></title>

          <itemizedlist>
            <listitem>
              <para>Click on the <emphasis role="bold">Spray Fixed</emphasis>
              hyperlink below <emphasis role="bold">DFU</emphasis> in the menu
              on the left.</para>

              <para>The <emphasis role="bold">Spray Fixed</emphasis> page
              displays.</para>
            </listitem>

            <listitem>
              <para>Fill in <emphasis role="bold">Source
              </emphasis>information (<emphasis role="bold">Machine, IP, File
              Path, and record length)</emphasis> and the <emphasis
              role="bold">Destination</emphasis> information (<emphasis
              role="bold">Group</emphasis>and <emphasis role="bold">Label
              ).</emphasis></para>

              <para><informaltable colsep="0" frame="none" rowsep="0">
                  <tgroup cols="2">
                    <colspec colwidth="122.40pt" />

                    <colspec colwidth="333.00pt" />

                    <tbody>
                      <row>
                        <entry align="right"><emphasis
                        role="bold">Source:</emphasis></entry>
                      </row>

                      <row>
                        <entry><emphasis
                        role="bold">Machine</emphasis></entry>

                        <entry>Use the drop-list to select the machine where
                        the source file is located.</entry>
                      </row>

                      <row>
                        <entry><emphasis role="bold">IP</emphasis></entry>

                        <entry>IP address of machine from which to spray. This
                        is automatically completed when you select the
                        <emphasis role="bold">Source
                        Machine.</emphasis></entry>
                      </row>

                      <row>
                        <entry><emphasis role="bold">Local
                        Path</emphasis></entry>

                        <entry>The file path of source file to spray.</entry>
                      </row>

                      <row>
                        <entry><emphasis role="bold">Record
                        Length</emphasis></entry>

                        <entry>The size of each record.</entry>
                      </row>

                      <row>
                        <entry align="right"><emphasis
                        role="bold">Destination:</emphasis></entry>
                      </row>

                      <row>
                        <entry><emphasis role="bold">Group</emphasis></entry>

                        <entry>Select the name of THOR cluster to spray
                        to.</entry>
                      </row>

                      <row>
                        <entry><emphasis role="bold">Label</emphasis></entry>

                        <entry>The logical name that you choose for the
                        file.</entry>
                      </row>

                      <row>
                        <entry align="right"><emphasis
                        role="bold">Options:</emphasis></entry>
                      </row>

                      <row>
                        <entry><emphasis
                        role="bold">Overwrite</emphasis></entry>

                        <entry>Check this box to overwrite files of the same
                        name.</entry>
                      </row>

                      <row>
                        <entry><emphasis
                        role="bold">Replicate</emphasis></entry>

                        <entry>Check this box to create backup copies of all
                        file parts exist in the backup directory (by
                        convention on the secondary drive of the node
                        following in the cluster).</entry>
                      </row>

                      <row>
                        <entry><emphasis
                        role="bold">Compress</emphasis></entry>

                        <entry>Check this box to compress the files.</entry>
                      </row>
                    </tbody>
                  </tgroup>
                </informaltable></para>
            </listitem>

            <listitem>
              <para>Press the <emphasis role="bold">Submit</emphasis>
              button.</para>

              <para>The <emphasis role="bold">DFU Workunit</emphasis>
              displays.</para>
            </listitem>

            <listitem>
              <?dbfo keep-together="always"?>

              <para>Press the <emphasis role="bold">Refresh</emphasis>button
              periodically until the status of your request indicates it is
              <emphasis role="bold">Finished </emphasis>or click on the
              <emphasis role="bold">View Progress</emphasis> hyperlink to see
              a progress indicator.</para>

              <para><informaltable colsep="1" frame="all" rowsep="0">
                  <tgroup cols="2">
                    <colspec colwidth="52.60pt" />

                    <colspec colwidth="384.80pt" />

                    <tbody>
                      <row>
                        <entry><graphic fileref="images/tip.jpg" /></entry>

                        <entry>You can use the Choose File button to look up
                        files on the selected source machine.</entry>
                      </row>
                    </tbody>
                  </tgroup>
                </informaltable></para>
            </listitem>
          </itemizedlist>
        </sect3>

        <sect3 id="Spray_CSV">
          <title><emphasis role="bold">Spray CSV</emphasis></title>

          <itemizedlist>
            <listitem>
              <para>Click on the <emphasis role="bold">Spray CSV</emphasis>
              hyperlink below <emphasis role="bold">DFU </emphasis>in the menu
              on the left.</para>

              <para>The <emphasis role="bold">Spray CSV</emphasis> page
              displays.</para>
            </listitem>

            <listitem>
              <para>Fill in <emphasis role="bold">Source
              </emphasis>information (<emphasis role="bold">Machine, IP, File
              Path, and record information)</emphasis> and the <emphasis
              role="bold">Destination</emphasis> information (<emphasis
              role="bold">Group</emphasis> and <emphasis
              role="bold">Label</emphasis>).</para>

              <para><informaltable colsep="0" frame="none" rowsep="0">
                  <tgroup cols="2">
                    <colspec colwidth="122.40pt" />

                    <colspec colwidth="333.00pt" />

                    <tbody>
                      <row>
                        <entry align="right"><emphasis
                        role="bold">Source:</emphasis></entry>
                      </row>

                      <row>
                        <entry><emphasis
                        role="bold">Machine</emphasis></entry>

                        <entry>Use the drop-list to select the machine where
                        the source file is located.</entry>
                      </row>

                      <row>
                        <entry><emphasis role="bold">IP</emphasis></entry>

                        <entry>IP address of machine from which to spray. This
                        is automatically completed when you select the
                        <emphasis role="bold">Source
                        Machine.</emphasis></entry>
                      </row>

                      <row>
                        <entry><emphasis role="bold">Local
                        Path</emphasis></entry>

                        <entry>The file path of source file to spray.</entry>
                      </row>

                      <row>
                        <entry><emphasis role="bold">Max Record
                        Length</emphasis></entry>

                        <entry>The length of longest record in the
                        file.</entry>
                      </row>

                      <row>
                        <entry><emphasis
                        role="bold">Separator</emphasis></entry>

                        <entry>The character used as a separator in the source
                        file.</entry>
                      </row>

                      <row>
                        <entry><emphasis role="bold">Line
                        Terminator</emphasis></entry>

                        <entry>The character used as a line terminator in the
                        source file.</entry>
                      </row>

                      <row>
                        <entry><emphasis role="bold">Quote</emphasis></entry>

                        <entry>The character used as a quote in the source
                        file.</entry>
                      </row>

                      <row>
                        <entry align="right"><emphasis
                        role="bold">Destination:</emphasis></entry>
                      </row>

                      <row>
                        <entry><emphasis role="bold">Group</emphasis></entry>

                        <entry>Select the name of THOR cluster to spray
                        to.</entry>
                      </row>

                      <row>
                        <entry><emphasis role="bold">Label</emphasis></entry>

                        <entry>The logical name that you choose for the
                        file.</entry>
                      </row>

                      <row>
                        <entry align="right"><emphasis
                        role="bold">Options:</emphasis></entry>
                      </row>

                      <row>
                        <entry><emphasis
                        role="bold">Overwrite</emphasis></entry>

                        <entry>Check this box to overwrite files of the same
                        name.</entry>
                      </row>

                      <row>
                        <entry><emphasis
                        role="bold">Replicate</emphasis></entry>

                        <entry>Check this box to create backup copies of all
                        file parts exist in the backup directory (by
                        convention on the secondary drive of the node
                        following in the cluster).</entry>
                      </row>

                      <row>
                        <entry><emphasis
                        role="bold">Compress</emphasis></entry>

                        <entry>Check this box to compress the files.</entry>
                      </row>
                    </tbody>
                  </tgroup>
                </informaltable></para>
            </listitem>

            <listitem>
              <para>Press the <emphasis role="bold">Submit</emphasis>
              button.</para>

              <para>The <emphasis role="bold">DFU Workunit</emphasis>
              displays.</para>
            </listitem>

            <listitem>
              <?dbfo keep-together="always"?>

              <para>Press the <emphasis role="bold">Refresh</emphasis> button
              periodically until the status of your request indicates it is
              <emphasis role="bold">Finished</emphasis>or click on the
              <emphasis role="bold">View Progress</emphasis> hyperlink to see
              a progress indicator.<informaltable colsep="1" frame="all"
                  rowsep="0">
                  <tgroup cols="2">
                    <colspec colwidth="52.60pt" />

                    <colspec colwidth="384.80pt" />

                    <tbody>
                      <row>
                        <entry><graphic fileref="images/tip.jpg" /></entry>

                        <entry>You can use the Choose File button to look up
                        files on the selected source machine.</entry>
                      </row>
                    </tbody>
                  </tgroup>
                </informaltable></para>
            </listitem>
          </itemizedlist>

          <beginpage />
        </sect3>

        <sect3 id="Spray_XML">
          <title><emphasis role="bold">Spray XML</emphasis></title>

          <itemizedlist>
            <listitem>
              <para>Click on the <emphasis role="bold">Spray
              XML</emphasis>hyperlink below <emphasis
              role="bold">DFU</emphasis>in the menu on the left.</para>

              <para>The <emphasis role="bold">Spray XML</emphasis> page
              displays.</para>
            </listitem>

            <listitem>
              <para>Fill in <emphasis role="bold">Source
              </emphasis>information (<emphasis role="bold">Machine, IP, File
              Path, and record information)</emphasis>and the <emphasis
              role="bold">Destination</emphasis> information (<emphasis
              role="bold">Group</emphasis> and <emphasis role="bold">Label
              ).</emphasis></para>

              <para><informaltable colsep="0" frame="none" rowsep="0">
                  <tgroup cols="2">
                    <colspec colwidth="122.40pt" />

                    <colspec colwidth="333.00pt" />

                    <tbody>
                      <row>
                        <entry align="right"><emphasis
                        role="bold">Source:</emphasis></entry>
                      </row>

                      <row>
                        <entry><emphasis
                        role="bold">Machine</emphasis></entry>

                        <entry>Use the drop-list to select the machine where
                        the source file is located.</entry>
                      </row>

                      <row>
                        <entry><emphasis role="bold">IP</emphasis></entry>

                        <entry>IP address of machine from which to spray. This
                        is automatically completed when you select the
                        <emphasis role="bold">Source
                        Machine.</emphasis></entry>
                      </row>

                      <row>
                        <entry><emphasis role="bold">Local
                        Path</emphasis></entry>

                        <entry>The file path of source file to spray.</entry>
                      </row>

                      <row>
                        <entry><emphasis role="bold">Format</emphasis></entry>

                        <entry>Select the file format from the
                        drop-list.</entry>
                      </row>

                      <row>
                        <entry><emphasis role="bold">Max Record
                        Length</emphasis></entry>

                        <entry>The length of longest record in the
                        file.</entry>
                      </row>

                      <row>
                        <entry><emphasis role="bold">Row
                        Tag</emphasis></entry>

                        <entry>The record separator tag in the XML
                        file</entry>
                      </row>

                      <row>
                        <entry align="right"><emphasis
                        role="bold">Destination:</emphasis></entry>
                      </row>

                      <row>
                        <entry><emphasis role="bold">Group</emphasis></entry>

                        <entry>Select the name of THOR cluster to spray
                        to.</entry>
                      </row>

                      <row>
                        <entry><emphasis role="bold">Label</emphasis></entry>

                        <entry>The logical name that you choose for the
                        file.</entry>
                      </row>

                      <row>
                        <entry align="right"><emphasis
                        role="bold">Options:</emphasis></entry>
                      </row>

                      <row>
                        <entry><emphasis
                        role="bold">Overwrite</emphasis></entry>

                        <entry>Check this box to overwrite files of the same
                        name.</entry>
                      </row>

                      <row>
                        <entry><emphasis
                        role="bold">Replicate</emphasis></entry>

                        <entry>Check this box to create backup copies of all
                        file parts exist in the backup directory (by
                        convention on the d: drive of the node following in
                        the cluster).</entry>
                      </row>

                      <row>
                        <entry><emphasis
                        role="bold">Compress</emphasis></entry>

                        <entry>Check this box to compress the files.</entry>
                      </row>
                    </tbody>
                  </tgroup>
                </informaltable></para>
            </listitem>

            <listitem>
              <para>Press the <emphasis role="bold">Submit
              </emphasis>button.</para>

              <para>The <emphasis role="bold">DFU Workunit</emphasis>
              displays.</para>
            </listitem>

            <listitem>
              <para>Press the <emphasis role="bold">Refresh </emphasis>button
              periodically until the status of your request indicates it is
              <emphasis role="bold">Finished </emphasis>or click on the
              <emphasis role="bold">View Progress</emphasis> hyperlink to see
              a progress indicator.</para>

              <para><informaltable colsep="1" frame="all" rowsep="0">
                  <tgroup cols="2">
                    <colspec colwidth="52.60pt" />

                    <colspec colwidth="384.80pt" />

                    <tbody>
                      <row>
                        <entry><graphic fileref="images/tip.jpg" /></entry>

                        <entry>You can use the Choose File button to look up
                        files on the selected source machine.</entry>
                      </row>
                    </tbody>
                  </tgroup>
                </informaltable></para>
            </listitem>
          </itemizedlist>

          <beginpage />
        </sect3>

        <sect3 id="Copy">
          <title><emphasis role="bold">Copy</emphasis></title>

          <itemizedlist>
            <listitem>
              <para>Locate the file to copy in the list of files, then click
              on the arrow icon, then select <emphasis role="bold">Copy
              </emphasis>from the pop-up menu..</para>
            </listitem>

            <listitem>
              <para>Fill in <emphasis role="bold">Destination</emphasis> and
              <emphasis role="bold">Options </emphasis>information.</para>

              <informaltable colsep="0" frame="none" rowsep="0">
                <tgroup cols="2">
                  <colspec colwidth="122.40pt" />

                  <colspec colwidth="333.00pt" />

                  <tbody>
                    <row>
                      <entry align="right"><emphasis
                      role="bold">Destination:</emphasis></entry>
                    </row>

                    <row>
                      <entry><emphasis role="bold">Group</emphasis></entry>

                      <entry>Select the name of THOR cluster to copy
                      to.</entry>
                    </row>

                    <row>
                      <entry align="right"><emphasis
                      role="bold">Note</emphasis></entry>

                      <entry>You can only choose from THOR clusters within the
                      current environment.</entry>
                    </row>

                    <row>
                      <entry><emphasis role="bold">Logical
                      File</emphasis></entry>

                      <entry>The logical name for the copied file.</entry>
                    </row>

                    <row>
                      <entry><emphasis role="bold">File
                      Mask</emphasis></entry>

                      <entry>Automatically updated based on logical file name
                      entered.</entry>
                    </row>

                    <row>
                      <entry align="right"><emphasis
                      role="bold">Options:</emphasis></entry>
                    </row>

                    <row>
                      <entry><emphasis
                      role="bold">Replicate</emphasis></entry>

                      <entry>Check this box to create backup copies of all
                      file parts exist in the backup directory (by convention
                      on the second drive of the node following in the
                      cluster).</entry>
                    </row>

                    <row>
                      <entry><emphasis role="bold">Wrap</emphasis></entry>

                      <entry>Check this box to keep the number of parts the
                      same and wrap if the target cluster is smaller that the
                      original.</entry>
                    </row>

                    <row>
                      <entry><emphasis
                      role="bold">Overwrite</emphasis></entry>

                      <entry>Check this box to overwrite files of the same
                      name.</entry>
                    </row>

                    <row>
                      <entry><emphasis role="bold">Compress</emphasis></entry>

                      <entry>Check this box to compress the files.</entry>
                    </row>

                    <row>
                      <entry><emphasis role="bold">Retain Superfile
                      Structure</emphasis></entry>

                      <entry>Check this box to retain the superfile
                      structure.</entry>
                    </row>
                  </tbody>
                </tgroup>
              </informaltable>
            </listitem>

            <listitem>
              <para>Press the <emphasis role="bold">Submit
              </emphasis>button.</para>

              <para>The <emphasis role="bold">DFU Workunit
              </emphasis>displays.</para>
            </listitem>

            <listitem>
              <para>Press the <emphasis role="bold">Refresh </emphasis>button
              periodically until the status of your request indicates it is
              <emphasis role="bold">Finished </emphasis>or click on the
              <emphasis role="bold">View Progress</emphasis> hyperlink to see
              a progress indicator.</para>
            </listitem>
          </itemizedlist>

          <beginpage />
        </sect3>

        <sect3 id="Remote_Copy">
          <title><emphasis role="bold">Remote Copy</emphasis></title>

          <para>Remote Copy allows you to copy data from a Thor cluster
          outside your environment to the one in your environment.</para>

          <itemizedlist>
            <listitem>
              <para>Click on the <emphasis role="bold">Remote Copy
              </emphasis>hyperlink below <emphasis role="bold">DFU
              </emphasis>in the menu on the left.</para>

              <para>The <emphasis role="bold">Copy File </emphasis>page
              displays.</para>
            </listitem>

            <listitem>
              <para>Fill in <emphasis role="bold">Source,
              Destination,</emphasis> and <emphasis role="bold">Options
              </emphasis>information.</para>

              <informaltable colsep="0" frame="none" rowsep="0">
                <tgroup cols="2">
                  <colspec colwidth="122.40pt" />

                  <colspec colwidth="333.00pt" />

                  <tbody>
                    <row>
                      <entry align="right"><emphasis
                      role="bold">Source:</emphasis></entry>
                    </row>

                    <row>
                      <entry><emphasis role="bold">Logical
                      File</emphasis></entry>

                      <entry>The logical file name in the remote
                      environment.</entry>
                    </row>

                    <row>
                      <entry><emphasis role="bold">Source
                      Dali</emphasis></entry>

                      <entry>The Dali Server in the remote environment</entry>
                    </row>

                    <row>
                      <entry><emphasis role="bold">Source
                      Username</emphasis></entry>

                      <entry>A valid user in the remote environment</entry>
                    </row>

                    <row>
                      <entry><emphasis role="bold">Source
                      Password</emphasis></entry>

                      <entry>The password for the user in the remote
                      environment</entry>
                    </row>

                    <row>
                      <entry align="right"><emphasis
                      role="bold">Destination:</emphasis></entry>
                    </row>

                    <row>
                      <entry><emphasis role="bold">Group</emphasis></entry>

                      <entry>Select the name of THOR cluster to copy
                      to.</entry>
                    </row>

                    <row>
                      <entry align="right"><emphasis
                      role="bold">Note</emphasis></entry>

                      <entry>You can only choose from THOR clusters within the
                      current environment.</entry>
                    </row>

                    <row>
                      <entry><emphasis role="bold">Logical
                      Name</emphasis></entry>

                      <entry>The logical name for the copied file.</entry>
                    </row>

                    <row>
                      <entry align="right"><emphasis
                      role="bold">Options:</emphasis></entry>
                    </row>

                    <row>
                      <entry><emphasis
                      role="bold">Replicate</emphasis></entry>

                      <entry>Check this box to create backup copies of all
                      file parts exist in the backup directory (by convention
                      on the d: drive of the node following in the
                      cluster).</entry>
                    </row>

                    <row>
                      <entry><emphasis role="bold">Wrap</emphasis></entry>

                      <entry>Check this box to keep the number of parts the
                      same and wrap if the target cluster is smaller that the
                      original.</entry>
                    </row>

                    <row>
                      <entry><emphasis
                      role="bold">Overwrite</emphasis></entry>

                      <entry>Check this box to overwrite files of the same
                      name.</entry>
                    </row>

                    <row>
                      <entry><emphasis role="bold">Compress</emphasis></entry>

                      <entry>Check this box to compress the files.</entry>
                    </row>

                    <row>
                      <entry><emphasis role="bold">Retain Superfile
                      Structure</emphasis></entry>

                      <entry>Check this box to retain the superfile
                      structure.</entry>
                    </row>
                  </tbody>
                </tgroup>
              </informaltable>
            </listitem>

            <listitem>
              <para>Press the <emphasis role="bold">Submit
              </emphasis>button.</para>

              <para>The <emphasis role="bold">DFU Workunit
              </emphasis>displays.</para>
            </listitem>

            <listitem>
              <para>Press the <emphasis role="bold">Refresh </emphasis>button
              periodically until the status of your request indicates it is
              <emphasis role="bold">Finished </emphasis>or click on the
              <emphasis role="bold">View Progress</emphasis> hyperlink to see
              a progress indicator.</para>
            </listitem>
          </itemizedlist>

          <beginpage />
        </sect3>

        <sect3 id="Modifying_a-DFU-Workunit">
          <title><emphasis role="bold">Modifying a DFU
          Workunit</emphasis></title>

          <para>From the DFU Workunit page, you can modify and run any DFU
          Spray, Despray, Copy, or Remote Copy action using the modified
          settings. This allows you to run similar jobs without filling in
          similar details again. It also allows you to correct any errors that
          might have caused a DFU workunit to fail.</para>

          <para>From any DFU Workunit page:</para>

          <itemizedlist>
            <listitem>
              <para>Press the Modify button.</para>

              <para>The page for the original action displays.</para>
            </listitem>

            <listitem>
              <para>Modify the details, as needed.</para>
            </listitem>

            <listitem>
              <para>Press the <emphasis role="bold">Submit
              </emphasis>button.</para>

              <para>The <emphasis role="bold">DFU Workunit
              </emphasis>displays.</para>
            </listitem>

            <listitem>
              <para>Press the <emphasis role="bold">Refresh </emphasis>button
              periodically until the status of your request indicates it is
              <emphasis role="bold">Finished </emphasis>or click on the
              <emphasis role="bold">View Progress</emphasis> hyperlink to see
              a progress indicator.</para>
            </listitem>
          </itemizedlist>

          <para></para>

          <para></para>
        </sect3>
      </sect2>
    </sect1>
  </chapter>
</book>