Boca Raton Documentation Team We welcome your comments and feedback about this document via email to docfeedback@hpccsystems.com Please include Documentation Feedback in the subject line and reference the document name, page numbers, and current Version Number in the text of the message. LexisNexis and the Knowledge Burst logo are registered trademarks of Reed Elsevier Properties Inc., used under license. Other products, logos, and services may be trademarks or registered trademarks of their respective companies. All names and example data used in this manual are fictitious. Any similarity to actual persons, living or dead, is purely coincidental. HPCC Systems Configuration Manager is the utility with which we configure the HPCC platform. The HPCC platform's configuration is stored in an XML file named environment.xml. When you install a package, a default single-node environment.xml is generated. After that, you can use the Configuration Manager to modify it and add nodes and configure components. The Configuration Manager Wizard creates a similar file, but after it is generated, you must rename it and put it into place on each node. Configuration Manager also offers an Advanced View which allows you to add instances of components or change the default settings for components. Even if you plane to use Advanced View, it is a good idea to start with a wizard generated configuration and use Advanced View to finish it. This document will guide you through configuring an HPCC environment using the Configuration Manager. This document will guide you through configuring an HPCC environment using the Configuration Manager. The HPCC packages should already be installed on ALL nodes. You can use any tool or shell script you choose. SSH to the first box in your environment and login as a user with sudo privileges. Start the Configuration Manager service on one node (usually the first node is considered the head node and is used for this task, but this is up to you). sudo /opt/HPCCSystems/sbin/configmgr Using a Web browser, go to the Configuration Manager's interface: http://<ip of installed system>:8015 The Configuration Manager startup wizard displays. There are a few different ways to configure your HPCC. You can use the Generate environment wizard or experienced users can use the Advanced View. To use the wizard select the Generate new environment using wizard button. Provide a name for the environment file. This will then be the name of the configuration xml. For example we will name our environment NewEnvironment. This will produce an XML file named NewEnvironment.xml. Press the Next button. Next you will need to define the IP addresses that your HPCC will be using. Enter the IP addresses. IP Addresses can be specified individually using semi-colon delimiters. You can also specify a range of IPs using a hyphen (for example, nnn.nnn.nnn.x-y). In the image below, we specified the IP addresses 10.239.219.1 through 10.239.219.100 using the range syntax, and also a single IP 10.239.219.111. Press the Next button. Alternatively, you could find the IP addresses using Auto Discovery by selecting the Auto Discovery button. Now you will define how many nodes to use for the Roxie and Thor clusters. Enter the appropriate values as indicated. Number of support nodes: Specify the number of nodes to use for support components. The default is 1. Number of nodes for Roxie cluster: Specify the number of nodes to use for your Roxie cluster. Enter zero (0) if you do not want a Roxie cluster. Number of slave nodes for Thor cluster Specify the number of slave nodes to use in your Thor cluster. A Thor master node will be added automatically. Number of Thor slaves per node (default 1) Specify the number of Thor slave processes to instantiate on each slave node. Enter zero (0) if you do not want a Thor cluster. Enable Roxie on demand Specify whether or not to allow queries to be run immediately on Roxie. This must be enabled to run the debugger. (Default is true) Press the Next button The wizard displays the configuration parameters. Press the Finish button to accept these values or press the Advanced View button to edit in advanced mode. You will now be notified that you have completed the wizard. At this point, you have created a file named NewEnvironment.xml in the /etc/HPCCSystems/source directory Keep in mind, that your HPPC configuration may be different depending on your needs. For example, you may not need a Roxie or you may need several smaller Roxie clusters. In addition, in a production [Thor] system, you would ensure that Thor and Roxie nodes are dedicated and have no other processes running on them. This document is intended to show you how to use the configuration tools. Capacity planning and system design is covered in a training module. Stop the HPCC Be sure HPCC is stopped before attempting to move the environment.xml file. Back up the original environment.xml file # for example sudo -u hpcc cp /etc/HPCCSystems/environment.xml /etc/HPCCSystems/environment.bak Note: the "live environment.xml file is located in your /etc/HPCCSystems/ directory. ConfigManager works on files in /etc/HPCCSystems/source directory. You must copy from this location to make an environment.xml file active. Copy the new .xml file from the source directory to the /etc/HPCCSystems and rename the file to environment.xml # for example sudo -u hpcc cp /etc/HPCCSystems/source/NewEnvironment.xml /etc/HPCCSystems/environment.xml Copy the /etc/HPCCSystems/environment.xml to the /etc/HPCCSystems/ on every node. You may want to use a script to push out XML file to all nodes. See the Example Scripts section in the Appendix of the Installing_and_RunningtheHPCCPlatform document. You can use the scripts as a model to create your own script to copy the environment.xml file out to all your nodes. Restart the HPCC platform. For the advanced user, the Advanced View offers access to adding additional instances of components or making configuration settings for individual components. This section shows some of the configuration options in Advanced Mode. SSH to the first box in your environment and login as a user with sudo privileges. If it is running, stop the HPCC system, using this command: sudo /sbin/service hpcc-init stop You can use this command to confirm HPCC processes are stopped: sudo /sbin/service hpcc-init status Start the Configuration Manager service on one node (usually the first node is considered the head node and is used for this task, but this is up to you). sudo /opt/HPCCSystems/sbin/configmgr Using a Web browser, go to the Configuration Manager's interface: http://<ip of installed system>:8015 The Configuration Manager startup wizard displays. There are a few different ways to configure your system. You can use the Generate environment wizard or experienced users can use the Advanced set up. Select Advanced View, then press the Next button. Select an XML file from the drop list. This list is populated from versions of an environment XML file in your server's ./etc/HPCCSystems/source/ directory. Press the Next button. The Configuration Manager interface displays. Default access is read-only. Many options are only available when write-access is enabled. Gain write access by checking the Write Access checkbox. Unchecking this box returns the environment to read-only mode. All menu items are disabled in read-only mode. Closing the web page automatically removes any write-access locks. Check the Write Access box. The Save button validates and saves the environment. The Save Environment As button validates and lets you specify the environment filename to save. The Validate Environment button just validates the current environment including any changes that have not yet been saved. The Open Environment button allows you to open a new environment file to work on. The Wizard button will bring up the Configuration Manager chooser form which will allow you to create or view an environment file where you can also launch the configuration wizard. These buttons are only enabled in Write Access mode. This section allows you to define your list of servers. When defining instances of components, you will choose from servers in this list. Select Hardware in the Navigator panel on the left side. Select the Computers tab. RT-CLICK on one of computers listed, then select New Range. Specify the following: Name Prefix - any name that will help you to identify the node or range Start IP Address Stop IP Address The IP Addresses can be specified in a range if all your host IP addresses are consecutively numbered. If the IP addresses are not sequential you should repeat the process for each individual IP address and just add the IP address in both the start and stop IP address field. You will then need to repeat the process for each node. Press the OK button. The list of nodes now displays with the nodes that you just added. Next, edit each System Server component instance and set it to a newly defined node. Click the disk icon to save Expand the Software section, if necessary, in the Navigator panel on the left side, by clicking on the button. Use the software components section to configure software components of the HPCC platform. Most software components are actual running processes; however, some are just definitions used by the system. These definitions are used by the configuration generator. Items that appear in red indicate optional values. They are only written to the environment if you add to or change that value. If untouched, they will not appear in the environment XML file. Select Dali Server in the Navigator panel on the left side. Select the Instances tab. In the computer column, choose a node from the drop list as shown below: Click the disk icon to save This section describes the DaliServer attributes. Attribute Definition AuditLogDir Audit Log Process description dataPath the path to the data description data path description environment the current environment file in use LogDir location of the log files msgLevel description of the msgLevel name Name of the process instance (AlphaNumeric and underscore only) This section describes the attributes configuring how Dali handles the system data store. Attribute name Definition EndTime End Time IdlePeriod Period to remain Idle IdleRate IdleRate keepStores lightweightCoalesce lightweight MinTime min. time. recoverDeltaErrors True or False StartTime start time This section describes the DaliServer LDAP tab. Attribute name Definition authMethod Authentication Method. Kerberos checkScopeScans True or False filesBasedn The Base File Type filesDefaultPassword Default Password filesDefaultUser Default User ldapProtocol Default Protocol use ldapServer LDAP Server This tab allows you to add any notes pertinent to the component's configuration. This can be useful to keep a record of changes and to communicate this information to peers. Dafilesrv is a helper process that every node needs. Select Dafilesrv in the Navigator panel on the left side. Select the Instances tab. RT-CLICK on a computer in the computer column, and select Add Instance . Select all computers in the list by checking the Select All box, then press the OK button. Click the disk icon to save This section describes the Dafilesrv attributes. Attribute Definition description DaFileSrv Process description name Name of the process instance (AlphaNumeric and underscore only) version current version number Select DFU Server in the Navigator panel on the left side. Select the Instances tab. In the computer column, choose a node from the drop list as shown below: Click the disk icon to save This section describes the DfuServer attributes. Attribute name Definition daliServers The Dali servers to use description Describes the component dfuLogDir The log directory monitorinterval Specifies the polling interval for DFU monitoring (in seconds). Default is 900 (15 minutes) monitorque the queue to monitor name Name of the process instance (AlphaNumeric and underscore only) queue the name of the dfu server queue transferBufferSize The value at which... This section describes the DfuServer SSH Options.. Attribute name Definition SSHidentityfile The SSH file used for authentication SSHpassword the SSH Password SSHretries no. of retries to authenticate SHtimeout the timeout interval SSHusername SSH username default is hpcc This tab allows you to add any notes pertinent to the component's configuration. This can be useful to keep a record of changes and to communicate this information to peers. The DIrectories component is a global definition used by other components to determine the directories they will use for various functions. Name Directory Description log /var/log/[NAME]/[INST] Location for Log files temp /var/lib/[NAME]/[INST]/temp Location for temp files data /var/lib/[NAME]/hpcc-data/[COMPONENT] Base Location for data files data2 /var/lib/[NAME]/hpcc-data2/[COMPONENT] additional location for data files data3 /var/lib/[NAME]/hpcc-data3/[COMPONENT] additional location for data files mirror /var/lib/[NAME]/hpcc-mirror/[COMPONENT] Base Location for mirror data files query /var/lib/[NAME]/queries/[INST] Base Location for Queries Select Drop Zone in the Navigator panel on the left side. Select the Attributes tab. In the Value column of the Computer row, choose a node from the drop list as shown below: Click the disk icon to save This tab allows you to add any notes pertinent to the component's configuration. This can be useful to keep a record of changes and to communicate this information to peers. Select ECL Agent in the Navigator panel on the left side. Select the Instances tab. In the computer column, choose a node from the drop list as shown below: Click the disk icon to save This section describes the EclAgent Attributes tab. Attribute name Definition description Describes the component name Name of the process instance (AlphaNumeric and underscore only) This section describes the EclAgent Options tab. Attribute name Definition allowedPipePrograms describes allowed Pipe Programs daliServers the Dali Servers pluginDirectory the plugin directory thorConnectTimeout the Thor timeout value traceLevel the trace level wuQueueName the name of the workunit queue This tab allows you to add any notes pertinent to the component's configuration. This can be useful to keep a record of changes and to communicate this information to peers. Select Ecl CC Server - myeclccserver in the Navigator panel on the left side. Select the Instances tab. In the computer column, choose a node from the drop list as shown below: Click the disk icon to save This section describes the Ecl CC Server Attributes tab. Attribute name Definition daliServers The Dali servers to use description Describes the component maxEclccProcesses The maximum number of eclcc processes that will be launched in parallel name Name of the process instance (AlphaNumeric and underscore only) queue Name of the Ecl cc server queue traceLevel The value *** To add a custom option, rt-click and select add. This tab allows you to add any notes pertinent to the component's configuration. This can be useful to keep a record of changes and to communicate this information to peers. Select ECL Scheduler in the Navigator panel on the left side. Select the Instances tab. In the computer column, choose a node from the drop list as shown below: Click the disk icon to save This section describes the EclScheduler Attributes tab. Attribute name Definition description Describes the component name Name of the process instance (AlphaNumeric and underscore only) This tab allows you to add any notes pertinent to the component's configuration. This can be useful to keep a record of changes and to communicate this information to peers. Select ESP - MyEsp in the Navigator panel on the left side. Select the Instances tab. In the computer column, choose a node from the drop list as shown below: Click the disk icon to save This section describes the Esp - myesp Attributes tab. Attribute name Definition componentfilesDir the component files directory daliServers the Dali servers description Describes the component enableSEHMapping True or False to enable*** formOptionsAccess True or False to enable...*** httpConfigAccess True or False to enable...*** logLevel The Level of ...*** logRequests True or False to enable...*** logResponses True or False to enable...*** maxBacklogQueueSize The size of the backlog queue maxConcurrentThreads The maximum number of concurrent threads maxRequestEntityLegnth The maximum length for entity requests name Name of the process instance (AlphaNumeric and underscore only) perfReportDelay The value at which...*** This section describes the Esp - myesp Service Bindings tab. Attribute name Definition ECLWatch provides ECL Watch interface Ws_ecl Provides Web access to published queries myecldirect Provides Ecl direct access This section describes the Esp - myesp Service Authentication tab. LDAP is a feature only available in Enterprise Edition. Attribute name Definition ldapAuthMethod the LDAP Authentication Method ldapConnections the number of LDAP Connections ldapServer the LDAP server method the method... passwordExpirationWarningDays the number of days advance warning of password expiration This section describes the Esp - myesp HTTPS tab. Attribute name Definition acceptSelfSigned True or False to enable self signed certificate... CA_Certificates_Path the location of the CA Certificates certificateFileName name of the CA Certificate Name of the city name of the city. Name of the country name of the country. Number of days the certificate is valid. The Number of days the certificate is valid. enableVerification True or False to enable verification of... organization name of the organization... organizationalUnit OU to use passphrase passphrase to use privateKeyFileName The privatekey name file (such as, privatekey.cer) regenerateCredentials True or False to enable generation of credentials. requireAddressMatch True or False to enable address match Name of the State name of the state TrustedPeers The names of trusted peers. This tab allows you to add any notes pertinent to the component's configuration. This can be useful to keep a record of changes and to communicate this information to peers. FTSlave is a helper process that every node needs. Select FTSlave in the Navigator panel on the left side. Select the Instances tab. RT-CLICK on a computer in the computer column, and select Add Instance . Select all computers in the list, then press the OK button. Click the disk icon to save This section describes the FTSlave attributes tab. Attribute Definition description FTSlave Process description name Name of the process instance (AlphaNumeric and underscore only) version current version number This tab allows you to add any notes pertinent to the component's configuration. This can be useful to keep a record of changes and to communicate this information to peers. Select Sasha Server in the menu on the left side. Select the Instances tab. In the computer column, choose a node from the drop list as shown below: This section describes the SashaServer Process Archiver tab. Attribute Definition cachedWUat SashaServer Archiver Process description cachedWUinterval SashaServer Archiver Process description cachedWUlimit SashaServer Archiver Process description DFUrecoveryAt SashaServer Archiver Process description DFUrecoveryCutoff SashaServer Archiver Process description DFUrecoveryInterval SashaServer Archiver Process description DFUrecoveryLimit SashaServer Archiver Process description DFUWUat SashaServer Archiver Process description DFUWUcutoff SashaServer Archiver Process description DFUWUduration SashaServer Archiver Process description DFUWUinterval SashaServer Archiver Process description DFUWUlimit SashaServer Archiver Process description DFUWUthrottle SashaServer Archiver Process description keepResultFiles Keep result files, True or False WUat SashaServer Archiver Process description WUbackup SashaServer Archiver Process description WUcutof SashaServer Archiver Process description WUduration SashaServer Archiver Process description WUinterval SashaServer Archiver Process description WUlimit SashaServer Archiver Process description WUretryinterval SashaServer Archiver Process description WUthrottle SashaServer Archiver Process description This section describes the SashaServer Process Coalescer tab. Attribute Definition coalesceAt SashaServer Coalesce Process description coalesceInterval SashaServer Coalesce Process description minDeltaSize SashaServer Coalesce Process description recoverDeltaErrors SashaServer Coalesce Process description This section describes the SashaServer Process DfuXref tab. Attribute Definition memoryLimit set value for Memory Limit xrefAt SashaServer DfuXRef Attribute description xrefCutoff SashaServer DfuXRef Attribute description. xrefEclWatchProvider Xref Ecl Watch Provider, True or False xrefInterval value of SashaServer DfuXRef Attribute xrefList SashaServer DfuXRef Attribute description xrefMaxScanThreads SashaServer DfuXRef Attribute description This section describes the SashaServer Process DfuExpiry tab. Attribute Definition ExpiryAt SashaServer DfuExpiry tab description ExpiryInterval Interval in hours at which the Dfu expires This section describes the SashaServer Process ThorQMon tab. Attribute Definition ThorQMonInterval ThorQMonInterval description ThorQMonQueues ThorQMonQueues description ThorQMonSwitchMinTime ThorQMonSwitchMinTime description This section describes the SashaServer Process DaFileSrvMonitor tab. Attribute Definition dafsmonAt dafsmon description dafsmonInterval dafsmon description dafsmonList dafsmon description This tab allows you to add any notes pertinent to the component's configuration. This can be useful to keep a record of changes and to communicate this information to peers. This section details how to define a Data Refinery (Thor) cluster. Before you begin, you should decide the width of the cluster (i.e., how many slave nodes will you have). Select Thor Cluster - mythor in the Navigator panel on the left side. Select the Topology tab. Expand the Topology, if needed, then RT-CLICK the Master and select Delete. This deletes the sample one-node Thor. You will replace this with a multi-node cluster. RT-CLICK on the Topology and select Add Master. Select a computer from the list, then press the OK button. RT-CLICK on the Master and select Add Slaves. Select the computers to use as slaves from the list, then press the OK button. Use CTRL+CLICK to multi-select or SHIFT+CLICK to select a range. The Nodes now display below the Thor Master node. Select Thor Cluster - mythor in the Navigator panel on the left side. Select the Attributes tab. Change the value of the localThor to false Click the disk icon to save The Topology tab allows you to to change aspects of your Thor Cluster. CLICK on your Thor cluster to expand the list and display your Thor nodes. RT-CLICK on the Master node to perform operations on the Master Node. If you want to designate a different node as the Thor master when setting up a multi-node system, follow these steps. Select Thor Cluster - mythor in the Navigator panel on the left side. Select the Topology tab. RT-CLICK on the Master node Select the Replace Master option. You should only use this feature when initially setting up your system. If there is data on the nodes when attempting to Swap Master, you run the risk of losing or corrupting some data. This section describes the Thor Cluster Attributes tab. Attribute Definition allowedPipePrograms ThorCluster Process description autoCopyBackup True or False checkPointRecovery True or False daliServers The Dali Server defaultOutputNodeGroup description ThorCluster Process description externalProgDir Location of externals globalMemorySize ThorCluster Process description idleRestartPeriod ThorCluster Process description largeMemSize ThorCluster Process description LCR True or False localThor True or False localThorPortBase localThorPortInc masterport monitorDaliFileServer Specifies whether to monitor for DaFileSrv process (True or False ) multiSlaves True or False multiThorExclusionLockName multiThorMemoryThreshold multiThorPriorityLock True or False name Name of cluster nodeGroup pluginsPath Location of plugins queueName replicateAsync True or False replicateOutputs True or False slaveDownTimeout slaveport smallSortThreshold verifyDaliConnectionInterval watchdogEnabled True or False watchdogProgressEnabled True or False watchdogProgressInterval ThorCluster Process description This section describes the ThorCluster SSH Options tab. Attribute Definition SSHidentityfile Location of the SSH keys component files. SSHpassword SSHretries The number of retries SSHtimeout Timeout SSHusername The user name of the account the SSH keys are associated with The debug tab is for internal use only This section describes the ThorCluster Swap Node tab. Attribute Definition AutoSwapNode True or False. Set to True to enable AutoSwapNode CheckAfterEveryJob True or False. Set to True to Check after Every Job SwapNodeCheckC True or False. Set to True if... SwapNodeCheckD True or False. Set to True if... SwapNodeCheckScript Swap Node Check Script... SwapNodeCheckScriptTimeout The number ... SwapNodeInterval The number ... SwapNodeMaxConcurrent value of Maximum number of concurrent... SwapNodeRestartJob True or False. Set to True... This section describes the ThorCluster Backup tab. Attribute Definition breakoutLimit value of... refreshRate value of... This tab allows you to add any notes pertinent to the component's configuration. This can be useful to keep a record of changes and to communicate this information to peers. This section details how to define a Rapid Data Delivery Engine (Roxie) cluster. Before you begin, you should decide the width of the cluster (i.e., how many agent nodes will you have).***Change the value of the localslave to false*** Select Roxie Cluster in the Navigator panel on the left side. Select the Servers tab. Expand the RoxieCluster >> farm1, if needed, then RT-CLICK the Server and select Delete. This deletes the sample one-node Roxie Server. You will replace this with several servers to act as Roxie Servers. RT-CLICK on the Roxie Cluster and select Add Farm. Select the computers to use as Servers from the list, then press the OK button. Use Ctrl+click to multi-select or shift+click to select a range. Select the Agents tab. Select the RoxieCluster, then RT-CLICK select Reconfigure Agents. This allows you to replace the single agent with the multiple nodes you added as Servers. Select the computers to use as Agents from the list, then press the OK button. Use Ctrl+click to multi-select or shift+click to select a range. Note: In this example, we are using the same five nodes that were made to be Servers. This is a typical configuration. Select the redundancy scheme to use. Typically, this is Circular Redundancy, as shown below. Click the disk icon to save Close Configuration Manager by pressing ctrl+C in the command window where it is running. This section describes the topology tab. Attribute name Definition Topology describes the system topology Cluster - thor describes the Thor clusters Cluster - hthor describes the hthor clusters Cluster - roxie describes the Roxie clusters Once your environment is set up as desired, you must copy the configuration file out to the other nodes. If it is running, stop the system Be sure system is stopped before attempting to move the Environment.xml file. Back up the original environment.xml file # for example sudo -u hpcc cp /etc/HPCCSystems/environment.xml /etc/HPCCSystems/environment.bak Note: the "live environment.xml file is located in your /etc/HPCCSystems/ directory. ConfigManager works on files in /etc/HPCCSystems/source directory. You must copy from this location to make an environment.xml file active. Copy the NewEnvironment.xml file from the source directory to the /etc/HPCCSystems and rename the file to environment.xml # for example sudo -u hpcc cp /etc/HPCCSystems/source/NewEnvironment.xml /etc/HPCCSystems/environment.xml Copy the /etc/HPCCSystems/environment.xml to the /etc/HPCCSystems/ on every node. Restart the HPCC system You might prefer to script this process, especially if you have many nodes. See the Example Scripts section in the Appendix of the Installing_and_RunningtheHPCCPlatform document. You can use the scripts as a model to create your own script to copy the environment.xml file out to all your nodes.