|
@@ -870,61 +870,12 @@
|
|
<chapter id="Capacity_Planning_for_Roxie_Clusters">
|
|
<chapter id="Capacity_Planning_for_Roxie_Clusters">
|
|
<title>Capacity Planning for Roxie Clusters</title>
|
|
<title>Capacity Planning for Roxie Clusters</title>
|
|
|
|
|
|
- <sect1 id="Capacity_Planning" role="nobrk">
|
|
|
|
- <title>Capacity Planning</title>
|
|
|
|
-
|
|
|
|
- <para>Roxie clusters are disk-based High Performance Computing Clusters
|
|
|
|
- (HPCC) , typically using indexed files. A cluster is capable of storing
|
|
|
|
- and manipulating as much data as its combined hard drive space; however,
|
|
|
|
- this does not produce optimal performance.</para>
|
|
|
|
-
|
|
|
|
- <para>For maximum performance, you should configure your cluster so
|
|
|
|
- slave nodes perform most jobs in memory.</para>
|
|
|
|
-
|
|
|
|
- <para>For example, if a query uses three data files with a combined file
|
|
|
|
- size of 60 GB, a 40-channel cluster is a good size, while a 60-channel
|
|
|
|
- is probably better.</para>
|
|
|
|
-
|
|
|
|
- <para>Another consideration is the size of the Thor cluster creating the
|
|
|
|
- data files and index files to be loaded. Your target Roxie cluster
|
|
|
|
- should be the same size as the Thor on which the data and index files
|
|
|
|
- are created or a number evenly divisible by the size of your Roxie
|
|
|
|
- cluster. For example, a 100-way Thor to a 20-way Roxie would be
|
|
|
|
- acceptable.</para>
|
|
|
|
-
|
|
|
|
- <para>This is due to the manner in which data is loaded and processed by
|
|
|
|
- Roxie slaves. If data is copied to slave nodes, the file parts are
|
|
|
|
- directly copied from source location to target locations. They are NOT
|
|
|
|
- split or resized to fit a different sized cluster. Therefore, if you
|
|
|
|
- load 50 file parts onto a 40-channel cluster, part one goes to channel
|
|
|
|
- one, part two to channel two, etc. Parts 41-50 start at the top again so
|
|
|
|
- that part 41 goes to channel 1, and part 42 goes to channel 2, etc. The
|
|
|
|
- result is an unevenly distributed workload and would result in reduced
|
|
|
|
- performance. A cluster will only perform as fast as its slowest
|
|
|
|
- node.</para>
|
|
|
|
-
|
|
|
|
- <para>The final consideration is the number of Server processes in a
|
|
|
|
- cluster. Each slave must also be a Server, but you can dedicate
|
|
|
|
- additional nodes to be only Server processes. This is useful for queries
|
|
|
|
- that require processing on the Server after results are returned from
|
|
|
|
- slaves. Those Server-intensive queries could be sent only to dedicated
|
|
|
|
- Server IP addresses so the load is removed from nodes acting as both
|
|
|
|
- Server and slave.</para>
|
|
|
|
-
|
|
|
|
- <sect2 id="Configuring_the_Channels">
|
|
|
|
- <title>Configuring the Channels</title>
|
|
|
|
-
|
|
|
|
- <para>In the illustration below, the nodes are configured using an N+5
|
|
|
|
- scheme to share channels. Channels can be configured in many ways,
|
|
|
|
- this is one example.</para>
|
|
|
|
-
|
|
|
|
- <para><graphic fileref="images/RM-15.jpg" /></para>
|
|
|
|
-
|
|
|
|
- <para>In this depiction, each enclosure holds five Roxie slave blades
|
|
|
|
- (a row of servers in the picture). We will use this example for the
|
|
|
|
- rest of this manual.</para>
|
|
|
|
- </sect2>
|
|
|
|
- </sect1>
|
|
|
|
|
|
+
|
|
|
|
+ <xi:include href="RoxieReference/RoxieRefMods/RoxieCapacityPlanning.xml"
|
|
|
|
+ xpointer="Capacity_Planning"
|
|
|
|
+ xmlns:xi="http://www.w3.org/2001/XInclude" />
|
|
|
|
+
|
|
|
|
+
|
|
</chapter>
|
|
</chapter>
|
|
|
|
|
|
<chapter id="PreFlight_and_Roxie_Metrics">
|
|
<chapter id="PreFlight_and_Roxie_Metrics">
|