Greg Panagiotatos 2010-12-17 StringLib.StringCleanSpaces(source)UnicodeLib.UnicodeCleanSpaces(source) source A string containing the data to clean. Return: CleanSpaces returns either a STRING or UNICODE value, as appropriate. All variations of the CleanSpaces function return the source string with all instances of multiple adjacent space characters (2 or more spaces together, or a tab character) reduced to a single space character. It also trims off all leading and trailing spaces. Example: A := StringLib.StringCleanSpaces('ABCDE ABCDE'); //A contains 'ABCDE ABCDE' UNICODE C := UnicodeLib.UnicodeCleanSpaces(U'ABCDE ABCDE'); //C contains U'ABCDE ABCDE' UnicodeLib.UnicodeCompareAtStrength(source1, source2, strength)UnicodeLib.UnicodeLocaleCompareAtStrength(source1,source2,locale,strength) source1 A string containing the data to compare. source2 A string containing the data to compare. strength An integer value indicating how to compare. Valid values are: 1 ignores accents and case, differentiating only between letters. 2 ignores case but differentiates between accents. 3 differentiates between accents and case but ignores e.g. differences between Hiragana and Katakana 4 differentiates between accents and case and e.g. Hiragana/Katakana, but ignores e.g. Hebrew cantellation marks 5 differentiates between all strings whose canonically decomposed forms (NFD—Normalization Form D) are non-identical locale A null-terminated string containing the language and country code to use to determine correct sort order and other operations. Return: CompareAtStrength returns an INTEGER value. The CompareAtStrength functions return zero (0) if the source1 and source2 strings contain the same data, ignoring any differences in the case of the letters. These functions return negative one (-1) if source1 < source2 or positive one (1) if source1 > source2. Example: base := u'caf\u00E9'; // U+00E9 is lowercase e with acute prim := u'coffee shop'; // 1st difference, different letters seco := u'cafe'; // 2nd difference, accents (no acute) tert := u'Caf\u00C9'; // 3rd, caps (U+00C9 is u/c E + acute) A := unicodeLib.UnicodeCompareAtStrength(base, prim, 1) != 0; // base and prim differ at all strengths A := unicodeLib.UnicodeCompareAtStrength(base, seco, 1) = 0; // base and seco same at strength 1 (differ only at strength 2) A := unicodeLib.UnicodeCompareAtStrength(base, tert, 1) = 0; // base and tert same at strength 1 (differ only at strength 3) A := unicodeLib.UnicodeCompareAtStrength(base, seco, 2) != 0; // base and seco differ at strength 2 A := unicodeLib.UnicodeCompareAtStrength(base, tert, 2) = 0; // base and tert same at strength 2 (differ only at strength 3) A := unicodeLib.UnicodeCompareAtStrength(base, seco, 3) != 0; // base and seco differ at strength 2 A := unicodeLib.UnicodeCompareAtStrength(base, tert, 3) != 0; // base and tert differ at strength 3 StringLib.StringCompareIgnoreCase(source1,source2)UnicodeLib.UnicodeCompareIgnoreCase(source1,source2)UnicodeLib.UnicodeLocaleCompareIgnoreCase(source1,source2, locale) source1 A string containing the data to compare. source2 A string containing the data to compare. locale A null-terminated string containing the language and country code to use to determine correct sort order and other operations. Return: CompareIgnoreCase returns an INTEGER value. The CompareIgnoreCase functions return zero (0) if the source1 and source2 strings contain the same data, ignoring any differences in the case of the letters. These functions return negative one (-1) if source1 < source2 or positive one (1) if source1 > source2. Example: A := StringLib.StringCompareIgnoreCase('ABCDE','abcde'); //A contains 0 -- they “match” B := StringLib.StringCompareIgnoreCase('ABCDE','edcba'); //B contains -1 -- they do not “match” StringLib.StringContains(source, pattern, nocase)UnicodeLib.UnicodeContains(source, pattern, nocase) source A string containing the data to search. pattern A string containing the characters to compare. An empty string ( ‘’ ) always returns true. nocase A boolean true or false indicating whether to ignore the case. Return: Contains returns a BOOLEAN value. The Contains functions return true if all the characters in the pattern appear in the source, otherwise they return false. Example: A := stringlib.stringContains( 'the quick brown fox jumps over the lazy dog', 'ABCdefghijklmnopqrstuvwxyz', true); B:= stringlib.stringContains( 'the speedy ochre vixen leapt over the indolent retriever', 'abcdefghijklmnopqrstuvwxyz', false); StringLib.Data2String(source) source A DATA string to convert. Return: Data2String returns a STRING value. The Data2String function returns a string containing the hexadeciomal characters from the source DATA value. Example: DATA2 CrLf := x’0A0D’; //2-bytes of hex STRING4 CrLfString := StringLib.Data2String(CrLf); //results in a ‘0A0D’ 4-byte string StringLib.EditDistance(string1, string2) string1 The first of a pair of strings to compare. string2 The second of a pair of strings to compare. Return: EditDistance returns an UNSIGNED4 value. The EditDistance function returns a standard Levenshtein distance algorithm score for the edit ditance between string1 and string2. This score i\reflects the minimum number of operations needed to transform string1 into string2. . Example: StringLib.EditDistance('CAT','CAT'); //returns 0 StringLib.EditDistance('CAT','BAT'); //returns 1 StringLib.EditDistance('BAT','BAIT'); //returns 1 StringLib.EditDistance('CAT','BAIT'); //returns 2 StringLib.StringExtract(source, instance)UnicodeLib.UnicodeExtract(source, instance) source A string containing a comma-delimited list of data. instance An integer specifying the ordinal position of the data item within the source to return. Return: Extract returns either a STRING or UNICODE value, as appropriate. The Extract function returns the data at the ordinal position specified by the instance within the comma-delimited source string. Example: //all these examples result in 'Success' A := IF(StringLib.StringExtract('AB,CD,,G,E',0) = '', 'Success', 'Failure -1'); B := IF(StringLib.StringExtract('AB,CD,,G,E',1) = 'AB', 'Success', 'Failure -2'); C := IF(StringLib.StringExtract('AB,CD,,G,E',2) = 'CD', 'Success', 'Failure -3'); D := IF(StringLib.StringExtract('AB,CD,,G,E',3) = '', 'Success', 'Failure -4'); E := IF(StringLib.StringExtract('AB,CD,,G,E',4) = 'G', 'Success', 'Failure -5'); F := IF(StringLib.StringExtract('AB,CD,,G,E',5) = 'E', 'Success', 'Failure -6'); G := IF(StringLib.StringExtract('AB,CD,,G,E',6) = '', 'Success', 'Failure -7'); StringLib.StringFilter(source, filterstring)UnicodeLib.UnicodeFilter(source, filterstring) source A string containing the data to filter. filterstring A string containing the characters to use as the filter. Return: Filter returns a STRING or UNICODE value, as appropriate. The StringFilter functions return the source string with all the characters except those in the filterstring removed. Example: //all these examples result in 'Success' A := IF(StringLib.StringFilter('ADCBE', 'BD') = 'DB', 'Success', 'Failure - 1'); B := IF(StringLib.StringFilter('ADCBEREBD', 'BDG') = 'DBBD', 'Success', 'Failure - 2'); C := IF(StringLib.StringFilter('ADCBE', '') = '', 'Success', 'Failure - 3'); D := IF(StringLib.StringFilter('', 'BD') = '', 'Success', 'Failure - 4'); E := IF(StringLib.StringFilter('ABCDE', 'EDCBA') = 'ABCDE', 'Success', 'Failure - 5'); StringLib.StringFilterOut(source, filterstring)UnicodeLib.UnicodeFilterOut(source, filterstring) source A string containing the data to filter. filterstring A string containing the characters to use as the filter. Return: FilterOut returns a STRING or UNICODE value, as appropriate. The FilterOut functions return the source string with all the characters in the filterstring removed. Example: //all these examples result in 'Success' A := IF(StringLib.StringFilterOut('ABCDE', 'BD') = 'ACE', 'Success', 'Failure - 1'); B := IF(StringLib.StringFilterOut('ABCDEABCDE', 'BD') = 'ACEACE', 'Success', 'Failure - 2'); C := IF(StringLib.StringFilterOut('ABCDEABCDE', '') = 'ABCDEABCDE', 'Success', 'Failure - 3'); D := IF(StringLib.StringFilterOut('', 'BD') = '', 'Success', 'Failure - 4'); StringLib.StringFind(source, target, instance)StringLib.EbcdicStringFind(source, target, instance)UnicodeLib.UnicodeFind(source, target, instance)UnicodeLib.UnicodeLocaleFind(source, target, instance, locale) source A string containing the data to search. target A string containing the substring to search for. instance An integer specifying which occurrence of the target to find. locale A null-terminated string containing the language and country code to use to determine correct sort order and other operations. Return: Find returns an INTEGER value. The Find functions return the beginning index position within the source string of the specified instance of the target string. If the target is not found or the specified instance is greater than the number of occurrences of the target in the source, StringFind returns zero (0). Example: A := IF(StringLib.StringFind('ABCDE', 'BC',1) = 2, 'Success', 'Failure - 1'); //success B := IF(StringLib.StringFind('ABCDEABCDE', 'BC', 2) = 7, 'Success', 'Failure - 2'); //success C := IF(StringLib.StringFind('ABCDEABCDE', '') = 0, 'Success', 'Failure - 3'); //syntax error, missing 3rd parameter D := IF(StringLib.StringFind('', 'BD', 1) = 0, 'Success', 'Failure - 4'); //success StringLib.StringUnboundedUnsafeFind(source, target)StringLib.EbcdicStringUnboundedUnsafeFind(source, target) source A string containing the data to search. target A string containing the substring to search for. Return: Find2 returns an INTEGER value. The StringUnboundedUnsafeFind functions return the beginning index position within the source string of the first instance of the target string. If the target string is not in the source string (or somewhere beyond), it creates a runtime exception. The StringUnboundedUnsafeFind functions scan the source string AND BEYOND until it finds what it is looking for. It is designed for use in variable length string input routines. Essentially you can pass it a source that is of length one and it will keep reading beyond the end of the source to find the target string and will tell you how far it had to go. Typically, this be used would search for an end-of-string delimiter to determine how many bytes of variable-length data there are. This can cause problems if the target string does not exist. Example: A := IF(StringLib.StringUnboundedUnsafeFind('ABCDE', 'BC') = 2, 'Success', 'Failure - 1'); //success B := IF(StringLib.StringFind2('ABCDEABCDE', 'BC') = 7, 'Success', 'Failure - 2'); //failure C := IF(StringLib.StringFind2('ABCDEABCDE', '') = 0, 'Success', 'Failure - 3'); //failure D := IF(StringLib.StringFind2('', 'BD') = 0, 'Success', 'Failure - 4'); //runtime exception StringLib.StringFindCount(source, target) source A string containing the data to search. target A string containing the substring to search for. Return: StringFindCount returns an INTEGER value. The FindCount function returns the number of non-overlapping instances of the target string within the source string. Example: A := IF(StringLib.StringFindCount('ABCDE', 'BC') = 1, 'Success', 'Failure - 1'); //success B := IF(StringLib.StringFindCount('ABCDEABCDE', 'BC') = 2, 'Success', 'Failure - 2'); //failure UnicodeLib.UnicodeLocaleFindAtStrength(source,target,instance,locale,strength) source A string containing the data to search. target A string containing the substring to search for. instance An integer specifying which occurrence of the target to find. locale A null-terminated string containing the language and country code to use to determine correct sort order and other operations. strength An integer value indicating how to compare. Valid values are: 1 ignores accents and case, differentiating only between letters 2 ignores case but differentiates between accents. 3 differentiates between accents and case but ignores e.g. differences between Hiragana and Katakana 4 differentiates between accents and case and e.g. Hiragana/Katakana, but ignores e.g. Hebrew cantellation marks 5 differentiates between all strings whose canonically decomposed forms (NFD—Normalization Form D) are non-identical Return: FindAtStrength returns an INTEGER value. The FindAtStrength function returns the beginning index position within the source string of the specified instance of the target string. If the target is not found or the specified instance is greater than the number of occurrences of the target in the source, StringFind returns zero (0). Example: base := u'caf\u00E9'; // U+00E9 is lowercase e with acute prim := u'coffee shop'; // 1st difference, different letters seco := u'cafe'; // 2nd difference, accents (no acute) tert := u'Caf\u00C9'; // 3rd, caps (U+00C9 is u/c E + acute) search := seco + tert + base; unicodelib.UnicodeLocaleFindAtStrength(search, base, 1, 'fr', 1) = 1; // at strength 1, base matches seco (only secondary diffs) unicodelib.UnicodeLocaleFindAtStrength(search, base, 1, 'fr', 2) = 5; // at strength 2, base matches tert (only tertiary diffs) unicodelib.UnicodeLocaleFindAtStrength(search, base, 1, 'fr', 3) = 9; // at strength 3, base doesn't match either seco or tert unicodelib.UnicodeLocaleFindAtStrength(u'le caf\u00E9 vert', u'cafe', 1, 'fr', 2) = 4; // however, an accent on the source, unicodelib.UnicodeLocaleFindAtStrength(u'le caf\u00E9 vert', u'cafe', 1, 'fr', 3) = 4; // rather than on the pattern, unicodelib.UnicodeLocaleFindAtStrength(u'le caf\u00E9 vert', u'cafe', 1, 'fr', 4) = 4; // is ignored at strengths up to 4, unicodelib.UnicodeLocaleFindAtStrength(u'le caf\u00E9 vert', u'cafe', 1, 'fr', 5) = 0; // and only counts at strength 5 UnicodeLib.UnicodeLocaleFindAtStrengthReplace(source, target, replacement, locale, strength ) source A string containing the data to search. target A string containing the substring to search for. replacement A string containing the replacement data. locale A null-terminated string containing the language and country code to use to determine correct sort order and other operations. strength An integer value indicating how to compare. Valid values are: 1 ignores accents and case, differentiating only between letters. 2 ignores case but differentiates between accents. 3 differentiates between accents and case but ignores e.g. differences between Hiragana and Katakana 4 differentiates between accents and case and e.g. Hiragana/Katakana, but ignores e.g. Hebrew cantellation marks 5 differentiates between all strings whose canonically decomposed forms (NFD—Normalization Form D) are non-identical Return: FindAtStrengthReplace returns a UNICODE value. The FindAtStrengthReplace functions return the source string with the replacement string substituted for all instances of the target string. If the target string is not in the source string, it returns the source string unaltered. Example: unicodelib.UnicodeLocaleFindAtStrengthReplace(u'e\u00E8E\u00C9eE', u'e\u00E9', u'xyz', 'fr', 1) = u'xyzxyzxyz'; unicodelib.UnicodeLocaleFindAtStrengthReplace(u'e\u00E8E\u00C9eE', u'e\u00E9', u'xyz', 'fr', 2) = u'e\u00E8xyzeE'; unicodelib.UnicodeLocaleFindAtStrengthReplace(u'e\u00E8E\u00C9eE', u'e\u00E9', u'xyz', 'fr', 3) = u'e\u00E8E\u00C9eE'; StringLib.StringFindReplace(source, target, replacement)UnicodeLib.UnicodeFindReplace(source, target, replacement)UnicodeLib.UnicodeLocaleFindReplace(source, target, replacement, locale ) source A string containing the data to search. target A string containing the substring to search for. replacement A string containing the replacement data. locale A null-terminated string containing the language and country code to use to determine correct sort order and other operations. Return: FindReplace returns a STRING or UNICODE value, as appropriate. The FindReplace functions return the source string with the replacement string substituted for all instances of the target string . If the target string is not in the source string, it returns the source string unaltered. Example: A := StringLib.StringFindReplace('ABCDEABCDE', 'BC','XY'); //A contains ‘AXYDEAXYDE’ A := unicodelib.UnicodeFindReplace(u'abcde', u'a', u'AAAAA'); //A contains u'AAAAAbcde' A := unicodelib.UnicodeFindReplace(u'aaaaa', u'aa', u'b'); //A contains u'bba' A := unicodelib.UnicodeFindReplace(u'aaaaaa', u'aa', u'b'); //A contains u'bbb' A := unicodelib.UnicodeLocaleFindReplace(u'gh\u0131klm', u'hyk', u'XxXxX', 'lt'); //A contains u'gXxXxXlm' A := unicodelib.UnicodeLocaleFindReplace(u'gh\u0131klm', u'hyk', u'X', 'lt'); //A contains u'gXlm' StringLib.GetBuildInfo() Return: GetBuildInfo returns a VARSTRING (null-terminated) value. The GetBuildInfo functions return a string containing information that specifies the build number and date/time stamp of the supercomputer software. Example: A := StringLib.GetBuildInfo(); StringLib.GetDateYYYYMMDD() Return: GetDateYYYYMMDD returns a STRING value. The GetDateYYYYMMDD function returns a string containing the current date in YYYYMMDD format. Example: A := GetDateYYYYMMDD(); StringLib.StringRepad(source, size)UnicodeLib.UnicodeRepad(source, size) source A string containing the data to resize. size A positive integer containing the number of characters in the result string. Return: Repad returns a STRING or UNICODE value, as appropriate. The Repad functions return the source string, stripped of any leading spaces, at the specified number of characters. Trailing spaces are added as needed to achieve the required size. Example: // All these examples result in 'Success' A := IF(StringLib.StringRepad('ABCDE ', 6) = 'ABCDE ', 'Success', 'Failure - 1'); B := IF(StringLib.StringRepad(' ',6) = ' ', 'Success', 'Failure - 2'); C := IF(StringLib.StringRepad('ABCDE ',0)='', 'Success', 'Failure - 3'); D := IF(StringLib.StringRepad('ABCDE ', 3) = 'ABC', 'Success', 'Failure - 4'); E := IF(StringLib.StringRepad(' ABCDE ', 3) = 'ABC', 'Success', 'Failure - 5'); StringLib.StringReverse(source)UnicodeLib.UnicodeReverse(source) source A string containing the data to reverse. Return: Reverse returns a STRING or UNICODE value, as appropriate. The Reverse functions return the source string with all characters in reverse order. Example: A := StringLib.StringReverse('ABCDE'); //A contains 'EDCBA' StringLib.String2Data(source) source A STRING to convert. Return: String2Data returns a DATA value. The String2Data function returns the DATA value for the hexadecimal characters contained in the source STRING. This function is the opposite of the Data2String function. Example: DATA2 CrLf := x’0A0D’; //2-bytes of hex STRING4 CrLfString := StringLib.Data2String(CrLf); //results in a ‘0A0D’ 4-byte string DATA2 NewCrLf := StringLib.String2Data(CrLfString); //and back again StringLib.StringSubstitute(source, target, replacement)UnicodeLib.UnicodeSubstitute(source, target, replacement) source A string containing the data to search. target A string containing the substring to search for. replacement A string containing the replacement character. Return: Substitute returns a STRING or UNICODE value, as appropriate. The Substitute functions return the source string with the replacement character substituted for all characters except those in the target string. If the target string is not in the source string, it returns the source string with all characters replaced by the replacement character. Example: A := unicodelib.UnicodeSubstitute(u'abcdeabcdec', u'cd', u'x'); //A contains u'xxcdxxxcdxc'; StringLib.StringSubstituteOut(source, target, replacement)UnicodeLib.UnicodeSubstituteOut(source, target, replacement) source A string containing the data to search. target A string containing the substring to search for. replacement A string containing the replacement character. Return: Substitute returns a STRING or UNICODE value, as appropriate. The Substitute functions return the source string with the replacement character substituted for all characters that exist in both the source and the target string. If no target string characters are in the source string, it returns the source string unaltered. Example: A := unicodelib.UnicodeSubstituteOut(u'abcde', u'cd', u'x'); //A contains u'abxxe'; StringLib.StringToLowerCase(source)UnicodeLib.UnicodeToLowerCase(source)UnicodeLib.UnicodeLocaleToLowerCase( source, locale ) source A string containing the data to change case. locale A null-terminated string containing the language and country code to use to determine correct sort order and other operations. Return: ToLowerCase returns a STRING or UNICODE value, as appropriate. The ToLowerCase functions return the source string with all upper case characters converted to lower case. Example: A := StringLib.StringToLowerCase('ABCDE'); //A contains ‘abcde’ StringLib.StringToProperCase(source)UnicodeLib.UnicodeToProperCase(source)UnicodeLib.UnicodeLocaleToProperCase( source, locale ) source A string containing the data to change case. locale A null-terminated string containing the language and country code to use to determine correct sort order and other operations. Return: ToProperCase returns a STRING or UNICODE value, as appropriate. The ToProperCase functions return the source string with the first letter of each word in upper case and all other letters left as-is. Example: A := StringLib.StringToProperCase('ABCDE ABCDE '); //A contains ‘Abcde Abcde’ StringLib.StringToUpperCase(source)UnicodeLib.UnicodeToUpperCase(source)UnicodeLib.UnicodeLocaleToUpperCase( source, locale ) source A string containing the data to change case. locale A null-terminated string containing the language and country code to use to determine correct sort order and other operations. Return: StringToUpperCase returns a STRING value. The ToUpperCase functions return the source string with all lower case characters converted to upper case. Example: A := StringLib.StringToUpperCase('abcde'); //A contains ‘ABCDE’ StringLib.StringWildMatch(source, pattern, nocase)UnicodeLib.UnicodeWildMatch(source, pattern, nocase) source A string containing the data to search. pattern A string containing the wildcard expression to match. Valid wildcards are ? (single character) and * (multiple character). nocase A boolean true or false indicating whether to ignore the case. Return: WildMatch returns a BOOLEAN value. The WildMatch function returns TRUE if the pattern matches the source. The case-insensitive version of UnicodeWildMatch has been optimized for speed over accuracy. For accurate case-folding, you should either use the UnicodeToUpperCase function explicity and then a case-sensitive UnicodeWildMatch, or use REGEXFIND. Example: stringlib.stringwildmatch('abcdeabcdec', 'a?c*', false) = TRUE; DataLib.AddressClean(source1,source2) source1 A string containing the street address line to clean. source2 A string containing the city/state/zip address line to clean. Return: AddressClean returns a STRING value. The AddressClean function returns a 261-byte string in the following format: Byte Position Component 1 primary address number 11 predirectional (N,NE,E,SE,S,SW,W,NW) 13 street name 41 suffix (ST, AVE, BLVD, etc.) 45 postdirectional (N,NE,E,SE,S,SW,W,NW) 47 secondary address indentifier (APT, STE, etc.) 57 secondary address range 65 city 90 state 92 5-digit zipcode 97 zip plus4 101 country 141 confidence level (0-9, 0=good 9=bad) 142 person (if source1 contains a name also) Example: A := DataLib.AddressClean('4590 NW 23rd Boulevard South Suite 307','Boca Raton, Fl 33434-6332'); /* A contains ‘4590 NW23RD BLVDS STE 307 BOCA RATON FL 33434 6332 1’ */ DataLib.CompanyClean(source) source A string containing the company name. Return: CompanyClean returns a STRING value. The CompanyClean function returns a 120-byte string in the following format: Byte Position Component 1 cleaned primary part of company name 41 cleaned secondary part of company name 81 “discardable” words (such as Corp. Inc. Ltd.) Example: A := DataLib.CompanyClean('The Seisint Data Mgmnt Company, Inc. Ltd'); /* A contains ‘THE SEISINT DATA MGMNT COMPANY INC LTD’ */ DataLib.DeDouble(source) source A string to process. Return: DeDouble returns a STRING value. The DeDouble function returns the source string with all instances of multiple adjacent characters (2 or more like characters together) reduced to a single instance. Example: A := DataLib.DeDouble('ABBBCCDE'); //A contains 'ABCDE' DataLib.Gender(name) name A string containing the forename to process. Return: Gender returns a STRING1 value, . The Gender function returns the gender most commonly associated with the name string as M (male), F (female), N (neutral), or U (unkown). Example: A := DataLib.Gender('DAVID'); //A contains 'M' B := DataLib.Gender('LESLIE'); //B contains 'N' C := DataLib.Gender('SUSAN'); //C contains 'F' D := DataLib.Gender('ABCDE'); //D contains 'U' DataLib.LeadMatch(source1,source2) source1 A string to compare. source2 A string to compare. Return: LeadMatch returns an UNSIGNED INTEGER value. The LeadMatch function returns the number of matching characters at the beginning of the source1 and source2 strings. Example: A := DataLib.LeadMatch('ABCDE','ABXDE'); //A contains 2 DataLib.NameClean(source) source A string containing the name. Return: NameClean returns a STRING value. The NameClean function returns a 142-byte string in the following format: Byte Position Component 1 first name 41 middle name 81 last name 121 name prefix (MR, MRS, DR, etc.) 131 name suffix (JR, SR, III, etc.) 141 gender code (M, F, N) 142 confidence flag (0-9, 0=good 9=bad) Example: A := DataLib.NameClean('Betty Ann Boop'); /*A contains ‘BETTY ANN BOOP F0’ */ B := DataLib.NameClean('Leslie Jean Boop'); /*B contains ‘LESLIE JEAN BOOP N0’ */ DataLib.NameMatch(leftfirst,leftmiddle,leftlast,rightfirst,rightmiddle,rightlast) leftfirst A string containing the first name. leftmiddle A string containing the middle name. leftlast A string containing the last name. rightfirst A string containing the first name. rightmiddle A string containing the middle name. rightlast A string containing the last name. Return: NameMatch returns an UNSIGNED INTEGER4 value. The NameMatch function returns a score indicating how closely the leftfirst, leftmiddle, and leftlast name matches the rightfirst, rightmiddle, and rightlast name. The lower the score, the closer the match, with zero (0) indicating a perfect match. This function compares the probability of the two incoming names referring to the same person versus one of the names referring to somebody else (this is not quite the same as them not being a reference to the same person). The main distinction is in the area of noise. Noise is defined as: a name (first or middle) that is not in our tables (we have more than 100K in our tables), or a last name we have never encountered with repeating characters or too few vowels, etc. Noise causes that portion of the name to be underwieghted in the final scoring. Example: A1 := DataLib.NameMatch('RICHARD', 'L', 'TAYLOR', 'RICHARD', 'L', 'TAYLOR'); //returns 0 A2 := DataLib.NameMatch('RICH', 'L', 'TAYLOR', 'RICHARD', 'L', 'TAYLOR'); //returns 1 A3 := DataLib.NameMatch('RICH', '', 'TAYLOR', 'RICHARD', 'L', 'TAYLOR'); //returns 2 A4 := DataLib.NameMatch('ROD', 'L', 'TAYLOR', 'RICHARD', 'L', 'TAYLOR'); //returns 99 //this gets a high score because there is a high chance // of ROD being a valid name of somebody else A5 := DataLib.NameMatch('ZXCVBNM', 'L', 'T', 'RICHARD', 'L', 'TAYLORSKY'); //returns 100 A6 := DataLib.NameMatch('ZXCVBNM', 'B', 'T', 'RICHARD', 'L', 'TAYLOR'); //returns 199 A7 := DataLib.NameMatch('T', 'L', 'ZXCVBNM', 'TAYLOR', 'L', 'RICHARD'); //returns 100 DataLib.NameSimilar(left,right, blanks) left A string containing the name. right A string containing the name. blanks A boolean value. When TRUE, it strips trailing spaces for the comparison between left and right. Return: NameSimilar returns an UNSIGNED INTEGER4 value. The NameSimilar function returns a score indicating how closely the left name matches the right name. The lower the score, the closer the match, with zero (0) indicating a perfect match. Example: A1 := datalib.NameSimilar('Fred Smith','Fred Smith',1); // returns 0 A2 := datalib.NameSimilar('Fred Smith','Fred Smythe',1); // returns 1 A3 := datalib.NameSimilar('Fred Smith','Fred Jones',1); // returns 99 DataLib.PositionalMatch(source1,source2) source1 A string to compare. source2 A string to compare. Return: PositionalMatch returns an UNSIGNED INTEGER value. The PositionalMatch function returns the number of matching characters at the exactly the same positions within the source1 and source2 strings. Example: A := DataLib.PositionalMatch(‘ABCDE’,’ABXDE’); //A contains 4 B := DataLib.PositionalMatch(‘ABCDE’,’ABCDE’); //B contains 5 C := DataLib.PositionalMatch(‘ABCDE’,’EABCDE’); //C contains 0 DataLib.PreferredFirst(source) source A string containing the first name. Return: PreferredFirst returns a STRING value. The PreferredFirst function returns the preferred first name replacement for the source first name given. This is used to replace nicknames with their more formal equivalents. Example: A := DataLib.PreferredFirst('Dick'); //A contains ‘RICHARD’ B := DataLib.PreferredFirst('RICHIE'); //B contains ‘RICHARD’ C := DataLib.PreferredFirst('GWEN'); //C contains ‘GWENDOLYN’ DataLib.SlidingMatch(source1,source2) source1 A string to compare. source2 A string to compare. Return: SlidingMatch returns an UNSIGNED INTEGER value. The SlidingMatch function returns the number of characters in source2 which appear in exactly the same sequence within the source1 string. There is a penalty imposed if subsequent slides happen without at least two matches in between. Example: A := DataLib.SlidingMatch('CBABT','CAT'); //A contains 2 (penalty) B := DataLib.SlidingMatch('CABT','CAT'); //B contains 3 (no penalty) C := DataLib.SlidingMatch('CBAT','CAT'); //C contains 3 (no penalty) D := DataLib.SlidingMatch('CAT','CAT'); //D contains 3 DataLib.StrCompare(source1, source2) source1 A string to compare. source2 A string to compare. Return: StrCompare returns an INTEGER value. The StrCompare function returns a score from 0 to 100 that indicates how closely the source1 and source2 strings match. Example: A := DataLib.StrCompare('ABCDEABCDE','ABCDEABCDE'); //A contains 100 B := DataLib.StrCompare('ABCDEABCDE','ABCDE ABCDE'); //B contains 95 C := DataLib.StrCompare('ABCDEABCDE','WXYZ ABCDE'); //C contains 50 D := DataLib.StrCompare('ABCDEABCDE','WXYZ WXYZ'); //D contains 0 DataLib.StringFind(source, target, instance) source A string containing the data to search. target A string containing the substring to search for. instance An integer specifying which occurrence of the substring to find. Return: StringFind returns an INTEGER value. The StringFind function returns the beginning index position within the source string of the specified instance of the target string. If the target is not found or the specified instance is greater than the number of occurrences of the target in the source, StringFind returns zero (0). Example: A := IF(DataLib.StringFind('ABCDE', 'BC',1) = 2, 'Success', 'Failure - 1'); //success B := IF(DataLib.StringFind('ABCDEABCDE', 'BC', 2) = 7, 'Success', 'Failure - 2'); //success C := IF(DataLib.StringFind('ABCDEABCDE', '') = 0, 'Success', 'Failure - 3'); //syntax error D := IF(DataLib.StringFind('', 'BD', 1) = 0, 'Success', 'Failure - 4'); //success DataLib.StringReplaceSmaller(source, target, replacement) source A string containing the data to search. target A string containing the substring to search for. replacement A string containing the replacement data. Return: StringReplaceSmaller returns a STRING value. The StringReplaceSmaller function returns the source string with the replacement string substituted for all instances of the target string, but only if the length of the replacement string is <= the length of the target string . If the target string is not in the source string or the replacement is too long, it returns the source string unaltered. Example: A := DataLib.StringReplaceSmaller('ABCDEABCDE', 'BC','XY'); //A contains ‘AXYDEAXYDE’ DataLib.StringSimilar100(left,right) left A string. right A string. Return: StringSimilar100 returns an UNSIGNED INTEGER4 value. The StringSimilar function returns a score indicating how closely the left string matches the right string. The lower the score, the closer the match, with zero (0) indicating a perfect match. Example: A1 := datalib.stringsimilar100('abc','abc'); //returns 0 A2 := datalib.stringsimilar100('abc','cde'); //returns 67 A3 := datalib.stringsimilar100('abc','abcde'); //returns 27 A4 := datalib.stringsimilar100('abc','ABC'); //returns 100 A5 := datalib.stringsimilar100('abc','xyz'); //returns 100 FileServices.CompareFiles( file1, file2 [, logicalonly ] [, usecrcs ] ) file1 A null-terminated string containing the logical name of the first file. file2 A null-terminated string containing the logical name of the second file. logicalonly Optional. A boolean TRUE/FALSE flag that, when TRUE, does not compare physical information from disk but only the logical information in the system datastore (Dali). If omitted, the default is TRUE. usecrcs Optional. A boolean TRUE/FALSE flag indicating that, when TRUE, compares physical CRCs of all the parts on disk. This may be slow on large files. If omitted, the default is FALSE. Return: CompareFiles returns returns an INTEGER4 value. The CompareFiles function compares file1 against file2 and returns the following values: 0 file1 and file2 match exactly 1 file1 and file2 contents match, but file1 is newer than file2 -1 file1 and file2 contents match, but file2 is newer than file1 2 file1 and file2 contents do not match and file1 is newer than file2 -2 file1 and file2 contents do not match and file2 is newer than file1 Example: A := FileServices.CompareFiles('Fred1', 'Fred2'); FileServices.DeleteLogicalFile( filename [, ifexists ] ) filename A null-terminated string containing the logical name of the file. ifexists Optional. A boolean value indicating whether to post an error if the filename does not exist. If omitted, the default is FALSE. The DeleteLogicalFile function removes the named file from disk. Example: A := FileServices.DeleteLogicalFile('Fred'); FileServices.LogicalFileList( [ pattern ] [, includenormal ] [, includesuper ] [, unknownszero ] ) pattern Optional. A null-terminated string containing the mask of the files to list. If omitted,the default is '*' (all files). includenormal Optional. A boolean flag indicating whether to include “normal” files. If omitted, the default is TRUE. includesuper Optional. A boolean flag indicating whether to include SuperFiles. If omitted, the default is FALSE. unknownszero Optional. A boolean flag indicating to set file sizes that are unknown to zero (0) instead of minus-one (-1). If omitted, the default is FALSE. Return: LogicalFileList returns returns a dataset in the following format: EXPORT FsLogicalFileNameRecord := RECORD STRING name; END; EXPORT FsLogicalFileInfoRecord := RECORD(FsLogicalFileNameRecord) BOOLEAN superfile; UNSIGNED8 size; UNSIGNED8 rowcount; STRING19 modified; STRING owner; STRING cluster; END; The LogicalFileList function returns a list of the logical files in the environment files as a dataset in the format listed above. Example: OUTPUT(FileServices.LogicalFileList()); //returns all normal files OUTPUT(FileServices.LogicalFileList(,FALSE,TRUE)); //returns all SuperFiles FileServices.FileExists( filename [, physicalcheck] ) filename A null-terminated string containing the logical name of the file. physicalcheck Optional. A boolean TRUE/FALSE to indicate whether to check for the physical existence the filename on disk. If omitted, the default is FALSE. Return: FileExists returns a BOOLEAN value. The FileExists function returns TRUE if the specified filename is present in the Distributed File Utility (DFU) and is not a SuperFile (use the FileServices.SuperFileExists function to detect their presence or absence). If physicalcheck is set to TRUE, then the file’s physical presence on disk is also checked. Example: A := FileServices.FileExists('~CLASS::RT::IN::People'); FileServices.ForeignLogicalFileName(filename [,foreigndali ] [,absolutepath] ) filename A null-terminated string containing the logical name of the file. foreigndali A null-terminated string containing the IP address of the foreign Dali. If omitted, the filename is presumed to be a foreign logical file name, which is converted to a local logical file name. absolutepath Optional. A boolean TRUE/FALSE to indicate whether to prepend a tilde (~) to the resulting foreign logical file name. If omitted, the default is FALSE. Return: ForeignLogicalFileName returns returns a VARSTRING (null-terminated) value. The ForeignLogicalFileName function returns either a foreign logical file name (if the foreigndali parameter is present) or a local logical file name. Example: sf := '~thor_data400::BASE::Business_Header'; ff := FileServices.ForeignLogicalFileName(sf,'10.150.29.161',true); //results in: ~foreign::10.150.29.161::thor_data400::base::business_header lf := FileServices.ForeignLogicalFileName(ff,'',true); //results in: ~thor_data400::base::business_header FileServices.GetBuildInfo() Return: GetBuildInfo returns a VARSTRING (null-terminated) value. The GetBuildInfo functions return a string containing information that specifies the build number and date/time stamp of the supercomputer software. Example: A := FileServices.GetBuildInfo(); ThorLib.GetExpandLogicalFileName( filename ) filename A null-terminated string containing the logical name of the file. Return: GetExpandLogicalFileName returns a VARSTRING (null-terminated) value. The GetExpandLogicalFileName function returns a string containing the expanded logical filename (ncluding the default scope, if the filename does not contain a leading tilde), all in lowercase. This is the same value as is used internally by DATASET and OUTPUT. Example: A := ThorLib.GetExpandLogicalFileName('Fred'); FileServices.GetFileDescription( filename ) filename A null-terminated string containing the logical name of the file. Return: GetFileDescription returns a VARSTRING (null-terminated) value. The GetFileDescription function returns a string containing the description information stored by the DFU about the specified filename. This description is set either through ECL watch or by using the FileServices.SetFileDescription function. Example: A := FileServices.GetFileDescription('Fred'); FileServices.RemoteDirectory( machineIP, directory [, mask ][, includesubs ] ) machineIP A null-terminated string containing the IP address of the remote machine. directory A null-terminated string containing the path to the directory to read. This must be in the appropriate format for the operating system running on the remote machine. mask Optional. A null-terminated string containing the filemask specifying which files to include in the result. If omitted,the default is '*' (all files). includesubdir Optional. A boolean flag indicating whether to include files from sub-directories under the directory. If omitted, the default is FALSE. Return: RemoteDirectory returns a dataset in the following format: EXPORT FsFilenameRecord := RECORD STRING name; //filename UNSIGNED8 size; //filesize STRING19 modified; //date-time stamp END; The RemoteDirectory function returns a list of files as a dataset in the format listed above from the specified machineIP and directory. If includesubdir is set to TRUE, then the name field contains the relative path to the file from the specified directory. Example: OUTPUT(FileServices.RemoteDirectory('edata12','\in','*.d00')); OUTPUT(FileServices.RemoteDirectory('10.150.254.6', '/c$/training',,TRUE)); FileServices.RenameLogicalFile( filename, newname ) filename A null-terminated string containing the current logical name of the file. newname A null-terminated string containing the new logical name for the file. The RenameLogicalFile function changes the logical filename to the newname. Example: A := FileServices.RenameLogicalFile('Fred', 'Freddie'); FileServices.Replicate( filename [, timeout ] [, espserverIPport])dfuwuid := FileServices.fReplicate( filename [, timeout ] [, espserverIPport]); filename A null-terminated string containing the logical name of the file. timeout Optional. An integer value indicating the timeout setting. If omitted, the default is -1. If set to zero (0), execution control returns immediately to the ECL workunit without waiting for the DFU workunit to complete. espserverIPport Optional. A null-terminated string containing the protocol, IP, port, and directory, or the DNS equivalent, of the ESP server program. This is usually the same IP and port as ECL Watch, with “/FileSpray” appended. If omitted, the default is the value contained in the lib_system.ws_fs_server attribute. dfuwuid The attribute name to recieve the null-terminated string containing the DFU workunit ID (DFUWUID) generated for the job. The Replicate function copies the individual parts of the filename to the mirror disks for the cluster. Typically, this means that the file part on one node’s C drive is copied to its neighbors D drive. Example: A := FileServices.Replicate('Fred'); FileServices.SendEmail( sendto, subject, body, server, port, sender ) sendto A null-terminated string containing a comma-delimited list of the addresses of the intended recipients. The validity of the addresses is not checked, so it is the programmer’s responsibility to ensure they are all valid. subject A null-terminated string containing the subject line. body A null-terminated string containing the text of the email to send. This must be character encoding “ISO-8859-1 (latin1)” (the ECL default character set). Text in any other character set must be sent as an attachment (see the FileServices.SendEmailAttachText() function). server Optional. A null-terminated string containing the name of the mail server. If omitted, defaults to the value in the lib_system.SMTPserver attribute. port Optional. An UNSIGNED4 integer value containing the port number. If omitted, defaults to the value in the lib_system.SMTPport attribute. sender Optional. A null-terminated string containing the address of the sender. If omitted, defaults to the value in the lib_system.emailAddress attribute. The SendEmail function sends an email message. Example: FileServices.SendEmail( 'me@mydomain.com', 'testing 1,2,3', 'this is a test message'); FileServices.SendEmailAttachData( sendto, subject, body, attachment, mimietype, filename, server, port, sender ) sendto A null-terminated string containing a comma-delimited list of the addresses of the intended recipients. The validity of the addresses is not checked, so it is the programmer’s responsibility to ensure they are all valid. subject A null-terminated string containing the subject line. body A null-terminated string containing the text of the email to send. This must be character encoding “ISO-8859-1 (latin1)” (the ECL default character set). Text in any other character set must be sent as an attachment. attachment A DATA value containing the binary data to attach. mimetype A null-terminated string containing the MIME-type of the attachment, which may include parameters (such as ‘text/plain; charset=ISO-8859-3’). When attaching general binary data for which no specific MIME type exists, use ‘application/octet-stream’. filename A null-terminated string containing the name of the attachment for the mail reader to display. server Optional. A null-terminated string containing the name of the mail server. If omitted, defaults to the value in the lib_system.SMTPserver attribute. port Optional. An UNSIGNED4 integer value containing the port number. If omitted, defaults to the value in the lib_system.SMTPport attribute. sender Optional. A null-terminated string containing the address of the sender. If omitted, defaults to the value in the lib_system.emailAddress attribute. The SendEmailAttachData function sends an email message with a binary attachment. Example: DATA15 attachment := D'test attachment'; FileServices.SendEmailAttachData( 'me@mydomain.com', 'testing 1,2,3', 'this is a test message', attachment, 'application/octet-stream', 'attachment.txt'); FileServices.SendEmailAttachText( sendto, subject, body, attachment, mimietype, filename, server, port, sender ) sendto A null-terminated string containing a comma-delimited list of the addresses of the intended recipients. The validity of the addresses is not checked, so it is the programmer’s responsibility to ensure they are all valid. subject A null-terminated string containing the subject line. body A null-terminated string containing the text of the email to send. This must be character encoding “ISO-8859-1 (latin1)” (the ECL default character set). Text in any other character set must be sent as an attachment. attachment A null-terminated string containing the text to attach. mimetype A null-terminated string containing the MIME-type of the attachment, which may include parameters (such as ‘text/plain; charset=ISO-8859-3’). filename A null-terminated string containing the name of the attachment for the mail reader to display. server Optional. A null-terminated string containing the name of the mail server. If omitted, defaults to the value in the lib_system.SMTPserver attribute. port Optional. An UNSIGNED4 integer value containing the port number. If omitted, defaults to the value in the lib_system.SMTPport attribute. sender Optional. A null-terminated string containing the address of the sender. If omitted, defaults to the value in the lib_system.emailAddress attribute. The SendEmailAttachText function sends an email message with a text attachment. Example: FileServices.SendEmailAttachText( 'me@mydomain.com', 'testing 1,2,3', 'this is a test message', 'this is a test attachment', 'text/plain; charset=ISO-8859-3', 'attachment.txt'); FileServices.SetFileDescription( filename , value ) filename A null-terminated string containing the logical name of the file. value A null-terminated string containing the description to place on the file. The SetFileDescription function changes the description information stored by the DFU about the specified filename to the specified value. This description is seen either through ECL watch or by using the FileServices.GetFileDescription function. Example: A := FileServices.SetFileDescription('Fred','All the Freds in the world'); FileServices.SetReadOnly( filename , flag ) filename A null-terminated string containing the logical name of the file. flag A boolean value indicating which way to set the read-only attribute of the filename. The SetReadOnly function toggles the read-only attribute of the filename. If the flag is TRUE, read-only is set on. Example: A := FileServices.SetReadOnly('Fred',TRUE); //set read only flag on FileServices.VerifyFile( file, usecrcs ) file A null-terminated string containing the logical name of the file. usecrcs A boolean TRUE/FALSE flag indicating that, when TRUE, compares physical CRCs of all the parts on disk. This may be slow on large files. Return: VerifyFile returns returns a VARSTRING value. The VerifyFile function checks the system datastore (Dali) information for the file against the physical parts on disk and returns the following values: OK The file parts match the datastore information Could not find file: filename The logical filename was not found Could not find part file: partname The partname was not found Modified time differs for: partname The partname has a different timestamp File size differs for: partname The partname has a file size File CRC differs for: partname The partname has a different CRC Example: A := FileServices.VerifyFile('Fred1', TRUE); FileServices.ExternalLogicalFileName( machineIP, filename ) machineIP A null-terminated string containing the IP address of the remote machine. filename A null-terminated string containing the path/name of the file. Return: ExternalLogicalFileName returns returns a VARSTRING (null-terminated) value. The ExternalLogicalFileName function returns an appropriately encoded external logical file name that can be used to directly read a file from any node that is running the dafilesrv utility (typically a landing zone). It handles upper case characters by escaping those characters inthe return string. Example: IP := '10.150.254.6'; file := '/c$/training/import/AdvancedECL/people'; DS1 := DATASET(FileServices.ExternalLogicalFileName(IP,file), Training_Advanced.Layout_PeopleFile, FLAT); OUTPUT(FileServices.ExternalLogicalFileName(IP,file)); //returns: //~file::10.150.254.6::c$::training::import::^advanced^e^c^l::people OUTPUT(DS1); //returns records from the external file result := FileServices.GetHostName( ip ); ip A null-terminated string containing the IP address of the remote machine. Return: GetHostName returns returns a VARSTRING (null-terminated) value. The GetHostName function does a reverse DNS lookup to return the host name for the machine at the specified ip address. Example: IP := '10.150.254.6'; OUTPUT(FileServices.GetHostName(IP)); result := FileServices.ResolveHostName( host ); host A null-terminated string containing the DNS name of the remote machine. Return: ResolveHostName returns returns a VARSTRING (null-terminated) value. The ResolveHostName function does a DNS lookup to return the ip address for the specified host name. Example: host := 'dataland_dali.br.seisint.com'; OUTPUT(FileServices.ResolveHostName(host)); FileServices.MoveExternalFile( location, frompath, topath ) location A null-terminated string containing the IP address of the remote machine. frompath A null-terminated string containing the path/name of the file to move. topath A null-terminated string containing the path/name of the target file. The MoveExternalFile function moves the single physical file specified by the frompath to the topath. Both frompath and topath are on the same remote machine, identified by the location. The dafileserv utility program must be running on the location machine. Example: IP := '10.150.254.6'; infile := '/c$/training/import/AdvancedECL/people'; outfile := '/c$/training/import/DFUtest/people'; FileServices.MoveExternalFile(IP,infile,outfile); FileServices.DeleteExternalFile( location, path ) location A null-terminated string containing the IP address of the remote machine. path A null-terminated string containing the path/name of the file to remove. The DeleteExternalFile function removes the single physical file specified by the path from the location. The dafileserv utility program must be running on the location machine. Example: IP := '10.150.254.6'; infile := '/c$/training/import/AdvancedECL/people'; FileServices.DeleteExternalFile(IP,infile); FileServices.CreateExternalDirectory( location, path ) location A null-terminated string containing the IP address of the remote machine. path A null-terminated string containing the directory path to create. The CreateExternalDirectory function creates the path on the location (if it does not already exist). The dafileserv utility program must be running on the location machine. Example: IP := '10.150.254.6'; path := '/c$/training/import/NewDir'; FileServices.CreateExternalDirectory(IP,path); result := FileServices.RfsQuery( server, query ); server A null-terminated string containing the ip:port address for the remote file server. query A null-terminated string containing the query to send to the server. Return: RfsQuery returns a null-terminated string containing the result of the query. The RfsQuery function returns a string that can be used in a DATASET declaration to read data from an RFS (Remote File Server) instance (e.g. rfsmysql) on another node. Example: rfsserver := '10.173.207.1:7080'; rec := RECORD,MAXLENGTH(8192) STRING mydata; END; OUTPUT(DATASET(FileServices.RfsQuery( rfsserver, 'SELECT data FROM xml_testnh'),rec,CSV(MAXLENGTH(8192)))); FileServices.RfsAction( server, query ); server A null-terminated string containing the ip:port address for the remote file server. query A null-terminated string containing the query to send to the server. The RfsAction function sends the query to the server. This is used when there is no expected return value Example: rfsserver := '10.173.207.1:7080'; FileServices.RfsAction(rfsserver,'INSERT INTO xml_testnh (data) VALUES (\''+TRIM(A)+'\' )'); result := NameLib.FirstNameToToken( name ); name A string expression containing the name to tokenize. Maximum sixe: 20 characters. Return: FirstNameToToken returns a string. The FirstNameToToken function returns a string containing a tokenized representation of the name.. Example: x := namelib.FirstNameToToken('Thompson'); result := NameLib.TokenToFirstName( token ); token A string expression containing the token resulting from the FirstNameToToken function. Return: TokenToFirstName returns a 20-character string. The TokenToFirstName function returns a string containing the name represented by the token. Example: y := namelib.TokenToFirstName(x); //returns 'Thompson' result := NameLib.TokenToLength( token ); token A string expression containing the token resulting from the FirstNameToToken function. Return: TokenToLength returns an unsigned 4-byte integer value. The TokenToLength function returns the number of characters in the token. Example: x := namelib.FirstNameToToken('Thompson'); y := namelib.TokenToFirstName(x); z := namelib.TokenToLength(x); ds := dataset([{x,y,z}],{string Tok, string Name, unsigned4 Len}); output(ds); FileServices.SetColumnMapping( file, mapping ); file A null-terminated string containing the logical filename. mapping A null-terminated string containing a comma-delimited list of field mappings. The SetColumnMapping function defines how the data in the fields of the file mist be transformed between the actual data storage format and the input format used to query that data. The format for each field in the mapping list is: <field>{set(<transform>(args),...),get(<transform>,...),displayname(<name>)} <field> The name of the field in the file. set Optional. Specifies the transforms applied to the values supplied by the user to convert them to values in the file. <transform> Optional. The name of a function to apply to the value. This is typically the name of a plugin function. The value being converted is always provided as the first parameter to the function, but extra parameters can be specified in brackets after the transform name (similar to SALT hygiene). get Optional. Specifies the transforms applied to the values in the file to convert them to the formatted values as they are understood by the user. displayname Optional. Allows a different name to be associated with the field than the user would naturally understand. Note that you may mix unicode and string functions, as the system automatically converts the parameters to the appropriate types expected for the functions. Example: // A file where the firstname(string) and lastname(unicode) are //always upper-cased: // There is no need for a displayname since it isn't really a // different field as far as the user is concerned, and there is // obviously no get transformations. firstname{set(stringlib.StringToUpperCase)}, surname{set(unicodelib.UnicodeToUpperCase)} // A name translated using a phonetic key // it is worth specifying a display name here, because it will make // more sense to the user, and the user may want to enter either the // translated or untranslated names. dph_lname{set(metaphonelib.DMetaPhone1), displayname(lname)} // A file where a name is converted to a token using the namelib // functions. (I don't think we have an example of this) // (one of the few situations where a get() attribute is useful) fnametoken{set(namelib.nameToToken), get(namelib.tokenToName), displayname(fname)} // upper case, and only include digits and alphabetic. searchname{set(stringlib.StringToUpperCase, stringlib.StringFilter( 'ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'))} // A file with a field that that needs to remove accents and then // uppercase: lastname{set(unicodeLIb.CleanAccents,stringLib.StringToUpperCase)} result := FileServices.GetColumnMapping( file ); file A null-terminated string containing the logical filename. Return: GetColumnMapping returns a null-terminated string containing the comma-delimited list of field mappings for the file. The GetColumnMapping function returns the field mappings for the file, in the same format specified for the SetColumnMapping function. Example: Maps := FileServices.GetColumnMapping('Thor::in::SomeFile'); FileServices.AddFileRelationship( primary, secondary, primaryfields, secondaryfields, [ relationship ], cardinality, payload [, description ] ); primary A null-terminated string containing the logical filename of the primary file. secondary A null-terminated string containing the logical filename of the secondary file. primaryfields A null-terminated string containing the name of the primary key field for the primary file. The value “__fileposition__” indicates the secondary is an INDEX that must use FETCH to access non-keyed fields. secondaryfields A null-terminated string containing the name of the foreign key field relating to the primary file. relationship A null-terminated string containing either “link” or “view” indicating the type of relationship between the primary and secondary files. If omitted, the default is “link.” cardinality A null-terminated string containing the kind of relationship between the primary and secondary files. The format is X:Y where X indicates the primary and Y indicates the secondary. Valid values for X and Y are “1” or ‘M’. payload A BOOLEAN value indicating whether the primary or secondary are payload INDEXes. description A null-terminated string containing the relationship description. The AddFileRelationship function defines the relationship between two files. These may be DATASETs or INDEXes. Each record in the primary file should be uniquely defined by the primaryfields (ideally), preferably efficiently. The primaryfields and secondaryfields parameters can have the same format as the column mappings for a file (see the SetColumnMappings function documentation) , although they will often be just a list of fields. They are currently used in two different ways: First, the roxie browser needs a way of determining which indexes are built from which files. A “view” relationship should be added each time an index is built from a file, like this: fileServices.AddFileRelationship(DG_FlatFileName, DG_IndexFileName, '', '', 'view', '1:1', false); To implement the roxie browser there is no need to define the primaryfields or secondaryfields, so passing blank strings is recommended. Second, the browser needs a way of finding all the original information from the file from an index. This stage depends on the nature of the index: a) If the index contains all the relevant data from the original file you don’t need to do anything. b) If the index uses a fileposition field to FETCH extra data from the original file then add a relationship between the original file and the index, using a special value of __fileposition__ to indicate the record is retrieved using a FETCH. fileServices.AddFileRelationship('fetch_file', 'new_index', '__fileposition__', 'index_filepos_field', 'link', '1:1', true); The original file is the primary, since the rows are uniquely identified by the fileposition (also true of the index), and the retrieval is efficient. c) If the index uses a payload field which needs to be looked up in another index to provide the information, then you need to define a relationship between the new index and the index that provides the extra information. The index providing the extra information is the primary. fileServices.AddFileRelationship('related_index', 'new_index', 'related_key_fields', 'index_filepos_field', 'link', '1:M', true); The payload flag is there so that the roxie browser can distinguish this link from a more general relationship between two files. You should ensure any super-file names are expanded if the relation is defined between the particular sub-files. While going through all the attributes it may be worth examining whether it makes sense to add relationships for any other combinations of files. It won’t have any immediate beneficial effect, but would once we add an ER diagram to the system. A couple of examples may help illustrate the syntax. For a typical example, datasets with a household and person file, the following defines a relationship linking by house hold id (hhid): fileServices.AddFileRelationship('HHFile','PersonFile', 'hhid', 'hhid', 'link', '1:M', false); Here’s a more hypothetical example—a file query with firstname, lastname related to an index with phonetic names you might have: fileServices.AddFileRelationship('names', 'inquiries', 'plastname{set(phonetic)},pfirstname{set(phonetic)}', 'lastname{set(fail)},firstname{set(fail)}', 'link', '1:M', false); Note, the fail mapping indicates that you can use the phonetic mapping from inquiries to names, but there is no way of mapping from names to inquiries. There could equally be get(fail) attributes on the index fields. Example: Maps := FileServices.GetColumnMapping('Thor::in::SomeFile'); FileServices.FileRelationshipList( primary, secondary [, primaryfields ] [, secondaryfields ] [, relationship ]); primary A null-terminated string containing the logical filename of the primary file. secondary A null-terminated string containing the logical filename of the secondary file. primaryfields A null-terminated string containing the name of the primary key field for the primary file. The value “__fileposition__” indicates the secondary is an INDEX that must use FETCH to access non-keyed fields. If omitted, the default is an empty string. secondaryfields A null-terminated string containing the name of the foreign key field relating to the primary file. If omitted, the default is an empty string. relationship A null-terminated string containing either “link” or “view” indicating the type of relationship between the primary and secondary files. If omitted, the default is “link.” Return: FileRelationshipList returns a dataset in the FsFileRelationshipRecord format. The FileRelationshipList function returns a list file relationships between the primary and secondary files. The return records are structured in the FsFileRelationshipRecord format: EXPORT FsFileRelationshipRecord := RECORD STRING primaryfile {MAXLENGTH(1023)}; STRING secondaryfile {MAXLENGTH(1023)}; STRING primaryflds {MAXLENGTH(1023)}; STRING secondaryflds {MAXLENGTH(1023)}; STRING kind {MAXLENGTH(16)}; STRING cardinality {MAXLENGTH(16)}; BOOLEAN payload; STRING description {MAXLENGTH(1023)}; END; Example: OUTPUT(fileServices.FileRelationshipList('names', 'inquiries')); See Also: AddFileRelationship FileServices.RemoveFileRelationship( primary, secondary, [, primaryfields ] [, secondaryfields ] [, relationship ]); primary A null-terminated string containing the logical filename of the primary file. secondary A null-terminated string containing the logical filename of the secondary file. primaryfields A null-terminated string containing the name of the primary key field for the primary file. The value “__fileposition__” indicates the secondary is an INDEX that must use FETCH to access non-keyed fields. If omitted, the default is an empty string. secondaryfields A null-terminated string containing the name of the foreign key field relating to the primary file. If omitted, the default is an empty string. relationship A null-terminated string containing either “link” or “view” indicating the type of relationship between the primary and secondary files. If omitted, the default is “link.” The RemoveFileRelationship function removes a file relationshuip between the primary and secondary files. Example: fileServices.RemoveFileRelationship('names', 'inquiries'); See Also: AddFileRelationship FileServices.MonitorFile( event, [ ip ] , filename, [,subdirs] [,shotcount] [,espserverIPport] )dfuwuid := FileServices.fMonitorFile( event, [ ip ] , filename, [,subdirs] [,shotcount] [,espserverIPport] ); event A null-terminated string containing the user-defined name of the event to fire when the filename appears. This value is used as the first parameter to the EVENT function. ip Optional. A null-terminated string containing the ip address for the file to monitor. This is typically a landing zone. This may be omitted only if the filename parameter contains a complete URL. filename A null-terminated string containing the full path to the file to monitor. This may contain wildcard characters (* and ?). subdirs Optional. A boolean value indicating whether to include files in sub-directories that match the wildcard mask when the filename contains wildcards. If omitted, the default is false. shotcount Optional. An integer value indicating the number of times to generate the event before the monitoring job completes. A negative one (-1) value indicates the monitoring job continues until manually aborted. If omitted, the default is 1. espserverIPport Optional. A null-terminated string containing the protocol, IP, port, and directory, or the DNS equivalent, of the ESP server program. This is usually the same IP and port as ECL Watch, with “/FileSpray” appended. If omitted, the default is the value contained in the lib_system.ws_fs_server attribute. dfuwuid The attribute name to recieve the null-terminated string containing the DFU workunit ID (DFUWUID) generated for the monitoring job. Return: fMonitorFile returns a null-terminated string containing the DFU workunit ID (DFUWUID). The MonitorFile function creates a file monitor job in the DFU Server. Once the job is received it goes into a 'monitoring' mode (which can be seen in the eclwatch DFU Workunit display), which polls at a fixed interval (settable by configenv but default 15 mins). If an appropriately named file arrives in this interval it will fire the event with the name of the triggering object as the event subtype (see the EVENT function). This process continues until either: 1) The shotcount number of events have been generated. 2) The user aborts the DFU workunit. The FileServices.AbortDfuWorkunit and FileServices.WaitDfuWorkunit functions can be used to abort or wait for the DFU job by passing them the returned dfuwuid. Note the following caveats and restrictions: 1) Events are only generated when the monitor job starts or subsequently on the polling interval. 2) Note that the event is generated if the file has been created since the last polling interval.-Therefore, the event may occur before the file is closed and the data all written. To ensure the file is not subsequently read before it is complete you should use a technique that will preclude this possibillity, such as using a separate 'flag' file instead of the file, itself or renaming the file once it has been created and completely written. 3) The EVENT function’s subtype parameter (its 2nd parameter) when monitoring physical files is the full URL of the file, with an absolute IP rather than DNS/netbios name of the file. This parameter cannot be retrieved but can only be used for matching a particular value. Example: EventName := 'MyFileEvent'; FileName := 'c:\\test\\myfile'; LZ := '10.150.50.14'; FileServices.MonitorFile(EventName,LZ,FileName); OUTPUT('File Found') : WHEN(EVENT(EventName,'*'),COUNT(1)); FileServices.MonitorLogicalFileName( event, filename, [, shotcount ] [, espserverIPport ] )dfuwuid := FileServices.fMonitorLogicalFileName( event, filename, [, shotcount ] [, espserverIPport ] ); event A null-terminated string containing the user-defined name of the event to fire when the filename appears. This value is used as the first parameter to the EVENT function. filename A null-terminated string containing the name of the logical file in the DFU to monitor. This may contain wildcard characters ( * and ?). shotcount Optional. An integer value indicating the number of times to generate the event before the monitoring job completes. A negative one (-1) value indicates the monitoring job continues until manually aborted. If omitted, the default is 1. espserverIPport Optional. A null-terminated string containing the protocol, IP, port, and directory, or the DNS equivalent, of the ESP server program. This is usually the same IP and port as ECL Watch, with “/FileSpray” appended. If omitted, the default is the value contained in the lib_system.ws_fs_server attribute. dfuwuid The attribute name to recieve the null-terminated string containing the DFU workunit ID (DFUWUID) generated for the monitoring job. Return: fMonitorLogicalFileName returns a null-terminated string containing the DFU workunit ID (DFUWUID). The MonitorLogicalFileName function creates a file monitor job in the DFU Server. Once the job is received it goes into a 'monitoring' mode (which can be seen in the eclwatch DFU Workunit display), which polls at a fixed interval (settable by configenv but default 15 mins). If an appropriately named file arrives in this interval it will fire the event with the name of the triggering object as the event subtype (see the EVENT function). This process continues until either: 1) The shotcount number of events have been generated. 2) The user aborts the DFU workunit. The FileServices.AbortDfuWorkunit and FileServices.WaitDfuWorkunit functions can be used to abort or wait for the DFU job by passing them the returned dfuwuid. Note the following caveats and restrictions: 1) If a matching file already exists when the DFU Monitoring job is started, that file will not generate an event. It will only generate an event once the file has been deleted and recreated. 2) If a file is created and then deleted (or deleted then re-created) between polling intervals, it will not be seen by the monitor and will not trigger an event. 3) Events are only generated on the polling interval. Example: EventName := 'MyFileEvent'; FileName := 'test::myfile'; IF (FileServices.FileExists(FileName), FileServices.DeleteLogicalFile(FileName)); FileServices.MonitorLogicalFileName(EventName,FileName); OUTPUT('File Created') : WHEN(EVENT(EventName,'*'),COUNT(1)); rec := RECORD STRING10 key; STRING10 val; END; afile := DATASET([{ 'A', '0'}], rec); OUTPUT(afile,,FileName); FileServices.AbortDfuWorkunit( dfuwuid [,espserverIPport ] ) dfuwuid A null-terminated string containing the DFU workunit ID (DFUWUID) for the job to abort. This value is returned by the “leading-f” versions of the Copy, DKC, SprayFixed, SprayVariable, SprayXML, and Despray FileServices functions. espserverIPport Optional. A null-terminated string containing the protocol, IP, port, and directory, or the DNS equivalent, of the ESP server program. This is usually the same IP and port as ECL Watch, with “/FileSpray” appended. If omitted, the default is the value contained in the lib_system.ws_fs_server attribute. The AbortDfuWorkunit function aborts the specified DFU workunit. Typically that workunit will have been launched with its timeout parameter set to zero (0). Example: FileServices.AbortDfuWorkunit('D20051108-143758'); FileServices.Copy( logicalname, destinationGroup, destinationLogicalname, [,scrDali] [,timeout] [,espserverIPport ] [,maxConnections] [,allowoverwrite] [,replicate] [,asSuperfile] );dfuwuid := FileServices.fCopy( logicalname, destinationGroup, destinationLogicalname, [,scrDali] [,timeout] [,espserverIPport ] [,maxConnections] [,allowoverwrite] [,replicate] [,asSuperfile] ); logicalname A null-terminated string containing the logical name of the file. destinationGroup A null-terminated string containing the destination cluster for the file. destinationLogicalnameA null-terminated string containing the new logical name of the file. srcDali Optional. A null-terminated string containing the IP and Port of the Dali containing the file to copy. If omitted, the default is an intra-Dali copy. timeout Optional. An integer value indicating the timeout setting. If omitted, the default is -1. If set to zero (0), execution control returns immediately to the ECL workunit without waiting for the DFU workunit to complete. espserverIPport Optional. A null-terminated string containing the protocol, IP, port, and directory, or the DNS equivalent, of the ESP server program. This is usually the same IP and port as ECL Watch, with “/FileSpray” appended. If omitted, the default is the value contained in the lib_system.ws_fs_server attribute. maxConnections Optional. An integer specifying the maximum number of connections. If omitted, the default is one (1). allowoverwrite Optional. A boolean TRUE or FALSE flag indicating whether to allow the new file to overwrite an existing file of the same name. If omitted, the default is FALSE. replicate Optional. A boolean TRUE or FALSE flag indicating whether to automatically replicate the new file. If omitted, the default is FALSE. asSuperfile Optional. A boolean TRUE or FALSE flag indicating whether to treat the file as a superfile. If omitted, the default is FALSE. If TRUE and the file to copy is a superfile, then the operation creates a superfile on the target, creating subfiles as needed while overwriting only those already existing subfiles whose content has changed. If FALSE and the file to copy is a superfile, then the operation consolidates all the superfile content into a single logical file on the target, not a superfile. dfuwuid The attribute name to recieve the null-terminated string containing the DFU workunit ID (DFUWUID) generated for the job. Return: fCopy returns a null-terminated string containing the DFU workunit ID (DFUWUID). The Copy function takes a logical file and copies it to another logical file. This may be done within the same cluster, or to another cluster, or to a cluster in a completely separate Dali. Example: FileServices.Copy('OUT::MyFile','400way','OUT::MyNewFile'); FileServices.DeSpray( logicalname, destinationIP , destinationpath, [,timeout] [,espserverIPport ] [,maxConnections] [,allowoverwrite])dfuwuid := FileServices.fDeSpray( logicalname, destinationIP , destinationpath, [,timeout] [,espserverIPport ] [,maxConnections] [,allowoverwrite]); logicalname A null-terminated string containing the logical name of the file. destinationIP A null-terminated string containing the destination IP address of the file. destinationpath A null-terminated string containing the path and name of the file. timeout Optional. An integer value indicating the timeout setting. If omitted, the default is -1. If set to zero (0), execution control returns immediately to the ECL workunit without waiting for the DFU workunit to complete. espserverIPport Optional. A null-terminated string containing the protocol, IP, port, and directory, or the DNS equivalent, of the ESP server program. This is usually the same IP and port as ECL Watch, with “/FileSpray” appended. If omitted, the default is the value contained in the lib_system.ws_fs_server attribute. maxConnections Optional. An integer specifying the maximum number of connections. If omitted, the default is one (1). allowoverwrite Optional. A boolean TRUE or FALSE flag indicating whether to allow the new file to overwrite an existing file of the same name. If omitted, the default is FALSE. dfuwuid The attribute name to recieve the null-terminated string containing the DFU workunit ID (DFUWUID) generated for the job. Return: fDeSpray returns a null-terminated string containing the DFU workunit ID (DFUWUID). The DeSpray function takes a logical file and desprays it (combines all parts on each supercomputer node into a single physical file) to the landing zone. Example: FileServices.DeSpray('OUT::MyFile', '10.150.50.14', 'c:\\OutputData\\MyFile.txt', -1, 'http://10.150.50.12:8010/FileSpray'); FileServices.DKC( logicalname, destinationIP , destinationpath, [,timeout] [,espserverIPport ] [,maxConnections] [,allowoverwrite])dfuwuid := FileServices.fDKC( logicalname, destinationIP , destinationpath, [,timeout] [,espserverIPport ] [,maxConnections] [,allowoverwrite]); logicalname A null-terminated string containing the logical name of the file. destinationIP A null-terminated string containing the destination IP address of the file. destinationpath A null-terminated string containing the path and name of the file. timeout Optional. An integer value indicating the timeout setting. If omitted, the default is -1. If set to zero (0), execution control returns immediately to the ECL workunit without waiting for the DFU workunit to complete. espserverIPport Optional. A null-terminated string containing the protocol, IP, port, and directory, or the DNS equivalent, of the ESP server program. This is usually the same IP and port as ECL Watch, with “/FileSpray” appended. If omitted, the default is the value contained in the lib_system.ws_fs_server attribute. maxConnections Optional. An integer specifying the maximum number of connections. If omitted, the default is one (1). allowoverwrite Optional. A boolean TRUE or FALSE flag indicating whether to allow the new file to overwrite an existing file of the same name. If omitted, the default is FALSE. dfuwuid The attribute name to recieve the null-terminated string containing the DFU workunit ID (DFUWUID) generated for the job. Return: fDKC returns a null-terminated string containing the DFU workunit ID (DFUWUID). The DKC function takes an INDEX file and desprays it (combines all parts on each supercomputer node into a single physical file) to the landing zone. Example: FileServices.DKC('OUT::MyIndex', '10.150.50.14', 'c:\\OutputData\\MyIndex.txt', -1, 'http://10.150.50.12:8010/FileSpray'); FileServices.RemotePull( remoteURL, sourcelogicalname, destinationGroup, destinationlogicalname, [,timeout] [,maxConnections] [,allowoverwrite] [,replicate] [,asSuperfile])dfuwuid := FileServices.RemotePull( remoteURL, sourcelogicalname, destinationGroup , destinationlogicalname, [,timeout] [,maxConnections] [,allowoverwrite] [,replicate] [,asSuperfile]); remoteURL A null-terminated string containing the protocol, IP, port, and directory, or the DNS equivalent, of the remote ESP server program. This is usually the same IP and port as its ECL Watch, with “/FileSpray” appended. sourcelogicalname A null-terminated string containing the local logical name of the file. destinationGroup A null-terminated string containing the name of the destination cluster. destinationlogicalname A null-terminated string containing the logical name to give the file on the remote cluster (this must be completely specified, including the domain). timeout Optional. An integer value indicating the timeout setting. If omitted, the default is -1. If set to zero (0), execution control returns immediately to the ECL workunit without waiting for the DFU workunit to complete. maxConnections Optional. An integer specifying the maximum number of connections. If omitted, the default is one (1). allowoverwrite Optional. A boolean TRUE or FALSE flag indicating whether to allow the new file to overwrite an existing file of the same name. If omitted, the default is FALSE. replicate Optional. A boolean TRUE or FALSE flag indicating whether to automatically replicate the new file. If omitted, the default is FALSE. asSuperfile Optional. A boolean TRUE or FALSE flag indicating whether to treat the file as a superfile. If omitted, the default is FALSE. If TRUE and the file to copy is a superfile, then the operation creates a superfile on the target, creating subfiles as needed while overwriting only those already existing subfiles whose content has changed. If FALSE and the file to copy is a superfile, then the operation consolidates all the superfile content into a single logical file on the target, not a superfile. dfuwuid The attribute name to recieve the null-terminated string containing the DFU workunit ID (DFUWUID) generated for the job. Return: fRemotePull returns a null-terminated string containing the DFU workunit ID (DFUWUID). The RemotePull function executes on the remoteURL, copying the sourcelogicalname from the local environment that instantiated the operation to the remote environment’s destinationGroup cluster, giving it the destinationlogicalname. This is very similar to using the FileServices.Copy function and specifying its espserverIPport parameter. Since the DFU workunit executes on the remote DFU server, the user name authentication must be the same on both systems, and the use must have rights to copy files on both systems. Example: FileServices.RemotePull('http://10.150.50.14:8010/FileSpray', '~THOR::LOCAL::MyFile', 'RemoteThor', '~REMOTETHOR::LOCAL::MyFile'); FileServices.SprayFixed( sourceIP , sourcepath, recordsize, destinationgroup, destinationlogicalname [,timeout] [, espserverIPport ] [, maxConnections ] [, allowoverwrite ] [, replicate ] [, compress ])dfuwuid := FileServices.fSprayFixed( sourceIP , sourcepath, recordsize, destinationgroup, destinationlogicalname [,timeout] [, espserverIPport ] [, maxConnections ] [, allowoverwrite ] [, replicate ] [, compress ]); sourceIP A null-terminated string containing the IP address of the file. sourcepath A null-terminated string containing the path and name of the file. recordsize An integer containing the size of the records in the file. destinationgroup A null-terminated string containing the name of the specific supercomputer within the target cluster. destinationlogicalnameA null-terminated string containing the logical name of the file. timeout Optional. An integer value indicating the timeout setting. If omitted, the default is -1. If set to zero (0), execution control returns immediately to the ECL workunit without waiting for the DFU workunit to complete. espserverIPport A null-terminated string containing the protocol, IP, port, and directory, or the DNS equivalent, of the ESP server program. This is usually the same IP and port as ECL Watch, with “/FileSpray” appended. If omitted, the default is the value contained in the lib_system.ws_fs_server attribute. maxConnections Optional. An integer specifying the maximum number of connections. If omitted, the default is one (1). allowoverwrite Optional. A boolean TRUE or FALSE flag indicating whether to allow the new file to overwrite an existing file of the same name. If omitted, the default is FALSE. replicate Optional. A boolean TRUE or FALSE flag indicating whether to replicate the new file. If omitted, the default is FALSE. compress Optional. A boolean TRUE or FALSE flag indicating whether to compress the new file. If omitted, the default is FALSE. dfuwuid The attribute name to recieve the null-terminated string containing the DFU workunit ID (DFUWUID) generated for the job. Return: fSprayFixed returns a null-terminated string containing the DFU workunit ID (DFUWUID). The SprayFixed function takes fixed-format file from the landing zone and distributes it across the nodes of the destination supercomputer. Example: FileServices.SprayFixed('10.150.50.14','c:\\InputData\\MyFile.txt', 255,'400way','IN::MyFile',-1, 'http://10.150.50.12:8010/FileSpray'); FileServices.SprayVariable( sourceIP , sourcepath , [ maxrecordsize ] , [ srcCSVseparator ] , [ srcCSVterminator ] , [ srcCSVquote ] , destinationgroup, destinationlogicalname [,timeout] [,espserverIPport] [,maxConnections] [,allowoverwrite] [,replicate] [, compress ])dfuwuid := FileServices.fSprayVariable( sourceIP , sourcepath , [ maxrecordsize ] , [ srcCSVseparator ] , [ srcCSVterminator ] , [ srcCSVquote ] , destinationgroup, destinationlogicalname [,timeout] [,espserverIPport] [,maxConnections] [,allowoverwrite] [,replicate] [, compress ]); sourceIP A null-terminated string containing the IP address of the file. sourcepath A null-terminated string containing the path and name of the file. maxrecordsize Optional. An integer containing the maximum size of the records in the file. If omitted, the default is 4096. srcCSVseparator Optional. A null-terminated string containing the CSV field separator. If omitted, the default is '\\,' srcCSVterminatorOptional. A null-terminated string containing the CSV record separator. If omitted, the default is '\\n,\\r,\\n' srcCSVquote Optional. A null-terminated string containing the CSV quoted field delimiter. If omitted, the default is '\'' destinationgroup A null-terminated string containing the name of the specific supercomputer within the target cluster. destinationlogicalnameA null-terminated string containing the logical name of the file. timeout Optional. An integer value indicating the timeout setting. If omitted, the default is -1. If set to zero (0), execution control returns immediately to the ECL workunit without waiting for the DFU workunit to complete. espserverIPport Optional. A null-terminated string containing the protocol, IP, port, and directory, or the DNS equivalent, of the ESP server program. This is usually the same IP and port as ECL Watch, with “/FileSpray” appended. If omitted, the default is the value in the lib_system.ws_fs_server attribute. maxConnections Optional. An integer specifying the maximum number of connections. If omitted, the default is one (1). allowoverwrite Optional. A boolean TRUE or FALSE flag indicating whether to allow the new file to overwrite an existing file of the same name. If omitted, the default is FALSE. replicate Optional. A boolean TRUE or FALSE flag indicating whether to replicate the new file. If omitted, the default is FALSE. compress Optional. A boolean TRUE or FALSE flag indicating whether to compress the new file. If omitted, the default is FALSE. dfuwuid The attribute name to recieve the null-terminated string containing the DFU workunit ID (DFUWUID) generated for the job. Return: fSprayVariable returns a null-terminated string containing the DFU workunit ID (DFUWUID). The SprayVariable function takes a variable length file from the landing zone and distributes it across the nodes of the destination supercomputer. Example: FileServices.SprayVariable('10.150.50.14', 'c:\\InputData\\MyFile.txt', ,,,, '400way', 'IN::MyFile', -1, 'http://10.150.50.12:8010/FileSpray'); FileServices.SprayXML( sourceIP , sourcepath , [ maxrecordsize ] , srcRowTag , [ srcEncoding ] , destinationgroup, destinationlogicalname [,timeout] [,espserverIPport] [,maxConnections] [,allowoverwrite] [,replicate] [, compress ])dfuwuid := FileServices.fSprayXML( sourceIP, sourcepath, [ maxrecordsize ] , srcRowTag , [ srcEncoding ] ,destinationgroup, destinationlogicalname [,timeout] [,espserverIPport] [,maxConnections] [,allowoverwrite] [,replicate] [, compress ]); sourceIP A null-terminated string containing the IP address of the file. sourcepath A null-terminated string containing the path and name of the file. maxrecordsize Optional. An integer containing the maximum size of the records in the file. If omitted, the default is 8192. srcRowTag A null-terminated string containing the row delimiting XML tag. srcEncoding Optional. A null-terminated string containing the encoding. If omitted, the default is 'utf8' destinationgroup A null-terminated string containing the name of the specific supercomputer within the target cluster. destinationlogicalnameA null-terminated string containing the logical name of the file. timeout Optional. An integer value indicating the timeout setting. If omitted, the default is -1. If set to zero (0), execution control returns immediately to the ECL workunit without waiting for the DFU workunit to complete. espserverIPport Optional. A null-terminated string containing the protocol, IP, port, and directory, or the DNS equivalent, of the ESP server program. This is usually the same IP and port as ECL Watch, with “/FileSpray” appended. If omitted, the default is the value contained in the lib_system.ws_fs_server attribute. maxConnections Optional. An integer specifying the maximum number of connections. If omitted, the default is one (1). allowoverwrite Optional. A boolean TRUE or FALSE flag indicating whether to allow the new file to overwrite an existing file of the same name. If omitted, the default is FALSE. replicate Optional. A boolean TRUE or FALSE flag indicating whether to replicate the new file. If omitted, the default is FALSE. compress Optional. A boolean TRUE or FALSE flag indicating whether to compress the new file. If omitted, the default is FALSE. dfuwuid The attribute name to recieve the null-terminated string containing the DFU workunit ID (DFUWUID) generated for the job. Return: fSprayXML returns a null-terminated string containing the DFU workunit ID (DFUWUID). The SprayXML function takes a well-formed XML file from the landing zone and distributes it across the nodes of the destination supercomputer, producing a well-formed XML file on each node. Example: FileServices.SprayXML('10.150.50.14','c:\\InputData\\MyFile.txt',, 'Row',,'400way','IN::MyFile',-1, 'http://10.150.50.12:8010/FileSpray'); FileServices.WaitDfuWorkunit( dfuwuid [,timeout] [,espserverIPport ] ) dfuwuid A null-terminated string containing the DFU workunit ID (DFUWUID) for the job to wait for. This value is returned by the “leading-f” versions of the Copy, DKC, SprayFixed, SprayVariable, SprayXML, and Despray FileServices functions. timeout Optional. An integer value indicating the timeout setting. If omitted, the default is -1. If set to zero (0), execution control returns immediately to the ECL workunit without waiting for the DFU workunit to complete. espserverIPport Optional. A null-terminated string containing the protocol, IP, port, and directory, or the DNS equivalent, of the ESP server program. This is usually the same IP and port as ECL Watch, with “/FileSpray” appended. If omitted, the default is the value contained in the lib_system.ws_fs_server attribute. Return: WaitDfuWorkunit returns a null-terminated string containing the final status string of the DFU workunit (such as: scheduled, queued, started, aborted, failed, finished, or monitoring). The WaitDfuWorkunit function waits for the specified DFU workunit to finish. Typically that workunit will have been launched with its timeout parameter set to zero (0). Example: FileServices.WaitDfuWorkunit('D20051108-143758'); FileServices.CreateSuperFile( superfile [, sequentialflag ] [, ifdoesnotexist ] ) superfile A null-terminated string containing the logical name of the superfile. sequentialflag Optional. A boolean value indicating whether to the sub-files must be sequentially ordered. If omitted, the default is FALSE. ifdoesnotexist Optional. A boolean value indicating whether to post an error if the superfile already exists. If TRUE, no error is posted. If omitted, the default is FALSE. Return: Null. The CreateSuperFile function creates an empty superfile. This function is not included in a superfile transaction. The sequentialflag parameter set to TRUE governs the unusual case where the logical numbering of sub-files must be sequential (for example, where all sub-files are already globally sorted). With sequentialflag FALSE (the default) the subfile parts are interleaved so the parts are found locally. For example, if on a 4-way cluster there are 3 files (A, B, and C) then the parts are as follows: A._1_of_4, B._1_of_4, and C_1_of_4 are on node 1 A._2_of_4, B._2_of_4, and C_2_of_4 are on node 2 A._3_of_4, B._3_of_4, and C_3_of_4 are on node 3 A._4_of_4, B._4_of_4, and C_4_of_4 are on node 4 Reading the superfile created with sequentialflag FALSE on Thor will read the parts in the order: [A1,B1,C1,] [A2,B2,C2,] [A3,B3,C3,] [A4,B4,C4] so the reads will all be local (i.e. A1,B1,C1 on node 1 etc). Setting sequentialflag to TRUE will read the parts in subfile order, like this: [A1,A2,A3,] [A4,B1,B2] [,B3,B4,C1,] [C2,C3,C4] so that the global order of A,B,C,D is maintained. However, the parts cannot all be read locally (e.g. A2 and A3 will be read on part 1). Because of this it is much less efficient to set sequentialflag true, and as it is unusual anyway to have files that are partitioned in order, it becomes a very unusual option to set. Example: FileServices.CreateSuperFile('~CLASS::RT::IN::SF1'); FileServices.SuperFileExists( filename ) filename A null-terminated string containing the logical name of the superfile. Return: SuperFileExists returns a BOOLEAN value. The SuperFileExists function returns TRUE if the specified filename is present in the Distributed File Utility (DFU) and is a SuperFile. It returns FALSE if the filename does exist in the DFU but is not a SuperFile (i.e. is a normal DATASET—use the FileServices.FileExists function to detect their presence or absence). This function is not included in a superfile transaction. Example: A := FileServices.SuperFileExists('~CLASS::RT::IN::SF1'); FileServices.DeleteSuperFile( superfile [, subdeleteflag ] ) superfile A null-terminated string containing the logical name of the superfile. subdeleteflag A boolean value indicating whether to delete the sub-files. If omitted, the default is FALSE. This option should not be used if the superfile contains any foreign file or foreign superfile. Return: Null. The DeleteSuperFile function deletes the superfile. This function is not included in a superfile transaction. Example: FileServices.DeleteSuperFile('~CLASS::RT::IN::SF1'); FileServices.GetSuperFileSubCount( superfile ) superfile A null-terminated string containing the logical name of the superfile. Return: GetSuperFileSubCount returns an INTEGER4 value. The GetSuperFileSubCount function returns the number of sub-files comprising the superfile. This function is not included in a superfile transaction. Example: A := FileServices.GetSuperFileSubCount('~CLASS::RT::IN::SF1'); FileServices.GetSuperFileSubName( superfile, subfile [,absolutepath] ) superfile A null-terminated string containing the logical name of the superfile. subfile An integer in the range of one (1) to the total number of sub-files in the superfile specifying the ordinal position of the sub-file whose name to return. absolutepath Optional. A boolean TRUE/FALSE to indicate whether to prepend a tilde (~) to the resulting foreign logical file name. If omitted, the default is FALSE. Return: GetSuperFileSubName returns a VARSTRING value. The GetSuperFileSubName function returns the logical name of the specified subfile in the superfile. This function is not included in a superfile transaction. Example: A := FileServices.GetSuperFileSubName('~CLASS::RT::IN::SF1', 1); //get name of first sub-file //this example gets the name of the first sub-file in // a foreign superfile sf := '~thor_data400::BASE::Business_Header'; sub := FileServices.GetSuperFileSubName( FileServices.ForeignLogicalFileName (sf, '10.150.29.161', TRUE), 1,TRUE); OUTPUT(FileServices.ForeignLogicalFileName(sub,'')); FileServices.LogicalFileSuperOwners( filename ) filename A null-terminated string containing the logical name of the file. Return: LogicalFileSuperOwners returns a dataset in the following format: EXPORT FsLogicalFileNameRecord := RECORD STRING name; END; The LogicalFileSuperOwners function returns a list of the logical filenames of all the SuperFiles that contain the filename as a sub-file. This function is not included in a superfile transaction. Example: OUTPUT(FileServices.LogicalFileSuperowners('~CLASS::RT::SF::Daily1')); //returns all SuperFiles that “own” the Daily1 file FileServices.LogicalFileSuperSubList( ) Return: LogicalFileSuperSubList returns a dataset in the following format: EXPORT FsLogicalSuperSubRecord := RECORD STRING supername{MAXLENGTH(255)}; STRING subname{MAXLENGTH(255)}; END; The LogicalFileSuperSubList function returns a list of the logical filenames of all the SuperFiles and their component sub-files. This function is not included in a superfile transaction. Example: OUTPUT(FileServices.LogicalFileSuperSubList()); //returns all SuperFiles and their sub-files FileServices.SuperFileContents( filename [, recurse ] ) filename A null-terminated string containing the logical name of the SuperFile. recurse A boolean flag indicating whether to expand nested SuperFiles withinthe filename so that only logical files are returned. If omitted, the default is FALSE. Return: SuperFileContents returns a dataset in the following format: EXPORT FsLogicalFileNameRecord := RECORD STRING name; END; The SuperFileContents function returns a list of the logical filenames of all the sub-files in the filename. This function is not included in a superfile transaction. Example: OUTPUT(FileServices.SuperFileContents('~CLASS::RT::SF::Daily')); //returns all files in the SuperFile FileServices.FindSuperFileSubName( superfile, subfile ) superfile A null-terminated string containing the logical name of the superfile. subfile A null-terminated string containing the logical name of the sub-file. Return: FindSuperFileSubName returns an INTEGER4 value. The FindSuperFileSubName function returns the ordinal position of the specified subfile in the superfile. This function is not included in a superfile transaction. Example: A := FileServices.GetSuperFileSubName('~CLASS::RT::IN::SF1', 'Sue'); //get position of sub-file Sue FileServices.StartSuperFileTransaction( ) Return: Null. The StartSuperFileTransaction function begins a transaction frame for superfile maintenance. The transaction frame is terminated by calling the FinishSuperFileTransaction function. Within the transaction frame, multiple superfiles may be maintained by adding, removing, clearing, swapping, and replacing sub-files. The FinishSuperFileTransaction function does an automatic rollback of the transaction if any error or failure occurs during the transaction frame. If no error occurs, then the commit or rollback of the transaction is controlled by the rollback parameter to the FinishSuperFileTransaction function. Example: FileServices.StartSuperFileTransaction(); FileServices.AddSuperFile( superfile, subfile [, atpos ] [, addcontent ] [,strict ]) superfile A null-terminated string containing the logical name of the superfile. subfile A null-terminated string containing the logical name of the sub-file. Ths may be another superfile. atpos An integer specifying the position of the subfile in the superfile. If omitted, the default is zero (0), which places the subfile at the end of the superfile. addcontent A boolean flag specifying whether the contents of a subfile that is itself a superfile are added to the superfile by copying or by reference. If omitted, the default is to add by reference. strict A boolean flag specifying, in the case of a subfile that is itself a superfile, whether to check for the existence of the superfile and raising an error if it does not. Also, if addcontent is set to TRUE, it will ensure the subfile that is itself a superfile is not empty. If omitted, the default is false. Return: Null. The AddSuperFile function adds the subfile to the list of files comprising the superfile. All subfiles in the superfile must have exactly the same structure type and format. This function may be included in a superfile transaction. Example: SEQUENTIAL( FileServices.StartSuperFileTransaction(), FileServices.AddSuperFile('MySuperFile','MySubFile'), FileServices.FinishSuperFileTransaction() ); FileServices.RemoveSuperFile( superfile, subfile [, delete] [, removecontents]) superfile A null-terminated string containing the logical name of the superfile. subfile A null-terminated string containing the logical name of the sub-file. This may be another superfile or a foreign file or superfile. delete A boolean flag specifying whether to delete the subfile from disk or just remove it from the superfile list of files. If omitted, the default is to just remove it from the superfile list of files. This option should not be used if the subfile is a foreign file or foreign superfile. removecontents A boolean flag specifying whether the contents of a subfile that is itself a superfile are recursively removed. Return: Null. The RemoveSuperFile function removes the subfile from the list of files comprising the superfile. This function may be included in a superfile transaction. Example: SEQUENTIAL( FileServices.StartSuperFileTransaction(), FileServices.RemoveSuperFile('MySuperFile','MySubFile'), FileServices.FinishSuperFileTransaction() ); FileServices.ClearSuperFile( superfile, [, delete ] ) superfile A null-terminated string containing the logical name of the superfile. delete A boolean flag specifying whether to delete the sub-files from disk or just remove them from the superfile list of files. If omitted, the default is to just remove them from the superfile list of files. Return: Null. The ClearSuperFile function removes all sub-files from the list of files comprising the superfile. This function may be included in a superfile transaction. Example: SEQUENTIAL( FileServices.StartSuperFileTransaction(), FileServices.ClearSuperFile('MySuperFile'), FileServices.FinishSuperFileTransaction() ); FileServices.SwapSuperFile( superfile1, superfile2 ) superfile1 A null-terminated string containing the logical name of the superfile. superfile2 A null-terminated string containing the logical name of the superfile. Return: Null. The SwapSuperFile function moves all sub-files from superfile1 to superfile2 and vice versa. This function may be included in a superfile transaction. Example: SEQUENTIAL( FileServices.StartSuperFileTransaction(), FileServices.SwapSuperFile('MySuperFile','YourSuperFile'), FileServices.FinishSuperFileTransaction() ); FileServices.ReplaceSuperFile( superfile, subfile1 , subfile2 ) superfile A null-terminated string containing the logical name of the superfile. subfile1 A null-terminated string containing the logical name of the sub-file. Ths may be another superfile. subfile2 A null-terminated string containing the logical name of the sub-file. Ths may be another superfile. Return: Null. The ReplaceSuperFile function removes the subfile1 from the list of files comprising the superfile and replaces it with subfile2. This function may be included in a superfile transaction. Example: SEQUENTIAL( FileServices.StartSuperFileTransaction(), FileServices.ReplaceSuperFile('MySuperFile', 'MyOldSubFile', 'MyNewSubFile'), FileServices.FinishSuperFileTransaction() ); FileServices.PromoteSuperFileList( superfiles [, addhead ] [, deltail ] [, createjustone ] [, reverse ] )oldlist := FileServices.fPromoteSuperFileList( superfiles [, addhead ] [, deltail ] [, createjustone ] [, reverse ] ); superfiles A set of null-terminated strings containing the logical names of the superfiles to act on. Any that don’t exist will be created. The contents of each superfile will be moved to the next in the list (NB -- each superfile must contain different sub-files). addhead Optional. A null-terminated string containing a comma-delimited list of logical file names to add to the first superfile after the promotion process is complete. deltail Optional. A boolean value specifying whether to physically delete the contents moved out of the last superfile. If omitted, the default is FALSE. createjustone Optional. A boolean value specifying whether to only create a single superfile (truncate the list at the first non-existent superfile). If omitted, the default is FALSE. reverse Optional. A boolean value specifying whether to reverse the order of procesing the superfiles list, effectively “demoting” instead of “promoting” the sub-files. If omitted, the default is FALSE. oldlist The name of the attribute that receives the returned string containing the list of the previous subfile contents of the emptied superfile. Return: PromoteSupeFileList returns Null; fPromoteSupeFileList returns a string. The PromoteSuperFileList function moves the subfiles from the first entry in the list of superfiles to the next in the list, subsequently repeating the process through the list of superfiles. This function does not use superfile transactions, it is an atomic operation. Example: FileServices.PromoteSuperFileList(['Super1','Super2','Super3'], 'NewSub1'); //Moves what was in Super1 to Super2, // what was in Super2 to Super3, replacing what was in Super3, // and putting NewSub1 in Super1 FileServices.FinishSuperFileTransaction( [ rollback ] ) rollback Optional. A boolean flag that indicates whether to commit (FALSE) or roll back (TRUE) the transaction. If omitted, the default is FALSE. Return: Null. The FinishSuperFileTransaction function terminates a superfile maintenance transaction frame. If the rollback flag is FALSE, the transction is committed atomically and the transaction frame closes. Otherwise, the transaction is rolled back and the transaction frame closes. Example: FileServices.FinishSuperFileTransaction(); Thorlib.Cluster( ) Return: Cluster returns a VARSTRING value. The Cluster function returns the name of the cluster running the workunit. Not supported on Roxie clusters. This name is used by ECLplus.exe to specify the the target cluster for a workunit. Example: A := Thorlib.Cluster(); Thorlib.DaliServers( ) Return: Daliservers returns a VARSTRING value. The Daliservers function returns the IP and port of the system data store (Dali) servers for the environment running the workunit. Example: A := Thorlib.Daliservers(); Thorlib.GetEnv( name, default ) name A null-terminated string containing the name of the environment variable. default A null-terminated string containing the default value of the environment variable. Return: GetEnv returns a VARSTRING value. The GetEnv function returns the value stored in the name environment variable. If the environment variable does not exist or contains no value, the default value is returned. Example: A := Thorlib.GetEnv('SomeEnvironmentVar','Default'); Thorlib.Group( ) Return: Group returns a VARSTRING value. The Group function returns the name of the node group running the workunit. Not supported on Roxie clusters. This name is used in ECL code to specify the target CLUSTER for an OUTPUT action or a PERSISTed attribute. Example: A := Thorlib.Group(); Thorlib.JobName( ) Return: JobName returns a VARSTRING value. The JobName function returns the name of the workunit. Example: A := Thorlib.JobName(); Thorlib.JobOwner( ) Return: JobOwner returns a VARSTRING value. The JobOwner function returns the username of the person running the workunit. Example: A := Thorlib.JobOwner(); Thorlib.LogString( message ) message A string expression containinig the text to place in the log file. Return: LogString returns an INTEGER value. The LogString function outputs “USER:” followed by the message text to the eclagant or Roxie log file and returns the length of the text written to the file. Example: A := Thorlib.LogString('The text message to log'); //places USER:The text message to log //in the log file Thorlib.OS( ) Return: OS returns a VARSTRING value. The OS function returns the operating system (windows or linux) of the cluster running the workunit. Example: A := Thorlib.OS(); Thorlib.Platform( ) Return: Platform returns a VARSTRING value. The Platform function returns the platform name (hthor, thor, or roxie) of the cluster running the workunit. Example: A := Thorlib.Platform(); Thorlib.Priority( ) Return: Priority returns a VARSTRING value. The Priority function returns the priority level of the workunit. Example: A := Thorlib.Priority(); Thorlib.L2P( filename [, createflag ] ) filename A null-terminated string containing the logical name of the file. createflag A boolean value indicating whether to create the filename. If omitted, the default is FALSE. Return: L2P returns a VARSTRING value. The L2P function (Logical to Physical) returns the physical name of the file represented by the logical filename. Example: A := Thorlib.L2P('Fred'); Thorlib.Node( ) Return: Node returns an UNSIGNED INTEGER4 value. The Node function returns the (zero-based) number of the Data Refinery (THOR) or Rapid Data Delivery Engine (ROXIE) node. Example: A := Thorlib.Node(); Thorlib.Nodes( ) Return: Nodes returns an UNSIGNED INTEGER4 value. The Nodes function returns the number of nodes in the cluster. This number is the same as the CLUSTERSIZE compile time constant. The Nodes function is evaluated each time it is called, so the choice to use the function versus the constant depends upon the circumstances. Example: A := Thorlib.Nodes(); Thorlib.WUID( ) Return: WUID returns a VARSTRING value. The WUID function returns the workunit identifier of the current job. This is the same as the WORKUNIT compile time constant. The WUID function is evaluated each time it is called, so the choice to use the function versus the constant depends upon the circumstances. Example: A := Thorlib.WUID(); LIB_Word.Lead_Contains( leftstring , rightstring ) leftstring A string value. rightstring A string value. Return: Lead_Contains returns a BOOLEAN value. The Lead_Contains function returns TRUE if the leftstring and rightstring both begin with the same characters. It compares the shorter of the two parameters against the same number of characters in the longer and returns TRUE if they match. Example: Import LIB_Word; A := LIB_Word.Lead_Contains('Fred','Jo'); //A contains False -- 'Jo' != 'Fr' B := LIB_Word.Lead_Contains('Fred','Freddie'); //B contains True -- 'Fred' = 'Fred' LIB_Word.StripTail( leftstring , stripstring ) leftstring A string containing the word to strip. stripstring A string containing the characters to strip. Return: StripTail returns a STRING value. The StripTail function returns the leftstring with any stripstring characters removed from the end. Example: Import LIB_Word; A := 'FRED\'S'; // value is FRED’S B := LIB_Word.StripTail(LIB_Word.StripTail(A,'\''),'\'S'); //B contains 'FRED' stripped of the possesive form FRED'S LIB_Word.StripUpToWord( leftstring , n ) leftstring A string containing the words to strip. n An integer containing the number of words to strip. Return: StripUpToWord returns a STRING value. The StripUpToWord function returns the leftstring with n number of words removed. Example: Import LIB_Word; A := 'FRED JO ME YOU'; B := LIB_Word.StripUpToWord(A,1); //B contains 'JO ME YOU' C := LIB_Word.StripUpToWord(A,2); //C contains 'ME YOU' LIB_Word.Tails( leftstring , rightstring ) leftstring A string value. rightstring A string value. Return: Tails returns a BOOLEAN value. The Tails function returns TRUE if the leftstring ends with the same characters as the rightstring. Example: Import LIB_Word; A1 := 'FRED\'S'; // value is FRED'S A2 := 'FRED'; // value is FRED B := '\'S'; //B contains 'S C := LIB_Word.Tails(A1,B); // returns TRUE D := LIB_Word.Tails(A2,B); // returns FALSE LIB_Word.Word( leftstring , n ) leftstring A string of space-delimited words.. n A 1-byte integer value specifying which word to return. Return: Word returns a STRING value. The Word function returns the nth word from the leftstring. Example: Import LIB_Word; A := 'FRED JO ME YOU'; B := LIB_Word.Word(A,1); //B contains 'FRED' C := LIB_Word.Word(A,2); //C contains 'JO' LIB_Date.Date_I2_YYYYMM( date ) date An integer value containing a date value in the form of the number of months since 1900 (January, 1900 = 1 while January 2001 = 1213). Return: Date_I2_YYYYMM returns a 6-byte STRING value. The Date_I2_YYYYMM function returns the date integer value translated into a YYYYMM string value. Example: Import LIB_Date; A := LIB_Date.Date_I2_YYYYMM(1213); //A contains '200101' B := LIB_Date.Date_I2_YYYYMM(1); //B contains '190001' LIB_Date.Date_MMDDYY_I2( date ) date A 6-byte string value containing a date in MMDDYY format. Return: Date_MMDDYY_I2 returns an UNSIGNED INTEGER value. The Date_MMDDYY_I2 function returns the date string value translated into an integer containing the date in YYYYMMDD format. If the YY postion of the date is < 40, then the date is considered to be after 2000, otherwise the date is considered to be in the 1900’s. Therefore, the valid range of years this function can handle is 1940 - 2039. Example: Import LIB_Date; A := LIB_Date.Date_MMDDYY_I2(' 010101'); //A contains the integer value 20010101 (20,010,101) B := LIB_Date.Date_MMDDYY_I2(' 010145'); //B contains the integer value 19450101 (19,450,101) LIB_Date.Date_Overlap( leftfirst, leftlast, rightfirst, rightlast ) leftfirst An integer value containing the start date of a date range in YYYYMM format. leftlast An integer value containing the end date of a date range in YYYYMM format. rightfirst An integer value containing the start date of a date range in YYYYMM format. rightlast An integer value containing the end date of a date range in YYYYMM format. Return: Date_Overlap returns an INTEGER value. The Date_Overlap function returns the number of months of overlap between the leftfirst to leftlast and rightfirst to rightlast date ranges. Example: Import LIB_Date; A := LIB_Date.date_overlap(200001,200303,200201,200302); //A contains 13 -- the overlap from 200201 to 200302 LIB_Date.Date_Overlap_First( leftfirst, leftlast, rightfirst, rightlast ) leftfirst An integer value containing the start date of a date range in YYYYMM format. leftlast An integer value containing the end date of a date range in YYYYMM format. rightfirst An integer value containing the start date of a date range in YYYYMM format. rightlast An integer value containing the end date of a date range in YYYYMM format. Return: Date_Overlap_First returns an INTEGER value. The Date_Overlap_First function returns the starting overlap date between the leftfirst to leftlast and rightfirst to rightlast date ranges. Example: Import LIB_Date; A := LIB_Date.date_overlap_first(200001,200303,200201,200302); //A contains 200201 -- the two ranges overlap from 200201 to 200302 LIB_Date.Date_Overlap_Last( leftfirst, leftlast, rightfirst, rightlast ) leftfirst An integer value containing the start date of a date range in YYYYMM format. leftlast An integer value containing the end date of a date range in YYYYMM format. rightfirst An integer value containing the start date of a date range in YYYYMM format. rightlast An integer value containing the end date of a date range in YYYYMM format. Return: Date_Overlap_First returns an INTEGER value. The Date_Overlap_Last function returns the ending overlap date between the leftfirst to leftlast and rightfirst to rightlast date ranges. Example: Import LIB_Date; A := LIB_Date.date_overlap_last(200001,200303,200201,200302); //A contains 200302 -- the two ranges overlap from 200201 to 200302 LIB_Date.Date_YYYYMM_I2( date ) date A 6-byte STRING value containing a date in YYYYMM format. Return: Date_YYYYMM_I2 returns an INTEGER value. The Date_I2_YYYYMM function returns the date string translated into an integer representing the number of months since 1900 (January, 1900 = 1 while January 2001 = 1213). Example: Import LIB_Date; A := LIB_Date.Date_YYYYMM_I2('200101' ); //A contains 1213 B := LIB_Date.Date_YYYYMM_I2('190001' ); //B contains 1 LIB_Date.DayOfYear( year, month, day ) year An integer value containing the 4-digit year. month An integer value containing the month number. day An integer value containing the day number within the month. Return: DayOfYear returns an INTEGER value. The DayOfYear function returns the julian date for the date passed to it as the year, month, and day parameters. It does properly handle leap years. Example: Import LIB_Date; A := LIB_Date.DayOfYear(2000,1,1 ); //A contains 1 B := LIB_Date.DayOfYear(2000,3,1 ); //B contains 61 LIB_Date.DaysApart( date1, date2 ) date1 An 8-byte STRING value containing a date in YYYYMMDD format. date2 An 8-byte STRING value containing a date in YYYYMMDD format. Return: DaysApart returns an INTEGER value. The DaysApart function returns the number of days between date1 and date2. It does properly handle leap years, but only works with dates since 1900. Example: Import LIB_Date; A := LIB_Date.DaysApart('20030101' , '20030110' ); //A contains 9 LIB_Date.DaysSince1900( year, month, day ) year An integer value containing the 4-digit year. month An integer value containing the month number. day An integer value containing the day number within the month. Return: DaysSince1900 returns an INTEGER value. The DaysSince1900 function returns a julian-type date containng the number of days since 1/1/1900 and the date passed to it as the year, month, and day parameters. It does properly handle leap years. Example: Import LIB_Date; A := LIB_Date.DaysSince1900(2003,1,1); //A contains 37621 B := LIB_Date.DaysSince1900(1900,1,1); //B contains 1 LIB_Date.EarliestDate( date1, date2 ) date1 An INTEGER value containing a date. date2 An INTEGER value containing a date. Return: EarliestDate returns an INTEGER value. The EarliestDate function returns the earlier of the passed date1 and date2 parameters. Both the date1 and date2 parameters should contain values derived from similar processes. Example: Import LIB_Date; A := LIB_Date.EarliestDate(20030101 , 20030110 ); //A contains 20030101 B := LIB_Date.EarliestDate(LIB_Date.DaysSince1900(2003,1,1), LIB_Date.DaysSince1900(2003,1,10) ); //B contains 37621 // (the return value from LIB_Date.DaysSince1900(2003,1,1)) LIB_Date.EarliestDateString( date1, date2 ) date1 An STRING value containing a date. date2 An STRING value containing a date. Return: EarliestDateString returns a STRING value. The EarliestDateString function returns the earlier of the passed date1 and date2 parameters. Both the date1 and date2 parameters should be in YYYYMMDD or YYYYMM format for proper evaluation. Example: Import LIB_Date; A := LIB_Date.EarliestDateString('20030101' , '20030110' ); //A contains 20030101 LIB_Date.Format_Date( year, month, day ) year An integer value containing the 4-digit year. month An integer value containing the month number. day An integer value containing the day number within the month. Return: FormatDate returns a STRING value. The Format_Date function returns a date string in YYYY/MM/DD format. Example: Import LIB_Date; A := LIB_Date.Format_Date(2003,1,1); //A contains “2003/01/01” LIB_Date.GetAge( date ) date An 8-byte STRING value containing a date in YYYYMMDD format. Return: GetAge returns an INTEGER value. The GetAge function returns the number of years between the current system date and the passed date. Example: Import LIB_Date; A := LIB_Date.GetAge('20000101' ); //A contains 3, assuming the current date is // sometime within the 2003 calendar year LIB_Date.LatestDate( date1, date2 ) date1 An INTEGER value containing a date. date2 An INTEGER value containing a date. Return: LatestDate returns an INTEGER value. The LatestDate function returns the later of the passed date1 and date2 parameters. Both the date1 and date2 parameters should contain values derived from similar processes. Example: Import LIB_Date; A := LIB_Date.LatestDate(20030101 , 20030110 ); //A contains 20030110 B := LIB_Date.LatestDate(LIB_Date.DaysSince1900(2003,1,1), LIB_Date.DaysSince1900(2003,1,10) ); //B contains 37630 // (the return value from LIB_Date.DaysSince1900(2003,1,10)) LIB_Date.LeapYear( year ) year An integer value containing the 4-digit year. Return: LeapYear returns a BOOLEAN value. The LeapYear function returns TRUE if the passed year parameter is a leap year. Example: Import LIB_Date; A := LIB_Date.LeapYear(2000); //A contains TRUE B := LIB_Date.LeapYear(2001); //B contains FALSE AuditLib.Audit( type, message ) type A string constant containing the type of audit entry. Currently, only INFO is provided. message A string containing the audit entry text. Return: Audit returns a BOOLEAN value indicating whether it was successful or not. The Audit function writes the message into the Windows event log or Linux system log on the ECL Agent computer. The entries can be retrieved from the logs using standard operating system tools. Example: AuditLib.Audit('INFO','Audit Message'); AuditLib.Auditdata( type, message, datablock ) type A string constant containing the type of audit entry. Currently, only INFO is provided. message A string containing the audit entry text. datablock An unstructured block of data, which the application reading the log entry must know how to read. Return: AuditData returns a BOOLEAN value indicating whether it was successful or not. The AuditData function writes the message into the Windows event log or Linux system log on the ECL Agent computer. The entries can be retrieved from the logs using standard operating system tools. Example: DATA2 DataBlock := x'0D0A'; AuditLib.AuditData('INFO','Audit Message',DataBlock); ParseLib.GetParseTree( ) Return: GetParseTree returns a STRING value. The GetParseTree function returns a textual representation of the match that occurred, using square brackets (such as: a[b[c]d] ) to indicate nesting. This function is only used within the RECORD or TRANSFORM structure that defines the result of a PARSE operation. This function is useful for debugging PARSE operations. Example: r := {string150 line}; d := dataset([ {'Ge 34:2 And when Shechem the son of Hamor the Hivite, '+ 'prince of the country, saw her, he took her, and lay with her, '+ 'and defiled her.'}, {'Ge 36:10 These are the names of Esaus sons; Eliphaz the son of '+ 'Adah the wife of Esau, Reuel the son of Bashemath the wife of '+ 'Esau.'} ],r); PATTERN ws := [' ','\t',',']*; PATTERN patStart := FIRST | ws; PATTERN patEnd := LAST | ws; PATTERN article := ['A','The','Thou','a','the','thou']; TOKEN patWord := PATTERN('[a-zA-Z]+'); TOKEN Name := PATTERN('[A-Z][a-zA-Z]+'); RULE Namet := name OPT(ws 'the' ws name); PATTERN produced_by := OPT(article ws) ['son of','daughter of']; PATTERN produces_with := OPT(article ws) ['wife of']; RULE progeny := namet ws ( produced_by | produces_with ) ws namet; results := RECORD STRING LeftName := MATCHTEXT(Namet[1]); STRING RightName := MATCHTEXT(Namet[2]); STRING LinkPhrase := IF(MATCHTEXT(produced_by[1])<>'', MATCHTEXT(produced_by[1]), MATCHTEXT(produces_with[1])); STRING Tree := 'Tree: ' + parseLib.getParseTree(); END; outfile1 := PARSE(d,line,progeny,results,SCAN ALL); /* the Tree field output looks like this: Tree: [namet[name"Shechem"] ws" " produced_by"the son of" ws" " namet[name"Hamor"]] */ ParseLib.GetXMLParseTree( ) Return: GetXMLParseTree returns a STRING value. The GetXMLParseTree function returns a textual representation of the match that occurred, using XML tags to indicate nesting. This function is only used within the RECORD or TRANSFORM structure that defines the result of a PARSE operation. This function is useful for debugging PARSE operations. Example: r := {string150 line}; d := dataset([ {'Ge 34:2 And when Shechem the son of Hamor the Hivite, '+ 'prince of the country, saw her, he took her, and lay with her, '+ 'and defiled her.'}, {'Ge 36:10 These are the names of Esaus sons; Eliphaz the son of '+ 'Adah the wife of Esau, Reuel the son of Bashemath the wife of '+ 'Esau.'} ],r); PATTERN ws := [' ','\t',',']*; PATTERN patStart := FIRST | ws; PATTERN patEnd := LAST | ws; PATTERN article := ['A','The','Thou','a','the','thou']; TOKEN patWord := PATTERN('[a-zA-Z]+'); TOKEN Name := PATTERN('[A-Z][a-zA-Z]+'); RULE Namet := name OPT(ws 'the' ws name); PATTERN produced_by := OPT(article ws) ['son of','daughter of']; PATTERN produces_with := OPT(article ws) ['wife of']; RULE progeny := namet ws ( produced_by | produces_with ) ws namet; results := RECORD STRING LeftName := MATCHTEXT(Namet[1]); STRING RightName := MATCHTEXT(Namet[2]); STRING LinkPhrase := IF(MATCHTEXT(produced_by[1])<>'', MATCHTEXT(produced_by[1]), MATCHTEXT(produces_with[1])); STRING Tree := 'Tree: ' + parseLib.getXMLParseTree(); END; outfile1 := PARSE(d,line,progeny,results,SCAN ALL); /* the Tree field output looks like this: Tree: <namet> <name>Shechem</name> </namet> <ws> </ws> <produced_by>the son of</produced_by> <ws> </ws> <namet> <name>Hamor</name> </namet> */ MetaphoneLib.DMetaphone1( source ) source The string to process. Return: DMetaphone1 returns a STRING value. The DMetaphone1 function returns a textual representation of the source data, similar to a soundex code. This function returns the first return value from the Double Metaphone algorithm. Example: r := RECORD STRING source; STRING M1; STRING M2; STRING Mboth; END; r XF(ProgGuide.Person.File L) := TRANSFORM SELF.source := L.LastName; SELF.M1 := MetaphoneLib.DMetaphone1( L.LastName ); SELF.M2 := MetaphoneLib.DMetaphone2( L.LastName ); SELF.Mboth := MetaphoneLib.DMetaphoneBoth( L.LastName ); END; ds := PROJECT(ProgGuide.Person.File,XF(LEFT)); COUNT(ds); COUNT(ds(M1 <> M2)); OUTPUT(ds); OUTPUT(ds(M1 <> M2)); MetaphoneLib.DMetaphone2( source ) source The string to process. Return: DMetaphone2 returns a STRING value. The DMetaphone2 function returns a textual representation of the source data, similar to a soundex code. This function returns the second return value from the Double Metaphone algorithm. Example: r := RECORD STRING source; STRING M1; STRING M2; STRING Mboth; END; r XF(ProgGuide.Person.File L) := TRANSFORM SELF.source := L.LastName; SELF.M1 := MetaphoneLib.DMetaphone1( L.LastName ); SELF.M2 := MetaphoneLib.DMetaphone2( L.LastName ); SELF.Mboth := MetaphoneLib.DMetaphoneBoth( L.LastName ); END; ds := PROJECT(ProgGuide.Person.File,XF(LEFT)); COUNT(ds); COUNT(ds(M1 <> M2)); OUTPUT(ds); OUTPUT(ds(M1 <> M2)); MetaphoneLib.DMetaphoneBoth( source ) source The string to process. Return: DMetaphoneBoth returns a STRING value. The DMetaphoneBoth function returns a textual representation of the source data, similar to a soundex code. This function returns both return value from the Double Metaphone algorithm, concatenating the two into a single result string. Example: r := RECORD STRING source; STRING M1; STRING M2; STRING Mboth; END; r XF(ProgGuide.Person.File L) := TRANSFORM SELF.source := L.LastName; SELF.M1 := MetaphoneLib.DMetaphone1( L.LastName ); SELF.M2 := MetaphoneLib.DMetaphone2( L.LastName ); SELF.Mboth := MetaphoneLib.DMetaphoneBoth( L.LastName ); END; ds := PROJECT(ProgGuide.Person.File,XF(LEFT)); COUNT(ds); COUNT(ds(M1 <> M2)); OUTPUT(ds); OUTPUT(ds(M1 <> M2)); WorkunitServices.WorkunitList( lowwuid [, highwuid ] [, username ] [, cluster ] [, jobname ] [, state ] [, priority ] [, fileread ] [, filewritten ] [, roxiecluster ] [, eclcontains ] [, online ] [, archived ]) lowwuid A null-terminated string containing the lowest WorkUnit IDentifier to list. This may be an empty string. highwuid Optional. A null-terminated string containing the highest WorkUnit IDentifier to list. If omitted, the default is an empty string. cluster Optional. A null-terminated string containing the name of the cluster the workunit ran on. If omitted, the default is an empty string. jobname Optional. A null-terminated string containing the name of the workunit. This may contain wildcard ( * ? ) characters. If omitted, the default is an empty string. state Optional. A null-terminated string containing the state of the workunit. If omitted, the default is an empty string. priority Optional. A null-terminated string containing the priority of the workunit. If omitted, the default is an empty string. fileread Optional. A null-terminated string containing the name of a file read by the workunit. This may contain wildcard ( * ? ) characters. If omitted, the default is an empty string. filewritten Optional. A null-terminated string containing the name of a file written by the workunit. This may contain wildcard ( * ? ) characters. If omitted, the default is an empty string. roxiecluster Optional. A null-terminated string containing the name of the Roxie cluster. If omitted, the default is an empty string. eclcontains Optional. A null-terminated string containing text to search for in the workunit’s ECL code. This may contain wildcard ( * ? ) characters. If omitted, the default is an empty string. online Optional. A Boolean true/false value specifying whether the search is performed online. If omitted, the default is TRUE. archived Optional. A Boolean true/false value specifying whether the seacrh is performed in the archives. If omitted, the default is FALSE. Return: WorkunitList returns a DATASET. The WorkunitList function returns a dataset of all workunits that meet the search criteria specified by the parameters passed to the function. All the parameters are search values and all but the first are omittable, therefore the easiest way to pass a particular single search parameter would be to use the NAMED parameter passing technique. The resulting DATASET is in this format: WorkunitRecord := RECORD STRING24 wuid; STRING owner{MAXLENGTH(64)}; STRING cluster{MAXLENGTH(64)}; STRING roxiecluster{MAXLENGTH(64)}; STRING job{MAXLENGTH(256)}; STRING10 state; STRING7 priority; STRING20 created; STRING20 modified; BOOLEAN online; BOOLEAN protected; END; Example: OUTPUT(WorkunitServices.WorkunitList('')); //list all current workunits OUTPUT(WorkunitServices.WorkunitList('', NAMED eclcontains := 'COUNT')); //list only those where the ECL code contains the word 'COUNT' //this search is case insensitive and does include comments WorkunitServices.WorkunitExists( wuid [, online ] [, archived ] ) wuid A null-terminated string containing the WorkUnit IDentifier to locate. online Optional. A Boolean true/false value specifying whether the search is performed online. If omitted, the default is TRUE. archived Optional. A Boolean true/false value specifying whether the seacrh is performed in the archives. If omitted, the default is FALSE. Return: WorkunitExists returns a BOOLEAN value. The WorkunitExists function returns whether the wuid exists. Example: OUTPUT(WorkunitServices.WorkunitExists('W20070308-164946')); WorkunitServices.WUIDonDate( year, month, day, hour, minute ) year An unsigned integer containing the year value. month An unsigned integer containing the month value. day An unsigned integer containing the day value. hour An unsigned integer containing the hour value. minute An unsigned integer containing the minute value. Return: WUIDonDate returns a VARSTRING value. The WUIDonDate function returns a valid WorkUnit IDentifier for a workunit that meets the passed parameters. Example: lowwuid := WorkunitServices.WUIDonDate(2008,02,13,13,00); highwuid := WorkunitServices.WUIDonDate(2008,02,13,14,00); OUTPUT(WorkunitServices.WorkunitList(lowwuid,highwuid)); //returns a list of workunits between 1 & 2 PM on 2/13/08 WorkunitServices.WUIDdaysAgo( daysago ) daysago An unsigned integer containing the number of days to go back. Return: WUIDdaysAgo returns a VARSTRING value. The WUIDdaysAgo function returns a valid WorkUnit IDentifier for a workunit that would have run within the last daysago days. Example: daysago := WorkunitServices.WUIDdaysAgo(3); OUTPUT(WorkunitServices.WorkunitList(daysago)); //returns a list of workunits run in the last 72 hours WorkunitServices.WorkunitTimeStamps( wuid ) wuid A null-terminated string containing the WorkUnit IDentifier. Return: WorkunitTimeStamps returns a DATASET value. The WorkunitTimeStamps function returns a DATASET with this format: EXPORT WsTimeStamp := RECORD STRING32 application; STRING16 id; STRING20 time; STRING16 instance; END; Each record in the returned dataset specifies a step in the workunit’s excution process (creation, compilation, etc.). Example: OUTPUT(WorkunitServices.WorkunitTimeStamps('W20070308-164946')); /* produces output like this: 'workunit ','Created ','2008-02-13T18:28:20Z',' ' 'workunit ','Modified','2008-02-13T18:32:47Z',' ' 'EclServer ','Compiled','2008-02-13T18:28:20Z','10.173.9.2:0 ' 'EclAgent ','Started ','2008-02-13T18:32:35Z','training009003' 'Thor - graph1','Finished','2008-02-13T18:32:47Z','training009004' 'Thor - graph1','Started ','2008-02-13T18:32:13Z','training009004' 'EclAgent ','Finished','2008-02-13T18:33:09Z','training009003' */ WorkunitServices.WorkunitMessages( wuid ) wuid A null-terminated string containing the WorkUnit IDentifier. Return: WorkunitMessages returns a DATASET value. The WorkunitMessages function returns a DATASET with this format: EXPORT WsMessage := RECORD UNSIGNED4 severity; INTEGER4 code; STRING32 location; UNSIGNED4 row; UNSIGNED4 col; STRING16 source; STRING20 time; STRING message{MAXLENGTH(1024)}; END; Each record in the returned dataset specifies a message in the workunit. Example: OUTPUT(WorkunitServices.WorkunitMessages('W20070308-164946')); WorkunitServices.WorkunitFilesRead( wuid ) wuid A null-terminated string containing the WorkUnit IDentifier. Return: WorkunitFilesRead returns a DATASET value. The WorkunitFilesRead function returns a DATASET with this format: EXPORT WsFileRead := RECORD STRING name{MAXLENGTH(256)}; STRING cluster{MAXLENGTH(64)}; BOOLEAN isSuper; UNSIGNED4 usage; END; Each record in the returned dataset specifies a file read by the workunit. Example: OUTPUT(WorkunitServices.WorkunitFilesRead('W20070308-164946')); /* produces results that look like this 'rttest::difftest::superfile','thor','true','1' 'rttest::difftest::base1','thor','false','1' */ WorkunitServices.WorkunitFilesWritten( wuid ) wuid A null-terminated string containing the WorkUnit IDentifier. Return: WorkunitFilesWritten returns a DATASET value. The WorkunitFilesWritten function returns a DATASET with this format: EXPORT WsFileRead := RECORD STRING name{MAXLENGTH(256)}; STRING10 graph; STRING cluster{MAXLENGTH(64)}; UNSIGNED4 kind; END; Each record in the returned dataset specifies a file written by the workunit. Example: OUTPUT(WorkunitServices.WorkunitFilesWritten('W20070308-164946')); /* produces results that look like this 'rttest::testfetch','graph1','thor','0' */ WorkunitServices.WorkunitTimings( wuid ) wuid A null-terminated string containing the WorkUnit IDentifier. Return: WorkunitTimings returns a DATASET value. The WorkunitTimings function returns a DATASET with this format: EXPORT WsTiming := RECORD UNSIGNED4 count; UNSIGNED4 duration; UNSIGNED4 max; STRING name{MAXLENGTH(64)}; END; Each record in the returned dataset specifies a timing for the workunit. Example: OUTPUT(WorkunitServices.WorkunitTimings('W20070308-164946')); /* produces results that look like this '1','4','4','EclServer: tree transform' '1','0','0','EclServer: tree transform: normalize.scope' '1','1','1','EclServer: tree transform: normalize.initial' '1','18','18','EclServer: write c++' '1','40','40','EclServer: generate code' '1','1010','1010','EclServer: compile code' '1','33288','33288','Graph graph1 - 1 (1)' '1','33629','33629','Total thor time: ' '2','1','698000','WorkUnit_lockRemote' '1','2','2679000','SDS_Initialize' '1','0','439000','Environment_Initialize' '1','33775','3710788928','Process' '1','1','1942000','WorkUnit_unlockRemote' */