Efficient Processing of Dynamic Arrays

Advanced Revelation developers frequently write programs to sequentially process large dynamic arrays. How you process a dynamic array can affect the performance of your program.

The least efficient way to process all the elements of an array is, unfortunately, the most straight forward. Using angle bracket operators or the EXTRACT function you loop through the array like this:

FOR CNTR = 1 TO NUM_ELEMENTS
  TEMP = ARRAY<CNTR>
  /* processing here */
NEXT CNTR

To determine what data to return, the system starts at the beginning of ARRAY each time through the loop and counts the number of delimiters. For a 10,000 element array this method is 100 times slower than an empty FOR…NEXT loop.

A much faster method is to use the REMOVE command. The program looks like:

POS = 1
FOR CNTR = 1 TO NUM_ELEMENTS
  REMOVE TEMP FROM ARRAY AT POS SETTING ï FLAG
  /* processing here*/
NEXT CNTR

Each time through the loop the variable POS is updated. The system starts at POS and searches forward in the array until a system delimiter is found. It then returns everything between POS and the new position and updates POS.

This method is only 1.93 times slower than an empty FOR…NEXT loop for a 10,000 element array.

The drawback to the REMOVE statement is that it stops scanning as soon as it encounters any system delimiter. This makes it unsuitable for dynamic arrays that mix, for example, field marks and value marks.

By using the bracket operators you can write a program that emulates the function of REMOVE, but retains control over which delimiter stops the scanning. The program looks like:

POS = 1
FOR CNTR = 1 TO NUM_ELEMENTS
 TEMP = ARRAY[POS, "F":@FM]
 POS = COL2() + 1
 /* processing here */
NEXT CNTR

For the same 10,000 element array this method is only 3.47 times slower than an empty FOR…NEXT loop.

The method works by maintaining a pointer into ARRAY to keep track of the current position. The brackets function instructs the system to scan forward in the array until a field mark is found. All other characters, including system characters, are ignored.

Figure 1 is a function, SEQ_DYN, that encapsulates this approach. You pass the dynamic array to be searched, the delimiter to use, and the position in the array to begin the search. Figure 2 shows how SEQ_DYN could be used in a program.

FUNCTION SEQ_DYN(DYN_ARRAY, DELIM, POSITION)

/* A Function to allow fast sequential access through a dynamic array */

DUM = DYN_ARRAY[POSITION,"F":DELIM]
POSITION = COL2() + 1
RETURN DUM

DECLARE SUBROUTINE MSG
DECLARE FUNCTION SEQ_DYN

ARRAY = "Apple,Pear,Orange,Banana,Kiwi"
POS = 1
LOOP
  VAL = SEQ_DYN(ARRAY, ",", POS)
WHILE VAL
  MSG("VAL is %1%", "", "", VAL)
REPEAT
STOP