Reduce/Select vs Push/Pop.Select (AREV Specific) [Revelation On-Line Wiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
[[https://www.revelation.com/|Sign up on the Revelation Software website to have access to the most current content, and to be able to ask questions and get answers from the Revelation community]]


==== Reduce/Select vs Push/Pop.Select (AREV Specific) ====
=== At 09 JUL 2007 06:49:28PM David Craig wrote: ===
 
{{tag>"AREV Specific"}}
 
<QUOTE>
I'm working on a function that will be called from all of our output programs.  The output programs are processing a selected list and the function has to do a non key search so if I use "Perform select.." it blows out the calling programs selected list.  I've found two ways around this; reduce/select and push/pop.select.  



I read that Reduce/Select is very slow and that Push/Pop.Select is the way to go, but my concern is that the select from the calling program(s) are typically quite long searches, and the function will be called once for each record processed, which will potentially be thousands of calls.  



Sorry for the long question but here's the point; when you pop the select does it rerun the query or does it just retrieve the selected keys?  I read something about 'seeing all those queries in the TCL stack which makes me fear the former.  



Thanks in advance for any help;



David Craig.  
</QUOTE>
----

=== At 10 JUL 2007 10:17AM Dave Harmacek wrote:  ===

<QUOTE>Where did you hear "I read that Reduce/Select is very slow and that Push/Pop.Select is the way to go"?



I suggest a Btree.extract, then do the REDUCE on a different cursor if you need returns on non-indexed columns.



Dave

</QUOTE>

----

=== At 10 JUL 2007 11:29AM Victor Engel wrote:  ===

<QUOTE>PUSH.SELECT stores details of the cursor in a set of 4 variables. These variables are restored with POP.SELECT. Certain other variables, such as @REC.COUNT have to be maintained separately.



The select list is NOT reperformed each time.

</QUOTE>

----

=== At 10 JUL 2007 11:54AM David Craig wrote:  ===

<QUOTE>I read it in this forum, this is the reply suggesting push/pop;



[url=http://www.revelation.com/discuss.nsf/7a591c01171830eb8525652b0083c06b/13E18BE2334E9E688525693D006A0410?OpenDocument]Reply[/url]



I don't know if that link will display correctly, I found it by searching the Arev forum for 'reduce', then it was a thread titled "REDUCE/SELECT by issues".  In addition to what I read there, there is the general sentiment that cursors in general are performance bottlenecks to be avoided, although that is probably in comparison to set based operations.  



I will look at b tree extract, I have been thinking I'll need to index those fields as the table grows larger, which it will quickly, so this might be the best approach.  Right now there's just a few test records in there so putting an index on it seems pointless.  



Thanks Victor for answering my question, that makes the push pop approach viable, then I can look at b tree extract as a way to deal with performance as the table grows larger.  



Thanks to you both;



David Craig

ABC-Clio

</QUOTE>

----

=== At 10 JUL 2007 12:25PM Warren Auyong wrote:  ===

<QUOTE>Obviously the method I was using was the wrong way to do it. I had not worked in ARev for several years, having switched to FoxPro, when I made that post and had forgotten about push/pop select.

</QUOTE>

----

=== At 11 JUL 2007 12:47PM David Craig wrote:  ===

<QUOTE>Testing yesterday showed that even when I can get it to work, the reduce /select approach will slow down our processing by a factor of about 15 (from 6 minutes to 1.5 hours), which will get worse as the file grows larger.  I don't believe this is related to reduce/select vs push pop but rather the search in the subroutine.  



So now I'm working on a btree.extract version.  I've got it to find the record but the 6th parameter 'flag' returns 0 whether or not it finds the records - per the documentation 'flag' should return -1 when it finds nothing.  I can check the 4th parameter 'key' to see if it found anything but at this point I have to ask - is the documentation incorrect, has the subroutine been changed or am I missing something?  



The documentation I'm working from is version 3.0 and I'm working with 3.12, if that is relevant.  



I have another question - again apologies for the long post - btree.extract requires the dictionary of the file being accessed to be open, the documentation uses @DICT.  Of course, the calling programs will be using @DICT, is the way around this issue to use something other than @DICT as a variable in the open statement, add push/pop, or  grab @DICT before the OPEN and restore it at the end of the subroutine.  



Thanks again for your patience and help;



David Craig.  

</QUOTE>

----

=== At 11 JUL 2007 01:14PM Victor Engel wrote:  ===

<QUOTE>I'm not sure I understand your issue with @DICT. Does your routine entail opening two different dictionaries to @DICT? If not, where is the problem?



</QUOTE>

----

=== At 11 JUL 2007 03:00PM Dave Harmacek wrote:  ===

<QUOTE>Craig, @DICT is a system variable you are allowed to set.  The current value is used by a number of system routines.



So, just open each dictionary to a different variable.  Set @DICT to the value of the one you need at the time you need it!



open "DICT","MAINFILE" to mainFV...

@DICT=mainFV

open "DICT","OTHERFILE" to otherFV...

eof=0

loop

  readnext @ID else eof=1

until eof

  read @RECORD from mainFv, @ID then

    otherval={OTHER}

    @DICT=otherFV

    call btree.extract( .....)

    @DICT=mainFV

  end

repeat



Dave

</QUOTE>

----

=== At 11 JUL 2007 06:05PM David Craig wrote:  ===

<QUOTE>Please, my name is David.  



Thanks for the help with @DICT, that should get me around that problem.   I ran into an additional problem with btree.extract in that the subroutine has to search for a record, if it's not there, create one.  But the next record processed may be looking for the same record, which is in the table but not (yet) in the index.  



I'm getting around that with Index.flush(tablename, '').  



At least I don't feel like I'm not learning anything,  



Thanks again for your help, 



David Craig.  

</QUOTE>

----

=== At 11 JUL 2007 06:20PM David Craig wrote:  ===

<QUOTE>The problem is that the program calling the subroutine is using @DICT, but I'm handling that by grabbing it before using it for btree.extract, then restoring it afterwards.  



If I didn't do that data accessed in the calling program with @RECORD{'field'} would fail because @DICT is now for a different table.  This much, I (now) know from experience.   



Thanks again, 



David Craig.  



</QUOTE>


[[https://www.revelation.com/revweb/oecgi4p.php/O4W_HANDOFF?DESTN=O4W_RUN_FORM&INQID=NONWORKS_READ&SUMMARY=1&KEY=0AB05FB78EB82DAA85257313007D6127|View this thread on the  forum...]]