MFS operations, select (code 7) and readnext (code 8) (OpenInsight 64-bit)
At 18 OCT 2023 08:22:27AM Richard Hunt wrote:
My project is to enhance these items thru a MFS. I have already have done it by implementing custom MFS codes and basic subroutines to call the MFS with the new codes.
I already have many enhancements done for selecting (reduction criteria) and READNEXT also that I want to neatly insert into the custom MFS.
What I really want to do is to use to somehow pass a "list number) to the select (code 7) and READNEXT (code 8). I want these to be able to handle multiple SELECT and READNEXT basic commands at the same time. But the code 8, when "done" seems to end all READNEXT calls to the MFS, almost as if there is an internal set of variables in use.
I basically have a problem with the MFS call 8 is not called consonantly with every READNEXT basic command.
I have the ability to get record ids one LH frame at a time, I have solved storing multiple select information.
If I have I will stic with the new custom MFS call codes and basic subroutines.
Got any ideas?
At 18 OCT 2023 11:08AM bob carten wrote:
I want these to be able to handle multiple SELECT and READNEXT basic commands at the same timeIn Oi10 you can stack multiple selects by passing in the statements as an @FM delimited list.
OI will process the statements, stopping if it encounters zero records found.
BTW, We are currently updating the documentation for OI10, converting it from windows help to Wiki format. This is a work in progress, so not fully up to date. The RLIST documentation has been updated to describe the features available in OI10 as of the current beta release.
Here is a link to the Rlist documentation
re your MFS questions:
Most of the activity for filtering rows occurs in the basic+ programs REDUCE, RTP11,RTP11.INDEXER,RTP12,RTI_CHAIN_SELECT, SI_REDUCTION, RESOLVE_SELECT
For RTP57 The SELECT operation is more like pre-select. It has 2 functions: it determines if the columns used in a query are indexed, and it initializes the variables which will be used by readnext.
For RTP57 READNEXT operation is called to retrieve the next set of keys from the filing system (RTP57A). Without a UD this will return a frame's worth or keys. With a UD this returns a chunk of keys which may be several frames worth.
OI buffers the keys in a variable. The Basic+ READNEXT operation (RTP12) consumes the keys from this buffer. When finds one that meets your selection criteria it returns it. When RTP12 drains the buffer it calls the filing system's READNEXT call, which will hit your MFS. So, as you noticed, your MFS will not be called for each readnext operation.
Note that if the field in your query is indexed, then SI.MFS will perform the reduction and your MFS will not get called.
If you have a mix of indexed and non-indexed columns, then, in OI10, OI tries to optimize the query into sections where it can use the indexes on the indexed fields, then apply the rest of the filter by looping through the keys found in the indexes. RTI_CHAIN_SELECT is the routine which performs the optimization, RESOLVE_SELECT is the routine which does the final loop / readnext.
I think an MFS might be too low a level for what you are trying to do, but I may not understand your goal. Can you just use MAKE.LIST to activate a list of keys before running the query? That would cause OI to use your list as the starting point. If you do want to use the readnext to operate on the keys, then you could try using a non-zero cursor. If you use cursor 1-8 then you can see the keys in @CURSORS, in which case you could manipulate them as desired. The SELECT could populate the variables (identical to MAKE.LIST) so that the RTI READNEXT..BY operation would use your list without ever hitting the file system.
At 18 OCT 2023 09:15PM Richard Hunt wrote:
Thanks Bob. I already have the basic programming for doing all that outside the MFS structure. I just thought it would be good to be able to use the MFS structure . Although since the MFS structure handles one read next at a time in a kind of multi tasking environment, I find it to be inferior or not quite good enough.
Over the years I have researched and noticed that the select and read next is no so efficient on memory or efficiency.
I have looked into the RTP11, RTP11.INDEXER, and RTP12.
I suggest that it should get ids by frame, and then by correct order use the reduction criteria to eliminate the ids that do not meet the reduction criteria before returning the frame of ids.
Returning multiple frames of ids is not an efficient use of memory. Sorting the ids before checking if the ids meet reduction criteria is not efficient either. Having a progress meter based on frame id selection is good during timely processes is good also. Being to be able to abort is good too.
So many enhancements are needed in my opinion.
I got all that done and the enhancements by using MfS custom codes 107 (rather than 7) and code 108 (rather than 8). I might just have to keep that method.
I was just hoping there was a way to make the MFS trigger on every read next basic statement.
At 19 OCT 2023 08:55AM bob carten wrote:
I bet you could have your mfs return only one key per readnext call. That would make each Basic+ readnext go back to the filing system for the next key.
The FMC parameter is used for passing information about the readnext. See ix_select_constants. I think you can add information to that for your own use.
At 20 OCT 2023 12:33PM Richard Hunt wrote:
Thanks Bob.
Yes i found I can use the SELECT (code 7) read nest (code 8) and clear select (code 9) FMC calls . The FMC argument can be used to pass information, as long as you don't bother with it during the SELECT, READNEXT and CLEARSELECT basic calls.
I figured how to determine when the FMC argument has information for my own calls to the MFS. so far its working perfect. Got the progress display, abort function and the clear select clean up.
Calling the MFS every time for a key is not time consuming. I read a lh frame of keys at a time and store them kinda like the @CURSORS does. Got the multiple select/readnext working at the same time.
My next step is to, like you suggested, incorporate indexing. Gonna be a bit time consuming. Got my own MFS for indexing. So it won't be too hard, except trying to share between the two MFS's without either one getting upset or ignored is the issue.