Update-only Data Warehousing from AREV 3.12 (OpenInsight 32-bit Specific)
At 11 JAN 2006 07:22:20PM Christian McGregor wrote:
We are running a DOS AREV 3.12 system, and would like to use OI 7.2 to push the data from AREV into MS SQL 2000.
The most appropriate method for doing this appears to be to use the OI Updates-Only Data Warehouse method, which would automatically publish changes made in AREV to SQL.
When we enable updates-only processing on our AREV table(s), OI applies XQ_MFS to AREV - which, as discussed elsewhere on these forums, causes a problem as it is not compatible with AREV. See these threads for details:
Update-only - Data Warehousing
These discussion threads are over 5 years old, so we're hoping that either the problem has been solved, or that someone can post a fix, or suggest an alternative method for achieving what we want to do.
Can anyone assist, or shed any light on this problem?
Thanks.
At 11 JAN 2006 08:44PM dsig _at_ sigafoos.org wrote:
I am not sure that this is the best method for doing this. The reason is that the mfs triggers the 'sql' update BUT the specific routines and connections are not available for Arev.
the only way i can see to use the warehousing feature would be to have arev write to some 'secondary' table which OI (working like an index server .. just spinning and checking every once in a while) would find the data and write to tables set to warehouse. This method takes several more writes and not sure it is very practical.
the problem is that you need a way to separate the arev process from the OI process.
Hmmm .. this is a pretty intersting problem
At 11 JAN 2006 10:31PM Christian McGregor wrote:
Thanks for the rapid response.
Interestingly enough your proposed solution is exactly what we were/are doing - using an MFS and some code in AREV to determine what has been updated and dropping that data into a secondary 'publications' table. We were then running an AREV indexing machine which periodically exported that data to a DBASE file, which was in turn imported via an scheduled SQL procedure.
Trouble was we found that due to some hard to identify issues - probably file contention, or record locking, or ??? - we found that some data was going missing during this process. So we've implemented OI to bypass the DBASE step, and go instead directly from AREV to SQL via OI.
We had hoped the Update-only feature would mean we could also remove the exporting to the secondary tables, but it sounds like we may have to continue using that, and have OI poll that table. We have managed to get that working, just thought we should query the other method before giving up on it.
Thanks again.