My problem is that I cannot find the description of the background indexing process.

I often read about !INDEXING table, but I cannot find a table with this name.

…

Two or three weeks later I cut a huge portion of a table and put it into another one (around 400000 records moved). Next one and half week the dedicated indexor was trying to update pending transactions and the server was almost unable to support something else.

So we have decided to solve the problem. During the weekend we copied all volumes to a standalone machine. After rebuilding indexes we copied the volumes back.

Now the indexes have unexpected behaviour for me.

..Index.Flush reports … index description row DATE missing, (reindexed after that). It seems that transactions are not processed well. Was the reindexing on a standalone machine wrong decision? What I should do to correct the situation? A pointer to detailed description of the backgroud processing will be wellcomed. Thanks for answers and sorry if I will not respond, after 5 hours I left for a week of halydays. </QUOTE> —- === At 01 MAR 2002 04:29PM Hippo wrote: === <QUOTE>I have reindexed the one table where the errors occur once again (on network) starting with cleartable !file as recomended here in the conference in past. I hope, this solves the particular problem of the table, but does not give me what I am searching for… the background indexing rules. … Where are the pending transactions stored. Some are in !file I suppose, but probably before the distribution … I have read about the process here, but I cannot locate the topic, and more worse, there was something written about !INDEXING file. It is a real table name? Remains it in AREV 3.12? May be it is renamed? I cannot locate it. </QUOTE> —- === At 01 MAR 2002 04:38PM The Sprezzatura Group wrote: === <QUOTE>The !Indexing file was removed in version 2.0 I think. For details of the indexing file structures go to our site (http://www.sprezzatura.com) and search on "index structures". The first article is http://www.sprezzatura.com/revmedia/V2I1A3.HTM which is a good starting point. The Sprezzatura Group World Leaders in all things RevSoft </QUOTE> —- === At 01 MAR 2002 06:31PM Hippo wrote: === <QUOTE>Thanks for answer </QUOTE> —- === At 01 MAR 2002 06:38PM Hippo wrote: === <QUOTE>Unfortunately the 'starting point' uses !INDEXING… </QUOTE> —- === At 01 MAR 2002 08:44PM Victor Engel wrote: === <QUOTE>Just substitute !filename for !indexing and make the further adjustment that there is thus no step to transfer records from one to the other. The same records are created in version 2.x+ but they are written to !filename instead of !indexing. </QUOTE> —- === At 04 MAR 2002 11:21AM Dave Harmacek wrote: === <QUOTE>Oh, where to start? 1) You deleted key indexing control rows by cleartable-ing the !file. 2) When deleting or adding so many rows into an indexed table it would be best to suspend the index transaction creation process during the delete, then completely re-index the table. There used to be utilities available to suspend the indexing of a table. Also, when deleting so many rows you would be best to stop the automatic re-sizing of the table. Use DUMPLH and press the "+" key until the SizeLock parameter is equal to 2. ("-" to restore to 0, later) You will find that the DOS size of the table has not changed much, even after re-sizing. This is because the overflow file, .OV, only has frames removed when the frame is at the endmost section of the file. You may be better to remove indexing of that table, copy all rows to a temporary table, rename the original table to OLD_tablename, rename the temporary table to the original tablename, and finally re-install all indexes. Dave </QUOTE> —- === At 08 MAR 2002 03:19PM Hippo wrote: === <QUOTE>Thank you Dave for the response. Next time I shall remove indexing first. The problem is not with the cut table, but with the overloading of an indexor. I understand the authomatic resizing rules. The mentioned table is not very frequently accessed, mostly the records are added. I let the sizelock equal 0 as the speed is not an issue. I will probably pack the overflow portion later. Thanks once again </QUOTE> —- === At 08 MAR 2002 03:37PM Hippo wrote: === <QUOTE>Thanks for response. It will mean the indexing on a standalone machine is OK. Unfortunately during the week of holidays the message 'The index row "DATE" is missing … The indexed column … should be rebuilt' appeared again on the dedicated indexor.

I will repeat reindexing once again and study the material recomended by Sprezz with substitution recomended by you.

The problem is probably solved.

It was independent on local reidexing.

The !file framesize was too small and the problem of long overflow linked lists probably appeared. We remaketable !file with higher framesize and everything seems to be OK.

If I understand it well, SI.MFS checks DICT updates and recompiles ! record (rewrites !! record if it exists =source request).

SI.MFS than traps READ,WRITE, and DELETE MFS calls and calls ! in !file with @MV,@ID,@RECORD and @ANS properly set.

@MV=2 means index rebuild, @ID, @RECORD not needed? … ALL.IDS keys as pending transactions.

The ! program in !file creates pending transactions in !file (form of pending transactions is described at 'starting point' given by Sprezz)(Yes … there is no !INDEXING file).

(There is temporarily XTRANS record in the !file whenever the update item is huge. It remains here if there are problems during creation of pending transactions)

How the background process deals with pending transactions is another story…

I hope the problem is solved now :))

I found that during selecting rows according index the result does not correspond to a question (returns rows pointed by another value).

This problem was discussed here before, therefore I expected the @FM/@VM or something like that in a key.

There actually was one such strange row. (Containing @VM in @ID)

I have removed it using RBASIC and reindexed !file.

So many false suspections / or so many parallel bugs? :((

I don't see what frame size has to do with anything. Sure, increasing frame size will result in a better distribution of data between the .lk and .ov portions of the indexing files, but that is independent of everything else discussed in this thread.

OK, you are right, I am sorry.

We had tried almost everything :(, As I told, I was on a holidays,

therefore when they told me on return, this repaired the situation, I hoped its true. I was in a hurry:(.

I suspected even problems of hashing too many index records to the same frame. Increasing the frame size may change this behaviour (statistically it is worse after increasing the frame size:(()

After finding @VM in a key (,deleting the record using RBasic and reindexing) everything is OK.

View this thread on the forum...