Here's a bizarre one.

Arev 2.12

Existing record with key HAPV*10113 (EE*Internal Date).

User went into a screen to modify this record. Got a New Record item on the screen, so they completed the screen and saved the data with an F9 key. The first record was entered on a previous day. Now there are two records with identical keys (no spaces or anything funny). I verified this with an R/BASIC select of the table and a READNEXT loop to check for matching keys. Only found these two. Obviously a B-Tree select on this table returns only the original record since its pointer finds this one. Yet, if I select the table on a non-indexed value, only the 2'nd record is found. I got the bogus 2'nd record and

copied it to the LISTS table and deleted it. Next I got the first record and copied it to the LISTS table using a tagged key HAPV*10113X and deleted this one as well. Now the indexes are completely FUBAR and will have to rebuilt (the table is 100,000+ records). Has anybody ever seen this? It seems to me that the first attempt to read the data failed without any error and this caused the problem, yet, when the record was saved at the end of the screen, it did not replace the original (which I thought would have been the case).

I'd run a LH_VERIFY on the file. It sounds like a hashing screw up. I've seen records hashed to the wrong groups before. I think DUMP will fix these.

How it happens? I'd speculate a write was interrupted by network error, workstation crash/shutdown, lock failure on file auto-resizing…you name it. Very rare but again I've seen it.

I never checked for duplicates but I seem to recall DUMP moving the item to the correct(?) group and complaining it was there already.

Write a symbolic on DICT.VOC called LENID formula RETURN LEN(@ID)

LIST filename by LENID BY @ID LENID @ID

Do your identical keys appear side by side?

I'm tipping a trailing delimiter or CHAR(0) in @ID - check in DUMPLH

Steve

Steve..

Here's what I did:

SELECT INPUTFILE

KEYS='

SAVEKEYS='

NUM=0

L0:

READNEXT @ID THEN

 LOCATE @ID IN KEYS THEN

     READ @RECORD FROM INPUTFILE,@ID THEN

          NUM+=1

          KEY=@ID:'*':NUM

          WRITE @RECORD ON LISTS,KEY

          DELETE INPUTFILE,@ID

          SAVEKEYS=KEY

     END

 END ELSE

     KEYS=@ID

 END

 GOTO L0:

END ELSE

 SWAP @FM WITH '|' IN SAVEKEYS

 CALL MSG(NUM:' Records Had Dupes|':SAVEKEYS)

END

IF THE KEY BEING READ IS FOUND IN THE TABLE OF KEYS PROCESSED SO FAR I ASSUME IT IS IDENTICAL SINCE CHAR(0) WOULD MAKE THE KEY NON-IDENTICAL. MAYBE THIS ISN'T SO??? WHAT IS REALLY STRANGE IS THE THE KEYS IN THE LISTS TABLE ARE ABSOLUTELY IDENTICAL EXCEPT THAT THE FIRST IS [list=1]*1 AND THE SECOND IS [list=1]*2 AND THAT THERE ARE ONLY 2 ITEMS IN THE TABLE LIKE THIS. WHAT IS REALLY PUZZLING IS HOW THIS HAPPENED:

THE USER ENTERED EACH ELEMENT OF THE KEY IN THE FIELDS OF THE FORM. THE "NEW RECORD" LOGIC WAS INVOKED (APPARENTLY BECAUSE THE ORIGINAL COULD NOT BE FOUND VIA THE HASH ALGORITHM). THE USER DUTIFULLY ENTERED DATA AND PRESSED THE F9 KEY TO SAVE. NO COMPLAINTS OR INCREMENTAL UPDATING LOGIC CALLED AS WOULD BE THE CASE IF THE ORIGINAL RECORD WAS THERE (VIA THE @RECORD / OREC MATCHING LOGIC). OBVIOUSLY OREC WAS NULL SINCE THE "NEW RECORD" LOGIC WAS INVOKED. HOWEVER, A B-TREE SELECT ON THIS KEY ONLY RETURNED THE ORIGINAL RECORD (WHICH COULD NOT BE READ IN THE SCREEN). WHEN THE PROBLEM WAS REPORTED TO ME, I TRIED GETTING TO THE DATA RECORDS VIA THE EDITOR. IF I TYPED EDIT INPUTFILE KEY, I GOT THE NEW RECORD. IF I DID A SELECT ON THE B-TREE'D KEY AND GOT 1 RECORD AND THEN TYPED EDIT INPUTFILE, I GOT THE OLD RECORD. NEXT, I RAN MY LITTLE FIXIT PROGRAM. SINCE THE FILE IS RELATIVELY LARGE, DOING AN INDEX REBUILD WOULD INVOLVE REMOVING ALL INDEXES AND THEN RE-APPLYING THEM AND FIRING UP A BATCH UPDATE PROGRAM OV

ERNIGHT. LH_VERIFY DOES NOT INDICATE ANY ERRORS, BUT I BET DUMP WOULD HAVE FIXED THESE IF I COULD FIND THE GROUPS THEY WERE IN, BUT THE AREV ROUTINE ALWAYS RETURNS THE SAME FRAME FOR THAT KEY (EVEN IF THEY WERE FOUND IN PHYSICALLY DIFFERENT FRAMES, AS THE ABOVE EXAMPLE SHOWS). REALLY STRANGE…

DON MILLER

C3 INC.

Hi Don,

Just confirming that all you guys are right on this one and a few tips to help you along to tidy it up.

As Steve said, it is possible to sometimes think you have dupe keys by looking at a F2 browse list which will show keys with embeeded delimiters (eg @VMS) on separate rows…but thats not your problem here.

You are right you physically have two copies of the record both with the same key in the file…and they are located in different groups.\

As you say, your program has verified this. I have seen this a few times, and the other Warren is right, its because during a resizing happening as part of a incomplete write, the record didn't end up in the right group. SO. when your customer goes to retrieve the record, (which is say in Group 100), AREV hashes the key, says "Ah, that should be in (eg) Group 250", and goes looking there…when it aint there it returns "New Record".

BUT, when you SELECT the file on a NON-Indexed field, it actually WALKS the file in FRAME order, and finds the key and gives it to you. If you are READNEXTING at the time (an unresolved LIST) you will also get the actual data of the "Old" record. If however you force a resolved list, or if you don't check @REDUCTION.DONE in an RBASIC program at this point and just READ @RECORD FROM FILE,@ID then it will "re-hash" the key and go looking in the correct group - and return a failed read - or in your case, then "New" record.

I am a bit surprised LH_VERIFY didn't report a problem - I mean the whole idea of it was to do effective "DUMP" logic without actually having to go through the whole DUMP process - no wonder we still think about writing our own…

But Yes DUMP would fix it if you were FIXing the relevant group..it would give you one of those "HAPV*10113" found in group xxx but belongs in Group yyyyu (F)ix (T)hrowaway (N)ot a record messages….

Everything you said in your messages is consistent with standard AREV performance here except the suggestion that a BTREE select on the Key field then returned the old record, not the New one… I suggest that if you check my comments on RESOLVED vs UNRESOLVED selects, and knowing the way BTREE indedxes work, you will find it had to do with the way you did the Select, not anything to do woth BTREEs. BTREES ALWAYS return RESOLVED lists, so you would get the NEW record that way.

Anyway, how do you find which group the record is in ? (The Old record). One way is to do a straight SELECT FILE.NAME of the file and then locate the first occurrence of your key within this list of keys (ie determine its position). Say its at position 23000 out of 100000 keys. DUMP the file, and determine how many groups there are (Modulo). Work out the approximate position of the Key

eg 4500 groups on file. Key is at approx Group 4500 * 23000 / 100000

= Group 1035. Go to group 1035 (1035 Ctrl-g) and note down the key of the first record in the frame. Now go back to the List of keys from the SELECT (in the LISTS file) and locate THIS key. Is it before or after the rogue key ?

Basically from then on you should be able to zoom in pretty quick on the exact group using a binary search technique and repeating the steps above.

ANOTHER WAY to find the group is simply to OSOPEN the relevant DOS file and OSBREAD your way through a frame (ie usually a 1024 byte block) at a time. Simply look for your key string in the block

IF INDEX(DATA.BLOCK,KEY,1) THEN … and bingo you have your frame number !

FINALLY, if i suspect there is any problem with Keys or Indexes, i compare a "SELECT FILE.NAME BY (Indexed Single-value) FIELD" result (ie number of records selected)…with a COUNT or DUMP (Reccount=).

This will tell you straight away if you have a problem with dupe keys or lost index transactions.

Hope all that helps !

Warren Kinny

Exodus Systems

Australia

wjkinny@resmaster.com

View this thread on the forum...