Group Format Errors in Advanced Revelation
One of the questions asked most frequently of Technical Support concerns the nature and cause of Group Format Errors (GFEs). This bulletin provides basic information on what a GFE is and on how a GFE can occur.
Group Format Errors are rare. If GFEs are received on a regular basis, then check the hardware, power supply, and locking as discussed below. These errors are caused by problems that cannot be controlled by Advanced Revelation. Though we can attempt to fix them, good backups are always in order in case the corruption is irreversible.
Group Format Error (GFE)
Definition :
A Group Format Error is an internal error in an Advanced Revelation Linear Hash file.
A GFE indicates that information within the file has been corrupted. Usually one of three things has occurred:
1) Record header information has been corrupted.
2) Group header information has been corrupted.
3) File header (in Group 0) information has been corrupted.
These headers contain important information on the number of records and the number of bytes in the record, in the group, or in the file. They also contain pointers to the end of a record, the next group, or overflow frame. If any of this information becomes corrupted, Advanced Revelation reports a GFE. This error generally manifests itself when users are accessing a data file or when background indexing is taking place.
On occasion, a Group Read Error is received. Group Read Errors will be caused if the group a record should be in is not found or if the key hashes to a group that isn't there. Though the causes for this corruption vary, GFEs and Group Read Errors are handled in the same manner.
Causes of GFEs
GFEs are caused by hardware problems, power surges or power outages, locking problems, and network options.
Hardware Problems
Typically, the hardware problems that can cause GFEs are hard disk problems and controller card problems. If there is a problem with a controller card, corruption may be introduced because the information written to disk is garbled. There may be a bad spot or sector on the hard disk which is not holding information reliably. A GFE may also appear if DOS has a problem simply reading the file. This can occur when there is a file in Advanced Revelation, but there is no associated DOS file, or if there is a Group Read Error with the FAT table at the DOS level.
Another hardware-related cause for GFEs is slow memory. When the machine memory is too slow for the CPU and disk caching, a Group Read Error may occur. Hardware diagnostics (Norton Utilities, DOS CHKDSK, etc.) can be run to to attempt to determine whether hardware is the cause of the GFE.
Power Cutoff
Another possible cause of GFEs is problems with power outages or problems with a power supply. If the power goes down in the middle of an operation or if users simply power down rather than logging off normally, the disk cache may not be properly flushed before the power goes down, causing corruption. A way to avoid the most obvious of these problems is to flush any caches to disk before powering down, by logging out of Advanced Revelation.
Locking Problems
A third possible cause of GFEs is a problem with record locking. This is especially important to keep in mind when in a networking environment. If Advanced Revelation is being used in a multiuser environment, it must be bumped and a network driver must be assigned. This is true even if it can be guaranteed that only one user at a time will ever attempt to write the same record. Verifying Locking Record locking on a network is achieved through the assignment of a network driver. Verify that a network driver has been installed and that each station has a unique station number by typing WHO from TCL. If no network driver is assigned or the wrong driver is installed for the network being run, see the Network Bump Disk manual for instructions on assigning or changing a network driver. To check whether or not record locking is being achieved on a network, attempt to bring up the same record on two different workstations. The second workstation should display the message:
recordname is in use elsewhere
and cannot be saved
..Press any key to continue..
The record can be viewed, but if any changes are attempted a second message should appear:
This data is protected and cannot be edited.
..Press any key to continue..
If these messages do not appear and changes can be made, there is a problem. Check the network driver as described above.
Locking problems may also crop up when locking and unlocking of records is being controlled from within an R/BASIC program. Locks on records should be done before the write, and unlocks after a write. See the example in the Technical Reference manual under LOCK. If this is done improperly, GFEs can occur.
Novell TTS Problems
One final cause for GFEs is the use of Novell's TTS option. Advanced NetWare, version 2.1 and above, has an option to use Transaction Tracking (TTS). Do not use TTS with Advanced Revelation. We do not currently support using TTS on Novell Networks because it guarantees that GFEs will occur.
Fixing GFEs
When a GFE is encountered, be sure to make note of the exact error message. The message gives the name of the subdirectory where the file is located, the DOS name of the file, and the group number where the corruption has occurred. For example, if the following message is received:
Group Format Error INVOICE\REV00034.LK Group 0000006
the corruption is on the volume INVOICE, in the file REV00034.LK in Group 6.
There are several steps that must be taken to fix a GFE. First, find out the Advanced Revelation name of the file that has the GFE. The number (REV00034.LK) is the DOS filename. To locate the associated Advanced Revelation filename, execute a LISTMEDIA command for the appropriate volume (the volume name given in the error message in the example above, the volume name is INVOICE). From TCL, type:
LISTMEDIA volumename
LISTMEDIA provides a listing of the Advanced Revelation file name as well as the DOS name. Locate the DOS file name under the MS-DOS Name column, and note the associated Advanced Revelation file name. It is also necessary to have the group number. If this number was not noted when the original message appeared, type this at TCL:
LIST filename AVERAGE SIZE DET-SUPP
This will force the error message to be redisplayed. The correct method for fixing a GFE is dependent on where and how the error occurred. For instance, if the GFE is encountered in an index file, there is a different recommended method for fixing the file than if it occurs in a data file. Outlined below are several strategies for fixing GFEs. Follow the one best suited to the nature of your GFE.
Fixing GFEs Caused by Disk Corruption
If the problem originates with a bad disk, the problem can sometimes be resolved by renaming the DOS file at the DOS level, then copying this renamed file back to its original name. Renaming the file creates a new name and covers the problem area with a file that will no longer be used, locking out the bad spot. Copying the file will create a duplicate file in a different location on the disk.
Note: Use this method only if one of the other methods suggested below has not been successful.
To accomplish this, use the DOS RENAME command to rename the Advanced Revelation file to NEW.NAME, then DOS copy NEW.NAME to its original name. For example, if you encounter a GFE in the file REV00034.LK, follow these steps from the DOS prompt:
RENAME REV00034.LK NEW.NAME COPY NEW.NAME REV00034.LK
If this does not cure the GFE, follow one of the methods outlined below.
Fixing a GFE in the !INDEXING File
Follow these steps to correct a GFE in !INDEXING:
_ Log into the SYSPROG account.
_ From TCL, type: CLEARFILE !INDEXING This will clear all the pending transactions out of the !INDEXING file.
_ Rebuild all indexes. Select Tools-Files-Indexing-Update from the Advanced Revelation Main menu. Rebuild indexes for each file, one file at a time.
To do this, select option 5 from the "Index Update Choices" menu, "Rebuild all indexes in a specific file." Note: Indexing files (!INDEXING or !datafile) cannot be restored from a backup. The indexing files must be completely up to date for the integrity of the indexes to be maintained. If corruption has been introduced, it is very likely that one or more of the indexing transactions has been affected.
Fixing a GFE in an Index (!data) File
Follow these steps to fix a GFE in a !datafile:
_ Remove all indexes for this file. Select Tools-Files-Indexing-Update from the Advanced Revelation Main menu. Select option 6, "Remove indexing from a File."
_ Install and rebuild the index. Select Tools-Files-Indexing, and the type of index to be reinstalled (Btree, Crossreference, etc.). When prompted to rebuild, answer "Yes".
Fixing a GFE in a Data File
If there is a GFE in a datafile, there are two choices:
_ Restore the file from a backup (a backup made before the corruption was introduced).
_ Use the DUMP utility. The Advanced Revelation utility DUMP will attempt to fix the internal corruption. This fix may delete bad records from the group designated in the original error message. See "Using DUMP" below.
Fixing a GFE in a Dict File
If there is a GFE in the dictionary of a datafile, restoring from a backup is the best solution. When restoring from a backup is not possible, try using DUMP. Keep in mind that dictionaries store indexing information. If DUMP is used on the dictionary of a file with indexing, examine the records in DUMP.FIX.TEMP to verify whether or not records were deleted. See "Using DUMP" below. If a field with indexing has been deleted, add the dictionary record, then reestablish and rebuild the indexes for that field.
Fixing a GFE in a System File
If the GFE is in an Advanced Revelation system file (for example VERBS or REVMEDIA), these files will need to be restored from a backup. Performing DUMP on them may prove hazardous, since critical system records might be deleted. Alternately, you can reinstall your Advanced Revelation system to a different location, and copy the corrupted system file (using DOS copy) to your existing system.
Using DUMP
DUMP allows examination of the internal filing structure of a Linear Hash file. To access the DUMP utility, type the following from TCL:
DUMP filename
where filename is the name of the Advanced Revelation file in which you have a GFE.
To move from group to group, use the up and down arrow keys. To move from frame to frame within a group (between primary and overflow) use the left and right arrow keys. Move to the group with the GFE (the group number is displayed in the original error message) and copy each frame within that group to your printer using [Shift-PrtSc]. You will need this information for comparison later.
Note: Keep in mind that you are directly in the filing structure. Any changes made while in the edit mode in DUMP will make permanent changes to the file, even if [Esc] is pressed.
While in DUMP, type [Ctrl-F] to fix the GFE. When prompted for the group or range to fix, enter the group number reported in the error message and press [Enter]. The system will attempt to fix the error by recalculating the corrupted header information. If this is not possible, then any bad records will be deleted from the file. The deleted records are normally stored in a file called DUMP.FIX.GARBAGE, depending on the type of corruption that has occurred. Each record that is deleted should become a record in DUMP.FIX.GARBAGE. DUMP creates a record key that provides the group number that the record belonged to, as well as a number that indicates where in the group of deleted records this record falls.
For example, imagine that DUMP has been used to fix a file called MY.FILE. When MY.FILE was dumped, three records were created in DUMP.FIX.GARBAGE. The keys are GROUP.3.1, GROUP.3.2, and GROUP.3.3. The record keys indicate that 3 records were deleted from GROUP 3 in MY.FILE. All records in DUMP.FIX.GARBAGE will need to be re-entered. The structure of each field of these records is as follows:
<1> RECORDKEY FIELD1 (the record key may be preceded by ASCII characters) <2> FIELD2 <3> FIELD3, etc.
Depending on the type of corruption and the fix performed by DUMP, these records may not show up in DUMP.FIX.GARBAGE. This will need to be determined by examining the printed copies of the frames taken prior to dumping the file and comparing them to what currently exists in a file called DUMP.FIX.TEMP. All good records in a group are copied to DUMP.FIX.TEMP, then copied back into the re-initialized groups in the file.
Compressing Overflow Frames
While fixing the file, the following message will appear:
Overflow free list has been cleared You can issue a
compress to recover and reorder overflow frames.
..Press any key to continue..
Compressing the frames will remove any unused frames and reorder the overflow frames so that they are contiguous in the overflow (.OV) file. To compress the frames, enter [Ctrl-C] from within DUMP after the fix process has completed.
Note: this process can be quite lengthy for large files.
Advanced NetWare and TTS are trademarks of Novell, Inc.