MFS PROGRAMMING & USAGE

This book describes Advanced Revelation's MFS (Modifying Filing Structure) technology. The use of MFSs is a powerful and innovative means for developers to gain low-level control over file input and output operations.

MFSs are an outgrowth of Advanced Revelation's approach to modularized filing systems. Unlike traditional database management systems, Advanced Revelation is not bound to specific filing structures. Instead, Advanced Revelation uses a standard protocol for file access, and a series of base filing systems (BFSs) to translate this protocol into the data management requirements of a variety of different structures.

The result is that Advanced Revelation provides to developers a completely open architecture. Programmers are able to control data access all the way from the application (window) level down to the filing system level.

This book focuses on the ability of developers to add a modifying layer to a base filing structure. In effect, an MFS adds functionality without requiring the developer to change the underlying base filing system.

There are two divisions to this book. The first section, "MFS Introduction," provides basic information about what an MFS is and how it functions. The introductory section also describes how to install an MFS. Finally, this section contains general information about programming an MFS, which includes both general programming techniques as well as notes about specific MFS functions.

A second section, "MFS Operations Reference," is a reference guide to all MFS functions. Described in the second section are all the arguments that an MFS can examine and modify for each filing system function supported in Advanced Revelation.

Note: information in this book is current as of Version 1.13 of Advanced Revelation. Changes to filing system functionality are possible with each new release of the product. Developers should carefully examine documentation that accompanies new releases to determine whether changes have been made to filing system protocols that might affect an MFS.

An MFS is a routine that stands between a file operation, such as READ or WRITE, and the filing system itself. The MFS routine has an opportunity to examine the data being used for a file operation, and to modify it before passing the data on to the filing system, or back to the program that requested the operation. In effect, an MFS is a shell around an existing filing system, one that monitors all input and output to that filing system. Another way of defining an MFS is to think of it as a filter through which all file operation requests must pass before the file can be accessed.

This functionality provides access to data at the filing system level. Through use of an MFS, a developer can monitor or "interrupt" all file I/O activity to a file, and modify the data being read or written, or otherwise take action based on the file activity.

An MFS resembles an Advanced Revelation BFS (Base Filing System) in some ways, but is different in both structure and purpose.

A BFS is a program or series of programs that provide the interface between the true physical storage of the data, and the R/BASIC program that makes file I/O requests. A simpler definition is to say that a BFS constitutes a filing system.

A BFS turns R/BASIC file requests into the logic required to access the physical data. BFSs make R/BASIC independent of filing structures by providing this translation layer.

There are as many BFSs as there are ways to store data. Each BFS can have vastly different means for physically storing data, as long as it accepts from and provides to an R/BASIC program a conventional record, namely a dynamic array.

A simplified model for this relationship looks like this:

  
R/BASIC  
program  
 
R/BASIC  
program  
R/BASIC interpreter (AREV.EXE)  
BFS1  
BFS2  
physical data1  
physical data2  

Examples of BFSs in Advanced Revelation include Linear Hash (controlled by the program RTP57), ROS files (RTP51), the memory resident files FILES and VOLUMES (RTP50), Revelation G ROS and LINK files (RTP59), and the logical "DOS" file (RTP53A). In addition, all Environmental Bonds such as those used to access dBASE files, Lotus files, and others, must always contain a BFS to provide the link between Advanced Revelation and the data on the storage medium.

An MFS is an extra layer between R/BASIC and the BFS. Typically, the MFS does not contain logic to access the physical file directly. Instead, the MFS simply monitors file activity, and then passes the file I/O requests, along with any data used for the request, on for further processing.

Because of its much more limited scope, an MFS is usually much simpler than a BFS. Whereas a BFS must in one way or another provide for all possible file I/O operations, an MFS can ignore almost all activity, and concentrate instead on the particular operation it is designed to monitor. An MFS must work in conjunction with the BFS that controls the filing system.

A model for the relationship of an MFS to the BFS looks like this:

  
R/BASIC  
program  
R/BASIC  
program  
 
R/BASIC interpreter (AREV.EXE)  
MFS  
BFS1  
BFS2  
physical data1  
physical data2  

An MFS is a likely solution anytime a developer wishes to monitor or control all activity in a file. Because the MFS sits on top of the filing system itself, it can oversee all operations in the file, regardless of what means a user might find to access the file (except a direct call to the BFS – see "Programming an MFS").

Excellent candidates for MFS implementation include the following functions:

An MFS can be used to monitor all reads and writes to a file, and pass the record through appropriate encryption or compression routines before the data is filed away. When the data is read later, the MFS can call decryption or decompression routines to return the data to its original form. At the R/BASIC level, all file I/O appears normal.

Because an MFS can detect when there is an attempt to read, write or delete records in a file, it can maintain an audit trail of all such activity, or can update indexes based on changes to the record. An audit trail or index MFS is an example of an MFS that does not actually modify the data for the filing system, but simply tracks its usage.

An MFS can use system security information (user names and privilege levels) to grant or deny access to data in the file. For example, the MFS can monitor all reads to a file and deny access to a user who does not have sufficient access privileges. The MFS might also permit some users to write to the file, but not others.

This security can be implemented for the record as a whole, or for individual fields. A possible implementation for the latter might null out fields during a read that are not authorized to a particular user, and replace them before writing the data back to file.

Advanced Revelation makes use of MFSs for a variety of functions. MFSs that are available in Advanced Revelation include:

All files that contain indexed fields are monitored by SI.MFS. If this MFS detects that a change has been made to an indexed field, it creates a transaction that is later used to update the appropriate index. SI.MFS also traps certain retrieval calls such as READNEXT and fulfills them from an index, rather than directly from the file.

If installed, QUICKDEX.MFS and RIGHTDEX.MFS monitor all writes, selects, and deletes against a file. During a write, these MFSs update a hidden record in the file, maintaining a sorted list of keys for records in the file. During a SELECT and READNEXT, QUICKDEX.MFS and RIGHTDEX.MFS simply read this hidden record, providing almost instantly a sorted list of record keys. At the same time, the MFSs hide the record by removing its name from any select lists generated by the user.

Two major functions are rolled into DICT.MFS. First, the MFS monitors writes to any dictionary (any file with a name beginning with the characters "DICT."), and calls the dictionary compiler whenever an F or S type record is written to the file.

Second, DICT.MFS examines all records being written to look for indexing flags. If any are found (for example, if the sixth field is set, a Btree index has been established for that field), DICT.MFS calls additional system routines used to create an index for that field.

DICT.MFS differs from other MFSs in Advanced Revelation in that it is installed "on the fly." The mere presence of the trigger characters "DICT." at the front of a file name causes the ATTACH process to install DICT.MFS onto the file automatically. Other MFSs must be installed explicitly. (Information on installing an MFS appears in the chapter "Installing an MFS.")

Note: specific information about each system MFS appears in Appendix 2.

Although MFS are used for a great variety of purposes, the basic structure and logic is much the same for any MFS. This chapter provides general information about the construction and programming of an MFS.

An MFS is a subroutine like any other in Advanced Revelation. Because of this, developers are free to create an MFS in any manner they choose, as long as the calling protocols for the subroutine are observed.

An MFS can be written in R/BASIC, in C, or in assembly language. Developers can choose the language best suited to the application of the MFS, and to their coding preference. For example, an MFS that compresses data might best be coded in a low- level language for speed purposes.

Advanced Revelation file operation requests originate in R/BASIC commands such as READ, WRITE, DELETE, LOCK, UNLOCK, SELECT, READNEXT, or in TCL commands such as MAKEFILE, NAMEMEDIA and ATTACH. Developers write these commands into their programs, or execute the commands from TCL, to control the flow of data to and from the file.

When a program or TCL command executes, each file-oriented statement becomes a call to the filing system. For example, if an R/BASIC program encounters a READ statement, and the file being read is a Linear Hash file, Advanced Revelation generates a call to the program RTP57, the Linear Hash BFS, with the appropriate "read" code.

Because R/BASIC is independent of filing systems, the call arguments for filing system subroutines use a standard protocol. R/BASIC produces the same subroutine call for all filing systems, always using the same arguments. A READ statement, for example, always produces the same BFS call, regardless of what filing system is actually being accessed by the read logic. By extension, all MFSs must use the same calling conventions.

If an MFS has been installed onto the file being accessed, the MFS "traps" or "interrupts" the call to the underlying filing system. Instead of calling the BFS, Advanced Revelation calls the MFS first, passing it the same arguments that would otherwise have been passed to the BFS.

Because the full list of arguments is available to the MFS, including the name of the operation to be performed, the MFS can examine and manipulate any of the arguments.

Although an MFS can be created completely from scratch, it is easiest to use one of the R/BASIC MFS shells provided by Revelation Technologies. The shell simplifies MFS creation, since a developer can concentrate on the logic required by the MFS.

Two MFS shells are provided on the accompanying diskette. The first is called MFS.SHELL1, and the second MFS.SHELL2. The two shells differ only in the method they use to determine what function is being executed. The use of one shell or the other is primarily one of programmer preference.

An MFS is called from R/BASIC with seven arguments. As with any R/BASIC subroutine, the arguments can be named at the programmer's discretion within the MFS code. The convention used in this book, however, uses the following names for the seven arguments:

SUBROUTINE MFS(CODE, BFS, HANDLE, NAME, FMC, RECORD, STATUS)  

The value of the arguments is dependent on what function (open, read, write, etc.) has been called. Details about the values passed and the return values expected appear in the section "MFS Operations Reference" later in this book. The following list, however, provides general overview of the function of each argument:

An MFS is called once for each file operation (note that one R/BASIC or TCL file operation may translate into several MFS calls). Each file operation typically requires separate logic within the MFS. For example, an MFS will usually execute different logic for a read and a write operation.

The first argument passed to the MFS (the CODE argument) contains an integer value that indicates the operation that has been requested. For example, if the operation is "read a record," CODE will contain the value 1. If the operation is "delete a file," CODE will contain the value 15, and so forth. In version 1.1 of Advanced Revelation, there are 28 possible values for the CODE argument. The full range of values for the CODE argument is described in the "MFS Operations Reference" section.

Note: standard constant names for the CODE argument are provided in the record ON.FILE.SYSTEM.GOSUB (see Appendix 4 for a listing of these code constants). These standard names are used within this book for clarity.

The MFS must determine what operation is being performed, and branch or "dispatch" to the appropriate logic within the MFS. There are two common ways to accomplish this task (the two different methods are illustrated in the sample MFS shells MFS.SHELL1 and MFS.SHELL2).

The first method uses the R/BASIC ON … GOSUB statement to branch to a local subroutine for each file operation. The CODE argument provides the index value for the ON … GOSUB statement. The GOSUB portion is followed by a list of 28 statement labels. Each statement label identifies the logic within the body of the MFS program for a particular file operation.

A typical statement begins like this:

ON CODE GOSUB READ.RECORD, READO.RECORD,‹  
   WRITE.RECORD, DELETE.RECORD, LOCK.RECORD ...  

For example, if a record lock operation is being executed, the value of CODE will be 5. The ON … GOSUB statement will branch to the fifth statement label, in this case LOCK.RECORD.

Very frequently, the MFS is concerned with only a small number of file operations, such as READ.RECORD and WRITE.RECORD. In this case, the logic for most remaining operations can be lumped under a single label in the MFS. The MFS might "stack" statement labels that have code in common, as in this example:

READ.RECORD  
(record read logic here)  
RETURN  
  
CREATE.FILE:  
RENAME.FILE:  
DELETE.FILE:  
("do nothing" logic here)  
RETURN  

A second method to dispatch within the MFS is to use a CASE block. In this method, the value of CODE is tested in a series of CASE statements to determine the logic to be executed for the operation. For example, this code fragment illustrates a portion of the CASE logic:

BEGIN CASE  
CASE CODE = 1  
(read logic here)  
CASE CODE = 2  
(read-only logic here)  
CASE CODE = 3  
(write logic here)  
(etc.)  
END CASE