Memory Management In Revelation (TB #1) (Functions/Subroutines/Programs,Performance)

Memory Management In Revelation

 

The size and complexity of applications being developed with Revelation Release G and Advanced Revelation merits special consideration of memory management. Good applications design requires that the programmer consider the growth potential of the application, and assure that there will be adequate memory space to accommodate expansion.

 

The following is a discussion of some techniques that can be used to optimize memory use and availability. Most topics are applicable to both Advanced Revelation and Revelation Release G; topics not mutually applicable will be noted as such.

 

The topic of memory management will be addressed from both inside the Revelation environment, as well as at the DOS level. The techniques detailed in this discussion are offered to the developer as tools with which to fine-tune an application.

 

Note: No attempt is made in this article to address the effect on performance of various memory management techniques. Programmers should be aware of these performance issues when designing systems that use memory efficiently.

 

Addressing the Memory Crunch

 

Revelation handles dynamic allocation of memory for programs and data, manages file buffers, and maintains pointer tables. Its dynamic treatment of memory space makes it very flexible in changing environments. It will take advantage of all available memory, and tailor its behavior to the amount of memory that is available. If there is only a minimal amount of memory, Revelation will keep less data in RAM, and make disk accessses more frequently. If the full 640K addressable by Revelation is present, it will keep more data in RAM.

 

As applications grow in size and sophistication, added demands on memory space may reach a critical point and produce the error message:

 

Out of string space - Not enough memory

 

The solution to this problem is to make more memory available for Revelation. If the computer has less than 640K, the answer is easy: install more memory. If the computer already has 640K, the answer is a bit more difficult. Through version 1.0 of Advanced Revelation, the system will not address memory above the 640K limit. In version 1.1 of Advanced Revelation, memory up to 704K can be addressed, so long as this memory is not otherwise being used as video memory.

 

Given these limitations, adding more memory will not give Revelation extra space (see, however, the section on virtual disks in extended memory). The only solution is to examine the application and its environment closely and determine how they might be modified to use the memory space available more effectively, without adverely affecting functionality or performance.

 

The DOS Environment

 

The AUTOEXEC.BAT and CONFIG.SYS files are good places to start when evaluating an application's use of memory. These files are called each time the system is booted, and may be initializing routines, etc., that if not crucial, may be taking up memory unnecessarily.

 

CONFIG.SYS DEVICE

 

This command specifies a device driver to be used by the system. These might include mouse drivers, or drivers for storage devices, etc. Examples include MOUSE.SYS, ANSI.SYS, etc. If some of these are not being used, trim them from the list.

Special mention should be made of network shell programs. These are utilities that are loaded into workstations to allow them to communicate with a local area network. Since these are memory resident, the size of the shell should be taken into account when selecting LAN hardware and software, especially when memory is at a premium. These shells take up varying amounts of memory, depending on the network environment. Sizes vary from around 30K to 140K.

 

BUFFERS

 

This specifies the number of memory buffers that DOS will use in reading and writing to disks. Each buffer assignment allocates 528 bytes. The default number of buffers for an XT type machine is two, for an AT type, three. Additional buffers assigned will reduce the amount of memory available to Revelation.

 

SHELL

 

The shell designates the command processor that will be used, and also may be used to change the environment size from the default value of 160 bytes. Unless specific requirements dictate otherwise, the default should be used.

 

AUTOEXEC.BAT

 

Memory resident programs, or TSRs (Terminate, Stay Resident programs) may be loaded with the execution of the AUTOEXEC.BAT. TSRs such as Sidekick, Prokey, etc., remain in memory until the computer is restarted. Although convenient, TSRs reside in memory space that cannot be used by other applications, regardless of whether the TSR is used or not. In addition, some memory resident programs may conflict with Revelation by attempting to use the same memory space.

 

Some TSRs can be configured to use extended memory. If configured to do so, the TSR can be used without reducing memory space available to Revelation.

HARDWARE The system hardware configuration affects the overall performance of the application. There are several items to consider with respect to memory and performance evaluation.

 

Main Memory

 

While Revelation will run in 320K, and Advanced Revelation will run in 512K, 640K is recommended. Both systems will run faster and more effectively with the maximum addressable RAM.

 

An enhancement in version 1.1 of Advanced Revelation will permit the use of memory up to 704K. However, this is possible only if the area from 640K to 704K is not being used for video memory.

 

Extended Memory

 

While Revelation products do not address extended memory, it can be useful when implemented as a RAM disk. Refer to the section on virtual disks later in this article.

Coprocessor Both Revelation and Advanced Revelation will take advantage of a math coprocessor chip. If the coprocessor is not present, emulation software is used. If a coprocessor is present, about 13K is gained as the system frees the memory space otherwise taken by the emulation software.

 

Memory Considerations from within Revelation

 

The area over which the developer has the most control of memory management is in the design and integration of the application. Efficient use of memory requires careful, well planned programming techniques. Inefficient technique can lead to wasted memory and slower execution, but well planned applications with streamlined code and tightly coupled modules run fast and lean.

 

Program Management:

 

The Program Stack Revelation manages programs in use by maintaining a program stack. The program stack is an array of all object code currently residing in memory.

When an external subroutine is initally called, the object code for the routine is loaded into the progam stack from the disk. If the subroutine is called again, Revelation looks for the routine in the programs stack, and loads the program from there instead of from the disk. The object code for a subroutine is not cleared from the program stack until the main program terminates.

 

Nesting

 

Nesting occurs as routine calls an external subroutine. After the called routine is executed, control is passed back to the calling routine at the point that the call was made. Several levels of nesting are possible with subroutines calling other subroutines.

Deep levels of nesting in an application can be costly in terms of memory consumption. The system must 'freeze' the calling routine, save all variable values, and remember where to return. The demand on memory increases with each additional level of nexting.

Since a subroutine will remain in the program stack until the main calling routine terminates, subsequent calls to that subroutine within that span will not require additional memory fo the object code. Additional variable space, however, will be allocated as the routine requires.

 

Excessive and unnecessary nesting should be avoided because of the additional memory requirements. Restructuring the coupling of programs might lead to more efficient use of memory. Consider the following example:

 

Main

            Sub 1

                        Sub 2

                                    Sub 3

 

This example demonstrates multiple levels of nesting. The main program calls subrotuine 1, which calls subroutine 2, which calls subroutine 3. Return to main must pass back through each subroutine.

 

All subroutines will remain in the program stack until the main program terminates.

This process may be better served by the following structure:

 

Sub 1   Main Sub 2 Sub 3

 

This structure controls all calling of subroutines through the main program, and limits the level of nesting. The advantage of this structure, in terms of memory conservation, is that the local variables belonging to the subroutines are abandoned as the subroutine exists. This frees up string space and descriptors for use elsewhere. The subroutines, as before, are retained in the program stack until the main program terminates.

 

A variation on this structure will clear all of the subroutines from the program stack after they are called. This is accomplished by inserting a main program between the original main program and the subroutines. The inserted main program functions as a dispatch routine, accepting subroutine names and arguments from the original main program, and calling the appropriate subroutine. The logic can be illustrated like this:

 

Sub 1 Main1 Main2 Sub 2 Sub 3

 

This technique requires passing parameters from Main1 to Main2 via global variables. Main1 loads the subroutine name and arguments into @USER0 and @USER1, respectively, and calls Main2. The code for Main2 is:

 

SUB = @USER0

ARG = @USER1

CALL @SUB(ARG)

STOP

 

After each subroutine terminates, control returns to Main2. When Main2 executes its STOP statement, it (and the subroutine) is cleared from the program stack, and control is returned to Main1.

 

Another possible structure, depending on the program flow requirements, is possible with the R/BASIC CHAIN instruction:

 

PGM1—→PGM2—→ PGM3

 

This allows sequential execution of processes without nesting. When the program that executes the chain is terminated, it is removed from the program stack, and descriptors and string space are recovered.

 

R/BASIC routines called with CHAIN must be CATALOGed. The CHAIN statement is similar to the PERFORM statement except control is not returned to the calling program.

 

As an example of CHAIN, this program will execute, exit, and run PROG.NEXT:

 

PRG.1

* (Processing for PRG.1 here)

* all done now…

CHAIN "NEXT.PROG"

 

Using CHAIN does have the disadvantage of limited sharing of data. Any passing of parameters would have to be accomplished through global variables or labeled common variables.

 

Expendable Suboroutines and Virtual Disk Drives (Advanced Revelation only)

 

Expendable Subroutines

 

Advnace Revelation provides the EXPENDABLE SUBROUTINE declaration to provide flexibility in program stack maintenance. A subroutine declared as EXPENDABLE is removed from the program stack immediately after it terminates, freeing string space for other use. If the subroutine is called again, it is reloaded from disk.

 

The following is an example of an EXPENDABLE subroutine declaration:

 

EXPENDABLE SUBROUTINE TEST.IT

*/this is an expendable routine /*

 

 

Extended/Expanded Memory

 

In systems with extended memory, RAM disks that address memory above 640K can be used very effectively in conjunction with expendable subroutines.

 

Expendable subroutines loaded into a RAM disk conserve space in the program stack and maintain RAM resident speed.

 

Initialize the RAM drive as a Revelation volume with the NAMEMEDIA command. Files containing EXPENDABLE SUBROUTINES can be moved to the RAM disk with the FILECOPY command. Then reATTACH the RAM volume. The subroutines can now be accessed with virtually the same speed as those in the program stack. Because they are expendable, they conserve program stack string space.

 

Warning: Using a virtual disk in the development phase can be risky, as changes to records in the memory-resident files will not be saved to disk unless you explicity copy the file from the RAM volume back to the hard drive.

 

Developers may wish to make loading of expendable subroutines into RAM disk a part of the Advanced Revelation logon process.

 

Treatment of Variables

 

Descriptors In Revelation, descriptors refer to vaiables. The descriptor table is a list that is maintained in memory, used to store information about variables throughout the system. Upon initialization, the system attempts to allocate 64K for the descriptor table. If it cannot be allocated, the system splits available RAM, with 2/3 for string space and 1/3 for the descriptor table.

 

Each entry in the descriptor table is 10 bytes long. The first two bytes comprise a header, and the following eight bytes may contain either data or pointer information. If the variable holds a number or a string that is eight bytes or less in length, that actual value is stored in the descriptor table. Otherwise, a pointer to the data is entered into the descriptor following the header. Thus it is better to use short (less than nine bytes) strings for frequently used flags or for parameters frequently passed to external functions.

 

Variables take one descriptor each. Dimensioned arrays require a descriptor for each element in the array. Dynamic arrays require only one descriptor, since they are simply one string of characters.

 

After the system common variables and system program variables have been declared and allocated, there is space for about 5,000 descriptors in the table. If an attempt is made to assigne variables beyond the capacity of the descriptor table, the system produces as error message "Maximum number of variables exceeded."

 

Scope of Variables

 

Different variable types have varying scopes. The descriptors and contents of global variables such as @RECORD and labelled common variables will remain in memory until they are reassigned. The descriptors and contents of unlabelled common variables remain in memory until the main program that initialized them is terminated. The descriptors and contents of local variables remain in memory until the routine is terminated. Local variables assigned in a main program are not directly addressable by an external subroutine, but remain in memory until the main program is exited.

 

Clearing Variables

 

Watch for wasted string space taken by variables. The scope of variables should be considered when making assignments. consider the follwoing assignment X=Y. If X and Y are local variables, both will continue to hold the original value of Y until they are reassigned or the program is exited. If Y is not used later in the program, set it to null to release the memory space.

 

Pay special attention to assignments made to common variables. Unused variables will continue to take up memory space until the common is cleared or terminated. See the discussion below on common variables for their specific scopes.

 

Transfer Statement

 

The TRANSFER statement can be used to reassign variables without creating duplicate images in memory. The following code:

 

TRANSFER A TO B

 

will reassign the pointers in the descriptor table for variables A and B, giving B the value of A, and setting A to null. Becasue no additional memory needs to be allocated for this operation, it is faster than conventional assignment.

 

Utility Variables

 

Reuse variables: use the same variable for all FOR…NEXT loops (those not nested), inputs, temporary holding variables, etc. Developing standard variables for these functions conserves space by not assigning additional variables.

 

Dimensioned Arrays

 

Dimensioned arrays consume descriptor table space very quickly. An entry is made into the table for each element in the array. A 10 by 10 array, for example, consumes 101 descriptors (100 elements plus element 0). If the routine is called recuresively, another 101 descriptors are consumed for each level of recursion. If the data in the array elements exceeds 8 bytes, additional string space ouside the descriptor table is used.

 

Dynamic arrays can be used as a substitute for dimensioned arrays. Although they are slower, the entire array requires only a single entry in the descriptor table.

 

COMMON variables

 

Unlabeled common variables are initialized by a main program, and are addressable by the main program and any external subroutines that declare the common variables. Unlabeled common variables are available until the main calling program terminates. Unlabeled common area can contain up to 255 variable descriptions.

 

Labeled common variables are similar to unlabeled commons, except they have exteded scope. Labeled commons continue to be available after the initializing main program terminates. These are useful for sharing data between main programs, or between subroutines not related via the same calling program. Labeled common areas can contain up to 255 variable descriptions.

 

Filing Systems

 

The filing systems of Revelation use memory for buffering of information coming from or going to the disk. Buffers can speed data access time by holding frames of data in memory. When a disk access is made, the system checks the buffers first to see if the data is already in memory. Each of the different filing systems utilize memory differently.

 

Release G

 

The ROS filing system uses variable size frames. Buffers are allocated dynamically, and each may be up to 64K. While the system will buffer up to nine ROS frames, total buffer size cannot exceed available memory size.

 

When memory space is extermely tight, ROS files may present a problem. Before a frame is read, the system will attempt to allocate 64K to buffer it, even if the ROS frame only contains a small quantity of data (the strign space used byt he frame is adjusted after the frame has been read). If the initla 64K simpley cannot be allocated, the ROS filing system will generate an "Out of string space" error.

 

Converting a ROS file to a LINK file can save memory by eliminating the large buffer overhead. LINK files allocate a very small buffer space in memory (1K)

 

Advanced Revelation

 

Advanced Revelation's Linear Hash filing system, in non-network configuration, automatically buffers six frames. In a networked configuration, Advanced Revelation buffers only one frame.

 

The default size for LHASH frames is 1,024 bytes, resulting in a buffer size of 6K. The developer, however, may specify a different frame size when makeing or remaking a file. Frame size can be set from 256 to 10,000 bytes.

 

The size of the Linear Hash buffers is set to the largest frame size encountered during the current session. For example, if the frame size of a file is set to the maximum of 10,000 bytes on a non-networked machine, the buffer size increases to 60,000 bytes. The total buffer size will not decrease until Revelation is completely existed and executed again.

Using MEMSPACE The function MEMSPACE can be used to check the amount of memory currently available while inside Revelation. It requires one argument, which is the number of bytes requested. MEMSPACE returns zero if the memory requested can be allocated; otherwise it will return the number of bytes available. By passing it a very large number, it will always return the amount of string space available:

 

DECLARE FUNCTION MEMSPACE

X = MEMSPACE(99999999)

PRINT X

 

MEMSPACE is included in Advanced Revelation. Release G users may download MEMSPACE from the Revelation Technologies Bulletin Board System. The number is (206) 641-8110.

 

Conclusion Memory management in Revelation should be approached both from within the application and the DOS environment. The developer can control memory usage by addressing several different aspects of the operating system, hardware, and software configurations.

 

DOS

 

Make maximum memory available for Revelation by reducing memory use at the DOS level. Avoid excessive buffer assignments. Elimate unnecessary memory resident programs.

 

Hardware

 

The computer should have 640K installed. A 80x87 coprocessor will reduce memory requirements by about 13K.

 

 

Program Management

 

Maximize available memory by managing the number of programs held in the program stack. Advanced Revelation users with extended/expanded memory should load expendable subroutines from a RAM drive.

 

Variable Management

 

Use short strings (less than nine bytes) as much as possible. Longer strings should be nulled after they are used. The TRANSFER statement should be used in variable assignment. Common variables can be used to share arrays between programs.

 

Filing Systems

 

Link files have lower memory overhead for I/O than ROS files. Advanced Revelation Linear Hash filing system buffers six frames in memory. Increases in frame size for any file will reduce the available memory space as the buffer size must increase.