kb:kb_articles:kb1020

In Memory Hashtables in OpenInsight (Performance)

Created at 25 APR 2013 01:10PM

In Memory Hashtables in OpenInsight

Overview

Storing frequently accessed values in memory rather than on disk is a technique for improving your applications performance.  One useful tool is an in-memory hashtable, also known as key/value storage. The hashtable lets you write a value to a key and read it later.  OpenInsight offers several tools to help you hold values in memory.  Each tool has strengths and weaknesses.

 

Program Description Notes
RTP65 In memory hashtable, C dll Fast.  No more than 100 tables.  Not designed for large number of Items.  No keys method.
RTI_HASHTABLE A wrapper around RTP65 Same as RTP65, easier interface to program, adds keys  method
Revelation.RTIDotNetHashtableA COM interface to the .Net Hashtable Unlimited number of tables

Can store <idispatch> objects as well as strings.  Used by BFS connectors
RTI_HASHTABLE2 A Basic+ wrapper around the .Net hashtable Same as Revelation.RTIDotNetHashtable, with easier interface
Scripting.Dictionary A Microsoft hashtable Ole object you can use for key value storage. Unlimited number of tables.  Not designed for large number of keys.
RTI_HASHTABLE3 A Basic+ wrapper around the Scripting Dictionary  
Variable named commons You can use variable named commons to implement a key/value sort of storageCan store dimensioned arrays, multiple variables. Used by OpenInsight to implement window properties

 

See

RTP65

RTI_HASHTABLE

Revelation.RTIDotNetHashtable

Scripting.Dictionary

Variable named commons

RTP65Subroutine

Description The RTP65 subroutine allows you to read and write key/value pairs to an in memory table.results.
Syntax RTP65( Method, Handle, Key, Record, Reserved, Status )

 
Methods

  		0 - Create Table

1 - Open Table

2 - Close Table

3 - Clear Table

4 - Read Record

5 - Write Record

6 - Delete Record

 

You will want to make equates for these methods. For example

 

EQU CREATE_CACHE$  TO 0

EQU OPEN_CACHE$    TO 1

EQU CLOSE_CACHE$   TO 2

EQU CLEAR_CACHE$   TO 3

EQU READ_CACHE$    TO 4

EQU WRITE_CACHE$   TO 5

EQU DELETE_CACHE$  TO 6

EQU MAX_CACHE_CNT$ to 100

 
Method = Create table rtp65 (CREATE_CACHE$, Handle,  tableName, , , Status)

 

Parameter  Description
Method In 0
Handle   Null
Key In Name of the table to create
Record   Null
Reserved   Null
Status Out0 - Success

1 - Table already exists

5 - Max number of tables exceeded

 

 
Method = Open Table rtp65 (OPEN_CACHE$, Handle,  tableName, , , Status)

 

Parameter  Description
Method In 1
Handle OutHandle for the table
Key In Name of the table to open
Record   Null
Reserved   Null
Status Out0 - Success

2 - Table does not exist

 

 
Method = Clear table rtp65 (CLEAR_CACHE$, Handle,  , , , )

 

Parameter  Description
Method In3
Handle InHandle from previously opened table
Key   Null
Record   Null
Reserved   Null
Status   Null

 

Clear erases all keys and values from the table. The table remains open.
Method = Read Record rtp65 (READ_CACHE$, Handle,  Key, Record, , Status)

 

|Parameter|  |Description |
|Method |In |4 |
|Handle |In |Handle from previously opened table |
|Key |In |Key of the record |
|Record |Out|Value of the record |
|Reserved |  |Null |
|Status |Out|0 - Success

3 - Invalid handle

4 - Table not open or Record not found|

 

  | |Method = Write Record |rtp65 (WRITE_CACHE$, Handle,  Key, Record, , Status)

 

Parameter  Description
Method In 5
Handle In Handle from previously opened table
Key In Key of the record
Record In Value of the record
Reserved   Null
Status Out0 - Success

3 - Invalid handle

4 - Table not open

6 - Memory Error

 

 
Method = Delete Recordrtp65 (DELETE_CACHE$, ,  tableName, , , Status)

 

|Parameter|  |Description |
|Method |In |6 |
|Handle |In |Handle from previously opened table |
|Key |In |Key of the record |
|Record |  |Null |
|Reserved |  |Null |
|Status |Out|0 - Success

3 - Invalid handle

4 - Table not open or Record not found|

 

  |   |Remarks|RTP65 is simple and fast.  Before OI 9 is is limited to a total of 10 tables. As of 9.1 the limit is raised to 100 tabled. Openinsight uses a few tables itself.  RTP65 will slow down when there are a large number of keys.| |  |  |   |Examples |  | |  |  | |Example 1 |Subroutine RTP65_EXAMPLE(table)

/*

**  Demonstrate reading and writing cached recrods

*/

 

Declare Subroutine RTP65

$insert Logical

 

EQU CREATE_CACHE$  TO 0

EQU OPEN_CACHE$    TO 1

EQU CLOSE_CACHE$   TO 2

EQU CLEAR_CACHE$   TO 3

EQU READ_CACHE$    TO 4

EQU WRITE_CACHE$   TO 5

EQU DELETE_CACHE$  TO 6

EQU MAX_CACHE_CNT$ to 100

 

If assigned(table) Else table =

If table = then

    table = 'SYSPROCS'

End

               

Open table To f_table Else

                isOk = false$

                Return

end

 

* Create Cache

CacheName = "MYCACHE"

cache_status = 0

hCache =

 

rtp65 (CREATE_CACHE$, ,  CacheName, , , cache_status)

isOk = ( cache_status eq 0 )

If isOK then

                rtp65 (OPEN_CACHE$, hCache, CacheName, , , cache_status)

                isOk = ( cache_status eq 0 )

End

 

* put all the records in a cache

Call Rlist('SELECT ' : table, 5, , , )

done = false$

ids =

Loop

                Readnext id Else done = true$

Until done Or Not(isOk)

                Read record From f_table, id Then

                                ids := id:@fm

                                rtp65(WRITE_CACHE$, hCache, id, record, , cache_status)

                                isOk = ( cache_status eq 0 )

                end

repeat

 

* pull them all back out

col =

Loop

                Remove id From ids At col Setting mark

While id ne "" And isOK

                record =

                rtp65(READ_CACHE$, hCache, id, record, , cache_status)           

                isOk = ( cache_status eq 0 )

repeat

 

rtp65 (CLEAR_CACHE$, hCache, , , , cache_status)

 

Return | |Example 2 - CACHE_MFS|Subroutine CACHE_MFS(CODE, BFS, HANDLE, NAME, FMC, RECORD, STATUS)

 

/* Name : CACHE_MFS

* Description:

* Cache records in lists

 

* Open – put the cache number in the handle

* Read/Reado – add the record to the cache

* write/delete/clear – Update remote, local copy

* omnievent, FMC = 3 - clear cache

*

*

* Side Effects:

* Cache does not expire, so once read, will not refresh from disk

* Notes :

can call RTP57(OMNI.SCRIPT, BFS, HANDLE, NAME, CLEAR_CACHE$, RECORD, STATUS) clear the cache manually

Will not cache records with key[1,1] eq ‘%’

 

*/

 

$insert Logical

$insert FSErrors_100

$Insert FILE.SYSTEM.EQUATES

 

*$insert Cache_mfs_common

EQU CREATE_CACHE$  TO 0

EQU OPEN_CACHE$    TO 1

EQU CLOSE_CACHE$   TO 2

EQU CLEAR_CACHE$   TO 3

EQU READ_CACHE$    TO 4

EQU WRITE_CACHE$   TO 5

EQU DELETE_CACHE$  TO 6

EQU MAX_CACHE_CNT$ to 100

 

EQU CACHE_HANDLE_DELIM$ to \F7\

Equ CACHE_HANDLE_POS$ To 6

 

declare subroutine rtp65

 

common /cache_mfs_Common/init@,handle_cnt@,names@,file_handles@(max_cache_cnt$),cache_handles@,full_cache_flags@

cache_nr = Field(handle, cache_handle_delim$, cache_handle_pos$,1)

if cache_nr then

                transfer handle to hold

                handle = file_handles@(cache_nr)

                hCache = cache_handles@<cache_nr>

               

                $Insert File.System.OnGoSub

               

                transfer hold to handle

end else

                $Insert File.System.OnGoSub

end

 

Return

* ————– Main Subs —————–

READ.RECORD:

READO.RECORD:

RECORD =

cache_status =

if name[1,1] = '%' then

                GOSUB NEXT.MFS

end else

                rtp65(READ_CACHE$, hCache, NAME, RECORD, , cache_status)

                If cache_status = 0 then

                                status = true$

                End else

                                * read from file

                                GOSUB NEXT.MFS                          

                                if status then

                                                * cache for next read

                                                rtp65(WRITE_CACHE$, hCache, NAME, RECORD, , cache_status)

                                end

                End

end

Return

 

WRITE.RECORD:

GOSUB NEXT.MFS

if status then

                rtp65(WRITE_CACHE$, hCache, NAME, RECORD, , cache_status)

End

Return

 

DELETE.RECORD:

GOSUB NEXT.MFS

rtp65(DELETE_CACHE$, hCache, NAME, , , cache_status)

Return

 

CLEARFILE:

GOSUB NEXT.MFS

rtp65(CLEAR_CACHE$, hCache, NAME, , , cache_status)

Return

 

DELETE.FILE:

GOSUB NEXT.MFS

rtp65(CLEAR_CACHE$, hCache, NAME, , , cache_status)

Return

* —————————————

* non-chained filing system calls

* —————————————

Flush:

Unlock.All:

Record = ""

Status = TRUE$

Return

 

Install:

if Assigned(init@) Else init@ = false$

If init@ else

                init@ = true$

                handle_cnt@ = 0

                mat file_handles@ =

                cache_handles@ =

                names@ =

                full_cache_flags@ =

end

Status = TRUE$

 

Return

* —————————————

* —————————————

* Chained Filing System Calls

* —————————————

LOCK.RECORD:

UNLOCK.RECORD:

GOSUB NEXT.MFS

RETURN

 

*————————————————–

SELECT:

READNEXT:

CLEARSELECT:

RECORD.COUNT:

* —————————————

CREATE.INDEX:

DELETE.INDEX:

UPDATE.INDEX:

SELECT.INDEX:

READNEXT.INDEX:

* —————————————

Omni.Script:

omniCall = If Assigned(FMC) then FMC<1> Else

If omniCall eq CLEAR_CACHE$ Then

   If hCache then

      rtp65 (CLEAR_CACHE$, hCache, , , , cache_status)

   end

End else

   gosub Next.MFS

end

Return

Reserved:

* there is a critical error if this line is reached

Status = FALSE$

Return

* —————————————

OPEN.FILE:

GOSUB NEXT.MFS

if status then

                cache_status =

                locate name in names@ using @fm setting pos then

                                * File has already been opened once

                                * Is it the same file or a new one with same name.  The handles should tell.

                                * If the new handle (RECORD) is different, then the cached records are invalid

                                hCache = cache_handles@<pos>

                                orig_handle  = file_handles@(pos)

                                if record # orig_handle then

                                                rtp65 (CLEAR_CACHE$, hCache, , , , cache_status)

                                end

                end else

                                hcache =

                                handle_cnt@+=1

                                if handle_cnt@ < = max_cache_cnt$ then

                                                * Create Cache

                                                rtp65 (CREATE_CACHE$, ,  NAME, , , cache_status)

                                                rtp65 (OPEN_CACHE$, hCache, NAME, , , cache_status)

                                                rtp65 (CLEAR_CACHE$, hCache, , , '', cache_status)

                                               

                                                * add it to the list

                                                handle_cnt@+=1

                                                names@<handle_cnt@> = name

                                                cache_handles@<handle_cnt@>=hCache

                                                file_handles@(handle_cnt@)= record

                                                full_cache_flags@<handle_cnt@>= false$

                                end

                end

                * embed the cache in RECORD, which is the handle of the file being opened

                if hCache then

                                record = FieldStore(record, CACHE_HANDLE_DELIM$, CACHE_HANDLE_POS$,1, handle_cnt@ )

                end

End

retval = status

Return

 

CREATE.FILE:

RENAME.FILE:

MOVE.FILE:

REMAKE.FILE:

* —————————————-

Open.Media:

CREATE.MEDIA:

READ.MEDIA:

WRITE.MEDIA:

Close.Media:

gosub Next.MFS

Return

* ———- End of Subroutine

* ==================================

* execute filing system chain

* ==================================

Next.MFS:

* Strips this MFS leaving the next fs as first element in array

FSList = delete(BFS, 1, 1, 1)

NextFS = FSList<1,1,1>

if len(NextFS) then

                call @NextFS(Code, FSList, Handle, Name, Fmc, Record, Status)

End

Return

                 

  • kb/kb_articles/kb1020.txt
  • Last modified: 2024/01/30 13:36
  • by 127.0.0.1