Okay, this issue has me tearing my hair out. We recently migrated from Novell servers running NLM 5 to Windows 2003 servers running UD4.6. We are running Ceridian's HR-1, which is basically ARev 3.02.

From a client PC, HR-1 runs just fine. It is also running just fine on the Citrix server we initially tested on. Record locking works, unique station IDs are set, EMS is allocated, shortcut is using /x /m4096, the whole nine yards. However, an issue arose when we published HR-1 on our Production Citrix server.

Basically, EM Used from the WHO screen is incrementing constantly and practically never goes back down. Where this is causing the biggest problem is with SELECT statements hitting multiple indexes. After only 2 or 3 processes, anything running a select breaks to the debugger with INDEX.REDUCER throwing a B10 VNAV error.

Again, on the Dev Citrix server, all of this worked flawlessly. I can find absolutely no differences between the two Citrix servers. I have compared the autoexec.nt, config.nt and _default.pif settings. To the best of my knowledge, no registry tweaks have been applied to either server. We are using TameDOS on both as well.

We really don't want to go back to deploying on the desktop if we don't have to, but a majority of our reports are breaking, as well as some programs doing large selects. Also, if I break the selects down to sequentially hit the indexed fields instead of doing it all in one big select, it works, but I really don't want to rewrite several hundred processes if I don't have to.

Any thoughts, ideas, help, or mystical wizardry would be greatly appreciated.

Matt Sorrell

Does $IDX_SETS exist in that version of AREV?

The Sprezzatura Group

The Sprezzatura Blog

World leaders in all things RevSoft

Yes, it appears so. I found $IDX_SETS in SYSOBJ with a version number of 3.1.13.

Is the deployed machine multi-processor/multi core and the other not?

The Sprezzatura Group

The Sprezzatura Blog

World leaders in all things RevSoft

I believe they are both multi-processor, multi-core, but I'm not positive. Let me check with the server admins and verify that.

Does multi-core cause a problem with ARev?

Yes, specifically with IDX_SETs. We started to write a replacement but got side tracked on other projects. Naturally this refers to the processor the AREV is running on.

The Sprezzatura Group

The Sprezzatura Blog

World leaders in all things RevSoft

Both Citrix servers are physical (not virtual) servers with a multi-core multi-processor architecture (multiple multi-core processors).

I'm assuming that the CPU of the server actually hosting REVBOOT is immaterial, just the CPU of the server where AREV.EXE is being executed.

Oh, and according to my SysAdmin, the two servers are identical in hardware architecture. Basically, it looks like they bought two units of the exact same model and config and implemented one as the Prod server and one as the Dev server.

Hmmm well there is a known IDX_SETS issue that causes long and/or multi column selects to fail. You could TRY setting the affinity of AREV.EXE to a specific processort?

The Sprezzatura Group

The Sprezzatura Blog

World leaders in all things RevSoft

I asked our admin about that, and he didn't think it was possible in Citrix. I'm trying to research it now and see if there's a way to do that.

Try without TameDOS using a separate short-cut / desktop Icon / batch file. I vaguely recall that one version of TameDOS upsets AREV's stack pointers in EMS memory (which is basically a string space extension). Arev uses the stack segment register unconventionally - not for the stack but to get at stuff in its own memory space.

I have a substitute if this is the case. Citrix also has its own TameDOS equivalent, if I recall.

Also check the Environment size on both servers. The program segment prefix for the AREV session is offset under some CMD / command shells.

CMD versus command.com (multiple versions?) is another consideration. Any stray command.com may be used in shelling out by AREV and can hiccup EMS segment pointers.

The "START" command has a switch named /AFFINITY . This may do what you need if you use START to launch arev.exe with the /AFFINITY switch.

Without TameDOS we can only run ~10 instances of the app before it brings the whole server to its knees. We are using the most recent version of TameDOS. Also, we are using the same version of TameDOS in both environments, the one that is working and the one that isn't.

When you say check the Environment size, what exactly do you mean? Check it where?

We are using Command.com to run AREV.EXE as CMD.EXE was giving us all types of weird errors.

Well - I'm thinking, what could change between two implementations, same hardware, same OS.

And it must therefore be a config issue. I'm looking for a problem which would upset the implementations. It has to be memory related. So what can affect available memory? The EMS is used for string space, if I recall, and is divided into a couple of 1MB sections and a 2MB section (could be mistaken - Pat McN discussed this on compuserve back around 95). So how would this creep? By something being offset in memory by around 16 bytes. Then the creep begins.

Video resolution I'd check. Video themes I'd turn off. COMSPEC I'd set in the environment.

The DOS environment can be had by doing a set]junk.txt and checking the resultant file size.

I reckon you've got COMMAND.COM and CMDs being invoked involuntarily.

You could create an expendable version of calculatex, but I think this would only be a cludge.

We solved the idx_sets problem by stopping searches against multiple indexed fields using an AND condition. If searching against multiple indexed fields we now chain multiple PERFORM SELECT.. statements. It's not as fast, but it is acceptable and it does not break.

We have not found the multi-processor/multi-core issue to be true. We found one article about an older dual core Intel processor that had a bug, but it was quickly fixed. This bug did not apply to quad core processors, nor to Xenon processors. It was only one iteration of the dual core chip.

Further, we have had numerous multi-processor machines, as well at 2 and 4 core desktop machines in our office for the last 9 or 10 years. Currently, it is all we use. We never see idx_sets problems on these machines.

However, we do see the idx_sets problem on servers running multiple sessions of arev. For example, we see it on servers running multiple Terminal Services sessions (Citrix is essentially Terminal Services).

Since most servers have multiple processors/cores, and most workstations don't, it might be easy to make this interpretation. But if the problem was caused by the number of cores, why would the problem appear only on a server and not a multi-processor/core workstation?

I suspect it has more to do with one session of arev stepping on the temporary work files created by another arev session on that same machine. Check your temp files after this error. You will find they don't clean up as they should. Check their contents.

Concerning a work-around, the idx_sets problem is triggered only when a select or btree.extract involves 2 or more indexed columns using an AND condition. We avoid this problem by using multiple, chained select statements instead of a single btree.extract or select statement that uses AND conditions against multiple indexed columns. It's slower, but it does not break.

My guess is that arev creates a temporary work file to handle the merging of the two result sets and the temporary file gets stepped on by a different session. (arev is not very good at selecting unique random temporary file names). I understand the problem does not occur in arev32. Perhaps it merges results sets differently, or does a better job of creating unique temp file names.

View this thread on the forum...