The Arev/OI sessions on our XP workstations are locking up intermittently for variable periods of time. We can't quite figure out what is happening since we don't usually don't get any error messages except for maybe a rare FS1019.
I'm running Process Monitor on my workstation and when my OI session stops responding, the State shows "Wait:UserRequest".
Does anyone have any suggestions for troubleshooting this issue? We are thinking about unplugging one workstation at a time for about 10 to 15 minutes to see if the problem goes away when the workstation is disconnected from the switch.
We probably have about 90 workstations that attach to the database server and the vast majority of them are joined to the domain. Ironically, we have no issues surfing any of the folders on that database server when the Arev/OI sessions lock up. We can open folders and documents when surfing with Windows Explorer.
Also, LH45SRVC.EXE goes to 0% CPU usage when our Arev/OI sessions lock up. When the sessions are running ok, the CPU usage for LH45SRVC is 10% to 30% on average. We can't quite figure out why the usage would drop to 0% as all of our users are pretty active.
We have confirmed that there are no IP address conflicts with the database server.
Yesterday, when we rebooted the database server and switched the LAN connection to the other NIC, the problem went away but it came back this morning.
PRIMARY DOMAIN CONTROLLER
Mfg: Dell
Model: PowerEdge 2950
Onboard NIC (dual): Broadcom BCM5708C NetExtreme II GigE
RAM: 8 GB
OS: Windows Server 2003 Enterprise Edition (32-bit)
This one is also running as a terminal server.
BACKUP DOMAIN CONTROLLER
Mfg: Dell
Model: PowerEdge R510
Onboard NIC (dual): Broadcom BCM5716C NetExtreme II GigE
RAM: 4 GB
OS: Windows Server 2003 R2 Enterprise x64 Edition SP2
DATABASE SERVER
Mfg: Dell
Model: PowerEdge 2850
Onboard NIC (dual): Intel Pro/1000MT (auto-negotiate best speed)
RAM: 2 GB
OS: Windows Server 2003 Standard Edition SP2
Universal Driver version: 4.5.00
One other note. If I run a LIST command for a large data table (about 1M records that consumes roughly 200MB of disk space), I can continue to scroll through the records (with the Page Down key) while everyone else is hung up. The NTVDM session for Arev shows 10.5MB for the Working Set (RAM?).
Also, it seems that when I try to access another resource (say by listing another file that's not yet open), Arev will sometimes lock up after I press Enter. It's almost as if a request for a new resource sets a chain of events in motion that causes a temporary seizure. However, I can't duplicate this one every time so it may not mean anything.
We just ran LH45SRVC.EXE in debug mode. It would pause periodically when the Arev/OI sessions locked up. We were unable to tell what operation might have been causing it to pause.
We put all of our servers and a couple of workstations on a single Ethernet Gigabit switch (a Cisco Power Over Ethernet model). We were still getting the intermittent freezes. Then we shut down the primary domain controller, which also handles DHCP for the network. The seizures stopped happening.
We now think that there was a bottleneck of some sort on the PDC that was affecting traffic going to the database server. The traffic begins to build up on the network and the terminal server users on the PDC also start using more of the PDC's resources (lots of Arev sessions). The PDC's performance slowly decays and finally hits a tipping point (maybe there is a queue that gets overloaded with requests) and the seizures start happening. It has to have something to do with the PDC's ability to process the IP-related traffic while also handling its role as a terminal server.
We recently added a backup domain controller and are going to transfer the role of PDC to it, which will hopefully clear up this mess.
Do you have any tables that are around 1 or 2 GB in the LK portion?
World leaders in all things RevSoft
Use Process Monitor on the server to watch the file activity when the pauses occur. Sprezzatura's post starts to look at a badly hashed/sized .LK/.OV file combination. Process monitor can reveal heavy I/O access to the physical files. If the file I/O is slow stations may appear to lock up. I strongly recommend looking at this first.
Running AREV sessions in a terminal server on the PDC is a brave setup. DOS mode applications aren't very friendly at sharing CPU resources, are you running TAME or another utility to ease this? If not, migrating to AREV32 can move your DOS text screens into a windows text screen environment.