I am seeing a massive slowdown in OI when running under Windows 7.

UTF-8 is off.

Here are some benchmarks

CPU

OS

Passmark

CPU mark

OI speed

Core2 Duo T7700 2.4GHz

Win Vista

660

1331

5min

Core i7 870 2.93GHz

Win 7 32 bit

1575

5431

5min

Core2 T7200 2GHz

Win 7 32 bit

678

1153

17min

Pentium 4 2.80GHz

Win XP

281

383

18min

The CPU mark and Passmark are from http://www.passmark.com/products/pt.htm

The OI speed is my RBasic code running a bunch of calculations and string manipulations.

If you compare the two Windows 7 machines, the second machine is about 3x faster than the third machine and speed of OI is about 3x as fast. That makes sense.

However the first machine is running OI at the same speed as the second machine and that makes no sense at all. The second machine should be much faster.

Why is Windows 7 so much slower?

Does anyone else see OI running under Windows 7 to be so much slower than XP or Vista?

Jim,

Have you done any benchmarking on Win7x64?

I ask this because a) on client sites we initially started rolling out Win7x32 (presuming compatability); then re-focussed and re-installed / continued rollout Win7x64. Winx64 solved sooo many bottleneck issues; on a informal basis seemed OI was OK / better than XP - but that may have been influenced recovering from Win7x32.

Richard Bright

No I have only tried Win7 32-bit, that's what I have on my machine and also what the customer has.

Would any one from Revelation care to comment on this, I have a customer giving me hell because the software is running so slow.

Let me clarify the questions so I understand. The first observation is that the Core i7 machine scores better on the PassMark tests than the T7700 machine but the same performance gain isn't realized in OpenInsight. Is this correct? If so, I would consider the bottleneck isn't the CPU but a network or disk I/O constraint. Windows 7 really accelerates disk I/O on subsequent read operations by using free RAM as a system file cache.

The second observation about the T7700 and the T7200 systems being so dis-similar is interesting and I need more information. Can you post the test code? Is OpenInsight run from a network or local installation? What version of the network service/OpenInsight was tested? Does it perform the same on subsequent tests (i.e. warm starts)?

How is OI benchmarked. Is OI a local copy or network installation? What network LinearHash Service is used? Can he post a copy of the test code?

There could be a variety of reasons, the most obvious may be that the CPU isn't the bottleneck in the test since he primarily compares the CPU score. It may be a disk/network bottleneck that limits his performance on the vista and windows 7 machine.

The question is why does OI run slower on Windows 7?

The benchmarks are from a commercial program written to benchmark different machines, you can get a copy here: http://www.passmark.com/products/pt.htm

The CPU mark is an average of a series of different CPU benchmarks. The PassMark is an average of a series of benchmarks CPU/Video/Disk/Memory etc.

The code I am running in OI and using for the benchmark runs almost entirely in memory and runs calculations and manipulates strings. It does next to no disk I/O. Just to test, on my machine I created a ram disk and copied the OI data files that are accessed to the ram disk. I saw no change in speed hence disk I/O is not a bottleneck. The tests are all local copies of OI. All are running exactly the same code against the exact same data-set. We are using All Networks driver 2.1 and OI 9.2.1

If you look at the CPU / PassMark and compare them to the values for the OI benchmark, the OI benchmark is way too slow when running on Windows 7. When not running Windows 7 the relative speed of OI roughly matches the benchmarks.

As to posting the test code I cannot as it was a complete run of our scheduling system with a massive data-set. I have written a short of piece of code that replicates the problem. I know I can speed this code up by pre-allocating memory to the string, that however is not the point. Why does the following code produce the following results?

Tree_Item=ABC":@FM:"DEF":@FM:"GHI":@FM:"JKL":@FM:"MNOP":@FM:"QRS":@FM:"TUW":@FM:"XYZ"

Tree_Export="

For I=1 To 500000

If Tree_Item # '' Then  

  Tree_Export := I  

  Tree_Export := Char(1)  

  Length_T=Len(Tree_Item)  

  Tree_Export := Length_T 

  Tree_Export := Char(1)  

  Tree_Export := Tree_Item  

End  

Next I

* write eof

Tree_Export := Char(26)

OsWrite Tree_Export To "C:\test.dat"

CPU

OS

Passmark

CPU mark

OI speed

Core2 Duo T7700 2.4GHz

Win Vista

660

1331

88 secs

Pentium 4 2.80GHz

Win XP

281

383

108 secs

Based on the CPU benchmark the first machine is over 2x as fast. Based on PassMark it's over 3x as fast. However the OI code runs at almost the same speed, why?

I posted the wrong benchmarks, cut and paste error. Here are the correct ones.

CPU

OS

Passmark

CPU mark

OI speed

Core2 T7200 2GHz

Win 7 32 bit

678

1153

88 secs

Pentium 4 2.80GHz

Win XP

281

383

108 secs

Jim,

Just wanted to post some of my thoughts based on your testing. I think your OI script is testing primaryly one aspect of the faster machine where Passmark is providing a better overall gauge.

Most of what you did in the script was string manipulation which would be a good test of how fast the processor can read/write to memory but probably not a good test of raw computing power.

How about changing your code to do integer math without any string manipulation and see how it differs between your test machines? I think you will see better results…

HTH,

John

My application as a whole exhibits the same slowdown. I agree that the code I posted only tests one aspect (string building) but it also reflects the same overall slowdown that I see.

…and a more importantly similar machine not running Windows 7 does not exhibit this slowdown.

Yes it would be a fairer test to see if the same hardware peforms differently under the two OSes

I have tested several different systems. If the O/S is Vista or XP then the benchmarks roughly match the relative speed of OI. However when I test on Windows 7 system my app written in OI runs much slower than it should.

Jim,

I am not disputing the fact. I do know that you have to be careful when comparing apples and oranges. Btw, I ran your tests on my Windows 7x64 system (Intel Core i7 950 running at 3.07 GHz). With OI 9.2 your test took 22.9 seconds. Running in XP-Mode (Virtual Machine in Windows 7) the same test took 29.5 seconds. Which is really quite good. I will have to fire up an old XP system to see what the difference is.

When I changed the test to:

For I=1 To 5000000

 x=i + i

next i

it took 0.83700000000681

Try running this test on your XP and Windows 7 to see if there is a difference…

I don't care what speed an integer calculation takes.

The whole point is my application slows down under Windows 7. I have shown this to be the case with a very simple piece of code.

Running an VM is meaningless.

Right, Rev is always touting that one of the things OI/ARev does best is string handling. And I'm willing to bet that the majority of OI/ARev applications are primarily concerned with string manipulation rather than integer math.

A benchmark involving string handling seems more relevant.

This post has two main points.

First, are you comparing the PassMark results and trying to correlate it to gains in OI performance? I.e. the Core2 T7200 2GHz is X% faster than the Pentium 4 2.80GHz so OI should be X% faster too. Is this one of the points you are trying to make?

Second, I've run the test on XP and 7 32 bit using the SAME HARDWARE. The test system was a OptiPlex 740 with an AMD 64 3500+ CPU which runs at about 2.2 Ghz. It should be comparable to the Pentium 4 system you have.

Over several iterations in Windows 7 the loop completed in 91 seconds (average). In windows XP the loop completed in 80 seconds average. In all cases the deviation was +/- 3 seconds from the average even though the string handling wasn't properly optimized (for testing sake). The difference between XP and 7 wasn't anywhere near the difference you reported in the first post.

There are some specific performance monitoring steps that you can take but let's talk about them in another post. PerfMon and ProcessExplorer can be used to monitor performance. Perhaps the windows 7 machine is sharing more of the CPU or isn't threading all of the processes across multiple cores.

I would expect OI to run somewhere between the CPU mark and the PassMark. The CPU would be the best case scenario; OI would have to be only using the CPU. Equally the PassMark is an average of the machine as a whole, including video/Disk/Memory etc. In my testing next to no disk access is taking place, nor is the video being used.

I just installed Windows XP as a dual boot on my laptop and re-tested the speed. It scheduled in 17 minutes, exactly the same amount of time it took under Windows 7, see below.

I conclude therefore that Windows 7 is not slowing anything down.

If you look at the customers machine, below. This machine is 4.7 (5431/1153 CPU Mark) and 2.3 (1575/678 PassMark) times the speed of mine. His OI speed is 3.4 (17/5) times the speed of mine so that makes complete sense. The speed of OI falls between the CPU mark and the PassMark as I expected and I therefore conclude his machine is performing as expected.

I have no idea why the other two machines seem to run OI so much faster than the benchmarks would seem to indicate. As it stands the CJM machine performs 3.4 (17/5) times as fast as mine, yet the machines have very similar hardware (and the benchmarks reflect that they are very similar). The ARG machine performs all most identically to my machine 18 min compared to 17 min, but has hardware that is far less capable (which again is reflected in the benchmarks).

I will get back to you when I figure out what on earth is going on.

Thanks for your help Jared, I appreciate you spending the time to help me get to the bottom of this.

CPU

OS

Passmark

CPU mark

OI speed

Core2 Duo T7700 2.4GHz (CJM)

Win Vista

660

1331

5min

Core i7 870 2.93GHz (Customer)

Win 7 32 bit

1575

5431

5min

Core2 T7200 2GHz (My machine)

Win 7 32 bit

678

1153

17min

Pentium 4 2.80GHz (ARG)

Win XP

281

383

18min

I understand your reasoning and comparison of the PassMark results to OpenInsight. You've been very exact in your testing and reasoning but there are a couple reasons why I would not recommend doing a direct comparison of PassMark and OI.

PassMark may thread better across multiple cores for certain operations which will skew the CPU results.

BASIC+ code isn't directly comparable to compiled code. PassMark may be able to implement optimizations that aren't available in the current OEngine.

Here are a couple suggestions to help figure out what is happening to your application:

1) Compare your tests in safe-mode or boot using only essential windows services. Perhaps an Antivirus or 3rd party utility is stealing CPU cycles or interrupting the benchmark.

2) Use sysinternals Process Explorer to monitor the CPU usage. Check the "context switches" to see how often OInsight.exe is interrupted.

3) Sysinternals Process Monitor can be used to monitor the entire application or system. Maybe you'll see some activity from OInsight or another process that explains a delay.

Hope this helps. Let me know how it goes.

View this thread on the Works forum...