When calculating Variance and Standard Deviation statistics my program calculated an incorect subtraction sub value.

a=91458486830000

b=4.6116861842739E18

x=a - b

Result x=-4.61159455994056E18

To isolate the problem I reduced the number of records from 4700 to 1000 and re ran the program. The new values are:

a=4765650000000

b=0

x=a - b

result=4765000000 this is the correct result.

I ran the same data in AREV for 4000 and 1000 records and received the correct result for both tests.

Does OI have a problem with very large numbers?

Have just tried the same figures in a stand-alone test and got the correct result (-4.61159472578707E18) !!

There has been a problem in the past with OI not recognising differences in the 16th digit in long numbers, but that isnt the case here.

Simon

What version of OI? Short answer is a fix has been applied to more recent OIs.

The Sprezzatura Group

World leaders in all things RevSoft

My test was in 4.02 …..

Mark,

Running that on OI 7.2.1 gives

-4.61159472578707E18

while running it on MS calc gives the same:

-4611594725787070000

So I think OI 7.2.1 looks OK…

The Sprezzatura Group

World leaders in all things RevSoft

Thanks for the replies.

I am using OI 7.2.1 and I have applied the latest patch.

If OI and MS are calculating the same result I must be missing something quite fundemental.

As I am subtracting a smaller number from a very large number I expect the result to be poitive?

Mark

Mark,

Your original posting shows numbers in the range A=nE14 and B=nE18. Subtracting B from A should have a negative result, provided your scaling factor is adequately caffeinated.

Gerald

Thanks Gerald.

Now that I have got my head around scientific notation I have solved the problem.

If one squares a large number, OI changes the original value of the variable and this is the cause of my problem.

Example:

xsum= 8617900

Temp=xsum * xsum After the calculation the value of xsum was changed to

"-214836483648” and Temp=4.61168601842739E18 Both values are incorrect

I changed the logic to:

xsum= 8617900 xsumx=8617900

Temp=xsum * xsumx Temp=74268200410000

The calculated value for Temp is correct and the value of xsum was not changed. The changing of the xsum value was the cause of my original problem.

I doubled checked the answers by taking the square root of Temp after the calculation. In the first case the answer was calculated to be 2147483648 and in the second case 8617900 (original value). The second case is correct.

Is this an OI problem or is the statement Temp=(xsum * xsum) invalid?

Why not use the PWR function?

temp=PWR(xsum,2)

To my mind that skirts the issue. If x=a * a produces different results from x=pwr(a,2), there is a bug in the system. I don't know about OI's implementation, but in some languages, using the simple multiplication is more efficient, hence it's commonly used.

There is always the exponentiation operators also and ^ temp=xsum 2

temp=xsum ^ 2

Mark,

There must be something else at play here - I cut and pasted your example and got the correct result.

When you run your program is 'xsum' the result of previous calculations in the code, or are you trying it with the literal value as per your example?

The only way I could see xsum being changed is in the process that convert strings to numbers, floats to integers and so on - this has caused precision issues in the past. The basic multiplication opcode doesn't modify its arguments at all.

The Sprezzatura Group

World leaders in all things RevSoft

The Sprezzatura Group

World leaders in all things RevSoft

Thanks for your help.

The code is as follows:

FOR QNUM =1 TO NUM.QUS

 QIND    =QUS.INDEX(QNUM)

 QUES.ID =QUS.LIST

 NO_ANS  =MAX.ANS

 STEMP1  =RTYPE

 AVERAGE =0.00

 STD.DEV =0.00

 VARIANCE=0.00

 MEDIAN  =0.00

 MODE    =0.00

 MIN     =0.00

 MAX     =0.00

 RANGE   =0.00

 SUM     =0.00

 XOPAT=MD2"

• XOPAT=x_index

 STDEV =00

 XSUM=NUM.SUMS(QNUM)

 XSUMx=NUM.SUMS(QNUM)

 XY=TOT.RESP(QNUM)

 BEGIN CASE

 CASE INDEX("LMRKNC",STEMP1,1)

    IF xy  0  THEN

       AVERAGE=xsum/xy

       IF xy ] 1 THEN

          XXST1=XY * NUM.SUM.SQS(QNUM)

          XXST2=xsumx * xsum

          VARIANCE=(XXST1 - XXST2)

          temp=SQRT(VARIANCE)

          STDEV=temp/XY

          VARIANCE=stdev * stdev/100

       end

       gosub numeric.median

       gosub numeric.mode

    end

    NTEMP1=TOT.NUM.QUS + QIND

    RMAT(NTEMP1) =TOT.RESP(QNUM)

    RMAT(NTEMP1) =xSUM

    RMAT(NTEMP1)  =(RMAT(NTEMP1) - RMAT(NTEMP1))  ;* RANGE

    square             =ICONV(sqrt(tot.resp(qnum)),XOPAT)

    RMAT(NTEMP1) =AVERAGE

    RMAT(NTEMP1) =STDEV

    RMAT(NTEMP1) =VARIANCE

    RMAT(NTEMP1) =ICONV(MEDIAN,XOPAT)

    RMAT(NTEMP1) =ICONV(MODE,XOPAT)

 END CASE

NEXT QNUM

RMAT(2 * TOT.NUM.QUS + 1)=NUM.FORMS

num.qus=1 and stemp1=L" and xy is ] 1. The routine is only executed once.

The "problem" of xsum being changed only occurs when the the original value of xsum is very large.

This code was copied from AREV where it works fine. The XSUMX variable was added and the program works fine and calculates the correct result.

Mark

Unless the compiler has special integral power optimizations, A*A is way more efficient that PWR(A,2). Accuracy could be different too, as PWR may use exp(log(A)*X) for its internal computation.

If accuracy and more important verifiability of the results is such a concern then it's better off to use a statistical package like SPSS to do the number crunching.

If there is any doubt as to the results the raw data can be given out and the results verified independently. The Biostatistics Department of one research hospital is using ARev for data collection and basic reports and the number crunching with SPSS and other packages. It's a lot simpler than having to have your ARev/OI software audited and having to prove the validity and accuracy of your algorithms.

View this thread on the forum...