Strange Results (AREV Specific)
At 08 JUN 1998 06:11:03PM Laura Randall wrote:
I'm still getting strange division results with a simple calc.
RESULT=TOTAL_HRS / TOTAL_BONUS
In the debugger:
TOTAL_HRS=12650
TOTAL_BONUS=1000
RESULT=7.90513833992075E-2
I don't have a clue as to why this is doing this. Not all records calculated incorrectly.
Anyone had this happen before?
Help, I'm desperate!
At 08 JUN 1998 07:40PM Victor Engel wrote:
I'm still getting strange division results with a simple calc. RESULT=TOTAL_HRS / TOTAL_BONUS In the debugger: TOTAL_HRS=12650 TOTAL_BONUS=1000 RESULT=7.90513833992075E-2
I assume you got the values reversed, in which case, this is the correct answer expressed in scientific notation. Arev does this with small numbers that don't quickly go to all zeroes. If you don't want to use scientific notation, you can use one of the MD conversions. Which you use depends on how many significant digits you want:
RESULT=10 * TOTAL_BONUS / TOTAL_HRS "MD1" produces .1
RESULT=100 * TOTAL_BONUS / TOTAL_HRS "MD2" produces .08
RESULT=1000 * TOTAL_BONUS / TOTAL_HRS "MD3" produces .079
RESULT=10000 * TOTAL_BONUS / TOTAL_HRS "MD4" produces .0791
etc.
It's important to note that the result you were getting is correct. It is simply in scientific notation. To interpret this notation, consider the letter E as an abbreviation for "times 10 to the power of", so in your example, 7.90513833992075E-2 really means 7.90513833992075 times 10^-2, which is the correct answer.
At 09 JUN 1998 02:00PM Laura Randall wrote:
Thanks, I'll give it a try. I had tried a scientific notation conversion but still was getting an incorrect answer. Will try again with your suggestion. thanks.