Sign up on the Revelation Software website to have access to the most current content, and to be able to ask questions and get answers from the Revelation community

At 08 JUN 1998 06:11:03PM Laura Randall wrote:

I'm still getting strange division results with a simple calc.

RESULT=TOTAL_HRS / TOTAL_BONUS

In the debugger:

TOTAL_HRS=12650

TOTAL_BONUS=1000

RESULT=7.90513833992075E-2

I don't have a clue as to why this is doing this. Not all records calculated incorrectly.

Anyone had this happen before?

Help, I'm desperate!


At 08 JUN 1998 07:40PM Victor Engel wrote:

I'm still getting strange division results with a simple calc. RESULT=TOTAL_HRS / TOTAL_BONUS In the debugger: TOTAL_HRS=12650 TOTAL_BONUS=1000 RESULT=7.90513833992075E-2

I assume you got the values reversed, in which case, this is the correct answer expressed in scientific notation. Arev does this with small numbers that don't quickly go to all zeroes. If you don't want to use scientific notation, you can use one of the MD conversions. Which you use depends on how many significant digits you want:

RESULT=10 * TOTAL_BONUS / TOTAL_HRS "MD1" produces .1

RESULT=100 * TOTAL_BONUS / TOTAL_HRS "MD2" produces .08

RESULT=1000 * TOTAL_BONUS / TOTAL_HRS "MD3" produces .079

RESULT=10000 * TOTAL_BONUS / TOTAL_HRS "MD4" produces .0791

etc.

It's important to note that the result you were getting is correct. It is simply in scientific notation. To interpret this notation, consider the letter E as an abbreviation for "times 10 to the power of", so in your example, 7.90513833992075E-2 really means 7.90513833992075 times 10^-2, which is the correct answer.


At 09 JUN 1998 02:00PM Laura Randall wrote:

Thanks, I'll give it a try. I had tried a scientific notation conversion but still was getting an incorrect answer. Will try again with your suggestion. thanks.

View this thread on the forum...

  • third_party_content/community/commentary/forums_nonworks/9d4f81584d53ff568525661d0079dc8b.txt
  • Last modified: 2023/12/28 07:40
  • by 127.0.0.1