Scientific Notation (AREV Specific)
At 21 FEB 2000 05:53:48PM Bruce wrote:
Hello,
What is the cleanest way to convert from e-notation to a standard format in arev. For example, If I divide 17.00/38016 I get a number in e-notation. Is there another way i should be making this calculation in Arev?
Thanks Bruce.
At 21 FEB 2000 05:57PM Matt Sorrell wrote:
Bruce,
If you knew ahead of time what level of precision that you needed, then you can use the MDx formatting / CONV codes, where x is the number of decimal places.
At 21 FEB 2000 06:02PM Matt Sorrell wrote:
Bruce,
My fingers got ahead of me, and I didn't put in an example.
For instance, if you calculated this result as:
OCONV( ICONV( 17.00 / 38016 , "MD6"), "MD6")
then ARev returns the result of .000447.
Basically, you have to force ARev to ignore it's level of precision by defining your own level of precision. I can get this to work reliably up to MD9.
FWIW,
Matt Sorrell
msorrel@greyhound.com
At 21 FEB 2000 07:18PM [url=http://www.sprezzatura.com" onMouseOver=window.status=Click here to visit our web site?';return(true)]The Sprezzatura Group[/url] wrote:
ICONV/OCONV can accept MD values from 0 through F. However, there might be a limitation to how long the result can be before an error occurs. It appears that if the result exceeds 15 chars, a runtime error is encountered.
World Leaders in all things RevSoft
At 22 FEB 2000 10:08AM Matt Sorrell wrote:
Andrew / Aaron / Carl,
Never know which one of you I'm talking to any more. That explains why MD10 blew up (well, just returned 0.0) on me. I didn't even think to try hex. Oh well.
Thx for the info,
Matt Sorrell
msorrel@greyhound.com
At 22 FEB 2000 11:37AM [url=http://www.sprezzatura.com" onMouseOver=window.status=Click here to visit our web site?';return(true)]The Sprezzatura Group[/url] wrote:
Spooky isn't it . It is documented in OI - unsure of AREV.
World Leaders in all things RevSoft