Join The Works program to have access to the most current content, and to be able to ask questions and get answers from Revelation staff and the Revelation community

At 17 MAR 2006 10:12:24PM Chris Meyer wrote:

As per client request, need to enter a decimal number with a variable number of decimal points. e.g enter a qty of 2 then 2 is displayed, enter 3.456 then this is to be displayed. Client does not wish to display in MD2 format. Using AREV the validation patterns were 0N or 0N'.'0N. Using this validation in OI the value after the decimal point gets trucated (MD0) to a single digit therefore 3.456 gets changed to 3.5 using the 0N'.'0N validation.

What is the correct validation pattern for a floating point decimal number.

Any suggestions is appreciated.

TIA

Chris


At 17 MAR 2006 10:50PM Warren Auyong wrote:

What happens if you use a range e.g. (0,100000)?


At 18 MAR 2006 06:02AM Chris Meyer wrote:

Sorry for this problem. I had some post processing on the control on the form. I will be a little more careful next time before shooting of the hip.

Chris


At 19 MAR 2006 12:23PM Gerald Lovel wrote:

Chris,

Actually, this was a very good question. I use a conversion routine, CNV_FD,.mn, to allow numbers to be printed with variable precision. For example, CNV_FD,.62 would print 123.45 as 123.45, but the number 123.4567 would be printed as 123.456700. Internally the numbers stored would be 12345 and 12345.67 respectively. You could think of this as an (MD2) conversion with extended decimal precision, and in fact that is exactly what it is.

I use this conversion in my billing system, where a user might quote a rate of $1.51/ton. This is fine until he realizes that he is using weights in pounds, so the rate has to be converted to $.000755/lb. (Well, I actually have conversion factors in the billing to avoid this, but you get the idea.)

View this thread on the Works forum...

  • third_party_content/community/commentary/forums_works/6db9d693ff41d0e785257135000c1f26.txt
  • Last modified: 2023/12/30 11:57
  • by 127.0.0.1