Real32.realRound is broken

[John Reppy-2]

The Real32.realRound function in mlton does not work.  Here is a  
simple test
program:

        fun snap f = 0.001 * Real32.realRound(f * 1000.0);
        fun g x = print(concat["x = ", Real32.toString x, ", snap(x) = ",  
Real32.toString(snap x), "\n"]);
        g 45.0000038147;

The Real64 version of this program produces the correct output

        x = 45.0000038147, snap(x) = 45

but the Real32 version produces

        x = 45.0000038147, snap(x) = 45.0000038147

This is using MLton 20070826 on Mac OS X 10.5.2/Intel.

        - John

_______________________________________________
MLton-user mailing list
MLton-user@...
http://mlton.org/mailman/listinfo/mlton-user

[John Reppy-2]

You're right.  The difference between Real64 and Real32 is that the  
fractional
part is not visible with the default number of significant digits used  
by toString.

        - John

On Apr 24, 2008, at 9:48 AM, Matthew Fluet wrote:

> On Wed, 23 Apr 2008, John Reppy wrote:
>> The Real32.realRound function in mlton does not work.  Here is a  
>> simple test
>> program:
>>
>> fun snap f = 0.001 * Real32.realRound(f * 1000.0);
>> fun g x = print(concat["x = ", Real32.toString x, ", snap(x) = ",
>> Real32.toString(snap x), "\n"]);
>> g 45.0000038147;
>>
>> The Real64 version of this program produces the correct output
>>
>> x = 45.0000038147, snap(x) = 45
>>
>> but the Real32 version produces
>>
>> x = 45.0000038147, snap(x) = 45.0000038147
>>
>> This is using MLton 20070826 on Mac OS X 10.5.2/Intel.
>
> Isn't this simply a consequence of IEEE floating point values being  
> represented with radix 2, so the base 10 value 0.001 cannot be  
> represented exactly?
>
> The following program:
>
> structure Real = Real32
> fun r2s x = Real.toString x
> val s = 0.001
> val z = 45000.0
> val sz = s * z
> val () = print(concat["s = ", r2s s, ", z = ", r2s z, ", s * z = ",  
> r2s sz, "\n"])
>
> produces
>
> s = 0.0010000000475, z = 45000, s * z = 45.0000038147
>
> so, if there is a bug, it isn't with Real32.realRound.

_______________________________________________
MLton-user mailing list
MLton-user@...
http://mlton.org/mailman/listinfo/mlton-user

[Matthew Fluet-3]

On Wed, 23 Apr 2008, John Reppy wrote:

> The Real32.realRound function in mlton does not work.  Here is a simple test
> program:
>
> fun snap f = 0.001 * Real32.realRound(f * 1000.0);
> fun g x = print(concat["x = ", Real32.toString x, ", snap(x) = ",
> Real32.toString(snap x), "\n"]);
> g 45.0000038147;
>
> The Real64 version of this program produces the correct output
>
> x = 45.0000038147, snap(x) = 45
>
> but the Real32 version produces
>
> x = 45.0000038147, snap(x) = 45.0000038147
>
> This is using MLton 20070826 on Mac OS X 10.5.2/Intel.

Isn't this simply a consequence of IEEE floating point values being
represented with radix 2, so the base 10 value 0.001 cannot be represented
exactly?

The following program:

  structure Real = Real32
  fun r2s x = Real.toString x
  val s = 0.001
  val z = 45000.0
  val sz = s * z
  val () = print(concat["s = ", r2s s, ", z = ", r2s z, ", s * z = ", r2s sz, "\n"])

produces

  s = 0.0010000000475, z = 45000, s * z = 45.0000038147

so, if there is a bug, it isn't with Real32.realRound.

_______________________________________________
MLton-user mailing list
MLton-user@...
http://mlton.org/mailman/listinfo/mlton-user