MLton misbehaving?

[Johan Grönqvist-2]

Hi,

I have a program that, in many situations, seems to produce reasonable
results.
I have had sporadic problems, however, and now have a case that is
repeatable.

This version of teh program raises an unhandled exception. In order to
debug,
I compile with -const 'Exn.keepHistory true', but then no unhandled
exception is raised,
and the program keeps runing happily.

Also, commenting out a few unused lines of code (for details, see below),
seems to make the unhandled exception go away, which seems very strange
to me.

As I do not expect this to be normal behaviour for mlton, I wanted to
also ask
if this is known, if there are situations where I should expect it to
happen and/or if
someone has advice on how to debug such problems and avoid them in the
future.

More details are available below

Thanks in advance for any help.

/ johan






More details follow here:

I use mlton verion
MLton 20070826 (built Wed Aug 29 09:44:07 2007 on
silver-star.cs.uchicago.edu)
that I installed from precompiled archive downloaded from mlton.org on a
computer running suse (10.0, I believe)
I also have a computer running debian testing, but have not yet tested
on that one.
I will do so tomorrow.

The program is a few thousand lines of code. Most of those are not used
at all in this example run.
The program essentially minimizes a polynomial in many variables, and
then writes data to files
before exiting. Most of the code is for bookkeeping on the variables,
and for extracting data.
The unhandled exception Bug that is raised by the rogram should only
be raised when I have made a mistake in the code or an incorrect
assumption about the code.
I never hadle such exceptions. I do not use any additional libraries,
and do not use any C-like facilities.
I do not use infinite precision integers.



To test the problem I run the following script:

------------------------------------------
johan@aeki:~/projects/elasticity/bug> cat expose.sh

echo "Running normally."

mlton vff.mlb
./vff


echo "Running with 'Exn.keepHistory true'"
mlton -const 'Exn.keepHistory true' vff.mlb
./vff

echo "Done."
-------------------------------------------------------



Below is the output in the terminal window.
I deleted compiler warnings of the form "* is not exhaustive".
At the end I exit the program by force with ctrl-c, as it would
otherwise run for very long.

-------------------------------
johan@aeki:~/projects/elasticity/bug> ./expose.sh
Running normally.
Starting nanowire calculation for side 3 and height 2.
Minimising pure case.
unhandled exception: Bug

Running with 'Exn.keepHistory true'
Starting nanowire calculation for side 3 and height 2.
Minimising pure case.
Minimization done. Energy: 3.57008568294E~20.
Minimising mixed case.
--------------------------------------------------------------

In the first run, an unhandled exception is raised before any result
has been computed. In the second case, the program goes on without problem,
and I exit by force.

Instead of exiting by force, I thought I could comment-out the rest of
the program (5 lines),
but that seems to change the behaviour.
 

The program used here ends in
------------------------
   
val _ = csnw_minimized (3, 2)

val _ = csnw_minimized (4, 2)
   
val _ = csnw_minimized (12, 3)
val _ = csnw_minimized (12, 10)
   
val _ = csnw_minimized (20, 3)
val _ = csnw_minimized (20, 10)
------------------------


Commenting out the last five of those lines prevents the unhandled
exception from being raised.

I comment out the 5 lines, and run the same script again
The output is then (warnings deleted again)

---------------------------------------
johan@aeki:~/projects/elasticity/bug> ./expose.sh
Running normally.
Starting nanowire calculation for side 3 and height 2.
Minimising pure case.
Minimization done. Energy: 3.01984609884E~20.
Minimising mixed case.
Minimization done. Energy: 5919.45929566.
Done.
Running with 'Exn.keepHistory true'
Starting nanowire calculation for side 3 and height 2.
Minimising pure case.
Minimization done. Energy: 9.3088938774E~21.
Minimising mixed case.
Minimization done. Energy: 5937.34353531.
Done.
Done.
johan@aeki:~/projects/elasticity/bug>

------------------------------------------

Thus, I conclude that commenting out unused code
alters the program behaviour.


Side note: As I do not use any kind of random numbers,
I am also puzzled by the fact that I get different numerical answers
depending on compiler options.


_______________________________________________
MLton-user mailing list
MLton-user@...
http://mlton.org/mailman/listinfo/mlton-user

[Vesa Karvonen-2]

On Wed, Apr 30, 2008 at 3:30 PM, Johan Grönqvist
<johan.gronqvist@...> wrote:
>  I have a program that, in many situations, seems to produce reasonable
>  results.

Seeing the program would most likely help.  Without it, others can only guess.

>  In order to debug, I compile with -const 'Exn.keepHistory true', but then
>  no unhandled exception is raised, and the program keeps runing happily.

The option does change the produced code.  I would not find it
surprising if the option could change floating point results on the
x86 without the -ieee-fp true option.

>  Thus, I conclude that commenting out unused code
>  alters the program behaviour.

Based on your description, it does not seem strictly unused.  MLton is
a whole-program compiler and commenting out code can change the
results of some program transformations. For example, if you have a
function that is initially called from two or more places and you then
delete all but one call to the function, then the function may become
a subject for a number of optimizations.  This can change, for
example, register allocation and might change floating point results.

>  Side note: As I do not use any kind of random numbers,
>  I am also puzzled by the fact that I get different numerical answers
>  depending on compiler options.

One possibility would be floating point imprecision.  Have you tried
compiling with -ieee-fp true?  The numerical values of "Energy"
(3.01984609884E~20 and  9.3088938774E~21) are near the extended
precision limit and might suggest that floating point issues are to
blame.  If that is the case, then you might need to rethink the
algorithm.

-Vesa Karvonen

_______________________________________________
MLton-user mailing list
MLton-user@...
http://mlton.org/mailman/listinfo/mlton-user

[Matthew Fluet-3]

On Wed, 30 Apr 2008, Johan Grönqvist wrote:
> This version of teh program raises an unhandled exception. In order to
> debug, I compile with -const 'Exn.keepHistory true', but then no
> unhandled exception is raised, and the program keeps runing happily.

Keeping the exception history shouldn't change the behavior of the
program, except for the MLton.Exn.history function and (therefore) the
top-level uncaught exception handler.

However, keeping the exception history does change the program as seen by
the compiler, so different compiler optimizations may occur (or fail to
occur).  As Vesa noted, x86 floating-point computation rounding is
affected by the generated code (in particular, when floating-point values
are written to memory due to register spilling or calling conventions).
Thus, it is not surprising that you get different floating-point results
when compiled in the two modes (or even if you make seemingly inoccous
changes to the source program).

Though, it is not impossible that the exception history leads to an
incorrect program transformation by the compiler.  Seeing the program
might help.

> I use mlton verion MLton 20070826 (built Wed Aug 29 09:44:07 2007 on
> silver-star.cs.uchicago.edu) that I installed from precompiled archive
> downloaded from mlton.org on a computer running suse (10.0, I believe)

So, this occurs on an x86-linux platform.

> The program essentially minimizes a polynomial in many variables, and
> then writes data to files before exiting. Most of the code is for
> bookkeeping on the variables, and for extracting data. The unhandled
> exception Bug that is raised by the rogram should only be raised when I
> have made a mistake in the code or an incorrect assumption about the
> code. I never hadle such exceptions. I do not use any additional
> libraries, and do not use any C-like facilities. I do not use infinite
> precision integers.

Is the Bug exception raised in response to floating-point calculations?

> Commenting out the last five of those lines prevents the unhandled
> exception from being raised.
>
> I comment out the 5 lines, and run the same script again
> The output is then (warnings deleted again)

Changing the input program in this manner, for example, might result in
csnw_minimized being inlined, if there is now only one call to the
function in the program.  That, in turn, might change how floating-point
values are spilled, flattened, etc.

As Vesa noted, the '-ieee-fp true' compiler option will instruct the
compiler to write every floating-point value to memory immediately after
it is computed.  That maximizes the rounding (and approximates true 64-bit
IEEE calculations) by always discarding the extra precision of the x86
80-bit floating-point registers.

Alternatively, if you have access to an amd64 machine, the MLton
native-codegen will use the SSE instructions to perform all (primitive)
floating-point calculations at exactly 64-bit IEEE.  That tends to be
much more robust to program transformations.
_______________________________________________
MLton-user mailing list
MLton-user@...
http://mlton.org/mailman/listinfo/mlton-user

[Johan Grönqvist]

ons 2008-04-30 klockan 13:06 -0500 skrev Matthew Fluet:
> On Wed, 30 Apr 2008, Johan Grönqvist wrote:

> As Vesa noted, x86 floating-point computation rounding is
> affected by the generated code [...].
> Thus, it is not surprising that you get different floating-point results[..].
>
> Is the Bug exception raised in response to floating-point calculations?
>

Yes!

I have now found that with '-ieee-fp true' the result of some floating point calculations is nan, and
at that point I do indeed raise the Bug exception because of the floating point results. I assume then
that a similar situation may be the reason for my previous problems as well.

Thanks to both of you for your helpful answers.

I will now go on trying to find out why the nan appears.

/ johan



_______________________________________________
MLton-user mailing list
MLton-user@...
http://mlton.org/mailman/listinfo/mlton-user