Nabble - MLton

Re: How to access large amounts of data through FFI

2008-10-11T05:47:49Z

On Sat, Oct 11, 2008 at 7:49 AM, Ville Tuulos <tuulos@...> wrote:
[...]
> Actually when I got the Florian's reply about MLton.Pointer, I started
> to consider writing something like RawVector which would support map,
> foldl etc. over mmaped memory. Going through your code will be even
> better exercise for me.

I extended my example somewhat. The most significant change in the
attached version is that it uses MLton's finalizers
(http://mlton.org/MLtonFinalizable) to munmap the mapped region
automatically when the RawVector is garbage collected. This can be
convenient, but assuming that the mapped file is large, it is usually
best to free (munmap) it explicitly (which is also supported), because
it might take a long time before the finalizers are run. The use of
finalizers also adds some "access overhead" to the RawVector
implementation and also has some other effects. For something like
the sub operation the access overhead may be significant. Aggregate
operations (foldl, find, app, ...) only incur the access overhead
once.

Implementing map, i.e. a function of type ('a -> 'b) -> 'a t -> 'b t,
for something like RawVector is not entirely straightforward. The
main problem is that if one is using it to map very large files to
memory, like suggested in your initial message, then creating a new
RawVector by mapping a function over it is likely to be inefficient.
Rather than use map, it is probably better to use "lower level"
functions like foldl directly or use lazy sequence of some form. One
possibility would be to use "iterator combinators" or something
similar that allows you to lazily map over the raw vector. Iterator
combinators are explained on the http://mlton.org/ForLoops page (note
that RawVector.for is essentially an iterator function and can be used
with iterator combinators) and there is a module implementing iterator
combinators in the extended basis library
(http://mlton.org/cgi-bin/viewsvn.cgi/mltonlib/trunk/com/ssh/extended-basis/unstable/public/control/iter.sig?view=auto).

-Vesa Karvonen

[mmap.c]

#include <fcntl.h> #include <sys/mman.h> #include <sys/stat.h> #include <sys/types.h> #include <unistd.h> static inline size_t align(size_t granularity, size_t value) { return value - value % granularity; } void *map_file(const char *file, size_t granularity, size_t *size) { int fd = open(file, O_RDONLY, S_IRUSR); if (-1 == fd) goto open; struct stat stat; if (-1 == fstat(fd, &stat)) goto stat; if (stat.st_size > (size_t)-1) goto size; size_t aligned_size = align(granularity, stat.st_size); void *ptr = mmap(0, aligned_size, PROT_READ, MAP_SHARED, fd, 0); if ((void*)(-1) == ptr) goto mmap; if (-1 == close(fd)) goto fclose; *size = aligned_size; return ptr; fclose: munmap(ptr, aligned_size); mmap: size: stat: close(fd); open: return 0; } void unmap_file(void *addr, size_t bytes) { if (addr) munmap(addr, bytes); }

_______________________________________________
MLton-user mailing list
MLton-user@...
http://mlton.org/mailman/listinfo/mlton-user

mmap.mlb (312 bytes) Download Attachment
mmap.sml (6K) Download Attachment
Build.sh (348 bytes) Download Attachment

From forum: MLton - User

Re: How to access large amounts of data through FFI

2008-10-10T21:49:05Z

On Fri, Oct 10, 2008 at 4:06 PM, Vesa Karvonen <vesa.a.j.k@...> wrote:
> On Fri, Oct 10, 2008 at 9:24 PM, Ville Tuulos <tuulos@...> wrote:
>> How to construct and pass a valid int vector from C to SML through
>> Mlton's FFI, preferably without copying the data first?
>
> Unfortunately, you can't access a mmapped region of memory as an
> ordinary SML vector, because ordinary SML vectors live in MLton's
> garbage collected heap (and have a special header for MLton's runtime
> and may be moved by MLton's GC).

ok. Good to know.

>> The FFI interface seems pretty straighforward but I couldn't find any
>> documentation how arrays and vectors should be initialized based on a
>> C pointer and size of the data. Or, if the int vector is not a good
>> idea, what would be a better way to get access to the memory space?
>
> Like suggested by Florian Weimer, likely the most efficient approach
> is to access the data using MLton.Pointer.

ok. Thanks for the suggestion.

> The attached files contain an example of a way to do it. The example
> contains the beginnings of a RawVector module that allows one to mmap
> a file and access it similarly to an ordinary vector. As an example,
> it opens a file ("data"), prints the length of the resulting RawVector
> and then prints all the elements of the RawVector.
>
> For (my) convenience, the example makes (rather minimal) use of an
> extended basis library from the "mltonlib" repository. You can check
> out the mltonlib repository with the following command:
>
> svn co svn://mlton.org/mltonlib/trunk mltonlib
>
> You'll then need to modify the Build.sh script to point to the
> location of mltonlib in order to build the example with the script.

Perfect! This is exactly what I need.

Actually when I got the Florian's reply about MLton.Pointer, I started
to consider writing something like RawVector which would support map,
foldl etc. over mmaped memory. Going through your code will be even
better exercise for me.

Thanks!

Ville

_______________________________________________
MLton-user mailing list
MLton-user@...
http://mlton.org/mailman/listinfo/mlton-user

From forum: MLton - User

Re: How to access large amounts of data through FFI

2008-10-10T16:06:40Z

On Fri, Oct 10, 2008 at 9:24 PM, Ville Tuulos <tuulos@...> wrote:
> I would like to access the data in Mlton without copying it first, due
> to obvious performance reasons. The data itself is a flat array of
> native 32-bit ints. Since it's an immutable array of native ints, I
> was hoping that I could access it as an int vector (or something
> similar) in Mlton.
>
> How to construct and pass a valid int vector from C to SML through
> Mlton's FFI, preferably without copying the data first?

Unfortunately, you can't access a mmapped region of memory as an
ordinary SML vector, because ordinary SML vectors live in MLton's
garbage collected heap (and have a special header for MLton's runtime
and may be moved by MLton's GC).

> The FFI interface seems pretty straighforward but I couldn't find any
> documentation how arrays and vectors should be initialized based on a
> C pointer and size of the data. Or, if the int vector is not a good
> idea, what would be a better way to get access to the memory space?

Like suggested by Florian Weimer, likely the most efficient approach
is to access the data using MLton.Pointer.

The attached files contain an example of a way to do it. The example
contains the beginnings of a RawVector module that allows one to mmap
a file and access it similarly to an ordinary vector. As an example,
it opens a file ("data"), prints the length of the resulting RawVector
and then prints all the elements of the RawVector.

For (my) convenience, the example makes (rather minimal) use of an
extended basis library from the "mltonlib" repository. You can check
out the mltonlib repository with the following command:

svn co svn://mlton.org/mltonlib/trunk mltonlib

You'll then need to modify the Build.sh script to point to the
location of mltonlib in order to build the example with the script.

-Vesa Karvonen

[mmap.c]

#include <fcntl.h> #include <sys/mman.h> #include <sys/stat.h> #include <sys/types.h> #include <unistd.h> void *map_file(const char *file, size_t *size) { int fd = open(file, O_RDONLY, S_IRUSR); if (-1 == fd) goto open; struct stat stat; if (-1 == fstat(fd, &stat)) goto stat; void *ptr = mmap(0, stat.st_size, PROT_READ, MAP_SHARED, fd, 0); if ((void*)(-1) == ptr) goto mmap; if (-1 == close(fd)) goto fclose; *size = stat.st_size; return ptr; fclose: munmap(ptr, stat.st_size); mmap: stat: close(fd); open: return 0; }

_______________________________________________
MLton-user mailing list
MLton-user@...
http://mlton.org/mailman/listinfo/mlton-user

Build.sh (348 bytes) Download Attachment
data (12 bytes) Download Attachment
mmap.mlb (312 bytes) Download Attachment
mmap.sml (3K) Download Attachment

From forum: MLton - User

Re: How to access large amounts of data through FFI

2008-10-10T11:36:24Z

* Ville Tuulos:

> Or, if the int vector is not a good idea, what would be a better way
> to get access to the memory space?

You probably should use MLton.Pointer: <http://mlton.org/MLtonPointer>

Perhaps with a wrapper to add bounds checking.

_______________________________________________
MLton-user mailing list
MLton-user@...
http://mlton.org/mailman/listinfo/mlton-user

From forum: MLton - User

How to access large amounts of data through FFI

2008-10-10T11:24:09Z

Hi

I'm a total SML / Mlton newbie, so please excuse my ignorance.

I'm trying to find out if I could use Mlton to perform some data
processing tasks that I used to do in C, or slowly in Python. I have
large amounts (hundreds of megs to a gigabyte) of data memory mapped
(using mmap) to my process' address space in C. The mapping is
read-only.

I would like to access the data in Mlton without copying it first, due
to obvious performance reasons. The data itself is a flat array of
native 32-bit ints. Since it's an immutable array of native ints, I
was hoping that I could access it as an int vector (or something
similar) in Mlton.

How to construct and pass a valid int vector from C to SML through
Mlton's FFI, preferably without copying the data first? The FFI
interface seems pretty straighforward but I couldn't find any
documentation how arrays and vectors should be initialized based on a
C pointer and size of the data. Or, if the int vector is not a good
idea, what would be a better way to get access to the memory space?

Thank you,

Ville Tuulos

_______________________________________________
MLton-user mailing list
MLton-user@...
http://mlton.org/mailman/listinfo/mlton-user

From forum: MLton - User

Re: Code being dropped that shouldn't

2008-10-09T20:00:13Z

On Wed, 8 Oct 2008, Wesley W. Terpstra wrote:
> I've been trying to track down the last bug in the library regression
> test, but I think now it's not a bug in the test at all. Try an i386
> compiler (windows and linux both show the same behaviour). Run:
> ./library-test -debug true -keep g -codegen c -profile time
...
> I know there's some sort of basis-specific code
> elimination. Is it possible that librarySuffix is getting hit by this?

No.

Turns out that it was a (long-standing) bug with 'MLton_callFromC' in
c-main.h. In the library test, there is a point when the call-stack for
libm5 looks like:
m5_open()
libm5.sml (* top-level SML-code, via m5_open trampoline *)
libm5confirmC()
libm5smlFn{Private,Public}() (* _export-ed from libm5.sml *)
MLton_callFromC()
libm5.sml (* libm5smlFn{Private,Public}, via MLton_callFromC trampoline *)

The _export-ed 'libm5smlFn{Private,Public}' functions terminate the
'MLton_callFromC' trampoline by setting the global 'returnToC' to 'TRUE'.
This returns to the top-level SML-code from libm5.sml, executed via the
'm5_open' trampoline. However, the 'returnToC' global is set to 'TRUE',
so the trampoline terminates at the first inter-chunk transfer, without
completing the top-level SML-code from libm5.sml; in particular, it
doesn't execute to the 'suffixArchiveOrLibrary' and doesn't suspend the ML
stack at the point just before the 'clean atExit'. Rather, it suspends
the ML stack much earlier in the execution, and when the top-level C-code
(as executed by check.sml) executes 'm5_close', it starts a new trampoline
(setting 'returnToC' to 'FALSE') and resumes the ML code somewhere shortly
after the call to 'libm5confirmC()', at which point it executes to the
first 'Primitive.MLton.Thread.returnToC' in 'suffixArchiveOrLibrary',
returns to 'm5_close'. That explains why the 'clean atExit' never
executed.

The reasons that this erroneous behavior doesn't occur with all of the
C-codegen library tests is that it depends upon how the program is
chunkify-ed. This is simply the coarse grouping of RSSA IL functions into
larger 'chunks' to create larger/fewer .c files. However, it is a
sized-based grouping, so small changes to the RSSA IL (such as the extra
code inserted for time profiling) can change the grouping and cause an
inter-chunk transfer to manifest the bug. One can cause the bug to
manifest in the other C-codegen tests by compiling with '-chunkify func',
which uses the finest chunking (increasing the number of inter-chunk
transfers).

The fix is trivial; set 'returnToC' to 'FALSE' after completing a
'MLton_callFromC' trampoline:

Modified: mlton/trunk/include/c-main.h
===================================================================
--- mlton/trunk/include/c-main.h 2008-10-09 21:35:20 UTC (rev 6918)
+++ mlton/trunk/include/c-main.h 2008-10-10 02:22:13 UTC (rev 6919)
@@ -39,6 +39,7 @@
do { \
cont=(*(struct cont(*)(void))cont.nextChunk)(); \
} while (not returnToC); \
+ returnToC = FALSE; \
s->atomicState += 1; \
GC_switchToThread (s, GC_getSavedThread (s), 0); \
s->atomicState -= 1; \

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

From forum: MLton - Dev

Code being dropped that shouldn't

2008-10-08T06:26:43Z

I've been trying to track down the last bug in the library regression
test, but I think now it's not a bug in the test at all. Try an i386
compiler (windows and linux both show the same behaviour). Run:
./library-test -debug true -keep g -codegen c -profile time

Expected (wrong) output:
check starting up
libm5 starting up
libm4 starting up
libm3 starting up
libm2 starting up
libm1 starting up
m1 pointer test complete.
m2 pointer test complete.
m3 pointer test complete.
m4 pointer test complete.
m5 pointer test complete.
check pointer test complete.
check exits
profiling is on
profiling is on
profiling is on
profiling is on
profiling is on

The problem is that the atExit code for libm1-5 doesn't get run.
However, just adding a print statement to mlton/exit.sml:60:
(* Return to 'lib_open'. *)
val () = Primitive.MLton.Thread.returnToC ()
(* Enter from 'lib_close'. *)
val () = print "lib_close invoked\n"
val _ = exiting := true
val () = let open Cleaner in clean atExit end
(* Return to 'lib_close'. *)
val () = Primitive.MLton.Thread.returnToC ()
... changes the output to
check starting up
libm5 starting up
libm4 starting up
libm3 starting up
libm2 starting up
libm1 starting up
m1 pointer test complete.
m2 pointer test complete.
m3 pointer test complete.
m4 pointer test complete.
m5 pointer test complete.
check pointer test complete.
lib_close invoked
lib_close invoked
lib_close invoked
lib_close invoked
lib_close invoked
libm1 exits
libm2 exits
libm3 exits
libm4 exits
libm5 exits
check exits

So it seems that the library suffix doesn't get run until the print
statement is added. Tracing the execution of m5_close with gdb
confirms this. I know there's some sort of basis-specific code
elimination. Is it possible that librarySuffix is getting hit by this?

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

From forum: MLton - Dev

DAMP 2009: Final Call For Papers

2008-10-06T18:11:42Z

[Note the revised full paper submission date.]

CALL FOR PAPERS

DAMP 2009: Workshop on
Declarative Aspects of Multicore Programming
Savannah, GA, USA
(co-located with POPL 2009)
January 20, 2009

ABSTRACT SUBMISSION DEADLINE: OCTOBER 10

http://www.cse.unsw.edu.au/~pls/damp09/

Important Dates:

Deadline for abstract submission: October 10, 2008
Deadline for full paper submission: October 14, 2008
Notification of acceptance: November 10, 2008
Final papers due: November 17, 2008
DAMP 2009: January 20, 2009

DAMP 2009 is the fourth in a series of one-day workshops seeking to
explore ideas in programming language design that will greatly
simplify programming for multicore architectures, and more generally
for tightly coupled parallel architectures. DAMP 2009 is co-located
with the ACM SIGPLAN - SIGACT Symposium on Principles of Programming
Languages (POPL 2009).

Scope

The emphasis will be on functional and (constraint-)logic programming,
but any programming language ideas that aim to raise the level of
abstraction are welcome. DAMP seeks to gather together researchers in
declarative approaches to parallel programming and to foster cross
fertilization across different approaches.

Specific topics include, but are not limited to:

* suitability of functional and (constraint-)logic programming
languages to multicore applications;
* run-time issues such as garbage collection or thread scheduling;
* architectural features that may enhance the parallel performance of
declarative languages;
* type systems and analysis for accurately knowing or limiting
dependencies, aliasing, effects, and nonpure features;
* ways of specifying or hinting at parallelism;
* ways of specifying or hinting at data placement which abstract away
from any details of the machine;
* compiler techniques, automatic parallelization, automatic
granularity control;
* experiences of and challenges arising from making declarative
programming practical;
* technology for debugging parallel programs;
* design and implementation of domain-specific declarative languages
for multi-core;

Accepted papers will be published in the ACM Digital Library and in a
physical proceedings. Papers must adhere to the SIGPLAN Republication
Policy:

http://www.sigplan.org/republicationpolicy.htm

Concurrent submissions to other conferences, workshops, journals, or
similar forums of publication are not allowed. However, DAMP is
intended to be a venue for discussion and exploration of
works-in-progress, and so publication of a paper at DAMP 2009 is not
intended to preclude later publication as appropriate.

Submission Details:

Authors should submit an abstract of at most 300 words and a full
paper of no more than 10 pages (including bibliography and appendices)
in the ACM SIGPLAN conference format. Further details are on the
workshop web page:

http://www.cse.unsw.edu.au/~pls/damp09/submission.html

Program Chair:
Manuel Chakravarty University of New South Wales

Program Committee:
Guy Blelloch Carnegie Mellon University
Manuel Carro Universidad Politécnica de Madrid
Koen Claessen Chalmers University of Technology
Matthew Fluet Toyota Technological Institute at Chicago
Vivek Sarkar Rice University
Sven-Bodo Scholz University of Hertfordshire
Satnam Singh Microsoft Research, Cambridge
Martin Sulzmann IT University of Copenhagen
Don Syme Microsoft Research, Cambridge
Phil Trinder Heriot-Watt University

General Chair:
Leaf Petersen Intel Corporation

Past DAMPs:

http://www.cliplab.org/Conferences/DAMP08
http://glew.org/damp2006
http://www.cs.cmu.edu/~damp

_______________________________________________
MLton-user mailing list
MLton-user@...
http://mlton.org/mailman/listinfo/mlton-user

From forum: MLton - User

Re: MLton import headers

2008-10-04T16:05:15Z

On Sun, Oct 5, 2008 at 12:02 AM, Matthew Fluet <fluet@...> wrote:

> That seems like a reasonable rationale. If I understand things correctly,
> though, one advantage of the
> #include "xyz.h"
> #include "xyz-private.h"
> approach is that is textually separates the public (in the sense of the
> documented portions of the library API available for use in other projects)
> functions from the private (in the sense of the undocumented portions of the
> library API) functions. That is, clients of "xyz.h" can't even see the
> prototypes of the private functions.

Correct. On the flip-side, having only one file is convenient.

While I agree that header files can serve a documentation purpose, I
don't think that's true for us. An automatically generated header is
full of extra-trash and lacks comments. The header doesn't declare the
symbols to C and output library doesn't expose them to the linker. If
you want to document your API, do a better job than an automatic
header file.

>> What I do think we need is a new annotation, 'defaultImport
>> public/external'. This way your 'prim.sml' that does all the
>> _import/_address/_symbol'ing can be easily switched between
>> static/dynamic import.
>
> This shifts the point of change from the 'prim.sml' file to the 'xyz.mlb'
> file.

That's exactly what I wanted: one place to change. Some of my projects
have prim.sml files that run over 1000 lines. These aren't
auto-generated, so there's no possiible autogenerated ML import header
to have in this case. An annotation could solve this neatly, without
trying to 'sed' the file, which I think is a pretty crude and fragile
approach.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

From forum: MLton - Dev

Re: MLton import headers

2008-10-04T15:02:24Z

On Sat, 4 Oct 2008, Wesley W. Terpstra wrote:

> My personal opinion is that removing the default and requiring the
> user to always specify the linkage would be surprising when compared
> to how one typically uses C libraries. There are only two cases where
> the default is 'wrong'. 1) PIC archives, which are a corner case very
> few projects need. In this case the header has no default and forces
> you to pay attention. 2) You are compiling a library including ML code
> and C code. In this case we require:
> #define PART_OF_XYZ
> #include "xyz.h"
> inside your C files instead of:
> #include "xyz.h"
> #include "xyz-private.h"
> This doesn't seem particularly onerous or strange to me. You're
> cooperating with MLton to build a library in this case, so it's fair
> for us to have a (not-so-uncommon) convention you need to be aware of.

That seems like a reasonable rationale. If I understand things correctly,
though, one advantage of the
#include "xyz.h"
#include "xyz-private.h"
approach is that is textually separates the public (in the sense of the
documented portions of the library API available for use in other
projects) functions from the private (in the sense of the undocumented
portions of the library API) functions. That is, clients of "xyz.h" can't
even see the prototypes of the private functions. On the other hand, if
the platform supports the right visibility attributes, then although a
client might be able to see the prototype of a private function in a
(combined) "xyz.h" header, they would fail at link time, because the
function wouldn't be visible in the binary library. I guess you
acheive this effect in the (combined) "xyz.h" by using
#define MLLIB_PRIVATE(x)
when not PART_OF_XYZ to hide the private functions.

> What I do think we need is a new annotation, 'defaultImport
> public/external'. This way your 'prim.sml' that does all the
> _import/_address/_symbol'ing can be easily switched between
> static/dynamic import.

This shifts the point of change from the 'prim.sml' file to the 'xyz.mlb'
file. While one could use "-default-ann 'defaultImport public'", this
could have undesirable effects if you import from two different libraries
that demand different linking, since one global default doesn't apply.

Anyways, it doesn't appear that there are clear-cut solutions. But, I
tend to agree that an auto-generated SML import header is a sufficiently
rare corner case that it doesn't really demand extra compiler support.

-Matthew

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

From forum: MLton - Dev

Re: MLton import headers

2008-10-04T09:37:31Z

On Sat, Oct 4, 2008 at 4:50 PM, Matthew Fluet <fluet@...> wrote:
>> So you would propose that I always require the user to
>> #define PART_OF_X / STATIC_LINK_X / DYNAMIC_LINK_X before #include'ing
>> the header?
>
> Not necessarily. What is 'best/common' practice? Doing a
> 'grep -r "(visibility(" /usr/include/*' (on an x86-linux) turns up some
> headers that use it, but none appear to demand a
> #define PART_OF/STATIC_LINK/DYNAMIC_LINK in order to control the visibility.

On linux you can ignore the difference of STATIC_LINK and
DYNAMIC_LINK. Thus most projects do. Libraries that must also work as
a dll on windows are the only ones which have to care.

All library projects provide some form of 'PART_OF', but it differs on
a library-to-library basis. The most common approach is to provide a
public header and one or more private headers. Examples of this are
gdtoa, gmp, and sqlite3. Some libraries also use a macro. For example,
gmp uses __GMP_WITHIN_GMP.

gmp also distinguishes static/dynamic linkage with __GMP_LIBGMP_DLL.
My approach follows gmp very closely. gmp sets GMP_LIBGMP_DLL at
library creation time depending on if it is a static or dynamic
library. It also has this to say:
AC_MSG_ERROR([cannot build both static and DLL, since gmp.h is
different for each. Use "--disable-static --enable-shared" to build
just a DLL.])

I also set STATIC/DYNAMIC_LINK to some default in our export header
based on static/dynamic output. I've just allowed the user to override
it.

Very few libraries can compile to a PIC archive. glibc is one example,
but it clearly has no need to be portable to a non-ELF platform. This
is the only case where I currently leave no default. There is
definitely no 'best practice' for this type of library.

The current MLton approach is: the export header is exactly the same
for all output formats, except that the default
PART_OF/STATIC_LINK/DYNAMIC_LINK differs. The user can be explicit and
override this default.

On an ELF system you can use the export headers for both
static/dynamic, because EXERNAL/PUBLIC are identical on that target.
Thus you can do like most linux programs and supply one header that
works for both static and dynamic libraries.

My personal opinion is that removing the default and requiring the
user to always specify the linkage would be surprising when compared
to how one typically uses C libraries. There are only two cases where
the default is 'wrong'. 1) PIC archives, which are a corner case very
few projects need. In this case the header has no default and forces
you to pay attention. 2) You are compiling a library including ML code
and C code. In this case we require:
#define PART_OF_XYZ
#include "xyz.h"
inside your C files instead of:
#include "xyz.h"
#include "xyz-private.h"
This doesn't seem particularly onerous or strange to me. You're
cooperating with MLton to build a library in this case, so it's fair
for us to have a (not-so-uncommon) convention you need to be aware of.

> I don't see how functors and/or the ml basis sytem help here. Neither allow
> for control-flow or conditional compilation.

I meant to use the mlb-path-map approach. You can write your
library-dependent code as a functor and then bind it to one of the two
structures in a file chosen depending on a path variable.

At any rate, I no longer think the ML export header is necessary. The
only situtation it seems reasonable for an ML program to use an ML
library via FFI is if there was a pure C library between them, in
which case you probably shouldn't be using the internal ML library
anyway, but rather the C library's wrappers.

What I do think we need is a new annotation, 'defaultImport
public/external'. This way your 'prim.sml' that does all the
_import/_address/_symbol'ing can be easily switched between
static/dynamic import.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

From forum: MLton - Dev

Re: MLton import headers

2008-10-04T07:50:19Z

On Sat, 4 Oct 2008, Wesley W. Terpstra wrote:

> On Fri, Oct 3, 2008 at 10:21 PM, Matthew Fluet <fluet@...> wrote:
>> Things seem
>> sufficiently complex that it is unclear whether a complicated set of
>> implicit defaults is enough to shield a programmer completely (or even
>> mostly) from the details of which they need to be aware. If that is the
>> case, then it is often simpler to allow/force someone to be explicit up
>> front, since that may be easier than trying to back out of the implicit
>> defaults when the need arises.
>
> So you would propose that I always require the user to
> #define PART_OF_X / STATIC_LINK_X / DYNAMIC_LINK_X before #include'ing
> the header?

Not necessarily. What is 'best/common' practice? Doing a
'grep -r "(visibility(" /usr/include/*' (on an x86-linux) turns up some
headers that use it, but none appear to demand a
#define PART_OF/STATIC_LINK/DYNAMIC_LINK in order to control the
visibility.

>> ---foo_import.src---
>> structure Foo = struct
>> val libopen = _import "foo_open" FOO_SCOPE: int * string vector -> unit;
>> val libclose = _import "foo_close" FOO_SCOPE: unit -> unit;
>> end
>> ------
>> ---Makefile---
>> FOO_SCOPE=public
>>
>> foo_import.sml: foo_import_src
>> sed 's|FOO_SCOPE|$(FOO_SCOPE)|' < foo_import.scc > foo_import.sml
>
> Well, I thought it would be nicer to have
> structure FooStatic = struct
> val libopen = _import "foo_open" public: int * string vector -> unit;
> val libclose = _import "foo_close" public: unit -> unit;
> end
> structure FooDynamic = struct
> val libopen = _import "foo_open" public: int * string vector -> unit;
> val libclose = _import "foo_close" public: unit -> unit;
> end
> structure Foo = FooStatic (* or whatever a sane default is *)
>
> ... my way you can use functors and/or the basis system to do what you
> want without having to drop out to make.

I don't see how functors and/or the ml basis sytem help here. Neither
allow for control-flow or conditional compilation. In your example above,
there is no way to change Foo from FooStatic to FooDynamic without
changing the source file. As Vesa noted, you can use MLB variables to get
a poor-man's form of conditional compilation, but then you have to (in the
Makefile) load an appropriate mlb-path-map.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

From forum: MLton - Dev

Re: MLton import headers

2008-10-03T16:55:22Z

On Fri, Oct 3, 2008 at 10:21 PM, Matthew Fluet <fluet@...> wrote:
> Things seem
> sufficiently complex that it is unclear whether a complicated set of
> implicit defaults is enough to shield a programmer completely (or even
> mostly) from the details of which they need to be aware. If that is the
> case, then it is often simpler to allow/force someone to be explicit up
> front, since that may be easier than trying to back out of the implicit
> defaults when the need arises.

So you would propose that I always require the user to
#define PART_OF_X / STATIC_LINK_X / DYNAMIC_LINK_X before #include'ing
the header?

> ---foo_import.src---
> structure Foo = struct
> val libopen = _import "foo_open" FOO_SCOPE: int * string vector -> unit;
> val libclose = _import "foo_close" FOO_SCOPE: unit -> unit;
> end
> ------
> ---Makefile---
> FOO_SCOPE=public
>
> foo_import.sml: foo_import_src
> sed 's|FOO_SCOPE|$(FOO_SCOPE)|' < foo_import.scc > foo_import.sml

Well, I thought it would be nicer to have
structure FooStatic = struct
val libopen = _import "foo_open" public: int * string vector -> unit;
val libclose = _import "foo_close" public: unit -> unit;
end
structure FooDynamic = struct
val libopen = _import "foo_open" public: int * string vector -> unit;
val libclose = _import "foo_close" public: unit -> unit;
end
structure Foo = FooStatic (* or whatever a sane default is *)

... my way you can use functors and/or the basis system to do what you
want without having to drop out to make.

> Finally, importing a MLton library (either static or dynamic) into another
> MLton library or executable seems to be a fairly obscure usage.

I was aiming for completeness. Perhaps it's not necessary then to
output an ML import header; if you really need one, you can write it
yourself.

While I agree it's pretty bizarre to have two MLton libraries at once,
I think this could happen if you have pure C programs/libraries that
in turn use ML libraries.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

From forum: MLton - Dev

Re: MLton import headers

2008-10-03T14:28:26Z

On Fri, Oct 3, 2008 at 11:21 PM, Matthew Fluet <fluet@...> wrote:
> Finally, importing a MLton library (either static or dynamic) into another
> MLton library or executable seems to be a fairly obscure usage. [...]

I could certainly imagine some more or less plausible practical
reasons to do so, such as cutting compilation time. Although, given
the limited set of types available for interfacing with such
libraries, many such uses could just as well be written by compiling
the MLton "library" as a regular program that is spawned by the host
program as a separate process and communicated with via some form of
IPC. BTW, according to Armstrong's book this is a preferred way of
using foreign language libraries in Erlang.

> It is almost certainly a win to have all high-level language code sharing
> the same instance of the runtime/GC/etc.

Yes, if you are directly linking to the code, then I would tend to
agree (modulo exceptional practical concerns). Isolating parts of a
program to different processes, OTOH, may have practical advantages
such as fault isolation (bug in a subsystem cannot crash the entire
program) that can be more valuable than efficiency.

-Vesa Karvonen

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

From forum: MLton - Dev

Re: MLton import headers

2008-10-03T13:21:54Z

On Thu, 2 Oct 2008, Wesley W. Terpstra wrote:
> Thoughts?

The whole set of shared-library/visibility issues seems fairly opaque (a
reflection on the ABI itself, not its realization in MLton). From some
of the scenarios that Wesley has described, it often seems to be the
case that one has to be very clear on how a program/library links to other
libraries, how a library is generated depending on its future use, etc.
Things seem sufficiently complex that it is unclear whether a complicated
set of implicit defaults is enough to shield a programmer completely (or
even mostly) from the details of which they need to be aware. If that is
the case, then it is often simpler to allow/force someone to be explicit
up front, since that may be easier than trying to back out of the implicit
defaults when the need arises.

Also, picking up a theme similar to one that Vesa raised, is it so
difficult to set the visibility of an imported library at configure/build
time of a target. For example, something like:

---foo_import.src---
structure Foo = struct
val libopen = _import "foo_open" FOO_SCOPE: int * string vector -> unit;
val libclose = _import "foo_close" FOO_SCOPE: unit -> unit;
end
------
---Makefile---
FOO_SCOPE=public

foo_import.sml: foo_import_src
sed 's|FOO_SCOPE|$(FOO_SCOPE)|' < foo_import.scc > foo_import.sml
------

where the 'FOO_SCOPE=public' in the Makefile could either have been
determined at configure time (depending on the availability of libfoo.a or
libfoo.so) or left blank to be set at the 'make' invocation.

Finally, importing a MLton library (either static or dynamic) into another
MLton library or executable seems to be a fairly obscure usage.
Independent of the fact that MLton is a whole-program compiler (which
benefits from exposing all of the SML code code the compiler at once), I
don't know of any high-level language implementation (e.g., OCaml, GHC)
that prefers to import language-native libraries as though they were
language-independent system libraries. [One might argue that CLR/.NET is
an exception, but, really, the 'language-native library' in that instance
happens to be .NET assemblies.] It is almost certainly a win to have all
high-level language code sharing the same instance of the runtime/GC/etc.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

From forum: MLton - Dev

Re: MLton import headers

2008-10-03T06:04:22Z

On Fri, Oct 3, 2008 at 10:01 AM, Vesa Karvonen > In C and C++, and
particularly on Windows, one often uses macros in
> library headers and source files to insert appropriate compiler
> specific declarations to mark symbols ...

Yes, this is what MLton/svn does in the generated C import header.
However, in some cases a programmer still needs to specify how he
links a library. The MLton import headers do this with a macro:

#define STATIC_LINK_FOOBAR
#include <foobar.h>

> One thing that comes to mind from this is that it would be nice to be
> able to switch between static and dynamic linking easily without using
> a complicated mechanism at the source level. In fact, I think that
> ideally code that uses a library should really be exactly the same
> regardless of whether the library is linked statically or
> (unoptionally) dynamically. I'm not saying that it isn't possible with
> the above format (you need to bind one of the modules, STATIC_LINK_M1
> or DYNAMIC_LINK_M1, to a module that you use in the client code), but
> rather that you might want to consider this from that perspective.

I agree this would be nice, but unfortunately it isn't. The assembly/C
generated by MLton to use a library must know whether those symbols
are imported from a dynamically or statically linked library. MLton
does not know this. When we pass '-lfoo' to the linker, it picks some
library out of paths we know nothing about.

> One alternative that comes to mind would be to specify as a
> command-line option whether one wishes to link statically or
> dynamically and MLton would then generate only one set of imports and
> the module name would be the same (e.g. just M1) in either case.

I am not sure where these options are to be passed, but it sounds like
you mean at library creation. What I think you've overlooked is that
the same library can be linked three different ways.

Consider a PIC archive library foo.

During compilation of C that will be included in the final PIC library
you have access to private symbols and shouldn't import symbols that
will be in the final foo product.

A shared library built against foo will statically link foo in. This
means it no longer has access to private symbols, but shouldn't be
importing the symbols either as they will be in the same final DSO.
MLton import headers call this PART_OF_LIBNAME.

A user of the resulting shared library wants to call functions that
were in foo. He has no access to private symbols and needs to import
the public ones. MLton import headers call this STATIC_LINK_LIBNAME

Each of these three uses are distinct and if you try to coflate any of
them I can present you with a platform where your solution breaks.
MLton import headers call this DYNAMIC_LINK_LIBNAME.

While it's certainly true that for many libraries there is a sane
default, sometimes there isn't. In MLton/svn terminology, we have four
output formats. The generated header access modes are described below:

executable -- only ever makes sense to use PART_OF_LIBNAME linkage
archive -- currently defaults to STATIC_LINK_LIBNAME, but if C is
included into the resulting library, PART_OF_LIBNAME should be defined
to override this default.
library -- currently defaults to DYNAMIC_LINK_LIBNAME, but override to
PART_OF_LIBNAME makes sense.
libarchive -- the case I detailed above, the import header has no
default and you have to specify the linkage.

I imagine we would output ML import files in the same way, eg: default
M1 = STATIC_LINK_M1 for archive, M1 = DYNAMIC_LINK_M1 for library. No
default for libarchive. Happily, there is no PART_OF_M1 case. At any
rate, this is what I am proposing: an automatic M1 that "does the
right thing" when it can, but includes both options in case you need
them.

Another problem on my mind, is that it's fairly common (on linux) for
libraries to be shipped as both dynamic (PIC .so) and static (non-PIC
archive), with only one header file. Sometimes important libraries
also include a PIC archive. This works because happily on ELF systems
public/external end up being the same. I'm not sure how to support
this model and at the same time also support platforms like windows
where public/external are critically different.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

From forum: MLton - Dev

Re: MLton import headers

2008-10-03T01:01:46Z

On Thu, Oct 2, 2008 at 1:20 PM, Wesley W. Terpstra <wesley@...> wrote:
[...]
> One of the problems with using a library is that you need to know how
> you are linking against it. Often programmers can get away without
> knowing, but this breaks down on some combinations of operating
> system, architecture, and definitions.

In C and C++, and particularly on Windows, one often uses macros in
library headers and source files to insert appropriate compiler
specific declarations to mark symbols that are either exported from
the currently compiled DLL or imported from an external DLL. These
macros are often defined in a configuration header of some sort and
one can then easily switch between various linkage options
(static/dynamic library, import/export symbols).

> When using a MLton generated library from another MLton program, there
> is the same problem. _import "foo" public: ...; is appropriate for a
> static link, while _import "foo" external: ...; is appropriate for a
> dynamic link. I was thinking of an output file something like:
>
> signature M1 =
> sig
> val m1_open : int * string vector -> unit;
> val m1_close : unit -> unit;
> end
>
> structure STATIC_LINK_M1 :> M1 =
> struct
> val m1_open = _import "m1_open" public;
> val m1_close = _import "m1_close" public;
> end
>
> structure DYNAMIC_LINK_M1 :> M1 =
> struct
> val m1_open = _import "m1_open" external;
> val m1_close = _import "m1_close" external;
> end
>
> I have intentionally left out PART_OF_M1 because you're better off not
> using the FFI in this case.
>
> Thoughts?

One thing that comes to mind from this is that it would be nice to be
able to switch between static and dynamic linking easily without using
a complicated mechanism at the source level. In fact, I think that
ideally code that uses a library should really be exactly the same
regardless of whether the library is linked statically or
(unoptionally) dynamically. I'm not saying that it isn't possible with
the above format (you need to bind one of the modules, STATIC_LINK_M1
or DYNAMIC_LINK_M1, to a module that you use in the client code), but
rather that you might want to consider this from that perspective.

One alternative that comes to mind would be to specify as a
command-line option whether one wishes to link statically or
dynamically and MLton would then generate only one set of imports and
the module name would be the same (e.g. just M1) in either case. This
import file generation step could be called as a part of the build
script (for the library) and the resulting module would then be used
to call the library. Alternatively, the module could be selected in a
MLB file based on a path variable and the library would have
pregenerated import files for both cases (static/dynamic). In either
case, the SML client code would be exactly the same regardless of
whether the library is linked statically or dynamically.

-Vesa Karvonen

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

From forum: MLton - Dev

MLton import headers

2008-10-02T03:20:17Z

Currently MLton outputs C import headers (ironically with
-export-header). I think it might also be useful to output an ML
import file.

One of the problems with using a library is that you need to know how
you are linking against it. Often programmers can get away without
knowing, but this breaks down on some combinations of operating
system, architecture, and definitions. One of the tasks the C import
header is to make the details transparent to the user.

When using a MLton generated library from another MLton program, there
is the same problem. _import "foo" public: ...; is appropriate for a
static link, while _import "foo" external: ...; is appropriate for a
dynamic link. I was thinking of an output file something like:

signature M1 =
sig
val m1_open : int * string vector -> unit;
val m1_close : unit -> unit;
end

structure STATIC_LINK_M1 :> M1 =
struct
val m1_open = _import "m1_open" public;
val m1_close = _import "m1_close" public;
end

structure DYNAMIC_LINK_M1 :> M1 =
struct
val m1_open = _import "m1_open" external;
val m1_close = _import "m1_close" external;
end

I have intentionally left out PART_OF_M1 because you're better off not
using the FFI in this case.

Thoughts?

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

From forum: MLton - Dev

TLDI 2009: call for papers

2008-10-01T13:05:57Z

[ Just a quick reminder that the TLDI deadline is Oct 8th... ]

*********************************************************************
CALL FOR PAPERS

TLDI 2009

ACM SIGPLAN Workshop on
Types in Language Design and Implementation

24 January 2009
Savannah, Georgia, USA

To be held in conjunction with POPL 2009

http://ttic.uchicago.edu/~amal/tldi2009/
*********************************************************************

IMPORTANT DATES

Submission: 8 Oct 2008, 5PM EDT (Wed)
Notification: 8 Nov 2008 (Sat)
Camera ready: 19 Nov 2008 (Wed)
TLDI'09: 24 January 2009 (Sat)

SCOPE

The role of types and proofs in all aspects of language design,
compiler construction, and software development has expanded greatly
in recent years. Type systems, type analyses, and formal deduction
have led to new concepts in compilation techniques for modern
programming languages, verification of safety and security properties
of programs, program transformation and optimization, and many other
areas. In light of this expanding role of types, the ACM SIGPLAN
Workshop on Types in Language Design and Implementation (TLDI'09)
follows six previous International Workshops on types in compilation
and language design (TIC'97, TIC'98, TIC'00, TLDI'03, TLDI'05, and
TLDI'07), with the hope of bringing together researchers to share new
ideas and results in this area.

Submissions for this event are invited on all interactions of types
with language design, implementation, and programming methodology.
This includes both practical applications and theoretical aspects.
TLDI'09 specifically encourages papers from a broad field of
programming language and compiler researchers, including those working
in object-oriented, dynamically-typed, late-binding, systems
programming, and mobile-code paradigms, as well as traditional
fully-static type systems. Topics of interest include:

- Typed intermediate languages and type-directed compilation
- Type-based language support for safety and security
- Types for interoperability
- Type systems for system programming languages
- Type-based program analysis, transformation, and optimization
- Dependent types and type-based proof assistants
- Types for security protocols, concurrency, and distributed computing
- Type inference and type reconstruction
- Type-based specifications of data structures and program invariants
- Type-based memory management
- Proof-carrying code and certifying compilation

This is not meant to be an exhaustive list; papers on novel
utilizations of type information are welcome. Authors concerned about
the suitability of a topic are encouraged to inquire via electronic
mail to the program chair prior to submission.

SUBMISSION GUIDELINES

Authors should submit a full paper of no more than 12 pages (including
bibliography and appendices) by Wednesday, October 8, 2008 5PM Eastern
Daylight Savings Time. The submission deadline and length limitations
are firm. Submissions that do not meet these guidelines will not be
considered.

All submissions should be in standard ACM SIGPLAN conference format:
two columns, nine-point font on a ten-point baseline. Detailed
formatting guidelines are available on the SIGPLAN Author Information
page, along with a LaTeX class file and template:

http://www.sigplan.org/authorInformation.htm

Papers must be submitted in Adobe Portable Document Format (PDF) and
must be formatted for US Letter size (8.5"x11") paper. Authors for
whom this is a hardship should contact the program chair before the
deadline.

Submitted papers must adhere to the SIGPLAN Republication Policy:

http://www.sigplan.org/republicationpolicy.htm

Submissions should contain original research not published or
submitted for publication elsewhere.

The URL for submission will be announced closer to the deadline.

GENERAL CHAIR

Andrew Kennedy Microsoft Research, Cambridge

PROGRAM CHAIR

Amal Ahmed Toyota Technological Institute, Chicago

PROGRAM COMMITTEE

Amal Ahmed Toyota Technological Institute, Chicago (Chair)
Juan Chen Microsoft Research
Peter Dybjer Chalmers University of Technology
Jeff Foster University of Maryland, College Park
Neal Glew Intel
Robert Harper Carnegie Mellon University
Andrew Myers Cornell University
Atsushi Ohori Tohoku University
Matthew Parkinson University of Cambridge
Didier Remy INRIA Paris-Rocquencourt
Andreas Rossberg Max Planck Institute for Software Systems

STEERING COMMITTEE

Craig Chambers University of Washington
Robert Harper Carnegie Mellon University (Chair)
Xavier Leroy INRIA Paris-Rocquencourt
Greg Morrisett Harvard University
George Necula Rinera Networks and UC Berkeley
Atsushi Ohori Tohoku University
Francois Pottier INRIA Paris-Rocquencourt
Zhong Shao Yale University

_______________________________________________
MLton-user mailing list
MLton-user@...
http://mlton.org/mailman/listinfo/mlton-user

From forum: MLton - User

Re: Re: [MLton-commit] r6883

2008-09-30T15:31:07Z

On Tue, Sep 30, 2008 at 11:31 PM, Matthew Fluet <fluet@...> wrote:

> On Tue, 23 Sep 2008, Wesley Terpstra wrote:
>>
>> When building a library, assume -default-ann "allowFFI true".
>
>> + (* It doesn't make sense to have a library without FFI *)
>> + val () =
>> + case !format of
>> + Executable => ()
>> + | _ => ignore (Control.Elaborate.processDefault "allowFFI
>> true")
>> +
>
> While I agree that it doesn't make sense to build a library that doesn't
> export any functions (and, thus, uses FFI), I don't think that the default
> annotation should be 'allowFFI true'. Yes, it is a little more concise for
> a one-file *.sml library, but it is an abstraction violation for a
> well-designed, modular, multi-file *.mlb project. In that setting, you
> would expect a project like:
>
> foo.mlb (* 'pure' SML *)
> bar.mlb (* 'pure' SML *)
> baz.sml (* 'pure' SML *)
> qux.sml (* 'pure' SML *)
> ann "allowFFI true" in
> export.sml (* export functions *)
> end
>
> Not defaulting to "allowFFI true" ensures that no FFI is used in the 'pure'
> SML portions of the library.

Fair enough. It might make sense to enable it for just sml files then,
just as MLton.* is available for sml files but not mlb files by
default. On the other hand, I don't really care that much. =)

Feel free to revert it and update regresson/library/library-test.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

From forum: MLton - Dev

Re: Re: [MLton-commit] r6887

2008-09-30T15:24:58Z

On Tue, Sep 30, 2008 at 11:17 PM, Matthew Fluet <fluet@...> wrote:
> On Tue, 23 Sep 2008, Wesley Terpstra wrote:
>> feround.c and IEEEReal.c carried duplicated fesetround code.
>> The IEEEReal i386 version also works on x86_64.
>
> This is not technically true, as it only sets the control word on the x87
> unit and not on the SSE unit.

An earlier commit by me added the SSE instructions to update the SSE
unit at the same time (check IEEEReal.c -- I added 64 bit assembly) as
the x87 unit. At the time I didn't realize there were two copies so
only updated the one. The commit you saw merged them and removed the
old version.

> For amd64, we use the SSE instructions exclusively for floating-point computation, so changing the x87 control word has no effect.
> I doubt that there is any amd64 platform that does not provide
> fe{get,set}round in libm, so I don't think it is necesary to provide
> assembly for amd64.

Not true. I added that code because it was needed on MinGW/win64. The
"libc" there also uses x87 ops, so we need to set both since MLton
native codegen uses the other. I assume it's supposed to affect
cos/sin/etc? At least it passes the regressions as it is now.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

From forum: MLton - Dev

Re: [MLton-commit] r6883

2008-09-30T14:31:13Z

On Tue, 23 Sep 2008, Wesley Terpstra wrote:
> When building a library, assume -default-ann "allowFFI true".

> + (* It doesn't make sense to have a library without FFI *)
> + val () =
> + case !format of
> + Executable => ()
> + | _ => ignore (Control.Elaborate.processDefault "allowFFI true")
> +

While I agree that it doesn't make sense to build a library that doesn't
export any functions (and, thus, uses FFI), I don't think that the default
annotation should be 'allowFFI true'. Yes, it is a little more concise
for a one-file *.sml library, but it is an abstraction violation for a
well-designed, modular, multi-file *.mlb project. In that setting, you
would expect a project like:

foo.mlb (* 'pure' SML *)
bar.mlb (* 'pure' SML *)
baz.sml (* 'pure' SML *)
qux.sml (* 'pure' SML *)
ann "allowFFI true" in
export.sml (* export functions *)
end

Not defaulting to "allowFFI true" ensures that no FFI is used in the
'pure' SML portions of the library.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

From forum: MLton - Dev

Re: Re: [MLton-commit] r6887

2008-09-30T14:28:13Z

On Tue, 30 Sep 2008, Matthew Fluet wrote:

> On Tue, 23 Sep 2008, Wesley Terpstra wrote:
>> feround.c and IEEEReal.c carried duplicated fesetround code.
>> The IEEEReal i386 version also works on x86_64.
>
> This is not technically true, as it only sets the control word on the x87
> unit and not on the SSE unit. For amd64, we use the SSE instructions
> exclusively for floating-point computation, so changing the x87 control word
> has no effect. See
> http://sources.redhat.com/cgi-bin/cvsweb.cgi/~checkout~/libc/sysdeps/x86_64/fpu/fesetround.c?rev=1.2&cvsroot=glibc
>
> I doubt that there is any amd64 platform that does not provide
> fe{get,set}round in libm, so I don't think it is necesary to provide assembly
> for amd64.

Hmm, I got confused looking at just the patch in the commit log. I guess
the SSE control word changes were added some time ago. And, looking at
that commit entry (r6693), it seems that MinGW on Win64 does not provide
(working) fe{get,set}round in libm.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

From forum: MLton - Dev

Re: [MLton-commit] r6887

2008-09-30T14:17:58Z

On Tue, 23 Sep 2008, Wesley Terpstra wrote:
> feround.c and IEEEReal.c carried duplicated fesetround code.
> The IEEEReal i386 version also works on x86_64.

This is not technically true, as it only sets the control word on the x87
unit and not on the SSE unit. For amd64, we use the SSE instructions
exclusively for floating-point computation, so changing the x87 control
word has no effect. See
http://sources.redhat.com/cgi-bin/cvsweb.cgi/~checkout~/libc/sysdeps/x86_64/fpu/fesetround.c?rev=1.2&cvsroot=glibc

I doubt that there is any amd64 platform that does not provide
fe{get,set}round in libm, so I don't think it is necesary to provide
assembly for amd64.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

From forum: MLton - Dev

Re: getText{Start,End} considered harmful

2008-09-30T12:19:51Z

On Tue, 23 Sep 2008, Ville Laurikari wrote:
> On Mon, Sep 22, 2008 at 06:33:43PM +0200, Wesley W. Terpstra wrote:
>> MLton currently uses two functions, GC_getText{Start,End}, to create a giant
>> lookup table for profiling. These two functions are among the hardest things
>> to supply when porting MLton. It becomes even harder once shared libraries
>
> I can vouch for the difficulty of porting these. One can always of
> course just #define HAS_TIME_PROFILING FALSE and live without
> profiling...

On an architecture that doesn't support native codegen (i.e., that must
use the C-codegen), then one could probably have defined the
GC_getText{Start,End} functions to return NULL (or to even 'die'), since
the C-codegen uses the 'time-field' approach to time profiling (rather
than the 'time-label' approach).

But, a moot point after Wesley's commit.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

From forum: MLton - Dev

Re: getText{Start,End} considered harmful

2008-09-30T12:15:39Z

On Sun, 28 Sep 2008, Wesley W. Terpstra wrote:
> On Sun, Sep 28, 2008 at 7:01 PM, Matthew Fluet <fluet@...> wrote:
>> There are/were other issues with the approach to time profiling:
>> http://mlton.org/pipermail/mlton/2005-November/028283.html
>> http://mlton.org/pipermail/mlton-commit/2005-November/000238.html
>
> Those emails propose to use code-modification for the C codegen and
> the labels for the native codegen. I didn't intend to change this.
> AFAIK, the C-codegen profiling path is completely unaffected.

Agreed that the C-codegen profiling path is unaffected by your changes.
But, I brought up that old thread because it lays out the arguments for
(and against) keeping the time overhead of profiling low. Something not
brought up in that thread is that, while using code labels in the native
codegens works, it does complicate the native codegens somewhat, because
everytime a new assembly basic block is introduced, it needs to conjure up
the right profiling information to associate with the new profiling label.
Also, it is unlikely that we could easily use labels in a reusable codegen
framework (e.g., MLRISC, C--, LLVM).

Just as you noted that the sourceLabels could be compressed by eliminating
duplicates, the 'field' approach to time profiling in the C-codegen (and
available on the native codegens) could probably be improved by avoiding
idempotent assignments to the curSourceSeqsIndex.

Anyways, your approach is certainly expedient for the problem at hand.
And the self-compile didn't suffer from the binary search. So, consider
the above just general comments.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

From forum: MLton - Dev

Re: getText{Start,End} considered harmful

2008-09-28T14:27:58Z

On Sun, Sep 28, 2008 at 7:01 PM, Matthew Fluet <fluet@...> wrote:
> My guess is that profiling with shared libraries is a bit much to ask. For
> one, you've worked to ensure that multiple ML shared libraries can be used
> within the same executable, but what would it mean for both of them to be
> profiling libraries -- they would both want to use SIGPROF for their own
> purposes.

Yes, this is clear. However, it seems reasonable to expect to profile
one library. The GC_getText{Start,End} were causing segfaults in this
case.

> There are/were other issues with the approach to time profiling:
> http://mlton.org/pipermail/mlton/2005-November/028283.html
> http://mlton.org/pipermail/mlton-commit/2005-November/000238.html

Those emails propose to use code-modification for the C codegen and
the labels for the native codegen. I didn't intend to change this.
AFAIK, the C-codegen profiling path is completely unaffected.

> Running some actual benchmarks (e.g., with the hamlet benchmark, as is
> reported in the above messages) provides real evidence; "I imagine..." and
> "I would contend..." are good hypotheses, but are not experiments.

I should probably run some more experiments, yes.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

From forum: MLton - Dev

Re: getText{Start,End} considered harmful

2008-09-28T10:06:50Z

On Sun, 28 Sep 2008, Matthew Fluet wrote:
> I see that such a patch has already been committed, but it would be nice to
> report the actual runtime (and profiling results) impact of such a change.

Sorry, my fault for reading/responding to messages in sequential order.
Your later results with the self-compile seem to support the argument that
a binary search suffices.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

From forum: MLton - Dev

Re: getText{Start,End} considered harmful

2008-09-28T10:01:05Z

On Mon, 22 Sep 2008, Wesley W. Terpstra wrote:
> MLton currently uses two functions, GC_getText{Start,End}, to create a giant
> lookup table for profiling. These two functions are among the hardest things
> to supply when porting MLton. It becomes even harder once shared libraries
> are considered, because then one needs to consider which text segment is
> intended. After reading the i386 sys v abi specification, I still have no
> idea how to do this correctly for shared libraries on elf.

My guess is that profiling with shared libraries is a bit much to ask.
For one, you've worked to ensure that multiple ML shared libraries can be
used within the same executable, but what would it mean for both of them
to be profiling libraries -- they would both want to use SIGPROF for their
own purposes.

Nonetheless, I appreciate the GC_getText{Start,End} difficulties.
There are/were other issues with the approach to time profiling:
http://mlton.org/pipermail/mlton/2005-November/028283.html
http://mlton.org/pipermail/mlton-commit/2005-November/000238.html

> It seems to me that not only are these a portability problem, they are
> completely unnecessary. Making a giant lookup table for every address in the
> text segment hardly seems like a worthy goal. I imagine the justification is
> that it has minimal impact on the performance of the program during handling
> of SIGPROF. However, I would contend that a giant lookup table could very
> well be more intrusive than just using a binary search of a compressed
> (repeated sourceSeqIndexes eliminated) sourceMaps.sourceLabels. Even if the
> binary search does have a greater impact, at 1kHZ it will hardly matter.

Running some actual benchmarks (e.g., with the hamlet benchmark, as is
reported in the above messages) provides real evidence; "I imagine..." and
"I would contend..." are good hypotheses, but are not experiments.

> So to sum up: could I just kill these off and use a binary search?

I see that such a patch has already been committed, but it would be nice
to report the actual runtime (and profiling results) impact of such a
change.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

From forum: MLton - Dev

Re: bogus warning messages about scope annotations

2008-09-25T01:09:32Z

Upon deeper digging, I realize that I was getting confused between two
different
copies of the source tree (I was editing one that had the annotations,
while compiling
the other). Sorry for the false alarm.

- John

On Sep 25, 2008, at 2:16 AM, Wesley W. Terpstra wrote:

> On Wed, Sep 24, 2008 at 10:38 PM, John Reppy <jhreppy@...> wrote:
> Using a fresh version of mlton on Mac OS X, I'm getting messages of
> the form
>
> Warning: /Users/jhr/Work/ICFPContest08/sml3d/src/glut/glut.sml 413.28.
> Symbol 'glutCreateMenuCB' redeclared as external (previously
> public). This may cause linker errors.
>
> when the code at lines 412-413 is
>
> val exportCreateMenu = _export "glutCreateMenuCB" private
> cdecl : (int -> unit) -> unit;
> val glutCreateMenuCB = _address "glutCreateMenuCB"
> private : Ptr.t;
>
> That's strange. I copy and pasted those lines to a file (replacing
> Ptr with MLton.Pointer) and saw no warnings. Can you please provide
> a complete source file which causes these warnings?
>
> I've tried various other annotations instead of private, but the
> message is
> consistent. In fact, it turns out that one can put any identifier
> as an annoation and
> still get the same message. E.g.,
>
> val exportCreateMenu = _export "glutCreateMenuCB" bogus
> cdecl : (int -> unit) -> unit;
> val glutCreateMenuCB = _address "glutCreateMenuCB" bogus :
> Ptr.t;
>
> The default visibility is public for export and external for
> address. So that's not unexpected as bogus is invalid and didn't
> override the defaults. I assume you also got errors that 'bogus' was
> unrecognized in this case?
>
> I did not get this warning on the r6841 version on my windows box at
> work.
>
> This check did not exist in r6841.
>

_______________________________________________
MLton-user mailing list
MLton-user@...
http://mlton.org/mailman/listinfo/mlton-user

From forum: MLton - User