Nabble - MLton - Dev

Re: [MLton-user] How to write performant network code

2009-01-07T10:31:20Z

I certainly am not arguing against calling memcpy(), but note that for small
chunks of memory, the overhead of the the extra call to C (including passing
parameters on the stack) could slow things down. Still, probably the way to
go.

For large copies, it can often be worth while checking the alignment and then
moving a word at a time.

I don't have any idea on the hashing problem.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

Training & jobs

2008-12-18T22:59:13Z

“Join us to make future”
FOR OPT/F-1 STUDENTS
FREE TRAINING
FOOD ACCOMODATION
H1B PROCESSING
FOR L1/ L2/ H1/ H4/ EAD/ GC
EXCELLENT OPPRTUNITIES
TRAINING
PLACEMENT
Highly Competitive offers for New H1b Transfers

About Us:
V2 technology inc is serving NJ since 2005, for us our employees are
of utmost importance. Our highly skilled and dedicated instructors
train you explicitly in market-related technologies for today and
tomorrow. We will work with you in developing marketing strategies
and finding the assignments of your choice.
We not only help you get a job but we build your long lasting
careers!!!
What are we Looking for:
Excellent communication skills.
Valid F1/ OPT/ CPT/ H1/ H4/ L1 or valid work status
(EAD, GC).
Bachelors’ degree in CS/ IT or previous IT experience.
Preferably (not mandatory) Master's Degree in Computers/
Electronics or previous IT experience.
Willing to relocate anywhere in USA.

What’s The Deal:
H1-B sponsorship to F1/ OPT/ H4/ L1/ L2/ E3/ EAD
Transfer of H1 & L1 visas.
Green Card sponsorship through PERM
Job focused professional training in
Java/ J2EE
.NET
Documentum
Share point
Oracle
Data Modeling
Relocation assistance- airfare, hotel accommodation, car rental
etc
Guarantee lowest bench period.
Employee referral program.
Effective Resume writing help, Marketing and Placement.
Technical mock interviews. Preparation of In Person Interviews

100% Guaranteed placement
Get in Touch:
Contact immediately to know more about the opportunities with us
at:609-843-0312
or
write mail at jobs@v2techinc.com
Contact person : John

Re: MLton bug: unhandled exception in Basis Library.

2008-12-16T11:26:47Z

On Tue, 16 Dec 2008, Henry Cejtin wrote:
> Wouldn't it be easy if the emergeny handler just went straight to C code
> (via the FFI) to print out the history?

Good point. Then it would probably suffice to move the implementation of
MLton.CallStack.current to <src>/basis-library/prim-mlton.sml, also do the
non-trivial setExtendExtra in <src>/basis-library/prim-mlton.sml, and, for
the emergency handler, hand off the entire call-stack array to C to
look-up and print the source strings.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

Re: MLton bug: unhandled exception in Basis Library.

2008-12-16T10:10:54Z

Wouldn't it be easy if the emergeny handler just went straight to C code (via the FFI) to print out the history?

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

RE: MLton bug: unhandled exception in Basis Library.

2008-12-16T08:37:38Z

On Mon, 15 Dec 2008, Nicolas Bertolotti wrote:
> Not really what I am looking for... (I already know how to use the exception history)
>
> The problem here is that the exception occurs at program startup during the initialisation of the basis library.
>
> At this stage, there is no real way to catch the exception and print the history.

The "fancy" top-level exception handler, that prints the history, is
implemented in
<src>/basis-library/primitive/mlton/exn.sml
and "installed" by 'Primitive.TopLevel.setHandler' in
<src>/basis-library/primitive/mlton/mlton.sml

An unhandled exception that is raised during the evaluation of the Basis
Library before the installation of the "fancy" top-level exception handler
is handled by the "emergency" top-level exception handler that is
"installed" in
<src>/basis-library/primitive/prim2.sml

If you want the emergency exception handler to print an exception history,
then you need to move (some portion of) the fancy exception handler
implementation back to the implementation of the emergency handler. In
particular, you need to call 'Primitive.MLton.Exn.setExtendExtra' with a
non-trivial function that captures the call-stack and you need to walk the
call-stack, extract the entries, and print them. It won't be trivial,
since, at the point of definition of the emergency exception handler, you
have almost no Basis Library functions available; everything will need to
be done with very primitive operations.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

RE: MLton bug: unhandled exception in Basis Library.

2008-12-15T04:20:37Z

Not really what I am looking for... (I already know how to use the exception history)

The problem here is that the exception occurs at program startup during the initialisation of the basis library.

At this stage, there is no real way to catch the exception and print the history.

> -----Original Message-----
> From: Vesa Karvonen [mailto:vesa.a.j.k@...]
> Sent: Friday, December 12, 2008 2:44 PM
> To: Nicolas Bertolotti
> Cc: mlton@...
> Subject: Re: [MLton] MLton bug: unhandled exception in Basis Library.
>
> On Thu, Dec 11, 2008 at 7:18 PM, Nicolas Bertolotti
> <Nicolas.Bertolotti@...> wrote:
> [...]
> > I am now trying to identify the call that raises this error but I can
> not
> > figure how to retrieve and print the exception history.
>
> The following post might be helpful:
> http://mlton.org/pipermail/mlton/2006-December/029466.html . As this
> is at least the second time this issue has shown up, it might be
> useful to add this to the FAQ (http://mlton.org/FAQ).
>
> -Vesa Karvonen

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

Re: MLton bug: unhandled exception in Basis Library.

2008-12-12T05:44:21Z

On Thu, Dec 11, 2008 at 7:18 PM, Nicolas Bertolotti
<Nicolas.Bertolotti@...> wrote:
[...]
> I am now trying to identify the call that raises this error but I can not
> figure how to retrieve and print the exception history.

The following post might be helpful:
http://mlton.org/pipermail/mlton/2006-December/029466.html . As this
is at least the second time this issue has shown up, it might be
useful to add this to the FAQ (http://mlton.org/FAQ).

-Vesa Karvonen

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

MLton bug: unhandled exception in Basis Library.

2008-12-11T09:18:50Z

Hello,

After changing the way my SML binary is launched on Windows (previously using spawnve(), now using CreateProcess()), I am facing this crash at the beginning of its execution.

The crash is version dependent (e.g. works fine on Windows Vista ; no longer works on Windows XP).

I added a « print » in the function raiseSys() which shows that the error corresponds to a « Bad file descriptor ».

I am now trying to identify the call that raises this error but I can not figure how to retrieve and print the exception history.

Any hint ?

Thanks in advance

Accelerating the pace of engineering and science

Nicolas Bertolotti
Senior Development Engineer

2 Rue de Paris
92196 Meudon Cedex

France

Nicolas.Bertolotti@...

tel:
fax:
mobile:

+33.1.41.14.88.55

+33.1.55.64.06.64

+33.6.86.41.87.15

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

Re: Including dlfcn-win32 in MLton

2008-11-25T15:45:19Z

Ville Laurikari wrote:
> Dan, have you had any luck in getting commit access to get your patch
> included in the official SVN repo
I'm afraid I haven't been able to get in touch with Ramiro Polla. If you
find his e-mail address,
please let me know, too.

-Dan

--
Dan DuVarney
Senior Software Engineer Reactive Systems Inc.
duvarney@... +1 (919) 324-3507 ext. 103
OpenPGP (GnuPG) public key file:
http://www.reactive-systems.com/~duvarney/pgp_public_key.txt

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

Including dlfcn-win32 in MLton

2008-11-24T06:55:57Z

Hi,

There was discussion a while ago regarding removing the built-in
implementation on dlopen() and friends from the MLton runtime (on
MinGW), and using dlfcn-win32 instead.

An unpatched version of dlfcn-win32 does not work at all with MLton,
which Dan DuVarney no doubt has noticed and made a patch:
http://code.google.com/p/dlfcn-win32/issues/detail?id=2

With this patch, it at least appears to work. Before, programs would
just throw a DynLinkError with an empty messag. I didn't check if it
leaks handles or not, but I believe Vesa in that it does leak.

Dan, have you had any luck in getting commit access to get your patch
included in the official SVN repo?

My point is that if we plan to use dlfcn-win32 in MLton, it would be
good that the upstream version works as-is. Having to patch it sort
of defeats the purpose of not having to maintain our own code for
dlopen().

BTW, I could not find an email address for Ramiro Polla, the author of
dlfcn-win32. Does anyone have it?

--
http://www.iki.fi/vl/

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

Re: Re: [MLton-user] SVN r6941 MLton/MinGW32 and FFI

2008-11-21T06:42:47Z

On Mon, Nov 17, 2008 at 04:00:08PM +0100, Wesley W. Terpstra wrote:
> Ok, so we can remove the MLton ones? I think they are broken atm anyway
> since they get prefix with MLton_ by the mingw.h.

I can confirm that the current dlopen and friends from mingw.c don't
work, exactly because they are prefixed with MLton_.

> I was thinking we could just include their (fixed) library in the
> MLton mingw installer.

This seems like a good idea.

--
http://www.iki.fi/vl/

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

RE: A few changes proposals for MLton

2008-11-18T19:46:48Z

On Mon, 17 Nov 2008, Nicolas Bertolotti wrote:
> I am also not really sure that the proposed operation:
>> + maxSize = alignDown(SIZE_MAX, s->sysvals.pageSize);
>> + maxSize -= sizeofCardMapAndCrossMap(s, maxSize);
> really returns the exact maximum size for a heap. It is not really an
> issue but you may think of a better way.

True, your operation yields a maxSize that is slightly smaller than
necessary, because it uses a larger than necessary card/cross map size.
One could write an invertSizeofCardMapAndCrossMap function such that:

heapSize = invertSizeofCardMapAndCrossMap (s, heapWithMapsSize);

implies

heapSize + sizeofCardMapAndCrossMap (s, heapSize)
<= heapWithMapsSize
< (heapSize + s->sysvals.pageSize)
+ sizeofCardMapAndCrossMap (s, heapSize + s->sysvals.pageSize)

which yields the largest (up to the pageSize) heapSize such that adding
its card/cross map size requires space less than or equal to the input
heapWithMapsSize.

According to my calculations, your operation yields:
maxSize = 4,294,963,200 - 33,554,432
= 4,261,408,768
with
sizeofCardMapAndCrossMap(maxSize) = 33,292,288
and
maxSize + sizeofCardMapAndCrossMap(maxSize) = 4,294,701,056

However, one could take:
maxSize = 4,261,662,720
with
sizeofCardMapAndCrossMap(maxSize) = 33,300,480
and
maxSize + sizeofCardMapAndCrossMap(maxSize) = 4,294,963,200

In practice, though, either of these values for maxSize is well above what
is available in a contiguous mmap allocation (either with mremap or after
dumping the heap to disk).

> It could also be slightly more efficient to compute this value only once
> at the beginning of the execution.

Agreed, although sizeofHeapDesired isn't on a critical path, so I doubt
that it would have any measurable effect.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

RE: A few changes proposals for MLton

2008-11-17T23:37:51Z

The only reason why I do this is that it is not possible to do a « chdir » to “\\pumpkin”. This is how Windows works (too bad) and I’m afraid there is no other way to support UNC paths.

I will join the discussion…

From: Wesley W. Terpstra [mailto:wesley@...]
Sent: Tuesday, November 18, 2008 12:45 AM
To: Nicolas Bertolotti
Cc: Matthew Fluet; mlton@...
Subject: Re: [MLton] A few changes proposals for MLton

On Mon, Nov 17, 2008 at 11:49 PM, Nicolas Bertolotti <Nicolas.Bertolotti@...> wrote:

I am sorry but I don't really understand what you mean with « You should bring up having UNC volumes as that is currently forbidden by the definition » (maybe it is just a problem with my English understanding).

Your patch changes OS.Path.fromString "\\pumpkin\media\foo" to return { vol="\\pumpkin\media", isAbs=true, arcs=["foo"] }. The problem with that is that the standard <http://mlton.org/basis/os-path.html> states "Under Microsoft Windows, the valid volume names have the form "a:", "A:", "b:", "B:", etc". That list pretty clearly doesn't intend to allow \\pumpkin\media as a volume. There was a discussion going on between Andreas and John Reppy (the basis specification editor) about the OS.Path structure. I was proposing that you join the discussion and see if the basis specification could be changed to allow vol="\\pumpkin\media" the way your patch does.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

RE: A few changes proposals for MLton

2008-11-17T23:33:25Z

Well, I also use the patch to build the cross-runtimes for solaris, MinGW, Win64, Linux 32 and Linux 64 (for Linux, I also use Linux -> Linux cross compilers in order to produce some code that is independent from the O.S. the build machine is running).

From: Wesley W. Terpstra [mailto:wesley@...]
Sent: Tuesday, November 18, 2008 12:33 AM
To: Nicolas Bertolotti
Cc: Matthew Fluet; mlton@...
Subject: Re: [MLton] A few changes proposals for MLton

On Tue, Nov 18, 2008 at 12:04 AM, Nicolas Bertolotti <Nicolas.Bertolotti@...>

Well, I actually use the changes to rebuild a 32 bit version of the compiler (to use less RAM) after building it natively on a 64 bit platform.

Just run "i386 make"... The latest svn should build you a 32 bit MLton on your 64 bit machine. No patches or cross-compiling needed.

If you have debian, make sure you have gcc-multilib and installed. You might need a few other lib32 packages as well.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

Re: A few changes proposals for MLton

2008-11-17T15:44:37Z

On Mon, Nov 17, 2008 at 11:49 PM, Nicolas Bertolotti <Nicolas.Bertolotti@...> wrote:

I am sorry but I don't really understand what you mean with « You should bring up having UNC volumes as that is currently forbidden by the definition » (maybe it is just a problem with my English understanding).

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

Re: A few changes proposals for MLton

2008-11-17T15:32:49Z

On Tue, Nov 18, 2008 at 12:04 AM, Nicolas Bertolotti <Nicolas.Bertolotti@...>

Well, I actually use the changes to rebuild a 32 bit version of the compiler (to use less RAM) after building it natively on a 64 bit platform.

Just run "i386 make"... The latest svn should build you a 32 bit MLton on your 64 bit machine. No patches or cross-compiling needed.

If you have debian, make sure you have gcc-multilib and installed. You might need a few other lib32 packages as well.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

RE: A few changes proposals for MLton

2008-11-17T15:04:37Z

> It isn't clear to me why upgrade-basis needs to know the target. In any

upgrade-basis generates the MLton structure with a value for Platform.host and OS.host that is based on the target so I guess this has an impact on those values when they are used in the compiler binary itself.

Anyway, before I implemented those changes, I remember my cross compiler already worked fine so I am not really sure this is really important.

> case, your changes don't seem to completely support cross compiling the
> compiler. For example, in <src>/Makefile, in the 'world-no-check' target,
> you don't execute the compiler on the target system, which would be
> necessary after a
> make TARGET=xx TARGET_ARCH=xx TARGET_OS=xx TARGET_MACHINE=xx compiler
> to have both build/lib/mlton-compile and build/lib/world.mlton for the
> target.

Well, I actually use the changes to rebuild a 32 bit version of the compiler (to use less RAM) after building it natively on a 64 bit platform. I do this by running:
$ touch mlton/main.sml
$ make compiler TARGET=... ...
and finally:
$ make world-no-check TARGET=... ... (to rebuild the main compiler binary only)
or:
$ make world libraries tools TARGET=... ... (to rebuild the tools as well)
After running:
$ make all-no-docs
But I do this using an external script so I guess I should add a bit of additional packaging (Makefile modifications) in order to propose an efficient complete cross compilation entry.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

RE: A few changes proposals for MLton

2008-11-17T14:49:25Z

Let's wait to hear what happens in the sml-basis thread. You should bring up having UNC volumes as that is currently forbidden by the definition.

I am sorry but I don’t really understand what you mean with « You should bring up having UNC volumes as that is currently forbidden by the definition » (maybe it is just a problem with my English understanding).

* 05-windows-exit.patch
* This patch adds support for Posix.Process.exit() on MinGW. I
guess it has been deactivated on purpose but I don't know why and I
have not experienced any issue with it.

To be honest, I have no idea why it's disabled either. I was even more mystified when I found that it was added in r3662 which was supposedly a commit on my behalf. I tracked down the email that included the patch I sent. <http://terpstra.ca/lurker/message/20041202.044539.0cb22ee4.en.html> ... but the changes to stub-mingw.sml are nowhere to be seen.

Now would be a good time for Stephen to speak up about where this came from! :-)

I don't really see any reason NOT to re-enable it, but as neither of us know why it was disabled in the first place... :-/

Funny !!! Wait and see…

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

RE: A few changes proposals for MLton

2008-11-17T14:46:51Z

Fine for me (I just didn't know we had this function available).

I am also not really sure that the proposed operation:
> + maxSize = alignDown(SIZE_MAX, s->sysvals.pageSize);
> + maxSize -= sizeofCardMapAndCrossMap(s, maxSize);
really returns the exact maximum size for a heap. It is not really an issue but you may think of a better way.

It could also be slightly more efficient to compute this value only once at the beginning of the execution.

Nicolas

> -----Original Message-----
> From: Matthew Fluet [mailto:fluet@...]
> Sent: Monday, November 17, 2008 8:21 PM
> To: Nicolas Bertolotti
> Cc: mlton@...
> Subject: Re: [MLton] A few changes proposals for MLton
>
> On Thu, 13 Nov 2008, Nicolas Bertolotti wrote:
> > * 21-prevent-null-heap-size.patch
> > * This patch addresses the issue I submitted a few days ago
> > (Segmentation fault on a 6 GB machine with ram-slop 0.85)
>
> --- mlton-r6698.patched/runtime/gc/heap.c 2008-11-03
> 23:45:47.000000000 +0100
> +++ mlton-r6698/runtime/gc/heap.c 2008-11-05 23:53:54.000000000
> +0100
> @@ -39,6 +39,7 @@
> size_t sizeofHeapDesired (GC_state s, size_t liveSize, size_t
> currentSize) {
> size_t liveWithMapsSize;
> size_t res;
> + size_t maxSize;
> float withMapsRatio;
>
> liveSize = align (liveSize, s->sysvals.pageSize);
> @@ -82,6 +83,9 @@
> res = s->sysvals.ram;
> } else { /* Required live ratio. */
> res = liveSize * s->controls.ratios.markCompact;
> + /* Avoid overflow */
> + if (res < liveSize)
> + res = SIZE_MAX;
> /* If the current heap is bigger than res, then shrinking always
> * sounds like a good idea. However, depending on what pages the
> * VM keeps around, growing could be very expensive, if it
> @@ -104,6 +108,16 @@
> die ("Out of memory with max heap size %s.",
> uintmaxToCommaString(s->controls.maxHeap));
> }
> +
> + /* Avoid overflow. */
> + maxSize = align(SIZE_MAX - s->sysvals.pageSize, s->sysvals.pageSize);
> + maxSize -= sizeofCardMapAndCrossMap(s, maxSize);
> + if (res > maxSize)
> + res = maxSize;
> + if (res < liveSize)
> + die ("Out of memory with system max heap size %s.",
> + uintmaxToCommaString(res));
> +
> if (DEBUG_RESIZING)
> fprintf (stderr, "%s = sizeofHeapDesired (%s, %s)\n",
> uintmaxToCommaString(res),
>
>
> Looks simple and effective. Equivalent, but perhaps easier to understand
> would be to replace
>
> + maxSize = align(SIZE_MAX - s->sysvals.pageSize, s->sysvals.pageSize);
>
> with
>
> + maxSize = alignDown(SIZE_MAX, s->sysvals.pageSize);

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

Re: A few changes proposals for MLton

2008-11-17T11:21:21Z

On Thu, 13 Nov 2008, Nicolas Bertolotti wrote:
> * 21-prevent-null-heap-size.patch
> * This patch addresses the issue I submitted a few days ago
> (Segmentation fault on a 6 GB machine with ram-slop 0.85)

--- mlton-r6698.patched/runtime/gc/heap.c 2008-11-03 23:45:47.000000000 +0100
+++ mlton-r6698/runtime/gc/heap.c 2008-11-05 23:53:54.000000000 +0100
@@ -39,6 +39,7 @@
size_t sizeofHeapDesired (GC_state s, size_t liveSize, size_t currentSize) {
size_t liveWithMapsSize;
size_t res;
+ size_t maxSize;
float withMapsRatio;

liveSize = align (liveSize, s->sysvals.pageSize);
@@ -82,6 +83,9 @@
res = s->sysvals.ram;
} else { /* Required live ratio. */
res = liveSize * s->controls.ratios.markCompact;
+ /* Avoid overflow */
+ if (res < liveSize)
+ res = SIZE_MAX;
/* If the current heap is bigger than res, then shrinking always
* sounds like a good idea. However, depending on what pages the
* VM keeps around, growing could be very expensive, if it
@@ -104,6 +108,16 @@
die ("Out of memory with max heap size %s.",
uintmaxToCommaString(s->controls.maxHeap));
}
+
+ /* Avoid overflow. */
+ maxSize = align(SIZE_MAX - s->sysvals.pageSize, s->sysvals.pageSize);
+ maxSize -= sizeofCardMapAndCrossMap(s, maxSize);
+ if (res > maxSize)
+ res = maxSize;
+ if (res < liveSize)
+ die ("Out of memory with system max heap size %s.",
+ uintmaxToCommaString(res));
+
if (DEBUG_RESIZING)
fprintf (stderr, "%s = sizeofHeapDesired (%s, %s)\n",
uintmaxToCommaString(res),

Looks simple and effective. Equivalent, but perhaps easier to understand
would be to replace

+ maxSize = align(SIZE_MAX - s->sysvals.pageSize, s->sysvals.pageSize);

with

+ maxSize = alignDown(SIZE_MAX, s->sysvals.pageSize);

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

Re: Re: [MLton-user] SVN r6941 MLton/MinGW32 and FFI

2008-11-17T07:00:08Z

On Mon, Nov 17, 2008 at 3:51 PM, Vesa Karvonen <vesa.a.j.k@...> wrote:

>> Looks like the origin was to get mlnlffi working on MinGW:
>> http://mlton.org/pipermail/mlton/2006-November/029349.html

Yes, that is why I initially wrote them. It is a minimal
implementation and doesn't necessarily work as an absolute complete
replacement for dl* functions.

Ok, so we can remove the MLton ones? I think they are broken atm anyway since they get prefix with MLton_ by the mingw.h.

> Using dlfcn-win32 does seem the best approach. I'd like to hear
> something from Vesa before removing them.

Frankly, after a very brief look at the code of dlfcn-win32
(http://code.google.com/p/dlfcn-win32/source/browse/trunk/dlfcn.c),
I'm not very impressed with it. I may be mistaken and it might be
mostly harmless, but it seems to leak library handles when RTLD_GLOBAL
is specified. Specifically, the code seems to first execute
LoadLibraryEx, which increments the reference count of the library,
then if RTLD_GLOBAL is specified, the library handle is added to the
doubly linked list (#1) --- except when the library was already
loaded, in which case no matching call to FreeLibrary seems to be
made, which means that the handle was leaked.

But, it is probably better to use it anyway (at least after all the
obvious bugs have been fixed), because then we don't have to maintain
a similar library.

Since you are obviously the resident expert, maybe help them it? I was thinking we could just include their (fixed) library in the MLton mingw installer. That way things will just work for users by adding -ldl, but otherwise we don't throw these symbols into your program. This seems the ideal solution to me since as DLL the wiki _import libdl examples will work unchanged with their default external scope.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

Re: Re: [MLton-user] SVN r6941 MLton/MinGW32 and FFI

2008-11-17T06:51:53Z

On Mon, Nov 17, 2008 at 1:42 PM, Wesley W. Terpstra <wesley@...> wrote:

> Never heard back from you about this ...
>
> ---------- Forwarded message ----------
> From: Wesley W. Terpstra <wesley@...>
> Date: Wed, Nov 12, 2008 at 12:56 AM
> Subject: Re: [MLton] Re: [MLton-user] SVN r6941 MLton/MinGW32 and FFI
> To: Matthew Fluet <fluet@...>
> Cc: "mlton@..." <mlton@...>
>
>
> On Wed, Nov 12, 2008 at 12:26 AM, Matthew Fluet <fluet@...> wrote:
>>> So why were the dl* functions added to mingw.c in the first place?
>> Looks like the origin was to get mlnlffi working on MinGW:
>> http://mlton.org/pipermail/mlton/2006-November/029349.html

Yes, that is why I initially wrote them. It is a minimal
implementation and doesn't necessarily work as an absolute complete
replacement for dl* functions.

> Using dlfcn-win32 does seem the best approach. I'd like to hear
> something from Vesa before removing them.

Frankly, after a very brief look at the code of dlfcn-win32
(http://code.google.com/p/dlfcn-win32/source/browse/trunk/dlfcn.c),
I'm not very impressed with it. I may be mistaken and it might be
mostly harmless, but it seems to leak library handles when RTLD_GLOBAL
is specified. Specifically, the code seems to first execute
LoadLibraryEx, which increments the reference count of the library,
then if RTLD_GLOBAL is specified, the library handle is added to the
doubly linked list (#1) --- except when the library was already
loaded, in which case no matching call to FreeLibrary seems to be
made, which means that the handle was leaked.

But, it is probably better to use it anyway (at least after all the
obvious bugs have been fixed), because then we don't have to maintain
a similar library.

-Vesa Karvonen

#1) I might add here that a singly linked list would work just as well.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

Re: A few changes proposals for MLton

2008-11-14T17:25:57Z

On Sat, Nov 15, 2008 at 12:58 AM, Matthew Fluet <fluet@...> wrote:

On Thu, 13 Nov 2008, Nicolas Bertolotti wrote:

* 03-windows-versions.patch

* This patch extends the setSysname() function that is used in
order to generate the "uname" string on MinGW. It makes it possible
to detect whether we are running Windows XP x64, Vista, Vista 64,
2003 or 2008.

Clearly an improvement. Should be committed.

* 04-windows-paths.patch
* This patch is highly related to the request Dan DuVarney has
submitted today about OS.Path. It provides support for UNC paths
(\\foo\bar\...) and handles paths like "\foo" as absolute paths
without a volume specification.

Let's wait to hear what happens in the sml-basis thread. You should bring up having UNC volumes as that is currently forbidden by the definition.

* 05-windows-exit.patch
* This patch adds support for Posix.Process.exit() on MinGW. I
guess it has been deactivated on purpose but I don't know why and I
have not experienced any issue with it.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

Re: A few changes proposals for MLton

2008-11-14T16:02:34Z

On Thu, 13 Nov 2008, Nicolas Bertolotti wrote:
> * 01-cross-compile-mlton.patch
> * This patch modifies the MLton Makefiles in order to make it
> easier to build a MLton cross compiler by cross compiling the
> runtime and only use the target machine to run the needed binaries
> (gen-sizes, print-constants etc...). It introduces a Makefile
> parameter TARGET_MACHINE that can be used in order to provide the
> hostname of the machine that should be used to run those binaries.
> The machine is then accessed through "ssh".

I don't have any experience with using cross compilers, so if it works for
you and doesn't impact things when non cross compiling, then it is
probably fine.

Two notes:

diff -Naur mlton-r6658.orig/runtime/Makefile mlton-r6658/runtime/Makefile
--- mlton-r6658.orig/runtime/Makefile 2008-06-16 16:03:08.000000000 +0200
+++ mlton-r6658/runtime/Makefile 2008-06-16 16:42:12.000000000 +0200
@@ -285,7 +296,7 @@
gen/basis-ffi.h gen/basis-ffi.sml: gen/gen-basis-ffi.sml gen/basis-ffi.def
mlton -output gen/gen-basis-ffi gen/gen-basis-ffi.sml
rm -f gen/basis-ffi.h gen/basis-ffi.sml
- cd gen && ./gen-basis-ffi
+ cd gen && $(TARGET_EXEC_BEGIN) ./gen-basis-ffi $(TARGET_EXEC_FILE)
rm -f gen/gen-basis-ffi

basis-ffi.h: gen/basis-ffi.h

This is (a) broken and (b) unecessary. This will have compiled
gen-basis-ffi.sml using the host mlton with no -target option, so the
resulting gen-basis-ffi executable will be for the host, but it is being
executed on the target machine. Also, gen-basis-ffi reads directly from
basis-ffi.def and writes directly to basis-ffi.h and basis-ffi.sml, which
aren't being copied to/from the target machine. You probably never
encountered this because the derived basis-ffi.{h,sml} files are in the
repository. In any case, the gen-basis-ffi program simply textually
processes basis-ffi.def, with no target dependencies. Hence, there is no
need to execute it on the target machine.

diff -Naur mlton-r6658.orig/bin/upgrade-basis mlton-r6658/bin/upgrade-basis
--- mlton-r6658.orig/bin/upgrade-basis 2008-06-16 16:08:02.000000000 +0200
+++ mlton-r6658/bin/upgrade-basis 2008-06-16 16:40:47.000000000 +0200
@@ -11,14 +11,15 @@
name=`basename "$0"`

usage () {
- die "usage: $name <PATH> <ARCH> <OS>"
+ die "usage: $name <PATH> <ARCH> <OS> <TARGET>"
}

case "$#" in
-3)
+4)
PATH="$1"
ARCH="$2"
OS="$3"
+ TARGET="$4"
;;
*)
usage
@@ -28,7 +29,7 @@
tmp="$$.sml"

echo "val () = print \"I work\"" >"$tmp"
-if ! mlton "$tmp" 1>&2; then
+if ! mlton -target $TARGET "$tmp" 1>&2; then
die "Error: cannot upgrade basis because the compiler doesn't work"
fi

@@ -36,7 +37,7 @@
feature="$1"
sml="$2"
echo "$feature" >"$tmp"
- if ! mlton -stop tc "$tmp" >/dev/null 2>&1; then
+ if ! mlton -target $TARGET -stop tc "$tmp" >/dev/null 2>&1; then
echo "$sml"
fi
}
diff -Naur mlton-r6658.orig/mlton/Makefile mlton-r6658/mlton/Makefile
--- mlton-r6658.orig/mlton/Makefile 2008-06-16 16:04:17.000000000 +0200
+++ mlton-r6658/mlton/Makefile 2008-06-16 16:40:47.000000000 +0200
@@ -9,8 +9,8 @@
SRC := $(shell cd .. && pwd)
BUILD := $(SRC)/build
BIN := $(BUILD)/bin
-HOST_ARCH := $(shell "$(SRC)/bin/host-arch")
-HOST_OS := $(shell "$(SRC)/bin/host-os")
+TARGET_ARCH := $(shell "$(SRC)/bin/host-arch")
+TARGET_OS := $(shell "$(SRC)/bin/host-os")
LIB := $(BUILD)/lib
MLTON := mlton
TARGET := self
@@ -27,12 +27,12 @@
FLAGS += -default-ann 'warnUnused true'
# FLAGS += -type-check true -show-types true
else
-ifeq (cygwin, $(HOST_OS))
+ifeq (cygwin, $(TARGET_OS))
# The stubs don't work on Cygwin, since they define spawn in terms of
# fork, and fork doesn't work on Cygwin. So, make without the stubs.
FILE := mlton.cm
else
-ifeq (mingw, $(HOST_OS))
+ifeq (mingw, $(TARGET_OS))
# Ditto for MinGW.
FILE := mlton.cm
else
@@ -87,7 +87,7 @@
#! Pass $(PATH) to upgrade-basis because it is run via #!/usr/bin/env
# bash, which resets the path.
$(UP):
- "$(SRC)/bin/upgrade-basis" '$(PATH)' "$(HOST_ARCH)" "$(HOST_OS)" >$(UP)
+ "$(SRC)/bin/upgrade-basis" '$(PATH)' "$(TARGET_ARCH)" "$(TARGET_OS)" "$(TARGET)" >$(UP)

mlton.sml: $(SOURCES)
rm -f mlton.sml && mlton -stop sml mlton.cm && chmod -w mlton.sml

It isn't clear to me why upgrade-basis needs to know the target. In any
case, your changes don't seem to completely support cross compiling the
compiler. For example, in <src>/Makefile, in the 'world-no-check' target,
you don't execute the compiler on the target system, which would be
necessary after a
make TARGET=xx TARGET_ARCH=xx TARGET_OS=xx TARGET_MACHINE=xx compiler
to have both build/lib/mlton-compile and build/lib/world.mlton for the
target.

It seems to me that your changes support the following workflow:

# start from a clean slate
make clean

# bootstrap; resulting compiler only supports -target self
make

# cross compile the runtime
## first, get self objects/archives out of the way
make -C runtime clean
make TARGET=xx TARGET_ARCH=xx TARGET_OS=xx TARGET_MACHINE=xx runtime

# there are new target specific files in basis-library/config/c
## this is just a target agnostic copy operation
make basis-no-check

# get the new target constants
make TARGET=xx TARGET_ARCH=xx TARGET_OS=xx TARGET_MACHINE=xx constants

# register the new target
make TARGET=xx TARGET_ARCH=xx TARGET_OS=xx TARGET_MACHINE=xx targetmap

Now, "build/bin/mlton -target new" works, but is a self executable. It
wouldn't be wise to do
make TARGET=xx TARGET_ARCH=xx TARGET_OS=xx TARGET_MACHINE=xx compiler
because that would overwrite build/lib/mlton-compile with a target
executable.

So, despite the existence of the TARGET variable in the mlton/Makefile,
I'm not sure how easy/useful it is to produce a cross-compiled
mlton-compile from within the Makefile system. But, I could be wrong.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

Re: A few changes proposals for MLton

2008-11-14T15:58:00Z

On Thu, 13 Nov 2008, Nicolas Bertolotti wrote:

> * 03-windows-versions.patch
> * This patch extends the setSysname() function that is used in
> order to generate the "uname" string on MinGW. It makes it possible
> to detect whether we are running Windows XP x64, Vista, Vista 64,
> 2003 or 2008.
>
>
> * 04-windows-paths.patch
> * This patch is highly related to the request Dan DuVarney has
> submitted today about OS.Path. It provides support for UNC paths
> (\\foo\bar\...) and handles paths like "\foo" as absolute paths
> without a volume specification.
>
>
> * 05-windows-exit.patch
> * This patch adds support for Posix.Process.exit() on MinGW. I
> guess it has been deactivated on purpose but I don't know why and I
> have not experienced any issue with it.

I have little to nothing to say with regards to Windows; it seems as
though there is a chance to make some global decisions regarding OS.Path
and Windows on the other thread. Please weigh in on the discussion, as it
is clear that you have some practical experiences that motivate changes.

I might be wary of extending the standard to include UNC paths, but it
does seem as though one can reasonably take UNC paths to denote paths with
volumes of the form "\\\\Server\\SharedFolder" (SML-string syntax). It
does, however, appear as though your patch allows either of the Windows
separators in UNC volume strings. I don't know if Windows interprets
"//Server/SharedFolder/foo" as a UNC path.

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton

Re: A few changes proposals for MLton

2008-11-14T15:57:11Z

On Thu, 13 Nov 2008, Nicolas Bertolotti wrote:
> * 18-fix-big-heap-disk-dump.patch
> * This patch fixes a crash I could observe when a very large heap
> is paged to disk. It seems that in some cases, fwrite() fails to
> dump the whole heap at once and we have to split.

I encountered this as well on x86-linux; an
fwrite (data, size, count, file) with size * count >= 2G immediately
returns 0 without writing any data and without setting ferror or errno.
(I.e., there is no indication as to why the operation failed.)
This doesn't happen on x86-darwin or on amd64-linux (though, I didn't try
size * count >= 2^63 (which ought to fail anyways, as the memory system
and file system can't take it)). Nor is there a corresponding problem
with fread. This is rather annoying, as fwrite is supposed to be the
abstraction that removes the need for looping writes.

With regards to the patch:

--- mlton-r6698.patched/runtime/util/safe.h 2008-09-09 10:54:08.000000000 +0200
+++ mlton-r6698/runtime/util/safe.h 2008-08-13 15:07:57.000000000 +0200
@@ -68,8 +68,21 @@
static inline void fwrite_safe (const void *buf, size_t size, size_t count,
FILE *f) {
size_t res;
-
- res = fwrite (buf, size, count, f);
+ enum {
+ WRITE_CHUNK_SIZE = 0x2000000, /* 32M */
+ };
+
+ res = 0;
+ while (res < count) {
+ size_t n;
+ n = count - res;
+ if (n > WRITE_CHUNK_SIZE)
+ n = WRITE_CHUNK_SIZE;
+ n = fwrite ((void *) ((char *) buf + res), size, n, f);
+ if (n == 0)
+ break;
+ res += n;
+ }
if (res != count)
diee ("fwrite (_, %"PRIuMAX", %"PRIuMAX", _) failed "
"(only wrote %"PRIuMAX").\n",

This kind of misses the point of the *_safe functions --- they are
supposed to signal errors, not mask them. The specification of fwrite
indicates that any short object count return value is an error. Also, the
code above only handles a large count, not a large size. So, it won't
help if one makes the seemingly meaning preserving change (in
diskBack.unix.c) from:
fwrite_safe (buf, 1, size, f);
to
fwrite_safe (buf, size, 1, f);

I would isolate the extreme call to fwrite_safe as in this alternative
patch. Although fread doesn't fail on large reads, it seems prudent to
limit extreme I/O; also, it makes the program more responsive to ctrl-C
(as control frequently returns to user-mode).
diff --git a/runtime/platform/diskBack.unix.c b/runtime/platform/diskBack.unix.c
index f9df82d..95bcea0 100644
--- a/runtime/platform/diskBack.unix.c
+++ b/runtime/platform/diskBack.unix.c
@@ -34,9 +34,21 @@ typedef struct {
void GC_diskBack_read (void *data, pointer buf, size_t size) {
FILE *f;

+ const size_t READ_CHUNK_SIZE = 0x2000000; /* 32M */
+
f = ((WriteToDiskData)data)->f;
fseek_safe (f, 0, SEEK_SET);
- fread_safe (buf, 1, size, f);
+ /* fread (_, 1, size, _) succeeds
+ * with size >= 2^31
+ * for a 32-bit executable on 64-bit linux.
+ * Nonetheless, match GC_diskBack_write.
+ */
+ while (size > 0) {
+ size_t s = min (READ_CHUNK_SIZE, size);
+ fread_safe (buf, 1, s, f);
+ buf += s;
+ size -= s;
+ }
}

void GC_diskBack_close (void *data) {
@@ -51,8 +63,20 @@ void *GC_diskBack_write (pointer buf, size_t size) {
FILE *f;
WriteToDiskData d;

+ const size_t WRITE_CHUNK_SIZE = 0x2000000; /* 32M */
+
f = tempFileDes ();
- fwrite_safe (buf, 1, size, f);
+ /* fwrite (_, 1, size, _) fails
+ * (with no helpful error conditions!)
+ * with size >= 2^31
+ * on x86-linux.
+ */
+ while (size > 0) {
+ size_t s = min (WRITE_CHUNK_SIZE, size);
+ fwrite_safe (buf, 1, s, f);
+ buf += s;
+ size -= s;
+ }
d = (WriteToDiskData)(malloc_safe (sizeof(*d)));
d->f = f;
return d;

_______________________________________________
MLton mailing list
MLton@...
http://mlton.org/mailman/listinfo/mlton