Feature request: MLton.Finalizable

[John Reppy-2]

I'm using finalization as a backstop for explicit management
of resources (e.g., textures and shaders).  I typically use the type

        Int32.int option ref MLton.Finalizable.t

to represent these resources and set the ref to NONE when it
has been explicitly deallocated.  I could streamline this
process with the addition of the following two operations to
the Finalizable structure:

       (* remove all finalization functions from the object *)
        val cancelFinalizers : 'a t -> unit

       (* change the state of the finalized object *)
        val update ('a t * 'a) -> unit

The implementation is straightforward:

        fun cancelFinalizers (T{finalizers, ...}) = finalizers := []

        fun update (T{value, ...}, x) = value := x

Any chance of getting these added to MLton?

        - John


_______________________________________________
MLton-user mailing list
MLton-user@...
http://mlton.org/mailman/listinfo/mlton-user

[Vesa Karvonen-2]

On Mon, Jun 2, 2008 at 5:29 PM, John Reppy <jhreppy@...> wrote:
> I'm using finalization as a backstop for explicit management
> of resources (e.g., textures and shaders).

Is this for the sml3d library/project (at
http://smlnj-gforge.cs.uchicago.edu/projects/sml3d/)?

At any rate, is there some reason why checkout on that project has (at
some point) been restricted to developers?  (I know it was previously
open for public checkout as I tried some of the examples.)  I've
recently been thinking about doing a little 3D graphics programming in
SML.  I wrote a very simple stripifying algorithm (for static
geometry) last week and now I've been looking at how to best interface
with Open GL vertex buffer objects in SML.  (From about 1997 to 2002 I
worked at Housemarque (http://www.housemarque.com), which is an
independent games developer.)  If nothing else, I would be interested
in looking at the library and might have some feedback.

> I could streamline this
> process with the addition of the following two operations to
> the Finalizable structure:
>
>      (* remove all finalization functions from the object *)
>        val cancelFinalizers : 'a t -> unit
>      (* change the state of the finalized object *)
>        val update ('a t * 'a) -> unit
>
> The implementation is straightforward:
>
>        fun cancelFinalizers (T{finalizers, ...}) = finalizers := []
>        fun update (T{value, ...}, x) = value := x
>
> Any chance of getting these added to MLton?

These seem reasonable and useful to me, but I can't immediately say
whether the above implementations might have some subtle semantic
oddities.  For example, would it be possible that when update is given
the only reference to the finalizable object, finalizers might
actually be run before the assignment takes place.  So, for example,
the following code

  fun delete resource =
     (withValue (resource, fn SOME r => reallyDelete r)
    ; update (resource, NONE))

would be incorrect and could cause reallyDelete to be called twice for
the same resource (once in delete and again in the finalizer).

-Vesa Karvonen

_______________________________________________
MLton-user mailing list
MLton-user@...
http://mlton.org/mailman/listinfo/mlton-user

[John Reppy-2]

On Jun 2, 2008, at 12:29 PM, Vesa Karvonen wrote:

> On Mon, Jun 2, 2008 at 5:29 PM, John Reppy <jhreppy@...> wrote:
>> I'm using finalization as a backstop for explicit management
>> of resources (e.g., textures and shaders).
>
> Is this for the sml3d library/project (at
> http://smlnj-gforge.cs.uchicago.edu/projects/sml3d/)?

Yes.

>
>
> At any rate, is there some reason why checkout on that project has (at
> some point) been restricted to developers?  (I know it was previously
> open for public checkout as I tried some of the examples.)  I've
> recently been thinking about doing a little 3D graphics programming in
> SML.  I wrote a very simple stripifying algorithm (for static
> geometry) last week and now I've been looking at how to best interface
> with Open GL vertex buffer objects in SML.  (From about 1997 to 2002 I
> worked at Housemarque (http://www.housemarque.com), which is an
> independent games developer.)  If nothing else, I would be interested
> in looking at the library and might have some feedback.

When we upgraded the version of gforge that we use, the way that  
anonymous
access was handled changed, which is probably why you can no longer  
access
it.  There is an awful lot of partially finished stuff, which is why  
I've
held off announcing it, but I've enabled anonymous access if you want to
take a look at the source.  Note that it requires the latest version  
of MLton
from svn to compile.

>
>
>> I could streamline this
>> process with the addition of the following two operations to
>> the Finalizable structure:
>>
>>     (* remove all finalization functions from the object *)
>>       val cancelFinalizers : 'a t -> unit
>>     (* change the state of the finalized object *)
>>       val update ('a t * 'a) -> unit
>>
>> The implementation is straightforward:
>>
>>       fun cancelFinalizers (T{finalizers, ...}) = finalizers := []
>>       fun update (T{value, ...}, x) = value := x
>>
>> Any chance of getting these added to MLton?
>
> These seem reasonable and useful to me, but I can't immediately say
> whether the above implementations might have some subtle semantic
> oddities.  For example, would it be possible that when update is given
> the only reference to the finalizable object, finalizers might
> actually be run before the assignment takes place.  So, for example,
> the following code
>
>  fun delete resource =
>     (withValue (resource, fn SOME r => reallyDelete r)
>    ; update (resource, NONE))
>
> would be incorrect and could cause reallyDelete to be called twice for
> the same resource (once in delete and again in the finalizer).

I don't think that this is a problem because the implementation of  
finalizable
objects creates a weak pointer to "value" to track the reachability of  
the
finalized object, but one could also change update to

        fun update (f as {value, ...}, x) = (value := x; touch f)

        - John


_______________________________________________
MLton-user mailing list
MLton-user@...
http://mlton.org/mailman/listinfo/mlton-user

[Matthew Fluet-3]

On Mon, 2 Jun 2008, John Reppy wrote:
> I'm using finalization as a backstop for explicit management
> of resources (e.g., textures and shaders).  I typically use the type
>
> Int32.int option ref MLton.Finalizable.t
>
> to represent these resources and set the ref to NONE when it
> has been explicitly deallocated.

Any reason not to simply let finalization take care of deallocation?  I
guess if you have limited resources, you might want to proactively
release them.

> I could streamline this
> process with the addition of the following two operations to
> the Finalizable structure:
>
>       (* remove all finalization functions from the object *)
> val cancelFinalizers : 'a t -> unit
>
>       (* change the state of the finalized object *)
> val update ('a t * 'a) -> unit

Why would you need both?

If you can remove the finalization functions, then you won't need to
change the state of the resource, since at an explicit deallocation you
will first cancel the finalization functions and then really delete:

   type res = Int32.int MLton.Finalizable.t

   fun delete res =
       (MLton.Finalizable.cancelFinalizers res
        ; MLton.Finalizable.withValue (res, fn r => reallyDelete r))

If you can update the state of the finalized object, then you won't need
to delete the finalization functions, since at an explicit deallocation
you will really delete the resource and then change its state:

   type res = Int32.int option MLton.Finalizable.t

   fun delete res =
       (MLton.Finalizable.withValue (res, fn SOME r => reallyDelete r
                                           | NONE => raise Fail "Resource gone")
        ; MLton.Finalizable.update (res, NONE))

> The implementation is straightforward:
>
> fun cancelFinalizers (T{finalizers, ...}) = finalizers := []
>
> fun update (T{value, ...}, x) = value := x
>
> Any chance of getting these added to MLton?

Your 'update' function doesn't work with the Finalizable implementation.
The finalization functions are invoked on the original value v used to
construct the finalizable value.  The closure passed off to the GC handler
can't be closed over the reference cell pointed to by the weak pointer,
else the reference cell would always be live.

You can do it with another level of indirection, but in that case you
might as well leave it to the client.


_______________________________________________
MLton-user mailing list
MLton-user@...
http://mlton.org/mailman/listinfo/mlton-user

[John Reppy-2]

On Jun 2, 2008, at 7:05 PM, Matthew Fluet wrote:

> On Mon, 2 Jun 2008, John Reppy wrote:
>> I'm using finalization as a backstop for explicit management
>> of resources (e.g., textures and shaders).  I typically use the type
>>
>> Int32.int option ref MLton.Finalizable.t
>>
>> to represent these resources and set the ref to NONE when it
>> has been explicitly deallocated.
>
> Any reason not to simply let finalization take care of  
> deallocation?  I guess if you have limited resources, you might want  
> to proactively release them.

The lag of waiting for GC could cause performance problems or even  
prevent code from
working.  I don't think that finalization will every be a substitute  
for explicit
management of system resources, but it is a useful failstop.  I may  
even have my
finalizers print a warning message, since failure to deallocate unused  
resources is
a bug.

>
>
>> I could streamline this
>> process with the addition of the following two operations to
>> the Finalizable structure:
>>
>>      (* remove all finalization functions from the object *)
>> val cancelFinalizers : 'a t -> unit
>>
>>      (* change the state of the finalized object *)
>> val update ('a t * 'a) -> unit
>
> Why would you need both?
>

...

Strictly speaking, I don't need either, but the update is more  
important, since we want
to catch attempts to use a resource after it has been deleted.  
Removing the finalizers
means that the finalizer can be a bit simpler, but is less important.  
Unfortunately,
as you point out below, update is not attainable with the current  
implementation of
Finalizable.

>
>
>> The implementation is straightforward:
>>
>> fun cancelFinalizers (T{finalizers, ...}) = finalizers := []
>>
>> fun update (T{value, ...}, x) = value := x
>>
>> Any chance of getting these added to MLton?
>
> Your 'update' function doesn't work with the Finalizable  
> implementation. The finalization functions are invoked on the  
> original value v used to construct the finalizable value.  The  
> closure passed off to the GC handler can't be closed over the  
> reference cell pointed to by the weak pointer, else the reference  
> cell would always be live.

That's a more serious objection, since I'm trying to avoid the extra  
level of indirection :(.
I guess that this could be made to work if finalization was  
implemented as a functor with
the finalization functions as functor arguments.  I guess I'll just  
stick with my current
implementation then.

        - John


_______________________________________________
MLton-user mailing list
MLton-user@...
http://mlton.org/mailman/listinfo/mlton-user

[Florian Weimer]

* Matthew Fluet:

> Why would you need both?
>
> If you can remove the finalization functions, then you won't need to
> change the state of the resource, since at an explicit deallocation
> you will first cancel the finalization functions and then really
> delete:
>
>   type res = Int32.int MLton.Finalizable.t
>
>   fun delete res =
>       (MLton.Finalizable.cancelFinalizers res
>        ; MLton.Finalizable.withValue (res, fn r => reallyDelete r))

This makes it difficult to detect use-after-delete on the ML level, I
think.

_______________________________________________
MLton-user mailing list
MLton-user@...
http://mlton.org/mailman/listinfo/mlton-user

[Matthew Fluet-3]

On Mon, 2 Jun 2008, John Reppy wrote:

> On Jun 2, 2008, at 7:05 PM, Matthew Fluet wrote:
>> On Mon, 2 Jun 2008, John Reppy wrote:
>> > The implementation is straightforward:
>> >
>> >  fun cancelFinalizers (T{finalizers, ...}) = finalizers := []
>> >
>> >  fun update (T{value, ...}, x) = value := x
>> >
>> > Any chance of getting these added to MLton?
>>
>> Your 'update' function doesn't work with the Finalizable implementation.
>> The finalization functions are invoked on the original value v used to
>> construct the finalizable value.  The closure passed off to the GC handler
>> can't be closed over the reference cell pointed to by the weak pointer,
>> else the reference cell would always be live.
>
> That's a more serious objection, since I'm trying to avoid the extra level of
> indirection :(.

MLton can often flatten references into their containing structure.  Since
the ref used by the weak pointer is not updated, it is a good candidate
container.  So, while you may pay the indirection in the source code, the
executable may not.


_______________________________________________
MLton-user mailing list
MLton-user@...
http://mlton.org/mailman/listinfo/mlton-user