re: Generic pickler and MLB files
|
View:
New views
2 Messages
—
Rating Filter:
Alert me
|
 |
re: Generic pickler and MLB files
by Vesa Karvonen-2
::
Rate this Message:
I got an email with some questions on my generics library in Finnish. I
took the liberty of translating the questions to English and answering them here, because the questions and answers may be useful to others. > What is the correct way to take Generic.Pickle to use with MLB-files? > The only combination I've gotten to work looks like this: > > $(MLTON_LIB)/com/ssh/generic/unstable/lib.mlb > $(MLTON_LIB)/com/ssh/generic/unstable/with/generic.sml > $(MLTON_LIB)/com/ssh/generic/unstable/with/type-info.sml > $(MLTON_LIB)/com/ssh/generic/unstable/with/type-hash.sml > $(MLTON_LIB)/com/ssh/generic/unstable/with/hash.sml > $(MLTON_LIB)/com/ssh/generic/unstable/with/uniplate.sml > $(MLTON_LIB)/com/ssh/generic/unstable/with/pretty.sml > $(MLTON_LIB)/com/ssh/generic/unstable/with/eq.sml > $(MLTON_LIB)/com/ssh/generic/unstable/with/some.sml > $(MLTON_LIB)/com/ssh/generic/unstable/with/pickle.sml > $(MLTON_LIB)/com/ssh/generic/unstable/with/seq.sml > $(MLTON_LIB)/com/ssh/generic/unstable/with/read.sml > $(MLTON_LIB)/com/ssh/generic/unstable/with/reduce.sml > $(MLTON_LIB)/com/ssh/generic/unstable/with/transform.sml > $(MLTON_LIB)/com/ssh/generic/unstable/with/fmap.sml > $(MLTON_LIB)/com/ssh/generic/unstable/with/size.sml > $(MLTON_LIB)/com/ssh/generic/unstable/with/ord.sml > $(MLTON_LIB)/com/ssh/generic/unstable/with/shrink.sml > $(MLTON_LIB)/com/ssh/generic/unstable/with/close-pretty-with-extra.sml > $(MLTON_LIB)/com/ssh/generic/unstable/with/reg-basis-exns.sml > > There is probably something extra here. Yes, the above includes most of the generics and also registers a bunch of standard exceptions (reg-basis-exns.sml) and also makes it so that values of the product type are printed in infix notation (close-pretty-with-extra.sml). If one just wants to use just the Pickle generic, then the following should do: (* First we need the generics library itself: *) $(MLTON_LIB)/com/ssh/generic/unstable/lib.mlb (* Then we begin the definition of the Generic module: *) $(MLTON_LIB)/com/ssh/generic/unstable/with/generic.sml (* Then we extends the Generic module with the desired combination: *) $(MLTON_LIB)/com/ssh/generic/unstable/with/type-info.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/type-hash.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/hash.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/eq.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/some.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/pickle.sml (* Then we "close" the combination to make it convenient to use: *) $(MLTON_LIB)/com/ssh/generic/unstable/with/close.sml (* Optionally, we can add support for some standard type constructors (e.g. option, order, tuples, ...): *) $(MLTON_LIB)/com/ssh/generic/unstable/with/extra.sml So, how does one know how to write the required combination as above? A short answer could be: A proper combination is a topological ordering of the graph whose root vertices are the desired generics and whose edges are specified by the WITH_?_DOM signatures of the generics. However, it probably helps to understand a bit about how the generics library works. Each generic, like the Pickle generic, is implemented by a functor, that extends an existing combination of generics. The with -files, one for each generic, like $(MLTON_LIB)/com/ssh/generic/unstable/with/hash.sml http://mlton.org/cgi-bin/viewsvn.cgi/*checkout*/mltonlib/trunk/com/ssh/generic/unstable/with/hash.sml?rev=6071 instantiate the functor (and perform a bit of plumbing). In the case of the Pickle generic, the functor is named WithPickle. Looking at the export file of the generics library http://mlton.org/cgi-bin/viewsvn.cgi/*checkout*/mltonlib/trunk/com/ssh/generic/unstable/public/export.sml?rev=6464 one can see that the WithPickle functor has the "signature" functor WithPickle (Arg : WITH_PICKLE_DOM) : PICKLE_CASES This means that WithPickle takes as an argument a module with the signature WITH_PICKLE_DOM and produces a module with the signature PICKLE_CASES. Looking at the signature file of the Pickle generic http://mlton.org/cgi-bin/viewsvn.cgi/*checkout*/mltonlib/trunk/com/ssh/generic/unstable/public/value/pickle.sig?rev=6389 we can see the WITH_PICKLE_DOM signature signature WITH_PICKLE_DOM = sig include CASES EQ HASH SOME TYPE_HASH TYPE_INFO sharing Open.Rep = EqRep = HashRep = SomeRep = TypeHashRep = TypeInfoRep end >From this one can see that to instantiate the WithPickle functor, one first needs to produce a combination of generics that includes the Eq (EQ), Hash (HASH), Some (SOME), TypeHash (TYPE_HASH), and TypeInfo (TYPE_INFO) generics. Coincidentally, in this case, that is the whole set of generics needed. However, in order to know the topological ordering, one still needs to peek into the WITH_?_DOM signatures of the required generics. For example, looking at http://mlton.org/cgi-bin/viewsvn.cgi/*checkout*/mltonlib/trunk/com/ssh/generic/unstable/public/value/hash.sig?rev=6378 we can see the WITH_HASH_DOM signature signature WITH_HASH_DOM = sig include CASES TYPE_HASH TYPE_INFO sharing Open.Rep = TypeHashRep = TypeInfoRep end and it tells us that TypeHash and TypeInfo must be in the combination before Hash. (Unfortunately, the MLB system is not expressive enough to specify MLB files that would automatically compute a topological ordering (or dependencies) allowing one to just specify which generics are actually wanted (and not their dependencies). So, one need to do it by hand or write a tool to do it. > I tried to use some extras from the with/ directory on top of the > lib-with-default.mlb, but it didn't seem to work. It can be made to work, but one needs to (re)close the combination and observe the topological ordering requirements as discussed above. Looking at lib-with-default.mlb, one can see that it defines the combination as follows: with/generic.sml with/eq.sml with/type-hash.sml with/type-info.sml with/hash.sml with/ord.sml with/pretty.sml with/read.sml with/close-pretty-with-extra.sml Comparing to the previous minimal combination for the Pickle generic, one can see that only the Some generic is missing. So, to extend the default with Pickle, one could write the following in an MLB file: (* First the default combination: *) $(MLTON_LIB)/com/ssh/generic/unstable/lib-with-default.mlb (* Then extend it with Some and then Pickle: *) $(MLTON_LIB)/com/ssh/generic/unstable/with/some.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/pickle.sml (* Finally (re)close the combination for use: *) $(MLTON_LIB)/com/ssh/generic/unstable/with/close-pretty-with-extra.sml Now, one may wonder what is the close-pretty-with-extra.sml file doing. It is, in effect, a convenience for the following frequently desired combination: $(MLTON_LIB)/com/ssh/generic/unstable/with/close.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/extra.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/infix-product.sml Another thing that one might find puzzling is that starting with lib-with-default.mlb and extending it with the Pickle generic, the combination was first started (in lib-with-default.mlb by including with/generic.sml), then closed (in lib-with-default.mlb by including with/close-pretty-with-extra.sml), and then closed again (in our extended combination by including with/close-pretty-with-extra.sml) (without starting a combination). The answer to the puzzle is that closing a combination does not close it in the sense of making it impossible to extend it later. Effectively, closing a combination just produces a set of combinators, specified by the CLOSED_CASES signature, that build type representations having just the closed combination of generics. > It seems that type representations made with lib-with-default.mlb and > with the above combination are incompatible. Is it possible that the > type of type representations changes depending on which "with/" -files > from the generics library have been taken into use? [To clarify, "above combination" refers to the previously mentioned "only combination" of generics that includes almost all the generics.] Yes, they are incompatible and, yes, the type of the type representation depends on which with -files are included and in which order. The functors for generics are generative. That is, each time a functor like WithPickle is instantiated, it creates a new type for the part of the type representation that corresponds just to the specific generic, like the Pickling generic. > This causes problems if one wants to write modular code in such a way > that modules export type representations of their own types. Yes, it is quite true that a weakness of the approach is that the combination must be specified explicitly and this has an effect on program organization. This is mentioned in my paper and in the README file of the generics library. However, it is not difficult to avoid (or solve) the problem although it requires extra linguistic measures (i.e. programming conventions / idioms). The simplest basic approach (although not the only one) to avoiding the problem is to organize the program in such a way that only a single combination of generics is ever produced. The simplest programming convention to make this happen is to make it so that the combination is specified in the context of the application being specified. This means that libraries using generics need to be parameterized with respect to the actual combination of generics, which is only known in the context of a complete application, to use. There are a number of ways to achieve such a parameterization. One way to parameterize libraries with respect to a combination of generics is functorize the libraries and only instantiate them in the final application. For example, the unit-test library is currently functorized. This works, but can be tedious (although programming in fully functorial style definitely has a number advantages; e.g. it helps to ensure that each module can be tested in isolation). Perhaps a more practical technique is to have the parameterization in the build files. With MLton's MLB system, one can use MLB path variables to implement such parameterization. The idea is that, in an MLB file for a library, one refers to the MLB file for the generics via a MLB path variable. For example, a plausible convention would be to use the path variable "GENERICS" to refer to the MLB file that defines the Generic module. In an MLB file, you would then specify $(GENERICS).mlb to get the generics. For example, in an application whose libraries just uses the Pickle generic, the file $(GENERIC).mlb could look like this: $(MLTON_LIB)/com/ssh/generic/unstable/lib.mlb $(MLTON_LIB)/com/ssh/generic/unstable/with/generic.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/type-info.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/type-hash.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/hash.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/eq.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/some.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/pickle.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/close.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/extra.sml When the application is compiled, one must then simply specify the MLB path variable GENERIC using either the -mlb-path-map or the -mlb-path-var (recently added) command line switch. I'll probably delete the lib-with-default.* files and change the libraries to use the convention (using MLB path variable GENERICS) discussed here. > How should one use the pickling generic? I suppose what the real question here is: How one should write a library that uses the Pickle generic? I believe the previous discussion already answers the question, but for concreteness, here is a complete example: <-- application.mlb --- local $(GENERICS).mlb library.mlb in main.sml end --- application.mlb --> <-- main.sml --- local open Generic in val () = Library.doStuff (list int) [1, 2, 3] end --- main.sml --> <-- generics.mlb --- $(MLTON_LIB)/com/ssh/generic/unstable/lib.mlb $(MLTON_LIB)/com/ssh/generic/unstable/with/generic.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/type-info.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/type-hash.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/hash.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/eq.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/some.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/pickle.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/close.sml $(MLTON_LIB)/com/ssh/generic/unstable/with/extra.sml --- generics.mlb --> <-- library.mlb --- local $(MLTON_LIB)/com/ssh/extended-basis/unstable/basis.mlb $(GENERICS).mlb in library.sml end --- library.mlb --> <-- library.sml --- structure Library :> sig val doStuff : ('a, 'x) Generic.Open.Rep.t -> 'a Effect.t end = struct open Cvt Generic fun doStuff t x = (print "Hex bytes:" ; String.app (fn c => prints [" ", P#L#"0"2 (I#x (ord c))]) (Generic.pickle t x) ; print "\n") end --- library.sml --> <-- mlb-path-map --- GENERICS generics MLTON_LIB *** NOTE: path to mltonlib root; you'll have to change this line *** --- mlb-path-map --> To try the above, just create files with the indicated names and contents to a single directory and compile with: mlton -mlb-path-map mlb-path-map application.mlb Running the produced application should produce the following output (at least with the current implementation of the Pickle generic): Hex bytes: 00 03 01 01 01 02 01 03 -Vesa Karvonen _______________________________________________ MLton-user mailing list MLton-user@... http://mlton.org/mailman/listinfo/mlton-user |
|
|
|
| Free Forum Powered by Nabble | Forum Help |