Prize and Milestone as mutable state idioms

[Tyler Close]

Hi all,

I was recently refactoring the JSON serialization code in the Waterken
Server to make auditing its correctness easier. In doing so, I wrote
two small utility classes, each one to enforce the update pattern the
code was using for one of its variables. For example, I was using a
boolean variable to keep track of whether or not the most recent JSON
value had been fully and successfully output. The update pattern here
was that the variable started out false and then transitioned to
forever being true. Verifying that this property was correctly
implemented previously required examining the full file for mentions
of the boolean variable to check that it was only ever assigned the
value true. The Milestone class encapsulates a boolean variable that
can only be updated in this way. There's a similar story for the Prize
class. I'm thinking both of these classes may be generally useful and
help communicate the programmer's intention for a mutable variable. If
they are, I think I'll upgrade them to being reusable parts of the
ref_send library. So, I'm looking for answers to a few questions:

1. Do the Prize and Milestone classes make the JSON serialization code
more reviewable?
2. Do you suspect these classes would make other algorithms more reviewable?
3. Is the ref_send library the right place to host a collection of
these utility classes for capability programming idioms?

See the attached Joe-E source file for the current JSON serialization code.

--Tyler

[ValueWriter.java]

---

// Copyright 2008 Waterken Inc. under the terms of the MIT X license
// found at http://www.opensource.org/licenses/mit-license.html
package org.waterken.syntax.json;

import java.io.IOException;
import java.io.Writer;
import java.math.BigDecimal;
import java.math.BigInteger;

import org.joe_e.Struct;

/**
 * A JSON writer.
 * <p>
 * A client can only output a syntactically correct JSON value, or leave the
 * {@link ValueWriter} in a {@linkplain #isWritten detectable} error state.
 * </p>
 */
/* package */ final class
ValueWriter extends Struct {
    static protected final String newLine = "\r\n";
    static private   final String tab = "  ";
   
    static protected final class
    Prize<T> {
    private T value;
   
    protected
    Prize(final T value) {
    this.value = value;
    }
   
    protected T
    claim() {
    final T r = value;
    value = null;
    return r;
    }
    }
   
    static protected final class
    Milestone {
    private boolean marked;
   
    protected
    Milestone(final boolean marked) {
    this.marked = marked;
    }
   
    protected boolean
    is() { return marked; }
   
    protected void
    mark() {
    marked = true;
    }
    }
   
    private final String indent; // indentation for this JSON value
    private final Prize<Writer> output;
    private final Milestone written;
   
    protected
    ValueWriter(final String indent, final Writer out) {
        this.indent = indent;
        this.output = new Prize<Writer>(out);
        written = new Milestone(null == out);
    }
   
    /**
     * @return <code>true</code> if a single JSON value was successfully
     *   written, else <code>false</code>
     */
    protected boolean
    isWritten() { return written.is(); }
   
    protected ObjectWriter
    startObject() throws IOException {
        final Writer out = output.claim();
        out.write('{');
        return new ObjectWriter(out);
    }
   
    protected final class
    ObjectWriter {
        static private final String comma = "," + newLine;
       
        private final String inset; // indentation for each member
        private final Writer out;
        private       String prefix; // current member separator prefix
        private       ValueWriter member; // most recent member started
       
        protected
        ObjectWriter(final Writer out) {
            inset = indent + tab;
            this.out = out;
            prefix = newLine;
            member = new ValueWriter(inset, null);
        }
       
        protected void
        close() throws IOException {
            if (!member.isWritten()) { throw new NullPointerException(); }
           
            member = null; // prevent future calls to this object
            out.write(newLine);
            out.write(indent);
            out.write('}');
            written.mark(); // mark the containing value successful
        }
       
        protected ValueWriter
        startMember(final String name) throws IOException {
            if (!member.isWritten()) { throw new NullPointerException(); }
           
            member = new ValueWriter(inset, out); // prevent calls until after
             // new member is complete
            out.write(prefix);
            out.write(inset);
            writeStringTo(name, out);
            out.write(" : ");
            prefix = comma;
            return member;
        }
    }
   
    protected ArrayWriter
    startArray() throws IOException {
        final Writer out = output.claim();
        out.write('[');
        return new ArrayWriter(out);
    }
   
    protected final class
    ArrayWriter {
        static private final String comma = ", ";
       
        private final String inset; // indentation for each element
        private final Writer out;
        private       String prefix; // current element separator prefix
        private       ValueWriter element; // most recent element started
       
        protected
        ArrayWriter(final Writer out) {
            inset = indent + tab;
            this.out = out;
            prefix = " ";
            element = new ValueWriter(inset, null);
        }

        protected void
        close() throws IOException {
            if (!element.isWritten()) { throw new NullPointerException(); }
           
            element = null; // prevent future calls to this object
            out.write(" ]");
            written.mark(); // mark the containing value successful
        }
       
        protected ValueWriter
        startElement() throws IOException {
            if (!element.isWritten()) { throw new NullPointerException(); }
           
            element = new ValueWriter(inset, out); // prevent calls until after
                            // new element is complete
            out.write(prefix);
            prefix = comma;
            return element;
        }
    }
   
    protected void
    writeLink(final String URL) throws IOException {
        final Writer out = output.claim();
        out.write("{ \"@\" : ");
        writeStringTo(URL, out);
        out.write(" }");
        written.mark();
    }
   
    protected void
    writeNull() throws IOException {
        output.claim().write("null");
        written.mark();
    }
   
    protected void
    writeBoolean(final boolean value) throws IOException {
        output.claim().write(value ? "true" : "false");
        written.mark();
    }
   
    protected void
    writeByte(final byte value) throws IOException {
        output.claim().write(Byte.toString(value));
        written.mark();
    }
   
    protected void
    writeShort(final short value) throws IOException {
        output.claim().write(Short.toString(value));
        written.mark();
    }
   
    protected void
    writeInt(final int value) throws IOException {
        output.claim().write(Integer.toString(value));
        written.mark();
    }
   
    protected void
    writeLong(final long value) throws IOException {
        output.claim().write(Long.toString(value));
        written.mark();
    }
   
    protected void
    writeInteger(final BigInteger value) throws IOException {
        output.claim().write(value.toString());
        written.mark();
    }
   
    protected void
    writeFloat(final float value) throws IOException {
        output.claim().write(Float.toString(value));
        written.mark();
    }
   
    protected void
    writeDouble(final double value) throws IOException {
        output.claim().write(Double.toString(value));
        written.mark();
    }
   
    protected void
    writeDecimal(final BigDecimal value) throws IOException {
        output.claim().write(value.toString());
        written.mark();
    }
   
    protected void
    writeString(final String value) throws IOException {
        writeStringTo(value, output.claim());
        written.mark();
    }
   
    static private void
    writeStringTo(final String value, final Writer out) throws IOException {
        out.write('\"');
        final int len = value.length();
        for (int i = 0; i != len; ++i) {
            final char c = value.charAt(i);
            switch (c) {
            case '\"':
                out.write("\\\"");
                break;
            case '\\':
                out.write("\\\\");
                break;
            case '\b':
                out.write("\\b");
                break;
            case '\f':
                out.write("\\f");
                break;
            case '\n':
                out.write("\\n");
                break;
            case '\r':
                out.write("\\r");
                break;
            case '\t':
                out.write("\\t");
                break;
            case ' ':
                out.write(c);
                break;
            default:
                switch (Character.getType(c)) {
                case Character.UPPERCASE_LETTER:
                case Character.LOWERCASE_LETTER:
                case Character.TITLECASE_LETTER:
                case Character.MODIFIER_LETTER:
                case Character.OTHER_LETTER:
                case Character.NON_SPACING_MARK:
                case Character.ENCLOSING_MARK:
                case Character.COMBINING_SPACING_MARK:
                case Character.DECIMAL_DIGIT_NUMBER:
                case Character.LETTER_NUMBER:
                case Character.OTHER_NUMBER:
                case Character.DASH_PUNCTUATION:
                case Character.START_PUNCTUATION:
                case Character.END_PUNCTUATION:
                case Character.CONNECTOR_PUNCTUATION:
                case Character.OTHER_PUNCTUATION:
                case Character.MATH_SYMBOL:
                case Character.CURRENCY_SYMBOL:
                case Character.MODIFIER_SYMBOL:
                case Character.INITIAL_QUOTE_PUNCTUATION:
                case Character.FINAL_QUOTE_PUNCTUATION:
                    out.write(c);
                    break;
                default:
                    out.write("\\u");
                    final int unicode = c;
                    for (int shift = 16; 0 != shift;) {
                        shift -= 4;
                        final int hex = (unicode >> shift) & 0x0F;
                        out.write(hex < 10 ? '0' + hex : 'A' + (hex - 10));
                    }
                }
            }
        }
        out.write('\"');
    }
}

---

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

[Tyler Close]

There are at least 2 errors in the code I posted yesterday. If you
can't find them, the updated code is at:

http://waterken.svn.sourceforge.net/viewvc/waterken/server/trunk/waterken/remote/src/org/waterken/syntax/json/ValueWriter.java?view=markup

--Tyler

On Mon, Jun 30, 2008 at 4:20 PM, Tyler Close <tyler.close@...> wrote:

> Hi all,
>
> I was recently refactoring the JSON serialization code in the Waterken
> Server to make auditing its correctness easier. In doing so, I wrote
> two small utility classes, each one to enforce the update pattern the
> code was using for one of its variables. For example, I was using a
> boolean variable to keep track of whether or not the most recent JSON
> value had been fully and successfully output. The update pattern here
> was that the variable started out false and then transitioned to
> forever being true. Verifying that this property was correctly
> implemented previously required examining the full file for mentions
> of the boolean variable to check that it was only ever assigned the
> value true. The Milestone class encapsulates a boolean variable that
> can only be updated in this way. There's a similar story for the Prize
> class. I'm thinking both of these classes may be generally useful and
> help communicate the programmer's intention for a mutable variable. If
> they are, I think I'll upgrade them to being reusable parts of the
> ref_send library. So, I'm looking for answers to a few questions:
>
> 1. Do the Prize and Milestone classes make the JSON serialization code
> more reviewable?
> 2. Do you suspect these classes would make other algorithms more reviewable?
> 3. Is the ref_send library the right place to host a collection of
> these utility classes for capability programming idioms?
>
> See the attached Joe-E source file for the current JSON serialization code.
>
> --Tyler
>

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

[Monty Zukowski-2]

I really like those idioms of Milestone and Prize, however in the code
for ValueWriter, I'm not clear on why Prize is important to
understanding the code.  I also don't understand:

  116         protected ValueWriter
  117         startMember(final String name) throws IOException {
  118             if (!member.isWritten()) { throw new NullPointerException(); }
  119
  120             member = new ValueWriter(inset, out); // prevent
calls until after
  121

In general I'm having trouble understanding the expected flow through
the code, and after 15 minutes of studying it I'm giving up.

I was going to comment that the below code will throw a
NullPointerException if called twice, but then I noticed
NullPointerExceptions being thrown on purpose elsewhere for
essentially the same reason.  Oh, maybe that's why you use Prize, to
force that NPE instead of possibly writing the output twice?

   81     protected ObjectWriter
   82     startObject() throws IOException {
   83         final Writer out = output.claim();
   84         out.write('{');

Monty
_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

[Tyler Close]

On Tue, Jul 1, 2008 at 11:26 AM, Monty Zukowski <monty@...> wrote:
> I really like those idioms of Milestone and Prize, however in the code
> for ValueWriter, I'm not clear on why Prize is important to
> understanding the code.

...

>
> In general I'm having trouble understanding the expected flow through
> the code, and after 15 minutes of studying it I'm giving up.

The goal of the ValueWriter is to ensure that its client can only
output a syntactically correct JSON value. The ValueWriter provides an
API for generating all the different kinds of JSON values and the
client gets to choose which one to use, but it can only choose one!
The Prize class is used to ensure that once the client starts creating
a particular kind of JSON value, it sticks to that decision;
otherwise, the output stream would be corrupted. Each of the
writeXXX() methods starts by claiming the prize, the output stream,
and then writes to it. If the prize has already been claimed, an
exception is thrown. A correctly coded client will never encounter
this exception, only a buggy or malicious one.

>  I also don't understand:
>
>  116         protected ValueWriter
>  117         startMember(final String name) throws IOException {
>  118             if (!member.isWritten()) { throw new NullPointerException(); }
>  119
>  120             member = new ValueWriter(inset, out); // prevent
> calls until after

The "member" variable acts almost like a Prize, but not quite. A JSON
object can contain an arbitrary number of members. Once a client has
started creating a member, it must finish it, but once it's done, a
new member can be started. Line 120 above is like claiming a newly
created Prize, where the prize is the ability to output a new member.
The check on line 118 ensures that the previously created member is
complete before allowing this new prize to be created and claimed.

> I was going to comment that the below code will throw a
> NullPointerException if called twice, but then I noticed
> NullPointerExceptions being thrown on purpose elsewhere for
> essentially the same reason.  Oh, maybe that's why you use Prize, to
> force that NPE instead of possibly writing the output twice?
>
>   81     protected ObjectWriter
>   82     startObject() throws IOException {
>   83         final Writer out = output.claim();
>   84         out.write('{');

Yes, that's it exactly.

So it looks like you understood the concept of a Prize, and eventually
understood why the output stream was being treated as one. Is that
right? How could this be made clearer? A comment at the Prize
construction site?

Thanks,
--Tyler
_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

[Monty Zukowski-2]

On Tue, Jul 1, 2008 at 1:43 PM, Tyler Close <tyler.close@...> wrote:
> The ValueWriter provides an
> API for generating all the different kinds of JSON values and the
> client gets to choose which one to use, but it can only choose one!
> The Prize class is used to ensure that once the client starts creating
> a particular kind of JSON value, it sticks to that decision

That description above is what I was missing, that concept of only
choosing one JSON Value.  It may be my lack of knowledge about JSON
prevented me from seeing that as obvious.

> So it looks like you understood the concept of a Prize, and eventually
> understood why the output stream was being treated as one. Is that
> right? How could this be made clearer? A comment at the Prize
> construction site?

What would've been even more helpful would be a javadoc comment for
ValueWriter that explained the basic usage--create ValueWriter, call
one and only one output method, NPE thrown if that is violated.

Monty
_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

[Kevin Reid-2]

On Jun 30, 2008, at 19:20, Tyler Close wrote:

> For example, I was using a boolean variable to keep track of  
> whether or not the most recent JSON value had been fully and  
> successfully output. The update pattern here was that the variable  
> started out false and then transitioned to forever being true.  
> Verifying that this property was correctly implemented previously  
> required examining the full file for mentions of the boolean  
> variable to check that it was only ever assigned the value true.  
> The Milestone class encapsulates a boolean variable that can only  
> be updated in this way.

 From the problem description, I immediately think of a promise.

That is (in E),

def [p, r] := Ref.promise()
def yet() { return Ref.isResolved(p) }
def now() { r.resolve(null) }

In this case it's probably silly, as the update pattern is almost  
equally short (and, I would expect, shorter in ref-send/Joe-E),

def [yet, now] := {
   var v := false
   [v.get,
    def now() { v := true }]
}

but promises can be a nice primitive for more complex happens-once  
cases (and require no external encapsulation).

--
Kevin Reid                            <http://homepage.mac.com/kpreid/>


_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

[Mark Miller-2]

On Wed, Jul 2, 2008 at 5:20 PM, Kevin Reid <kpreid@...> wrote:
> def [yet, now] := {
>   var v := false
>   [v.get,
>    def now() { v := true }]
> }


Did you mean

    (&v).get

?


--
Text by me above is hereby placed in the public domain

 Cheers,
 --MarkM
_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

[Kevin Reid-2]

On Jul 2, 2008, at 20:31, Mark Miller wrote:

> On Wed, Jul 2, 2008 at 5:20 PM, Kevin Reid <kpreid@...> wrote:
>> def [yet, now] := {
>>   var v := false
>>   [v.get,
>>    def now() { v := true }]
>> }
>
>
> Did you mean
>
>     (&v).get
>
> ?

Yes.

--
Kevin Reid                            <http://homepage.mac.com/kpreid/>


_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

[Karp, Alan H]

It's a good read, but it's not intended for the JavaScript newbie, as Crock notes.  I'll be changing my coding style to follow his recommendations.  One thing did bother me.  The code examples look so much like E that they looked funny without "def" and "to".

________________________
Alan Karp
Principal Scientist
Virus Safe Computing Initiative
Hewlett-Packard Laboratories
1501 Page Mill Road
Palo Alto, CA 94304
[phone], fax [phone]
http://www.hpl.hp.com/personal/Alan_Karp

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang