How to match EOF token?

[Kris De Volder]

Ok, this seems like a trivial question. But I've searched and searched
the internet with google and looked all over the place and couldn't find
the answer.

How do you deal with EOF token?

What I mean is, how do I make sure my parser looks at the whole file and
not just the beginning part?

I have this trivial grammar file:

--- begin of file ---
Helpers
        digit = ['0'..'9'] ;

Tokens
       
Productions

        goal = expression ;

        expression
                = literal
                ;
--- end of file ---

But the parser accepts any file that starts with a number and simply
ignores any spurious text after that. I want an error!

I found some mailing list messages about EOF token. So I tried

goal = expression EOF ;
goal = expression eof ;
etc.

But no luck there.

None of the example grammars I've looked at mentioned eof either. I've
tried renaming the initial production to "start" "goal" etc. making an
assumption maybe there is some "magic" eof matching being added by a
special "goal" production. But it didn't work.

I noticed that there is an EOF token class in the generated code. But...
what's the mechanism in sablecc that let's me match it?

Kris

PS: I'm using sablecc 3.2.


_______________________________________________
SableCC-Discussion mailing list
SableCC-Discussion@...
http://lists.sablecc.org/listinfo/sablecc-discussion

[Kris De Volder]

OK... I will answer my own question.

The reason it didn't work had nothing to do with sablecc but with some
dumb mistake in how I was calling the parser (or rather... I was
actually not calling it at all, so nothing was getting parsed :).

So, the answer to the question:

"How do I match EOF token?" is

It is matched by sablecc automatically after reducing the
first/start/goal production in the Productions section.

Kris

On Mon, 2008-01-07 at 15:48 -0800, Kris De Volder wrote:

> Ok, this seems like a trivial question. But I've searched and searched
> the internet with google and looked all over the place and couldn't find
> the answer.
>
> How do you deal with EOF token?
>
> What I mean is, how do I make sure my parser looks at the whole file and
> not just the beginning part?
>
> I have this trivial grammar file:
>
> --- begin of file ---
> Helpers
> digit = ['0'..'9'] ;
>
> Tokens
>
> Productions
>
> goal = expression ;
>
> expression
> = literal
> ;
> --- end of file ---
>
> But the parser accepts any file that starts with a number and simply
> ignores any spurious text after that. I want an error!
>
> I found some mailing list messages about EOF token. So I tried
>
> goal = expression EOF ;
> goal = expression eof ;
> etc.
>
> But no luck there.
>
> None of the example grammars I've looked at mentioned eof either. I've
> tried renaming the initial production to "start" "goal" etc. making an
> assumption maybe there is some "magic" eof matching being added by a
> special "goal" production. But it didn't work.
>
> I noticed that there is an EOF token class in the generated code. But...
> what's the mechanism in sablecc that let's me match it?
>
> Kris
>
> PS: I'm using sablecc 3.2.

_______________________________________________
SableCC-Discussion mailing list
SableCC-Discussion@...
http://lists.sablecc.org/listinfo/sablecc-discussion