[digester] xml attribute values containing "]" character get scrambled

[Ken Tanaka]

I'm configuring a program with regex patterns to inventory filenames to
a database. I have 8 types of files I wish to deal with, with 8 separate
filename patterns I want to match against to choose an appropriate
action. I'm not sure if I'm doing something wrong or if I've hit a bug.
I'm using digester 1.8, with jdk1.5.0_11 on Red Hat Enterprise Linux
Client release 5.1.

So I'm configuring my program with regular expressions that match parts
of a directory path with "/([^/]+)/" to match one or more characters
other than "/" between directory separators of "/". I encountered some
weirdness in parsing the configuration file. I simplified my more
elaborate regular expressions from this down to a configuration file
containing:

<?xml version="1.0" encoding="UTF-8"?>
<!--     Document   : digester_conf.xml  -->
<toplevel>
    <!--    Set of strings -->
    <!--    If all the strings contain "]", then string seven gets garbled
            by content from string eight. -->
    <stringset
        attr1="/one/([^/]+)"
        attr2="/two/([^/]+)"
        attr3="/three/([^/]+)"
        attr4="/four/([^/]+)"
        attr5="/five/([^/]+)"
        attr6="/six/([^/]+)"
        attr7="/seven/([^/]+)"
        attr8="/eight/([^/]+)"
        attr9="/nine/([^/]+)"
        />
</toplevel>

I print out the stringset values and get (note attr7 doesn't look right):

String set:
  attr1 =/one/([^/]+)
  attr2 =/two/([^/]+)
  attr3 =/three/([^/]+)
  attr4 =/four/([^/]+)
  attr5 =/five/([^/]+)
  attr6 =/six/([^/]+)
  attr7 =/eight/([^/]+)
  attr8 =/eight/([^/]+)
  attr9 =/nine/([^/]+)
  attr10=null

If the configuration file sets attr1...attr9 to strings not containing
the ']' character, then all works as expected. Am I using special
characters that I should I be escaping in the values somehow? Do you see
any errors in my addRules method? Or could this be a bug? I thought at
first there was a limit to the number of attributes allowed, but
variations on the program work with up to 15 stringset attributes, which
is as far as I've tested. But the seventh string containing ']' gets
corrupted by the eighth string containing ']', even with other values
interspersed.

The main Java file contains

public class TryDigester
{  
    /** Creates a new instance of TryDigester */
    public TryDigester() {
    }
   
    public static void main( String[] args )
    {
        System.out.println( "Starting configuration test" );
        if (args.length < 1) {
            System.out.println("Usage: java -jar target/TryDigester.jar
digester_conf.xml");
            System.exit(-1);
        }
       
        TryDigester app = new TryDigester();
       
        //configure loading session
        String configFile = args[0];
        // Create a Digester instance
        Digester d = new Digester();
       
        // Prime the digester stack with an object for rules to
        // operate on.
        ConfigData cfg = new ConfigData();
        d.push(cfg);
       
        // Add rules to the digester that will be triggered while
        // parsing occurs.
        addRules(d);
       
        // Process the input file.
        try {
            java.io.File srcfile = new java.io.File(configFile);
            d.parse(srcfile);
        } catch(java.io.IOException ioe) {
            System.out.println("Error reading input file:" +
ioe.getMessage());
            System.exit(-1);
        } catch(org.xml.sax.SAXException se) {
            System.out.println("Error parsing input file:" +
se.getMessage());
            System.exit(-1);
        }
       
        // For debugging, see the results of configuration file processing.
        cfg.print();
       
        // Do something with configuration data
//        app.doSomething(cfg);
       
        System.out.println("Done");
    } // end main(String[] args)
   
   
    /**
     * The parsing rules for filling in session parameters from the XML
     * configuration file.
     */
    private static void addRules(Digester d) {      
        d.addObjectCreate("toplevel/stringset", StringSet.class);
        d.addSetProperties("toplevel/stringset");
        d.addSetNext("toplevel/stringset", "addStringset");      
    } // addRules(Digester d)


    private void doSomething(ConfigData cfgData) {
        if (cfgData.getStringset() == null) {
            throw new NullPointerException("stringset is null");
        }
        // ============ processing code goes here ==========
    } // doSomething(ConfigData cfgData)
}

ConfigData is an object representing anything of interest from the
configuration file. I contains just a single object StringSet, which is
a databean with 10 String properties (attr1 through attr10). If people
want, I can post the source files (3 small java files, 1 maven2
pom.xml), the first listing above shows the configuration file in its
entirety.

Thanks in advance for any suggestions,
-Ken

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@...
For additional commands, e-mail: user-help@...

[Ken Tanaka]

> I'm using digester 1.8, with jdk1.5.0_11 on Red Hat Enterprise
> Linux
> Client release 5.1.
>
OK, I've tried running the same jar file on my home system and it works
fine there. The same executable jar on MacOS 10.4.11, java version
"1.5.0_13" gives correct output--I didn't change anything, just
downloaded the jar and ran it. I previously had used "maven
assembly:assembly", so I believe that means all the dependent jar files
are included (digester 1.8, etc.), except for system libraries, right?
Does this mean it's more difficult to debug? ;-)

For now I'm just hard-coding the strings in my processing class. I'm
avoiding adding rules to load these as XML configuration sub-elements,
because I'm using a pre-existing framework where I can't add the
RuleSets to do this unless I duplicate that portion of the code; however
attributes are handled (usually) without changing the framework code by
just providing the getters and setters in my bean class.

-Ken

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@...
For additional commands, e-mail: user-help@...

Ken.Tanaka@... schrieb:

>> I'm using digester 1.8, with jdk1.5.0_11 on Red Hat Enterprise
>> Linux
>> Client release 5.1.
>>
>>    
> OK, I've tried running the same jar file on my home system and it works
> fine there. The same executable jar on MacOS 10.4.11, java version
> "1.5.0_13" gives correct output--I didn't change anything, just
> downloaded the jar and ran it. I previously had used "maven
> assembly:assembly", so I believe that means all the dependent jar files
> are included (digester 1.8, etc.), except for system libraries, right?
> Does this mean it's more difficult to debug? ;-)
>
> For now I'm just hard-coding the strings in my processing class. I'm
> avoiding adding rules to load these as XML configuration sub-elements,
> because I'm using a pre-existing framework where I can't add the
> RuleSets to do this unless I duplicate that portion of the code; however
> attributes are handled (usually) without changing the framework code by
> just providing the getters and setters in my bean class.
>  

Can you post your ConfigData class?

You could also try enabling debug logging for category
    org.apache.commons.digester
which will give a lot of info about which methods are called when.

Regards, Simon


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@...
For additional commands, e-mail: user-help@...

[Ken Tanaka]

----- Original Message -----
From: "simon.kitching@..." <simon.kitching@...>
Date: Friday, May 16, 2008 2:44 am
Subject: Re: [digester] xml attribute values containing "]" character
get scrambled

> Ken.Tanaka@... schrieb:
> >> I'm using digester 1.8, with jdk1.5.0_11 on Red Hat Enterprise
> >> Linux
> >> Client release 5.1.
> >>
> >>    
> > OK, I've tried running the same jar file on my home system and it
> works> fine there. The same executable jar on MacOS 10.4.11, java
> version> "1.5.0_13" gives correct output--I didn't change anything,
> just> downloaded the jar and ran it. I previously had used "maven
> > assembly:assembly", so I believe that means all the dependent jar
> files> are included (digester 1.8, etc.), except for system
> libraries, right?
> > Does this mean it's more difficult to debug? ;-)

...

> >  
>
> Can you post your ConfigData class?
>
> You could also try enabling debug logging for category
>    org.apache.commons.digester
> which will give a lot of info about which methods are called when.
>
> Regards, Simon
>

Thanks for the suggestion. I'll try adding logging to the simplified
example. Logging on my working application looked as if the wrong
information was in my XML input, although there was a lot of output to
wade through. Logging on the example code should be much easier to analyze.

Here is my ConfigData class: ============================
/*
 * ConfigData.java
 */

package gov.noaa.eds.byExample.tryDigesterConfigLimit;

public class ConfigData {
    private StringSet stringset = null;

    public StringSet getStringset() {
        return this.stringset;
    }

    public void addStringset(StringSet ss) {
        this.stringset = ss;
    }
   
    public void print() {
        System.out.println("Configuration contents:");
        this.stringset.print();
    }
} // end ConfigData


Here is my StringSet class: ============================
/*
 * StringSet.java
 */

package gov.noaa.eds.byExample.tryDigesterConfigLimit;

public class StringSet {
    private String attr1 = null;
    private String attr2 = null;
    private String attr3 = null;
    private String attr4 = null;
    private String attr5 = null;
    private String attr6 = null;
    private String attr7 = null;
    private String attr8 = null;
    private String attr9 = null;
    private String attr10 = null;

    /**
     * Print the contents of this class. Mainly for debugging.
     */
    public void print() {
        System.out.println("String set:");
        System.out.println("  attr1 =" + this.attr1);
        System.out.println("  attr2 =" + this.attr2);
        System.out.println("  attr3 =" + this.attr3);
        System.out.println("  attr4 =" + this.attr4);
        System.out.println("  attr5 =" + this.attr5);
        System.out.println("  attr6 =" + this.attr6);
        System.out.println("  attr7 =" + this.attr7);
        System.out.println("  attr8 =" + this.attr8);
        System.out.println("  attr9 =" + this.attr9);
        System.out.println("  attr10=" + this.attr10);
    }

    public String getAttr1() {
        return attr1;
    }

    public void setAttr1(String attr1) {
        this.attr1 = attr1;
    }

    public String getAttr2() {
        return attr2;
    }

    public void setAttr2(String attr2) {
        this.attr2 = attr2;
    }

    public String getAttr3() {
        return attr3;
    }

    public void setAttr3(String attr3) {
        this.attr3 = attr3;
    }

    public String getAttr4() {
        return attr4;
    }

    public void setAttr4(String attr4) {
        this.attr4 = attr4;
    }

    public String getAttr5() {
        return attr5;
    }

    public void setAttr5(String attr5) {
        this.attr5 = attr5;
    }

    public String getAttr6() {
        return attr6;
    }

    public void setAttr6(String attr6) {
        this.attr6 = attr6;
    }

    public String getAttr7() {
        return attr7;
    }

    public void setAttr7(String attr7) {
        this.attr7 = attr7;
    }

    public String getAttr8() {
        return attr8;
    }

    public void setAttr8(String attr8) {
        this.attr8 = attr8;
    }

    public String getAttr9() {
        return attr9;
    }

    public void setAttr9(String attr9) {
        this.attr9 = attr9;
    }

    public String getAttr10() {
        return attr10;
    }

    public void setAttr10(String attr10) {
        this.attr10 = attr10;
    }
} // end StringSet



---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@...
For additional commands, e-mail: user-help@...

[Ken Tanaka]

>> Can you post your ConfigData class?
>>
>> You could also try enabling debug logging for category
>>    org.apache.commons.digester
>> which will give a lot of info about which methods are called when.
>>
>> Regards, Simon
>>
>>    
> Thanks for the suggestion. I'll try adding logging to the simplified
> example. Logging on my working application looked as if the wrong
> information was in my XML input, although there was a lot of output to
> wade through. Logging on the example code should be much easier to analyze.

I should clarify, it just looked like the wrong information was in the
XML even though it was correct. Some additional examples:

This digester_conf.xml
  <toplevel>
      <stringset
      attr1="1]"
      attr2="2]"
      attr3="3]"
      attr4="4]"
      attr5="5]"
      attr6="6]"
      attr7="7]seven"
      attr8="8]!"
      attr9="9]"
      attr10="ten"
      />
  </toplevel>

results in output (attr7 has attr8 overlaid):
  String set:
    attr1 =1]
    attr2 =2]
    attr3 =3]
    attr4 =4]
    attr5 =5]
    attr6 =6]
    attr7 =8]!even
    attr8 =8]!
    attr9 =9]
    attr10=ten

and this digester_conf.xml (same as before, but attr10 changed)
  <toplevel>
      <stringset
      attr1="1]"
      attr2="2]"
      attr3="3]"
      attr4="4]"
      attr5="5]"
      attr6="6]"
      attr7="7]seven"
      attr8="8]!"
      attr9="9]"
      attr10="ten]"
      />
  </toplevel>

results in output (attr7 has attr8 overlaid, attr9 has attr10 overlaid):
  String set:
    attr1 =1]
    attr2 =2]
    attr3 =3]
    attr4 =4]
    attr5 =5]
    attr6 =6]
    attr7 =8]!even
    attr8 =8]!
    attr9 =te
    attr10=ten]

Here is the logging output (log4j) for the first digester_conf.xml file.
I'm not sure what to make of this.

-Ken

main INFO  tryDigesterConfigLimit.TryDigester - Beginning TryDigester
session -------------
main DEBUG Digester.sax -
setDocumentLocator(com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser$LocatorProxy@7854a328)
main DEBUG Digester.sax - startDocument()
main DEBUG Digester.sax - startElement(,,toplevel)
main DEBUG digester.Digester -   Pushing body text ''
main DEBUG digester.Digester -   New match='toplevel'
main DEBUG digester.Digester -   No rules found matching 'toplevel'.
main DEBUG Digester.sax - characters(

    )
main DEBUG Digester.sax - characters(
    )
main DEBUG Digester.sax - characters(
    )
main DEBUG Digester.sax - startElement(,,stringset)
main DEBUG digester.Digester -   Pushing body text '



    '
main DEBUG digester.Digester -   New match='toplevel/stringset'
main DEBUG digester.Digester -   Fire begin() for
ObjectCreateRule[className=gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet,
attributeName=null]
main DEBUG digester.Digester - [ObjectCreateRule]{toplevel/stringset}New
gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet
main DEBUG digester.Digester -   Fire begin() for SetPropertiesRule[]
main DEBUG digester.Digester - [SetPropertiesRule]{toplevel/stringset}
Setting property 'attr1' to '1]'
main DEBUG digester.Digester - [SetPropertiesRule]{toplevel/stringset}
Setting property 'attr2' to '2]'
main DEBUG digester.Digester - [SetPropertiesRule]{toplevel/stringset}
Setting property 'attr3' to '3]'
main DEBUG digester.Digester - [SetPropertiesRule]{toplevel/stringset}
Setting property 'attr4' to '4]'
main DEBUG digester.Digester - [SetPropertiesRule]{toplevel/stringset}
Setting property 'attr5' to '5]'
main DEBUG digester.Digester - [SetPropertiesRule]{toplevel/stringset}
Setting property 'attr6' to '6]'
main DEBUG digester.Digester - [SetPropertiesRule]{toplevel/stringset}
Setting property 'attr7' to '8]!even'
main DEBUG digester.Digester - [SetPropertiesRule]{toplevel/stringset}
Setting property 'attr8' to '8]!'
main DEBUG digester.Digester - [SetPropertiesRule]{toplevel/stringset}
Setting property 'attr9' to '9]'
main DEBUG digester.Digester - [SetPropertiesRule]{toplevel/stringset}
Setting property 'attr10' to 'ten'
main DEBUG digester.Digester - [SetPropertiesRule]{toplevel/stringset}
Set gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet properties
main DEBUG beanutils.BeanUtils -
BeanUtils.populate(gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet@692a3722,
{attr9=9], attr10=ten, attr2=2], attr1=1], attr4=4], attr3=3], attr6=6],
attr5=5], attr8=8]!, attr7=8]!even})
main DEBUG beanutils.BeanUtils -  
setProperty(gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet@692a3722,
attr9, 9])
main DEBUG beanutils.ConvertUtils - Convert string '9]' to class
'java.lang.String'
main DEBUG beanutils.ConvertUtils -   Using converter
org.apache.commons.beanutils.converters.StringConverter@6262937c
main DEBUG beanutils.PropertyUtils - setSimpleProperty: Invoking method
public void
gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet.setAttr9(java.lang.String)
with value 9] (class java.lang.String)
main DEBUG beanutils.BeanUtils -  
setProperty(gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet@692a3722,
attr10, ten)
main DEBUG beanutils.ConvertUtils - Convert string 'ten' to class
'java.lang.String'
main DEBUG beanutils.ConvertUtils -   Using converter
org.apache.commons.beanutils.converters.StringConverter@6262937c
main DEBUG beanutils.PropertyUtils - setSimpleProperty: Invoking method
public void
gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet.setAttr10(java.lang.String)
with value ten (class java.lang.String)
main DEBUG beanutils.BeanUtils -  
setProperty(gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet@692a3722,
attr2, 2])
main DEBUG beanutils.ConvertUtils - Convert string '2]' to class
'java.lang.String'
main DEBUG beanutils.ConvertUtils -   Using converter
org.apache.commons.beanutils.converters.StringConverter@6262937c
main DEBUG beanutils.PropertyUtils - setSimpleProperty: Invoking method
public void
gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet.setAttr2(java.lang.String)
with value 2] (class java.lang.String)
main DEBUG beanutils.BeanUtils -  
setProperty(gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet@692a3722,
attr1, 1])
main DEBUG beanutils.ConvertUtils - Convert string '1]' to class
'java.lang.String'
main DEBUG beanutils.ConvertUtils -   Using converter
org.apache.commons.beanutils.converters.StringConverter@6262937c
main DEBUG beanutils.PropertyUtils - setSimpleProperty: Invoking method
public void
gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet.setAttr1(java.lang.String)
with value 1] (class java.lang.String)
main DEBUG beanutils.BeanUtils -  
setProperty(gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet@692a3722,
attr4, 4])
main DEBUG beanutils.ConvertUtils - Convert string '4]' to class
'java.lang.String'
main DEBUG beanutils.ConvertUtils -   Using converter
org.apache.commons.beanutils.converters.StringConverter@6262937c
main DEBUG beanutils.PropertyUtils - setSimpleProperty: Invoking method
public void
gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet.setAttr4(java.lang.String)
with value 4] (class java.lang.String)
main DEBUG beanutils.BeanUtils -  
setProperty(gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet@692a3722,
attr3, 3])
main DEBUG beanutils.ConvertUtils - Convert string '3]' to class
'java.lang.String'
main DEBUG beanutils.ConvertUtils -   Using converter
org.apache.commons.beanutils.converters.StringConverter@6262937c
main DEBUG beanutils.PropertyUtils - setSimpleProperty: Invoking method
public void
gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet.setAttr3(java.lang.String)
with value 3] (class java.lang.String)
main DEBUG beanutils.BeanUtils -  
setProperty(gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet@692a3722,
attr6, 6])
main DEBUG beanutils.ConvertUtils - Convert string '6]' to class
'java.lang.String'
main DEBUG beanutils.ConvertUtils -   Using converter
org.apache.commons.beanutils.converters.StringConverter@6262937c
main DEBUG beanutils.PropertyUtils - setSimpleProperty: Invoking method
public void
gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet.setAttr6(java.lang.String)
with value 6] (class java.lang.String)
main DEBUG beanutils.BeanUtils -  
setProperty(gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet@692a3722,
attr5, 5])
main DEBUG beanutils.ConvertUtils - Convert string '5]' to class
'java.lang.String'
main DEBUG beanutils.ConvertUtils -   Using converter
org.apache.commons.beanutils.converters.StringConverter@6262937c
main DEBUG beanutils.PropertyUtils - setSimpleProperty: Invoking method
public void
gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet.setAttr5(java.lang.String)
with value 5] (class java.lang.String)
main DEBUG beanutils.BeanUtils -  
setProperty(gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet@692a3722,
attr8, 8]!)
main DEBUG beanutils.ConvertUtils - Convert string '8]!' to class
'java.lang.String'
main DEBUG beanutils.ConvertUtils -   Using converter
org.apache.commons.beanutils.converters.StringConverter@6262937c
main DEBUG beanutils.PropertyUtils - setSimpleProperty: Invoking method
public void
gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet.setAttr8(java.lang.String)
with value 8]! (class java.lang.String)
main DEBUG beanutils.BeanUtils -  
setProperty(gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet@692a3722,
attr7, 8]!even)
main DEBUG beanutils.ConvertUtils - Convert string '8]!even' to class
'java.lang.String'
main DEBUG beanutils.ConvertUtils -   Using converter
org.apache.commons.beanutils.converters.StringConverter@6262937c
main DEBUG beanutils.PropertyUtils - setSimpleProperty: Invoking method
public void
gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet.setAttr7(java.lang.String)
with value 8]!even (class java.lang.String)
main DEBUG digester.Digester -   Fire begin() for
SetNextRule[methodName=addStringset, paramType=null]
main DEBUG Digester.sax - endElement(,,stringset)
main DEBUG digester.Digester -   match='toplevel/stringset'
main DEBUG digester.Digester -   bodyText=''
main DEBUG digester.Digester -   Fire body() for
ObjectCreateRule[className=gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet,
attributeName=null]
main DEBUG digester.Digester -   Fire body() for SetPropertiesRule[]
main DEBUG digester.Digester -   Fire body() for
SetNextRule[methodName=addStringset, paramType=null]
main DEBUG digester.Digester -   Popping body text '



    '
main DEBUG digester.Digester -   Fire end() for
SetNextRule[methodName=addStringset, paramType=null]
main DEBUG digester.Digester - [SetNextRule]{toplevel/stringset} Call
gov.noaa.eds.byExample.tryDigesterConfigLimit.ConfigData.addStringset(gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet@692a3722)
main DEBUG beanutils.MethodUtils - Matching name=addStringset on class
gov.noaa.eds.byExample.tryDigesterConfigLimit.ConfigData
main DEBUG beanutils.MethodUtils - Found straight match: public void
gov.noaa.eds.byExample.tryDigesterConfigLimit.ConfigData.addStringset(gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet)
main DEBUG beanutils.MethodUtils - isPublic:true
main DEBUG digester.Digester -   Fire end() for SetPropertiesRule[]
main DEBUG digester.Digester -   Fire end() for
ObjectCreateRule[className=gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet,
attributeName=null]
main DEBUG digester.Digester - [ObjectCreateRule]{toplevel/stringset}
Pop gov.noaa.eds.byExample.tryDigesterConfigLimit.StringSet
main DEBUG Digester.sax - characters(

)
main DEBUG Digester.sax - endElement(,,toplevel)
main DEBUG digester.Digester -   match='toplevel'
main DEBUG digester.Digester -   bodyText='





'
main DEBUG digester.Digester -   No rules found matching 'toplevel'.
main DEBUG digester.Digester -   Popping body text ''
main DEBUG Digester.sax - endDocument()


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@...
For additional commands, e-mail: user-help@...