[jira] Created: (JIBX-228) ArrayIndexOutOfBoundsException caused by org.jibx.extras.DomMapperBase

ArrayIndexOutOfBoundsException caused by org.jibx.extras.DomMapperBase
----------------------------------------------------------------------

                 Key: JIBX-228
                 URL: http://jira.codehaus.org/browse/JIBX-228
             Project: JiBX
          Issue Type: Bug
          Components: core
    Affects Versions: JiBX 1.1.6
         Environment: JiBX v1.1.6a
JDK v1.5.0_06
Eclipse IDE "Ganymede" v3.4 + JiBX eclipse plugin activated
            Reporter: Alexandre Delarge
             Fix For: JiBX 1.1.6
         Attachments: jibx-jira.zip

Hello,


I refactored a web service application that made heavy usage of DOM generation/parsing to use JiBX instead.

While working on this, I wrote a couple JiBX bindings that rely on the "org.jibx.extras.DomElementMapper" class that comes as a part of JiBX extras.

Using this mapper to marshall arbitrary portions of DOM trees raised a number of ArrayIndexOutOfBoundsException that prevented my application to success in generating response messages.

I drilled down these error causes into JiBX source code, and here is what I found (note: all references to source code below are taken from v1.1.6a).


1) Lookup of known namespaces & prefixes
==================================

In class "org.jibx.extras.DomMapperBase", in many places inside the method "findNamespaceIndex" you return -1 if you don't find a namespace prefix among the list owned by the m_xmlWriter object.
Here is an example on line 123:
---
  [...]
  int index = m_xmlWriter.getPrefixIndex(prefix);
  if (index >= 0) {
      return getNamespaceUri(index).equals(uri) ?
          index : -1;
  } else {        // line 122
      return -1;  // line 123
  }
  [...]
---

This -1 value greatly increases the chances of getting an ArrayIndexOutOfBoundsException from method "getNamespacePrefix(int index)" in class "org.jibx.runtime.impl.XMLWriterNamespaceBase".

The -1 value results from a failure in finding the exact given prefix in the list of prefixes the m_xmlWriter object owns.
I think that looking for the namespace itself inside the list of namespace URIs owned by the m_xmlWriter object will do the right job, while suppressing many of the exceptions.
To me it sounds valid, because it happens that the namespace URI is known but just associated to a different prefix.

During the marshalling of various DOM elements, I got bothered by this missing lookup in several places, so I implemented it into a new private method:
---
  /**
   * Get index number for declared namespace.
   *
   * @param uri namespace URI (empty string if none)
   * @return namespace index number, or <code>-1</code> if not declared or
   * masked
   */
  private int findNamespaceURIIndex(String uri) {
  String[] uris = m_xmlWriter.getNamespaces();
  for (int i = 0; i < uris.length; i++) {
  if(uris[i].equals(uri)) {
  return i;
  }
  }
  return -1;
  }
---

Then I updated the "findNamespaceIndex" method accordingly, in the 3 following locations (lines 108, 115, and 123):
---
[...]
        } else {
           return findNamespaceURIIndex(uri);                     // line 108
        }
      } else {
        return -1;
      }
    } else {
    return m_defaultNamespaceURI.equals(uri) ?                    
            m_defaultNamespaceIndex : findNamespaceURIIndex(uri); // line 115
  }
} else {
    int index = m_xmlWriter.getPrefixIndex(prefix);
    if (index >= 0) {
        return getNamespaceUri(index).equals(uri) ?
            index : -1;
    } else {                                                      
    return findNamespaceURIIndex(uri);                           // line 123
    }
[...]
---


2) Discovery of namespaces & prefixes
==============================

I also had troubles with the strategy for handling some namespaces declarations coming from a DOM tree, with the same kind of symptoms showing up: ArrayIndexOutOfBoundsException rising from class "org.jibx.runtime.impl.StreamWriterBase", more exactly from method "writePrefix".

For that one I had to look closely at method "marshalElement" in class "org.jibx.extras.DomElementMapper", stepping into the algorithm for getting the right namespace index for the current element.

Actually, I think the following assignment at line 227 (by the way is the "==" operator wanted here, or is it "equals()" ?):
---
[...]
  if (uri == decluri) {
    nsi = defind;     // line 227
  }
[...]
---


Will occasionally de-activates the lookup for the right index among newly discovered namespaces, which is made at line 251:
---
[...]
  for (int i = 0; i < length; i++) {
    prefs[i] = (String)nss.get(i*2);
    uris[i] = (String)nss.get(i*2+1);
    nums[i] = base + i;
    if (nsi < 0 && uri.equals(uris[i])) {                   // line 251
      if ((prefix == null && prefs[i] == "") ||
        (prefix != null && prefix.equals(prefs[i]))) {
        nsi = base + i;
      }
    }
  }
[...]
---

This may result in an incorrect index being used, because the lookup made at line 251 takes into account the index shift introduced by the addition of new namespaces, while this assignment does not.
I therefore remove it.

However the main issue with that method in my humble opinion is the fact we don't check that we have found the element's namespace index just after we checked all the attributes for namespaces declarations.
I therefore added the following statements just before line 233:
---
[...]
  /*
   * if we get there without having found an index for the current
   * element's namespace, we add it to the list.
   */
  if(nsi == -1 && (nss == null || !nss.contains(uri))) {
  if(nss == null) nss = new ArrayList();
      nss.add(prefix == null ? "" : prefix);
      nss.add(uri == null ? "" : uri);
  }
[...]
---

I don't alter the value of "nsi", so the aforementioned lookup located at line 251 will assign it with the right value.


I attach a sample Eclipse project to this report, it contains a sample set of Java classes, XML file, and binding that exhibits the 2nd issue (I cannot easily cause the 1st one to happen).
The code I bundle features a JUnit test case in class "test.jibx.junit.TestDomMapper".

I include my version of the "DomMapperBase" class, which enables the code I wrote  to work fine.
I also include a visual summary of the differences between the original version, of the class and mine, hoping this will help.


Regards,
Alex.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
_______________________________________________
jibx-devs mailing list
jibx-devs@...
https://lists.sourceforge.net/lists/listinfo/jibx-devs

     [ http://jira.codehaus.org/browse/JIBX-228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alexandre Delarge updated JIBX-228:
-----------------------------------

    Attachment: jira-jibx-228.txt

Sorry I haven't used the preview feature, and did not see the code excerpts get broken by the formatting.

Here is the properly formatted text I submitted, which should be more readable.

regards,
Alex.

> ArrayIndexOutOfBoundsException caused by org.jibx.extras.DomMapperBase
> ----------------------------------------------------------------------
>
>                 Key: JIBX-228
>                 URL: http://jira.codehaus.org/browse/JIBX-228
>             Project: JiBX
>          Issue Type: Bug
>          Components: core
>    Affects Versions: JiBX 1.1.6
>         Environment: JiBX v1.1.6a
> JDK v1.5.0_06
> Eclipse IDE "Ganymede" v3.4 + JiBX eclipse plugin activated
>            Reporter: Alexandre Delarge
>             Fix For: JiBX 1.1.6
>
>         Attachments: jibx-jira.zip, jira-jibx-228.txt
>
>
> Hello,
> I refactored a web service application that made heavy usage of DOM generation/parsing to use JiBX instead.
> While working on this, I wrote a couple JiBX bindings that rely on the "org.jibx.extras.DomElementMapper" class that comes as a part of JiBX extras.
> Using this mapper to marshall arbitrary portions of DOM trees raised a number of ArrayIndexOutOfBoundsException that prevented my application to success in generating response messages.
> I drilled down these error causes into JiBX source code, and here is what I found (note: all references to source code below are taken from v1.1.6a).
> 1) Lookup of known namespaces & prefixes
> ==================================
> In class "org.jibx.extras.DomMapperBase", in many places inside the method "findNamespaceIndex" you return -1 if you don't find a namespace prefix among the list owned by the m_xmlWriter object.
> Here is an example on line 123:
> ---
>   [...]
>   int index = m_xmlWriter.getPrefixIndex(prefix);
>   if (index >= 0) {
>       return getNamespaceUri(index).equals(uri) ?
>           index : -1;
>   } else {        // line 122
>       return -1;  // line 123
>   }
>   [...]
> ---
> This -1 value greatly increases the chances of getting an ArrayIndexOutOfBoundsException from method "getNamespacePrefix(int index)" in class "org.jibx.runtime.impl.XMLWriterNamespaceBase".
> The -1 value results from a failure in finding the exact given prefix in the list of prefixes the m_xmlWriter object owns.
> I think that looking for the namespace itself inside the list of namespace URIs owned by the m_xmlWriter object will do the right job, while suppressing many of the exceptions.
> To me it sounds valid, because it happens that the namespace URI is known but just associated to a different prefix.
> During the marshalling of various DOM elements, I got bothered by this missing lookup in several places, so I implemented it into a new private method:
> ---
>   /**
>    * Get index number for declared namespace.
>    *
>    * @param uri namespace URI (empty string if none)
>    * @return namespace index number, or <code>-1</code> if not declared or
>    * masked
>    */
>   private int findNamespaceURIIndex(String uri) {
>   String[] uris = m_xmlWriter.getNamespaces();
>   for (int i = 0; i < uris.length; i++) {
>   if(uris[i].equals(uri)) {
>   return i;
>   }
>   }
>   return -1;
>   }
> ---
> Then I updated the "findNamespaceIndex" method accordingly, in the 3 following locations (lines 108, 115, and 123):
> ---
> [...]
>         } else {
>            return findNamespaceURIIndex(uri);                     // line 108
>         }
>       } else {
>         return -1;
>       }
>     } else {
>     return m_defaultNamespaceURI.equals(uri) ?                    
>             m_defaultNamespaceIndex : findNamespaceURIIndex(uri); // line 115
>   }
> } else {
>     int index = m_xmlWriter.getPrefixIndex(prefix);
>     if (index >= 0) {
>         return getNamespaceUri(index).equals(uri) ?
>             index : -1;
>     } else {                                                      
>     return findNamespaceURIIndex(uri);                           // line 123
>     }
> [...]
> ---
> 2) Discovery of namespaces & prefixes
> ==============================
> I also had troubles with the strategy for handling some namespaces declarations coming from a DOM tree, with the same kind of symptoms showing up: ArrayIndexOutOfBoundsException rising from class "org.jibx.runtime.impl.StreamWriterBase", more exactly from method "writePrefix".
> For that one I had to look closely at method "marshalElement" in class "org.jibx.extras.DomElementMapper", stepping into the algorithm for getting the right namespace index for the current element.
> Actually, I think the following assignment at line 227 (by the way is the "==" operator wanted here, or is it "equals()" ?):
> ---
> [...]
>   if (uri == decluri) {
>     nsi = defind;     // line 227
>   }
> [...]
> ---
> Will occasionally de-activates the lookup for the right index among newly discovered namespaces, which is made at line 251:
> ---
> [...]
>   for (int i = 0; i < length; i++) {
>     prefs[i] = (String)nss.get(i*2);
>     uris[i] = (String)nss.get(i*2+1);
>     nums[i] = base + i;
>     if (nsi < 0 && uri.equals(uris[i])) {                   // line 251
>       if ((prefix == null && prefs[i] == "") ||
>         (prefix != null && prefix.equals(prefs[i]))) {
>         nsi = base + i;
>       }
>     }
>   }
> [...]
> ---
> This may result in an incorrect index being used, because the lookup made at line 251 takes into account the index shift introduced by the addition of new namespaces, while this assignment does not.
> I therefore remove it.
> However the main issue with that method in my humble opinion is the fact we don't check that we have found the element's namespace index just after we checked all the attributes for namespaces declarations.
> I therefore added the following statements just before line 233:
> ---
> [...]
>   /*
>    * if we get there without having found an index for the current
>    * element's namespace, we add it to the list.
>    */
>   if(nsi == -1 && (nss == null || !nss.contains(uri))) {
>   if(nss == null) nss = new ArrayList();
>       nss.add(prefix == null ? "" : prefix);
>       nss.add(uri == null ? "" : uri);
>   }
> [...]
> ---
> I don't alter the value of "nsi", so the aforementioned lookup located at line 251 will assign it with the right value.
> I attach a sample Eclipse project to this report, it contains a sample set of Java classes, XML file, and binding that exhibits the 2nd issue (I cannot easily cause the 1st one to happen).
> The code I bundle features a JUnit test case in class "test.jibx.junit.TestDomMapper".
> I include my version of the "DomMapperBase" class, which enables the code I wrote  to work fine.
> I also include a visual summary of the differences between the original version, of the class and mine, hoping this will help.
> Regards,
> Alex.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
_______________________________________________
jibx-devs mailing list
jibx-devs@...
https://lists.sourceforge.net/lists/listinfo/jibx-devs

    [ http://jira.codehaus.org/browse/JIBX-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=141323#action_141323 ]

Alexandre Delarge commented on JIBX-228:
----------------------------------------

While conducing further testings with other, more complex, SOAP messages, I realized that changing the mechanism for the lookup of known namespaces & prefixes (1st issue I reported) does cause other problems because of the method "closeNamespaces" in class "org.jibx.runtime.impl.XMLWriterNamespaceBase".

The method "closeNamespaces" removes the prefixes associated to namespaces declarations that are nested deep inside an element. As I understand it, this cleanup happens right after the element got marshalled, and right before the next element's sibling is processed.

With the changes I made in the lookup of known namespaces, the following appears: if the sibling refers to a namespace of which prefix got cleaned up, then a NullPointerException will rise from method "writePrefix" in class "org.jibx.runtime.impl.GenericXMLWriter", because the cleaning can assign the prefix's value with a null String.

Consequently, it appears the changes I made to the method "findNamespaceIndex" in class "org.jibx.extras.DomElementMapper" by introducing a new method "findNamespaceURIIndex" are dangerous, as they introduce this other issue at least.

The changes made in the context of the 2nd seem to stay valid so far though.

Regards,
Alex.

> ArrayIndexOutOfBoundsException caused by org.jibx.extras.DomMapperBase
> ----------------------------------------------------------------------
>
>                 Key: JIBX-228
>                 URL: http://jira.codehaus.org/browse/JIBX-228
>             Project: JiBX
>          Issue Type: Bug
>          Components: core
>    Affects Versions: JiBX 1.1.6
>         Environment: JiBX v1.1.6a
> JDK v1.5.0_06
> Eclipse IDE "Ganymede" v3.4 + JiBX eclipse plugin activated
>            Reporter: Alexandre Delarge
>             Fix For: JiBX 1.1.6
>
>         Attachments: jibx-jira.zip, jira-jibx-228.txt
>
>
> Hello,
> I refactored a web service application that made heavy usage of DOM generation/parsing to use JiBX instead.
> While working on this, I wrote a couple JiBX bindings that rely on the "org.jibx.extras.DomElementMapper" class that comes as a part of JiBX extras.
> Using this mapper to marshall arbitrary portions of DOM trees raised a number of ArrayIndexOutOfBoundsException that prevented my application to success in generating response messages.
> I drilled down these error causes into JiBX source code, and here is what I found (note: all references to source code below are taken from v1.1.6a).
> 1) Lookup of known namespaces & prefixes
> ==================================
> In class "org.jibx.extras.DomMapperBase", in many places inside the method "findNamespaceIndex" you return -1 if you don't find a namespace prefix among the list owned by the m_xmlWriter object.
> Here is an example on line 123:
> ---
>   [...]
>   int index = m_xmlWriter.getPrefixIndex(prefix);
>   if (index >= 0) {
>       return getNamespaceUri(index).equals(uri) ?
>           index : -1;
>   } else {        // line 122
>       return -1;  // line 123
>   }
>   [...]
> ---
> This -1 value greatly increases the chances of getting an ArrayIndexOutOfBoundsException from method "getNamespacePrefix(int index)" in class "org.jibx.runtime.impl.XMLWriterNamespaceBase".
> The -1 value results from a failure in finding the exact given prefix in the list of prefixes the m_xmlWriter object owns.
> I think that looking for the namespace itself inside the list of namespace URIs owned by the m_xmlWriter object will do the right job, while suppressing many of the exceptions.
> To me it sounds valid, because it happens that the namespace URI is known but just associated to a different prefix.
> During the marshalling of various DOM elements, I got bothered by this missing lookup in several places, so I implemented it into a new private method:
> ---
>   /**
>    * Get index number for declared namespace.
>    *
>    * @param uri namespace URI (empty string if none)
>    * @return namespace index number, or <code>-1</code> if not declared or
>    * masked
>    */
>   private int findNamespaceURIIndex(String uri) {
>   String[] uris = m_xmlWriter.getNamespaces();
>   for (int i = 0; i < uris.length; i++) {
>   if(uris[i].equals(uri)) {
>   return i;
>   }
>   }
>   return -1;
>   }
> ---
> Then I updated the "findNamespaceIndex" method accordingly, in the 3 following locations (lines 108, 115, and 123):
> ---
> [...]
>         } else {
>            return findNamespaceURIIndex(uri);                     // line 108
>         }
>       } else {
>         return -1;
>       }
>     } else {
>     return m_defaultNamespaceURI.equals(uri) ?                    
>             m_defaultNamespaceIndex : findNamespaceURIIndex(uri); // line 115
>   }
> } else {
>     int index = m_xmlWriter.getPrefixIndex(prefix);
>     if (index >= 0) {
>         return getNamespaceUri(index).equals(uri) ?
>             index : -1;
>     } else {                                                      
>     return findNamespaceURIIndex(uri);                           // line 123
>     }
> [...]
> ---
> 2) Discovery of namespaces & prefixes
> ==============================
> I also had troubles with the strategy for handling some namespaces declarations coming from a DOM tree, with the same kind of symptoms showing up: ArrayIndexOutOfBoundsException rising from class "org.jibx.runtime.impl.StreamWriterBase", more exactly from method "writePrefix".
> For that one I had to look closely at method "marshalElement" in class "org.jibx.extras.DomElementMapper", stepping into the algorithm for getting the right namespace index for the current element.
> Actually, I think the following assignment at line 227 (by the way is the "==" operator wanted here, or is it "equals()" ?):
> ---
> [...]
>   if (uri == decluri) {
>     nsi = defind;     // line 227
>   }
> [...]
> ---
> Will occasionally de-activates the lookup for the right index among newly discovered namespaces, which is made at line 251:
> ---
> [...]
>   for (int i = 0; i < length; i++) {
>     prefs[i] = (String)nss.get(i*2);
>     uris[i] = (String)nss.get(i*2+1);
>     nums[i] = base + i;
>     if (nsi < 0 && uri.equals(uris[i])) {                   // line 251
>       if ((prefix == null && prefs[i] == "") ||
>         (prefix != null && prefix.equals(prefs[i]))) {
>         nsi = base + i;
>       }
>     }
>   }
> [...]
> ---
> This may result in an incorrect index being used, because the lookup made at line 251 takes into account the index shift introduced by the addition of new namespaces, while this assignment does not.
> I therefore remove it.
> However the main issue with that method in my humble opinion is the fact we don't check that we have found the element's namespace index just after we checked all the attributes for namespaces declarations.
> I therefore added the following statements just before line 233:
> ---
> [...]
>   /*
>    * if we get there without having found an index for the current
>    * element's namespace, we add it to the list.
>    */
>   if(nsi == -1 && (nss == null || !nss.contains(uri))) {
>   if(nss == null) nss = new ArrayList();
>       nss.add(prefix == null ? "" : prefix);
>       nss.add(uri == null ? "" : uri);
>   }
> [...]
> ---
> I don't alter the value of "nsi", so the aforementioned lookup located at line 251 will assign it with the right value.
> I attach a sample Eclipse project to this report, it contains a sample set of Java classes, XML file, and binding that exhibits the 2nd issue (I cannot easily cause the 1st one to happen).
> The code I bundle features a JUnit test case in class "test.jibx.junit.TestDomMapper".
> I include my version of the "DomMapperBase" class, which enables the code I wrote  to work fine.
> I also include a visual summary of the differences between the original version, of the class and mine, hoping this will help.
> Regards,
> Alex.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
_______________________________________________
jibx-devs mailing list
jibx-devs@...
https://lists.sourceforge.net/lists/listinfo/jibx-devs