[ exist-Patches-2015037 ] Parse i18n Strings

[SourceForge.net]

Patches item #2015037, was opened at 2008-07-10 22:10
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=317691&aid=2015037&group_id=17691

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Michael Westbay (westbay)
Assigned to: Nobody/Anonymous (nobody)
Summary: Parse i18n Strings

Initial Comment:
Environment:

Mac OS/X 10.4 (Japanese locale)
JDK 1.5
SVN build 7939 (Updated July 10, 2008 approx. 9:00pm JST)


Problem:

After retrieving a Japanese string (in UTF-8) from MySQL, the serialization of the XML result was failing with the following partial trace:

2008-07-10 20:26:46,373 [SocketListener0-4] DEBUG (ModuleUtils.java [stringToXML]:76) - Parsing XML response ...
2008-07-10 20:26:46,429 [SocketListener0-4] ERROR (ExecuteFunction.java [eval]:217) - sql:execute() Could not serialize SQL results to XML for SQL: "select articleId, title, intro from News where feature=1 and lang='ja'  order by date desc, aTimestamp desc limit 20"
org.apache.xerces.impl.io.MalformedByteSequenceException: Invalid byte 1 of 1-byte UTF-8 sequence.
        at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)
        at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)
        at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
        at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
        at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
        at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
        at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
        at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
        at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
        at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
        at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
        at org.exist.xquery.modules.ModuleUtils.stringToXML(ModuleUtils.java:83)
        at org.exist.xquery.modules.sql.ExecuteFunction.eval(ExecuteFunction.java:215)
        at org.exist.xquery.BasicFunction.eval(BasicFunction.java:68)


Analysis:

It turns out that in org.exist.xquery.modules.ModuleUtils that xml.getBytes() is called without an encoding parameter.  The system default is attempted to be used, and this causes the above failure.

By calling this with xml.getBytes("UTF-8") everything works fine.


Patch:

The supplied patch only modifies the getBytes function on line 78 as described above.  This fixes the problem that I'm having, and no other problems have cropped up.  But I don't know if other systems or setups will be effected (i.e. fetching from other databases in the SQL module using character encodings other than Unicode).

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=317691&aid=2015037&group_id=17691

-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
_______________________________________________
Exist-open mailing list
Exist-open@...
https://lists.sourceforge.net/lists/listinfo/exist-open