[jira] Created: (CODEC-73) Make string2byte conversions indepedent of platform default encoding

Make string2byte conversions indepedent of platform default encoding
--------------------------------------------------------------------

                 Key: CODEC-73
                 URL: https://issues.apache.org/jira/browse/CODEC-73
             Project: Commons Codec
          Issue Type: Bug
    Affects Versions: 1.3
         Environment: any platform whose default encoding is not a superset of ASCII, e.g. UTF-16 or EBCDIC
            Reporter: Benjamin Bentmann
         Attachments: CODEC-73.patch

Both the library itself and many of its tests are utterly dependent on the JVM's default charset. For example, {{DigestUtils}} calls {{String.getBytes()}} to convert an input string to a byte array, happily delivering different digests for the same input string if run on different platforms.

If you want to try out the havor yourself, just run the unit tests in a JVM with UTF-16, e.g. by adding the line
{code:xml}
<argLine>-Dfile.encoding=UTF-16</argLine>
{code}
to the configuration of the Surefire Plugin in the POM.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/CODEC-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Benjamin Bentmann updated CODEC-73:
-----------------------------------

    Attachment: CODEC-73.patch

> Make string2byte conversions indepedent of platform default encoding
> --------------------------------------------------------------------
>
>                 Key: CODEC-73
>                 URL: https://issues.apache.org/jira/browse/CODEC-73
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.3
>         Environment: any platform whose default encoding is not a superset of ASCII, e.g. UTF-16 or EBCDIC
>            Reporter: Benjamin Bentmann
>         Attachments: CODEC-73.patch
>
>
> Both the library itself and many of its tests are utterly dependent on the JVM's default charset. For example, {{DigestUtils}} calls {{String.getBytes()}} to convert an input string to a byte array, happily delivering different digests for the same input string if run on different platforms.
> If you want to try out the havor yourself, just run the unit tests in a JVM with UTF-16, e.g. by adding the line
> {code:xml}
> <argLine>-Dfile.encoding=UTF-16</argLine>
> {code}
> to the configuration of the Surefire Plugin in the POM.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/CODEC-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12613293#action_12613293 ]

Sebb commented on CODEC-73:
---------------------------

Agreed that the default charset dependency needs to be removed.

However, I have an alternative suggestion:

In the case of the Hex encode() method, one could completely avoid the need to use getBytes() by using a byte[] array for the conversion.

Likewise, I think the Hex decode() could just be performed on bytes rather than converting to char first.

Patch to follow.

> Make string2byte conversions indepedent of platform default encoding
> --------------------------------------------------------------------
>
>                 Key: CODEC-73
>                 URL: https://issues.apache.org/jira/browse/CODEC-73
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.3
>         Environment: any platform whose default encoding is not a superset of ASCII, e.g. UTF-16 or EBCDIC
>            Reporter: Benjamin Bentmann
>         Attachments: CODEC-73.patch
>
>
> Both the library itself and many of its tests are utterly dependent on the JVM's default charset. For example, {{DigestUtils}} calls {{String.getBytes()}} to convert an input string to a byte array, happily delivering different digests for the same input string if run on different platforms.
> If you want to try out the havor yourself, just run the unit tests in a JVM with UTF-16, e.g. by adding the line
> {code:xml}
> <argLine>-Dfile.encoding=UTF-16</argLine>
> {code}
> to the configuration of the Surefire Plugin in the POM.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/CODEC-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sebb updated CODEC-73:
----------------------

    Attachment: Hex.patch

Add byte[] conversion methods

> Make string2byte conversions indepedent of platform default encoding
> --------------------------------------------------------------------
>
>                 Key: CODEC-73
>                 URL: https://issues.apache.org/jira/browse/CODEC-73
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.3
>         Environment: any platform whose default encoding is not a superset of ASCII, e.g. UTF-16 or EBCDIC
>            Reporter: Benjamin Bentmann
>         Attachments: CODEC-73.patch, Hex.patch
>
>
> Both the library itself and many of its tests are utterly dependent on the JVM's default charset. For example, {{DigestUtils}} calls {{String.getBytes()}} to convert an input string to a byte array, happily delivering different digests for the same input string if run on different platforms.
> If you want to try out the havor yourself, just run the unit tests in a JVM with UTF-16, e.g. by adding the line
> {code:xml}
> <argLine>-Dfile.encoding=UTF-16</argLine>
> {code}
> to the configuration of the Surefire Plugin in the POM.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/CODEC-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sebb updated CODEC-73:
----------------------

    Attachment:     (was: Hex.patch)

> Make string2byte conversions indepedent of platform default encoding
> --------------------------------------------------------------------
>
>                 Key: CODEC-73
>                 URL: https://issues.apache.org/jira/browse/CODEC-73
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.3
>         Environment: any platform whose default encoding is not a superset of ASCII, e.g. UTF-16 or EBCDIC
>            Reporter: Benjamin Bentmann
>         Attachments: CODEC-73.patch
>
>
> Both the library itself and many of its tests are utterly dependent on the JVM's default charset. For example, {{DigestUtils}} calls {{String.getBytes()}} to convert an input string to a byte array, happily delivering different digests for the same input string if run on different platforms.
> If you want to try out the havor yourself, just run the unit tests in a JVM with UTF-16, e.g. by adding the line
> {code:xml}
> <argLine>-Dfile.encoding=UTF-16</argLine>
> {code}
> to the configuration of the Surefire Plugin in the POM.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/CODEC-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sebb updated CODEC-73:
----------------------

    Attachment: Hex.patch

Add byte[] conversion methods (with private toDigit(byte method)

> Make string2byte conversions indepedent of platform default encoding
> --------------------------------------------------------------------
>
>                 Key: CODEC-73
>                 URL: https://issues.apache.org/jira/browse/CODEC-73
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.3
>         Environment: any platform whose default encoding is not a superset of ASCII, e.g. UTF-16 or EBCDIC
>            Reporter: Benjamin Bentmann
>         Attachments: CODEC-73.patch, Hex.patch
>
>
> Both the library itself and many of its tests are utterly dependent on the JVM's default charset. For example, {{DigestUtils}} calls {{String.getBytes()}} to convert an input string to a byte array, happily delivering different digests for the same input string if run on different platforms.
> If you want to try out the havor yourself, just run the unit tests in a JVM with UTF-16, e.g. by adding the line
> {code:xml}
> <argLine>-Dfile.encoding=UTF-16</argLine>
> {code}
> to the configuration of the Surefire Plugin in the POM.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/CODEC-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sebb updated CODEC-73:
----------------------

    Comment: was deleted

> Make string2byte conversions indepedent of platform default encoding
> --------------------------------------------------------------------
>
>                 Key: CODEC-73
>                 URL: https://issues.apache.org/jira/browse/CODEC-73
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.3
>         Environment: any platform whose default encoding is not a superset of ASCII, e.g. UTF-16 or EBCDIC
>            Reporter: Benjamin Bentmann
>         Attachments: CODEC-73.patch, Hex.patch
>
>
> Both the library itself and many of its tests are utterly dependent on the JVM's default charset. For example, {{DigestUtils}} calls {{String.getBytes()}} to convert an input string to a byte array, happily delivering different digests for the same input string if run on different platforms.
> If you want to try out the havor yourself, just run the unit tests in a JVM with UTF-16, e.g. by adding the line
> {code:xml}
> <argLine>-Dfile.encoding=UTF-16</argLine>
> {code}
> to the configuration of the Surefire Plugin in the POM.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/CODEC-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Henri Yandell updated CODEC-73:
-------------------------------

    Fix Version/s: 1.4

> Make string2byte conversions indepedent of platform default encoding
> --------------------------------------------------------------------
>
>                 Key: CODEC-73
>                 URL: https://issues.apache.org/jira/browse/CODEC-73
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.3
>         Environment: any platform whose default encoding is not a superset of ASCII, e.g. UTF-16 or EBCDIC
>            Reporter: Benjamin Bentmann
>             Fix For: 1.4
>
>         Attachments: CODEC-73.patch, Hex.patch
>
>
> Both the library itself and many of its tests are utterly dependent on the JVM's default charset. For example, {{DigestUtils}} calls {{String.getBytes()}} to convert an input string to a byte array, happily delivering different digests for the same input string if run on different platforms.
> If you want to try out the havor yourself, just run the unit tests in a JVM with UTF-16, e.g. by adding the line
> {code:xml}
> <argLine>-Dfile.encoding=UTF-16</argLine>
> {code}
> to the configuration of the Surefire Plugin in the POM.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.