what to do about unicode?

[support-192]

I spent a-lot of time googling and reading archives, but i'm still not
sure how to proceed. I'm writing some crossplatform odbc code. The UI is
unicode. Since it will run on Windows, it would be nice if I could just
use wchar_t and the 'wide' functions for everything. Obviously they work
on Windows, and if I could get them to work on UNIX i'd be done with no
changes to code. I'm mainly interested in targeting just MySQL right
now. It looks like I have 3 choices.

1. forget it. use ANSI functions. The UI will have to do unicode to
multibyte type conversions when making odbc calls. To REALLY get this
right i'd have to support conversions to any and all known character
sets that might be used.. because i'm coming from unicode any character
is possible.

2. perform wchar_t to UTF16 conversions when making odbc calls. On
Windows this would be sort of a noop, but on UNIX it is necessary
because wchar_t is UCS-4 and unixODBC is UTF16( or ucs-2..?). I see no
standard functions for this conversion on UNIX. Guess I would need to
use a library like iconv or ICU.

3. I can make unixODBC take wchar_t/ucs-4 by using defining
SQL_WCHART_CONVERT. This would be great! ...but it will only work if my
drivers support it and if unixODBC has been built for it. How do I know?
>From what i've read it doesn't work or hasn't been tested.

thanks
_______________________________________________
unixODBC-support mailing list
unixODBC-support@...
http://mail.easysoft.com/mailman/listinfo/unixodbc-support

[Nick Gorham]

support wrote:

>I spent a-lot of time googling and reading archives, but i'm still not
>sure how to proceed. I'm writing some crossplatform odbc code. The UI is
>unicode. Since it will run on Windows, it would be nice if I could just
>use wchar_t and the 'wide' functions for everything. Obviously they work
>on Windows, and if I could get them to work on UNIX i'd be done with no
>changes to code. I'm mainly interested in targeting just MySQL right
>now. It looks like I have 3 choices.
>
>1. forget it. use ANSI functions. The UI will have to do unicode to
>multibyte type conversions when making odbc calls. To REALLY get this
>right i'd have to support conversions to any and all known character
>sets that might be used.. because i'm coming from unicode any character
>is possible.
>
>2. perform wchar_t to UTF16 conversions when making odbc calls. On
>Windows this would be sort of a noop, but on UNIX it is necessary
>because wchar_t is UCS-4 and unixODBC is UTF16( or ucs-2..?). I see no
>standard functions for this conversion on UNIX. Guess I would need to
>use a library like iconv or ICU.
>
>3. I can make unixODBC take wchar_t/ucs-4 by using defining
>SQL_WCHART_CONVERT. This would be great! ...but it will only work if my
>drivers support it and if unixODBC has been built for it. How do I know?
>>From what i've read it doesn't work or hasn't been tested.
>
>thanks
>_______________________________________________
>unixODBC-support mailing list
>unixODBC-support@...
>http://mail.easysoft.com/mailman/listinfo/unixodbc-support
>
>
>  
>

Well, one way, is to write the code in the same way for Windows and
unix, both then use UCS-2, all you then have to do, is add your own
version of the missing calls that handle the 16 bit codes. You can
certainly use iconv to convert to other encodings, but it depends on
what you want to do with the data.

I suspect that you will find you don't need that many conversion
functions, and you might find some gotchas if you let the stdlib handle
the codes, all it takes is someone to change the LANG and it all
changes. Just look at some of the problems Java and Perl have had
using/not using UTF8 LANGs

--
Nick Gorham
Easysoft Limited
http://www.easysoft.com, http://www.unixODBC.org

_______________________________________________
unixODBC-support mailing list
unixODBC-support@...
http://mail.easysoft.com/mailman/listinfo/unixodbc-support

[support-192]

Nick Gorham wrote:

> support wrote:
>
>> I spent a-lot of time googling and reading archives, but i'm still not
>> sure how to proceed. I'm writing some crossplatform odbc code. The UI is
>> unicode. Since it will run on Windows, it would be nice if I could just
>> use wchar_t and the 'wide' functions for everything. Obviously they work
>> on Windows, and if I could get them to work on UNIX i'd be done with no
>> changes to code. I'm mainly interested in targeting just MySQL right
>> now. It looks like I have 3 choices.
>>
>> 1. forget it. use ANSI functions. The UI will have to do unicode to
>> multibyte type conversions when making odbc calls. To REALLY get this
>> right i'd have to support conversions to any and all known character
>> sets that might be used.. because i'm coming from unicode any character
>> is possible.
>>
>> 2. perform wchar_t to UTF16 conversions when making odbc calls. On
>> Windows this would be sort of a noop, but on UNIX it is necessary
>> because wchar_t is UCS-4 and unixODBC is UTF16( or ucs-2..?). I see no
>> standard functions for this conversion on UNIX. Guess I would need to
>> use a library like iconv or ICU.
>>
>> 3. I can make unixODBC take wchar_t/ucs-4 by using defining
>> SQL_WCHART_CONVERT. This would be great! ...but it will only work if my
>> drivers support it and if unixODBC has been built for it. How do I know?
>>> From what i've read it doesn't work or hasn't been tested.
>>
>> thanks
>> _______________________________________________
>> unixODBC-support mailing list
>> unixODBC-support@...
>> http://mail.easysoft.com/mailman/listinfo/unixodbc-support
>>
>>
>>  
>>
> Well, one way, is to write the code in the same way for Windows and
> unix, both then use UCS-2, all you then have to do, is add your own
> version of the missing calls that handle the 16 bit codes. You can
> certainly use iconv to convert to other encodings, but it depends on
> what you want to do with the data.
>
> I suspect that you will find you don't need that many conversion
> functions, and you might find some gotchas if you let the stdlib
> handle the codes, all it takes is someone to change the LANG and it
> all changes. Just look at some of the problems Java and Perl have had
> using/not using UTF8 LANGs
>

Ok. I was planning a thin wrapper around the odbc calls anyway. I can
put the wchar_t <--> utf16 conversion in there. I don't even know that
unicode in the database is really necessary, but the UI will be unicode
so I might as well be doing odbc unicode to UI unicode conversions
instead of odbc ascii to UI unicode.

-john
_______________________________________________
unixODBC-support mailing list
unixODBC-support@...
http://mail.easysoft.com/mailman/listinfo/unixodbc-support

[Nick Gorham]

support wrote:

>>Ok. I was planning a thin wrapper around the odbc calls anyway. I can
>>put the wchar_t <--> utf16 conversion in there. I don't even know that
>>unicode in the database is really necessary, but the UI will be unicode
>>so I might as well be doing odbc unicode to UI unicode conversions
>>instead of odbc ascii to UI unicode.
>>
>>-john
>>_______________________________________________
>>
>>
>>    
>>

If you get the chance to make a list of the functions you use, it may be
worth us thinking about creating a helper library of such functions.

--
Nick Gorham
Easysoft Limited
http://www.easysoft.com, http://www.unixODBC.org

_______________________________________________
unixODBC-support mailing list
unixODBC-support@...
http://mail.easysoft.com/mailman/listinfo/unixodbc-support

[support-192]

Nick Gorham wrote:

> support wrote:
>
>>> Ok. I was planning a thin wrapper around the odbc calls anyway. I can
>>> put the wchar_t <--> utf16 conversion in there. I don't even know that
>>> unicode in the database is really necessary, but the UI will be unicode
>>> so I might as well be doing odbc unicode to UI unicode conversions
>>> instead of odbc ascii to UI unicode.
>>>
>>> -john
>>> _______________________________________________
>>>
>>>
>>>  
> If you get the chance to make a list of the functions you use, it may
> be worth us thinking about creating a helper library of such functions.
>

I just made a light c++ class to encapsulate SQLWCHAR strings.
It can be constructed from a std::wstring (and therefore a wchar_t string).
It has a static member function that can take a SQLWCHAR string and
return std::wstring.
It has a SQLWCHAR * operator overload so it can be passed directly to
ODBC calls.

On Linux iconv() is used for WCHAR_T <-> UTF16 conversions.
On windows it just internally stores a std::wstring reference, no
conversion is done.

I think that will cover my needs. At least so far. If interested I can
make the code available.

-john

_______________________________________________
unixODBC-support mailing list
unixODBC-support@...
http://mail.easysoft.com/mailman/listinfo/unixodbc-support

[Nick Gorham]

support wrote:

>Nick Gorham wrote:
>  
>
>>support wrote:
>>
>>    
>>
>>>>Ok. I was planning a thin wrapper around the odbc calls anyway. I can
>>>>put the wchar_t <--> utf16 conversion in there. I don't even know that
>>>>unicode in the database is really necessary, but the UI will be unicode
>>>>so I might as well be doing odbc unicode to UI unicode conversions
>>>>instead of odbc ascii to UI unicode.
>>>>
>>>>-john
>>>>_______________________________________________
>>>>
>>>>
>>>>  
>>>>        
>>>>
>>If you get the chance to make a list of the functions you use, it may
>>be worth us thinking about creating a helper library of such functions.
>>
>>    
>>
>I just made a light c++ class to encapsulate SQLWCHAR strings.
>It can be constructed from a std::wstring (and therefore a wchar_t string).
>It has a static member function that can take a SQLWCHAR string and
>return std::wstring.
>It has a SQLWCHAR * operator overload so it can be passed directly to
>ODBC calls.
>
>On Linux iconv() is used for WCHAR_T <-> UTF16 conversions.
>On windows it just internally stores a std::wstring reference, no
>conversion is done.
>
>I think that will cover my needs. At least so far. If interested I can
>make the code available.
>
>-john
>
>_______________________________________________
>unixODBC-support mailing list
>unixODBC-support@...
>http://mail.easysoft.com/mailman/listinfo/unixodbc-support
>
>
>  
>

Its things like that that C++ does make much simpiler :-)

--
Nick Gorham
Easysoft Limited
http://www.easysoft.com, http://www.unixODBC.org

_______________________________________________
unixODBC-support mailing list
unixODBC-support@...
http://mail.easysoft.com/mailman/listinfo/unixodbc-support