Nabble - Php - Internationalization (i18n)

Re: UTF-8 Japanese goes into MySQL OK, comes out question marks

2008-08-25T06:10:52Z

Hi,

I have the same problem, can you say me how you resolve this?

Thank you in advance!

Richard Pavonarius wrote:

OK, this is officially driving me nuts.

In phpMyAdmin, I can see the Japanese text in the db. By sending the
query SET NAMES 'utf8' before inserting data, I can get my own PHP
scripts to input correctly.

However, in my PHP scripts the Japanese text is coming out of the DB
as question marks. Static Japanese text on the page is OK, so it's not
a problem with the browser font.

PHP 5.2.2, MySQL 4.13-beta-standard

php.ini:
mbstring.detect_order = auto
mbstring.encoding_translation = On
mbstring.func_overload = 0
mbstring.http_input = auto
mbstring.http_output = UTF-8
mbstring.internal_encoding = UTF-8
mbstring.language = Neutral
mbstring.script_encoding = none
mbstring.strict_detection = Off
mbstring.substitute_character = none

/etc/my.cfg
[client]
default-character-set=utf8

[mysqld]
init-connect=SET NAMES 'utf8'
collation_server=utf8_unicode_ci
character_set_server=utf8
default-character-set=utf8

Thanks,

Rich

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Using php_intl in PHP 5.3 alpha on windows

2008-08-15T06:35:42Z

I was wondering if someone could help me with a problem I've been having
using the new php_intl.dll extension. I've taken the latest Win32
snapshot from 2008/08/15 and I have enabled the intl extension. I have
also placed all the ICU 3.6 DLLs, which I have downloaded from the ICU
project site, and placed them in my system32 folder. The problem I have
is that when I try to use the php command to perform a simple test I get
the following error.

"The procedure entry point
?extract@UnicodeString@icu_3_6@@QBEHPAGHAAW4UErrorCode@@@Z could not be
located in the dynamic link library icuuc36.dll."

Would anyone happen to know why this is being caused? I have even built
my own ICU DLLs from source just to see if I could fix it but
unfortunately it made no difference.

I realize that this is an alpha release and that there will be bugs but
I would really like to start using this extension now as I can not wait
until the official release date. Previous attempts I made to build the
intl DLL myself have been unsuccessful and so I was hopping that someone
here would have a solution.

Thanks,

Mike (trulogik at gmail dot com)

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

RE: Language detection

2008-08-13T07:50:11Z

> Thanks Ed. Unless I've misunderstood, this is just doing charset
> detection, with language as a bonus when the charset implies it?

That wouldn't be very useful. No, it uses recognizers for charset/language
combinations.

> difference between say English, French and German, all in UTF-8
> encoding, please let me know.

It does not have data to do any utf-8 language detection, but the structure
is in place.

You might want to consider adding data to their framework for what you want
to do. It isn't complicated. The most important thing you need is good
sample text in quantity so you can generate the n-gram probability table.

I believe the code was taken from Mozilla, so you might look there. Maybe
they've already done what you are looking for.

=Ed

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Language detection

2008-08-12T21:06:21Z

> http://pear.php.net/package/Text_LanguageDetect

Thanks to both people who suggested this; it is just what I was hoping
to find (and that google didn't). I'll start evaluating it then
roll-my-own on top if it isn't accurate enough.

Darren

--
Darren Cook, Software Researcher/Developer
http://dcook.org/mlsn/ (English-Japanese-German-Chinese-Arabic
open source dictionary/semantic network)
http://dcook.org/work/ (About me and my work)

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Language detection

2008-08-12T21:06:16Z

>> I need to write (PHP) code to detect the language of a given block of
>> text.

> In any case, ICU has this. See
>
> http://icu-project.org/userguide/charsetDetection.html
>
> It has both charset and language detection.

Thanks Ed. Unless I've misunderstood, this is just doing charset
detection, with language as a bonus when the charset implies it? If
someone is actually using this and can confirm it can tell the
difference between say English, French and German, all in UTF-8
encoding, please let me know.

Thanks,

Darren

--
Darren Cook, Software Researcher/Developer
http://dcook.org/mlsn/ (English-Japanese-German-Chinese-Arabic
open source dictionary/semantic network)
http://dcook.org/work/ (About me and my work)

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Language detection

2008-08-12T00:58:57Z

Zitat von Darren Cook <darren@...>:

> I need to write (PHP) code to detect the language of a given block of
> text. (For my purposes I want to initially distinguish between English,
> Japanese, German, Simplified Mandarin, Traditional Mandarin, Arabic,
> Korean, French) I want it to be reliable so my plan was to have a list
> of unicode points only found in each given language [1], and use that to
> return a high confidence answer. If none found, then have a list of high
> frequency words for each language [2] and use that to return a lower
> confidence answer.

http://pear.php.net/package/Text_LanguageDetect

Jan.

--
Do you need professional PHP or Horde consulting?
http://horde.org/consulting/

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

RE: Language detection

2008-08-11T15:15:17Z

> I need to write (PHP) code to detect the language of a given block of
> text.

Your proposed approach is very simplistic and probably won't be extensible
if it works at all. Usually a statistical approach is taken using groups of
characters.

In any case, ICU has this. See

http://icu-project.org/userguide/charsetDetection.html

It has both charset and language detection. This is also available via Win32
and .NET APIs in case that helps at all.

If you roll your own you might want to be aware that there are a lot of
patents in this area.

=Ed

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Language detection

2008-08-10T23:11:48Z

I need to write (PHP) code to detect the language of a given block of
text. (For my purposes I want to initially distinguish between English,
Japanese, German, Simplified Mandarin, Traditional Mandarin, Arabic,
Korean, French) I want it to be reliable so my plan was to have a list
of unicode points only found in each given language [1], and use that to
return a high confidence answer. If none found, then have a list of high
frequency words for each language [2] and use that to return a lower
confidence answer.

Like most of my i18n-related php code I'll release as BSD-license open
source. But I wondered if there already existed something I could build
on. (Or comprehensive lists of unicode points only used in certain
languages; I have some small ad hoc lists, but the more I have the more
useful the algorithm is.)

(I'm aware of letter-frequency techniques,
http://en.wikipedia.org/wiki/Letter_frequencies but haven't worked out
where that is ever more useful than word analysis?)

Darren

[1]: E.g. scharfes-s for German, katakana/hiragana for Japanese (also,
http://en.wiktionary.org/wiki/Category:Japanese-only_CJKV_Characters ).
Arabic and Korean also have unique alphabets. Accents for French.

[2]: E.g. for English "the", "be", "to", etc.
http://en.wikipedia.org/wiki/Most_common_words_in_English
Same list for German:
http://de.wikipedia.org/wiki/Liste_der_h%C3%A4ufigsten_W%C3%B6rter_der_deutschen_Sprache

--
Darren Cook, Software Researcher/Developer
http://dcook.org/mlsn/ (English-Japanese-German-Chinese-Arabic
open source dictionary/semantic network)
http://dcook.org/work/ (About me and my work)
http://darrendev.blogspot.com/ (blog on php, flash, i18n, linux, ...)

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: intl merged into core

2008-07-14T13:10:49Z

Am 14.07.2008 um 08:50 schrieb Stanislav Malyshev:

> Hi!
>
>> http://pecl.php.net/bugs/bug.php?id=14263 should be fixed asap, as
>> well as http://pecl.php.net/bugs/bug.php?id=14265.
>
> These are fixed now.

Cool!

>> http://pecl.php.net/bugs/bug.php?id=14266 might also be worthwile
>> to simply change before it gets widespread public attention :)
>
> I'll add it soon.

Well I was wondering if it wouldn't be wise to remove the current
options altogether and replace them with the array variants... hence
the comment "widespread public attention", as doing it now would only
break four or five people's code ;)

David

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: intl merged into core

2008-07-13T23:50:39Z

Hi!

> http://pecl.php.net/bugs/bug.php?id=14263 should be fixed asap, as well
> as http://pecl.php.net/bugs/bug.php?id=14265.

These are fixed now.

> http://pecl.php.net/bugs/bug.php?id=14266 might also be worthwile to
> simply change before it gets widespread public attention :)

I'll add it soon.
--
Stanislav Malyshev, Zend Software Architect
stas@... http://www.zend.com/
(408)253-8829 MSN: stas@...

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: intl merged into core

2008-07-11T14:52:16Z

Hi Stas and company,

Congratulations on reaching this milestone! I know it's been a long
process and a lot of work to get here, and having this extension will
make the development of internationalized applications in PHP much,
much easier. Yes, there's always more, but this is a huge step forward.

Thank you very much to all who contributed!
Norbert

On Jul 7, 2008, at 17:21 , Stanislav Malyshev wrote:

> Hi!
>
> Following more than a year of planning and development, and the
> release of version 1.0.0 on PECL, I have merged today the intl
> extension into PHP core. The list of its current capabilities is
> here: http://php.net/intl - please try it out and enjoy. Version
> compatible with 5.2.x starting 5.2.4 is available from PECL, and
> for 5.3 is in the CVS tree.
>
> I would like to say big thanks to everybody who contributed to this
> project, by helping to design it, write the code, test it or
> provided feedback and bug reports. Thanks a lot!
>
> With this, it is not by any means the end of the road for this
> project. There's still much to be done. We need to integrate it
> much better with PHP 6, improve support for other modules like
> DateTime, and support more capabilities of ICU, like resources,
> calendars, IDN - and you probably could think of more. And, of
> course, bugfixes :)
>
> If you have any comments, wishes, feedback etc. please tell. And if
> you encounter a bug or have specific feature request please go to
> http://pecl.php.net/package/intl and go to "Package Bugs"->"Report
> new bug".
>
> --
> Stanislav Malyshev, Zend Software Architect
> stas@... http://www.zend.com/
> (408)253-8829 MSN: stas@...
>
> --
> PHP Unicode & I18N Mailing List (http://www.php.net/)
> To unsubscribe, visit: http://www.php.net/unsub.php
>

-------------------------------------
Norbert Lindenberg
Yahoo! Internationalization Architect

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: intl merged into core

2008-07-09T05:52:06Z

I think

http://pecl.php.net/bugs/bug.php?id=14263 should be fixed asap, as
well as http://pecl.php.net/bugs/bug.php?id=14265.
http://pecl.php.net/bugs/bug.php?id=14266 might also be worthwile to
simply change before it gets widespread public attention :)

And... what about the class name prefix now? :S

David

Am 09.07.2008 um 10:48 schrieb Marcus Boerger:

> Hello Stanislav,
>
> nice work all involved.
>
> marcus
>
> Tuesday, July 8, 2008, 2:21:23 AM, you wrote:
>
>> Hi!
>
>> Following more than a year of planning and development, and the
>> release
>> of version 1.0.0 on PECL, I have merged today the intl extension into
>> PHP core. The list of its current capabilities is here:
>> http://php.net/intl - please try it out and enjoy. Version compatible
>> with 5.2.x starting 5.2.4 is available from PECL, and for 5.3 is in
>> the
>> CVS tree.
>
>> I would like to say big thanks to everybody who contributed to this
>> project, by helping to design it, write the code, test it or provided
>> feedback and bug reports. Thanks a lot!
>
>> With this, it is not by any means the end of the road for this
>> project.
>> There's still much to be done. We need to integrate it much better
>> with
>> PHP 6, improve support for other modules like DateTime, and support
>> more
>> capabilities of ICU, like resources, calendars, IDN - and you
>> probably
>> could think of more. And, of course, bugfixes :)
>
>> If you have any comments, wishes, feedback etc. please tell. And if
>> you
>> encounter a bug or have specific feature request please go to
>> http://pecl.php.net/package/intl and go to "Package Bugs"->"Report
>> new
>> bug".
>
>> --
>> Stanislav Malyshev, Zend Software Architect
>> stas@... http://www.zend.com/
>> (408)253-8829 MSN: stas@...
>
>
>
>
> --
> Best regards,
> marcus
>
>
> --
> PHP Unicode & I18N Mailing List (http://www.php.net/)
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: intl merged into core

2008-07-09T01:48:04Z

Hello Stanislav,

nice work all involved.

marcus

Tuesday, July 8, 2008, 2:21:23 AM, you wrote:

> Hi!

> Following more than a year of planning and development, and the release
> of version 1.0.0 on PECL, I have merged today the intl extension into
> PHP core. The list of its current capabilities is here:
> http://php.net/intl - please try it out and enjoy. Version compatible
> with 5.2.x starting 5.2.4 is available from PECL, and for 5.3 is in the
> CVS tree.

> I would like to say big thanks to everybody who contributed to this
> project, by helping to design it, write the code, test it or provided
> feedback and bug reports. Thanks a lot!

> With this, it is not by any means the end of the road for this project.
> There's still much to be done. We need to integrate it much better with
> PHP 6, improve support for other modules like DateTime, and support more
> capabilities of ICU, like resources, calendars, IDN - and you probably
> could think of more. And, of course, bugfixes :)

> If you have any comments, wishes, feedback etc. please tell. And if you
> encounter a bug or have specific feature request please go to
> http://pecl.php.net/package/intl and go to "Package Bugs"->"Report new
> bug".

> --
> Stanislav Malyshev, Zend Software Architect
> stas@... http://www.zend.com/
> (408)253-8829 MSN: stas@...

--
Best regards,
marcus

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

intl merged into core

2008-07-07T17:21:23Z

Hi!

Following more than a year of planning and development, and the release
of version 1.0.0 on PECL, I have merged today the intl extension into
PHP core. The list of its current capabilities is here:
http://php.net/intl - please try it out and enjoy. Version compatible
with 5.2.x starting 5.2.4 is available from PECL, and for 5.3 is in the
CVS tree.

I would like to say big thanks to everybody who contributed to this
project, by helping to design it, write the code, test it or provided
feedback and bug reports. Thanks a lot!

With this, it is not by any means the end of the road for this project.
There's still much to be done. We need to integrate it much better with
PHP 6, improve support for other modules like DateTime, and support more
capabilities of ICU, like resources, calendars, IDN - and you probably
could think of more. And, of course, bugfixes :)

If you have any comments, wishes, feedback etc. please tell. And if you
encounter a bug or have specific feature request please go to
http://pecl.php.net/package/intl and go to "Package Bugs"->"Report new
bug".

--
Stanislav Malyshev, Zend Software Architect
stas@... http://www.zend.com/
(408)253-8829 MSN: stas@...

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: intl 1.0.0RC1

2008-06-16T12:42:47Z

Hi!

Norbert Lindenberg ☮ wrote:
> I like Guillaume's proposal - use Collator and Locale as is for 5.2,
> make them Intl::Collator and Intl::Locale when namespaces are available,
> i.e., from 5.3.
>
> Would that work for everybody? How much effort is it to implement this?

If we make this exclusively, this would mean that 5.2 and 5.3/6 users
would have incompatible code. I don't think it's a good idea.

We may make Intl::Collator as alias for Collator in 5.3+, and that'd be
easy, but I'm not sure that'd what people here want.
--
Stanislav Malyshev, Zend Software Architect
stas@... http://www.zend.com/
(408)253-8829 MSN: stas@...

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

RE: intl 1.0.0RC1

2008-06-16T01:24:37Z

Are we keeping the 5.2 names in 5.3 as well or is the proposal that 5.2 apps get rewritten for 5.3?

I don't really see the need to make changes for 5.3 for the sake of satisfying this particular naming convention.
For php 6, it can make sense given the other changes being required.

tex

> -----Original Message-----
> From: Norbert Lindenberg ☮ [mailto:Norbert.Lindenberg@...]
> Sent: Friday, June 13, 2008 11:34 PM
> To: Guillaume Rossolini
> Cc: Norbert Lindenberg ☮; Darren Cook; Stanislav Malyshev;
> php-i18n@...
> Subject: Re: [PHP-I18N] intl 1.0.0RC1
>
> I like Guillaume's proposal - use Collator and Locale as is
> for 5.2, make them Intl::Collator and Intl::Locale when
> namespaces are available, i.e., from 5.3.
>
> Would that work for everybody? How much effort is it to
> implement this?
>
> Norbert
>
>
> On Jun 10, 2008, at 02:44 , Guillaume Rossolini wrote:
>
> > Hi,
> >
> > Here's my user point of view, if you don't mind.
> >
> > I think your argument is about namespaces, which will be in
> 6 but not
> > 5.2... So you might want to have IntCollator and IntLocale
> in 5.2 to
> > avoid possible collisions, but you would rather have something like
> > Intl::Collator and Intl::Locale from 5.3 and 6 onward. I guess the
> > question is:
> > can you
> > bear with Collator and Locale until we have namespaces? I
> know I can.
> >
> > Guillaume Rossolini
> >
>
> -------------------------------------
> Norbert Lindenberg
> Yahoo! Internationalization Architect
>
>
>
> --
> PHP Unicode & I18N Mailing List (http://www.php.net/) To
> unsubscribe, visit: http://www.php.net/unsub.php
>
>

Re: intl 1.0.0RC1

2008-06-16T00:20:36Z

Zitat von Lars Strojny <lars@...>:

> Hi,
>
> Am Freitag, den 13.06.2008, 23:34 -0700 schrieb Norbert Lindenberg ☮:
>> Would that work for everybody? How much effort is it to implement
>> this?
>
> +1 for this solution. Stas, what do you think?

That would work for me too.

Jan.

--
Do you need professional PHP or Horde consulting?
http://horde.org/consulting/

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: intl 1.0.0RC1

2008-06-14T12:21:35Z

Hi,

Am Freitag, den 13.06.2008, 23:34 -0700 schrieb Norbert Lindenberg ☮:
> Would that work for everybody? How much effort is it to implement
> this?

+1 for this solution. Stas, what do you think?

cu, Lars

signature.asc (852 bytes) Download Attachment

Re: intl 1.0.0RC1

2008-06-13T23:34:20Z

I like Guillaume's proposal - use Collator and Locale as is for 5.2,
make them Intl::Collator and Intl::Locale when namespaces are
available, i.e., from 5.3.

Would that work for everybody? How much effort is it to implement this?

Norbert

On Jun 10, 2008, at 02:44 , Guillaume Rossolini wrote:

> Hi,
>
> Here's my user point of view, if you don't mind.
>
> I think your argument is about namespaces, which will be in 6 but not
> 5.2... So you might want to have IntCollator and IntLocale in 5.2
> to avoid
> possible collisions, but you would rather have something like
> Intl::Collator
> and Intl::Locale from 5.3 and 6 onward. I guess the question is:
> can you
> bear with Collator and Locale until we have namespaces? I know I can.
>
> Guillaume Rossolini
>

Re: Unicode Transliteration & ICU

2008-06-10T14:37:21Z

The full text transformation support is not there yet, but there is a
simple transliteration function - str_transliterate().

-Andrei

Darren Cook wrote:

>> Does anyone know anything about transliteration in PHP6. I have noticed
>> that PHP has an ICU extension. ICU has a very comprehensive
>> transliteration/transform module that is not documented.
>
> It is documented here:
> http://www.icu-project.org/userguide/Transform.html
>
> (But I don't think Transform in the PHP intl extension?)
>
> No Arabic support, which is the transliteration code I'm working on at
> the moment (in native PHP; it'll be in the next (MIT open-source) fclib
> library release).
>
> I'm also not sure the Japanese one will be useful, as it sounds like
> they do things slightly differently from normal Hepburn romaji to allow
> the conversions to be reversible. (which also suggests they don't
> transliterate the katakana long vowel but keep it as a hyphen??)
>
>> Currently I am using iconv and a PLEC extension to transliterate, but
>> they area neither comprehensive or widely supported.
>
> Which languages are you trying to transliterate for?
>
> Darren
>
>

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: intl 1.0.0RC1

2008-06-10T04:29:11Z

Hi!

> I don't. Maybe you do. Or, rather, Zend does. So the Zend Framework can
> use it, I guess.

Do you seriously argue that I have to redesign the extension, throwing
out all the original requirements for all the participating parties in
the project and all people who contributed to it and already use it -
and make it only what *you* personally want and need? Please tell me you
were joking.
--
Stanislav Malyshev, Zend Software Architect
stas@... http://www.zend.com/
(408)253-8829 MSN: stas@...

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

RE: intl 1.0.0RC1

2008-06-10T04:28:16Z

It wasn't Zend. Other companies wanted it in 5.2.
tex

> -----Original Message-----
> From: David Zülke [mailto:dz@...]
> Sent: Tuesday, June 10, 2008 4:14 AM
> To: Stanislav Malyshev
> Cc: Darren Cook; php-i18n@...
> Subject: Re: [PHP-I18N] intl 1.0.0RC1
>
> Am 10.06.2008 um 09:02 schrieb Stanislav Malyshev:
>
> > Hi!
> >
> >> What are the good reasons for not changing the names?
> >
> > This was already discussed on the list, so reading the
> archives would
> > help. But OK, here it goes again:
> >
> > 1. We want the extension to be in 5.2 to 6.x
>
> I don't. Maybe you do. Or, rather, Zend does. So the Zend
> Framework can use it, I guess.
>
> The reasonable and responsible thing to do would be making it
> 5.3+ only.
>
>
> David
>
> --
> PHP Unicode & I18N Mailing List (http://www.php.net/) To
> unsubscribe, visit: http://www.php.net/unsub.php
>
>

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: intl 1.0.0RC1

2008-06-10T04:27:13Z

Hi!

> What kind of attitude is that? You personally think it sucks, so it's
> not going to happen?

Not only myself, but myself too. Why do you think that your opinion
matters but mine and everybody's that agrees with me doesn't?

> It's not artificial or awkward at all, it's poor man's namespacing that

Poor man's namespacing is not exactly what we want. I just call you to
think wider than narrow focus on "all names should start with the same
letter".

> we use in PHP since ages, because we don't have namespacing yet.
> Everybody is using it, especially in PHP internally.

No, not everybody. Exception class is not called StandardException, and
ArrayIterator is not called SplArrayIterator, Directory is not called
StandardDirectory, etc.

> Maybe because you DO live in the ivory tower and have the luxury to be

I'm with PHP since 1998 at least, maybe earlier (hard to remember 10
years back). I don't think it's especially smart to try and accuse me in
not knowing what happens with real-world projects in PHP.

> ext/intl, chances are high they have to rewrite their whole code,
> because they happen to use Collator or Locale already. Not very unlikely

And they should. If they are going to use ICU extension, why would they
need to use their own Collator? It makes little sense to both have your
own collators and ICU collator and use those in the same application.
--
Stanislav Malyshev, Zend Software Architect
stas@... http://www.zend.com/
(408)253-8829 MSN: stas@...

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: intl 1.0.0RC1

2008-06-10T04:14:08Z

Am 10.06.2008 um 09:02 schrieb Stanislav Malyshev:

> Hi!
>
>> What are the good reasons for not changing the names?
>
> This was already discussed on the list, so reading the archives
> would help. But OK, here it goes again:
>
> 1. We want the extension to be in 5.2 to 6.x

I don't. Maybe you do. Or, rather, Zend does. So the Zend Framework
can use it, I guess.

The reasonable and responsible thing to do would be making it 5.3+ only.

David

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

RE: intl 1.0.0RC1

2008-06-10T04:07:04Z

1) I don't see any reason for name calling or personal attacks.
Besides, Stas's ivory tower is more of a penthouse than a tower.

2) I don't understand your argument that it takes a major code rewrite to use the collator.
That has not been my experience.
Why do you make that claim or what did you have to do to introduce date class that is relevant?

tex

> -----Original Message-----
> From: Jan Schneider [mailto:jan@...]
> Sent: Tuesday, June 10, 2008 3:33 AM
> To: php-i18n@...
> Subject: Re: [PHP-I18N] intl 1.0.0RC1
>
> Zitat von "Stanislav Malyshev" <stas@...>:
>
> > Hi!
> >
> >> Yes. Why can't it be IntlCollator in both 5.x and 6.x? In
> other words
> >> (unless I'm missing something) compatibility between 5.x and 6.x
> >> doesn't affect the naming choice.
> >
> > The thing is that having collators in PHP called IntlCollator and
> > locales in PHP called IntlLocale sucks. It looks so awkward and
> > artificial.
>
> What kind of attitude is that? You personally think it sucks,
> so it's not going to happen?
> It's not artificial or awkward at all, it's poor man's
> namespacing that we use in PHP since ages, because we don't
> have namespacing yet.
> Everybody is using it, especially in PHP internally.
>
> >> I'm advocating uniformity for the sake of usability :-).
> (for reasons
> >
> > I don't see how IntlLocale is more usable than Locale. If you'd be
> > total stranger to all the things discussed here and I'd tell you -
> > there's a thing called "locale", what do you think would be
> the name
> > of the class that works with it - would you answer "Locale" or
> > "IntlLocale"?
>
> If I would be a stranger, I wouldn't try guessing class names
> at all.
> Instead I would look into the manual. And I would easily spot
> that in the manual: extension == manual chapter ==
> function/class prefix. THAT is usability.
>
> >> already given). (I'm a freelance programmer, working on real-world
> >> i18n-related code, not an ivory tower academic.)
> >
> > OK, great - and you'd prefer all your functions and classes to have
> > prefix Intl because of that? I know I wouldn't. I would
> like to have
> > class named Locale to work with locales.
>
> Maybe because you DO live in the ivory tower and have the
> luxury to be able to start each project from scratch with the
> newest and freshest code.
> Nobody else does. Anybody in the real-world is dealing with
> large portions of existing code, slowly integrating new
> features where see fit.
> If anybody who is currently using i18n in his code (and only
> those are interested in ext/intl anyway), wants to use
> functionality from ext/intl, chances are high they have to
> rewrite their whole code, because they happen to use Collator
> or Locale already. Not very unlikely in i18n related code,
> isn't it? This is not different from trying introduce the
> Date class a few years back.
>
> Jan.
>
> --
> Do you need professional PHP or Horde consulting?
> http://horde.org/consulting/
>
>
> --
> PHP Unicode & I18N Mailing List (http://www.php.net/)
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: intl 1.0.0RC1

2008-06-10T03:33:13Z

Zitat von "Stanislav Malyshev" <stas@...>:

> Hi!
>
>> Yes. Why can't it be IntlCollator in both 5.x and 6.x? In other words
>> (unless I'm missing something) compatibility between 5.x and 6.x doesn't
>> affect the naming choice.
>
> The thing is that having collators in PHP called IntlCollator and
> locales in PHP called IntlLocale sucks. It looks so awkward and
> artificial.

What kind of attitude is that? You personally think it sucks, so it's
not going to happen?
It's not artificial or awkward at all, it's poor man's namespacing
that we use in PHP since ages, because we don't have namespacing yet.
Everybody is using it, especially in PHP internally.

>> I'm advocating uniformity for the sake of usability :-). (for reasons
>
> I don't see how IntlLocale is more usable than Locale. If you'd be
> total stranger to all the things discussed here and I'd tell you -
> there's a thing called "locale", what do you think would be the name
> of the class that works with it - would you answer "Locale" or
> "IntlLocale"?

If I would be a stranger, I wouldn't try guessing class names at all.
Instead I would look into the manual. And I would easily spot that in
the manual: extension == manual chapter == function/class prefix. THAT
is usability.

>> already given). (I'm a freelance programmer, working on real-world
>> i18n-related code, not an ivory tower academic.)
>
> OK, great - and you'd prefer all your functions and classes to have
> prefix Intl because of that? I know I wouldn't. I would like to have
> class named Locale to work with locales.

Maybe because you DO live in the ivory tower and have the luxury to be
able to start each project from scratch with the newest and freshest
code.
Nobody else does. Anybody in the real-world is dealing with large
portions of existing code, slowly integrating new features where see
fit.
If anybody who is currently using i18n in his code (and only those are
interested in ext/intl anyway), wants to use functionality from
ext/intl, chances are high they have to rewrite their whole code,
because they happen to use Collator or Locale already. Not very
unlikely in i18n related code, isn't it? This is not different from
trying introduce the Date class a few years back.

Jan.

--
Do you need professional PHP or Horde consulting?
http://horde.org/consulting/

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: intl 1.0.0RC1

2008-06-10T02:44:30Z

Hi,

Here's my user point of view, if you don't mind.

I think your argument is about namespaces, which will be in 6 but not
5.2... So you might want to have IntCollator and IntLocale in 5.2 to avoid
possible collisions, but you would rather have something like Intl::Collator
and Intl::Locale from 5.3 and 6 onward. I guess the question is: can you
bear with Collator and Locale until we have namespaces? I know I can.

Guillaume Rossolini

On Tue, Jun 10, 2008 at 11:17 AM, Stanislav Malyshev <stas@...> wrote:

> Hi!
>
> Yes. Why can't it be IntlCollator in both 5.x and 6.x? In other words
>> (unless I'm missing something) compatibility between 5.x and 6.x doesn't
>> affect the naming choice.
>>
>
> The thing is that having collators in PHP called IntlCollator and locales
> in PHP called IntlLocale sucks. It looks so awkward and artificial.
>
> I'm advocating uniformity for the sake of usability :-). (for reasons
>>
>
> I don't see how IntlLocale is more usable than Locale. If you'd be total
> stranger to all the things discussed here and I'd tell you - there's a thing
> called "locale", what do you think would be the name of the class that works
> with it - would you answer "Locale" or "IntlLocale"?
>
> already given). (I'm a freelance programmer, working on real-world
>> i18n-related code, not an ivory tower academic.)
>>
>
> OK, great - and you'd prefer all your functions and classes to have prefix
> Intl because of that? I know I wouldn't. I would like to have class named
> Locale to work with locales.
> --
> Stanislav Malyshev, Zend Software Architect
> stas@... http://www.zend.com/
> (408)253-8829 MSN: stas@...
>
> --
> PHP Unicode & I18N Mailing List (http://www.php.net/)
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>

Re: intl 1.0.0RC1

2008-06-10T02:17:00Z

Hi!

> Yes. Why can't it be IntlCollator in both 5.x and 6.x? In other words
> (unless I'm missing something) compatibility between 5.x and 6.x doesn't
> affect the naming choice.

The thing is that having collators in PHP called IntlCollator and
locales in PHP called IntlLocale sucks. It looks so awkward and artificial.

> I'm advocating uniformity for the sake of usability :-). (for reasons

I don't see how IntlLocale is more usable than Locale. If you'd be total
stranger to all the things discussed here and I'd tell you - there's a
thing called "locale", what do you think would be the name of the class
that works with it - would you answer "Locale" or "IntlLocale"?

> already given). (I'm a freelance programmer, working on real-world
> i18n-related code, not an ivory tower academic.)

OK, great - and you'd prefer all your functions and classes to have
prefix Intl because of that? I know I wouldn't. I would like to have
class named Locale to work with locales.
--
Stanislav Malyshev, Zend Software Architect
stas@... http://www.zend.com/
(408)253-8829 MSN: stas@...

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: intl 1.0.0RC1

2008-06-10T01:35:14Z

>> I don't understand why the choice between IntlCollator and Collator
>> affects compatibility between 5.x and 6.x. Can you explain this further?
>
> Because if you want Collator in 6, and you want same code to run both
> 5.x and 6.x, then it's Collator in 5.x. If A==B, then B==A.

Yes. Why can't it be IntlCollator in both 5.x and 6.x? In other words
(unless I'm missing something) compatibility between 5.x and 6.x doesn't
affect the naming choice.

> I however think that here usability is
> more important than uniformity for the sake of uniformity.

I'm advocating uniformity for the sake of usability :-). (for reasons
already given). (I'm a freelance programmer, working on real-world
i18n-related code, not an ivory tower academic.)

Darren

--
Darren Cook, Software Researcher/Developer
http://dcook.org/mlsn/ (English-Japanese-German-Chinese-Arabic
open source dictionary/semantic network)
http://dcook.org/work/ (About me and my work)
http://dcook.org/work/charts/ (My flash charting demos)

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: intl 1.0.0RC1

2008-06-10T00:46:08Z

Hi!

> I don't understand why the choice between IntlCollator and Collator
> affects compatibility between 5.x and 6.x. Can you explain this further?

Because if you want Collator in 6, and you want same code to run both
5.x and 6.x, then it's Collator in 5.x. If A==B, then B==A.

> I don't think IntlCollator is either long, or weird. Actually I feel

Well, of course it would be strange for you to say "I think it's weird
but I still advocate it's usage". I however think that here usability is
more important than uniformity for the sake of uniformity. There would
be no problem for people to know which class name to use for collator.

> prefixing the extension name aids code clarity (as it tells me which
> extension the class belongs to, and I can search on "Intl" to find all
> places I'm using anything from the extension).

Now, why would it be important to you which directory contains the
source file that produced your function? So important that you would
want to sacrifice usability for it? I think most users couldn't care
less once it works for them.

> * Following PHP naming guidelines

PHP naming guidelines do not say it must always be extension name. It
has it has to be by functional group.

> * Avoid clashes with existing classes (someone having written a class
> called Collator is more likely than someone having written one called
> IntlCollator)

That's because IntlCollator is so awkward a name nobody would use it in
the code unless forced to. Why would I want to force all PHP users to
use awkward names? To show off how "consistent" we are? To hell with the
consistency if the consistency means being consistently hard to use.
--
Stanislav Malyshev, Zend Software Architect
stas@... http://www.zend.com/
(408)253-8829 MSN: stas@...

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: intl 1.0.0RC1

2008-06-10T00:27:44Z

>> What are the good reasons for not changing the names?
>
> 1. We want the extension to be in 5.2 to 6.x
> 2. We want PHP source code compatibility between 5.x and 6.x code

I don't understand why the choice between IntlCollator and Collator
affects compatibility between 5.x and 6.x. Can you explain this further?

> 3. We want PHP 6 users use class Collator when they need collator, not
> some weird long name, same with Locale etc.

I don't think IntlCollator is either long, or weird. Actually I feel
prefixing the extension name aids code clarity (as it tells me which
extension the class belongs to, and I can search on "Intl" to find all
places I'm using anything from the extension).

> 4. intl extension, unlike many others, do not have one functional
> module, but multiple ones, related only by underlying library.

I think the users will see it as a single extension:
a) All the classes are based on ICU classes of the same name;
b) I type "pecl install intl"; (I cannot just install one part)
c) All classes are for i18n, which most people will see as a single task.

The arguments for prefixing all classes with "Intl" are:
* Consistency with IntlDateFormatter

* Following PHP naming guidelines

* Avoid clashes with existing classes (someone having written a class
called Collator is more likely than someone having written one called
IntlCollator)

(I'd also add "code clarity", but that is more of a subjective, personal
thing.)

Darren

--
Darren Cook, Software Researcher/Developer
http://dcook.org/mlsn/ (English-Japanese-German-Chinese-Arabic
open source dictionary/semantic network)
http://dcook.org/work/ (About me and my work)
http://dcook.org/work/charts/ (My flash charting demos)

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: intl 1.0.0RC1

2008-06-10T00:02:51Z

Hi!

> What are the good reasons for not changing the names?

This was already discussed on the list, so reading the archives would
help. But OK, here it goes again:

1. We want the extension to be in 5.2 to 6.x
2. We want PHP source code compatibility between 5.x and 6.x code
3. We want PHP 6 users use class Collator when they need collator, not
some weird long name, same with Locale etc.
4. intl extension, unlike many others, do not have one functional
module, but multiple ones, related only by underlying library. Thus
prefixing all modules with extension name makes as much sense as
prefixing all functions using libc or libm with common prefix. Each
functional module, of course, does have its own prefix.
--
Stanislav Malyshev, Zend Software Architect
stas@... http://www.zend.com/
(408)253-8829 MSN: stas@...

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Unicode Transliteration & ICU

2008-06-09T16:05:21Z

> Thanks for your very informative reply, Darren. I guess that maybe
> PHP6 has implemented this from ICU. I was told by a PECL developer
> that there is something in PHP6 but he didn't elaborate.

The intl extension:
http://pecl.php.net/package/intl/
You can use it from php 5.2.4 onwards (or 5.2.3 with some
modifications). Also see php|a magazine,Mar 2008.

> The one I am using at the moment is:
> http://derickrethans.nl/translit.php

Thanks, I'd not heard of that. The Chinese conversion seems to be done
by a huge lookup table, which is interesting.

> Your work sounds interesting. I have downloaded your library, but am
> having trouble navigating through it.

Yes, fclib is quite informal :-).

> What files should I be looking at for the transliteration?

utf8.inc, e.g. fclib_katakana_to_hepburn_romaji().
See also my articles in php|a, Aug and Sep 2007.

> I would like to be able to transliterate absolutely everything in
> unicode. I have no idea if that is unreasonable as I am just getting
> into character sets. I want them to make a bulletproof string to url
> function for search engine friendliness and I also believe it is not
> really a good thing to have high unicode in the url. For example
>
> Héllo Thìs is a URL Ælfred => hello-this-is-a-url-aelfred

If URLs are the only concern I think I'd do this using urlencode(). What
does a transliteration approach gain you?

> Another thing that I started working on was a strtoupper, strtolower
> and ucfirst function for cyrillic and anything else that can be upper
> and lower case. However, being new to character set and unicode I am
> having trouble converting the hex codes to actual character and
> cannot get preg_replace to work with high unicode.

See fclib_utf8_chr() and uniord() in utf8.inc, which are UTF-8 versions
of PHP's chr() and ord() functions.

I'm not sure about using preg as I'm not sure I've done it that way. The
manual http://jp2.php.net/manual/en/regexp.reference.php has a section
on unicode, but still doesn't seem to support giving a 4-character hex
code. Perhaps you just use \x twice in a row? E.g.
\x06\x28
to match U+0628 (Arabic BEH).

Darren

--
Darren Cook, Software Researcher/Developer
http://dcook.org/mlsn/ (English-Japanese-German-Chinese-Arabic
open source dictionary/semantic network)
http://dcook.org/work/ (About me and my work)
http://dcook.org/work/charts/ (My flash charting demos)

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Unicode Transliteration & ICU

2008-06-09T02:55:00Z

Hi!

> It is documented here:
> http://www.icu-project.org/userguide/Transform.html
>
> (But I don't think Transform in the PHP intl extension?)

No, not yet.
--
Stanislav Malyshev, Zend Software Architect
stas@... http://www.zend.com/
(408)253-8829 MSN: stas@...

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Unicode Transliteration & ICU

2008-06-08T16:47:18Z

> Does anyone know anything about transliteration in PHP6. I have noticed
> that PHP has an ICU extension. ICU has a very comprehensive
> transliteration/transform module that is not documented.

It is documented here:
http://www.icu-project.org/userguide/Transform.html

(But I don't think Transform in the PHP intl extension?)

No Arabic support, which is the transliteration code I'm working on at
the moment (in native PHP; it'll be in the next (MIT open-source) fclib
library release).

I'm also not sure the Japanese one will be useful, as it sounds like
they do things slightly differently from normal Hepburn romaji to allow
the conversions to be reversible. (which also suggests they don't
transliterate the katakana long vowel but keep it as a hyphen??)

> Currently I am using iconv and a PLEC extension to transliterate, but
> they area neither comprehensive or widely supported.

Which languages are you trying to transliterate for?

Darren

--
Darren Cook, Software Researcher/Developer
http://dcook.org/mlsn/ (English-Japanese-German-Chinese-Arabic
open source dictionary/semantic network)
http://dcook.org/work/ (About me and my work)
http://dcook.org/work/charts/ (My flash charting demos)

--
PHP Unicode & I18N Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php