Working with encoded values in more recent versions of ZPT

[Charlie Clark-2]

Hi,

I've noticed in Zope 2.10 that the behaviour of ZPT with regard to encoded  
values returned by a database has changed which leads to errors when using  
string: and non-ascii characters.

ie.

<tal:content tal:content="string: ${result/firstname} ${result/surname}"  
/> will raise UnicodeDecodeError
for result = {'firstname':'charlie', 'surname':'Düsseldorf'}
but not if Python string formatting is used
<tal:content tal:content="python: '%s %s' % (result['firstname'],  
result['surname'])" />

Is this the only workaround apart from decoding all return values?

Charlie
--
Charlie Clark
eGenix.com

Professional Python Services directly from the Source
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::

     eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611
_______________________________________________
ZPT mailing list
ZPT@...
http://mail.zope.org/mailman/listinfo/zpt

[Dieter Maurer]

Charlie Clark wrote at 2007-8-20 14:41 +0200:
>I've noticed in Zope 2.10 that the behaviour of ZPT with regard to encoded  
>values returned by a database has changed which leads to errors when using  
>string: and non-ascii characters.

This is caused by switching in the Zope3 (unicode based) ZPT implementation
for Zope 2.10.

The new ZPT implementaion required serveral minor releases before
it became stable. Maybe, you do not yet have the most current version.

Andreas tried hard to control the inevitable string/unicode integration
problems. Maybe, you read the available documentation to find out
about your choices.



--
Dieter
_______________________________________________
ZPT mailing list
ZPT@...
http://mail.zope.org/mailman/listinfo/zpt

[Charlie Clark-2]

Am 20.08.2007, 20:53 Uhr, schrieb Dieter Maurer <dieter@...>:

> This is caused by switching in the Zope3 (unicode based) ZPT  
> implementation
> for Zope 2.10.

Yes, I know the cause and can see it in the tracebacks. It's just a bit  
weird that string formatting works fine but TALES string: expressions  
treat everything as ASCII. So maybe I need to look at the underlying  
implementation of ZPT.

> The new ZPT implementaion required serveral minor releases before
> it became stable. Maybe, you do not yet have the most current version.

Zope 2.10.3-final. There has been another version since then but the  
changelog makes no reference to ZPT.

> Andreas tried hard to control the inevitable string/unicode integration
> problems. Maybe, you read the available documentation to find out
> about your choices.

Have the ZPT docs been updated?

Charlie
--
Charlie Clark
eGenix.com

Professional Python Services directly from the Source
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::

     eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611
_______________________________________________
ZPT mailing list
ZPT@...
http://mail.zope.org/mailman/listinfo/zpt

[Andreas Jung-5]

--On 20. August 2007 21:04:34 +0200 Charlie Clark <charlie@...>
wrote:

> Am 20.08.2007, 20:53 Uhr, schrieb Dieter Maurer <dieter@...>:
>
>> This is caused by switching in the Zope3 (unicode based) ZPT
>> implementation
>> for Zope 2.10.
>
> Yes, I know the cause and can see it in the tracebacks. It's just a bit
> weird that string formatting works fine but TALES string: expressions
> treat everything as ASCII. So maybe I need to look at the underlying
> implementation of ZPT.
>
>> The new ZPT implementaion required serveral minor releases before
>> it became stable. Maybe, you do not yet have the most current version.
>
> Zope 2.10.3-final. There has been another version since then but the
> changelog makes no reference to ZPT.
>
>> Andreas tried hard to control the inevitable string/unicode integration
>> problems. Maybe, you read the available documentation to find out
>> about your choices.
>
> Have the ZPT docs been updated?

If you think this is a bug, please put it into a unittest (ZPT has
enough related tests which can serve as an example).

Andreas



_______________________________________________
ZPT mailing list
ZPT@...
http://mail.zope.org/mailman/listinfo/zpt

[Charlie Clark-2]

Am 20.08.2007, 21:12 Uhr, schrieb Andreas Jung <lists@...>:

> If you think this is a bug, please put it into a unittest (ZPT has
> enough related tests which can serve as an example).

Will do. I've noticed that the newer templates have explicit  
output-encodings and I thought that this might be related but it seems  
not. Currently a major issue for me for migrating a site to Zope 2.10

FWIW this is current traceback

Traceback (innermost last):
   Module ZPublisher.Publish, line 119, in publish
   Module ZPublisher.mapply, line 88, in mapply
   Module ZPublisher.Publish, line 42, in call_object
   Module Shared.DC.Scripts.Bindings, line 313, in __call__
   Module Shared.DC.Scripts.Bindings, line 350, in _bindAndExec
   Module Products.PageTemplates.ZopePageTemplate, line 331, in _exec
   Module Products.PageTemplates.ZopePageTemplate, line 427, in pt_render
   Module Products.PageTemplates.PageTemplate, line 89, in pt_render
   Module zope.pagetemplate.pagetemplate, line 117, in pt_render
   Module zope.tal.talinterpreter, line 271, in __call__
   Module zope.tal.talinterpreter, line 346, in interpret
   Module zope.tal.talinterpreter, line 861, in do_defineMacro
   Module zope.tal.talinterpreter, line 346, in interpret
   Module zope.tal.talinterpreter, line 824, in do_loop_tal
   Module zope.tal.talinterpreter, line 346, in interpret
   Module zope.tal.talinterpreter, line 534, in do_optTag_tal
   Module zope.tal.talinterpreter, line 516, in no_tag
   Module zope.tal.talinterpreter, line 346, in interpret
   Module zope.tal.talinterpreter, line 824, in do_loop_tal
   Module zope.tal.talinterpreter, line 346, in interpret
   Module zope.tal.talinterpreter, line 534, in do_optTag_tal
   Module zope.tal.talinterpreter, line 516, in no_tag
   Module zope.tal.talinterpreter, line 346, in interpret
   Module zope.tal.talinterpreter, line 754, in do_insertStructure_tal
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 4:  
ordinal not in range(128)


Charlie
--
Charlie Clark
eGenix.com

Professional Python Services directly from the Source
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::

     eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611
_______________________________________________
ZPT mailing list
ZPT@...
http://mail.zope.org/mailman/listinfo/zpt

[Charlie Clark-2]

Am 21.08.2007, 05:27 Uhr, schrieb Andreas Jung <lists@...>:

> See
> <http://www.zope.org/Collectors/Zope/2339>

Thanks. So this means I don't need to write any additional tests? Glad to  
know you've resolved it.

Charlie

--
Charlie Clark
eGenix.com

Professional Python Services directly from the Source
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::

     eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611
_______________________________________________
ZPT mailing list
ZPT@...
http://mail.zope.org/mailman/listinfo/zpt