On __findattr__, __getattr__ and __getattribute__

[Leo Soto M.]

Here is a short history of __findattr__, __getattr__ and
__getattribute__, and why PyObject#object___getattribute__ can't
directly call PyObject#__getattr__. I learned this in the hard way,
but that's what you get when you attempt to modify a core component of
the runtime without understanding all this:

What are __findattr__ and __getattr__?

  - PyObject#__findattr__: Has the semantics of Python's
__getattribute__ (that is, is called before looking on the object
dict), but returns null instead of throwing Py.AttributeError when the
lookup fails.

  - PyObject#__getattr__: Same as __findattr__ but *does* throws
AttributeError and never returns null. Currently it is implemented in
terms of __findattr__, in the obvious way. PyObject subclasses are
expected to override __findattr__.

When are they used?

 - From Java code wanting to access attributes of PyObjects.
 - When you do getattr(foo, bar) on Python code [and I'd bet that when
you do foo.bar too, but I've not looked at the exact locations].

And what about __getattribute__?

 - There are methods *exposed* through the __getattribute__ descriptor
which get called when you do foo.__getattribute__(bar) or
type(foo).__getattribute__(bar)

But who uses that syntax?

 - Typical use is in custom implementations of __getattribute__:

    class Foo(object):
        def __getattribute__(self, attr):
            if attr == "bar":
                return "bar!"
            else:
                Foo.__getattribute__(self, attr)

 - And who knows who other, what's important is that it should work!

So foo.__getattribute__('bar') should always be the same as
getattribute(foo, 'bar') for new-style classes?

  - Yes. That's why every new-style PyObject subclass overriding
__findattr__ should also expose __getattribute__. And both should
follow exactly the same logic.

Cool. So what if we avoid all that repetition by making
object___getattribute__ call __getattr__?

  - Does *not* work. By doing that object.__getattribute__(x, y) would
call x.object___getattribute__(y) and would end calling
x.__findattr__(y) which would invoke the *overriden* __getattr__
logic, not the *original* object's __getattr__ logic. That's quite bad
for derived types with custom __getattribute__ implementation (such as
the "Foo" class shown above), because this means going into an
infinite loop.

Doh.

What *seems* reasonable now is that each overriden __findattr__()
simply call the exposed __getattribute__ and not the other way around.
If the lookup could be delegated to super, then the __getattribute__
method should invoke super.__findattr__().

--
Leo Soto M.
http://blog.leosoto.com

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Jython-dev mailing list
Jython-dev@...
https://lists.sourceforge.net/lists/listinfo/jython-dev

[Jim Baker]

foo.bar is compiled to call __getattr__ if invoked in a get context; see CodeCompiler#visitAttribute

On Mon, Jun 23, 2008 at 2:44 PM, Leo Soto M. <leo.soto@...> wrote:

Here is a short history of __findattr__, __getattr__ and
__getattribute__, and why PyObject#object___getattribute__ can't
directly call PyObject#__getattr__. I learned this in the hard way,
but that's what you get when you attempt to modify a core component of
the runtime without understanding all this:

What are __findattr__ and __getattr__?

 - PyObject#__findattr__: Has the semantics of Python's
__getattribute__ (that is, is called before looking on the object
dict), but returns null instead of throwing Py.AttributeError when the
lookup fails.

 - PyObject#__getattr__: Same as __findattr__ but *does* throws
AttributeError and never returns null. Currently it is implemented in
terms of __findattr__, in the obvious way. PyObject subclasses are
expected to override __findattr__.

When are they used?

 - From Java code wanting to access attributes of PyObjects.
 - When you do getattr(foo, bar) on Python code [and I'd bet that when
you do foo.bar too, but I've not looked at the exact locations].

And what about __getattribute__?

 - There are methods *exposed* through the __getattribute__ descriptor
which get called when you do foo.__getattribute__(bar) or
type(foo).__getattribute__(bar)

But who uses that syntax?

 - Typical use is in custom implementations of __getattribute__:

   class Foo(object):
       def __getattribute__(self, attr):
           if attr == "bar":
               return "bar!"
           else:
               Foo.__getattribute__(self, attr)

 - And who knows who other, what's important is that it should work!

So foo.__getattribute__('bar') should always be the same as
getattribute(foo, 'bar') for new-style classes?

 - Yes. That's why every new-style PyObject subclass overriding
__findattr__ should also expose __getattribute__. And both should
follow exactly the same logic.

Cool. So what if we avoid all that repetition by making
object___getattribute__ call __getattr__?

 - Does *not* work. By doing that object.__getattribute__(x, y) would
call x.object___getattribute__(y) and would end calling
x.__findattr__(y) which would invoke the *overriden* __getattr__
logic, not the *original* object's __getattr__ logic. That's quite bad
for derived types with custom __getattribute__ implementation (such as
the "Foo" class shown above), because this means going into an
infinite loop.

Doh.

What *seems* reasonable now is that each overriden __findattr__()
simply call the exposed __getattribute__ and not the other way around.
If the lookup could be delegated to super, then the __getattribute__
method should invoke super.__findattr__().

--
Leo Soto M.
http://blog.leosoto.com

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Jython-dev mailing list
Jython-dev@...
https://lists.sourceforge.net/lists/listinfo/jython-dev

--
Jim Baker
jbaker@...
-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Jython-dev mailing list
Jython-dev@...
https://lists.sourceforge.net/lists/listinfo/jython-dev

[Alan Kennedy-2]

[Leo]
> Here is a short history of __findattr__, __getattr__ and
> __getattribute__, and why PyObject#object___getattribute__ can't
> directly call PyObject#__getattr__. I learned this in the hard way,
> but that's what you get when you attempt to modify a core component of
> the runtime without understanding all this:

I found this document to be most enlightening.

http://www.python.org/doc/2.2.1/whatsnew/sect-rellinks.html

Sample excerpt: """
A fair number of sophisticated Python classes define hooks for
attribute access using __getattr__; most commonly this is done for
convenience, to make code more readable by automatically mapping an
attribute access such as obj.parent into a method call such as
obj.get_parent(). Python 2.2 adds some new ways of controlling
attribute access.

First, __getattr__(attr_name) is still supported by new-style classes,
and nothing about it has changed. As before, it will be called when an
attempt is made to access obj.foo and no attribute named "foo" is
found in the instance's dictionary.

New-style classes also support a new method,
__getattribute__(attr_name). The difference between the two methods is
that __getattribute__ is always called whenever any attribute is
accessed, while the old __getattr__ is only called if "foo" isn't
found in the instance's dictionary.
"""

A|ll the best,

Alan.

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Jython-dev mailing list
Jython-dev@...
https://lists.sourceforge.net/lists/listinfo/jython-dev

[Leo Soto M.]

On Tue, Jun 24, 2008 at 5:39 AM, Alan Kennedy <jython-dev@...> wrote:

> [Leo]
>> Here is a short history of __findattr__, __getattr__ and
>> __getattribute__, and why PyObject#object___getattribute__ can't
>> directly call PyObject#__getattr__. I learned this in the hard way,
>> but that's what you get when you attempt to modify a core component of
>> the runtime without understanding all this:
>
> I found this document to be most enlightening.
>
> http://www.python.org/doc/2.2.1/whatsnew/sect-rellinks.html
>
> Sample excerpt: """
> A fair number of sophisticated Python classes define hooks for
> attribute access using __getattr__; most commonly this is done for
> convenience, to make code more readable by automatically mapping an
> attribute access such as obj.parent into a method call such as
> obj.get_parent(). Python 2.2 adds some new ways of controlling
> attribute access.
>
> First, __getattr__(attr_name) is still supported by new-style classes, [...]

Yeah, but I was talking about PyObject#__getattr__, not Python's __getattr__ :).

And it's counter-intuitive, but PyObject#__getattr__ has the semantics
of __getattribute__. Historical reasons, I'd bet.

--
Leo Soto M.
http://blog.leosoto.com

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Jython-dev mailing list
Jython-dev@...
https://lists.sourceforge.net/lists/listinfo/jython-dev

[Samuele Pedroni-3]

> Yeah, but I was talking about PyObject#__getattr__, not Python's __getattr__ :).
>
> And it's counter-intuitive, but PyObject#__getattr__ has the semantics
> of __getattribute__. Historical reasons, I'd bet.
>
>  
__getattr__ is what is called to implement the . operator for attribute
access. It existed with that name before
I got involved with Jython. Considering the naming of other method with
similar intent (_eq, _add) _loadattr or something
would have been a better name. It would not unreasonable to do that
renaming except for backward compatibility issues but
see the next point.

__findattr__ is really the part of the public API that someone
interacting from Java should use to do attribute access,
that's what the Jython code base usually does itself, the null return is
usually more convenient and somewhat faster.

They are layered as they are because is saner performance-wise to have
the null returning version be the foundation for the raising version.

__getattribute__ came later with new-style classes, the main point is
that it needs to exists as a descriptor in the type dictionary of the
required types.

__findattr__ was intended to be the method where to implement the
attribute access semantics of each object type, the other are derivatives.

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Jython-dev mailing list
Jython-dev@...
https://lists.sourceforge.net/lists/listinfo/jython-dev

[Leo Soto M.]

On 6/25/08, Samuele Pedroni <pedronis@...> wrote:

Hi Samuele, good to see you continue reading Jython-dev!

[...]

>  __findattr__ is really the part of the public API that someone interacting
> from Java should use to do attribute access,
>  that's what the Jython code base usually does itself, the null return is
> usually more convenient and somewhat faster.
>
>  They are layered as they are because is saner performance-wise to have the
> null returning version be the foundation for the raising version.
>
>  __getattribute__ came later with new-style classes, the main point is that
> it needs to exists as a descriptor in the type dictionary of the required
> types.
>
>  __findattr__ was intended to be the method where to implement the attribute
> access semantics of each object type, the other are derivatives.

Well, I'm now experimenting with changing this. The __findattr__ API
(null return value when no attribute is found) doesn't let inherited
classes throw their own, customs, AttributeErrors.  It's a very
obscure and marginal use case. Following the rule which says that
every obscure CPython feature is used by at least one prominent
codebase, Django has descriptors which throw slightly customized
AttributeError instances. AFAICS, there is no way to propagate them
without allowing PyObject subclasses to override __getattr__ (The
option of raising AttributeError from *Derived#__findattr__ sounds
like breaking the __findattr__ API to me).

As allowing PyObject subclasses to override __getattr__ will bring a
confusing interaction between __findattr__ and __getattr__, I think it
is better to make __getattr__  overridable and implement __findattr__
in terms of __getattr__ (to keep the public API).

A very similar situations happens with __finditem__ , __getitem__ and
custom KeyErrors.
--
Leo Soto M.
http://blog.leosoto.com

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Jython-dev mailing list
Jython-dev@...
https://lists.sourceforge.net/lists/listinfo/jython-dev

[Samuele Pedroni-3]

Leo Soto M. wrote:

> On 6/25/08, Samuele Pedroni <pedronis@...> wrote:
>
> Hi Samuele, good to see you continue reading Jython-dev!
>
> [...]
>  
>>  __findattr__ is really the part of the public API that someone interacting
>> from Java should use to do attribute access,
>>  that's what the Jython code base usually does itself, the null return is
>> usually more convenient and somewhat faster.
>>
>>  They are layered as they are because is saner performance-wise to have the
>> null returning version be the foundation for the raising version.
>>
>>  __getattribute__ came later with new-style classes, the main point is that
>> it needs to exists as a descriptor in the type dictionary of the required
>> types.
>>
>>  __findattr__ was intended to be the method where to implement the attribute
>> access semantics of each object type, the other are derivatives.
>>    
>
>
> Well, I'm now experimenting with changing this. The __findattr__ API
> (null return value when no attribute is found) doesn't let inherited
> classes throw their own, customs, AttributeErrors.  It's a very
> obscure and marginal use case. Following the rule which says that
> every obscure CPython feature is used by at least one prominent
> codebase, Django has descriptors which throw slightly customized
> AttributeError instances. AFAICS, there is no way to propagate them
> without allowing PyObject subclasses to override __getattr__ (The
> option of raising AttributeError from *Derived#__findattr__ sounds
> like breaking the __findattr__ API to me).
>
> As allowing PyObject subclasses to override __getattr__ will bring a
> confusing interaction between __findattr__ and __getattr__, I think it
> is better to make __getattr__  overridable and implement __findattr__
> in terms of __getattr__ (to keep the public API).
>  

but as I said layering things in the other order may have performance
effects. You would have to
benchmark things.
> A very similar situations happens with __finditem__ , __getitem__ and
> custom KeyErrors.
>  
at the moment it seems that is solved by having the *Derived classes
implement both __finditem__ and __getitem__ as appropriate instead of
relying on the default implementation in PyObject. I personally don't
think that's a bad solution.

As I said the real questions are more what is the performance impact of
the different approaches and whether for the largest number of types is
easier to implement findxxx vs getxxx, straightforwardly  supporting
obscure features should not come if possible with large impacts in these
respects.

Things have changed over time so maybe indeed the answer to both
questions is different now than in the past. But I think unless answers
in these respects motivate a change the option of only touching things
at the *Derived level is a reasonable one from my POV, they are
generated classes and they are really not meant to be subclassed
manually, so a bit of duplication there is not really a problem.






-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Jython-dev mailing list
Jython-dev@...
https://lists.sourceforge.net/lists/listinfo/jython-dev

[Leo Soto M.]

On Fri, Jun 27, 2008 at 4:13 PM, Samuele Pedroni <pedronis@...> wrote:

[...]

> at the moment it seems that is solved by having the *Derived classes
> implement both __finditem__ and __getitem__ as appropriate instead of
> relying on the default implementation in PyObject. I personally don't think
> that's a bad solution.

It's not good either. Failed lookups will run the lookup code twice.
That's why I discarded a previous patch for __getattr__/__findattr__
which followed the same strategy.

But thinking this again looks like I could have avoided the double
lookup by copying the trivial PyObject#__getitem__ code to
*Derived#__getitem__. Hmm.

> the option of only touching things at the *Derived level is a reasonable one
> from my POV, they are generated classes and they are really not meant to be
> subclassed manually, so a bit of duplication there is not really a problem.

Yeah, seems like I should have tried with that too. *Derived could try
to get over by overriding __getattr__ without calling super (at the
cost of a bit of duplication of the PyObject#__getattr__ code).

--
Leo Soto M.
http://blog.leosoto.com

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Jython-dev mailing list
Jython-dev@...
https://lists.sourceforge.net/lists/listinfo/jython-dev

[Leo Soto M.]

On Fri, Jun 27, 2008 at 5:01 PM, Leo Soto M. <leo.soto@...> wrote:

> On Fri, Jun 27, 2008 at 4:13 PM, Samuele Pedroni <pedronis@...> wrote:
>
> [...]
>
>> at the moment it seems that is solved by having the *Derived classes
>> implement both __finditem__ and __getitem__ as appropriate instead of
>> relying on the default implementation in PyObject. I personally don't think
>> that's a bad solution.
>
> It's not good either. Failed lookups will run the lookup code twice.
> That's why I discarded a previous patch for __getattr__/__findattr__
> which followed the same strategy.
>
> But thinking this again looks like I could have avoided the double
> lookup by copying the trivial PyObject#__getitem__ code to
> *Derived#__getitem__.

To make it more clear: I was talking about removing the super call on
*Derived#__getitem__.

But now I see that this isn't a good idea, because it would break the
principle of least surprise if __getitem__ is overrided on other
classes. For example, if we implement PyFoo#__getitem__, it wouldn't
be invoked on PyFooDerived instances.

So, it looks like I'm going in circles, because "correctness first"
would put me again to the "lets change __finditem__ to call
__getitem__" camp.

--
Leo Soto M.
http://blog.leosoto.com

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Jython-dev mailing list
Jython-dev@...
https://lists.sourceforge.net/lists/listinfo/jython-dev

[Samuele Pedroni-3]

Leo Soto M. wrote:

> On Fri, Jun 27, 2008 at 5:01 PM, Leo Soto M. <leo.soto@...> wrote:
>  
>> On Fri, Jun 27, 2008 at 4:13 PM, Samuele Pedroni <pedronis@...> wrote:
>>
>> [...]
>>
>>    
>>> at the moment it seems that is solved by having the *Derived classes
>>> implement both __finditem__ and __getitem__ as appropriate instead of
>>> relying on the default implementation in PyObject. I personally don't think
>>> that's a bad solution.
>>>      
>> It's not good either. Failed lookups will run the lookup code twice.
>> That's why I discarded a previous patch for __getattr__/__findattr__
>> which followed the same strategy.
>>
>> But thinking this again looks like I could have avoided the double
>> lookup by copying the trivial PyObject#__getitem__ code to
>> *Derived#__getitem__.
>>    
>
> To make it more clear: I was talking about removing the super call on
> *Derived#__getitem__.
>
> But now I see that this isn't a good idea, because it would break the
> principle of least surprise if __getitem__ is overrided on other
> classes. For example, if we implement PyFoo#__getitem__, it wouldn't
> be invoked on PyFooDerived instances.
>
>  

but apart PyInstance which is a special case anyway and has no Derived
class, all classes that override
__getitem__ do so to control the raised exception, which means that a
slight refactoring of PyObject.__getitem__
in terms of an helper lookup_error or so, would allow to avoid
overriding __getitem__ completely. Subclasses
of PyObject would simply replace the error condition helper. This is
just one of the possible approaches.
It would reduce code here and there too.
> So, it looks like I'm going in circles, because "correctness first"
> would put me again to the "lets change __finditem__ to call
> __getitem__" camp.
>
>  
But it seems that in all cases, because of the semantics of Java
collections, defining finditem is the natural and fast thing,
exept for PyInstance and Derived classes.


On a related/unrelated note skimming the code of PyDictionary it seems
that the semantics of the python level __getitem__ (dict__getitem__) and
__finditem__ for it got out of sync, __finditem__ will not respect
__missing__. I may be confused though.

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Jython-dev mailing list
Jython-dev@...
https://lists.sourceforge.net/lists/listinfo/jython-dev

[Samuele Pedroni-3]

Samuele Pedroni wrote:

> Leo Soto M. wrote:
>  
>> On Fri, Jun 27, 2008 at 5:01 PM, Leo Soto M. <leo.soto@...> wrote:
>>  
>>    
>>> On Fri, Jun 27, 2008 at 4:13 PM, Samuele Pedroni <pedronis@...> wrote:
>>>
>>> [...]
>>>
>>>    
>>>      
>>>> at the moment it seems that is solved by having the *Derived classes
>>>> implement both __finditem__ and __getitem__ as appropriate instead of
>>>> relying on the default implementation in PyObject. I personally don't think
>>>> that's a bad solution.
>>>>      
>>>>        
>>> It's not good either. Failed lookups will run the lookup code twice.
>>> That's why I discarded a previous patch for __getattr__/__findattr__
>>> which followed the same strategy.
>>>
>>> But thinking this again looks like I could have avoided the double
>>> lookup by copying the trivial PyObject#__getitem__ code to
>>> *Derived#__getitem__.
>>>    
>>>      
>> To make it more clear: I was talking about removing the super call on
>> *Derived#__getitem__.
>>
>> But now I see that this isn't a good idea, because it would break the
>> principle of least surprise if __getitem__ is overrided on other
>> classes. For example, if we implement PyFoo#__getitem__, it wouldn't
>> be invoked on PyFooDerived instances.
>>
>>  
>>    
>
> but apart PyInstance which is a special case anyway and has no Derived
> class, all classes that override
> __getitem__ do so to control the raised exception, which means that a
> slight refactoring of PyObject.__getitem__
> in terms of an helper lookup_error or so, would allow to avoid
> overriding __getitem__ completely. Subclasses
> of PyObject would simply replace the error condition helper. This is
> just one of the possible approaches.
> It would reduce code here and there too.
>  

Ah, __getattr__ already uses that approach, it seems it comes from my
new-style work :) . We would need to remove
the final there though :( (we will be left with only conventions to
avoid it to be overridden) to allow Derived to deal with the
subclasses of AttributeError case.


-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Jython-dev mailing list
Jython-dev@...
https://lists.sourceforge.net/lists/listinfo/jython-dev

[Samuele Pedroni-3]

>
> On a related/unrelated note skimming the code of PyDictionary it seems
> that the semantics of the python level __getitem__ (dict__getitem__) and
> __finditem__ for it got out of sync, __finditem__ will not respect
> __missing__. I may be confused though.
>  
ah, this is amended by PyDefaultDict.java (and the normal Derived
behavior for the subclasses) which has a __finditem__
that calls __getitem__, of course this one is breaking the interface
because of the KeyError vs null issue.

I suppose a good thing to know would be how many people have written
their own subclasses of PyObject or PyXXX
overridding __finditem__/__getitem__. I suppose they would have to
rework them anyway with the next Jython version.

Because given the various constraints and the current tangle one
possible approach and compromise would be to simply
make both __finditem__ and __getitem__ final, and base them on a new
interface, some other method
that would be allowed to either return null or raise KeyError/IndexError
whatever is most convenient, efficient.

Sketching:

public final __finditem__(key) {
      try {
                return  do_find_item(key)  // just making a name up
      } catch (PyException exc) {
              if (Py.matchException(exc,Py.LookupError))
                    return null;
              throw exc;
      }
}

public final __getitem__(key) {
      res = do_find_item(key)
      if (res == null) {
            lookupError(key) // overridable
      }
      return res
}

so one would have the option to have a null returning do_find_item and
override lookupError possibly,
or simply raise KeyError/IndexError directly if that's more convenient.

The exception catching in __finditem__ would probably have no influence
in the common cases/fast path.

With this approach I think the code in *Derived could be much simpler,
and PyDictionary do_find_item would
still simply be table.get(key).

May be missing something though, is kind of bed time around here now.



-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Jython-dev mailing list
Jython-dev@...
https://lists.sourceforge.net/lists/listinfo/jython-dev

[Leo Soto M.]

On Fri, Jun 27, 2008 at 7:05 PM, Samuele Pedroni <pedronis@...> wrote:

[...]

> Because given the various constraints and the current tangle one possible
> approach and compromise would be to simply
> make both __finditem__ and __getitem__ final, and base them on a new
> interface, some other method
> that would be allowed to either return null or raise KeyError/IndexError
> whatever is most convenient, efficient.
>
> Sketching:
>
> public final __finditem__(key) {
>     try {
>               return  do_find_item(key)  // just making a name up
>     } catch (PyException exc) {
>             if (Py.matchException(exc,Py.LookupError))
>                   return null;
>             throw exc;
>     }
> }
>
> public final __getitem__(key) {
>     res = do_find_item(key)
>     if (res == null) {
>           lookupError(key) // overridable
>     }
>     return res
> }
>
> so one would have the option to have a null returning do_find_item and
> override lookupError possibly,
> or simply raise KeyError/IndexError directly if that's more convenient.

Hey, I really like this approach.

I think that the same idea could be applied to __getitem__ /
__finditem__ if it turns out that the cost of raising AttributeError
instead of returning null is too much.

--
Leo Soto M.
http://blog.leosoto.com

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Jython-dev mailing list
Jython-dev@...
https://lists.sourceforge.net/lists/listinfo/jython-dev

[Leo Soto M.]

On Mon, Jun 30, 2008 at 10:04 AM, Leo Soto M. <leo.soto@...> wrote:

[...]

> I think that the same idea could be applied to __getitem__ /
> __finditem__

Err, I meant __getattr__ / __findattr__

> if it turns out that the cost of raising AttributeError
> instead of returning null is too much.

Indeed, it costs too much: 2000% overhead benchmarking
hasattr(object(), 'foo') using pybench.

So, my approach introduces a huge performance regression with failed
lookups on non-derived objects (Derived already shows exactly the same
horrible performance, because the exposed object___getattribute__ is
found on the type dict, and it throws NoAttributeError).

Time to try Samuele suggestion... :)
--
Leo Soto M.
http://blog.leosoto.com

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Jython-dev mailing list
Jython-dev@...
https://lists.sourceforge.net/lists/listinfo/jython-dev

[Leo Soto M.]

On Fri, Jun 27, 2008 at 7:05 PM, Samuele Pedroni <pedronis@...> wrote:

>
>>
>> On a related/unrelated note skimming the code of PyDictionary it seems
>> that the semantics of the python level __getitem__ (dict__getitem__) and
>> __finditem__ for it got out of sync, __finditem__ will not respect
>> __missing__. I may be confused though.
>>
>
> ah, this is amended by PyDefaultDict.java (and the normal Derived behavior
> for the subclasses) which has a __finditem__
> that calls __getitem__, of course this one is breaking the interface because
> of the KeyError vs null issue.
>
> I suppose a good thing to know would be how many people have written their
> own subclasses of PyObject or PyXXX
> overridding __finditem__/__getitem__. I suppose they would have to rework
> them anyway with the next Jython version.
>
> Because given the various constraints and the current tangle one possible
> approach and compromise would be to simply
> make both __finditem__ and __getitem__ final, and base them on a new
> interface, some other method
> that would be allowed to either return null or raise KeyError/IndexError
> whatever is most convenient, efficient.

OK, I've implemented this approach. The patch is up for review at
<http://codereview.appspot.com/2888>.

And this time pybench don't show any performance regression:
<http://pylonshq.com/pasties/924>
--
Leo Soto M.
http://blog.leosoto.com

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Jython-dev mailing list
Jython-dev@...
https://lists.sourceforge.net/lists/listinfo/jython-dev