handling /usr/local/lib/python2.x/site-packages in sys.path

View: New views
16 Messages — Rating Filter:   Alert me  

handling /usr/local/lib/python2.x/site-packages in sys.path

by Matthias Klose-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Currently Debian's python has /usr/local/lib/python2.x/site-packages
in sys.path allowing for installation of local modules.  Barry did
point out that this conflicts with a python installation, where
/usr/local is the default prefix, and the system python gets modules
from the local installation.  Some suggestions were made to fix this:

 - add an env var to not include /usr/local/lib/python2.x/site-packages
   when the env var is set. Keeps standard behaviour without
   modifications and allows people to remove it from sys.path. But
   requires the user to know about that option.

 - add another path (e.g. /usr/local/python/lib2.x/site-packages),
   and remove the /usr/local/lib/python2.x/site-packages path after
   the next release. Does provide an upgrade path, but doesn't solve
   the probem immediately.

Matthias


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...


Re: handling /usr/local/lib/python2.x/site-packages in sys.path

by wichert :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Matthias Klose wrote:
> Currently Debian's python has /usr/local/lib/python2.x/site-packages
> in sys.path allowing for installation of local modules.  Barry did
> point out that this conflicts with a python installation, where
> /usr/local is the default prefix, and the system python gets modules
> from the local installation.

In other words you are trying to make things a bit easier for people who are
a) expert enough to build and use their own python version and b) notice
things breaking because they use a site-packages path which is also used by
the system python.

I can sympathise with building your own python (Debian's python uses twice
the amount of memory needed for most unicode-using applications due to the
use of UCS4 for example) but I think people who know enough to do that also
know enough to deal with the resulting possible conflict over
/usr/local/lib/python2.x use. Personally I have a number of local python's
but they are all isolated in zc.buildout instances.

   Some suggestions were made to fix this:
>
>  - add an env var to not include /usr/local/lib/python2.x/site-packages
>    when the env var is set. Keeps standard behaviour without
>    modifications and allows people to remove it from sys.path. But
>    requires the user to know about that option.

That makes things unpredicatable: use su or something else that can affect
the environment and suddenly python behaves quite differently. That does not
feel like a good idea.

>  - add another path (e.g. /usr/local/python/lib2.x/site-packages),
>    and remove the /usr/local/lib/python2.x/site-packages path after
>    the next release. Does provide an upgrade path, but doesn't solve
>    the probem immediately.

That just means you will break people's python at a later point again. That
certainly can not be the right thing to do.

Iff we really must come up with a way to make it a bit easier for people to
install their own python version make it a system-wide switch in
/etc/default/python2.x and keep the default behaviour as it is now so you
will not break existing installations.

Wichert.

--
Wichert Akkerman <wichert@...>    It is simple to make things.
http://www.wiggy.net/                   It is hard to make things simple.


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...


Re: handling /usr/local/lib/python2.x/site-packages in sys.path

by Barry Warsaw :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 11, 2008, at 3:43 AM, Matthias Klose wrote:

> Currently Debian's python has /usr/local/lib/python2.x/site-packages
> in sys.path allowing for installation of local modules.  Barry did
> point out that this conflicts with a python installation, where
> /usr/local is the default prefix, and the system python gets modules
> from the local installation.  Some suggestions were made to fix this:
>
> - add an env var to not include /usr/local/lib/python2.x/site-packages
>   when the env var is set. Keeps standard behaviour without
>   modifications and allows people to remove it from sys.path. But
>   requires the user to know about that option.
>
> - add another path (e.g. /usr/local/python/lib2.x/site-packages),
>   and remove the /usr/local/lib/python2.x/site-packages path after
>   the next release. Does provide an upgrade path, but doesn't solve
>   the probem immediately.

Thanks for raising this issue Matthias.  There are a couple of other  
points I'd like to make.

This isn't /just/ a problem for Python experts or developers.  
Sometimes we recommend that Ordinary Users install Python from source,  
say to use an application that isn't supported by a distro's packaging  
system.  There can be lots of reasons for this: maybe the app is in  
beta testing, maybe they want to try things out without affecting the  
system version, etc.

Python developers may not be Debian experts and may not know about  
this system peculiarity.  So when we tell people "just do configure;  
make; make install" we can actually do inadvertent damage to their  
Debian system.

It also makes Debian the odd man out.  Instructions we publish for  
every other *nix have to have a caveat or FAQ for Debian's (and  
derivatives) difference.  These can go unnoticed until things break,  
and then we can get difficult to debug problem reports.  An  
experienced helper will be conditioned to first ask "Are you on Debian  
or Ubuntu? Well, you have to do things differently there."

This feature is difficult to discover, and highly surprising to any  
Python person, especially more so if they are not Debian experts.  
When I first hit this problem (and complained to Matthias ;), it took  
me several hours to figure out what was going on, where the hack was  
being inserted, and that it was actually intentional.

A more fundamental question though is, why do this in the first  
place?  What's the use case?  What convenience are you providing  
users, and if that's overwhelmingly beneficial, can it be done in a  
way that doesn't go counter to the conventions of the upstream project?

What confuses me is that this doesn't appear to be a convenience based  
on permissions, because on my Ubuntu box it takes root to install  
things in /usr/local just as it would to install things in /usr.  
Meaning, if I'm going to install an egg in either location, either  
manually or via easy_install, I still need to be root.  Although this  
doesn't affect things /now/ I think Python 2.6 will allow for user-
specific sys.paths so that you could for example install an egg in  
~/.local/lib/python2.6 and have it just show up.

I'm also uncomfortable with the way that Debian accomplishes this: it  
hacks Python's standard site.py.  There really should be a better way  
to do this (I have to think about this a bit though).

I would opt for Matthias's second option, and would be happy if there  
was a roadmap for fixing this problem.

- -Barry

P.S. System applications written in Python should /never/ use "#! /usr/
bin/env python".  They should always hardcode /usr/bin/pythonX.Y.  
Otherwise you can break your system just by tweaking your $PATH.  You  
should have to work harder to break your system <wink>.

This is why setuptools rewrites the #! line when it installs scripts  
in packages.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR9aM8XEjvBPtnXfVAQKpBwP+OBvovf+MZsHTsU5N42TKqqHXi+IQwHKp
rnCtqkBju+jYAWmJpE5BWHmYkBIyc9Me4N7tPMM04YYKwipub5tLEa1CUaHmtyWp
5sEHK2rlOJoluBvJy/S2NBgkSExw2RHmgkn5ZUb8IlLHO9HFh67Egc6vWG6Bg6xq
GScfgB1bEfQ=
=yk2u
-----END PGP SIGNATURE-----


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...


Re: handling /usr/local/lib/python2.x/site-packages in sys.path

by Barry Warsaw :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 11, 2008, at 3:58 AM, Wichert Akkerman wrote:
>
>> - add another path (e.g. /usr/local/python/lib2.x/site-packages),
>>   and remove the /usr/local/lib/python2.x/site-packages path after
>>   the next release. Does provide an upgrade path, but doesn't solve
>>   the probem immediately.
>
> That just means you will break people's python at a later point  
> again. That certainly can not be the right thing to do.

Depends on how you look at it I guess.  I see it as eventually fixing  
everyone's Python ;).

> Iff we really must come up with a way to make it a bit easier for  
> people to install their own python version make it a system-wide  
> switch in /etc/default/python2.x and keep the default behaviour as  
> it is now so you will not break existing installations.

I'm not sure what the "it" is you want to make a system-wide switch.  
Would that be which Python to use, or whether to add the sys.path hack?

I don't know who's going to be at Pycon later this week, but if enough  
distro people are around, I'd be very happy to have a discussion about  
how to make this more consistent across distros, more amenable to  
users and developers, and how to make things less mysterious,  
confusing, or fragile.  Given how many system tools are being built in  
Python now (a good thing!), there should be some agreement on the  
right way to do it.

I'll be at Pycon and would be willing to chat about this at a BOF,  
over lunch, or over beers. :)

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR9aPNHEjvBPtnXfVAQJCzgP/QaadzyGxnvA6WqT8B3QsouSVvZvsmg9n
LAOnoLYzZOhcLzSUlYiukPcU6DMX8HAA8AaAvwpiyCLmk6IZDGMJxDZH+R8u4+w7
9E4KRdt3yxa5nbcJAMa/fWzjpDTlXSecZBAHw7IAYngy55+fv7cJ7UvWgNqsPdIB
h4mUoEU32CU=
=I8kv
-----END PGP SIGNATURE-----


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...


Re: handling /usr/local/lib/python2.x/site-packages in sys.path

by Sebastian Rittau :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Tue, Mar 11, 2008 at 08:43:06AM +0100, Matthias Klose wrote:

> Currently Debian's python has /usr/local/lib/python2.x/site-packages
> in sys.path allowing for installation of local modules.  Barry did
> point out that this conflicts with a python installation, where
> /usr/local is the default prefix, and the system python gets modules
> from the local installation.  Some suggestions were made to fix this:

Well, I would expect that package I install in /usr/local are found
automatically by the standard Python install. I think locally installing
packages is a far more common use case than installing a hand-compiled
python. It would be very unusual and inconvenient if installing modules
with "python setup.py --prefix=/usr/local" wouldn't work with the
standard Python installation.

 - Sebastian


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...


Re: handling /usr/local/lib/python2.x/site-packages in sys.path

by Piotr Ożarowski-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

[Matthias Klose, 2008-03-11]
>  - add another path (e.g. /usr/local/python/lib2.x/site-packages),
>    and remove the /usr/local/lib/python2.x/site-packages path after
>    the next release. Does provide an upgrade path, but doesn't solve
>    the probem immediately.

I would choose this option or even remove /usr/local/* from sys.path
completely. If one wants to play with unsupported Python modules or
Eggs, he should sys.path.append(his_path) by hand, IMHO. If we
additionally force ez_* mess to install to /usr/local/ by default, we
will get rid of one of the most common problems I receive from users of my
packages.
--
-=[     Piotr Ozarowski     ]=-
-=[ http://www.ozarowski.pl ]=-


attachment0 (196 bytes) Download Attachment

Re: handling /usr/local/lib/python2.x/site-packages in sys.path

by Sebastian Rittau :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Tue, Mar 11, 2008 at 06:58:56PM +0100, Piotr Ożarowski wrote:

> I would choose this option or even remove /usr/local/* from sys.path
> completely. If one wants to play with unsupported Python modules or
> Eggs, he should sys.path.append(his_path) by hand, IMHO.

I would not consider installing packages that are not packaged in Debian
"playing". Installing such packages is a common use case for system
administrators and advanced users. With the same reasoning we could
remove /usr/local/lib from /etc/ld.so.conf.

 - Sebastian


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...


Re: handling /usr/local/lib/python2.x/site-packages in sys.path

by Thomas Viehmann :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Matthias Klose wrote:
>  - add an env var to not include /usr/local/lib/python2.x/site-packages
>    when the env var is set. Keeps standard behaviour without
>    modifications and allows people to remove it from sys.path. But
>    requires the user to know about that option.

I would much prefer this. Debian users with local (=unpackaged) packages are
probably much more common than Debian users with (=unpackaged) versions of all
python, and I would rather reasonably support those than leaving them in the
cold. Quite frankly, installing stuff that is also present in the system under
user local is a recipe for disaster (also happens with libraries) and rather
hard to cater for.

>  - add another path (e.g. /usr/local/python/lib2.x/site-packages),
>    and remove the /usr/local/lib/python2.x/site-packages path after
>    the next release. Does provide an upgrade path, but doesn't solve
>    the probem immediately.
No. Let's not break stuff for people that use the software in Debian that they
got from Debian.

Quite frankly, the use case seems a bit of bad practice. Installing to
/usr/local software to compete with that in /usr leads to all sorts of breakage
where one doesn't expect it and if python.org needs people to install
experimental versions they should either recommend chroots / test machines or
provide packages. Anyone capable of installing a local version of python is also
able to run debootstrap to create a chroot.

Kind regards

T.
(who used to maintain a set of libraries where local installations caused no end
 of trouble for users, maintainers, and upstreams)
--
Thomas Viehmann, http://thomas.viehmann.net/


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...


Re: handling /usr/local/lib/python2.x/site-packages in sys.path

by Piotr Ożarowski-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

[Sebastian Rittau, 2008-03-11]
> On Tue, Mar 11, 2008 at 06:58:56PM +0100, Piotr Ożarowski wrote:
> > I would choose this option or even remove /usr/local/* from sys.path
> > completely. If one wants to play with unsupported Python modules or
> > Eggs, he should sys.path.append(his_path) by hand, IMHO.
>
> I would not consider installing packages that are not packaged in Debian
> "playing". Installing such packages is a common use case for system
> administrators and advanced users. With the same reasoning we could
> remove /usr/local/lib from /etc/ld.so.conf.

true.

/me still doesn't know how to (at the same time) eat a cookie (get rid
of Egg problems) and still be able to keep it (give advanced
users/administrators possibility to add local modifications).
--
-=[     Piotr Ozarowski     ]=-
-=[ http://www.ozarowski.pl ]=-


attachment0 (196 bytes) Download Attachment

Re: handling /usr/local/lib/python2.x/site-packages in sys.path

by Floris Bruynooghe-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Tue, Mar 11, 2008 at 09:45:21AM -0400, Barry Warsaw wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Mar 11, 2008, at 3:43 AM, Matthias Klose wrote:
>
>> Currently Debian's python has /usr/local/lib/python2.x/site-packages
>> in sys.path allowing for installation of local modules.  Barry did
>> point out that this conflicts with a python installation, where
>> /usr/local is the default prefix, and the system python gets modules
>> from the local installation.  Some suggestions were made to fix this:
>>
>> - add an env var to not include /usr/local/lib/python2.x/site-packages
>>   when the env var is set. Keeps standard behaviour without
>>   modifications and allows people to remove it from sys.path. But
>>   requires the user to know about that option.
>>
>> - add another path (e.g. /usr/local/python/lib2.x/site-packages),
>>   and remove the /usr/local/lib/python2.x/site-packages path after
>>   the next release. Does provide an upgrade path, but doesn't solve
>>   the probem immediately.
>
> Thanks for raising this issue Matthias.  There are a couple of other  
> points I'd like to make.
>
> This isn't /just/ a problem for Python experts or developers.  Sometimes
> we recommend that Ordinary Users install Python from source, say to use
> an application that isn't supported by a distro's packaging system.  
> There can be lots of reasons for this: maybe the app is in beta testing,
> maybe they want to try things out without affecting the system version,
> etc.
>
> Python developers may not be Debian experts and may not know about this
> system peculiarity.  So when we tell people "just do configure; make;
> make install" we can actually do inadvertent damage to their Debian
> system.

Problem is that both are very natural, having local packages being
picked up by the system python in
/usr/local/lib/python2.X/site-packages as well as being able to
install a new python with a prefix of /usr/local.

Given this choice I'd prefer the second option as well, it seems more
natural then the first and an upgrade path is important (Wichert's
argument about the first having problems with su is also important).
It would be good if the distutils could be changed to install local
packages there by default then, so as to provide the least surprises:
just "python setup.py install" as root would do the correct thing.

When looking at the grand scale of things the change for the system
Python to look for local packages in
/usr/local/python/lib2.X/site-packages (or
/usr/local/python/site-packages2.X?[1]) is probably best done upstream
so it is on all systems the same.  The build system could do this when
a prefix of /usr is passed to configure.

This can not be done by using a system wide configuration file IIRC,
since the Python make system will pick up those configuration files
hence breaking the /usr/local install again as well as generating
FTBFS on systems that have python already installed during the
build[0].  This is possibly a bug in the Python make system though,
what is the rationale for picking up the distutils configuration files
for building python itself?  Surely all this info is passed to
configure?


> What confuses me is that this doesn't appear to be a convenience based  
> on permissions,

It has nothing to do with permissions.  The sysadmin should never
install things into /usr/ directly, /usr/local/ is their playground.


> I'm also uncomfortable with the way that Debian accomplishes this: it  
> hacks Python's standard site.py.  There really should be a better way to
> do this (I have to think about this a bit though).

In the python we ship to our customers (living in /opt/vendor/python2X
as per FHS) we place a .pth file in the site-packages directory to
accomplish extra locations on sys.path (pointing somewhere else in our
/opt/vendor hierarchy).  Maybe
/usr/lib/python2.X/site-packages/debian.pth would be a better way to
handle this on Debian?  Or are there reasons not to do this?  Although
the suggestion I make above, changing it upstream, would make this
unnecessary.

[As an aside, I find what happens with python-support much worse.  I
regularly waste time re-discovering that a whole load of system
installed packages can't be found in /usr/lib/python2.X/site-packages/
but live somewhere in /var/lib/python-support/.  It's a violation of
the FHS as well, /var/lib/ is for program state information, .pyc's
are not state).]


Regards
Floris


[0] This has bitten me in the past with FTBFS when having a
    ~/.pydistutils.cfg defining a `home=~' and trying to rebuild the
    Debian Python package.  I think it also happens when trying to
    build a new upstream python with an existing distutils
    configuration somewhere.

[1] Both violate FHS though.  The directories allowed in /usr/local
    after a fresh install are limited
    (http://www.pathname.com/fhs/pub/fhs-2.3.html#USRLOCALLOCALHIERARCHY).
    But /usr/local/lib/python/site-packages2.X does seem a little
    cumbersome and confusing (since a local python will use
    /usr/local/lib/python2.X/site-packages).  Maybe a
    /usr/local/lib/system-python/ structure would work, but that is
    getting cumbersome again.

--
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...


Re: handling /usr/local/lib/python2.x/site-packages in sys.path

by wichert :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Barry Warsaw wrote:
> It also makes Debian the odd man out.  Instructions we publish for
> every other *nix have to have a caveat or FAQ for Debian's (and
> derivatives) difference.  These can go unnoticed until things break,
> and then we can get difficult to debug problem reports.  An
> experienced helper will be conditioned to first ask "Are you on Debian
> or Ubuntu? Well, you have to do things differently there."

This is a point of perspective. From my point of view python is the odd
one out: every other tool and compiler seems to look in both /usr/local
and /usr *except* for (non-Debian) python. That would be highly
confusing for me, and I fail to understand why python made that choice.

Debian has always setup /usr/local so that anyone in the staff group can
write to it. This is very convenient: it gives you a clear point of
demarkation between OS-managed applications, which install in /usr, and
manually managed applications which install in /usr/local and can be
installed by trusted non-root users. I don't know why your Ubuntu system
only makes /usr/local writable for root, to me that sounds like a bug in
Ubuntu.

I find virtualenv to be a better tool to setup local python
environments: it does not require people to recompile all of python
while still providing a clean environment to work in. And as far as I
can see it is a good solution for almost all use cases where you
currently tell people to compile a local python version.

Wichert.

--
Wichert Akkerman <wichert@...>   It is simple to make things.
http://www.wiggy.net/                  It is hard to make things simple.



--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...


Re: handling /usr/local/lib/python2.x/site-packages in sys.path

by Barry Warsaw :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 11, 2008, at 6:36 PM, Floris Bruynooghe wrote:

> On Tue, Mar 11, 2008 at 09:45:21AM -0400, Barry Warsaw wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On Mar 11, 2008, at 3:43 AM, Matthias Klose wrote:
>>
>>> Currently Debian's python has /usr/local/lib/python2.x/site-packages
>>> in sys.path allowing for installation of local modules.  Barry did
>>> point out that this conflicts with a python installation, where
>>> /usr/local is the default prefix, and the system python gets modules
>>> from the local installation.  Some suggestions were made to fix  
>>> this:
>>>
>>> - add an env var to not include /usr/local/lib/python2.x/site-
>>> packages
>>>  when the env var is set. Keeps standard behaviour without
>>>  modifications and allows people to remove it from sys.path. But
>>>  requires the user to know about that option.
>>>
>>> - add another path (e.g. /usr/local/python/lib2.x/site-packages),
>>>  and remove the /usr/local/lib/python2.x/site-packages path after
>>>  the next release. Does provide an upgrade path, but doesn't solve
>>>  the probem immediately.
>>
>> Thanks for raising this issue Matthias.  There are a couple of other
>> points I'd like to make.
>>
>> This isn't /just/ a problem for Python experts or developers.  
>> Sometimes
>> we recommend that Ordinary Users install Python from source, say to  
>> use
>> an application that isn't supported by a distro's packaging system.
>> There can be lots of reasons for this: maybe the app is in beta  
>> testing,
>> maybe they want to try things out without affecting the system  
>> version,
>> etc.
>>
>> Python developers may not be Debian experts and may not know about  
>> this
>> system peculiarity.  So when we tell people "just do configure; make;
>> make install" we can actually do inadvertent damage to their Debian
>> system.
>
> Problem is that both are very natural, having local packages being
> picked up by the system python in
> /usr/local/lib/python2.X/site-packages as well as being able to
> install a new python with a prefix of /usr/local.

I asked quite a few Python people about this at Pycon 2008, and I  
raised it in a Python packaging session with Matthias and a few Fedora  
people.

I actually think this is natural only for Debian users, and not the  
wider Python or Linux crowd.  The Fedora people don't do this, nor  
does Gentoo AFAICT.  It surprised quite a few Python folks I spoke  
with, even some that use Debian or Ubuntu.

> Given this choice I'd prefer the second option as well, it seems more
> natural then the first and an upgrade path is important (Wichert's
> argument about the first having problems with su is also important).
> It would be good if the distutils could be changed to install local
> packages there by default then, so as to provide the least surprises:
> just "python setup.py install" as root would do the correct thing.

I agree that for Debian and Ubuntu, where this tradition is firmly  
ingrained, the second option makes the most sense.  It lets you have  
the functionality you want without breaking the expectations of the  
wider Python community.  Everyone can live together peacefully! :)

I'm not sure your other suggestion make sense from a default distutils  
perspective.  The reason is that distutils doesn't see the world that  
way, and it really doesn't know anything about the rest of the file  
system, other than the paths it gets from Python or that you tell it  
about.  So having it install in Python's own site-packages by default  
still makes sense, but it should be easily configurable to do  
otherwise, either through command line switch or other mechanism.  
That's the case today though :).

> When looking at the grand scale of things the change for the system
> Python to look for local packages in
> /usr/local/python/lib2.X/site-packages (or
> /usr/local/python/site-packages2.X?[1]) is probably best done upstream
> so it is on all systems the same.  The build system could do this when
> a prefix of /usr is passed to configure.

I don't think it makes sense to get this into upstream because this is  
really a distro question.  What about MacPorts installing into /opt/
local?  What about if I install from source into ~barry/development/
python2.6?   None of those should have anything to do with /usr/local.

>> What confuses me is that this doesn't appear to be a convenience  
>> based
>> on permissions,
>
> It has nothing to do with permissions.  The sysadmin should never
> install things into /usr/ directly, /usr/local/ is their playground.

This is Debian policy (which is fine), but I don't think all distros  
agree.  I'm not a distro guy though. :)  Mattias, didn't the Fedora  
guys say they were going to try to create a mailing list for  
discussing these kinds of cross-distro issues?

>> I'm also uncomfortable with the way that Debian accomplishes this: it
>> hacks Python's standard site.py.  There really should be a better  
>> way to
>> do this (I have to think about this a bit though).
>
> In the python we ship to our customers (living in /opt/vendor/python2X
> as per FHS) we place a .pth file in the site-packages directory to
> accomplish extra locations on sys.path (pointing somewhere else in our
> /opt/vendor hierarchy).  Maybe
> /usr/lib/python2.X/site-packages/debian.pth would be a better way to
> handle this on Debian?  Or are there reasons not to do this?  Although
> the suggestion I make above, changing it upstream, would make this
> unnecessary.

I just looked in Ubuntu Gutsy and Hardy, and in those distros, there's  
is actually a hack to site.py to add /usr/local explicitly.  Maybe  
Debian does it differently, but I don't have a system to check.  I do  
think using debian.pth would be a much better way to do it than  
hacking Lib/site.py from upstream.

> [As an aside, I find what happens with python-support much worse.  I
> regularly waste time re-discovering that a whole load of system
> installed packages can't be found in /usr/lib/python2.X/site-packages/
> but live somewhere in /var/lib/python-support/.  It's a violation of
> the FHS as well, /var/lib/ is for program state information, .pyc's
> are not state).]

Right, pycs are cache! :)  I can't comment on the rest of this  
paragraph. ;)

Cheers,
- -Barry


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)

iQCVAwUBR+mHb3EjvBPtnXfVAQL2DwP/XGIQFmXcwWabeo4KtLBKvbk3AQOENqcd
ndS4dK1v6oijNC98mc14syjR16wszU9+LPwYTgUXIkP7Q9JAWSIE3/A+UL0lKxet
4IDEswdjcmRMDI27UTSuf09fin+YJ2icuIwTpKbuXbH3tFxxvP/wcwFZSFmMPx0L
/2wpSSW15gk=
=4cYg
-----END PGP SIGNATURE-----


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...


Re: handling /usr/local/lib/python2.x/site-packages in sys.path

by Ben Finney-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Barry Warsaw <barry@...> writes:

> On Mar 11, 2008, at 6:36 PM, Floris Bruynooghe wrote:
> > The sysadmin should never install things into /usr/ directly,
> > /usr/local/ is their playground.
>
> This is Debian policy (which is fine), but I don't think all distros
> agree.

Those distros that claim to adhere to the FHS agree.

    /usr is shareable, read-only data. That means that /usr should be
    shareable between various FHS-compliant hosts and must not be
    written to. Any information that is host-specific or varies with
    time is stored elsewhere.

    <URL:http://www.pathname.com/fhs/pub/fhs-2.3.html#THEUSRHIERARCHY>


    The /usr/local hierarchy is for use by the system administrator
    when installing software locally. It needs to be safe from being
    overwritten when the system software is updated. It may be used
    for programs and data that are shareable amongst a group of hosts,
    but not found in /usr.

    <URL:http://www.pathname.com/fhs/pub/fhs-2.3.html#USRLOCALLOCALHIERARCHY>

--
 \           “The cost of education is trivial compared to the cost of |
  `\                                     ignorance.” —Thomas Jefferson |
_o__)                                                                  |
Ben Finney


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...


Re: handling /usr/local/lib/python2.x/site-packages in sys.path

by Emilio Pozuelo Monfort :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Barry Warsaw wrote:
> This is Debian policy (which is fine), but I don't think all distros
> agree.  I'm not a distro guy though. :)  Mattias, didn't the Fedora guys
> say they were going to try to create a mailing list for discussing these
> kinds of cross-distro issues?

There's http://lists.freedesktop.org/mailman/listinfo/distributions, you may
want to raise this issue there.

Cheers,
Emilio



signature.asc (196 bytes) Download Attachment

Re: handling /usr/local/lib/python2.x/site-packages in sys.path

by Barry Warsaw :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 26, 2008, at 6:23 AM, Emilio Pozuelo Monfort wrote:

> Barry Warsaw wrote:
>> This is Debian policy (which is fine), but I don't think all distros
>> agree.  I'm not a distro guy though. :)  Mattias, didn't the Fedora  
>> guys
>> say they were going to try to create a mailing list for discussing  
>> these
>> kinds of cross-distro issues?
>
> There's http://lists.freedesktop.org/mailman/listinfo/distributions,  
> you may
> want to raise this issue there.

Thanks for the pointer.  I'll keep an eye on this, but I don't have  
time for yet another mailing list. ;)

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)

iQCVAwUBR+pKpnEjvBPtnXfVAQI+2wQAiR+DmSgWzuWxFAw9T2TiztGouquKsqSC
tD+CNuu7gF8HsjDKxdWltU2rIpxkrSVhPZSQPp5L3/4yL3LHAtANdKv5AyPQOAqz
t7pSenmuyLdwcyuIzRQQRi3MrB9OSgnjfNHnBet6jw7H9o0RM+5pbakgo26xVIqt
2pyyQhDibLg=
=NVYT
-----END PGP SIGNATURE-----


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...


Re: handling /usr/local/lib/python2.x/site-packages in sys.path

by Floris Bruynooghe-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Tue, Mar 11, 2008 at 08:43:06AM +0100, Matthias Klose wrote:
> Currently Debian's python has /usr/local/lib/python2.x/site-packages
> in sys.path allowing for installation of local modules.  Barry did
> point out that this conflicts with a python installation, where
> /usr/local is the default prefix, and the system python gets modules
> from the local installation.  Some suggestions were made to fix this:
[...]
>  - add another path (e.g. /usr/local/python/lib2.x/site-packages),
>    and remove the /usr/local/lib/python2.x/site-packages path after
>    the next