Packaging question on private extension modules

[Manuel Prinz-2]

Hi debian-python,

I do have a question on Python packaging that was unfortunately not answered
by the available documentation. I guess it is a misunderstanding on my side.

I have a Python application with private extension modules, so the package is
Arch: any. Because there is no way (at least I'm not aware of that) to split
the package because the modules are private, I can't seperate the large part
of the app out to /usr/share/package/ because the extension module parts
under /usr/lib/package/ can't be loaded. Is there a way to do that or is the
recommended way to put everything to /usr/lib/package/ even thought large
parts are arch-indep?

Also, is it OK to build just against the default Python version in Debian or
should all versions be supported? Upstream's build system does not enable to
build against a specific version of Python.

Thanks in advance!

Best regards
Manuel

P.S.: Please CC me in replies, I'm not subscribed.


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Andreas Tille]

On Sat, 31 May 2008, Manuel Prinz wrote:

> I have a Python application with private extension modules, so the package is
> Arch: any. Because there is no way (at least I'm not aware of that) to split
> the package because the modules are private, I can't seperate the large part
> of the app out to /usr/share/package/ because the extension module parts
> under /usr/lib/package/ can't be loaded. Is there a way to do that or is the
> recommended way to put everything to /usr/lib/package/ even thought large
> parts are arch-indep?

Perhaps I'm missing something, but why do you think you can not split the
packages?  If you set Depends correctly this will ensure that the private
modules will be available.  Perhaps giving a link to some Vcs / download
URL which concrete package you are working on might be helpful to understand.

> Also, is it OK to build just against the default Python version in Debian or
> should all versions be supported? Upstream's build system does not enable to
> build against a specific version of Python.

I guess it is sufficient to support default Python version if it seems to
hard to tweak the build system for building against more Python versions.

Kind regards

        Andreas.

--
http://fam-tille.de


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Manuel Prinz-2]

Hi Andreas!

Am Sonntag, 1. Juni 2008 08:12:41 schrieb Andreas Tille:
> Perhaps I'm missing something, but why do you think you can not split the
> packages?  If you set Depends correctly this will ensure that the private
> modules will be available.  Perhaps giving a link to some Vcs / download
> URL which concrete package you are working on might be helpful to
> understand.

The package is pdb2pqr, you can find it here:
svn://svn.debian.org/svn/debian-med/trunk/packages/pdb2pqr/trunk/

The current version puts everything to /usr/lib/pdb2pqr as that seems to
please lintian. I tried to split the package into two and also tried to put
the arch-indep modules into /usr/share/pdb2pqr and the arch-dep modules
(propka and pdb2pka) into /usr/lib/pdb2pqr in the same package. Both
approaches did not work, pdb2pqr.py was not able to load the propka modules
anymore. I thought this was due to the fact that they are private modules and
pdb2pqr.py is started from /usr/share/pdb2pqr, so it's not able to load the
parts that reside in /usr/lib/pdb2pqr. I guess this is were I do not
understand things completely.

It's my first Python package, so there might be other things that are not OK.
I took Steffen's work on it but that did not produce a working version of
PDB2PQR, so I'm trying to work things out. The current package works but has
everything under /usr/lib/pdb2pqr which I think is not optimal since it uses
more archive space than needed. (The binary parts are only a few kb.) Any
feedback it welcome! Thanks in advance!

Best regards
Manuel


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Andreas Tille]

On Sun, 1 Jun 2008, Manuel Prinz wrote:

> The package is pdb2pqr, you can find it here:
> svn://svn.debian.org/svn/debian-med/trunk/packages/pdb2pqr/trunk/

Without having a look into this ...

> The current version puts everything to /usr/lib/pdb2pqr as that seems to
> please lintian. I tried to split the package into two and also tried to put
> the arch-indep modules into /usr/share/pdb2pqr and the arch-dep modules
> (propka and pdb2pka) into /usr/lib/pdb2pqr in the same package. Both
> approaches did not work, pdb2pqr.py was not able to load the propka modules
> anymore.

Did you tried symlinks to join all files needed in one directory?  You could
point symlinks from /usr/lib/pdb2pqr to /usr/share/pdb2pqr and make the
Arch: any package

     Depends: <arch-all_package>

to make sure these symlinks are properly resolved.

> I thought this was due to the fact that they are private modules and
> pdb2pqr.py is started from /usr/share/pdb2pqr, so it's not able to load the
> parts that reside in /usr/lib/pdb2pqr. I guess this is were I do not
> understand things completely.

Or if /usr/share/pdb2pqr is the directory to start from just do the symlinking
and dependency the other way around.  I'm not enough of a python expert to
give some hints how private modules could be found otherwise - but let the
application think they are right in place seems to be a working circumvention of
the problem.

Kind regards

         Andreas.

--
http://fam-tille.de


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Manuel Prinz-2]

Hi Andreas!

Am Sonntag, 1. Juni 2008 14:43:38 schrieb Andreas Tille:
> Or if /usr/share/pdb2pqr is the directory to start from just do the
> symlinking and dependency the other way around.  I'm not enough of a python
> expert to give some hints how private modules could be found otherwise -
> but let the application think they are right in place seems to be a working
> circumvention of the problem.

Thanks a lot for your help! This way round it might actually work. I was not
sure if I should do symlinking between packages but since it must be a
Depends anyway this seems to be fine.

Best regards
Manuel


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Josselin Mouette]

Le samedi 31 mai 2008 à 22:00 +0200, Manuel Prinz a écrit :
> I have a Python application with private extension modules, so the package is
> Arch: any. Because there is no way (at least I'm not aware of that) to split
> the package because the modules are private, I can't seperate the large part
> of the app out to /usr/share/package/ because the extension module parts
> under /usr/lib/package/ can't be loaded. Is there a way to do that or is the
> recommended way to put everything to /usr/lib/package/ even thought large
> parts are arch-indep?

At least, current practice in this case is to put everything
in /usr/lib.

> Also, is it OK to build just against the default Python version in Debian or
> should all versions be supported? Upstream's build system does not enable to
> build against a specific version of Python.

For private extensions, it is quite complex to support multiple python
versions as you need a specific rtupdate script, so in general you
should not do that.

Cheers,
--
 .''`.
: :' :      We are debian.org. Lower your prices, surrender your code.
`. `'       We will add your hardware and software distinctiveness to
  `-        our own. Resistance is futile.

[Aaron M. Ucko]

Manuel Prinz <debian@...> writes:

> I have a Python application with private extension modules, so the package is
> Arch: any. Because there is no way (at least I'm not aware of that) to split
> the package because the modules are private, I can't seperate the large part
> of the app out to /usr/share/package/ because the extension module parts
> under /usr/lib/package/ can't be loaded. Is there a way to do that or is the
> recommended way to put everything to /usr/lib/package/ even thought large
> parts are arch-indep?

You're welcome to copy the approach I take with BitPim, which is in
the same boat: the main package (bitpim) is architecture-independent
and installs its Python modules under /usr/share (specifically, in
/usr/share/bitpim/code).  However, because a couple of modules are
architecture-dependent, they go into a separate bitpim-lib package on
which bitpim depends, with a carefully crafted __init__.py that lets
Python find them:

$ cat /usr/share/bitpim/code/native/__init__.py
# Handle Debian's split-up layout
__path__.insert(0, '/usr/lib/bitpim/native')

I suspect one could do the same with a .pth file.

> Also, is it OK to build just against the default Python version in Debian or
> should all versions be supported? Upstream's build system does not enable to
> build against a specific version of Python.

You shouldn't need to build against multiple Python versions as long
as you arrange for inter-package dependencies to be bin-NMU safe (no
strict all -> any dependencies).

> P.S.: Please CC me in replies, I'm not subscribed.

Done; thanks for warning us.

--
Aaron M. Ucko, KB1CJC (amu at alum.mit.edu, ucko at debian.org)
http://www.mit.edu/~amu/ | http://stuff.mit.edu/cgi/finger/?amu@...


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Manuel Prinz-2]

Am Montag, 2. Juni 2008 14:52:05 schrieb Josselin Mouette:
> At least, current practice in this case is to put everything
> in /usr/lib.

This is something I did and of course works. Currently I do install everything
into /usr/share except the binary parts (.so files). I have those in /usr/lib
and symlink to them in /usr/lib. I did so because it is trivial to split the
packages into Arch: all and any but I'm still uncertain about splitting the
package since it is rather small: about ~650kb total, binary parts are just a
few kb. Is there a rule of thumb when it's reasonable to split?

Thanks to everyone who answered so far!

Best regards
Manuel


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...