multi-variant packages and bulk builds

[Aleksey Cheusov-3]

In pkgsrc there are a number of packages (category/package) that can
be built with different PKGNAME depending on a value of some special
variables. For example, www/ap2-* packages, */py-* and many others.

AFAIU in order to buils these packages pbulk uses hardcoded list of
special variables in mk/pbulk/pbulk-index.mk. For multi-variant
packages my distbb currently builds only package, the default one, but
I don't thing the way pbulk works is the right way.

Instead, I propose to introduce one special variable, e.g. VARIANTS,
and to support it in all multi-variant packages.

Example:
  cd /usr/pkgsrc/www/py-clearsilver
  make show-var VARNAME=VARIANTS
  > PYTHON_VERSION_DEFAULT=15,20,21,22,23,25
  cd /usr/pkgsrc/www/ap2-auth-external
  make show-var VARNAME=VARIANTS
  > PKG_APACHE_DEFAULT=apache2,apache22
  cd /usr/pkgsrc/www/ap2-python
  > PKG_APACHE_DEFAULT=apache2,apache22 PYTHON_VERSION_DEFAULT=23,24,25

Using cartesian product of all supported values for all listed
variables, bulk build software can easily build all variants all such
packages. I hope the idea is clear.

Of course the code that sets VARIANTS variable should use appropriate
xxx_ACCEPTED variables.

One of the main benefit of this approch is that pkgsrc itself and bulk
build programs become more isolated and independant from each other
because pkgsrc API becomes clearer and easier.

Any thoughts?

--
Best regards, Aleksey Cheusov.

[Aleksey Cheusov-3]

> In pkgsrc there are a number of packages (category/package) that can
> be built with different PKGNAME depending on a value of some special
> variables. For example, www/ap2-* packages, */py-* and many others.

Patch in the attached file implements a proposed idea.
It is very small and simple.

For remind: idea is to develop a pkgsrc API for multi-variant packages
(py23-xxx, py24-xxx, py25-xxx etc.). Using this API bulk build
programs can generate appropriate binaries easily and without
hardcodes.

How it works (tested)

  0 ~>cd /srv/pkgsrc/wip/dict-server/
  0 dict-server>bmake show-var VARNAME=VARIANTS

  0 dict-server>cd ../../www/ap-php
  0 ap-php>bmake show-var VARNAME=VARIANTS
  PHP_VERSION_DEFAULT=4,5 PKG_APACHE_DEFAULT=apache13,apache2,apache22
  0 ap-php>cd ../../www/ap2-python
  0 ap2-python>bmake show-var VARNAME=VARIANTS
  PYTHON_VERSION_DEFAULT=25,24,23 PKG_APACHE_DEFAULT=apache2,apache22
  0 ap2-python>cd ../../www/ap-python
  0 ap-python>bmake show-var VARNAME=VARIANTS
  PYTHON_VERSION_DEFAULT=23,21
  0 ap-python>cd ../../www/ap2-chroot
  0 ap2-chroot>bmake show-var VARNAME=VARIANTS
  PKG_APACHE_DEFAULT=apache2,apache22
  0 ap2-chroot>cd ../../net/php-sockets
  0 php-sockets>bmake show-var VARNAME=VARIANTS
  PHP_VERSION_DEFAULT=4,5
  0 php-sockets>

--
Best regards, Aleksey Cheusov.

[Hubert Feyrer-4]

On Tue, 22 Jul 2008, Aleksey Cheusov wrote:
> Patch in the attached file implements a proposed idea.

I think the attachment's missing.

Does what you propose work only for bulk builds, or will a simple "make
package" start building multiple versions?

Also: If something says "i need <some python module", how many versions of
that module (and how many versions of python, if) will be built?


  - Hubert

[Aleksey Cheusov-3]

 >> Patch in the attached file implements a proposed idea.

> I think the attachment's missing.
Oops. Pardon. It is below.

> Does what you propose work only for bulk builds, or will a simple
> "make package" start building multiple versions?
No. But this new variable can be used by, e.g., make all-packages.

> Also: If something says "i need <some python module", how many
> versions of that module (and how many versions of python, if) will
> be built?
No changes to 'make package', just new variable for
all python, php and apache.

---

Index: lang/php/phpversion.mk
===================================================================
RCS file: /cvsroot/pkgsrc/lang/php/phpversion.mk,v
retrieving revision 1.9
diff -u -r1.9 phpversion.mk
--- lang/php/phpversion.mk 11 Oct 2007 08:51:43 -0000 1.9
+++ lang/php/phpversion.mk 22 Jul 2008 15:22:50 -0000
@@ -72,6 +72,9 @@
 _PHP_VERSION_${pv}_OK= yes
 .endfor
 
+# for multi-variant packages and bulk build software
+VARIANTS+= PHP_VERSION_DEFAULT=${PHP_VERSIONS_ACCEPTED:ts,}
+
 # check what is installed
 .if exists(${LOCALBASE}/lib/php/20020429)
 _PHP_VERSION_4_INSTALLED= yes
Index: lang/python/pyversion.mk
===================================================================
RCS file: /cvsroot/pkgsrc/lang/python/pyversion.mk,v
retrieving revision 1.59
diff -u -r1.59 pyversion.mk
--- lang/python/pyversion.mk 25 Apr 2008 16:27:45 -0000 1.59
+++ lang/python/pyversion.mk 22 Jul 2008 15:22:51 -0000
@@ -83,6 +83,9 @@
 .endif
 .endfor
 
+# for multi-variant packages and bulk build software
+VARIANTS+= PYTHON_VERSION_DEFAULT=${_PYTHON_VERSIONS_ACCEPTED:ts,}
+
 #
 # choose a python version where to add,
 # try to be intelligent
Index: mk/apache.mk
===================================================================
RCS file: /cvsroot/pkgsrc/mk/apache.mk,v
retrieving revision 1.25
diff -u -r1.25 apache.mk
--- mk/apache.mk 17 Dec 2007 22:09:58 -0000 1.25
+++ mk/apache.mk 22 Jul 2008 15:23:07 -0000
@@ -70,6 +70,9 @@
 _APACHE_PKG_PREFIX.apache22= ap22
 _APACHE_PKGSRCDIR.apache22= ../../www/apache22
 
+# for multi-variant packages and bulk build software
+VARIANTS+= PKG_APACHE_DEFAULT=${PKG_APACHE_ACCEPTED:ts,}
+
 #
 # Sanity checks.
 #

---

--
Best regards, Aleksey Cheusov.

[Hubert Feyrer-4]

On Tue, 22 Jul 2008, Aleksey Cheusov wrote:
> Oops. Pardon. It is below.

IC... it adds definitions to the pkg Makefiles (and friends),
but the bulk build systems in pkgsrc don't make use of it.
No idea what people think about plugging in support for external build
systems that way...


  - Hubert

[Aleksey Cheusov-3]

> On Tue, 22 Jul 2008, Aleksey Cheusov wrote:
 >> Oops. Pardon. It is below.

> IC... it adds definitions to the pkg Makefiles (and friends), but
> the bulk build systems in pkgsrc don't make use of it.  No idea what
> people think about plugging in support for external build systems
> that way...
First, this variable is not only for distbb. It obviously may be used
by pbulk of bulk build framework easily. Second, based on this
variable you/and_me_too can create 'make all-packages' target or
something else. I didn't think about applications much, though.

This addition just makes pkgsrc external API more functional and
universal. As I already said it is trivial for me to list/hardcode
python/php/apache special variables inside
distbb (share/distbb/distbb.mk) just like pbulk did.
But I think this is bad design. It is better to implemented such things
inside pkgsrc. In this case changes in packages should not lead to changes
in bulk build system. Say, what happened if
emacs20/21/22/xemacs becomes multi-variant package?
pbulk will need changes...

And finally, what does "external build system" mean?
Is distbb closed-source? ;-)

--
Best regards, Aleksey Cheusov.

[Joerg Sonnenberger]

On Wed, Jul 23, 2008 at 10:04:29AM +0300, Aleksey Cheusov wrote:
> This addition just makes pkgsrc external API more functional and
> universal. As I already said it is trivial for me to list/hardcode
> python/php/apache special variables inside
> distbb (share/distbb/distbb.mk) just like pbulk did.

You seem to have a misconception of what "hard-coding" means.
pbulk itself doesn't care at all about the list, it just calls a target
in the infrastructure to find out what packages are available.

Keeping the list of enumeration variables in one place in IMO much
preferable as this list is constant and means that filtering logic can
be done in one place. So this *is* done in pkgsrc.

Joerg

[Aleksey Cheusov-3]

> Keeping the list of enumeration variables in one place in IMO much
> preferable as this list is constant and means that filtering logic can
> be done in one place.
One place in pbulk, one place in distbb, one place in bb framework etc.
Three places in total. Is this still "one place"?

"This list is constant"? I don't think so. Now pkgsrc supports
multi-variant packages only for python/php/apache. But real candidate
to be fourth already exists. It is emacs. Currently binaries for emacs
"modules" are built for only one version of emacs according to
EMACS_TYPE variable.  I personally don't think this is good. As a
maintainer and author of textproc/dictem I'd like to see the following
binaries: emacs21-dictem, emacs22-dictem and xemacs-dictem. The same for
other emacs modules. There are lots of them.

So, "this list" is not constant and may be extended in the future.
And in this case there is no reason to add new multi-variant packages
(appropriate variable names) into "one place": pbulk, distbb
bb framework etc., even if "this list" is not very big in pkgsrc today.


A few weeks ago you've added new variable BOOTSTRAP_PKG to pkgsrc
which is set to "yes" for bootstrap packages. I guess this is for your
pbulk. Right? Why you don't keep this constant list in "one place"? ;-)

Newly created BOOTSTRAP_PKG variable does the same thing for bootstrap
packages as VARIANTS variable does for multi-variant packages.

--
Best regards, Aleksey Cheusov.

[Joerg Sonnenberger]

On Wed, Jul 23, 2008 at 02:00:14PM +0300, Aleksey Cheusov wrote:
> > Keeping the list of enumeration variables in one place in IMO much
> > preferable as this list is constant and means that filtering logic can
> > be done in one place.
> One place in pbulk, one place in distbb, one place in bb framework etc.
> Three places in total. Is this still "one place"?

Nothing stops you from extracting the same data.

> So, "this list" is not constant and may be extended in the future.

Just because this list can be extended doesn't mean it changes during
one scan.

> A few weeks ago you've added new variable BOOTSTRAP_PKG to pkgsrc
> which is set to "yes" for bootstrap packages. I guess this is for your
> pbulk. Right? Why you don't keep this constant list in "one place"? ;-)

You are confusing infrastructure and packages. BOOTSTRAP_PKG is
inherently a package attribute. It is not specific to pbulk as it will
be used to make replace/update handling smarter at some point in the
future.

Joerg

[Aleksey Cheusov-3]

 >> One place in pbulk, one place in distbb, one place in bb framework etc.
 >> Three places in total. Is this still "one place"?

> Nothing stops you from extracting the same data.

I have already explained several times why it is bad to copy-paste the
same code (CHANGEABLE list of special variables) to many places.
1) It is bad design/hardcode
2) If the list of these special variables is changed, all
   "one places" should be changed too. I don't see any reason for this.

The really good "one place" is ONE special variable that keeps
all PYTHONs, PHPs, APACHEs and all variations for this particular package.

Again. This "constant list" is not as constant as you may think.
You've completely ignored my arguments about emacs.

 >> So, "this list" is not constant and may be extended in the future.

> Just because this list can be extended doesn't mean it changes
> during one scan.
How "one scan" is connected with my proposal and all this discussion? :-)

 >> A few weeks ago you've added new variable BOOTSTRAP_PKG to pkgsrc
 >> which is set to "yes" for bootstrap packages. I guess this is for your
 >> pbulk. Right? Why you don't keep this constant list in "one place"? ;-)

> BOOTSTRAP_PKG is inherently a package attribute. It is not specific
> to pbulk

BOOTSTRAP_PKG is a package attribute just like VARIANTS.

BOOTSTRAP_PKG says "this package is bootstrap package, handle it specially".

VARIANTS says "this package can generate a list of binaries, value of this
               variable says what variables can be changed to what values"

VARIANTS is also not specific to distbb, just like BOOTSTRAP_PKG.

As you may know I've created and maintain wip/pkg_online - client/server
tool for search in pkgsrc packages. The following command

  pkg_online_find n:<search_strategy>:<query>
   or
  pkg_online_find PKGNAME:<search_strategy>:<query>

searches for packages by PKGNAME field. Raw data for pkg_online server
is generated by pkg_src_summary program, also a part of pkg_summary-utils.
Currently pkg_src_summary tool generates summaries (pkg_src_summary)
only for "default" variant of multi-variant packages. VARIANTS variable
allows me EASILY adapt pkg_src_summary to generate summaries for all
packages and therefore I'll be able to match not only emacs22-dictem,
but also dictem21-dictem and xemacs-dictem.

This ability is needed for those who prefer binary upgrades
and uses pkg_online.

Another examples is ability to build all packages by running, e.g.,
'make all-packages' or third party tool implemented outside pkgsrc.
By using VARIANTS variable you can implement this for years without
extra knowledge about pkgsrc.

If case "constant list" will change, you should not change
pkg_src_summary, "make all-packages", distbb or anything else.

--
Best regards, Aleksey Cheusov.

[Quentin Garnier]

On Sun, Jul 20, 2008 at 03:33:56PM +0300, Aleksey Cheusov wrote:

> In pkgsrc there are a number of packages (category/package) that can
> be built with different PKGNAME depending on a value of some special
> variables. For example, www/ap2-* packages, */py-* and many others.
>
> AFAIU in order to buils these packages pbulk uses hardcoded list of
> special variables in mk/pbulk/pbulk-index.mk. For multi-variant
> packages my distbb currently builds only package, the default one, but
> I don't thing the way pbulk works is the right way.
>
> Instead, I propose to introduce one special variable, e.g. VARIANTS,
> and to support it in all multi-variant packages.
>
> Example:
>   cd /usr/pkgsrc/www/py-clearsilver
>   make show-var VARNAME=VARIANTS
>   > PYTHON_VERSION_DEFAULT=15,20,21,22,23,25
>   cd /usr/pkgsrc/www/ap2-auth-external
>   make show-var VARNAME=VARIANTS
>   > PKG_APACHE_DEFAULT=apache2,apache22
>   cd /usr/pkgsrc/www/ap2-python
>   > PKG_APACHE_DEFAULT=apache2,apache22 PYTHON_VERSION_DEFAULT=23,24,25
>
> Using cartesian product of all supported values for all listed
> variables, bulk build software can easily build all variants all such
> packages. I hope the idea is clear.
>
> Of course the code that sets VARIANTS variable should use appropriate
> xxx_ACCEPTED variables.
>
> One of the main benefit of this approch is that pkgsrc itself and bulk
> build programs become more isolated and independant from each other
> because pkgsrc API becomes clearer and easier.
>
> Any thoughts?

You are mixing two different problems:

1.  Knowing exactly all the parameters that influence the build of a
    given package.

    The {PYTHON,PHP,APACHE}_ACCEPTED variables are used to know if you
    can build a package in a given environment.  While there may be a
    point in having a framework to handle multiple versions of packages,
    the goal is not to make a list of all possible combinations.

    Either the user controls his environment and sets the appropriate
    variables to set the specific version of such pieces of software she
    wants, or lets pkgsrc decide, using the _DEFAULT values.

    Back in pkgviews days, you could have multiple versions of a package
    in the same installation, but that approach ultimately failed
    because in the end it wasn't that much simpler than having several
    LOCALBASEs.

    What I feel is missing currently is a way for the user to retrieve
    the information about version dependencies.  A lot of effort was put
    in the options framework with great success, and my humble opinion
    is that such information ("I will build with apache22, per your
    environment" or "I will build with php5, per pkgsrc's default")
    belongs in "make show-options", even though this is slightly
    different from a regular option.

2.  Providing binary packages for all the world and its mother.

    Here I think both pbulk (iow, Joerg) and you are wrong.  pkgsrc's
    flexibility is both its greatest asset and its curse.  From the
    administrator's point of view, the options framework and the
    support for different versions of PHP, Apache and Python is an
    amazing feature.

    However, it is also part of the reason why our binary package
    distribution (actually, the lack of a proper one) is pkgsrc's
    greatest weakness.  Distributing the results of a bulk build is
    easy, but maintaining a binary distribution over the time is where
    we're not to the level of competitors.

    A binary packages distribution has to be seen as a whole, with a
    set of default options and settings that are consistent across all
    packages.  Red Hat, Debian and whoever else provide you with a
    static binary distribution, because that's the only way they can
    sanely maintain and support it.

    That said, there are different uses of bulk package building:  I
    covered the user-oriented one, for which I oppose the idea of
    building multiple versions of a package, with the exception of the
    case where they don't conflict, in which case they should have their
    own package directory anyway.  As for packages depending on any
    version of another package, they should only be built for the pkgsrc
    default version.

    The other use of bulk building is for testing purposes, in which
    case your VARIANTS idea could have merit.  The problem is, if you
    want to test all building possibilities of a package, you also have
    to test all subsets of the options list from the options framework.

    I'll leave as an exercise to the reader to compute how many times
    mplayer and mencoder would be built if we made pbulk or any other
    bulk building system do all possible options combinations.

    The point is, if you really want to formalise that part, you'd
    better go all the way, and I really wish you good luck with that.
    As for me, I think pbulk is already exceeding its expectations in
    allowing such possibilities, and while pbulk can do whatever it
    wants in its world, there is no point is pushing that feature into
    pkgsrc.

--
Quentin Garnier - cube@... - cube@...
"See the look on my face from staying too long in one place
[...] every time the morning breaks I know I'm closer to falling"
KT Tunstall, Saving My Face, Drastic Fantastic, 2007.

[Aleksey Cheusov-3]

> You are mixing two different problems:

> 1.  Knowing exactly all the parameters that influence the build of a
>     given package.
... and changes package's PKGNAME

This is significant. I hope it is always true for php/python/apache modules.
Otherwise this is a problem and it should be resolved.

>     While there may be a point in having a framework to handle
>     multiple versions of packages, the goal is not to make a list of
>     all possible combinations.
The goal is to generate binaries for all valid combinations
of apache/php/python. And a list of all possible combinations
in the variable VARIANTS allows me to do this.

>     Back in pkgviews days, you could have multiple versions of a package
>     in the same installation, but that approach ultimately failed
>     because in the end it wasn't that much simpler than having several
>     LOCALBASEs.
Whether py23-xxx, py24-xxx and py25-xxx packages conflicts
or not is a completely different question. My proposal and this
problem/question are completely independant.

>  A lot of effort was put in the options framework with great
>  success, and my humble opinion is that such information ("I will
>  build with apache22, per your environment" or "I will build with
>  php5, per pkgsrc's default") belongs in "make show-options", even
>  though this is slightly different from a regular option.
I didn't even try to talk about options framework. Again, my proposal
and options framework are completely independant
unless you reimplement php/apache/python
variations through options framework or something more powerful
combining both php/python/apache and options framework.

At this time option framework is outside of my interest and
I'm talking only about php/python/apache modules.

But you directs me to the following thought:

VARIANTS variable I proposed does the same thing for
php/python/apache/emacs?/something_else? as 'make show-option' does
for options framework. They are full analogs.
This is yet another reason why it is good to have it.

> 2.  Providing binary packages for all the world and its mother.

>     The other use of bulk building is for testing purposes, in which
>     case your VARIANTS idea could have merit.  The problem is, if you
>     want to test all building possibilities of a package, you also have
>     to test all subsets of the options list from the options framework.
Lets speak about option framework and VARIANTS differently.
My proposal doesn't relate to options at all.
I think it is possible for me to adapt distbb to
build ALL packages with ALL combinations of ALL options.
But this is not thing I'd like to discuss now.

> As for me, I think pbulk is already exceeding its expectations in
> allowing such possibilities, and while pbulk can do whatever it
> wants in its world, there is no point is pushing that feature into
> pkgsrc.
Oh :-( I'll try to explain once again why I don't want to do the same
things as pbulk/Joerg did.

There are three reasons.

A)

  1) One of the basic principle in programming is "do not copy-paste".
   Do not copy-paste big parts of code, do not copy-paste functions,
   classes, files etc. In general, do not copy-paste repeateable things,
   especially in case such things can easily be changed. Instead,
   separate such things into a separate substance, and then call
   this SUBSTANCE by its NAME. I hope all of you are familiar with
   this simple rule.

  2) In our case "repeated thing" is a list of variables by changing which
   we can build all variants of python/apache/php-based modules.
   These variable are PYTHON_VERSION_REQD, PHP_VERSION_REQD
   and PKG_APACHE_REQD.

   (My patch probably needs to be fixed. s/_DEFAULT/_REQD/ )

   A list of these variable MAY CHANGE in the future.

  3) Another "repeated thing" is a list of variables that keeps
   valid/accepted versions of php/python/apache for any particular
   package.

   These vars are: _PYTHON_VERSIONS_ACCEPTED, PHP_VERSIONS_ACCEPTED
   and PKG_APACHE_ACCEPTED.

   This list may also change in the future.

  4) These two lists are repeatable because they are used (directly
     or indirectly) in at least pbulk, my distbb, again my pkg_summary-utils
     (in pkg_online) and who knows 'make all-packages'.

     I think Four examples are enough to proof "repetition".
     Even two (pbulk and distbb) are enough.

  5) Items 1-4  =>  I ask you: Do not copy-paste these lists over many tools,
     separate them
     into a new substance and NAME it. BTW, I don't insist on the name
     VARIANTS, if you have better idea, rename it...

B) Currently there is no analog of 'make show-options'
   for php/python/apache modules. And I thing VARIANTS variable is
   good candidate for it.

C) There are some people which think that existence of my distbb is
   good thing even if it is not official tool. They said something like
   "competence is good, let's see". If you/they really think so,
   pleeeeeeeeeeease add VARIANTS variable to pkgsrc or implement
   similar functionality in any other way (I need FUNCTIONALITY).

   I hope pkgsrc developers are not against third party software.
   Is it SO HARD to add so small functionality to pkgsrc
   really needed by third party software? While developing distbb
   I NEVER asked you to extend pkgsrc API before, because it is
   enough in most cases.

   I tried to use your pbulk because I'm tired of it, sorry.  I don't
   use it anymore and I will not use it in nearest future.

   Also note, that I'm not the only person who uses distbb and
   binary packages it creates. I/we also need better support
   binary repository.

--
Best regards, Aleksey Cheusov.

[Aleksey Cheusov-3]

>     The other use of bulk building is for testing purposes, in which
>     case your VARIANTS idea could have merit.  The problem is, if you
>     want to test all building possibilities of a package, you also have
>     to test all subsets of the options list from the options framework.
[...]
>     The point is, if you really want to formalise that part, you'd
>     better go all the way, and I really wish you good luck with that.

After a few minutes after sending my previous email, I realised that
distbb with support for VARIANTS will be able to be used for
generating all binaries for option-enabled packages automatically,
that is without additional changes to distbb itself.

For this to work the following things should be implemented (except
VARIANTS and distbb support for it, of course)

(The following is make-like pseudocode)

1) category/package/Makefile:
   ...
   PKGNAME=general-package-name${OPTIONS_SUFX}-version
   ...

2) somewhere in pkgsrc .mk files
   .if defined(GENERATE_ALL_OPTIONS_PKGS) && defined(PKG_SUPPORTED_OPTIONS)
       VARIANTS+= ...
       # VARIANTS should look like OPTIONS.name1=0,1 OPTIONS.name2=0,1 ...
       # 0 means "feature is disabled", 1 - "enabled"
   .endif

3) .for opt in ${PKG_SUPPORTED_OPTIONS}
      .if ${OPTIONS.${opt}} = 1
          OPTIONS_SUFX += -${f} # string concatenation here, not +=
      .endif
   .endfor

That's all ;-)

That is bulk building for both multi-php/apache/python modules and
packages with multiple options can be implemented through universal
VARIANTS variable.

IIRC jlam@ wanted this functionality a few months ago.
And he was not alone.

--
Best regards, Aleksey Cheusov.

[Dennis den Brok-2]

Aleksey Cheusov <cheusov@...> schrieb:

> 1) category/package/Makefile:
>    ...
>    PKGNAME=general-package-name${OPTIONS_SUFX}-version
>    ...
>
> 2) somewhere in pkgsrc .mk files
>    .if defined(GENERATE_ALL_OPTIONS_PKGS) && defined(PKG_SUPPORTED_OPTIONS)
>        VARIANTS+= ...
>        # VARIANTS should look like OPTIONS.name1=0,1 OPTIONS.name2=0,1 ...
>        # 0 means "feature is disabled", 1 - "enabled"
>    .endif
>
> 3) .for opt in ${PKG_SUPPORTED_OPTIONS}
>       .if ${OPTIONS.${opt}} = 1
>           OPTIONS_SUFX += -${f} # string concatenation here, not +=
>       .endif
>    .endfor
>

How do you plan to organize the resulting binary packages for all
possible combinations of options?

--
Dennis den Brok

[Aleksey Cheusov-3]

 >> 1) category/package/Makefile:
 >>    ...
 >>    PKGNAME=general-package-name${OPTIONS_SUFX}-version
 >>    ...
 >>
 >> 2) somewhere in pkgsrc .mk files
 >>    .if defined(GENERATE_ALL_OPTIONS_PKGS) && defined(PKG_SUPPORTED_OPTIONS)
 >>        VARIANTS+= ...
 >>        # VARIANTS should look like OPTIONS.name1=0,1 OPTIONS.name2=0,1 ...
 >>        # 0 means "feature is disabled", 1 - "enabled"
 >>    .endif
 >>
 >> 3) .for opt in ${PKG_SUPPORTED_OPTIONS}
 >>       .if ${OPTIONS.${opt}} = 1
 >>           OPTIONS_SUFX += -${f} # string concatenation here, not +=
 >>       .endif
 >>    .endfor
 >>

> How do you plan to organize the resulting binary packages for all
> possible combinations of options?

When you say "to organize" you mean dependancies between
packages having options?

First of all, all options should be splitted into two categories.

Category 1: options with GLOBALLY THE SAME SEMANTICS,
   that is option with the same semantics for ALL packages.

   For example, disabling -x11 option should ALWAYS mean
   "no X support at all", and disabling -ssl option should ALWAYS
   mean "no ssl support at all" for ALL packages.

Category 2: options meaning different things for different packages.
   Solutions:
    a) avoid options of this category at all. That is, all package-specific
       options MUST have a uniq name.
    b) mark such options somehow and use this marker.

After this, I presume dependancies can be set relatively easily.
If, say, package A depends on package B and both they have option, e.g. x11,
then binary package A-version%x11.tgz depends on  B-version%x11.tgz
and package A-version%nox11.tgz depends on B-version%nox11.tgz.
In this example I assume that option x11 is of "category 1".
Here % is a char-separator between "old plain PKGNAME" and options part.

--
Best regards, Aleksey Cheusov.

[Quentin Garnier]

On Wed, Jul 23, 2008 at 08:23:28PM +0300, Aleksey Cheusov wrote:
[...]

> When you say "to organize" you mean dependancies between
> packages having options?
>
> First of all, all options should be splitted into two categories.
>
> Category 1: options with GLOBALLY THE SAME SEMANTICS,
>    that is option with the same semantics for ALL packages.
>
>    For example, disabling -x11 option should ALWAYS mean
>    "no X support at all", and disabling -ssl option should ALWAYS
>    mean "no ssl support at all" for ALL packages.
>
> Category 2: options meaning different things for different packages.
>    Solutions:
>     a) avoid options of this category at all. That is, all package-specific
>        options MUST have a uniq name.
>     b) mark such options somehow and use this marker.

This is already the policy.  Whether or not it is properly respected is
another debate, but feel free to send patches.  Also, they're supposed
to be documented.  Again, patches welcomed.

> After this, I presume dependancies can be set relatively easily.
> If, say, package A depends on package B and both they have option, e.g. x11,
> then binary package A-version%x11.tgz depends on  B-version%x11.tgz
> and package A-version%nox11.tgz depends on B-version%nox11.tgz.

There is no reason for options to correlate that way between packages.

--
Quentin Garnier - cube@... - cube@...
"See the look on my face from staying too long in one place
[...] every time the morning breaks I know I'm closer to falling"
KT Tunstall, Saving My Face, Drastic Fantastic, 2007.