Approach to Configuring Apache with Perl -- multiple servers in cluster

[Berg, Eric]

I've got a cluster of 4 Linux boxes that will be sitting behind a load
balancer using a VIP and round-robin load balancing.  There will be 3
different instances that I'll be serving up on 3 different ports that'll
be my DEV, STAGE and PROD instances.

Our environment is set up so that I autodeploy not only my code, but my
apache configurations via CVS hooks.  (Yes, it's very cool.)  I need to
build a configuration that will configure each instance based on a
single parameter.  Right now I'm using a file called ENVIRONMENT that
gets written during the deploy, so I have a value for DEV, STAGE and
PROD.

I need to set the following:

- DocumentRoot
- Listen
- ServerRoot
- HostName
- @INC manipulation to point to our dev, stage and prod lib dirs
- a number of environment variables

What I'm looking to you good people for is some ideas about best
practices for doing this kind of configuration.

Here are some of my questions:

- Apache2's configuration is, by default, broken up into a number of
Includes for files in the extras directory, so, what's the "right' way
to break out your configs into these files?
- Does it matter when you configure the items listed above.  Is there
any reason to set any of them earlier rather than later?
- Is there a way to set a sort of "global" config item that can easily
be retrieved so that I can set a variable early on that can be used
later in the configuration?

Thanks for any input you may have.

Eric

Eric D. Berg

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

This message is intended only for the personal and confidential use of the designated recipient(s) named above.  If you are not the intended recipient of this message you are hereby notified that any review, dissemination, distribution or copying of this message is strictly prohibited.  This communication is for information purposes only and should not be regarded as an offer to sell or as a solicitation of an offer to buy any financial product, an official confirmation of any transaction, or as an official statement of Lehman Brothers.  Email transmission cannot be guaranteed to be secure or error-free.  Therefore, we do not represent that this information is complete or accurate and it should not be relied upon as such.  All information is subject to change without notice.

--------
IRS Circular 230 Disclosure:
Please be advised that any discussion of U.S. tax matters contained within this communication (including any attachments) is not intended or written to be used and cannot be used for the purpose of (i) avoiding U.S. tax related penalties or (ii) promoting, marketing or recommending to another party any transaction or matter addressed herein.

[Perrin Harkins]

On Thu, Aug 7, 2008 at 11:22 AM, Berg, Eric <eric.berg@...> wrote:

> I need to
> build a configuration that will configure each instance based on a
> single parameter.  Right now I'm using a file called ENVIRONMENT that
> gets written during the deploy, so I have a value for DEV, STAGE and
> PROD.
>
> I need to set the following:
>
> - DocumentRoot
> - Listen
> - ServerRoot
> - HostName
> - @INC manipulation to point to our dev, stage and prod lib dirs
> - a number of environment variables
>
> What I'm looking to you good people for is some ideas about best
> practices for doing this kind of configuration.

I recommend just using a template and writing out your config files.
It's much simpler than trying to wedge dynamic stuff into apache
configuration and it works for proxy servers that don't run mod_perl.

> - Apache2's configuration is, by default, broken up into a number of
> Includes for files in the extras directory, so, what's the "right' way
> to break out your configs into these files?

There's no reason to follow that example if it doesn't appeal to you.
Just use one file if that's how you're used to doing it.

- Perrin

[Berg, Eric]

Thanks, Perrin and Jon.

I'm a little strapped for time at the moment, so I'm going to branch
each different version for dev, stage and prod.  The differences between
the different hosts is more easily handled within <Perl> sections.

Since our deployment process comprises Perl scripts that are called via
CVS hooks for tagging (i.e., we set a tag of 'PROD' and the code gets
deployed to our prod instance's server root), it will be pretty
straight-forward to replace the simple process of checking out of CVS
and copying the file into the server root with one that checks out a
template, renders it, and places the resulting string into a file in the
server root.  Or...I've never used ttree before.

I actually like the separate file approach.  I find the idea of
monolithic files difficult to manage, whereas that of having discrete
areas of functionality which have their configurations segregated
logically and in such a way as to make it easy to remove their Include
lines appeal to me.

Thanks again, guys.

Eric

> -----Original Message-----
> From: pharkins@... [mailto:pharkins@...] On
> Behalf Of Perrin Harkins
> Sent: Thursday, August 07, 2008 5:11 PM
> To: Berg, Eric
> Cc: modperl@...
> Subject: Re: Approach to Configuring Apache with Perl --
> multiple servers in cluster
>
> On Thu, Aug 7, 2008 at 11:22 AM, Berg, Eric
> <eric.berg@...> wrote:
> > I need to
> > build a configuration that will configure each instance based on a
> > single parameter.  Right now I'm using a file called
> ENVIRONMENT that
> > gets written during the deploy, so I have a value for DEV, STAGE and
> > PROD.
> >
> > I need to set the following:
> >
> > - DocumentRoot
> > - Listen
> > - ServerRoot
> > - HostName
> > - @INC manipulation to point to our dev, stage and prod lib dirs
> > - a number of environment variables
> >
> > What I'm looking to you good people for is some ideas about best
> > practices for doing this kind of configuration.
>
> I recommend just using a template and writing out your config files.
> It's much simpler than trying to wedge dynamic stuff into apache
> configuration and it works for proxy servers that don't run mod_perl.
>
> > - Apache2's configuration is, by default, broken up into a number of
> > Includes for files in the extras directory, so, what's the
> "right' way
> > to break out your configs into these files?
>
> There's no reason to follow that example if it doesn't appeal to you.
> Just use one file if that's how you're used to doing it.
>
> - Perrin
>

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

This message is intended only for the personal and confidential use of the designated recipient(s) named above.  If you are not the intended recipient of this message you are hereby notified that any review, dissemination, distribution or copying of this message is strictly prohibited.  This communication is for information purposes only and should not be regarded as an offer to sell or as a solicitation of an offer to buy any financial product, an official confirmation of any transaction, or as an official statement of Lehman Brothers.  Email transmission cannot be guaranteed to be secure or error-free.  Therefore, we do not represent that this information is complete or accurate and it should not be relied upon as such.  All information is subject to change without notice.

--------
IRS Circular 230 Disclosure:
Please be advised that any discussion of U.S. tax matters contained within this communication (including any attachments) is not intended or written to be used and cannot be used for the purpose of (i) avoiding U.S. tax related penalties or (ii) promoting, marketing or recommending to another party any transaction or matter addressed herein.

[Berg, Eric]

Just wanted to follow up and thank you guys for the template suggestion.
While it seemed like a big change at first, I did go down the path to
different versions being maintained on different branches...for about 5
minutes, and it reeeeaaaally pissed me off.  I immediately got confused
by the different versions, etc.

In any case, in essence, I was able to replace this in our deploy code:

    File::Copy::copy( $from_file, $deployfile )
      || die "Error moving file '$from_file' to '$deployfile': $!";

with this:

    $template->process( $from_file, $template_params, $deploy_file )
      || $log->logdie( $template->error() );

And it's really cool.  The best aspects of using this approach are that
it allows me to have a single set of files, and the resulting files are
very straight-forward.  Whereas I was movign toward using <PERL>
sections to compute DocumentRoot, ServerRoot and a number of other
things, I now have very standard-looking directives, which will make it
much easier to support.

Thanks again.

Eric

Thanks for the great idea, guys.

> -----Original Message-----
> From: Berg, Eric
> Sent: Thursday, August 07, 2008 5:53 PM
> To: 'Perrin Harkins'
> Cc: modperl@...
> Subject: RE: Approach to Configuring Apache with Perl --
> multiple servers in cluster
>
> Thanks, Perrin and Jon.
>
> I'm a little strapped for time at the moment, so I'm going to
> branch each different version for dev, stage and prod.  The
> differences between the different hosts is more easily
> handled within <Perl> sections.
>
> Since our deployment process comprises Perl scripts that are
> called via CVS hooks for tagging (i.e., we set a tag of
> 'PROD' and the code gets deployed to our prod instance's
> server root), it will be pretty straight-forward to replace
> the simple process of checking out of CVS and copying the
> file into the server root with one that checks out a
> template, renders it, and places the resulting string into a
> file in the server root.  Or...I've never used ttree before.
>
> I actually like the separate file approach.  I find the idea
> of monolithic files difficult to manage, whereas that of
> having discrete areas of functionality which have their
> configurations segregated logically and in such a way as to
> make it easy to remove their Include lines appeal to me.
>
> Thanks again, guys.
>
> Eric
>
> > -----Original Message-----
> > From: pharkins@... [mailto:pharkins@...] On
> > Behalf Of Perrin Harkins
> > Sent: Thursday, August 07, 2008 5:11 PM
> > To: Berg, Eric
> > Cc: modperl@...
> > Subject: Re: Approach to Configuring Apache with Perl --
> > multiple servers in cluster
> >
> > On Thu, Aug 7, 2008 at 11:22 AM, Berg, Eric
> > <eric.berg@...> wrote:
> > > I need to
> > > build a configuration that will configure each instance based on a
> > > single parameter.  Right now I'm using a file called
> > ENVIRONMENT that
> > > gets written during the deploy, so I have a value for
> DEV, STAGE and
> > > PROD.
> > >
> > > I need to set the following:
> > >
> > > - DocumentRoot
> > > - Listen
> > > - ServerRoot
> > > - HostName
> > > - @INC manipulation to point to our dev, stage and prod lib dirs
> > > - a number of environment variables
> > >
> > > What I'm looking to you good people for is some ideas about best
> > > practices for doing this kind of configuration.
> >
> > I recommend just using a template and writing out your config files.
> > It's much simpler than trying to wedge dynamic stuff into apache
> > configuration and it works for proxy servers that don't run
> mod_perl.
> >
> > > - Apache2's configuration is, by default, broken up into
> a number of
> > > Includes for files in the extras directory, so, what's the
> > "right' way
> > > to break out your configs into these files?
> >
> > There's no reason to follow that example if it doesn't
> appeal to you.
> > Just use one file if that's how you're used to doing it.
> >
> > - Perrin
> >

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

This message is intended only for the personal and confidential use of the designated recipient(s) named above.  If you are not the intended recipient of this message you are hereby notified that any review, dissemination, distribution or copying of this message is strictly prohibited.  This communication is for information purposes only and should not be regarded as an offer to sell or as a solicitation of an offer to buy any financial product, an official confirmation of any transaction, or as an official statement of Lehman Brothers.  Email transmission cannot be guaranteed to be secure or error-free.  Therefore, we do not represent that this information is complete or accurate and it should not be relied upon as such.  All information is subject to change without notice.

--------
IRS Circular 230 Disclosure:
Please be advised that any discussion of U.S. tax matters contained within this communication (including any attachments) is not intended or written to be used and cannot be used for the purpose of (i) avoiding U.S. tax related penalties or (ii) promoting, marketing or recommending to another party any transaction or matter addressed herein.