Bug#492157: ITP: apollo -- The Apollo Solr Server

[Paul Waite]

Package: wnpp
Severity: wishlist
Owner: Paul Waite <paul@...>

* Package name    : apollo
  Version         : 1.8.0
  Upstream Author :
* URL             : http://lucene.apache.org/solr/
* License         : GPL, Apache 2.0, CDDL 1.0, BSD, MIT
  Programming Lang: (C, C++, C#, Perl, Python, etc.)
  Description     : The Apollo Solr Server

The Apollo Solr Server is a debian packaging of the standard Solr Server
available from the Apache project (http://lucene.apache.org/solr/). This
package can be installed with replication enabled, either as a Master or
a Slave. The latter is set up for you to rsync from the Master via cron.

This apollo package also supports any number of instances of Solr,
running on separate ports. These are managed via a common utility 'apollo'
to provide create, remove, purge, start, stop, restart, and status.

The package also includes a MaoriMacronsFilter plugin which can be set up
in your schema.xml to map macronned characters to stright ascii on both
index and query operations. The default schema.xml has this set up for
the 'text' field type already. It is a trivial exercise to provide other
mappings.

-- System Information:
Debian Release: 4.0
  APT prefers stable
  APT policy: (900, 'stable'), (400, 'stable')
Architecture: i386 (x86_64)
Shell:  /bin/sh linked to /bin/bash
Kernel: Linux 2.6.18-6-amd64
Locale: LANG=en_NZ.UTF-8, LC_CTYPE=en_NZ.UTF-8 (charmap=UTF-8)



--
To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Stefano Zacchiroli]

On Thu, Jul 24, 2008 at 03:47:53PM +1200, Paul Waite wrote:
> The Apollo Solr Server is a debian packaging of the standard Solr Server
> available from the Apache project (http://lucene.apache.org/solr/). This
> package can be installed with replication enabled, either as a Master or
> a Slave. The latter is set up for you to rsync from the Master via cron.

Reading this paragraph I frankly do not understand what this package is
about (neither I do reading the forthcoming paragraphs): the first row
is almost useless as it is evident that we are talking about "debian
packaging" and "Solr Server" does not help in understanding what we are
talking about.

The short description is not explaining what the package is about
either.

Also, the Homepage does not belong to the long description, but to the
Homepage field in the source package stanza.

Please improve the descriptions.
Cheers.

--
Stefano Zacchiroli -*- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
I'm still an SGML person,this newfangled /\ All one has to do is hit the
XML stuff is so ... simplistic  -- Manoj \/ right keys at the right time

[Paul Waite]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Stefano Zacchiroli wrote:

> On Thu, Jul 24, 2008 at 03:47:53PM +1200, Paul Waite wrote:
>> The Apollo Solr Server is a debian packaging of the standard Solr Server
>> available from the Apache project (http://lucene.apache.org/solr/). This
>> package can be installed with replication enabled, either as a Master or
>> a Slave. The latter is set up for you to rsync from the Master via cron.
>
> Reading this paragraph I frankly do not understand what this package is
> about (neither I do reading the forthcoming paragraphs): the first row
> is almost useless as it is evident that we are talking about "debian
> packaging" and "Solr Server" does not help in understanding what we are
> talking about.
>
> The short description is not explaining what the package is about
> either.
>
> Also, the Homepage does not belong to the long description, but to the
> Homepage field in the source package stanza.
>
> Please improve the descriptions.
> Cheers.

Thanks for the feedback. Sometimes when one already knows what the package "is about" it seems
plain that the description is absolutely clear as crystal. But it quite obviously is not!

Here is a (hopefully) better one. Please let me know if it does the trick for you, and thanks for
your time on this.

Cheers,
Paul.


Long description of apollo:

The Apollo Solr Server is a debian packaging of Solr, the open source enterprise search engine which
is available from the Apache project (http://lucene.apache.org/solr/). Solr is provided by the
Apache project in the form of a tarball, which forms the upstream content of apollo. Apollo takes
the tarball, unpacks it and then re-organises it along Debian-compliant lines with configuration
files under /etc, logfiles under /var/log, data under /var/lib, start/stop scripts, and log rotation.

Aside from this basic re-organisation, apollo also allows the user to easily configure the way Solr
works via debconf, where the port Solr listens on, memory allocation and cron-driven replication
settings can be chosen. With the latter, apollo allows easy setup of master-slave replication
between a single master and one or more slave servers.

Apollo also provides a MaoriMacronsFilter plugin which can be set up in your schema.xml to map
macronned characters to stright ascii on both index and query operations. The default schema.xml has
this set up for the 'text' field type already. Although this filtering is specific to the Maori
language, it is also provided as a working example, and from that it is a trivial exercise to
provide other mappings.

Finally, the apollo package has been designed to allow the user to run multiple (only limited by
machine memory and cpu resources) instances of Solr in a cluster. New instances of Solr are created
by unique name, and run on a unique port, and are managed by a simple command-line utility which
provides access to create, remove, purge, start, stop, restart, and instance status functions.

In summary, apollo wraps up the Solr search engine in a very convenient single package, and allows
it to be easily deployed and managed on Debian systems.

(end long description)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFIiC4mtfkpAgkMOyMRAvAPAJ4nOY62fDrt4Wzn7s4QfVuU7YoUogCfavjT
v7X9qw+zLY3ix1L521h8IUk=
=EfYy
-----END PGP SIGNATURE-----



--
To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Stefano Zacchiroli]

On Thu, Jul 24, 2008 at 07:24:22PM +1200, Paul wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Thanks for the feedback. Sometimes when one already knows what the
> package "is about" it seems plain that the description is absolutely
> clear as crystal. But it quite obviously is not!

Hi Paul, thanks for the quick reply, a few more comments are reported
below.

> The Apollo Solr Server is a debian packaging of Solr, the open source enterprise search engine which
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Drop this, it is a description of a Debian package, obviously we are
talking about Debian packaging, rather go for  "Solr is the open source
enterprise search engine which ..."

> is available from the Apache project (http://lucene.apache.org/solr/). Solr is provided by the

As I told you, drop the homepage, it has no role in the long
description, rather use the Homepage field
(http://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Homepage)

> Apache project in the form of a tarball, which forms the upstream content of apollo. Apollo takes
> the tarball, unpacks it and then re-organises it along Debian-compliant lines with configuration
> files under /etc, logfiles under /var/log, data under /var/lib, start/stop scripts, and log rotation.

Drop this paragraph all together, these are packaging details, not
information useful for the sysadm which had to choose whether or not to
install Solr.

> Aside from this basic re-organisation, apollo also allows the user to easily configure the way Solr
> works via debconf, where the port Solr listens on, memory allocation and cron-driven replication
> settings can be chosen. With the latter, apollo allows easy setup of master-slave replication
> between a single master and one or more slave servers.

Most of this can be dropped as well, for similar reasons.

I suggest you to have a look at Developer's Reference 6.2.1
(http://www.debian.org/doc/developers-reference/best-pkging-practices.html#bpp-desc-basics)
to 6.2.4, it is full of useful hints on what should be part of
short/long descriptions and what shouldn't.

Hope this helps,
Cheers.

--
Stefano Zacchiroli -*- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
I'm still an SGML person,this newfangled /\ All one has to do is hit the
XML stuff is so ... simplistic  -- Manoj \/ right keys at the right time



--
To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Paul Waite]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Herewith a synopsis and long description update - please disregard earlier versions of these.

Synopsis:
Solr enterprise search engine built on Lucene

Long description:
Solr is the open source enterprise search engine built on the Lucene Java API and made available
from the Apache project. Solr features full-text search, is optimised for high traffic volumes,
provides a nice XML/HTTP protocol for indexing and querying, supports faceted search, built-in
caching, replication, and search hit highlighting.

This package supports multiple instances of Solr in a cluster, limited only by machine memory and
cpu resources. Instances of Solr can be created by unique name to run on a specified port, and are
easily managed by a command-line utility which provides access to create, remove, purge, start,
stop, restart, and instance status functions.

The package also provides a MaoriMacronsFilter plugin which allows text searching on text written in
the Maori language.


Cheers,
Paul.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFIiEkRtfkpAgkMOyMRAs3pAJ486MGn44b23Lj4dxOzdWH7okELoACguyur
XmZC8uy0SMIzLxWzhpzDTNI=
=zeyc
-----END PGP SIGNATURE-----



--
To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Steve Greenland]

On 23-Jul-08, 22:47 (CDT), Paul Waite <paul@...> wrote:
> * License         : GPL, Apache 2.0, CDDL 1.0, BSD, MIT

Really? All of those?

>   Programming Lang: (C, C++, C#, Perl, Python, etc.)

Really? All of those?

>   Description     : The Apollo Solr Server
>
> The Apollo Solr Server is a debian packaging of the standard Solr Server
> available from the Apache project (http://lucene.apache.org/solr/).

That's nice. Actually, I pretty much assumed is was a debian package,
what with this being Debian and all. I've read the whole long
description, and I still have no idea what this package does or why I
might want it.

Steve




--
To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Paul Waite]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Steve,

> On 23-Jul-08, 22:47 (CDT), Paul Waite <paul@...> wrote:
>> * License         : GPL, Apache 2.0, CDDL 1.0, BSD, MIT
>
> Really? All of those?

Unfortunately yes, I counted 'em ;-)


>>   Programming Lang: (C, C++, C#, Perl, Python, etc.)
>
> Really? All of those?

Nope - that's an 'oops' on my part - should have inserted "Java" there.


>>   Description     : The Apollo Solr Server
>>
>> The Apollo Solr Server is a debian packaging of the standard Solr Server
>> available from the Apache project (http://lucene.apache.org/solr/).
>
> That's nice. Actually, I pretty much assumed is was a debian package,
> what with this being Debian and all. I've read the whole long
> description, and I still have no idea what this package does or why I
> might want it.

Did you see the final submission for synopsis and description (with all
the noise about debian packaging etc remove)? That's a *lot* better. If
that doesn't do it, then I'm not sure what will.

Cheers,
Paul.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIiOsotfkpAgkMOyMRAuD1AJ922YZtESONNEmUmmt9ffH6zKyoxgCdFy6r
C8f7ZRKuNdXwqwV2gZdieZQ=
=VoLK
-----END PGP SIGNATURE-----



--
To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Guus Sliepen]

On Thu, Jul 24, 2008 at 03:47:53PM +1200, Paul Waite wrote:

>   Description     : The Apollo Solr Server

That is not a description, that's just the full name. What does this
package do? I cannot find any answer in the long description either. The
project website mentions that it is an "enterprise search server". Still
vague, but I guess it is something equivalent to htdig. Both the short
and long description should make clear that this is a web-based search
engine/server/whatever.

> The Apollo Solr Server is a debian packaging of the standard Solr Server
> available from the Apache project (http://lucene.apache.org/solr/). This

There is no need to mention that this is a Debian package. Also, the URL
to the project page is already in the Homepage: header, there is no need
to repeat it.

> package can be installed with replication enabled, either as a Master or
> a Slave. The latter is set up for you to rsync from the Master via cron.
>
> This apollo package also supports any number of instances of Solr,
> running on separate ports. These are managed via a common utility 'apollo'
> to provide create, remove, purge, start, stop, restart, and status.
>
> The package also includes a MaoriMacronsFilter plugin which can be set up
> in your schema.xml to map macronned characters to stright ascii on both
> index and query operations. The default schema.xml has this set up for
> the 'text' field type already. It is a trivial exercise to provide other
> mappings.

The rest of the description reads like a manual page, not like a
description of the features of Solr.

Then, where does the name "Apollo" come from? I do not see any
reference to that name on the Solr website.

Finally, it seems Solr is already packaged by the Debian Java
Maintainers, see http://packages.debian.org/solr-tomcat5.5. If there is
anything in your package that is not in theirs, please coordinate with
them.

--
Met vriendelijke groet / with kind regards,
      Guus Sliepen <guus@...>

[Paul Waite]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Guus,

Your other comments are out of date (see the BTS for full bug report
e-mail conversation), however many thanks for your reply. This feedback
is important though:

> Finally, it seems Solr is already packaged by the Debian Java
> Maintainers, see http://packages.debian.org/solr-tomcat5.5. If there is
> anything in your package that is not in theirs, please coordinate with
> them.

I'll start out by saying that I'm quite happy NOT to package this
separately if that's the general consensus. In fact, I hoped this
particular discussion would ensue as I certainly don't want to be
spending my time maintaining a package that nobody wants!

However in reply to the above suggestion, I would point out that apollo
takes a different approach to theirs, as it creates a single instance of
the jetty-based example from the Solr tarball, then constructs a
framework around that which provides two important features lacking in
the above:

  master/slave replication, out-of-the-box from debconf
  multiple Solr instances running on the same machine

Apollo also installs nicely on the current stable (etch) which was one
of the drivers for creating something myself, and not using the above,
which at the time of testing did not. Apollo has been running on etch in
our client production systems for a year now. Given that lenny is not
yet released, and even when it is there will be a *lot* of etch servers
running out there for a long while, I think this is still a very useful
attribute.

I obviously looked at the above Debian Maintainers packages solr-common,
solr-jetty and solr-tomcat initially, when starting out to build the
above-mentioned applications for our clients, however the fact that they
didn't install on our stable (etch) production servers was a blocker.
Hence apollo came into being.

I should also add that apollo is currently built as a native Debian
package, where the Solr tarball is downloaded in the build process, and
then bits of it used to construct the apollo binary package.

I look forward to your comments on the above, and also on this packaging
approach. If you (or anyone) wants a preview of the package, then let me
know.

As I began by saying, it could be that this package isn't deemed to be
useful or suitable for Debian. That's fine with me, my motivation was to
make available something that I had already built and am currently using
in production systems, to other potential users of Solr.

Cheers,
Paul.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIij8DtfkpAgkMOyMRAtZ8AJ94OwcLBVbaqpm0X4ZJXFosxfK5yACfQghi
ZQl/OaTf4xARojKGIDpQU0Y=
=ajyE
-----END PGP SIGNATURE-----



--
To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Guus Sliepen]

On Sat, Jul 26, 2008 at 09:00:52AM +1200, Paul wrote:

[...]
> I should also add that apollo is currently built as a native Debian
> package, where the Solr tarball is downloaded in the build process, and
> then bits of it used to construct the apollo binary package.

Ok I see now that you have added your own enhancements to Solr. In that
case, maybe it is better not to make a native Debian package, but to
create your own "upstream" project, or of course try to get your
enhancements merged with the original Solr project. In case they do not
want to merge, you should make an official Apollo Solr website (you can
use alioth.debian.org for that I think). That way other distributions
can also easily track your progress and make packages out of it.

--
Met vriendelijke groet / with kind regards,
      Guus Sliepen <guus@...>