Gentoo Clustering LiveCD/DVD (Google SoC 2008 project)

[Eric Thibodeau-2]

Hello all,

    As some of you may already know, I am working on a clustering
livedvd. Although a livecvd is flashy and all, my efforts are geared
towards making ebuilds and meta-ebuilds that ease setting up a freshly
installed stage3 system into a Master Beowulf server with
PXE+tftp+NFSroot booted nodes. My progressing work is available on the
unofficial Gentoo overlay server (http://git.overlays.gentoo.org/gitweb/ 
, direct link being:
http://git.overlays.gentoo.org/gitweb/?p=proj/clustering-livecd.git;a=summary).
Everything needed to create the livedvd as well as the NFSroot images
(notably catalyst specs and custom ebuilds) is available there. I also
make the resulting/progressing work available in my personal web server
http://wiki.neuralbs.com/~kyron/soc2008/ but, please, don't download the
iso image unless you've asked me first since it's not complete work and
is not "functional" per say (the current image does not contain the
node's nfsroot and the auto-configuration scripts are still being worked on)

    So, this means it's now the perfect time to send me your wish list.
Since we're all interested in seeing Gentoo "sell" as a powerful
platform for HPC, it would be nice if some of you could give me good
examples of apps that give some nice "marketable" GUI results ;) as well
as more seriously used HPC apps (less the marketing fluff).

    Apart from the wish list, I think a custom theme would also be nice.
So a bootsplah, I heard that _really_ sells with the CEOs, as well as
some other nifty fb stuff would be willingly integrated into the livedvd ;)

That's it for now, just wanted to make an official thread since I seem
to have forgotten to do so ;)

Eric Thibodeau (aka kyron on freenode.net)
--
gentoo-cluster@... mailing list

[Alexandre Racine-2]

Well... one vote for SGE.



Alexandre Racine
alexandre.racine@...
514-461-1300 poste 3304

> -----Original Message-----
> From: Eric Thibodeau [mailto:kyron@...]
> Sent: 7 juillet 2008 10:38
> To: gentoo-cluster@...
> Subject: [gentoo-cluster] Gentoo Clustering LiveCD/DVD (Google SoC
2008

> project)
>
> Hello all,
>
>     As some of you may already know, I am working on a clustering
> livedvd. Although a livecvd is flashy and all, my efforts are geared
> towards making ebuilds and meta-ebuilds that ease setting up a freshly
> installed stage3 system into a Master Beowulf server with
> PXE+tftp+NFSroot booted nodes. My progressing work is available on the
> unofficial Gentoo overlay server
> (http://git.overlays.gentoo.org/gitweb/
> , direct link being:
> http://git.overlays.gentoo.org/gitweb/?p=proj/clustering-
> livecd.git;a=summary).
> Everything needed to create the livedvd as well as the NFSroot images
> (notably catalyst specs and custom ebuilds) is available there. I also
> make the resulting/progressing work available in my personal web

server
> http://wiki.neuralbs.com/~kyron/soc2008/ but, please, don't download
> the
> iso image unless you've asked me first since it's not complete work
and
> is not "functional" per say (the current image does not contain the
> node's nfsroot and the auto-configuration scripts are still being
> worked on)
>
>     So, this means it's now the perfect time to send me your wish
list.

> Since we're all interested in seeing Gentoo "sell" as a powerful
> platform for HPC, it would be nice if some of you could give me good
> examples of apps that give some nice "marketable" GUI results ;) as
> well
> as more seriously used HPC apps (less the marketing fluff).
>
>     Apart from the wish list, I think a custom theme would also be
> nice.
> So a bootsplah, I heard that _really_ sells with the CEOs, as well as
> some other nifty fb stuff would be willingly integrated into the
> livedvd ;)
>
> That's it for now, just wanted to make an official thread since I seem
> to have forgotten to do so ;)
>
> Eric Thibodeau (aka kyron on freenode.net)
> --
> gentoo-cluster@... mailing list

--
gentoo-cluster@... mailing list

[Carlos Moyano Cubillos]

Is possible to include Globus Toolkit?



2008/7/7, Alexandre Racine <Alexandre.Racine@...>:

> Well... one vote for SGE.
>
>
>
>  Alexandre Racine
>
> alexandre.racine@...
>  514-461-1300 poste 3304
>
>
>  > -----Original Message-----
>  > From: Eric Thibodeau [mailto:kyron@...]
>  > Sent: 7 juillet 2008 10:38
>  > To: gentoo-cluster@...
>  > Subject: [gentoo-cluster] Gentoo Clustering LiveCD/DVD (Google SoC
>  2008
>  > project)
>  >
>  > Hello all,
>  >
>  >     As some of you may already know, I am working on a clustering
>  > livedvd. Although a livecvd is flashy and all, my efforts are geared
>  > towards making ebuilds and meta-ebuilds that ease setting up a freshly
>  > installed stage3 system into a Master Beowulf server with
>  > PXE+tftp+NFSroot booted nodes. My progressing work is available on the
>  > unofficial Gentoo overlay server
>  > (http://git.overlays.gentoo.org/gitweb/
>  > , direct link being:
>  > http://git.overlays.gentoo.org/gitweb/?p=proj/clustering-
>  > livecd.git;a=summary).
>  > Everything needed to create the livedvd as well as the NFSroot images
>  > (notably catalyst specs and custom ebuilds) is available there. I also
>  > make the resulting/progressing work available in my personal web
>  server
>  > http://wiki.neuralbs.com/~kyron/soc2008/ but, please, don't download
>  > the
>  > iso image unless you've asked me first since it's not complete work
>  and
>  > is not "functional" per say (the current image does not contain the
>  > node's nfsroot and the auto-configuration scripts are still being
>  > worked on)
>  >
>  >     So, this means it's now the perfect time to send me your wish
>  list.
>  > Since we're all interested in seeing Gentoo "sell" as a powerful
>  > platform for HPC, it would be nice if some of you could give me good
>  > examples of apps that give some nice "marketable" GUI results ;) as
>  > well
>  > as more seriously used HPC apps (less the marketing fluff).
>  >
>  >     Apart from the wish list, I think a custom theme would also be
>  > nice.
>  > So a bootsplah, I heard that _really_ sells with the CEOs, as well as
>  > some other nifty fb stuff would be willingly integrated into the
>  > livedvd ;)
>  >
>  > That's it for now, just wanted to make an official thread since I seem
>  > to have forgotten to do so ;)
>  >
>  > Eric Thibodeau (aka kyron on freenode.net)
>  > --
>  > gentoo-cluster@... mailing list
>
>  --
>  gentoo-cluster@... mailing list
>
>

--
C.M.C.
--
gentoo-cluster@... mailing list

[Eric Thibodeau-2]

Ok, I forgot to mention some restrictions:

1- please make sure the packages are in portage or at least in some
"official" overlay
2- verify that the package's license admits redistribution, there are
unfortunately many sci FOSS that are shy when it comes to
"redistribution" unfortunately

Note that I might accept some ebuilds to put directly into the CD's
overlay if they prove to be quite useful (can impress people by their
functions on a cluster ;P )

Eric
Eric Thibodeau wrote:

> Hello all,
>
>    As some of you may already know, I am working on a clustering
> livedvd. Although a livecvd is flashy and all, my efforts are geared
> towards making ebuilds and meta-ebuilds that ease setting up a freshly
> installed stage3 system into a Master Beowulf server with
> PXE+tftp+NFSroot booted nodes. My progressing work is available on the
> unofficial Gentoo overlay server
> (http://git.overlays.gentoo.org/gitweb/ , direct link being:
> http://git.overlays.gentoo.org/gitweb/?p=proj/clustering-livecd.git;a=summary).
> Everything needed to create the livedvd as well as the NFSroot images
> (notably catalyst specs and custom ebuilds) is available there. I also
> make the resulting/progressing work available in my personal web
> server http://wiki.neuralbs.com/~kyron/soc2008/ but, please, don't
> download the iso image unless you've asked me first since it's not
> complete work and is not "functional" per say (the current image does
> not contain the node's nfsroot and the auto-configuration scripts are
> still being worked on)
>
>    So, this means it's now the perfect time to send me your wish list.
> Since we're all interested in seeing Gentoo "sell" as a powerful
> platform for HPC, it would be nice if some of you could give me good
> examples of apps that give some nice "marketable" GUI results ;) as
> well as more seriously used HPC apps (less the marketing fluff).
>
>    Apart from the wish list, I think a custom theme would also be
> nice. So a bootsplah, I heard that _really_ sells with the CEOs, as
> well as some other nifty fb stuff would be willingly integrated into
> the livedvd ;)
>
> That's it for now, just wanted to make an official thread since I seem
> to have forgotten to do so ;)
>
> Eric Thibodeau (aka kyron on freenode.net)

--
gentoo-cluster@... mailing list

[NyZeta]

Hi Eric.

I find your current project really great.
I checked the information you've put out and I must say it really looks promising. I also downloaded your portage overlay from nabble. I'm going to give it a try to check it out. I've been playing around with drqueue and transcode farms.

[Eric Thibodeau-2]

Don't hesitate to contact me if you have any difficulties getting it all
off the ground. There is one not-so-trivial bug with the current setup,
diradm (which you _must_ use to create your users (this implies not to
use the gentoo user), has a bug. The workaround:

When creating a "cluster" user, you must specify the userid, ie:

diradm -u 1001 -p mysimplepassword twitt1
diradm -u 1002 -p hissimplepassword twitt2
etc...

I'm obviously forgetting some other quirks but this was the last one
from the last commit. Quite frankly, if you have a chance to actually
install onto an HDD, I would suggest that approach instead (use the
ebuilds in the git) since the environment is not volatile ;)

Eric

NyZeta wrote:
> Hi Eric.
>
> I find your current project really great.
> I checked the information you've put out and I must say it really looks
> promising. I also downloaded your portage overlay from nabble. I'm going to
> give it a try to check it out. I've been playing around with drqueue and
> transcode farms