compile speed

[Hans-J. Ullrich]

Hi all,

I would like to increase compiling speed of my system (AMD64, dual-core, 2GB
RAM).

I suppose, there are some settings in /etc/apt/apt-build.conf, I can use with
gcc or make (such like NICE=something) or let it run more processes at a
time.

Are there any experiences, somebody made at ? If yes, just let me know.


Thanks !


Hans


--
To UNSUBSCRIBE, email to debian-amd64-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Marcus Beranek-2]

Am Sonntag, den 15.06.2008, 14:01 +0200 schrieb Hans-J. Ullrich:
> Hi all,
>
> I would like to increase compiling speed of my system (AMD64, dual-core, 2GB
> RAM).
>
> I suppose, there are some settings in /etc/apt/apt-build.conf, I can use with
> gcc or make (such like NICE=something) or let it run more processes at a
> time.

Well, I guess, you can start your compiler via "make -j 2" and/or put
something like "make_options = -j2" into "/etc/apt/apt-build.conf" (but
I'm not sure about the correct syntax for apt-build.conf).

The "-j" option of the "make"-command specifies the number of jobs to
run simultaneously. You have a dual-core, so I guess -j2 is a good
start. Maybe -j3 or even more may give you some more speed, you'll have
to try...

You could also disable optimization in gcc (the -O switch), which speeds
up compiling. But that would result in slower/bigger binaries... so i
guess that is not what you want... ;-)


HTH,
Marcus


--
To UNSUBSCRIBE, email to debian-amd64-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Lennart Sorensen]

On Sun, Jun 15, 2008 at 02:01:18PM +0200, Hans-J. Ullrich wrote:
> I would like to increase compiling speed of my system (AMD64, dual-core, 2GB
> RAM).
>
> I suppose, there are some settings in /etc/apt/apt-build.conf, I can use with
> gcc or make (such like NICE=something) or let it run more processes at a
> time.
>
> Are there any experiences, somebody made at ? If yes, just let me know.

if the makefiles are proper (that is they list all dependancies for each
file correctly), then make -j 4 tends to work well for me (that is run 4
things at once, which is 2 per core).

--
Len Sorensen


--
To UNSUBSCRIBE, email to debian-amd64-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Hans-J. Ullrich]

Am Sonntag, 15. Juni 2008 schrieb Lennart Sorensen:

> On Sun, Jun 15, 2008 at 02:01:18PM +0200, Hans-J. Ullrich wrote:
> > I would like to increase compiling speed of my system (AMD64, dual-core,
> > 2GB RAM).
> >
> > I suppose, there are some settings in /etc/apt/apt-build.conf, I can use
> > with gcc or make (such like NICE=something) or let it run more processes
> > at a time.
> >
> > Are there any experiences, somebody made at ? If yes, just let me know.
>
> if the makefiles are proper (that is they list all dependancies for each
> file correctly), then make -j 4 tends to work well for me (that is run 4
> things at once, which is 2 per core).
>
> --
> Len Sorensen

Hi Len,

yes, that is the point, I wanted to know. I suppose, higher than "-j 4" will
let the system make too slow.

Just another last: Are the entries in /etc/apt/apt-build.conf used by any(!)
compiler applications, which are debian based ? I am thinking especially of
commands like "make-kpkg", ncurses tools  like "module-assistant" and dpkg
related things, which are executed during upgrading of packages.


Cheers

Hans  


--
To UNSUBSCRIBE, email to debian-amd64-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Lennart Sorensen]

On Sun, Jun 15, 2008 at 07:19:29PM +0200, Hans-J. Ullrich wrote:
> yes, that is the point, I wanted to know. I suppose, higher than "-j 4" will
> let the system make too slow.
>
> Just another last: Are the entries in /etc/apt/apt-build.conf used by any(!)
> compiler applications, which are debian based ? I am thinking especially of
> commands like "make-kpkg", ncurses tools  like "module-assistant" and dpkg
> related things, which are executed during upgrading of packages.

The thing is, many Makefiles are NOT proper and will result in broken
builds if you try to run them in parallel.  So there is no way you can
just say 'always do this' because sometimes it doesn't work.

I find that 2xCores is a good value, since it tries to make sure each
core has something to compile even while doing disk io for another
compile task.  Any more than that never seems to improve things and just
increases the number of context switches that need to be done.

Another option is to look at gcc's -pipe option, which makes it use
pipes with multiple processes rather than temporary files with each
compile step process run in turn.  It is completely safe no matter what
the Makefile does, but won't likely gain as much, but still better than
nothing.

--
Len Sorensen


--
To UNSUBSCRIBE, email to debian-amd64-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Goswin von Brederlow-2]

lsorense@... (Lennart Sorensen) writes:

> On Sun, Jun 15, 2008 at 07:19:29PM +0200, Hans-J. Ullrich wrote:
>> yes, that is the point, I wanted to know. I suppose, higher than "-j 4" will
>> let the system make too slow.
>>
>> Just another last: Are the entries in /etc/apt/apt-build.conf used by any(!)
>> compiler applications, which are debian based ? I am thinking especially of
>> commands like "make-kpkg", ncurses tools  like "module-assistant" and dpkg
>> related things, which are executed during upgrading of packages.
>
> The thing is, many Makefiles are NOT proper and will result in broken
> builds if you try to run them in parallel.  So there is no way you can
> just say 'always do this' because sometimes it doesn't work.
>
> I find that 2xCores is a good value, since it tries to make sure each
> core has something to compile even while doing disk io for another
> compile task.  Any more than that never seems to improve things and just
> increases the number of context switches that need to be done.

For the kernel I found that -j3 is a few seconds faster than -j4.

But what really makes a difference for me is ccache. It won't hep the
first time you compile something but the second time. I would be
interested to hear about cache hit/miss ratios for apt-build usage.

> Another option is to look at gcc's -pipe option, which makes it use
> pipes with multiple processes rather than temporary files with each
> compile step process run in turn.  It is completely safe no matter what
> the Makefile does, but won't likely gain as much, but still better than
> nothing.

I have half a mind to file a bugreport against gcc to default to
it. The only reason not to use -pipe is when you don't have much rum,
like < 128MiB.

MfG
        Goswin


--
To UNSUBSCRIBE, email to debian-amd64-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Lennart Sorensen]

On Mon, Jun 16, 2008 at 10:36:50AM +0200, Goswin von Brederlow wrote:
> For the kernel I found that -j3 is a few seconds faster than -j4.
>
> But what really makes a difference for me is ccache. It won't hep the
> first time you compile something but the second time. I would be
> interested to hear about cache hit/miss ratios for apt-build usage.

I have never looked at ccache.

> I have half a mind to file a bugreport against gcc to default to
> it. The only reason not to use -pipe is when you don't have much rum,
> like < 128MiB.

or perhaps 512MB if compiling C++.

--
Len Sorensen


--
To UNSUBSCRIBE, email to debian-amd64-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...