Bug#492077: debian-installer: avoid searching for libgcc_s.so

[John Reiser]

Subject: debian-installer: avoid searching for libgcc_s.so
Package: debian-installer
Version:
Severity: wishlist

*** Please type your report below this line ***
Many executables used by the installer depend on libgcc_s.so.  Searching for
this library (by ld-linux.so just after execve) takes more than 1% of install
time.  The search tries:
   2375 open("/lib/tls/libgcc_s.so.1", O_RDONLY) = -1 ENOENT
   2373 open("/lib/fast-mult/libgcc_s.so.1", O_RDONLY) = -1 ENOENT
   2372 open("/lib/v5l/libgcc_s.so.1", O_RDONLY) = -1 ENOENT
   2372 open("/lib/v5l/fast-mult/libgcc_s.so.1", O_RDONLY) = -1 ENOENT
   2356 open("/lib/tls/v5l/fast-mult/libgcc_s.so.1", O_RDONLY) = -1 ENOENT
   2355 open("/lib/tls/fast-mult/libgcc_s.so.1", O_RDONLY) = -1 ENOENT
   2351 open("/lib/tls/v5l/libgcc_s.so.1", O_RDONLY) = -1 ENOENT
before finding it in /lib.  [Those counts in the first column are for less than
1/3 of base install on a NSLU2 armel-eabi in low-memory mode.]

The search can be avoided, by forcing it to succeed on the first try:
   export LD_LIBRARY_PATH=/lib
Please apply.


The users of libgcc_s.so include:
ash busybox cat chmod chown cp cttyhack dd df dmesg echo egrep false grep
gunzip gzip ip kill ln ls mkdir mknod more mount mv parted_server pidof ps pwd
rm rmdir sed sh sleep sync tar touch true umount uname zcat libblkid.so.1.0
libext2fs.so.2 libext2fs.so.2.4 libnewt.so.0.52
libparted-1.7.so.1 libparted-1.7.so.1.0.0 libslang.so.2 libuuid.so.1
libuuid.so.1.2


-- System Information:
Debian Release: lenny/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (500, 'testing')
Architecture: armel (armv5tel)

Kernel: Linux 2.6.25-2-ixp4xx
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968)
Shell: /bin/sh linked to /bin/bash

--



--
To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Frans Pop-3]

reassign 492077 rootskel 1.64
retitle 492077 [optimization] d-i: avoid searching for libgcc_s.so
thanks

On Wednesday 23 July 2008, John Reiser wrote:
> The search can be avoided, by forcing it to succeed on the first try:
>    export LD_LIBRARY_PATH=/lib

Thanks for the suggestion.

Sounds reasonable for the Debian Installer environment, but we should
probably not have that set when commands are executed in the target
environment.

We can easily unset LD_LIBRARY_PATH in chroot-setup.sh for apt-install and
in-target, but it's less easy for direct calls of chroot. OTOH, as we've
been moving towards a situation where the rule is "use in-target if you
need a sane target environment" anyway that may not be a major objection.

Making this change should probably be delayed until after the release of
lenny as it could well introduce some subtle breakage.

What do others think?

Cheers,
FJP

[Otavio Salvador]

Frans Pop <elendil@...> writes:

> Making this change should probably be delayed until after the release of
> lenny as it could well introduce some subtle breakage.
>
> What do others think?

Yes. I believe we should avoid any change that can have a global
effect on the installer environment at this stage. This does looks to
be a great optimization to be done as soon as lenny release is out but
I fear to do that before.

 --
        O T A V I O    S A L V A D O R
---------------------------------------------
 E-mail: otavio@...      UIN: 5906116
 GNU/Linux User: 239058     GPG ID: 49A5F855
 Home Page: http://otavio.ossystems.com.br
---------------------------------------------
"Microsoft sells you Windows ... Linux gives
 you the whole house."



--
To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Jérémy Bobbio-2]

On Wed, Jul 23, 2008 at 09:13:06PM +0200, Frans Pop wrote:

> Sounds reasonable for the Debian Installer environment, but we should
> probably not have that set when commands are executed in the target
> environment.
>
> We can easily unset LD_LIBRARY_PATH in chroot-setup.sh for apt-install and
> in-target, but it's less easy for direct calls of chroot. OTOH, as we've
> been moving towards a situation where the rule is "use in-target if you
> need a sane target environment" anyway that may not be a major objection.
>
> Making this change should probably be delayed until after the release of
> lenny as it could well introduce some subtle breakage.
>
> What do others think?

Proposal sounds reasonable, waiting after Lenny too. :)

Cheers,
--
Jérémy Bobbio                        .''`.
lunar@...                    : :Ⓐ  :  # apt-get install anarchism
                                    `. `'`
                                      `-  

[Joey Hess]

John Reiser wrote:

> Many executables used by the installer depend on libgcc_s.so.  Searching for
> this library (by ld-linux.so just after execve) takes more than 1% of install
> time.  The search tries:
>    2375 open("/lib/tls/libgcc_s.so.1", O_RDONLY) = -1 ENOENT
>    2373 open("/lib/fast-mult/libgcc_s.so.1", O_RDONLY) = -1 ENOENT
>    2372 open("/lib/v5l/libgcc_s.so.1", O_RDONLY) = -1 ENOENT
>    2372 open("/lib/v5l/fast-mult/libgcc_s.so.1", O_RDONLY) = -1 ENOENT
>    2356 open("/lib/tls/v5l/fast-mult/libgcc_s.so.1", O_RDONLY) = -1 ENOENT
>    2355 open("/lib/tls/fast-mult/libgcc_s.so.1", O_RDONLY) = -1 ENOENT
>    2351 open("/lib/tls/v5l/libgcc_s.so.1", O_RDONLY) = -1 ENOENT
> before finding it in /lib.  [Those counts in the first column are for less than
> 1/3 of base install on a NSLU2 armel-eabi in low-memory mode.]

joey@kodama:~>time perl -e 'for (1..16625) { open(IN, "/no/file") }'      
0.03user 0.03system 0:00.06elapsed 96%CPU (0avgtext+0avgdata 0maxresident)k

So for a typical install on modern hardware, these 16 thousand extra syscalls
use less than 0.1 second.

I don't have my slug handy to try, but 1% of install time sounds very excessive
even there.

Also, when trying to optimise something, it's generally a good idea
to find an optimiation target that yields significantly better than 1%
improvement.. So, for example, optimising partman to not fork > 1 thousand
child processes when building its menu would probably yield a much more
noticeable benefit. There's a bug in the BTS about that too, IIRC.

> The search can be avoided, by forcing it to succeed on the first try:
>    export LD_LIBRARY_PATH=/lib

That stikes me as potentially a very dangerous hack. Also, it doesn't
really result in fewer syscalls here:

joey@kodama:~>strace busybox 2>&1 |grep open                    
open("/etc/ld.so.cache", O_RDONLY)      = 5
open("/lib/i686/cmov/libm.so.6", O_RDONLY) = 5
open("/lib/i686/cmov/libc.so.6", O_RDONLY) = 5
joey@kodama:~>LD_LIBRARY_PATH=/lib strace busybox 2>&1 |grep open
open("/lib/tls/i686/sse2/cmov/libm.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/lib/tls/i686/sse2/libm.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/lib/tls/i686/cmov/libm.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/lib/tls/i686/libm.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/lib/tls/sse2/cmov/libm.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/lib/tls/sse2/libm.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/lib/tls/cmov/libm.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/lib/tls/libm.so.6", O_RDONLY)    = -1 ENOENT (No such file or directory)
open("/lib/i686/sse2/cmov/libm.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/lib/i686/sse2/libm.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/lib/i686/cmov/libm.so.6", O_RDONLY) = 5
open("/lib/i686/cmov/libc.so.6", O_RDONLY) = 5


IIRC the linker/glibc already has a bug open about its innefficient
attempts to open libraries that arn't there. I don't think we should try
to work around such a bug in the installer.

--
see shy jo

[Joey Hess]

Joey Hess wrote:
> joey@kodama:~>time perl -e 'for (1..16625) { open(IN, "/no/file") }'      
> 0.03user 0.03system 0:00.06elapsed 96%CPU (0avgtext+0avgdata 0maxresident)k
>
> So for a typical install on modern hardware, these 16 thousand extra syscalls
> use less than 0.1 second.
>
> I don't have my slug handy to try, but 1% of install time sounds very excessive
> even there.

joey@slug:~>time perl -e 'for (1..16625) { open(IN, "/no/file") }'
0.97user 0.28system 0:01.26elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k

So even on the slug, if these syscalls take the claimed 1% of the install time,
then the entire installation must take less than 3 minutes!

--
see shy jo