Fun with ExtUtils::MakeMaker and Windows import libraries

[spicy jack]

I've been working on getting the Gtk2-Perl stack to compile under
MSWin32 without as much pain as there is now.  Windows has something
called 'import libraries' [1], which currently I have to create by
hand when I compile Cairo and Glib in order to get Gtk2 to compile.  I
create these import libraries by hacking up the Makefile and adding
something similar to this:

dlltool --input-def Cairo.def --dllname Cairo.dll --output-lib libCairoPerl.a
$(CP) libCairoPerl.a C:/camelbox/lib

I have been able to abstract the above and add it to the Makefile.PL
file.  My questions right now are:

1) I created a dynamic_lib method in the Makefile.PL for
Cairo/Glib/Gtk2, and added Makefile code similar to the above.  It
didn't work, as MakeHelper overrides my dynamic_lib method later on in
order to quiet things somewhere.  I disabled dynamic_lib in MakeHelper
and my overridden dynamic_lib method gets fired.  Is there a better
place for my extra code than the dynamic_lib method?  Oh,  I just
found the extra copy of MakeHelper.pm in the Glib source, I'm now
doing my sad panda face :(

2) What should I call my import libraries?  I've been using
libGlibPerl, libCairoPerl and libGtk2Perl.  They would only needed  if
( $^O ~= /MSWin32/), they shouldn't be needed on cygwin or anywhere
else for that matter.

3) I was going to add a test in Makefile.PL to decide if the above
import libraries get added to the Makefile via
ExtUtils::Depends->set_libs, it would end up looking something like
this (taken from a patch I have made to the Gtk2 Makefile.PL):

use Config;
if ( $^O =~ /Win32/ ) {
        $gtk2->set_libs (q(-L) . $Config{bin} . q( ) . $pkgcfg_gtk{libs}
                . q(-lGlibPerl -lCairoPerl));
} else {
        $gtk2->set_libs ($pkgcfg_gtk{libs});
}

Can someone think of a better way to do this?  Is this not worth
pursuing at all because?  Too much hassle?

My demo patches for the Makefile.PL included with Cairo/Glib/Gtk2 are
available at [2], [3] and [4].  They all work but only as long as
dynamic_lib has been disabled in $PERL5LIB/Glib/MakeHelper.pm (or
MakeHelper.pm in Glib source).

Thanks,

Brian

[1] http://docs.python.org/ext/dynamic-linking.html; yeah, I know,
python website
[2] http://code.google.com/p/camelbox/source/browse/trunk/build-extras/2008.1-odin/Cairo-Makefile.PL.diff
[3] http://code.google.com/p/camelbox/source/browse/trunk/build-extras/2008.1-odin/Glib-Makefile.PL.diff
[4] http://code.google.com/p/camelbox/source/browse/trunk/build-extras/2008.1-odin/Gtk2-Makefile.PL.diff
_______________________________________________
gtk-perl-list mailing list
gtk-perl-list@...
http://mail.gnome.org/mailman/listinfo/gtk-perl-list

[Paul Miller-13]

On Tue, Jul 22, 2008 at 4:00 AM, spicy jack <elspicyjack@...> wrote:
> Can someone think of a better way to do this?  Is this not worth
> pursuing at all because?  Too much hassle?

Oh god, this needs doing so bad I'd chip in if you had a paypal link.

I don't know the correct method, and I don't know what it'll take to
fix it, but I have documented several irritating build challenges,
mostly relating to the ridiculously short command length in win32:

https://voltar.org/grm/#gtk2-perl

I suspect you're already familiar with them if you're thinking of
tackling the problem.

--
If riding in an airplane is flying, then riding in a boat is swimming.
93 jumps, 38.4 minutes of freefall, 75.1 freefall miles.
_______________________________________________
gtk-perl-list mailing list
gtk-perl-list@...
http://mail.gnome.org/mailman/listinfo/gtk-perl-list

[Torsten Schoenfeld]

spicy jack wrote:
> I've been working on getting the Gtk2-Perl stack to compile under
> MSWin32 without as much pain as there is now.  Windows has something
> called 'import libraries' [1], which currently I have to create by
> hand when I compile Cairo and Glib in order to get Gtk2 to compile.  I
> create these import libraries by hacking up the Makefile and adding
> something similar to this:

I don't have this problem on Windows XP with gtk+ 2.12.9, Visual Studio 6.0,
perl 5.8.8 from ActiveState, and CVS checkouts of our modules.  Glib, Gtk2,
and Gtk2::GladeXML compile and pass all their tests.

I'm not sure if it's still necessary, but at some point in the past,
ExtUtils::MakeMaker had to be patched to not discard important linker
arguments.  See <http://rt.cpan.org/Ticket/Display.html?id=21430>.

If you're using perl 5.10, you'll need to patch ExtUtils::Depends as in the
attached patch.  I'll commit it shortly.

Also, Storable still seems to be broken on win32, so you might have to set the
environment variable FORCE_DATA_DUMPER.  muppet, maybe we should default to
using Data::Dumper on win32?

> 1) I created a dynamic_lib method in the Makefile.PL for
> Cairo/Glib/Gtk2, and added Makefile code similar to the above.  It
> didn't work, as MakeHelper overrides my dynamic_lib method later on in
> order to quiet things somewhere.  I disabled dynamic_lib in MakeHelper
> and my overridden dynamic_lib method gets fired.  Is there a better
> place for my extra code than the dynamic_lib method?  Oh,  I just
> found the extra copy of MakeHelper.pm in the Glib source, I'm now
> doing my sad panda face :(

MakeHelper.pm is the place for this kind of fiddling, I think.  Cairo and Glib
have separate MakeHelper.pm's because Cairo can't depend on Glib.  And both
aren't that similar anyway.

> 3) I was going to add a test in Makefile.PL to decide if the above
> import libraries get added to the Makefile via
> ExtUtils::Depends->set_libs, it would end up looking something like
> this (taken from a patch I have made to the Gtk2 Makefile.PL):
>
> use Config;
> if ( $^O =~ /Win32/ ) {
> $gtk2->set_libs (q(-L) . $Config{bin} . q( ) . $pkgcfg_gtk{libs}
> . q(-lGlibPerl -lCairoPerl));
> } else {
> $gtk2->set_libs ($pkgcfg_gtk{libs});
> }
>
> Can someone think of a better way to do this?  Is this not worth
> pursuing at all because?  Too much hassle?

This should be done by ExtUtils::Depends.  Specifically, find_extra_libs is
meant for this.

--
Bye,
-Torsten

Index: Depends.pm
===================================================================
RCS file: /cvsroot/gtk2-perl/gtk2-perl-xs/ExtUtils-Depends/lib/ExtUtils/Depends.pm,v
retrieving revision 1.18
diff -u -r1.18 Depends.pm
--- Depends.pm 30 Mar 2008 15:36:23 -0000 1.18
+++ Depends.pm 2 Aug 2008 12:45:00 -0000
@@ -302,7 +302,13 @@
 
  my %mappers = (
  MSWin32 => sub { $_[0] . '.lib' },
- cygwin  => sub { 'lib' . $_[0] . '.dll.a'},
+ cygwin  => sub {
+ '(?:' .
+ 'lib' . $_[0] . '.dll.a' .
+ '|' .
+ $_[0] . '.dll' .
+ ')'
+ },
  );
  my $mapper = $mappers{$^O};
  return () unless defined $mapper;


Index: ParseXSDoc.pm
===================================================================
RCS file: /cvsroot/gtk2-perl/gtk2-perl-xs/Glib/ParseXSDoc.pm,v
retrieving revision 1.34
diff -u -r1.34 ParseXSDoc.pm
--- ParseXSDoc.pm 30 Mar 2008 17:22:08 -0000 1.34
+++ ParseXSDoc.pm 1 Aug 2008 21:02:13 -0000
@@ -87,7 +87,7 @@
  # uses an obscene amount of ram on Gtk2's nearly 200 xs files.  Use
  # Storable unless the user really really wants to force us to fall
  # back to Data::Dumper.
- if ($ENV{FORCE_DATA_DUMPER}) {
+ if ($ENV{FORCE_DATA_DUMPER} || $^O eq 'MSWin32') {
  $Data::Dumper::Purity = 1;
  print Data::Dumper->Dump([$parser->{xspods}, $parser->{data}],
                        [qw($xspods            $data)]);


_______________________________________________
gtk-perl-list mailing list
gtk-perl-list@...
http://mail.gnome.org/mailman/listinfo/gtk-perl-list

[Ari Jolma]

Torsten Schoenfeld kirjoitti:

> spicy jack wrote:
>  
>> I've been working on getting the Gtk2-Perl stack to compile under
>> MSWin32 without as much pain as there is now.  Windows has something
>> called 'import libraries' [1], which currently I have to create by
>> hand when I compile Cairo and Glib in order to get Gtk2 to compile.  I
>> create these import libraries by hacking up the Makefile and adding
>> something similar to this:
>>    
>
> I don't have this problem on Windows XP with gtk+ 2.12.9, Visual Studio 6.0,
> perl 5.8.8 from ActiveState, and CVS checkouts of our modules.  Glib, Gtk2,
> and Gtk2::GladeXML compile and pass all their tests.
>
> I'm not sure if it's still necessary, but at some point in the past,
> ExtUtils::MakeMaker had to be patched to not discard important linker
> arguments.  See <http://rt.cpan.org/Ticket/Display.html?id=21430>.
>
> If you're using perl 5.10, you'll need to patch ExtUtils::Depends as in the
> attached patch.  I'll commit it shortly.
>
>  

The patch modifies the cygwin entry in the find_extra_libs. Spicy and me
as well are using MinGW, which have MSWin32 in $^O, which makes
find_extra_libs use ".lib", which is the extension MS compilers use AFAIK.

The GTK runtime has both ".dll.a" and ".lib" import libraries. Only the
".dll.a" ones have the "lib" prefix, which seems to be required by
Kid.pm (assumably ActiveState's version is modified to not to). I'm used
to patching Glib and Gtk2 Makefile.PL's so that they use the ".dll.a"
import libraries
(http://geoinformatics.tkk.fi/twiki/bin/view/Main/InstallingWithMinGW).
Thus I also create import libraries with .dll.a extensions for Glib.dll
etc (see the wiki page).

Because find_extra_libs does not find my Glib.dll.a I have to add them
by hand to Makefile too (see the wiki page, it says "sometimes" though).

My approach works but all this feels as a mixture between MS compilers
and cygwin, I'm not sure what should be fixed - at least I'd like to
change Glib et al Makefile.PL's so that I didn't need to hand-edit Makefile.

Best regards,

Ari

--
Prof. Ari Jolma
Geoinformatiikka / Geoinformatics
Teknillinen Korkeakoulu / Helsinki University of Technology
tel: [phone] address: POBox 1200, 02015 TKK, Finland
Email: ari.jolma at tkk.fi URL: http://www.tkk.fi/~jolma


_______________________________________________
gtk-perl-list mailing list
gtk-perl-list@...
http://mail.gnome.org/mailman/listinfo/gtk-perl-list

[Torsten Schoenfeld]

Ari Jolma wrote:

> My approach works but all this feels as a mixture between MS compilers
> and cygwin, I'm not sure what should be fixed - at least I'd like to
> change Glib et al Makefile.PL's so that I didn't need to hand-edit
> Makefile.

I don't have any experience with MinGW, so I can't help much.  But yes, the
goal should be that building just works.  So if you can come up with a patch
to EU::Depends or whatever else needs changing, we'd be happy to include it.

--
Bye,
-Torsten
_______________________________________________
gtk-perl-list mailing list
gtk-perl-list@...
http://mail.gnome.org/mailman/listinfo/gtk-perl-list

[Ari Jolma]

Torsten Schoenfeld kirjoitti:

> Ari Jolma wrote:
>
>  
>> My approach works but all this feels as a mixture between MS compilers
>> and cygwin, I'm not sure what should be fixed - at least I'd like to
>> change Glib et al Makefile.PL's so that I didn't need to hand-edit
>> Makefile.
>>    
>
> I don't have any experience with MinGW, so I can't help much.  But yes, the
> goal should be that building just works.  So if you can come up with a patch
> to EU::Depends or whatever else needs changing, we'd be happy to include it

EU::Depends::find_extra_libs assumes now that when OS is MSWin32, the
(import) library extension is ".lib". When MinGW is used, the extension
is usually .dll.a (MinGW could also happily use the actual .dll, but
EU::LibList::Kid does not allow that currently). Thus a (partial)
solution would be to change find_extra_libs to search for files with
.dll.a extension too. I don't have a very elegant solution how to allow
both extensions.

ActiveState seems to support MinGW and automatically detect MinGW (this
is old news
http://www.issociate.de/board/post/274396/ActivePerl_5.8.7.815_released.html).
Somehow this detection should be propagated to EU::LibList::Kid to look
for import libraries "lib<library>.dll.a" -- or the Makefile.PL's may
also alter the liblist by adding ".dll.a" before calling set_libs().

Ari

--
Prof. Ari Jolma
Geoinformatiikka / Geoinformatics
Teknillinen Korkeakoulu / Helsinki University of Technology
tel: [phone] address: POBox 1200, 02015 TKK, Finland
Email: ari.jolma at tkk.fi URL: http://www.tkk.fi/~jolma


_______________________________________________
gtk-perl-list mailing list
gtk-perl-list@...
http://mail.gnome.org/mailman/listinfo/gtk-perl-list

[rahed]

Ari Jolma <ari.jolma@...> writes:

> EU::Depends::find_extra_libs assumes now that when OS is MSWin32, the
> (import) library extension is ".lib". When MinGW is used, the
> extension is usually .dll.a (MinGW could also happily use the actual
> .dll, but EU::LibList::Kid does not allow that currently). Thus a
> (partial) solution would be to change find_extra_libs to search for
> files with .dll.a extension too. I don't have a very elegant solution
> how to allow both extensions.
>
> ActiveState seems to support MinGW and automatically detect MinGW
> (this is old news
> http://www.issociate.de/board/post/274396/ActivePerl_5.8.7.815_released.html).
> Somehow this detection should be propagated to EU::LibList::Kid to
> look for import libraries "lib<library>.dll.a" -- or the Makefile.PL's
> may also alter the liblist by adding ".dll.a" before calling
> set_libs().

I also follow your approach on Windows with perl 5.10 built with MinGW.
To find .dll.a libraries I trimmed EU::Liblist::Kid with the patch
below. Changes are only in _win32_ext procedure. Am not sure if cygwin
builts are then ok.

Still, I have to add one line to the $(INST_DYNAMIC) Makefile section
(an example for Glib, similar for Cairo and Gtk2).

$(NOECHO) $(LD) -o Glib.dll $(LDDLFLAGS) $(LDFROM)
-Wl,-out-implib,blib/arch/auto/Glib/Glib.lib $(EXPORT_LIST)
$(OTHERLDFLAGS) $(MYEXTLIB) $(PERL_ARCHIVE) $(LDLOADLIBS) dll.exp

Probably some changes in EU::MM_Win32 are needed to produce correct
Makefile without the necessity to add an additional line.


--- Kidorig.pm 2008-08-26 19:41:38.669502800 +0000
+++ Kid.pm 2008-08-26 19:54:56.446856800 +0000
@@ -308,12 +308,27 @@
  if (s/^-l// and $GC and !/^lib/i) {
     $_ = "lib$_";
  }
- $_ .= $libext if !/\Q$libext\E$/i;
+ my ($libext2,$lib1,$lib2);
+ $libext2 = '.dll'.$libext;
+ $lib1 = $_.$libext if !/\Q$libext\E$/i; # .a
+ $lib2 = $_.$libext2 if !/\Q$libext2\E$/i; # .dll.a
 
  my $secondpass = 0;
     LOOKAGAIN:
 
         # look for the file itself
+ if (-f $lib1) {
+    warn "'$thislib' found as '$lib1'\n" if $verbose;
+    $found++;
+    push(@extralibs, $lib1);
+    next;
+ }
+ if (-f $lib2) {
+    warn "'$thislib' found as '$lib2'\n" if $verbose;
+    $found++;
+    push(@extralibs, $lib2);
+    next;
+ }
  if (-f) {
     warn "'$thislib' found as '$_'\n" if $verbose;
     $found++;
@@ -323,7 +338,7 @@
 
  my $found_lib = 0;
  foreach my $thispth (@searchpath, @libpath){
-  unless (-f ($fullname="$thispth\\$_")) {
+  unless (-f ($fullname="$thispth\\$lib1") or (-f ($fullname="$thispth\\$lib2"))) {
     warn "'$thislib' not found as '$fullname'\n" if $verbose;
     next;
   }
@@ -341,7 +356,8 @@
  goto LOOKAGAIN if s/^lib//i;
     }
     elsif (!/^lib/i) {
- $_ = "lib$_";
+ $lib1 = "lib$lib1";
+ $lib2 = "lib$lib2";
  goto LOOKAGAIN;
     }
  }


--
Radek

_______________________________________________
gtk-perl-list mailing list
gtk-perl-list@...
http://mail.gnome.org/mailman/listinfo/gtk-perl-list

[Torsten Schoenfeld]

Ari Jolma wrote:

> EU::Depends::find_extra_libs assumes now that when OS is MSWin32, the
> (import) library extension is ".lib". When MinGW is used, the extension is
> usually .dll.a (MinGW could also happily use the actual .dll, but
> EU::LibList::Kid does not allow that currently). Thus a (partial) solution
> would be to change find_extra_libs to search for files with .dll.a extension
> too. I don't have a very elegant solution how to allow both extensions.

Elegance shouldn't be the primary concern.  If you have a patch that makes
linking work on MinGW, just post it.

> Somehow this detection should be propagated to EU::LibList::Kid to look for
> import libraries "lib<library>.dll.a" -- or the Makefile.PL's may also alter
> the liblist by adding ".dll.a" before calling set_libs().

I'd prefer a solution that doesn't involve changing every Makefile.PL.

-Torsten
_______________________________________________
gtk-perl-list mailing list
gtk-perl-list@...
http://mail.gnome.org/mailman/listinfo/gtk-perl-list

[Ari Jolma]

Torsten Schoenfeld kirjoitti:

> Elegance shouldn't be the primary concern.  If you have a patch that makes
> linking work on MinGW, just post it.
>
>  
>> Somehow this detection should be propagated to EU::LibList::Kid to look for
>> import libraries "lib<library>.dll.a" -- or the Makefile.PL's may also alter
>> the liblist by adding ".dll.a" before calling set_libs().
>>    
>
> I'd prefer a solution that doesn't involve changing every Makefile.PL.
>  

ok, almost there

1. Patch lib/ExtUtils/Liblist/Kid.pm with the attached patch (the patch
needs to be offered to guys maintaining ExtUtils::MakeMaker I guess)

2. When building Glib and Cairo, there are the extra steps before make
install:

using spicy jack's method:

dlltool --input-def Glib.def --dllname Glib.dll --output-lib Glib.dll.a
copy Glib.dll.a blib/arch/auto/Glib

and

dlltool --input-def Cairo.def --dllname Cairo.dll --output-lib Cairo.dll.a
copy Cairo.dll.a blib/arch/auto/Cairo

Brian, you wrote that you have been able to abstract the above and add
it to the Makefile.PL. How exactly? However, I think this should/could
be corrected somewhere in lib/ExtUtils/ and not change the Makefile.PL

3. Now the patch for ExtUtils/Depends.pm - that's attached

4. Gtk2 installs without problems - at least in my test just now with
everything freshly built.

These same steps

dlltool --input-def Gtk2.def --dllname Gtk2.dll --output-lib Gtk2.dll.a
copy Gtk2.dll.a blib/arch/auto/Gtk2

need to be run for Gtk2::GladeXML

Ari


--
Prof. Ari Jolma
Environmental Management Information Technology
Teknillinen korkeakoulu / Helsinki University of Technology
tel: [phone] address: POBox 5300, 02015 TKK, Finland
Email: ari.jolma at tkk.fi
http://geoinformatics.tkk.fi/twiki/bin/view/Main/AriJolmaHomePage


308,315c308,311
< # handle possible library arguments
< if (s/^-l// and $GC and !/^lib/i) {
<    $_ = "lib$_";
< }
< $_ .= $libext if !/\Q$libext\E$/i;
<
< my $secondpass = 0;
<     LOOKAGAIN:
---
>     # try $_$libext, lib$_$libext, $_.dll$libext, lib$_.dll$libext
>     s/^-l//;
>     my $found_lib = 0;
>     for my $libname ($_.$libext, 'lib'.$_.$libext, $_.'.dll'.$libext, 'lib'.$_.'.dll'.$libext) {
318,319c314,315
< if (-f) {
<    warn "'$thislib' found as '$_'\n" if $verbose;
---
> if (-f $libname) {
>    warn "'$thislib' found as '$libname'\n" if $verbose;
321,322c317,319
<    push(@extralibs, $_);
<    next;
---
>    $found_lib++;
>    push(@extralibs, $libname);
>    last;
325d321
< my $found_lib = 0;
327c323
<    unless (-f ($fullname="$thispth\\$_")) {
---
>    unless (-f ($fullname="$thispth\\$libname")) {
338,347d333
<
< # do another pass with (or without) leading 'lib' if they used -l
< if (!$found_lib and $thislib =~ /^-l/ and !$secondpass++) {
<    if ($GC) {
< goto LOOKAGAIN if s/^lib//i;
<    }
<    elsif (!/^lib/i) {
< $_ = "lib$_";
< goto LOOKAGAIN;
<    }
350c336
< # give up
---
> # express frustration

304,305c304,305
< MSWin32 => sub { $_[0] . '.lib' },
< cygwin  => sub { $_[0] . '.dll'},
---
> MSWin32 => sub { ($_[0] . '.lib', $_[0] . '.dll.a') },
> cygwin  => sub { ($_[0] . '.dll') },
313c313
< my $lib = $mapper->($stem);
---
> for my $lib ($mapper->($stem)) {
319c319,320
< if ((not $matching_file) && /$pattern/) {;
---
> if ((not $matching_file) && /$pattern/)
> {;
327a329
>    }

_______________________________________________
gtk-perl-list mailing list
gtk-perl-list@...
http://mail.gnome.org/mailman/listinfo/gtk-perl-list

[spicy jack]

Apologies, I just realized I hit 'reply' instead of 'reply to all' for
the below.  A month late :/

2008/9/10 Ari Jolma <ari.jolma@...>:
> Brian, you wrote that you have been able to abstract the above and add it to
> the Makefile.PL. How exactly? However, I think this should/could be
> corrected somewhere in lib/ExtUtils/ and not change the Makefile.PL

My patches to various Makefile.PL's are in my SVN tree [1].  My
patches are to have the import libraries built automatically.  They
don't fix the problem with the Kid.pm module where it can't find the
correct libraries.  My additions to build the import libraries are in
the wrong place in the Makefile.PL though; look for Torsten's comments
earlier in this thread about the proper place to add this extra stuff
for the import libraries to the Makefile.  I haven't spent the time
trying to make it work the way he suggested.

I have also looked at the differences between building using VC++
(which is what Torsten uses and he doesn't need to build import
libraries) and MinGW (how I do it and I do need to build import
libraries), and I've got a few ideas on that as well.  There's a page
on Red Hat's website [2] that has the best explanation I've found so
far about what's going on.  Again, I haven't tried to apply it
anywhere yet though.

Thanks,

Brian

[1] http://code.google.com/p/camelbox/source/browse/trunk/build-extras/2008.1-odin/
[2] http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/gnu-linker/win32.html
_______________________________________________
gtk-perl-list mailing list
gtk-perl-list@...
http://mail.gnome.org/mailman/listinfo/gtk-perl-list