====== Fixing --as-needed problems ======
===== Quick and ugly workaround (DON'T USE IT!) =====
Put this line in ''spec'' file:
%define filterout_ld -Wl,--as-needed
But **don't use it** unless you really need to quickly build a package. Never ever try to commit such a change. It might be tempting to do so but if you think about it ''--as-needed'' did it's job well and disabled unneeded libraries. If building fails, it's the package that needs to get fixed, not the linker flags.
===== How --as-needed works =====
With ''--as-needed'' enabled, only libraries containing symbols required by __object files__ are linked.
==== Why it fails for some packages ====
In other words, if you list 10 libraries and only 3 of them contain referenced symbols, others are silently discarded. This is a good thing but unfortunately some projects fail to list all their dependencies properly.
This normally works because if project requires libraries //A// and //B// and //A// itself lists //B// as a dependency, both //A// and //B// get pulled in even if the project itself only lists //A//.
Now assume that //A// is a superfluous (unneeded) dependency which gets dropped (or //A// requiring //B// is superfluous and //B// gets dropped from //A//). The symbols are no longer available.
==== About positions ====
Linker options are positional, and position is very important. It looks for missing symbols only in libraries coming after object file.
Correct positions are:
$ gcc $(LDFLAGS) -o $@ $(OBJECTS) $(LIBS)
and this generally works well.
If there is something like that:
$ gcc -Wl,--as-needed -l
the library will __never__ be linked, because it's not required at the time it's checked.
And with something like that:
$ gcc -l -Wl,--as-needed
library will __allways__ be linked, even if it isn't required.
==== Short examples ====
Most of problems are with readline/ncurses and tinfo, and it's nice example: tinfo is required by both readline and ncurses, and both are linked with this library (now, when [[http://cvs.pld-linux.org/cgi-bin/cvsweb/SOURCES/readline-shared.patch?r1=1.6&r2=1.7|fixed readline]] is fixed).
But some packages link with readline or ncurses while they use __only__ symbols from tinfo. //Without// ''--as-needed'' those executables work because they are linked with tinfo from readline/ncurses library. //With// ''--as-needed'' linking will not work, because readline/ncurses contain no symbols required by executable so they are not linked, it's dependencies naturally are neither linked. Thats why there is a need to pass ''-ltinfo''. If it requires only symbols from tinfo it's ok to ''s/ncurses/tinfo/''. But if it really requires readline/ncurses but there is some executable (or ./configure) which requires only tinfo both ''-lreadline''/''-lncurses'' and ''-ltinfo'' should be passed.
For a longer example take a look at real example nr. 1.
Other common problem is when package produces some shared libraries without linking them with all required libraries, and everything is linked at the end to one binary. So we have:
$ gcc -Wl,--as-needed -o executable -l2 -l1
It's usual situation when object files require only library 2, and //-l1// is required by //-l2//. As I said linker only checks for symbols from objects, so //-l1// is not linked. Normally it is easy to fix it, simply make sure while linking //-l2// it is linked to //-l1// (take a look at the second real example).
===== Real examples =====
==== 1. unresolved symbols in executable ====
Very common situation, [[http://cvs.pld-linux.org/cgi-bin/cvsweb/SPECS/xmoto.spec?rev=HEAD|xmoto.spec]]. Actually, it's nothing new. Every linking behaves this way if some library is missing.
With ''-Wl,--as-needed'' enabled it stops on something like this:
g++ -Wl,--as-needed -o xmoto-edit BuiltInFont.o [...object files...] Packager.o -lGL -lcurl -lode -llualib50 -llua50 -lSDL_mixer -lbz2 -lz -lpng -ljpeg
VApp.o: In function `vapp::App::getRealTime()':
src/VApp.cpp:287: undefined reference to `SDL_GetTicks'
VApp.o: In function `vapp::App::getTime()':
src/VApp.cpp:284: undefined reference to `SDL_GetTicks'
[... bunch of missing SDL functions ...]
Editor.o: In function `vapp::EditorApp::viewDrawGrid()':
src/Editor.cpp:777: undefined reference to `SDL_GetMouseState'
Editor.o:src/Editor.cpp:46: more undefined references to `SDL_GetMouseState' follow
EditorMain.o: In function `main':
src/EditorMain.cpp:59: undefined reference to `SDL_Quit'
collect2: ld returned 1 exit status
make[1]: *** [xmoto-edit] Error 1
So let's try to find some of missing symbols:
$ grep SDL_GetMouseState /usr/lib64/libSDL*.so
Binary file /usr/lib64/libSDL.so matches
They are in ''-lSDL'', but binary does not link with ''-lSDL''. Edit Makefile by hand and add ''-lSDL'' at the same place ''-lSDL_mixer'' is: ''LIBS = -lcurl -lode -llualib50 -llua50 -lSDL_mixer -lSDL -lbz2 -lz -lpng -ljpeg''
What we get after running make in build tree:
g++ -Wl,--as-needed -o xmoto-edit BuiltInFont.o [...object files...] Packager.o -lcurl -lode -llualib50 -llua50 -lSDL_mixer -lSDL -lbz2 -lz -lpng -ljpeg
VApp.o: In function `vapp::App::grabScreen()':
src/VApp.cpp:667: undefined reference to `glReadBuffer'
src/VApp.cpp:671: undefined reference to `glReadPixels'
[...more missing gl functions...]
src/Editor.cpp:1280: undefined reference to `glEnable'
src/Editor.cpp:1288: undefined reference to `glDisable'
collect2: ld returned 1 exit status
Same story:
$ grep glEnableClientState /usr/lib64/lib*.so
Binary file /usr/lib64/libGL.so matches
''LIBS = -lcurl -lode -llualib50 -llua50 -lSDL_mixer -lSDL -lGL -lbz2 -lz -lpng -ljpeg''
But [[http://cvs.pld-linux.org/cgi-bin/cvsweb/SPECS/xmoto.spec?rev=1.21|take a look at spec file]], ''-lGL'' thing was fixed there already:
%{__make} \
GL_LIBS="-lGL"
Anyway, lets run make, and what we get ?
g++ -Wl,--as-needed -o xmoto-edit BuiltInFont.o [...object files...] Packager.o -lcurl -lode -llualib50 -llua50 -lSDL_mixer -lSDL -lGL -lbz2 -lz -lpng -ljpeg
make[1]: Leaving directory `/home/users/sparky/rpm/BUILD/xmoto-0.2.0'
**It worked !**
== Why was it working without --as-needed ? ==
Answer is really easy: libSDL is required by SDL_mixer:
$ ldd /usr/lib64/libSDL_mixer-1.2.so.0.2.4 | grep SDL
libSDL-1.2.so.0 => /usr/lib64/libSDL-1.2.so.0 (0x00002ab425307000)
but xmoto-edit contains no SDL_mixer symbols, that's why it wasn't linked with SDL_mixer, and there was nothing providing SDL library.
Can you see now why was it stupid to disable ''--as-needed'' ? It worked just perfectly disabling unneeded library !
Finally, fix for this may be:
%configure \
LIBS="-lSDL -lGL"
or patching configure.in, and the result is:
''Wrote: /home/users/sparky/rpm/RPMS/xmoto-0.2.0-2.x86_64.rpm''
==== 2. unresolved symbols in library while linking executable ====
That's the most common and a little more difficult case, [[http://cvs.pld-linux.org/cgi-bin/cvsweb/SPECS/evolution-data-server.spec?rev=HEAD|evolution-data-server]]:
Compilation stops at this place:
/bin/sh ../libtool --tag=CC --mode=link gcc -Wl,--as-needed -o test-source-selector test-source-selector.o libedataserverui-1.2.la ../libedataserver/libedataserver-1.2.la -pthread -lglade-2.0 [...many -l libraries...]
-lgnome-keyring -lpthread
gcc -Wl,--as-needed -o .libs/test-source-selector test-source-selector.o -pthread ./.libs/libedataserverui-1.2.so [...many, many .so and -l libraries...] -lpthread
./.libs/libedataserverui-1.2.so: undefined reference to `glade_xml_new'
./.libs/libedataserverui-1.2.so: undefined reference to `gnome_keyring_find_items_sync'
./.libs/libedataserverui-1.2.so: undefined reference to `gnome_keyring_get_default_keyring_sync'
./.libs/libedataserverui-1.2.so: undefined reference to `gnome_keyring_attribute_list_free'
./.libs/libedataserverui-1.2.so: undefined reference to `glade_xml_get_widget'
./.libs/libedataserverui-1.2.so: undefined reference to `gnome_keyring_create_sync'
./.libs/libedataserverui-1.2.so: undefined reference to `gnome_keyring_item_delete_sync'
./.libs/libedataserverui-1.2.so: undefined reference to `gnome_keyring_item_create_sync'
collect2: ld returned 1 exit status
make[2]: *** [test-source-selector] Error 1
It's different case, now it isn't problem with binary file, but with shared library. And the difficulty is it shows up at executable-linking time, because undefined symbols in libraries are permited.
OK, lets try to fix that library:
$ grep gnome_keyring_item_delete_sync /usr/lib64/lib*.so
Binary file /usr/lib64/libgnome-keyring.so matches
$ grep glade_xml_new /usr/lib64/lib*.so
Binary file /usr/lib64/libglade-2.0.so matches
Editing Makefile we find:
''GNOME_KEYRING_LIBS = -lgnome-keyring -lglib-2.0''
So add this one and ''-lglade-2.0'' to that library deps:
libedataserverui_1_2_la_LIBADD = \
$(top_builddir)/addressbook/libebook/libebook-1.2.la \
$(GNOME_KEYRING_LIBS) -lglade-2.0 \
$(E_DATA_SERVER_LIBS)
As it uses libtool, remove .la to force regeneration: ''$ rm libedataserverui-1.2.la'' and try to build ''$ make'' passes without problems.
Now we only need to fix it correcly, after looking at configure.in and Makefile.am one can see it was only a typo:
libedataserverui_1_2_la_LIBADD = \
$(top_builddir)/addressbook/libebook/libebook-1.2.la \
$(E_DATA_SERVER_UI_LIBS)
^^^
== Why was it working without --as-needed ? ==
''test-source-selector'' binary was linked with all libraries needed by libedataserverui-1.2.so, and the binary was the one who provided missing symbols to libedataserverui-1.2.so
=== What if broken library comes from other package ? ===
You can check manually does library has all required symbols, simply running ' gcc //library// ', like this:
This is an example of correctly linked library:
$ gcc /usr/lib64/liblftp-tasks.so.0.0.0
/usr/lib64/gcc/x86_64-pld-linux/4.1.2/../../../../lib64/crt1.o: In function `_start':
(.text+0x20): undefined reference to `main'
collect2: ld returned 1 exit status
And incorrectly linked one:
$ gcc /usr/lib64/liblftp-jobs.so.0.0.0
/usr/lib64/gcc/x86_64-pld-linux/4.1.2/../../../../lib64/crt1.o: In function `_start':
(.text+0x20): undefined reference to `main'
/usr/lib64/liblftp-jobs.so.0.0.0: undefined reference to `StringSet::Remove(int)'
/usr/lib64/liblftp-jobs.so.0.0.0: undefined reference to `Buffer::Format(char const*, ...)'
[...]
/usr/lib64/liblftp-jobs.so.0.0.0: undefined reference to `SMTask::SuspendSlave()'
collect2: ld returned 1 exit status
**Warning:** some libraries may require to have unresolved symbols.
==== 3. unresolved symbols caused by incorrect order ====
Two of most difficult of common problems at once, [[http://cvs.pld-linux.org/cgi-bin/cvsweb/SPECS/evolution.spec?rev=HEAD|evolution.spec]]:
The difficulties are:
- it appears in configure, what may be difficult to fix and difficult to find what the real problem is
- the problem is not a missing library, but arguments order
configure stops with such message, which says nothing:
checking if pilot-link handles UTF-8 conversions... no
configure: error: evolution requires pilot-link to have working UTF-8 conversion routines
let's look at config.log:
gcc -o conftest -ggdb -O2 -DORBIT2=1 -pthread -I/usr/include/libgnome-2.0 [...many -I...] -I/usr/include/libxml2 -Wl,--as-needed -pthread -lgpilotd [...many -l...] -lglib-2.0 conftest.c >&5
/home/users/sparky/tmp/ccgrL9ll.o: In function `main':
/home/users/sparky/rpm/BUILD/evolution-2.7.90/conftest.c:64: undefined reference to `convert_ToPilotChar'
collect2: ld returned 1 exit status
function '''convert_ToPilotChar''', may be found in passed library, but take a look at section about positions, arguments order is incorrect: libraries go before objects (conftest.c). It's very common when someone puts ''-l<>'' in ''LDFLAGS'' instead of ''LIBS'', that was the case too.
Just take a look at my fix:
- LDFLAGS_save="$LDFLAGS"
- LDFLAGS="$LDFLAGS $GNOME_PILOT_LIBS"
+ LIBS_save="$LIBS"
+ LIBS="$LIBS $GNOME_PILOT_LIBS"
[...]
- LDFLAGS="$LDFLAGS_save"
+ LIBS="$LIBS_save"
With this simple change everything works perfectly.