1

I'm currently developing a C++ application which will reference multiple *.so libraries, each containing code for different machines - written in C. I also have one shared object containing code from a custom namespace Utilities, which (as the name implies) contains basic utilities useful for the application (written in C++, like the rest of the application).

Currently, utilities.so is the only (custom) library referenced by the application. The application compiles and links just fine, however executing it on the target hardware display the following error: bin/updater_v4test: error while loading shared libraries: ../../../bin/device_modules/utilities.so.1.0.0: cannot open shared object file: No such file or directory

When calling LDD on said application, the following output is shown: ../../../bin/device_modules/utilities.so.1.0.0 => not found

I've written a script which (upon make exiting without errors) collects all SOs and pushes them in to the /lib/ directory on the target hardware; meaning that upon each successful build of the application (whether in part or entirely), the updated files are pushed to the correct directory on the target hardware.

In an attempt to mimic other libraries I've used (e.g. zlib), I've attempted to create a symlink to the library file, with no luck (utilities.so.1.0.0 is the symlink, utilities.so is the actual SO). utilities.so.1.0.0 --> utilities.so

At the moment I'm at a loss, and I can't waste much more time in trying to figure this out myself.

Below is an excerpt from the utilities Makefile:

include ../../common/user.mk

LIB_DIR=../../../libs/lib/powerpc-linux-gnu
CFLAGS=-Wall -ggdb -I../../libs/include -I../../common -lpthread -lddc -std=c++0x -I../../libs/include/zlib
LDFLAGS=../../libs/lib/powerpc-linux-gnu/libcurl.so.4 ../../libs/include/minizip/.libs/libminizip.so.1.0.0 \
-L ../../libs/lib \
-L ../../libs/lib/powerpc-linux-gnu \
-Wl,-rpath-link,../../libs/lib/powerpc-linux-gnu \
-lrt -lddc -lpthread -shared
BIN_DIR=../bin
CANONICAL_BIN_DIR := $(shell readlink -f $(BIN_DIR))
CANONICAL_CUR_DIR := $(shell readlink -f "./")
COMP_OBJECTS := $(wildcard *.cpp)
OBJECTS := $(patsubst %.cpp,%.o,$(wildcard *.cpp))

all: objects utilities.so
    -cp --parents utilities.so $(GLOBAL_BIN_LIB)
    -ln -s utilities.so utilities.so.1.0.0
    -mv utilities.so.1.0.0 $(GLOBAL_BIN_LIB)
#   -cp --parents *.h $(GLOBAL_HEADER_DIR)
    -rm -f utilities.so
    -rm -f *.o
    for header in $(wildcard *.h); do \
        echo $$header; \
        ln -s $(CANONICAL_CUR_DIR)/$$header $(GLOBAL_HEADER_DIR)/$$header; \
    done;
    @printf "########## BUILT $^ ##########\n\n"

utilities.so: $(OBJECTS)
    ${CXX} $^ -o $@ ${LDFLAGS}

objects: $(COMP_OBJECTS)
    ${CXX} -c $^ ${CFLAGS}

And now an excerpt from the application Makefile. Here I've added a reference to the library in the linker flags, the header files are all included in from one directory. 

include ../../../common/user.mk

LIB_DIR=../../../libs/lib/powerpc-linux-gnu
CFLAGS=-Wall -ggdb -I../../../libs/include -I${COMMON_DIR} -lpthread -lddc -std=c++0x -I../../../libs/include/zlib -I${HEADER_DIR}
LDFLAGS=../../../libs/lib/powerpc-linux-gnu/libcurl.so.4 ../../../libs/include/minizip/.libs/libminizip.so.1.0.0 ../../../bin/device_modules/utilities.so.1.0.0 \
    -L ../../../libs/lib \
    -L ../../../libs/lib/powerpc-linux-gnu \
    -Wl,-rpath-link,../../../libs/lib/powerpc-linux-gnu \
    -lrt -lddc -lpthread -L ${SO_DIR}
BIN_DIR=../bin
CANONICAL_BIN_DIR := $(shell readlink -f $(BIN_DIR))
CANONICAL_CUR_DIR := $(shell readlink -f "./")

all: test_update.bin
    -cp --parents test_update.bin $(BIN_DIR)
    -ln -s $(CANONICAL_BIN_DIR)/test_update.bin $(GLOBAL_BIN_APP)/test_update.bin
    -rm -f *.bin
    @printf "########## BUILT $^ ##########\n\n"

test_update.bin: main.o updaterdelegate.o commonfunctions.o tinyxml.o
    ${CXX} $^ -o $@ ${LDFLAGS}

####################
#  Required Files  #
####################

main.o: Main.cpp
    ${CXX} -c $^ -o $@ ${CFLAGS}

updaterdelegate.o: UpdaterDelegate.cpp
    ${CXX} -c $^ -o $@ ${CFLAGS}

commonfunctions.o: $(shell python -c "import os.path; print os.path.relpath('${IMPL_CMN_FUNC}'.replace('\"', ''), '${CANONICAL_CUR_DIR}'.replace('\"', ''))")
    ${CXX} -c $^ -o $@ ${CFLAGS}

tinyxml.o: $(shell python -c "import os.path; print os.path.relpath('${IMPL_TXML}'.replace('\"', ''), '${CANONICAL_CUR_DIR}'.replace('\"', ''))")
    ${CXX} -c $^ -o $@ ${CFLAGS}

####################
# /Required Files  #
####################

Just for gits and shiggles, I'll add the build output of the utilities.so and application.

utilities.so:

(Clean ommitted)

=============== CLEAN COMPLETE... BUILDING... ===============


make: Entering directory `~/_workspace/upv4/common/utils'
powerpc-linux-gnu-g++  -c ArgumentHandling.cpp Extensions.cpp Logging.cpp -Wall -ggdb -I../../libs/include -I../../common -lpthread -lddc -std=c++0x -I../../libs/include/zlib
ArgumentHandling.cpp: In member function ‘void Utilities::ArgumentHandler::freeMemory()’:
ArgumentHandling.cpp:136: warning: deleting ‘void*’ is undefined
powerpc-linux-gnu-g++  ArgumentHandling.o Extensions.o Logging.o -o utilities.so ../../libs/lib/powerpc-linux-gnu/libcurl.so.4 ../../libs/include/minizip/.libs/libminizip.so.1.0.0 -L ../../libs/lib -L ../../libs/lib/powerpc-linux-gnu -Wl,-rpath-link,../../libs/lib/powerpc-linux-gnu -lrt -lddc -lpthread -shared
cp --parents utilities.so """~/_workspace/upv4""/bin/device_modules"
ln -s utilities.so utilities.so.1.0.0
mv utilities.so.1.0.0 """~/_workspace/upv4""/bin/device_modules"
rm -f utilities.so
rm -f *.o
for header in ArgumentHandling.h Enumerations.h Extensions.h Logging.h; do \
        echo $header; \
        ln -s ~/_workspace/upv4/common/utils/$header """"~/_workspace/upv4""/bin/device_modules"/headers"/$header; \
    done;
ArgumentHandling.h
Enumerations.h
Extensions.h
Logging.h
########## BUILT objects utilities.so ##########

make: Leaving directory `~/_workspace/upv4/common/utils'



=============== BUILD COMPLETE... PARSING... ===============


========== Warnings ==========

Total: 1

ArgumentHandling.cpp:136: warning: deleting ‘void*’ is undefined


========== Errors ==========

Total: 0

test_update.bin:

(Clean ommitted)

=============== CLEAN COMPLETE... BUILDING... ===============


make: Entering directory `~/_workspace/_workspace/upv4/test/app/src'
powerpc-linux-gnu-g++  -c Main.cpp -o main.o -Wall -ggdb -I../../../libs/include -I"""~/_workspace/_workspace/upv4""/common" -lpthread -lddc -std=c++0x -I../../../libs/include/zlib -I"""""~/_workspace/_workspace/upv4""/bin/device_modules"/headers""
powerpc-linux-gnu-g++  -c UpdaterDelegate.cpp -o updaterdelegate.o -Wall -ggdb -I../../../libs/include -I"""~/_workspace/_workspace/upv4""/common" -lpthread -lddc -std=c++0x -I../../../libs/include/zlib -I"""""~/_workspace/_workspace/upv4""/bin/device_modules"/headers""
powerpc-linux-gnu-g++  -c ../../../common/commonFunctions.cpp -o commonfunctions.o -Wall -ggdb -I../../../libs/include -I"""~/_workspace/_workspace/upv4""/common" -lpthread -lddc -std=c++0x -I../../../libs/include/zlib -I"""""~/_workspace/_workspace/upv4""/bin/device_modules"/headers""
powerpc-linux-gnu-g++  -c ../../../common/xmlreader/tinyxml2.cpp -o tinyxml.o -Wall -ggdb -I../../../libs/include -I"""~/_workspace/_workspace/upv4""/common" -lpthread -lddc -std=c++0x -I../../../libs/include/zlib -I"""""~/_workspace/_workspace/upv4""/bin/device_modules"/headers""
powerpc-linux-gnu-g++  main.o updaterdelegate.o commonfunctions.o tinyxml.o -o test_update.bin ../../../libs/lib/powerpc-linux-gnu/libcurl.so.4 ../../../libs/include/minizip/.libs/libminizip.so.1.0.0 ../../../bin/device_modules/utilities.so.1.0.0 -L ../../../libs/lib -L ../../../libs/lib/powerpc-linux-gnu -Wl,-rpath-link,../../../libs/lib/powerpc-linux-gnu -lrt -lddc -lpthread -L """"~/_workspace/_workspace/upv4""/bin/device_modules""
cp --parents test_update.bin ../bin
ln -s ~/_workspace/_workspace/upv4/test/app/bin/test_update.bin """~/_workspace/_workspace/upv4""/bin"/test_update.bin
ln: failed to create symbolic link `~/_workspace/_workspace/upv4/bin/test_update.bin': File exists
make: [all] Error 1 (ignored)
rm -f *.bin
########## BUILT test_update.bin ##########

make: Leaving directory `~/_workspace/_workspace/upv4/test/app/src'



=============== BUILD COMPLETE... PARSING... ===============


========== Warnings ==========

Total: 0




========== Errors ==========

Total: 0

The build script is custom, I built it myself to rid myself of the entire make output when I don't need it, and it parses out all the errors and warnings. Due to the issue, I have turned make output back on.

Did I miss something during compilation/linking of the library, or is something going wrong during the linking of the application? I'm inclined to say it's during compilation/linking of the library - although I don't know what exactly is going wrong, as everything is compiling and linking just fine. Why is the application looking for the .so file in a path it can't possibly have?

I also made sure that the .so files are found with the $PATH variable, so the application should be able to find them.

Improve this question
4
  • 1
    Use -Wl,-rpath= and learn about $ORIGIN in man ld. Never use LD_LIBRARY_PATH. Commented Feb 5, 2019 at 10:40
  • @Max Didn't end up using LD_LIBRARY_PATH, instead I added a simple script to install the .so files to /usr/lib on the development machine Commented Feb 5, 2019 at 10:54
  • Writing your own files into /usr/lib is not a good idea because this is a place for system libraries controlled by the package manager. Commented Feb 5, 2019 at 10:58
  • This is a very old (and unsupported, proprietary) development machine. I'm not too concerned about that, frankly. As long as it works and is reproducable. Commented Feb 5, 2019 at 11:00

1 Answer 1

Reset to default
1

The system normally searches for shared objects in a fixed set of directories. You can handle around this by defining the environment variable LD_LIBRARY_PATH and adding the directory where you install the shared libraries into.

Or you can add those libraries to some of the standard libraries directories and execute ldconfig -a to update the cache of shared libraries.

See ldconfig(8), for more info.

edit

There are two mechanisms to load a shared object.

  • The first is the normal library loading mechanism, that specifies what has to be loaded at start executable time, and implies linking your executable with the shared objects. This is what you do in your Makefile. You specify the shared executables, and the ld.so.xxx shared object (which is linked to your application when you are dynamically linking it) loads and follows all the unresolved identifiers to find a place for them in the virtual address space. The ld.so.xxx object uses the /etc/ld.so.cache file, that is just a hash table with the directories and shared executables available to be loaded this way. That file is indexed by what is called the soname (which is something useful to allow different versions of the same library to coexist in the same system) and normally maps to the last versioned shared found in the list of directories listed in the file /etc/ld.so.conf (this is static information to accelerate the process of loading libraries and is generated at each boot of the system). If the shared was found at link time (this mechanism is not used by ld, but only to load the libraries at program start time) to have a soname, then that soname is searched in the /etc/ld.so.cache to find the final file that has to be loaded. This cache is built on each boot of the system, so you don't have to cope with that, but only if you don't want to reboot and you install a new library for system use. It is important to note that shared objects must be given sonames for this to work, and that the list of directories in the file /etc/ld.so.conf are the only directories used to search for files. This means that for a standard library to be usable, you need to put it in one of this directories (or add the directory to the list in /etc/ld.so.conf) and then execute ldconfig -a to rebuild the cache and let it include a reference for the file under that soname.

    The other way it to add to the list of search places a list in the form of a PATH variable. The variable is LD_LIBRARY_PATH so, in case you have your shared objects in ${HOME}/libs, you can add this line to your .profile:

    export LD_LIBRARY_PATH=${HOME}/libs

    or

    setenv LD_LIBRARY_PATH ${HOME}/libs

    this allows your library to live outside of the standard directories, but think twice, as this is a far less efficient way to load a file (as it involves processing a list of directories to look for the final shared objects, while the previous approach is direct, you ask for the file matching the soname that ld.so.xxx asks for, only one search, only one step)

  • The second is to use a library function dlopen(3) that allow you to load a shared object and do some housekeeping before calling anything inside. The dl library allows you to search for a symbol in the shared executable and then decide if you interpret it as data or a jump target. dlopen() just opens and loads into the virtual address space a shared object. It resolves the dependencies (if requested to) and is the more flexible way (but it is also nontransparent) to load unknown code for execution. This is the way plugins normally work. You decide in a config file, or dynamically what to load, and then you load it. The program doesn't have to know previously the symbol table of what you are dealing with and you are free to implement whatever you want in the module you are loading.

All of these methods work with ELF binaries, so you have a lot of freedom, but also a lot of complexity on it.

more...

As i've seen from your compilation:

  • you are including -l in the compile only commands, the libraries are only needed if you are linking an executable, don't put libraries in the compile phase.

  • for a library to be searched and selected by the linker, it has to be named as lib<name>.so (without the version info at the end) so this means that you normally find three names for a standard library (let me use the math library -lm as an example):

    /usr/lib/libm.so.3.2.8 # this is the ELF file with the library contents. /usr/lib/libm.so.3 -> libm.so.3.2.8 # this is the soname used to create the library. /usr/lib/libm.so -> libm.so.3 # this is the actual file the ld(1) program searches for when using -lm.

those links have to be created, as the system normally doesn't. This is part of the installation procedure for a shared library. The soname link allows you to have different versions of a library and detect which one of them will be used at runtime (all must be compatible so you can interchange, when you make an incompatible modification, then you have to change the soname, so the system doesn't get confused on loading)

It is very important to know that the ld(1) program only selects a library if it is called lib<name>.so, with no version info. Indeed, the compiler first searchs for lib<name>.so, then for lib<name>.a, then it complains.

It is very important to put -L places to search for libraries before any -l option that will use those directories in the linking parameters.

You only need to run ldconfig -a and install the library in a system directory if you are not going to use the LD_LIBRARY_PATH mechanism. (this mechanism doesn't work for root account, for obvious reasons :))

Expecting this added comments give some light to the process.

Improve this answer
Sign up to request clarification or add additional context in comments.

10 Comments

this must be done for the library to be loaded automatically, so It must be done everywhere you want to link the application dynamically and run it. Just try the first approach (LD_LIBRARY_PATH, including the directory where the shared object resides) and execute the application. You'll see how the application loads and runs without any issue.
Be careful, that I'm not sure if ldd(1) uses the environment variable LD_LIBRARY_PATH to check for libraries. I think it only reads the cache of libraries ld.so.cache and doesn't follow path variables.
@SimonC, see my edit to the answer for a more thorough explanation.
@SimonC, please, check the order of the -L options you use for linking and the actual files, as the directories to search in appear after the actual files, they will not be searched for using the -L options specified. The order is important, put all the -L options before any library you want to search for.
also, the -rpath-link is embedding the path into the required executable, making it difficult to load the libraries if you move the application elsewhere, don't use it.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.