Calling dlsym() with a NULL handle doesn't return NULL, but rather returns a random function

Question

My title may not be clear, so allow me to explain. I have a piece of code that goes like this:

void* pluginFile = dlopen(fileName, RTLD_LAZY);
auto function = dlsym(pluginFile, "ExpectedFunction");

This works fine if dlopen returns the right file. My problem is when dlopen doesn't find a file and returns NULL. What currently happens is that this call is made:

dlsym(0x0, "ExpectedFunction");

The problem is that this returns a random function in my project called ExpectedFunction. What I thought would happen is that dlsym would return NULL since the passed handle is NULL. I'm not able to find the expected behavior for such a use case online.

My question is, what is supposed to happen when you pass a NULL handle to dlsym? Will it simply return NULL or will it interpret it as a handle at location 0x0? If the inteded behavior is the latter, then I'll simply add a check to make sure dlopen suceeded. If not, I'd like to know why it randomly returns a function with the same name from an other library if the handle is NULL.

My current use case is that I am loading 10 shared libraries that I made that all have a function ExpectedFunction(). However, if we call dlopen with a filename of a shared library that does not exist, it will return NULL. Then, dlsym will return a pointer to ExpectedFunction() of the last library that was loaded.

Looks like your implementation uses #define RTLD_DEFAULT ((void*)0), so your null pointer is indeed interpreted as "whichever you find first". — Quentin
– Quentin, Commented Jul 11, 2019 at 16:29
A good habit to get into: Whenever you call some API function (like dlopen) that has a return value that clearly indicates if it has failed, always check it and act appropriately as soon as you can, rather than relying the value being safe to use elsewhere. It pays to get religion about it. Not only does it save you from "gotcha" situations like this (which might not get discovered in testing) but it makes it clearer to those who come after you what ought to happen. — Joe
– Joe, Commented Jul 11, 2019 at 16:40

eerorika · Accepted Answer · 2021-08-15 17:38:17Z

9

My question is, what is supposed to happen when you pass a NULL handle to dlsym?

The specification says:

If handle does not refer to a valid object opened by dlopen() ... dlsym() shall return NULL.

However, there are some reserved handle values that have special behaviour. If you pass such reserved handle, then the behaviour is different. The exact values are unspecified by POSIX, but for example in glibc:

# define RTLD_NEXT        ((void *) -1l)
# define RTLD_DEFAULT        ((void *) 0)

(void *) 0 is null, and therefore you accidentally passed RTLD_DEFAULT into dlsym. Of this, the spec says:

RTLD_DEFAULT

The symbol lookup happens in the normal global scope; that is, a search for a symbol using this handle would find the same definition as a direct use of this symbol in the program code.

So, in conclusion, what is supposed to happen depends on whether NULL is a reserved value or not. It happens to be reserved in glibc, but is not necessarily so in other implementations.

You should check that dlopen does not return null (or check that dlerror does return null) before passing to dlsym.

edited Aug 15, 2021 at 17:38

answered Jul 11, 2019 at 16:32

eerorika

240k13 gold badges212 silver badges354 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

panzi Over a year ago

Note: RTLD_DEFAULT is defined for some (most) other POSIX C libraries (like the one for FreeBSD or macOS), but it is not equal to NULL there. Therefore better use RTLD_DEFAULT instead of NULL.

r3mus n0x · Accepted Answer · 2019-07-11 16:32:35Z

2

From the dlfcn.h on Ubuntu Linux:

/* If the first argument to `dlsym' or `dlvsym' is set to RTLD_DEFAULT
   the run-time address of the symbol called NAME in the global scope
   is returned.  */
# define RTLD_DEFAULT   ((void *) 0)

and from dlsym man-page:

RTLD_DEFAULT

Find the first occurrence of the desired symbol using the default shared object search order. The search will include global symbols in the executable and its dependencies, as well as symbols in shared objects that were dynamically loaded with the RTLD_GLOBAL flag.

answered Jul 11, 2019 at 16:32

r3mus n0x

6,1841 gold badge17 silver badges37 bronze badges

1 Comment

JigsawCorp Over a year ago

Thank you for answering my question!

Collectives™ on Stack Overflow

Calling dlsym() with a NULL handle doesn't return NULL, but rather returns a random function

2 Answers 2

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related