6

I have a closed-source 3rd-party shared library I need to link against. Unfortunately, the creators of the 3rd-party library didn't bother to limit which symbols are exported and exported all symbols. The 3rd-party library internally uses an incompatible version of a popular library that I am using in my code but exports conflicting symbols (google's protobuf library). This leads to a runtime error when the protobuffer library version check finds the compile time and run time versions of the library are not compatible. I can solve the problem by reverting to an older version of protobufs 2.3 that matches the version used in the 3rd party library. However, protbuf 2.3 has performance issues that make it unusable slow for my application. I need a way to use protobuf 2.4 in my code and let the 3rd-party library use it's own internal v 2.3.

Is there a way to generate a new version of the 3rd party library that doesn't export the symbols from the protobuf v 2.3 library used internally given only the so file? If I had the source, it would be an easier problem. It seems that tools like objcopy and strip can't actually modify the dynamic symbol table. The only idea I have so far is to create my own shim library that exports only the symbols I need by redirecting calls to the 3rd-party library (opened with dlopen perhaps?).

Is there a better solution?

2
  • 1
    which operating system do you use? Commented Apr 18, 2012 at 19:58
  • Linux (fedora, ubuntu)... My preference for affordable scientific / distributed computing... ;-) Commented Apr 18, 2012 at 22:05

2 Answers 2

5

I found a solution that works... I created a shim library that redirects calls to the third-party library allowing code outside the library to see protbuf v2.4 symbols while code inside the third-party library sees protobuf v2.3 symbols. This workaround was based on the idea posted here: http://www.linuxjournal.com/article/7795

I had to modify the dlopen flags to include RTLD_LAZY | RTLD_LOCAL | RTLD_DEEPBIND. The RTLD_LOCAL flag keeps symbols inside the third-party library from being seen outside the shim library (prevents symbols leaking out). The RTLD_DEEPBIND forces calls from inside the 3rd party library to see only the internal version of the symbols (keeps symbols from leaking in).

To be concrete, here is an example excerpt from my shim library.

#include <stdio.h>
#include <stdint.h>
#include <dlfcn.h>
#include "libhdfs/hdfs.h"

//#define PRINT_DEBUG_STUFF

// Helper function to retrieve a function pointer to a function from libMapRClient
// while isolating the symbols used internally from those already linked externaly
// to workaround symbol collision problem with the current version of libMapRClient.
void* GetFunc(const char* name){
  #ifdef PRINT_DEBUG_STUFF
    printf("redirecting %s\n", name);
  #endif
  void *handle;
  char *error;

  handle = dlopen("/opt/mapr/lib/libMapRClient.so", RTLD_LAZY | RTLD_LOCAL | RTLD_DEEPBIND);

  if (!handle) {
    fputs(dlerror(), stderr);
    exit(1);
  }
  void* fp = dlsym(handle, name);
  if ((error = dlerror()) != 0) {
    fprintf(stderr, "%s\n", error);
    exit(1);
  }
  return fp;
}


hdfsFS hdfsConnect(const char* host, tPort port) {
  typedef hdfsFS (*FP) (const char* host, tPort port);
  static FP ext = 0;
  if (!ext) {
    ext = (FP)GetFunc("hdfsConnect");
  }
  return ext(host, port);
}


int hdfsCloseFile(hdfsFS fs, hdfsFile file) {
  typedef int (*FP) (hdfsFS fs, hdfsFile file);
  static FP ext = 0;
  if (!ext) {
    ext = (FP)GetFunc("hdfsCloseFile");
  }
  return ext(fs, file);
}

... and so on for the other public API functions

Sign up to request clarification or add additional context in comments.

1 Comment

Isn't the shim basically just hiding the dlopen / dlsym nastiness from the main application? It doesn't seem necessary.
0

It might help to change the order of libraries you pass to the linker. If several libraries export the same symbol the linker should use the one from the first library. But, not all linkers are according to that, so have a look into its documentation.

2 Comments

Thanks for this idea... I tried this, it doesn't quite work because it forces all call sites to use the same version of the symbol (either from version 2.3 or 2.4). However, I need calls within the 3rd party library to call v2.3 of the symbol and calls within my code to call v2.4 of the symbol. Otherwise the library detects mismatched versions and aborts. Switching the library link order only changes if the abort happens when my code loads the library or the 3rd party code loads the library.
So you need a kind of partial linking. In the first stage link your code to v2.4 and independently 3rd party lib's code to v2.3 and in the second stage link the results together. Maybe there are linker options which allow such incomplete linkings of the first stage?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.