CMakeLists: Adding source files from Github with ExternalProject

Question

I'm writing a small catkin wrapper for the Beckhoff ADS library. I would like to install the files from the AdsLib folder but without using the CMake file from Beckhoff.

I just want to copy the files in order to add them to my library in my own CMakeLists. This works fine if I copy the files manually. But I would like to build directly using the latest files from the Github source.

I tried every possible combination I could find here on stackoverflow, but somehow couldn't make it work.

cmake_minimum_required(VERSION 2.8.3)
project(ads_catkin)

find_package(catkin_simple REQUIRED)
catkin_simple()

include(ExternalProject)


ExternalProject_Add(ads
    PREFIX ${CMAKE_BINARY_DIR}/ads
    GIT_REPOSITORY https://github.com/Beckhoff/ADS.git
    GIT_TAG master
    CONFIGURE_COMMAND ""
    #GIT_TAG 6b3a03009a757cf651fe44d8be7b6df698028f0e
    UPDATE_COMMAND ""
    BUILD_COMMAND ""
    INSTALL_COMMAND ""
)

ExternalProject_Get_Property(ads source_dir)

cs_add_library(${PROJECT_NAME}
    ${source_dir}/AdsLib/AdsDef.cpp
)
cs_install()
cs_export()

The first time I'm building with catkin_tools, this gives me:

Errors     << ads_catkin:cmake /home/xxx/xxx_ws/logs/ads_catkin/build.cmake.000.log                                 
CMake Error at /home/xxx/xxx_ws/devel/share/catkin_simple/cmake/catkin_simple-extras.cmake:150 (add_library):
  Cannot find source file:

    /home/xxx/xxx/build/ads_catkin/ads/src/ads/AdsLib/AdsDef.cpp

  Tried extensions .c .C .c++ .cc .cpp .cxx .m .M .mm .h .hh .h++ .hm .hpp
  .hxx .in .txx
Call Stack (most recent call first):
  CMakeLists.txt:23 (cs_add_library)


CMake Error: CMake can not determine linker language for target: ads_catkin
CMake Error: Cannot determine link language for target "ads_catkin".

Then, the second time I run the build tool over the same code, the files get actually cloned, but I end up with this error:

Errors     << ads_catkin:make /home/xxx/xxx_ws/logs/ads_catkin/build.make.000.log                                   
make[2]: *** No rule to make target 'CMakeFiles/ads_catkin.dir/build'.  Stop.
make[1]: *** [CMakeFiles/ads_catkin.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
Cloning into 'ads'...
Already on 'master'
make: *** [all] Error 2

Btw: cs_add_library is the catkin_simple form for add_library and target_link_libraries.

Update: Getting closer... adapting the solution from @Tsyvarev to my file:

cmake_minimum_required(VERSION 2.8.3)
project(ads_catkin)

find_package(catkin_simple REQUIRED)
catkin_simple()

include(ExternalProject)


ExternalProject_Add(ads
    PREFIX ${CATKIN_DEVEL_PREFIX}/ads
    GIT_REPOSITORY https://github.com/Beckhoff/ADS.git
    GIT_TAG 6b3a03009a757cf651fe44d8be7b6df698028f0e
    #GIT_TAG master
    CONFIGURE_COMMAND ""
    UPDATE_COMMAND ""
    BUILD_COMMAND ""
    INSTALL_COMMAND ""
    BUILD_BYPRODUCTS 
        <SOURCE_DIR>/AdsLib/AdsDef.cpp
        <SOURCE_DIR>/AdsLib/AdsLib.cpp
        <SOURCE_DIR>/AdsLib/AmsConnection.cpp
        <SOURCE_DIR>/AdsLib/AmsPort.cpp
        <SOURCE_DIR>/AdsLib/AmsRouter.cpp
        <SOURCE_DIR>/AdsLib/Frame.cpp
        <SOURCE_DIR>/AdsLib/Log.cpp
        <SOURCE_DIR>/AdsLib/NotificationDispatcher.cpp
        <SOURCE_DIR>/AdsLib/Sockets.cpp
        <SOURCE_DIR>/AdsLib/AdsDef.h
        <SOURCE_DIR>/AdsLib/AdsLib.h
        <SOURCE_DIR>/AdsLib/AdsNotification.h
        <SOURCE_DIR>/AdsLib/AmsConnection.h
        <SOURCE_DIR>/AdsLib/AmsHeader.h
        <SOURCE_DIR>/AdsLib/AmsPort.h
        <SOURCE_DIR>/AdsLib/AmsRouter.h
        <SOURCE_DIR>/AdsLib/Frame.h
        <SOURCE_DIR>/AdsLib/Log.h
        <SOURCE_DIR>/AdsLib/NotificationDispatcher.h
        <SOURCE_DIR>/AdsLib/RingBuffer.h
        <SOURCE_DIR>/AdsLib/Router.h
        <SOURCE_DIR>/AdsLib/Semaphore.h
        <SOURCE_DIR>/AdsLib/Sockets.h
        <SOURCE_DIR>/AdsLib/wrap_endian.h
        <SOURCE_DIR>/AdsLib/wrap_socket.h
)

ExternalProject_Get_Property(ads SOURCE_DIR)


include_directories(
    ${SOURCE_DIR}/AdsLib
)

cs_add_library(AdsLib
    ${SOURCE_DIR}/AdsLib/AdsDef.cpp
    ${SOURCE_DIR}/AdsLib/AdsLib.cpp
    ${SOURCE_DIR}/AdsLib/AmsConnection.cpp
    ${SOURCE_DIR}/AdsLib/AmsPort.cpp
    ${SOURCE_DIR}/AdsLib/AmsRouter.cpp
    ${SOURCE_DIR}/AdsLib/Frame.cpp
    ${SOURCE_DIR}/AdsLib/Log.cpp
    ${SOURCE_DIR}/AdsLib/NotificationDispatcher.cpp
    ${SOURCE_DIR}/AdsLib/Sockets.cpp
)
add_dependencies(AdsLib ads)


cs_install()

install(DIRECTORY ${SOURCE_DIR}/AdsLib/
        DESTINATION ${CATKIN_PACKAGE_INCLUDE_DESTINATION}
        PATTERN ".cpp" EXCLUDE   
)

cs_export()

I tried to include the header files in my AdsLib library. But I still get an error:

In file included from /home/xxx/xxx_ws/src/xxx/nav_controller/src/controller.cpp:19:0:
/home/xxx/xxx_ws/src/xxx/nav_controller/include/nav_controller/controller.h:27:10: fatal error: AdsLib.h: No such file or directory
 #include "AdsLib.h"
          ^~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [CMakeFiles/nav_controller.dir/src/controller.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
In file included from /home/xxx/xxx_ws/src/xxx/nav_controller/src/test.cpp:22:0:
/home/xxx/xxx_ws/src/xxx/nav_controller/include/nav_controller/controller.h:27:10: fatal error: AdsLib.h: No such file or directory
 #include "AdsLib.h"
          ^~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [CMakeFiles/nav_controller.dir/src/test.cpp.o] Error 1
make[1]: *** [CMakeFiles/nav_controller.dir/all] Error 2
make: *** [all] Error 2

It is not clear what may be causing the issue. Please provide the full CMake log, including the error message, in your question post. It would be helpful if you included a Minimal Reproducible Example in your post, as the CMake you posted does not look complete... — Kevin
– Kevin, Commented Jan 17, 2020 at 16:02
Related if not a duplicate: https://stackoverflow.com/questions/21223163/how-to-tell-cmake-to-download-some-necessary-header-files-more-precisely-glm-ma — drescherjm
– drescherjm, Commented Jan 17, 2020 at 16:04
@squareskittles I extended the example which is now the complete CMakeLists.txt. I use the catkin tools to build. — pixelpress
– pixelpress, Commented Jan 17, 2020 at 16:16
@drescherjm I saw this one. The second example seems to correspond to what I am doing... — pixelpress
– pixelpress, Commented Jan 17, 2020 at 16:17
Thanks, the problem is more clear now. I think the answers in this post would help solve your issue. Essentially, ExternalProject_Add runs a build time, so when CMake is running, it does not yet have access to artifacts of the External project. — Kevin
– Kevin, Commented Jan 17, 2020 at 16:57

Tsyvarev · Accepted Answer · 2025-03-29 17:40:52Z

5

The CMake error Cannot find source file means a simple thing: Nothing tells CMake that given source file is generated and the file itself is absent.

Because the file is generated in ExternalProject_Add, you need to adjust corresponded target-level dependency:

add_dependencies(ads_catkin ads)

This command should be issued after both add_library() and ExternalProject_Add calls which create corresponded targets. This command tells CMake that the library should be built only after all steps for the external project has been performed.

You still need to tell CMake that the source file is generated. There are two ways for doing this.

Set GENERATED property:

 set_source_files_properties(${source_dir}/AdsLib/AdsDef.cpp PROPERTIES GENERATED TRUE)

This prevents CMake to search the file on the configuration stage.

List file in the BYPRODUCTS option for the target which generates it. For the target created by ExternalProject_Add command this is achieved by additional option to that command
```
 BUILD_BYPRODUCTS <SOURCE_DIR>/AdsLib/AdsDef.cpp
```

This also sets GENERATED property for the source file, as in the first case. But BYPRODUCTS also makes your project usable for Ninja users.

(In the option above expression <SOURCE_DIR> is a special way to refer to ExternalProject's source directory. It is usable for some ExternalProject's options, and BUILD_BYPRODUCTS is one of them).

Technically, CMake should be smart enough for adding target-level dependencies (the effect of add_dependencies(ads_catkin ads)) automatically when it sees corresponded BYPRODUCTS option. But this feature is described only for 3.16 version, and I don't know whether it works in older versions.

edited Mar 29 at 17:40

answered Jan 17, 2020 at 21:13

Tsyvarev

67.7k18 gold badges142 silver badges189 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

pixelpress Over a year ago

Thanks! I tried this, but now I'm still missing the header files in my catkin library. I updated the post with my new file.

pixelpress Over a year ago

It seems like installation is not the problem. It's more that the include files are not properly linked. I tried to add catkin_package(INCLUDE_DIRS ${SOURCE_DIR}/AdsLib) since cs_export is only exporting the include folder. But even if I add this to the end of the Cmake file, I get an error that the files from the external project are not available.

d-karl Over a year ago

This answer is entirely correct. I'd like to point out that there is also the lesser known FetchContent CMake Module, which retrieves external projects or other content at configuration time, instead of build time. If you declare and provide the Content before using the files from the project, this should work right out of the box without managing dependencies.

Tsyvarev Over a year ago

@d-karl: Using FetchContent module has been discussed in the comments to the question. The asker comments that this module is not accessible for older (<3.10) versions of CMake, and this is true.

sdbbs Mar 29 at 15:28

Isn't it add_dependencies instead of add_dependency?

|

pixelpress · Accepted Answer · 2020-01-22 18:12:46Z

1

In the end, even with the good help from @Tsyvarev, I couldn't make it work using ExternalProject_Add. Part of the solution is to manually export the library with cs_export(INCLUDE_DIRS ${ads}), which didn't work with ExternalProject_Add, because the files were not available at this time.

In the end, I included the DownloadProject module, which is a predecessor of FetchContent, in my package. This does what it should.

answered Jan 22, 2020 at 18:12

pixelpress

1291 silver badge14 bronze badges

Collectives™ on Stack Overflow

CMakeLists: Adding source files from Github with ExternalProject

2 Answers 2

6 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related