diff options
| author | Alexandru Croitor <alexandru.croitor@qt.io> | 2021-09-27 18:32:37 +0200 |
|---|---|---|
| committer | Alexandru Croitor <alexandru.croitor@qt.io> | 2022-02-04 15:50:57 +0100 |
| commit | 5d0fd24f0ff9289dba2d41a38a7054770c3a3eee (patch) | |
| tree | 0e1963850c18812868286fa27d2c0ac947405d80 | |
| parent | b69d0262e0262274a8f050f9b627e9ca4a3b1833 (diff) | |
CMake: shiboken: Add support for cross-building
The shiboken project will be cross-compiled either if
CMAKE_CROSSCOMPILING is set to TRUE (which is usually set
by a CMake toolchain file or computed by CMake itself)
or if QFP_SHIBOKEN_HOST_PATH is provided on the command line.
Various code is added to ensure the correct target Qt and Python
libraries are found, as well as host Qt and Shiboken tools.
These are specified to the project by setting one of the following
vars:
- QFP_QT_TARGET_PATH (for device Qt libs and includes)
- QFP_PYTHON_TARGET_PATH (for device python libs and includes)
- QFP_SHIBOKEN_HOST_PATH (for host shiboken generator)
- QT_HOST_PATH (for host moc and friends)
To cross-compile the libraries, the project needs to find the host
Shiboken6Tools package (to use the host binding generator).
Any path specified in QFP_SHIBOKEN_HOST_PATH variable will be used to
search for that package.
When cross-compiling we now use the new FindPython CMake module in
shiboken_find_required_python instead of FindPythonInterp and
FindPythonLibs.
The old modules can't be used because they rely on executing the
python interpreter which can't be done if the host and target
platforms are different.
The new module instead internally relies on executing a python-config
shell script that is distributed with Python installations. This shell
script knows various details about the built Python and provides that
to the CMake module without having to run an executable of a different
architecture.
The CMake variables set by the new module have different names, so in
order not to modify too much build system code initially, we set the
old variable names with data from the new variables. This can
potentially be cleaned up in a follow up change.
The shiboken6 generator tool and its library dependency apiextractor
will not be built when cross-compiling (so there's no need to provide
a path to the LLVM libs).
Only the supporting libraries and python modules are built.
Tests are not built by default because running the target python
interpreter on a host machine will likely not work (different
platform / architecture). Support could potentially be added for
certain platforms so that the tests run via a user-space qemu.
Two new config tests are added to query information about the
target Qt and Python libraries. These will also be used by setup.py.
These will be used to extract necessary info instead of running
qmake / qtpaths / python because their arch is different from the
host one.
Just like with limited-api mode, when cross-compiling,
the signature embedding does not embed pre-compiled pyc files because
the host and python target can be of different versions.
PYTHON_WITH_COUNT_ALLOCS is always false when cross-compiling because
there's no way to query that information from the target python
installation without resorting to parsing certain files in a hacky
way.
Because we require a valid Python_SOABI value to be determined by
FindPython, we now require CMake version 3.17+ when cross-compiling.
We might be able to get rid of this requirement in the future at
the cost of some hacky code that could be difficult to maintain.
Task-number: PYSIDE-802
Task-number: PYSIDE-1033
Change-Id: I29a23b2d6366247090e55dd9176fbda7acdc29e8
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
| -rw-r--r-- | sources/shiboken6/CMakeLists.txt | 14 | ||||
| -rw-r--r-- | sources/shiboken6/cmake/ShibokenHelpers.cmake | 278 | ||||
| -rw-r--r-- | sources/shiboken6/cmake/ShibokenSetup.cmake | 55 | ||||
| -rw-r--r-- | sources/shiboken6/config.tests/target_python_info/CMakeLists.txt | 44 | ||||
| -rw-r--r-- | sources/shiboken6/config.tests/target_qt_info/CMakeLists.txt | 36 | ||||
| -rw-r--r-- | sources/shiboken6/doc/CMakeLists.txt | 12 | ||||
| -rw-r--r-- | sources/shiboken6/libshiboken/CMakeLists.txt | 20 | ||||
| -rw-r--r-- | sources/shiboken6/libshiboken/embed/embedding_generator.py | 12 | ||||
| -rw-r--r-- | sources/shiboken6/libshiboken/signature/signature_globals.cpp | 7 | ||||
| -rw-r--r-- | sources/shiboken6/tests/CMakeLists.txt | 16 |
10 files changed, 405 insertions, 89 deletions
diff --git a/sources/shiboken6/CMakeLists.txt b/sources/shiboken6/CMakeLists.txt index 4d5f279df..a6ed7681b 100644 --- a/sources/shiboken6/CMakeLists.txt +++ b/sources/shiboken6/CMakeLists.txt @@ -10,13 +10,17 @@ project(shiboken6) include(cmake/ShibokenSetup.cmake) -add_subdirectory(ApiExtractor) # Uses libclang -add_subdirectory(generator) # Uses ApiExtractor And QtCore +if(SHIBOKEN_BUILD_TOOLS) + add_subdirectory(ApiExtractor) # Uses libclang + add_subdirectory(generator) # Uses ApiExtractor And QtCore +endif() -add_subdirectory(libshiboken) # Uses Python -add_subdirectory(shibokenmodule) # Uses libshiboken +if(SHIBOKEN_BUILD_LIBS) + add_subdirectory(libshiboken) # Uses Python + add_subdirectory(shibokenmodule) # Uses libshiboken + add_subdirectory(data) +endif() -add_subdirectory(data) add_subdirectory(doc) if(BUILD_TESTS) diff --git a/sources/shiboken6/cmake/ShibokenHelpers.cmake b/sources/shiboken6/cmake/ShibokenHelpers.cmake index 5897f4bd2..3f571c331 100644 --- a/sources/shiboken6/cmake/ShibokenHelpers.cmake +++ b/sources/shiboken6/cmake/ShibokenHelpers.cmake @@ -75,25 +75,46 @@ endif() endmacro() -macro(set_python_site_packages) - execute_process( - COMMAND ${PYTHON_EXECUTABLE} -c "if True: - import sysconfig - from os.path import sep +macro(shiboken_internal_set_python_site_packages) + # When cross-building, we can't run the target python executable to find out the information, + # so we allow an explicit variable assignment or use a default / sensible value. + if(SHIBOKEN_IS_CROSS_BUILD OR PYSIDE_IS_CROSS_BUILD OR QFP_FIND_NEW_PYTHON_PACKAGE) + # Allow manual assignment. + if(QFP_PYTHON_SITE_PACKAGES) + set(PYTHON_SITE_PACKAGES "${QFP_PYTHON_SITE_PACKAGES}") + else() + # Assumes POSIX. + # Convention can be checked in cpython's source code in + # Lib/sysconfig.py's _INSTALL_SCHEMES + set(__version_major_minor + "${Python_VERSION_MAJOR}.${Python_VERSION_MINOR}") + + set(PYTHON_SITE_PACKAGES_WITHOUT_PREFIX + "lib/python${__version_major_minor}/site-packages") + set(PYTHON_SITE_PACKAGES + "${CMAKE_INSTALL_PREFIX}/${PYTHON_SITE_PACKAGES_WITHOUT_PREFIX}") + unset(__version_major_minor) + endif() + else() + execute_process( + COMMAND ${PYTHON_EXECUTABLE} -c "if True: + import sysconfig + from os.path import sep - # /home/qt/dev/env/lib/python3.9/site-packages - lib_path = sysconfig.get_path('purelib') + # /home/qt/dev/env/lib/python3.9/site-packages + lib_path = sysconfig.get_path('purelib') - # /home/qt/dev/env - data_path = sysconfig.get_path('data') + # /home/qt/dev/env + data_path = sysconfig.get_path('data') - # /lib/python3.9/site-packages - rel_path = lib_path.replace(data_path, '') + # /lib/python3.9/site-packages + rel_path = lib_path.replace(data_path, '') - print(f'${CMAKE_INSTALL_PREFIX}{rel_path}'.replace(sep, '/')) - " - OUTPUT_VARIABLE PYTHON_SITE_PACKAGES - OUTPUT_STRIP_TRAILING_WHITESPACE) + print(f'${CMAKE_INSTALL_PREFIX}{rel_path}'.replace(sep, '/')) + " + OUTPUT_VARIABLE PYTHON_SITE_PACKAGES + OUTPUT_STRIP_TRAILING_WHITESPACE) + endif() if (NOT PYTHON_SITE_PACKAGES) message(FATAL_ERROR "Could not detect Python module installation directory.") elseif (APPLE) @@ -152,21 +173,32 @@ macro(set_quiet_build) endmacro() macro(get_python_extension_suffix) - execute_process( - COMMAND ${PYTHON_EXECUTABLE} -c "if True: - import sys - import sysconfig - suffix = sysconfig.get_config_var('EXT_SUFFIX') - pos = suffix.rfind('.') - if pos > 0: - print(suffix[:pos]) - else: - print(f'Unable to determine PYTHON_EXTENSION_SUFFIX from EXT_SUFFIX: \"{suffix}\"', - file=sys.stderr) - " - OUTPUT_VARIABLE PYTHON_EXTENSION_SUFFIX - OUTPUT_STRIP_TRAILING_WHITESPACE) - message(STATUS "PYTHON_EXTENSION_SUFFIX: " ${PYTHON_EXTENSION_SUFFIX}) + # When cross-building, we can't run the target python executable to find out the information, + # so we rely on Python_SOABI being set by find_package(Python). + # Python_SOABI is only set by CMake 3.17+ + # TODO: Lower this to CMake 3.16 if possible. + if(SHIBOKEN_IS_CROSS_BUILD) + if(NOT Python_SOABI) + message(FATAL_ERROR "Python_SOABI variable is empty.") + endif() + set(PYTHON_EXTENSION_SUFFIX ".${Python_SOABI}") + else() + execute_process( + COMMAND ${PYTHON_EXECUTABLE} -c "if True: + import sys + import sysconfig + suffix = sysconfig.get_config_var('EXT_SUFFIX') + pos = suffix.rfind('.') + if pos > 0: + print(suffix[:pos]) + else: + print(f'Unable to determine PYTHON_EXTENSION_SUFFIX from EXT_SUFFIX: \"{suffix}\"', + file=sys.stderr) + " + OUTPUT_VARIABLE PYTHON_EXTENSION_SUFFIX + OUTPUT_STRIP_TRAILING_WHITESPACE) + endif() + message(STATUS "PYTHON_EXTENSION_SUFFIX: " ${PYTHON_EXTENSION_SUFFIX}) endmacro() macro(shiboken_parse_all_arguments prefix type flags options multiopts) @@ -177,21 +209,26 @@ macro(shiboken_parse_all_arguments prefix type flags options multiopts) endmacro() macro(shiboken_check_if_limited_api) - # On Windows, PYTHON_LIBRARIES can be a list. Example: - # optimized;C:/Python36/libs/python36.lib;debug;C:/Python36/libs/python36_d.lib - # On other platforms, this result is not used at all. - execute_process( - COMMAND ${PYTHON_EXECUTABLE} -c "if True: - import os - for lib in '${PYTHON_LIBRARIES}'.split(';'): - if '/' in lib and os.path.isfile(lib): - prefix, py = lib.rsplit('/', 1) - if py.startswith('python3'): - print(prefix + '/python3.lib') - break - " - OUTPUT_VARIABLE PYTHON_LIMITED_LIBRARIES - OUTPUT_STRIP_TRAILING_WHITESPACE) + # TODO: Figure out how to use limited API libs when cross-building to Windows, if that's ever + # needed. Perhaps use host python to walk the libs of the target python installation. + + if(NOT SHIBOKEN_IS_CROSS_BUILD) + # On Windows, PYTHON_LIBRARIES can be a list. Example: + # optimized;C:/Python36/libs/python36.lib;debug;C:/Python36/libs/python36_d.lib + # On other platforms, this result is not used at all. + execute_process( + COMMAND ${PYTHON_EXECUTABLE} -c "if True: + import os + for lib in '${PYTHON_LIBRARIES}'.split(';'): + if '/' in lib and os.path.isfile(lib): + prefix, py = lib.rsplit('/', 1) + if py.startswith('python3'): + print(prefix + '/python3.lib') + break + " + OUTPUT_VARIABLE PYTHON_LIMITED_LIBRARIES + OUTPUT_STRIP_TRAILING_WHITESPACE) + endif() if(FORCE_LIMITED_API STREQUAL "yes") if (${PYTHON_VERSION_MAJOR} EQUAL 3 AND ${PYTHON_VERSION_MINOR} GREATER 4) @@ -209,16 +246,72 @@ endmacro() macro(shiboken_find_required_python) - if(${ARGC} GREATER 0) - find_package(PythonInterp ${ARGV0} REQUIRED) - find_package(PythonLibs ${ARGV0} REQUIRED) + # This function can also be called by consumers of ShibokenConfig.cmake package like pyside, + # that's why we also check for PYSIDE_IS_CROSS_BUILD (which is set by pyside project) + # and QFP_FIND_NEW_PYTHON_PACKAGE for an explicit opt in. + # + # We have to use FindPython package instead of FindPythonInterp to get required target Python + # information. + if(SHIBOKEN_IS_CROSS_BUILD OR PYSIDE_IS_CROSS_BUILD OR QFP_FIND_NEW_PYTHON_PACKAGE) + set(_shiboken_find_python_version_args "") + if(${ARGC} GREATER 0) + list(APPEND _shiboken_find_python_version_args "${ARGV0}") + endif() + + # We want FindPython to look in the sysroot for the python-config executable, + # but toolchain files might set CMAKE_FIND_ROOT_PATH_MODE_PROGRAM to NEVER because + # programs are mostly found for running and you usually can't run a target executable on + # a host platform. python-config can likely be ran though, because it's a shell script + # to be run on a host Linux. + set(_shiboken_backup_CMAKE_FIND_ROOT_PATH_MODE_PROGRAM + "${CMAKE_FIND_ROOT_PATH_MODE_PROGRAM}") + set(_shiboken_backup_CMAKE_FIND_ROOT_PATH + "${CMAKE_FIND_ROOT_PATH}") + set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM ONLY) + if(Python_ROOT_DIR) + list(PREPEND CMAKE_FIND_ROOT_PATH "${Python_ROOT_DIR}") + endif() + + # We can't look for the Python interpreter because FindPython tries to execute it, which + # usually won't work on a host platform due to different architectures / platforms. + # Thus we only look for the Python include and lib directories which are part of the + # Development component. + find_package( + Python + ${_shiboken_find_python_version_args} + REQUIRED + COMPONENTS Development + ) + set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM + "${_shiboken_backup_CMAKE_FIND_ROOT_PATH_MODE_PROGRAM}") + set(CMAKE_FIND_ROOT_PATH + "${_shiboken_backup_CMAKE_FIND_ROOT_PATH}") + + # Mirror the variables that FindPythonInterp sets, instead of conditionally checking + # and modifying all the places where the variables are used. + set(PYTHON_EXECUTABLE "${Python_EXECUTABLE}") + set(PYTHON_VERSION "${Python_VERSION}") + set(PYTHON_LIBRARIES "${Python_LIBRARIES}") + set(PYTHON_INCLUDE_DIRS "${Python_INCLUDE_DIRS}") + set(PYTHONINTERP_FOUND "${Python_Interpreter_FOUND}") + set(PYTHONINTERP_FOUND "${Python_Interpreter_FOUND}") + set(PYTHONLIBS_FOUND "${Python_Development_FOUND}") + set(PYTHON_VERSION_MAJOR "${Python_VERSION_MAJOR}") + set(PYTHON_VERSION_MINOR "${Python_VERSION_MINOR}") + set(PYTHON_VERSION_PATCH "${Python_VERSION_PATCH}") else() - # If no version is specified, just use any interpreter that can be found (from PATH). - # This is useful for super-project builds, so that the default system interpeter - # gets picked up (e.g. /usr/bin/python and not /usr/bin/python2.7). - find_package(PythonInterp REQUIRED) - find_package(PythonLibs REQUIRED) + if(${ARGC} GREATER 0) + find_package(PythonInterp ${ARGV0} REQUIRED) + find_package(PythonLibs ${ARGV0} REQUIRED) + else() + # If no version is specified, just use any interpreter that can be found (from PATH). + # This is useful for super-project builds, so that the default system interpeter + # gets picked up (e.g. /usr/bin/python and not /usr/bin/python2.7). + find_package(PythonInterp REQUIRED) + find_package(PythonLibs REQUIRED) + endif() endif() + shiboken_validate_python_version() set(SHIBOKEN_PYTHON_INTERPRETER "${PYTHON_EXECUTABLE}") @@ -398,6 +491,85 @@ function(shiboken_internal_disable_pkg_config_if_needed) endif() endfunction() +function(shiboken_internal_detect_if_cross_building) + if(CMAKE_CROSSCOMPILING OR QFP_SHIBOKEN_HOST_PATH) + set(is_cross_build TRUE) + else() + set(is_cross_build FALSE) + endif() + set(SHIBOKEN_IS_CROSS_BUILD "${is_cross_build}" PARENT_SCOPE) + message(STATUS "SHIBOKEN_IS_CROSS_BUILD: ${is_cross_build}") +endfunction() + +function(shiboken_internal_decide_parts_to_build) + set(build_libs_default ON) + option(SHIBOKEN_BUILD_LIBS "Build shiboken libraries" ${build_libs_default}) + message(STATUS "SHIBOKEN_BUILD_LIBS: ${SHIBOKEN_BUILD_LIBS}") + + if(SHIBOKEN_IS_CROSS_BUILD) + set(build_tools_default OFF) + else() + set(build_tools_default ON) + endif() + option(SHIBOKEN_BUILD_TOOLS "Build shiboken tools" ${build_tools_default}) + message(STATUS "SHIBOKEN_BUILD_TOOLS: ${SHIBOKEN_BUILD_TOOLS}") + + if(SHIBOKEN_IS_CROSS_BUILD) + set(_shiboken_build_tests_default OFF) + elseif(SHIBOKEN_BUILD_LIBS) + set(_shiboken_build_tests_default ON) + endif() + option(BUILD_TESTS "Build tests." ${_shiboken_build_tests_default}) + message(STATUS "BUILD_TESTS: ${BUILD_TESTS}") +endfunction() + +function(shiboken_internal_find_host_shiboken_tools) + if(SHIBOKEN_IS_CROSS_BUILD) + set(find_package_extra_args) + if(QFP_SHIBOKEN_HOST_PATH) + list(APPEND find_package_extra_args PATHS "${QFP_SHIBOKEN_HOST_PATH}/lib/cmake") + list(PREPEND CMAKE_FIND_ROOT_PATH "${QFP_SHIBOKEN_HOST_PATH}") + endif() + find_package( + Shiboken6Tools 6 CONFIG + ${find_package_extra_args} + ) + + if(NOT Shiboken6Tools_DIR) + message(FATAL_ERROR + "Shiboken6Tools package was not found. " + "Please set QFP_SHIBOKEN_HOST_PATH to the location where the Shiboken6Tools CMake " + "package is installed.") + endif() + endif() +endfunction() + +function(shiboken_internal_set_up_extra_dependency_paths) + set(extra_root_path_vars + QFP_QT_TARGET_PATH + QFP_PYTHON_TARGET_PATH + ) + foreach(root_path IN LISTS extra_root_path_vars) + set(new_root_path_value "${${root_path}}") + if(new_root_path_value) + set(new_prefix_path "${CMAKE_PREFIX_PATH}") + list(PREPEND new_prefix_path "${new_root_path_value}/lib/cmake") + set(CMAKE_PREFIX_PATH "${new_prefix_path}") + set(CMAKE_PREFIX_PATH "${new_prefix_path}" PARENT_SCOPE) + + # Need to adjust the prefix and root paths so that find_package(Qt) and other 3rd + # party packages are found successfully when they are located outside of the + # default sysroot (whatever that maybe for the target platform). + if(SHIBOKEN_IS_CROSS_BUILD) + set(new_root_path "${CMAKE_FIND_ROOT_PATH}") + list(PREPEND new_root_path "${new_root_path_value}") + set(CMAKE_FIND_ROOT_PATH "${new_root_path}") + set(CMAKE_FIND_ROOT_PATH "${new_root_path}" PARENT_SCOPE) + endif() + endif() + endforeach() +endfunction() + macro(compute_config_py_values full_version_var_name ) diff --git a/sources/shiboken6/cmake/ShibokenSetup.cmake b/sources/shiboken6/cmake/ShibokenSetup.cmake index 5eaa01d2a..c9b022e1e 100644 --- a/sources/shiboken6/cmake/ShibokenSetup.cmake +++ b/sources/shiboken6/cmake/ShibokenSetup.cmake @@ -4,11 +4,22 @@ list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_LIST_DIR}") include(ShibokenHelpers) -option(BUILD_TESTS "Build tests." TRUE) option(USE_PYTHON_VERSION "Use specific python version to build shiboken6." "") option(DISABLE_DOCSTRINGS "Disable documentation extraction." FALSE) shiboken_internal_disable_pkg_config_if_needed() +shiboken_internal_detect_if_cross_building() + +if(SHIBOKEN_IS_CROSS_BUILD AND CMAKE_VERSION VERSION_LESS "3.17") + # TODO: We rely on FindPython shipped with CMake 3.17+ to provide the value of Python_SOABI. + # It might be possible to extract the Python_SOABI manually with CMake 3.16 if we reimplement + # the logic that FindPython does in 3.17 ourselves. + message(FATAL_ERROR "You need CMake version 3.17 or greater to cross-build.") +endif() + +shiboken_internal_decide_parts_to_build() +shiboken_internal_find_host_shiboken_tools() +shiboken_internal_set_up_extra_dependency_paths() set(QT_MAJOR_VERSION 6) message(STATUS "Using Qt ${QT_MAJOR_VERSION}") @@ -18,13 +29,15 @@ if(QUIET_BUILD) set_quiet_build() endif() -if (USE_PYTHON_VERSION) +if(USE_PYTHON_VERSION) shiboken_find_required_python(${USE_PYTHON_VERSION}) else() shiboken_find_required_python() endif() -setup_clang() +if(SHIBOKEN_BUILD_TOOLS) + setup_clang() +endif() set(shiboken6_VERSION "${shiboken_MAJOR_VERSION}.${shiboken_MINOR_VERSION}.${shiboken_MICRO_VERSION}") set(shiboken6_library_so_version "${shiboken_MAJOR_VERSION}.${shiboken_MINOR_VERSION}") @@ -70,7 +83,7 @@ message(STATUS "PYTHON_SHARED_LIBRARY_SUFFIX: ${PYTHON_SHARED_LIBRARY_SUFFIX}") if(NOT PYTHON_SITE_PACKAGES) - set_python_site_packages() + shiboken_internal_set_python_site_packages() endif() set_cmake_cxx_flags() @@ -120,20 +133,28 @@ execute_process( # Detect if python interpeter was compiled with COUNT_ALLOCS define # Linux distros are inconsistent in setting the sysconfig.get_config_var('COUNT_ALLOCS') value -execute_process( - COMMAND ${PYTHON_EXECUTABLE} -c "if True: - count_allocs = False - import sys - try: - if sys.getcounts: - count_allocs = True - except: - pass +# We can't detect it when cross-building, because we can't run the target python executable. +# TODO: Is there another way to detect this and is it relevant for cross-built python interpreters? +# At the very least, upstream CPython removed COUNT_ALLOCS support in Python 3.9. +if(SHIBOKEN_IS_CROSS_BUILD) + set(PYTHON_WITH_COUNT_ALLOCS 0) +else() + execute_process( + COMMAND ${PYTHON_EXECUTABLE} -c "if True: + count_allocs = False + import sys + try: + if sys.getcounts: + count_allocs = True + except: + pass + + print(bool(count_allocs)) + " + OUTPUT_VARIABLE PYTHON_WITH_COUNT_ALLOCS + OUTPUT_STRIP_TRAILING_WHITESPACE) +endif() - print(bool(count_allocs)) - " - OUTPUT_VARIABLE PYTHON_WITH_COUNT_ALLOCS - OUTPUT_STRIP_TRAILING_WHITESPACE) set(SHIBOKEN_BUILD_TYPE "${CMAKE_BUILD_TYPE}") diff --git a/sources/shiboken6/config.tests/target_python_info/CMakeLists.txt b/sources/shiboken6/config.tests/target_python_info/CMakeLists.txt new file mode 100644 index 000000000..55fafb7c3 --- /dev/null +++ b/sources/shiboken6/config.tests/target_python_info/CMakeLists.txt @@ -0,0 +1,44 @@ +cmake_minimum_required(VERSION 3.16) +project(proj LANGUAGES CXX) + +include("${CMAKE_CURRENT_LIST_DIR}/../../cmake/ShibokenHelpers.cmake") + +shiboken_internal_detect_if_cross_building() +shiboken_find_required_python() +shiboken_internal_set_python_site_packages() + +message(STATUS "qfp:python_info:interpreter_found: ${Python_Interpreter_FOUND}") +message(STATUS "qfp:python_info:development_found: ${Python_Development_FOUND}") +message(STATUS "qfp:python_info:version: ${Python_VERSION}") +message(STATUS "qfp:python_info:version_major: ${Python_VERSION_MAJOR}") +message(STATUS "qfp:python_info:version_minor: ${Python_VERSION_MINOR}") +message(STATUS "qfp:python_info:version_patch: ${Python_VERSION_PATCH}") +message(STATUS "qfp:python_info:executable: ${Python_EXECUTABLE}") +message(STATUS "qfp:python_info:include_dirs: ${Python_INCLUDE_DIRS}") +message(STATUS "qfp:python_info:libraries: ${Python_LIBRARIES}") +message(STATUS "qfp:python_info:library_dirs: ${Python_LIBRARY_DIRS}") +message(STATUS "qfp:python_info:runtime_library_dirs: ${Python_RUNTIME_LIBRARY_DIRS}") +# Python_SOABI will be empty with CMake < 3.17 +message(STATUS "qfp:python_info:so_abi: ${Python_SOABI}") +message(STATUS "qfp:python_info:site_packages_dir: ${PYTHON_SITE_PACKAGES_WITHOUT_PREFIX}") +message(STATUS "qfp:python_info:site_packages_dir_with_prefix: ${PYTHON_SITE_PACKAGES}") + +message(STATUS + "The following values might be unstable because they depend on private FindPython API") +# This is using internal FindPython API and is subject to break. +set(_PYTHON_PREFIX Python) +if(COMMAND _python_get_config_var) + if(_${_PYTHON_PREFIX}_CONFIG) + message(STATUS "qfp:python_info:config_executable: ${_${_PYTHON_PREFIX}_CONFIG}") + endif() + + _python_get_config_var(_${_PYTHON_PREFIX}_PREFIX PREFIX) + if(_${_PYTHON_PREFIX}_PREFIX) + message(STATUS "qfp:python_info:prefix: ${_${_PYTHON_PREFIX}_PREFIX}") + endif() + _python_get_config_var(_${_PYTHON_PREFIX}_CONFIGDIR CONFIGDIR) + if(_${_PYTHON_PREFIX}_CONFIGDIR) + message(STATUS "qfp:python_info:config_dir: ${_${_PYTHON_PREFIX}_CONFIGDIR}") + endif() +endif() + diff --git a/sources/shiboken6/config.tests/target_qt_info/CMakeLists.txt b/sources/shiboken6/config.tests/target_qt_info/CMakeLists.txt new file mode 100644 index 000000000..bb0e11b9e --- /dev/null +++ b/sources/shiboken6/config.tests/target_qt_info/CMakeLists.txt @@ -0,0 +1,36 @@ +cmake_minimum_required(VERSION 3.16) +project(proj LANGUAGES CXX) + +include("${CMAKE_CURRENT_LIST_DIR}/../../cmake/ShibokenHelpers.cmake") + +shiboken_internal_detect_if_cross_building() +shiboken_internal_set_up_extra_dependency_paths() +find_package(Qt6 REQUIRED COMPONENTS Core) + +include(FeatureSummary) + +feature_summary(INCLUDE_QUIET_PACKAGES + WHAT PACKAGES_FOUND + REQUIRED_PACKAGES_NOT_FOUND + RECOMMENDED_PACKAGES_NOT_FOUND + OPTIONAL_PACKAGES_NOT_FOUND + RUNTIME_PACKAGES_NOT_FOUND + FATAL_ON_MISSING_REQUIRED_PACKAGES) + +message(STATUS "qfp:qt_info:QT_VERSION: ${Qt6_VERSION}") +message(STATUS "qfp:qt_info:QT_INSTALL_PREFIX: ${QT6_INSTALL_PREFIX}") +message(STATUS "qfp:qt_info:QT_INSTALL_ARCHDATA: ${QT6_INSTALL_PREFIX}/${QT6_INSTALL_ARCHDATA}") +message(STATUS "qfp:qt_info:QT_INSTALL_BINS: ${QT6_INSTALL_PREFIX}/${QT6_INSTALL_BINS}") +message(STATUS "qfp:qt_info:QT_INSTALL_CONFIGURATION: ${QT6_INSTALL_CONFIGURATION}") +message(STATUS "qfp:qt_info:QT_INSTALL_DATA: ${QT6_INSTALL_PREFIX}/${QT6_INSTALL_DATA}") +message(STATUS "qfp:qt_info:QT_INSTALL_DOCS: ${QT6_INSTALL_PREFIX}/${QT6_INSTALL_DOCS}") +message(STATUS "qfp:qt_info:QT_INSTALL_EXAMPLES: ${QT6_INSTALL_PREFIX}/${QT6_INSTALL_EXAMPLES}") +message(STATUS "qfp:qt_info:QT_INSTALL_HEADERS: ${QT6_INSTALL_PREFIX}/${QT6_INSTALL_HEADERS}") +message(STATUS "qfp:qt_info:QT_INSTALL_LIBS: ${QT6_INSTALL_PREFIX}/${QT6_INSTALL_LIBS}") +message(STATUS "qfp:qt_info:QT_INSTALL_LIBEXECS: ${QT6_INSTALL_PREFIX}/${QT6_INSTALL_LIBEXECS}") +message(STATUS "qfp:qt_info:QT_INSTALL_PLUGINS: ${QT6_INSTALL_PREFIX}/${QT6_INSTALL_PLUGINS}") +message(STATUS "qfp:qt_info:QT_INSTALL_QML: ${QT6_INSTALL_PREFIX}/${QT6_INSTALL_QML}") +message(STATUS "qfp:qt_info:QT_INSTALL_TESTS: ${QT6_INSTALL_PREFIX}/${QT6_INSTALL_TESTS}") +message(STATUS "qfp:qt_info:QT_INSTALL_TRANSLATIONS: ${QT6_INSTALL_PREFIX}/${QT6_INSTALL_TRANSLATIONS}") +message(STATUS "qfp:qt_info:is_shared: ${QT6_IS_SHARED_LIBS_BUILD}") + diff --git a/sources/shiboken6/doc/CMakeLists.txt b/sources/shiboken6/doc/CMakeLists.txt index 7be47363b..25a4be1ec 100644 --- a/sources/shiboken6/doc/CMakeLists.txt +++ b/sources/shiboken6/doc/CMakeLists.txt @@ -57,8 +57,18 @@ except: message(STATUS "qhelpgenerator - found") file(TO_NATIVE_PATH ${CMAKE_CURRENT_BINARY_DIR}/html/Shiboken.qhp QHP_FILE) + + if(SHIBOKEN_IS_CROSS_BUILD) + set(python_executable "${QFP_PYTHON_HOST_PATH}") + else() + set(python_executable "${PYTHON_EXECUTABLE}") + endif() + if(NOT python_executable OR NOT EXISTS "${python_executable}") + message(FATAL_ERROR "No python executable found to build documentation.") + endif() + add_custom_command(TARGET doc POST_BUILD - COMMAND ${PYTHON_EXECUTABLE} py_script.py # ${CMAKE_CURRENT_BINARY_DIR}/html/Shiboken.qhp + COMMAND "${python_executable}" py_script.py # ${CMAKE_CURRENT_BINARY_DIR}/html/Shiboken.qhp COMMAND "${qhelpgenerator_binary}" ${QHP_FILE} COMMENT "Generating shiboken documentation QCH files based on the QHP files" VERBATIM) diff --git a/sources/shiboken6/libshiboken/CMakeLists.txt b/sources/shiboken6/libshiboken/CMakeLists.txt index 5c454b578..f9267b95a 100644 --- a/sources/shiboken6/libshiboken/CMakeLists.txt +++ b/sources/shiboken6/libshiboken/CMakeLists.txt @@ -23,13 +23,25 @@ else() set(embedding_option "") endif() +if(SHIBOKEN_IS_CROSS_BUILD) + set(host_python_path "${QFP_PYTHON_HOST_PATH}") + set(use_pyc_in_embedding FALSE) +else() + set(host_python_path "${PYTHON_EXECUTABLE}") + if(PYTHON_LIMITED_API) + set(use_pyc_in_embedding FALSE) + else() + set(use_pyc_in_embedding TRUE) + endif() +endif() + add_custom_command( OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/embed/signature_bootstrap_inc.h" OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/embed/signature_inc.h" - COMMAND ${PYTHON_EXECUTABLE} -E + COMMAND ${host_python_path} -E "${CMAKE_CURRENT_SOURCE_DIR}/embed/embedding_generator.py" --cmake-dir "${CMAKE_CURRENT_BINARY_DIR}/embed" - --limited-api ${PYTHON_LIMITED_API} + --use-pyc ${use_pyc_in_embedding} ${embedding_option} DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/embed/embedding_generator.py" "${CMAKE_CURRENT_SOURCE_DIR}/embed/signature_bootstrap.py" @@ -98,6 +110,10 @@ if (NOT "${NUMPY_INCLUDE_DIR}" STREQUAL "") endif() +if(SHIBOKEN_IS_CROSS_BUILD) + target_compile_definitions(libshiboken PRIVATE -DSHIBOKEN_NO_EMBEDDING_PYC=1) +endif() + shiboken_compute_python_includes() # On Windows we need to link against the python.lib import library. # On macOS and Linux we don't link against the python shared / static library, diff --git a/sources/shiboken6/libshiboken/embed/embedding_generator.py b/sources/shiboken6/libshiboken/embed/embedding_generator.py index 1477ff541..d1fb0bd6d 100644 --- a/sources/shiboken6/libshiboken/embed/embedding_generator.py +++ b/sources/shiboken6/libshiboken/embed/embedding_generator.py @@ -81,7 +81,7 @@ def runpy(cmd, **kw): subprocess.call([sys.executable, '-E'] + cmd.split(), **kw) -def create_zipfile(limited_api, quiet): +def create_zipfile(use_pyc, quiet): """ Collect all Python files, compile them, create a zip file and make a chunked base64 encoded file from it. @@ -110,7 +110,7 @@ def create_zipfile(limited_api, quiet): if embed_dir != work_dir: utils.copyfile(embed_dir / "signature_bootstrap.py", work_dir) - if limited_api: + if not use_pyc: pass # We cannot compile, unless we have folders per Python version else: files = ' '.join(fn for fn in os.listdir('.')) @@ -126,9 +126,9 @@ def create_zipfile(limited_api, quiet): tmp.close() # also generate a simple embeddable .pyc file for signature_bootstrap.pyc - boot_name = "signature_bootstrap.py" if limited_api else "signature_bootstrap.pyc" + boot_name = "signature_bootstrap.py" if not use_pyc else "signature_bootstrap.pyc" with open(boot_name, "rb") as ldr, open("signature_bootstrap_inc.h", "w") as inc: - _embed_bytefile(ldr, inc, limited_api) + _embed_bytefile(ldr, inc, not use_pyc) os.chdir(cur_dir) if quiet: return @@ -247,9 +247,9 @@ def str2bool(v): if __name__ == "__main__": parser = argparse.ArgumentParser() parser.add_argument('--cmake-dir', nargs="?") - parser.add_argument('--limited-api', type=str2bool) + parser.add_argument('--use-pyc', type=str2bool) parser.add_argument('--quiet', action='store_true') args = parser.parse_args() if args.cmake_dir: work_dir = Path(args.cmake_dir).resolve() - create_zipfile(args.limited_api, args.quiet) + create_zipfile(args.use_pyc, args.quiet) diff --git a/sources/shiboken6/libshiboken/signature/signature_globals.cpp b/sources/shiboken6/libshiboken/signature/signature_globals.cpp index 744ce0407..eb5e1c3bc 100644 --- a/sources/shiboken6/libshiboken/signature/signature_globals.cpp +++ b/sources/shiboken6/libshiboken/signature/signature_globals.cpp @@ -89,8 +89,9 @@ static safe_globals_struc *init_phase_1(PyMethodDef *init_meth) * Since we now have an embedding script, we can do this without any * Python strings in the C code. */ -#ifdef Py_LIMITED_API - // We must work for multiple versions, so use source code. +#if defined(Py_LIMITED_API) || defined(SHIBOKEN_NO_EMBEDDING_PYC) + // We must work for multiple versions or we are cross-building for a different + // Python version interpreter, so use source code. #else AutoDecRef marshal_module(PyImport_Import(PyName::marshal())); if (marshal_module.isNull()) @@ -104,7 +105,7 @@ static safe_globals_struc *init_phase_1(PyMethodDef *init_meth) AutoDecRef bytes(PyBytes_FromStringAndSize(bytes_cast, sizeof(PySide_SignatureLoader))); if (bytes.isNull()) goto error; -#ifdef Py_LIMITED_API +#if defined(Py_LIMITED_API) || defined(SHIBOKEN_NO_EMBEDDING_PYC) PyObject *builtins = PyEval_GetBuiltins(); PyObject *compile = PyDict_GetItem(builtins, PyName::compile()); if (compile == nullptr) diff --git a/sources/shiboken6/tests/CMakeLists.txt b/sources/shiboken6/tests/CMakeLists.txt index bfa92e229..296f8d8d7 100644 --- a/sources/shiboken6/tests/CMakeLists.txt +++ b/sources/shiboken6/tests/CMakeLists.txt @@ -39,7 +39,16 @@ list(SORT TEST_FILES) set(test_blacklist "") -find_package(PythonInterp REQUIRED) +if(SHIBOKEN_IS_CROSS_BUILD) + # PYTHON_EXECUTABLE will be empty when cross-building. + message(WARNING + "Running tests when cross-compiling is not supported because it would require running " + "a target python interpreter which might have a different architecture than the host." + ) +else() + find_package(PythonInterp REQUIRED) +endif() + if(NOT CTEST_TESTING_TIMEOUT) set(CTEST_TESTING_TIMEOUT 60) endif() @@ -59,7 +68,10 @@ foreach(test_file ${TEST_FILES}) endif() endforeach() -add_subdirectory(dumpcodemodel) +# dumpcodemodel depends on apiextractor which is not cross-built. +if(SHIBOKEN_BUILD_TOOLS) + add_subdirectory(dumpcodemodel) +endif() # FIXME Skipped until add an option to choose the generator # add_subdirectory(test_generator) |