1

I'm trying to compile a Go native binary to use in Java code on Mac. The binary needs to be executed on a linux host which has the following specifications:

# uname -a
Linux <hostname> 5.14.0-284.25.1.el9_2.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Jul 20 09:11:28 EDT 2023 x86_64 x86_64 x86_64 GNU/Linux

For local testing on mac, I simply generate the binary using go build -o libmybinary.so -buildmode=c-shared main.go but to compile it in the correct format for the linux host, I use this instead:

FROM source as builder 

RUN CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build -o libmybinary.so -buildmode=c-shared main.go

The problem is that if I don't use CGO_ENABLED=1 GOOS=linux GOARCH=amd64, the binary cannot be executed on the host, but if I try to use it, I get this error in docker build:

1.553 # runtime/cgo
1.553 gcc: error: unrecognized command line option '-m64'
------
ERROR: failed to solve: process "/bin/bash -eo pipefail -c CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build -o libmybinary.so -buildmode=c-shared main.go" did not complete successfully: exit code: 1

I also tried replacing

FROM source as builder 
RUN CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build -o libmybinary.so -buildmode=c-shared main.go

with

FROM --platform=linux/amd64 source as builder 
RUN go build -o libmybinary.so -buildmode=c-shared main.go

and it seems to compile the .so in correct format for the host but my java code couldn't find the exported methods as I think this way does not essentially "cross-compile" within the native image, leading to differences in behaviour with cgo and the underlying C toolchain. I wanted to check what would be the right way to achieve the correctly compiled binary?

Improve this question
4
  • Is there anything Java specific in this question? otherwise remove the Java tag. Commented Mar 14 at 1:37
  • Does this Mac have an Intel CPU or an ARM CPU (M series)? Commented Mar 14 at 1:56
  • (1) Before you use Docker, you can first run Go compilation on your Mac and test it with Java. Everything is done on your Mac. The results are for Mac. (2) After confirming that everything is OK, move to Docker to compile and test Go and Java for amd64. Commented Mar 14 at 1:59
  • The Mac is M series, I've already done the first part and Mac compiled binary works on Mac. The second part is the issue because I can't figure out the right way to compile the binary for linux amd64 using docker. Commented Mar 14 at 11:30

2 Answers 2

Reset to default
2

Since I don't have a Mac M, I can only provide my approach to compiling ARM on x86_64:

For my case, I compile programs on an x86_64 system targeting the ARM platform.

For your case, you should be compiling on your ARM environment to target the x86_64 (amd64) platform.

Project Directory

create a base directory: Java-JNA-Go-MultiArch

The following commands, unless otherwise stated, are executed in the Java-JNA-Go-MultiArch directory by default.

Verify that Docker's QEMU is supported.

docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

Test ARM container

Then test whether the ARM container can be executed:

docker run --rm --platform linux/arm64 alpine uname -m

If aarch64 is output, it means QEMU has started normally.

Test AMD container (x86_64)

docker run --rm --platform linux/amd64 alpine uname -m

output: x86_64, it means QEMU has started normally.

go_mylib

crate go_mylib directory.

Java-JNA-Go-MultiArch
└── go_mylib
    ├── mylib.go
    ├── linux-arm-64 (dir)
    └── linux-x86-64 (dir)

mylib.go

package main

import "C"

//export add_integers
func add_integers(a C.int, b C.int) C.int {
    return a + b
}

//export concat_strings
func concat_strings(x *C.char, y *C.char) *C.char {
    result := C.CString(C.GoString(x) + C.GoString(y))
    return result
}

func main() {}

Build

Build ARM64

docker run --rm --platform linux/arm64 \
  -v $(pwd)/go_mylib:/app -w /app \
  golang:latest \
  go build -o linux-arm-64/libmylib.so -buildmode=c-shared mylib.go

Build AMD 64

docker run --rm --platform linux/amd64 \
  -v $(pwd)/go_mylib:/app -w /app \
  golang:latest \
  go build -o linux-x86-64/libmylib.so -buildmode=c-shared mylib.go

Final Result

Java-JNA-Go-MultiArch
└── go_mylib
    ├── mylib.go
    ├── linux-arm-64
    │   ├── libmylib.h
    │   └── libmylib.so
    └── linux-x86-64
        ├── libmylib.h
        └── libmylib.so

Important

When compiling into a shared library (.so) using the Go language, it will reference the version of GLIBC, so you must know which GLIBC version is being used.

Run command:

docker run -it --rm --platform linux/arm64 \
  golang:latest \
  bash

and

docker run -it --rm --platform linux/amd64 \
  golang:latest \
  bash

In container:

ldd --version

get result: GLIBC 2.36

ldd (Debian GLIBC 2.36-9+deb12u9) 2.36

java_app_call_mylib

Project Driectory

Java-JNA-Go-MultiArch
└── java_app_call_mylib
    ├── pom.xml
    └── src
        └── main
            └── java
                └── com
                    └── example
                        └── JnaExample.java

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.example</groupId>
    <artifactId>go-jna-example</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <properties>
            <maven.compiler.source>17</maven.compiler.source>
                <maven.compiler.target>17</maven.compiler.target>
                <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
                <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        </properties>

    <dependencies>
        <!-- JNA dependency for native library access -->
        <dependency>
            <groupId>net.java.dev.jna</groupId>
            <artifactId>jna</artifactId>
            <version>5.11.0</version>
        </dependency>
    </dependencies>

    <build>
      <finalName>app</finalName>
    </build>
</project>

JnaExample.java

package com.example;

import com.sun.jna.Library;
import com.sun.jna.Native;
import com.sun.jna.Pointer;

public class JnaExample {

  // Define interface mapping to local library
  public interface MyLib extends Library {
  
    //Load dynamic library
    MyLib INSTANCE = Native.load("mylib", MyLib.class);

    int add_integers(int a, int b);

    Pointer concat_strings(String x, String y);
  }

  public static void main(String[] args) {
    // Call the add_integers function
    int sum = MyLib.INSTANCE.add_integers(10, 20);
    System.out.println("Sum: " + sum);

    //Call concat_strings function
    Pointer resultPointer = MyLib.INSTANCE.concat_strings("Hello, ", "World!");
    String resultString = resultPointer.getString(0); //Read string from pointer
    System.out.println("Concatenated String: " + resultString);

    // Manually release the memory allocated by malloc
    Native.free(Pointer.nativeValue(resultPointer));
  }  
}

Build

Build in docker container

run command in Java-JNA-Go-MultiArch

docker run --rm \
  -v $(pwd)/java_app_call_mylib:/app -w /app \
  maven:3.8.2-eclipse-temurin-17 \
  mvn clean package
  
docker run --rm \
  -v $(pwd)/java_app_call_mylib:/app -w /app \
  maven:3.8.2-eclipse-temurin-17 \
  mvn dependency:copy-dependencies -DoutputDirectory=target/libs

or

you can run command in Java-JNA-Go-MultiArch/java_app_call_mylib

mvn clean package

mvn dependency:copy-dependencies -DoutputDirectory=target/libs

Output Result

Java-JNA-Go-MultiArch
└── java_app_call_mylib
    ...
    └── target
        ├── app.jar
        └── libs
            └── jna-5.11.0.jar

app

We create an app directory and test it based on the above results.

create app under Java-JNA-Go-MultiArch

Java-JNA-Go-MultiArch
└── app
    ├── app.jar
    ├── libs
    │   └── jna-5.11.0.jar
    ├── linux-arm-64
    │   └── libmylib.so
    └── linux-x86-64
        └── libmylib.so
  • app.jar : copy it from java_app_call_mylib/target
  • libs/jna-5.11.0.jar : copy it from java_app_call_mylib/target
  • linux-arm-64/libmylib.so : copy it from go_mylib
  • linux-x86-64/libmylib.so : copy it from go_mylib

Test ARM64

in Java-JNA-Go-MultiArch

Run command:

docker run -it --rm --platform linux/arm64 \
  -v $(pwd)/app:/app -w /app \
  openjdk:24-ea-17-jdk-bookworm \
  bash

in container:

export LD_LIBRARY_PATH=`pwd`/linux-arm-64
java --enable-native-access=ALL-UNNAMED -cp "libs/*:app.jar" com.example.JnaExample

Test AMD64 (For your needs)

in Java-JNA-Go-MultiArch

Run command:

docker run -it --rm --platform linux/amd64 \
  -v $(pwd)/app:/app -w /app \
  openjdk:24-ea-17-jdk-bookworm \
  bash

in container:

export LD_LIBRARY_PATH=`pwd`/linux-x86-64
java --enable-native-access=ALL-UNNAMED -cp "libs/*:app.jar" com.example.JnaExample

output:

Sum: 30
Concatenated String: Hello, World!

in container:

ldd --version

output:

ldd (Debian GLIBC 2.36-9+deb12u8) 2.36

Why, it keeps checking the GLIBC version?

Run openjdk:17

docker run -it --rm --platform linux/amd64 \
  -v $(pwd)/app:/app -w /app \
  openjdk:17 \
  bash

in container:

# ldd --version
ldd (GNU libc) 2.28

export LD_LIBRARY_PATH=`pwd`/linux-x86-64
java --enable-native-access=ALL-UNNAMED -cp "libs/*:app.jar" com.example.JnaExample

output error:

Exception in thread "main" java.lang.UnsatisfiedLinkError: Unable to load library 'mylib':
/lib64/libc.so.6: version `GLIBC_2.34' not found (required by /app/linux-x86-64/libmylib.so)
Native library (linux-x86-64/libmylib.so) not found in resource path (libs/jna-5.11.0.jar:app.jar)
at com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:301)
...
...
Suppressed: java.lang.UnsatisfiedLinkError: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by /app/linux-x86-64/libmylib.so)

That is to say, if your x86_64 target machine has an older GLIBC version, the same error may occur when JDK calls libxxx.so generated by GO.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

0

Thank you life888888 for such a detailed answer above, taking inspiration from it, I did something simpler for my use case to make it work. Instead of trying to create the binary in Dockerfile itself, I simply used the Dockerfile to create the environment I wanted matching the one I had on the host. This is defined in the BASE_IMAGE argument where it pull the image I needed (which had the right OS for linux and go installed). So this is simply how my Dockerfile looked:

FROM ${BASE_IMAGE}:${BASE_TAG} as base

WORKDIR /workspace

COPY go.mod go.mod
COPY go.sum go.sum

ADD . go_mylib
RUN go mod download

Then I build the container on my M1 Mac using the command:

$ docker build -t go_mylib:v1 --platform linux/amd64 .

Run the image using:

$ docker run -i -t --sysctl net.ipv6.conf.all.disable_ipv6=0 --platform linux/amd64 --name go_mylib-v1 go_mylib:v1 /bin/bash

Once inside the container's bash, I go into the project folder and then run the command to create a shared library:

# cd go_mylib/
# go build -o libmybinary.so -buildmode=c-shared main.go

And finally exiting the container, I copy the binary generated to my local folder using this command:

$ docker cp go_mylib-v1:/workspace/go_mylib/libmybinary.so .

(where /workspace/go_mylib/libmybinary.so is the path of the file inside my container and . refers to the current folder in my local system.

I finally load this library from my Java code by using:

MyLib INSTANCE = Native.load("mybinary", MyLib.class);

and it works on the host as expected.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.