ctypes + cgo and ownership of strings

Question

Background

I am currently using a DLL I generated with golang in python. As a sanity check for debugging I added a function called return_string(), which takes a pointer to a string in C (char*) converts to go, then returns the result. Mostly this is a debugging tool to look for encoding issues between the two languages.

Specific issue

In python I've wrapped the function, and everything works, except if I free after copying the bytes in. Here's the function:

# import library
if platform().lower().startswith("windows"):
    lib = cdll.LoadLibrary(os.path.join(os.path.dirname(os.path.realpath(__file__)), "lib.dll"))
else:
    lib = cdll.LoadLibrary(os.path.join(os.path.dirname(os.path.realpath(__file__)), "lib.so")) 

lib.return_string.argtypes = [c_char_p]
lib.return_string.restype = c_char_p

lib.FreeCString.argtypes = [c_char_p]

def prepare_string(data: str | bytes) -> c_char_p:
    """Takes in a string and returns a C-compatible string"""
    if isinstance(data, str):
        return c_char_p(data.encode())
    return c_char_p(bytes(data))

def return_string(text: str | bytes) -> str:
    """Debugging function that shows you the Go representation of a C string and returns the python string version"""
    c_input = prepare_string(text)
    result = lib.return_string(c_input) # This is allocated in go using essentially malloc

    if not result:
        return ""

    copied_bytes = string_at(result) # This should be a COPY into new python-managed memory afaik
    decoded = copied_bytes.decode(errors="replace") # Gets to here no issues

    lib.FreeCString(result) # Program silently fails here
    return copied_bytes

If I remove the call to lib.FreeCString(result), everything works, but my understanding is that because the result variable is allocated in go with a malloc() call, it needs to be freed in python. But, when I free it, it's either double-freeing or free-after-use and I'm not sure which, or why.

Do I need to free result? and if not, where does the pointer allocated in go get cleaned up?

Go code

I don't think it's a go-side issue, but just in case, here's the go code as well:

// All code returns `unsafe.Pointer` because this is a package, and CGo breaks your types if you don't

// Used to convert a C-compatible string back to itself, good for debugging encoding issues
//
// Parameters:
//   - cString: Pointer to the C string (*C.char).
//
// Returns:
//   - Pointer to a new C string with the same content (*C.char).
//     Note: The caller is responsible for freeing the allocated memory using FreeCString.
//
//export return_string
func return_string(cString unsafe.Pointer) unsafe.Pointer {

    internalRepresentation := C.GoString((*C.char)(cString))
    result := StringToCString(internalRepresentation)
    return result
}

// Convert a string to a c-compatible C-string (glorified alias for C.CString)
//
// Parameters:
//   - input: The Go string to convert.
//
// Returns:
//   - A pointer to the newly allocated C string (*C.char).
//     Note: The caller is responsible for freeing the allocated memory using FreeCString.
func StringToCString(input string) unsafe.Pointer {
    return unsafe.Pointer(C.CString(input))
}

// Free a previously allocated C string from Go.
//
// Parameters:
//   - ptr: Pointer to the C string to be freed (*C.char).
//
//export FreeCString
func FreeCString(ptr unsafe.Pointer) {
    fmt.Println("FreeCString(): freeing", ptr)
    if ptr != nil {
        C.free(ptr)
    }
}

kostix · Accepted Answer · 2025-05-16 13:44:47Z

The cgo manual explicitly states about C.GoString:

A few special functions convert between Go and C types by making copies of the data. In pseudo-Go definitions:
<…>
// C string to Go string
func C.GoString(*C.char) string

(Emphasis mine.)

This basically means, that once you've called C.GoString on a string, the Go code asked the memory manager provided by the Go runtime powering that code to allocate a new string, and then copied over there the contents of the original string. As a corollary, the produced string is owned by the Go runtime and must be freed by it.

Note how this function is different from Go.CString which is documented in the same place:

// Go string to C string
// The C string is allocated in the C heap using malloc.
// It is the caller's responsibility to arrange for it to be
// freed, such as by calling C.free (be sure to include stdlib.h
// if C.free is needed).
func C.CString(string) *C.char

This function does the reverse: it takes a Go-managed string, allocates new memory chunk by calling libc and then copies the bytes there. Such a string must be deallocated by a matching call to libc.

Maybe you've been tripped by the idea that C.GoString "coerces" the source string to "look" like Go's (I'm judging by the name internalRepresentation you've picked), but that's simply not true. You can do that in Go via certain dances involving unsafe but it appears you do not need that in your code.

I would add that Go is a reasonably safe language which means no code in its standard library — other than that available via the package unsafe — provides for "tricks" such as reinterpreting memory or pointers to it. Compare the functions available via the C pseudo package to, say, unsafe.Slice which makes it possible to produce a slice out of an arbitrary pointer.

Collectives™ on Stack Overflow

ctypes + cgo and ownership of strings

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related