0

I have written a linux service in Go that processes a large number of image files, converting them from TIFF to JPEG and adding a text overlay on top. I'm using the Go package gographics/imagick/v3 based in CGO to do this. The machine I run this on has 64 logical cores, 96GB of RAM and I run Rocky Linux 9.5 with REMI's ImageMagick7 packages installed.

In my code I launch a worker pool of 50 goroutines, each running the func I use to convert the images. I initialize imagick globally at the top of the main func with a deferred imagick.Terminate(). In the image convert func I establish a new magickwand, drawingwand and pixelwand. If at any point there is an issue with an image, the wands are cleared and the goroutine loops back to check the channel for another job. After all images are completed, the goroutine exits and all the wands are destroyed.

Since this is a service, it never exits, but after it completes a processing job, I see in htop that there are 70-80 threads in an idle state owned by the service PID. If I process multiple jobs, that number of idle threads never gets any larger, but it never goes away either. I'm assuming these are helper threads established by the ImageMagick Wand library? If I stop the service, the threads are released and when I start my service there are only 6 threads until something is processed.

I have disabled any OpenMP, MKL, OpenBlas settings I can find via environment variables and setting whatever resource limits in the imagick library I can. I have each goroutine use runtime.LockOSThread to set affinity of the imagick function in each goroutine, but nothing changes. The leftover threads remain.

If I look at the threads in gdb or by cat-ing the status in /proc/PID/task/TID/status I'm shown they are in futex and nothing more than that. The threads aren't taking up any CPU resources and other than whatever RAM a native thread consumes, they're not using that up much either.

Has anyone else ever had experience with Go and Imagick and seen something like this? Have I stumbled on a bug or is this just normal for Go an Imagick when performing the kind of processing I'm doing in a long running app like a service?

Thanks

Improve this question
1
  • Yes, the Go runtime does not reclaim threads, so this is all expected. If you need to reduce the thread count, then you need to just not spawn as many threads. Commented Mar 5 at 13:18

1 Answer 1

Reset to default
0

Go tries to maintain a certain number of threads running Go code (customizable with the GOMAXPROCS environment variable). If there are fewer than that many threads running and no more threads available on which to schedule a goroutine, a new thread may be created. Threads which are not running Go code—including those blocked on system calls as well as those currently in a cgo call—do not count towards this limit, meaning that a new thread may be created even when GOMAXPROCS or more threads already exist.

Once the scheduler creates a thread, it never exits. There has been discussion on attempting to cull "superfluous" (for a not-so-clear definition of the word) threads, but as of 1.24, there is no attempt to do so. Making a cgo call—which ties a thread to a goroutine for the duration of the call—does require some additional care in this regard, since the runtime can't scale blocking calls in arbitrarily many goroutines to a small number of threads (like it can with, say, blocking I/O compatible with epoll/kqueue/overlapped I/O/etc.). I'd consider whether this actually represents an issue, however.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.