Using evaluate library to evaluate BertScore only uses 1 busy GPU

Question

I'm using evaluate library to evaluate the BertScore. Here are my codes:

import evaluate
bertscore = evaluate.load("bertscore")
bertscore_result = bertscore.compute(predictions=[sentence], references=references_sentence)
bertscore_avg = np.mean(bertscore_result["f1"])

print("BertScore:", bertscore_avg)

I am using GeForce 4090 GPU, which has two GPU units, each with 24GB RAM. Here is the output of the nvidia-smi:

Wed May 28 00:11:09 2025       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.183.01             Driver Version: 535.183.01   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 4090        Off | 00000000:81:00.0 Off |                  Off |
|  0%   31C    P8              21W / 450W |     14MiB / 24564MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
|   1  NVIDIA GeForce RTX 4090        Off | 00000000:C1:00.0 Off |                  Off |
| 30%   28C    P8              30W / 450W |  24004MiB / 24564MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
                                                                                         
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A      1720      G   /usr/lib/xorg/Xorg                            4MiB |
|    1   N/A  N/A      1720      G   /usr/lib/xorg/Xorg                           15MiB |
|    1   N/A  N/A      1896      G   /usr/bin/gnome-shell                         10MiB |
|    1   N/A  N/A    619637      C   /opt/tljh/user/bin/python                 23962MiB |
+---------------------------------------------------------------------------------------+

As you can see the GPU unit 1 is almost occupied, but the GPU unit 0 is idle. However, the code insists using GPU unit 1 and results in:

RuntimeError: CUDA error: out of memory

Of course I can kill other GPU-intensive process, but I would like to explore whether the evaluate library codes are smart enough to utlize the idle GPU unit.

Is this ever possible? Thanks in advance.

Raptor · Accepted Answer · 2025-10-08 06:01:23Z

0

To use the GPU with index 1 (the second GPU unit), we can use:

CUDA_VISIBLE_DEVICES=1 python evaluate.py

or

import os
os.environ['CUDA_VISIBLE_DEVICES'] = '1'

# evaluate codes here

answered Oct 8 at 6:01

Raptor

54.4k47 gold badges254 silver badges405 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Using evaluate library to evaluate BertScore only uses 1 busy GPU

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related