XGBoost model: train on GPU, run on CPU without GPU RAM allocation

Question

How can I train an XGBoost model on a GPU but run predictions on CPU without allocating any GPU RAM?

My situation: I create an XGBoot model (tree_method='gpu_hist') in Python with predictor='cpu_predictor', then I train it on GPU, then I save (pickle) it to disk, then I read the model from disk, then I use it for predictions.

My problem: once the model starts doing predictions, even though I run it on CPU, it still allocates some small amount of GPU RAM (around ~289MB). This is a problem for the following reasons:

I run multiple copies of the model to parallelize predictions and if I run too many, the prediction processes crash.
I can not use GPU for training other models, if I run predictions on the same machine at the same time.

So, how can one tell XGBoost to not allocate any GPU RAM and use CPU and regular RAM only for predictions?

Thank you very much for your help!

stumbling upon this same issue. did you end up solving it?

Pedro
– Pedro

2023-11-21 17:18:10 +00:00
Commented Nov 21, 2023 at 17:18 — Pedro
– Pedro, Commented Nov 21, 2023 at 17:18

grandrew · Accepted Answer · 2024-03-28 02:10:42Z

1

This is probably due to the model being saved as pickle

When I loaded and re-saved the model in either JSON or BIN formats, the problem was gone.

answered Mar 28, 2024 at 2:10

grandrew

7387 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

XGBoost model: train on GPU, run on CPU without GPU RAM allocation

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related