Sagemaker custom model : "The model data archive is too large" error when creation endpoint

Question

Recently, while using a known-good endpoint configuration, I keep on receiving a: "The model data archive is too large. Please reduce the size of the model data archive or move to an instance type with more memory." error. The tar.gz file is 7.7G but is not loaded in memory (only a small part of it is). I am wondering if anything changes recently that may be causing this issue.

Thanks for any insights

Emmanuel

Arun Nagarajan · Accepted Answer · 2019-06-27 12:48:06Z

1

When an Endpoint is created with SageMaker, the model data artifact is downloaded and uncompressed onto the associated disk/EBS volume on the instance. This volume size is proportional[1] to the instance type that you chose.

Please make sure that the instance type that you picked has enough disk space to accommodate the uncompressed .tar.gz file. (It does not matter if it is fully or partially loaded onto memory later on, it has to fit in the disk uncompressed).

[1] Volume size for instance types - https://docs.aws.amazon.com/sagemaker/latest/dg/host-instance-storage.html

answered Jun 27, 2019 at 12:48

Arun Nagarajan

762 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Bhavya Over a year ago

Is there any way to deploy models with huge size(around 60gb compressed)? Or should I go for multi-model endpoint in that case?

Collectives™ on Stack Overflow

Sagemaker custom model : "The model data archive is too large" error when creation endpoint

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related