ConnectException: Connection refused (Connection refused) Error while obtaining a new communication channel. error in databricks notebook

Question

I am facing below error message while running the python code(ML model) in the python databricks notebook

ConnectException: Connection refused (Connection refused) Error while obtaining a new communication channel

ConnectException error: This is often caused by an OOM error that causes the connection to the Python REPL to be closed. Check your query's memory usage. Spark tip settings

Pratik Lad · Accepted Answer · 2023-01-30 12:34:43Z

1

The driver may be experiencing a memory bottleneck, which is a frequent cause of this issue. When this occurs, the driver has an out of memory (OOM) crash, restarts often, or loses responsiveness. Any of the following factors might be the memory bottleneck's cause:

For the load placed on the driver, the driver instance type is not ideal.
Memory-intensive procedures are carried out on the driver.
The same cluster is hosting a large number of concurrent notebooks or processes.

Please try below options

Try increasing driver-side memory and then retry.
You can look at the spark job dag which give you more info on data flow.

answered Jan 30, 2023 at 12:34

Pratik Lad

8,7282 gold badges5 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

ConnectException: Connection refused (Connection refused) Error while obtaining a new communication channel. error in databricks notebook

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related