Catching details of exception in Python

I encountered something similar today when running python script for an LDA model using the Gensim package in Jupyter Lab.

I entered the print statements to see if there is anything wrong with the for loop code, which works fine outside a loop.

Jupyter Lab Input:

limit = 30
start=2
step=2

coherence_values = []

for num_topics in range(start, limit, step):
    lda_model = gensim.models.ldamodel.LdaModel(corpus=corpus,
                                       id2word=id2word,
                                       num_topics=num_topics,
                                       random_state=100,
                                       update_every=1,
                                       chunksize=len(corpus),
                                       passes=10,
                                       alpha='auto',
                                       per_word_topics=True)
    print(f'LDA model {num_topics} created')
    coherence_model_lda = CoherenceModel(model=lda_model, texts=data_lemmatized, dictionary=id2word, coherence='c_v')
    print(f'Coherence model {num_topics} created')
    coherence_lda = coherence_model_lda.get_coherence()
    print(f'Coherence value {num_topics} obtained')
    coherence_values.append(coherence_lda)

Jupyter Lab Output:

LDA model 2 created
Coherence model 2 created
Coherence value 2 obtained
LDA model 4 created
Coherence model 4 created
Coherence value 4 obtained
LDA model 6 created
Coherence model 6 created

No python errors occur -- the code cell simply keeps running without error, but in my terminal I observe 11 counts of the following error.

Terminal error:

libc++abi.dylib: terminating with uncaught exception of type 
std::runtime_error: Couldn't close file

It seems the code runs fine, and that the error occurs after 3 of the 14 loops are run. Sometimes one loop runs, sometimes two.

A simple solution is to set parameter processes of class CoherenceModel to a specific num (maybe 1 for safe). The source code of CoherenceModel about processes is like below, you can see by default the caculation is carried out with multiprocesses, the num of processes is decided by the num of cpus:

"""     
processes : int, optional
       Number of processes to use for probability estimation phase, any value less than 1 will be interpreted as num_cpus - 1.
"""
 self.processes = processes if processes >= 1 else max(1, mp.cpu_count() - 1)

I use macOS Monterey and python 3.7.

I have encountered the same issue too since upgrading to macOS Catalina (same error thrown); and unfortunately for me, the only sensible and reliable solution as I see it now is to rewrite the code to avoid using multiprocessing at all, then the issue goes away. So any thing having to do with multiprocessing in Python 3.7 as described here has to go...