airflow error:AttributeError: module 'airflow.utils.log' has no attribute 'file_processor_handler'

Question

My local airflow instant was up and running, but now when I run airflow webserver or any other airflow command I got the below error (with some traceback):

  Unable to load the config, contains a configuration error.
  Traceback (most recent call last):
  File "/anaconda3/lib/python3.6/logging/config.py", line 382, in resolve
  found = getattr(found, frag)
  AttributeError: module 'airflow.utils.log' has no attribute 'file_processor_handler'

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
  File "/anaconda3/lib/python3.6/logging/config.py", line 384, in resolve
self.importer(used)
   File "/anaconda3/lib/python3.6/site-packages/airflow/utils/log/file_processor_handler.py", line 25, in <module>
from airflow.utils.helpers import parse_template_string
   File "/anaconda3/lib/python3.6/site-packages/airflow/utils/helpers.py", line 25, in <module>
import psutil
 File "/anaconda3/lib/python3.6/site-packages/psutil/__init__.py", line 134, in <module>
 from . import _psosx as _psplatform
  File "/anaconda3/lib/python3.6/site-packages/psutil/_psosx.py", line 14, in <module>
from . import _psutil_osx as cext
  ImportError: dlopen(/anaconda3/lib/python3.6/site-packages/psutil/_psutil_osx.cpython-36m-darwin.so, 2): Symbol not found: _kIOMasterPortDefault
  Referenced from: /anaconda3/lib/python3.6/site-packages/psutil/_psutil_osx.cpython-36m-darwin.so
  Expected in: flat namespace
 in /anaconda3/lib/python3.6/site-packages/psutil/_psutil_osx.cpython-36m-darwin.so

  The above exception was the direct cause of the following exception:

   Traceback (most recent call last):
   File "/anaconda3/lib/python3.6/logging/config.py", line 558, in configure
   handler = self.configure_handler(handlers[name])
 ValueError: Cannot resolve 'airflow.utils.log.file_processor_handler.FileProcessorHandler': dlopen(/anaconda3/lib/python3.6/site-packages/psutil/_psutil_osx.cpython-36m-darwin.so, 2): Symbol not found: _kIOMasterPortDefault
  Referenced from: /anaconda3/lib/python3.6/site-packages/psutil/_psutil_osx.cpython-36m-darwin.so
  Expected in: flat namespace
 in /anaconda3/lib/python3.6/site-packages/psutil/_psutil_osx.cpython-36m-darwin.so

   During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
 File "/anaconda3/bin/airflow", line 21, in <module>
from airflow import configuration
  File "/anaconda3/lib/python3.6/site-packages/airflow/__init__.py", line 36, in <module>
from airflow import settings
 File "/anaconda3/lib/python3.6/site-packages/airflow/settings.py", line 259, in <module>
configure_logging()
 File "/anaconda3/lib/python3.6/site-packages/airflow/logging_config.py", line 72, in configure_logging
raise e
 File "/anaconda3/lib/python3.6/site-packages/airflow/logging_config.py", line 67, in configure_logging
dictConfig(logging_config)
 File "/anaconda3/lib/python3.6/logging/config.py", line 795, in dictConfig
dictConfigClass(config).configure()

I tried to upgrade the airflow but that didn't help

It looks like an issue with your environment. It would help to know what version you are using and how you installed/configured Airflow. — Alessandro Cosentino
– Alessandro Cosentino, Commented Jan 24, 2019 at 8:01
@AlessandroCosentino I installed the final version by running pip install apache-airflow its 1.10.2 airflow version — MT467
– MT467, Commented Jan 24, 2019 at 16:38

Yong Wang · Accepted Answer · 2019-03-18 09:42:15Z

2

I just encountered the same issue as your mentioned.

I will facing the issue even I just import pandas lib.

The reason is I have some dag files name email.py. which is same as python internal lib name. the conflict lead to python stop work. After I rename the email.py to email_test.py. The problem solved.

You can refer to the similar question. It is same issue and same solution. ImportError : cannot import DAG airflow

If you like my answer , pls vote it.

answered Mar 18, 2019 at 9:42

Yong Wang

1,31311 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

ransy · Accepted Answer · 2019-04-10 11:59:05Z

0

See my comment in this PR https://github.com/apache/airflow/pull/3585

essentially pinning psutil to 4.4.2 will fix, as the latest version of psutil at 5.6.1 in this context fails for me on osx

answered Apr 10, 2019 at 11:59

ransy

111 bronze badge

1 Comment

ransy Over a year ago

ValueError: Cannot resolve 'airflow.utils.log.file_processor_handler.FileProcessorHandler': dlopen(/miniconda3/envs/mvp/lib/python3.6/site-packages/psutil/_psutil_osx.cpython-36m-darwin.so, 2): Symbol not found: _kIOMasterPortDefault

seems to be heart of the issue as its in the error handling of the first error that everything falls flat on its face.

jmuhlenkamp · Accepted Answer · 2023-04-06 17:48:46Z

0

I think the problem is that the install of pustil has a mix of old and new files.

The "Cannot resolve 'airflow.utils.log.file_processor_handler.FileProcessorHandler" part of the error message is just Airflow saying while trying to FileProcessorHandler we got this other error.

You should see the exact same error form running python -c "import psutil" which would entirely remove Airflow from the equation.

I think the fix is to totally uninstall psutil (if conda wasn't involved I'd say pip uninstall psutil) and then reinstall it. I think in this case I'd try a pip-uninstall and to uninstall it from however you install conda packages.

edited Apr 6, 2023 at 17:48

jmuhlenkamp

2,1901 gold badge20 silver badges39 bronze badges

answered Apr 10, 2019 at 16:13

Ash Berlin-Taylor

4,06834 silver badges35 bronze badges

Collectives™ on Stack Overflow

airflow error:AttributeError: module 'airflow.utils.log' has no attribute 'file_processor_handler'

3 Answers 3

Comments

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related