How to split Python logging file by file size

Question

I am using Django and have included Python's default logging library. I have following configuration for logging (in settings.py):

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'verbose': {
            'format' : "[%(asctime)s %(name)s-%(levelname)s (%(filename)s:%(lineno)s %(funcName)s)]: %(message)s",
            'datefmt' : "%d/%b/%Y %H:%M:%S"
        },
        'simple': {
            'format': '[%(asctime)s %(name)s-%(levelname)s]: %(message)s',
            'datefmt' : "%d/%b/%Y %H:%M:%S"
        },
    },
    'handlers': {
        'file-django': {
            'level': 'DEBUG',
            'class': 'logging.FileHandler',
            'filename':  os.path.join(PROJECT_LOGS, 'django.log'),
            'formatter': 'simple'
        },
        'file-application': {
            'level': 'DEBUG',
            'class': 'logging.FileHandler',
            'filename':  os.path.join(PROJECT_LOGS, 'application.log'),
            'formatter': 'simple'
        },
        'file-core': {
            'level': 'DEBUG',
            'class': 'logging.FileHandler',
            'filename':  os.path.join(PROJECT_LOGS, 'core.log'),
            'formatter': 'verbose'
        },
    },
    'loggers': {
        'django': {
            'handlers':['file-django'],
            'propagate': True,
            'level':'DEBUG',
        },
        'application': {
            'handlers': ['file-application'],
            'propagate': True,
            'level': 'DEBUG',
        },
        'core': {
            'handlers': ['file-core'],
            'propagate': True,
            'level': 'DEBUG',
        },
    }
}

I am trying to split result logging files using only configuration. I "googled" and found few solutions but all of them are code based.

your request is not clear, are you looking for a some tool to parse the logs? what for? are you looking for a log viewer? — Guy Gavriely
– Guy Gavriely, Commented Jan 26, 2014 at 22:18
I am looking for "how to configure python logging to log into file and if file size for example equals to 100MB split it into another file". — KenanBek
– KenanBek, Commented Jan 27, 2014 at 6:00

Jingchao Luan · Accepted Answer · 2018-04-14 17:36:42Z

3

You can use the logging.handlers.RotatingFileHandler in your configuration file setting.py, take your handler 'file-django' as example and assume you split it into a new file when its size over than 100MB:

'file-django': {
    'level': 'DEBUG',
    'class': 'logging.handlers.RotatingFileHandler',
    'filename':  os.path.join(PROJECT_LOGS, 'django.log'),
    'maxBytes': 104857600, # 1024*1024*100B (100MB)
    'backupCount': 10, # keep at most 10 log files
    'formatter': 'simple'
},

Reference: https://xxx-cook-book.gitbooks.io/django-cook-book/Logs/Handlers/FileHandler/rotating-file-handler.html

answered Apr 14, 2018 at 17:36

Jingchao Luan

4514 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Vinay Sajip · Accepted Answer · 2014-01-27 08:35:49Z

1

Use Ignacio's comment and look up how to use RotatingFileHandler; it splits by file size and you can configure it declaratively in Django via the LOGGING dictionary.

answered Jan 27, 2014 at 8:35

Vinay Sajip

100k15 gold badges184 silver badges196 bronze badges

Collectives™ on Stack Overflow

How to split Python logging file by file size

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related