search by tags

for the user

adventures into the land of the command line

captains log, stardate 12345... how to log with python?

In a script

This is nice in place of using print statements, if you want.

import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

logger.info('Start getting some data')
# get data here
data = {'ice': 'cream', 'mars': 'bar'}
logger.debug('Data: %s', data)
logger.info('Updating data ...')
# update data here
logger.info('Finish updating data')

When you run it, you’ll get an output like this

$ python mygroovyscript.py
INFO:__main__:Start getting some data
INFO:__main__:Updating data ...
INFO:__main__:Finish updating data

If you change the log level to DEBUG

logging.basicConfig(level=logging.DEBUG)

and run it again

$ python mygroovyscript.py
INFO:__main__:Start getting some data
DEBUG:__main__:Data: {'ice': 'cream', 'mars': 'bar'}
INFO:__main__:Updating data ...
INFO:__main__:Finish updating data

In a script or app

This is better. Logs go into a file, you can control the format and you can write logging messages all over the place with different levels. You can change the logging level in one location to control what gets written to the file.

import logging
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)

# create a file handler
handler = logging.FileHandler('my_groovy_logfile.log')
handler.setLevel(logging.DEBUG)

# create a logging format
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)

# add the handlers to the logger
logger.addHandler(handler)

logger.info('Live long and prosper, baby')
logger.debug('Try and open some file which is not there')
try:
    open('/path/that/does/not/exist', 'rb')
except (SystemExit, KeyboardInterrupt):
    raise
except Exception, e:
    logger.error('Failed to open file', exc_info=True)

This method here called FileHandler, let’s you write to a file.

logging.FileHandler('my_groovy_logfile.log')

There is another one called RotatingFileHandler which will even handle log rotation for you.

logging.RotatingFileHandler('my_groovy_logfile.log', mode='a', maxBytes=0, backupCount=0)

As a developer this sounds like a nice thing, but as a system administrator, this is awful and I hate it. I prefer to let logrotate do its thing without having yet another application with it’s own log rotate mechanism step all over its toes.

Anyway after running a script or starting your app you’ll see stuff like this in the log file

$ python logs_baby.py
$ vim my_groove_logfile.log
2017-03-05 01:16:13,020 - __main__ - INFO - Live long and prosper, baby
2017-03-05 01:16:13,023 - __main__ - DEBUG - Try and open some file which is not there
2017-03-05 01:16:13,025 - __main__ - ERROR - Failed to open file
Traceback (most recent call last):
  File "logs_baby.py", line 21, in
    open('/path/to/does/not/exist', 'rb')
IOError: [Errno 2] No such file or directory: '/path/that/does/not/exist'

In a flask app

Copy pasta this into your flask app

import logging
log_path = '/var/log/something/'
log_file = '%serror.log' % log_path
log_formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
captains_log = logging.FileHandler(log_file, 'a')
captains_log.setLevel(logging.WARNING)
captains_log.setFormatter(log_formatter)

app = Flask(__name__)
app.logger.addHandler(captains_log)

if __name__ == '__main__':
    app.run()

I’ve seen examples where people also do it like this:

import logging

app = Flask(__name__)

if __name__ == '__main__':
  log_path = '/var/log/something/'
  log_file = '%serror.log' % log_path
  log_formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
  captains_log = logging.FileHandler(log_file, 'a')
  captains_log.setLevel(logging.WARNING)
  captains_log.setFormatter(log_formatter)
  app.logger.addHandler(captains_log)
    app.run()

I think both ways work, up to you I guess.