Disclaimer: This tutorial has been tested on Python3.8.

If you are short on time please find below the minimal required code to turn log messages into correct JSON format in Python using the builtin logging module:

import logging
import json
class JSONFormatter(logging.Formatter):
	def __init__(self):
		super().__init__()
	def format(self, record):
		record.msg = json.dumps(record.msg)
		return super().format(record)
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
loggingStreamHandler = logging.StreamHandler()
# loggingStreamHandler = logging.FileHandler("test.json",mode='a') #to save to file
loggingStreamHandler.setFormatter(JSONFormatter())
logger.addHandler(loggingStreamHandler)
logger.info({"data":123})

Getting A Logger Object

If you are still reading I am going to assume that you are interested in learning about how the above works. First of all we creating a logger object with logger = logging.getLogger(__name__). If you are not familiar with the special variable __name__, this change depending on where you are running the script. For example if you are running the above script above directly the value will be “__main__”. However, if this is being run from a module then it will take the module’s name.

Setting The Log Level

You can set different log levels to the logger object. There are 5 log levels in the following Hierarchy:

  1. DEBUG
  2. INFO
  3. WARNING
  4. ERROR
  5. CRITICAL

It is a hierarchy because if you set the log level let’s say at INFO, then INFO, WARNING, ERROR and CRITICAL only work i.e DEBUG messages are ignored. If you are accessing a logger and you are not sure which log levels it can log you can always call logger.isEnabledFor(logging.INFO) to check if a particular level is enabled (of course replace logging.INFO with any other level that you want to check).

Anyway we set the log level to DEBUG in this example: logger.setLevel(logging.DEBUG)

Handlers

Handlers are used to serve messages to destinations. In this case, the StreamHandler is used to send messages (like you guessed) to streams. This includes the console. there are 14 handlers which come inbuilt you can find more information on them here: https://docs.python.org/3/howto/logging.html#useful-handlers

As an alternative you can change the handler to a FileHandler to save to disk.

Formatter

In the code above we create a class JSONFormatter which inherits from class logging.Formatter. Then we override the format function of the class to write our own format class. Here, we have made use of the json library to make sure we are converting dictionaries correctly to json. Feel free to use any other formatting that you like. The message passed in .info here is stored record.msg. If you would like to find all the attributes that are stored in record you can call record.__dict__ You should something like the following:

{'name': 'context_name', 'msg': '{"data": 123}', 'args': (), 'levelname': 'INFO', 'levelno': 20, 'pathname': 'minimal.py', 'filename': 'filename.py', 'module': 'minimal', 'exc_info': None, 'exc_text': None, 'stack_info': None, 'lineno': 16, 'funcName': '<module>', 'created': 1615752571.359734, 'msecs': 359.73405838012695, 'relativeCreated': 14.586210250854492, 'thread': 140735484547968, 'threadName': 'MainThread', 'processName': 'MainProcess', 'process': 6774}

This gives you a few useful information that you can make use of based on your needs. In this case we have only used msg.

There is a bit more to this than what I have covered and I would recommend you to check out the official documentation and inspect the code of the module. You could also comment below if you would like to see any other logging article.

Thanks for reading!