Logging in Python : A Gentle Introduction

Debugging is one of the most important parts of writing code. Good debugging habits can spare us a lot of pain. In this article we look at logging, which indeed, is part of a healthy debugging hygiene. This article starts from the very basics of logging and is a great place to learn logging for the first time. For a more involved introduction, please refer to my other article.

Writing print statements all over the program (formally, printf debugging) is the most intuitive form of debugging a program. In fact, the first line of code all of us have ever written is an example of printf debugging. Every Hello World program is basically making sure that the piece of code we’re writing compiles and runs properly by displaying an output.

An example of logging in python

Logging, in my words, is a formalization of printf debugging which most of us use. But logging becomes especially useful as the size of our codes increase. How I like to encourage people who’re used to writing print statements while debugging is to just ask them replace the print statements with logging commands. So without any further ado, let’s look at how to do logging in python.

Hello Logging World

To begin with, let’s write a hello world version of logging.

The output of this python code looks like this :

WARNING:root:Hello Logging World

This code print Hello World on the console along with a few other things (which we’ll ignore for now). We begin by importing the logging module. Then we print Hello World using the warning method in the logging module. The obvious question here is, why do we use a function called warning to print something. Shouldn’t we be using a function that has the word print in its name?

This is because when we’re logging, we’re displaying/printing/writing things based on certain severity levels. There are 5 severity levels pre-defined in python, these are

• debug (10)
• info (20)
• warning (30)
• error (40)
• critical (50)

The severity levels are also referred by the integers given in the brackets. To understand severity levels a little better, let’s try and run the next code snippet.

The output of this code is

WARNING:root:Hello World
ERROR:root:Hello World
CRITICAL:root:Hello World

We find that only messages corresponding to warning, error and critical methods are displayed. The messages for the info and debug methods have been ignored. This is because the default severity level for a python program is set to 30. This means all the messages with severity level greater than or equal to 30 will be displayed, and the messages below that severity level are ignored.

This shows one of the many advantages of using logging over printf debugging. Once we’re done with debugging, we can just increase the severity level of the program to stop displaying messages rather than commenting out individual print statements.

Changing Severity Levels

To change severity levels in python, we use the basicConfig method of the logging module. The following code accomplishes this task.

The output of the above code is

DEBUG:root:Hello World
INFO:root:Hello World
WARNING:root:Hello World
ERROR:root:Hello World
CRITICAL:root:Hello World

All the logging statements are printed now because the logging level is set to logging.DEBUG, which is the lowest logging severity level. All the logging levels greater than or equal to logging.DEBUG will thus be printed.

It’s important to note that logging.debug and logging.DEBUG are two different things. logging.debug() is a function in the logging module wheres logging.DEBUG is a constant. For a sanity check, let’s print the different log levels in the logging module.

The output is

Debug : 10
Info : 20
Warning : 30
Error : 40
Critical : 50

Logging into a File

As our codebase becomes large, it is just not possible to keep track of all the logs that are displayed on the console. Also, if you write programs that run for days in and days out, one can’t assign a person to keep an eye on the console 24*7 for the off chance that a log message shows up on the console. Thus it is important, especially for large codebases, to write logs separately into a file.

To do this, we again use the basicConfig method in the logging module.

The output of the code is then written into a file. The filemode follows the same conventions as when writing into a file. When filemode = ‘w’, a new file is created every time the program is run. Thus, logs from previous runs are overwritten. In case you want to keep the previous logs, use the append mode (filemode = ‘a’), which appends the logs to the end of the file if a file is present. It also creates a new file if a file with that name is not present.

Formatting Messages

In this section, we discuss a few useful ways of displaying logs. One of the most useful way is to just use logs as a print statement.

Output:

ERROR:root: My name is Akshat and my purpose is to introduce you to logging

The logging.info message is not displayed because the logging level is set higher than info. This clearly shows us that logging can easily replace a print statements in your daily debugging practice.

One another useful way to format logs is shown in the code given below.

The output of the above code is

06/26/2020 04:28:38 AM :: CRITICAL : This is the end of the article.

Remember that variables defined in capital letters by convention are constants in python. We first define the message format for logging. The message shows the date and time, the severity level followed by the message. This format, in my opinion, should suffice most of the logging needs you might have. After that, we define the date format which is also sent as an argument to the basicConfig method. An exhaustive list of all possible message formats can be found here.

Hopefully after reading this article you’ve got a much better grasp of logging in python. I would now encourage you to replace all the print statements you use to debugging your code to logging. This will be your first step towards building a healthy debugging hygiene.

So what’s next? As your codebase increases, you’ll need more a few more pieces to implement efficient logging. My article here introduces logging for large codebases with multiple files.

In case there are any comments, suggestions or questions, feel free to contact me through my linkedin.