Python - logging

• Introduction
• logging
• Log levels
• Log format
• Custom loggers
• Pending
• Tasks

Introduction

Logging is essential for building dependable software. It records software events, creating an audit trail that details various system operations. This trail helps diagnose issues, understand runtime behavior, and analyze user interactions, offering insights into design decisions.

logging

Python’s built-in logging module is incredibly powerful and boasts the necessary logging features that one might require from a separate logging framework. Therefore, it remains the most popular way to log in the Python ecosystem despite the existence of a few third-party logging frameworks like Loguru.

Providing a logging module in the standard library ensures that all Python programs can benefit from having consistent logging features, making it easier to adopt and understand when working on different projects.

To get started with the logging module, you need to import it to your program first, as shown below:

import logging

logging.debug("A debug message")
logging.info("An info message")
logging.warning("A warning message")
logging.error("An error message")
logging.critical("A critical message")

When you execute the above program, you should observe the following output:

WARNING:root:A warning message
ERROR:root:An error message
CRITICAL:root:A critical message

Notice that the debug() and info() messages are missing from the output while the others are present. This is due to the default log level configured on the logging module, which we will get to shortly. For now, understand that this is the intended behavior of the example.

In each record above, you will observe that the log level (WARNING, ERROR, etc.) is printed in all caps followed by root, which is the name of the default logger. Finally, you have the log message that was passed as an argument to the function.

This output format is summarized below:

<LEVEL>:<name>:<message>

Log levels

Log levels define the severity of the event that is being logged. They convey implicit meaning about the program state when the message was recorded, which is crucial when sieving through large logs for specific events.

For example, a message logged at the INFO level indicates a normal and expected event, while one that is logged at the ERROR level signifies that some unexpected error has occurred.

The available log levels in the logging module are listed below in increasing order of severity:

Here’s the table without the numeric values:

It’s important to always use the most appropriate log level so that you can quickly find the information you need.

For instance, logging a message at the WARNING level will help you find potential problems that need to be investigated, while logging at the ERROR or CRITICAL level helps you discover problems that need to be rectified immediately.

By default, the logging module will only produce records for events that have been logged at a severity level of WARNING and above.

This is why the debug() and info() messages were omitted in the previous example since they are less severe than WARNING.

You can change this behavior using the logging.basicConfig() method as demonstrated below:

import logging

logging.basicConfig(level=logging.INFO)

logging.debug("A debug message")
logging.info("An info message")
logging.warning("Something bad is going to happen")
logging.error("An error message")
logging.critical("A critical message")

When you execute the program now, you will observe the output below:

INFO:root:An info message
WARNING:root:A warning message
ERROR:root:An error message
CRITICAL:root:A critical message

Since we’ve set the minimum level to INFO, we are now able to view the log message produced by logging.info() in addition levels with greater severity while the DEBUG message remains suppressed.

Setting a default log level in the manner shown above is useful for controlling the number of logs that are generated by your application.

For example, during development you could set it to DEBUG to see all the log messages produced by the application, while production environments could use INFO or WARNING.

Log format

Beyond the log message itself, it is essential to include as much diagnostic information about the context surrounding each event being logged. Standardizing the structure of your log entries makes it much easier to parse them and extract only the necessary details.

Generally, a good log entry should include at least the following properties:

These properties allow for basic filtering by log management tools so you must ensure they are present in all your Python log entries. At the moment, the timestamp is missing from our examples so far which prevents us from knowing when each log entry was created.

Let’s fix this by changing the log format through the logging.basicConfig() method as shown below:

import logging

logging.basicConfig(format="%(asctime)s | %(levelname)s | %(message)s")

logging.warning("Something bad is going to happen")

The format argument above configures how log messages are displayed.

It uses the following LogRecord attributes for formatting the logs:

With the above configuration in place, all log entries produced by the logging module will now be formatted in following manner:

2023-02-05 11:41:31,862 | WARNING | Something bad is going to happen

Notice that the root name no longer appears in the log entry because the log format has been redefined to exclude it. The fields are also now separated by spaces and the | character to make them easier to read.

After the log level, you have the log creation time which can be customized further using the datefmt argument to basicConfig():

import logging

logging.basicConfig(
    format="%(levelname)s | %(asctime)s | %(message)s",
    datefmt="%H:%M:%S,%f",

)

logging.warning("Something bad is going to happen")

WARNING | 2025-11-27T08:52:58Z | Something bad is going to happen

The datefmt argument accepts the same directives as the time.strftime() method.

The ISO-8601 format shown above it is widely recognized and supported, which makes it easy to process and sort timestamps consistently.

Please refer to the LogRecord documentation to examine all the available attributes that you can use to format your log entries.

Here’s another example that uses a few more of these attributes to customize the log format:

import logging

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s | %(levelname)s | %(filename)s:%(lineno)s | %(message)s",
    datefmt="%Y-%m-%d %H:%M:%S"
)

logging.debug("Database query executed in 0.234ms: SELECT * FROM users WHERE id=42")
logging.info("User authentication successful: user_id=12345, username='john_doe'")
logging.warning("System disk is full")
logging.error("Failed to connect to database: Connection timeout")

2025-11-27 09:10:31 | INFO | main.py:10 | User authentication successful: user_id=12345, username='john_doe'
2025-11-27 09:10:31 | WARNING | main.py:11 | System disk is full
2025-11-27 09:10:31 | ERROR | main.py:12 | Failed to connect to database: Connection timeout

Custom loggers

So far, we’ve used the default logger (named root) to write logs in the examples.

The default logger is accessible and configurable through module-level methods on the logging module (such as logging.info(), logging.basicConfig()).

In general, it’s best to avoid using the root logger and instead create a separate logger for each module in your application. This allows you to easily control the logging level and configuration for each module, and provide better context for log messages.

Creating a new custom logger can be achieved by calling the getLogger() method as shown below:

import logging

logger = logging.getLogger("toto")

logger.info("An info")
logger.warning("A warning")

The getLogger() method returns a Logger object whose name is set to the specified name argument (toto in this case). A Logger provides all the functions for you to log messages in your application. Unlike the root logger, its name is not included in the log output by default. In fact, we don’t even get the log level; only the log message is present. However, it defaults to logging to the standard error just like the root logger.

To change the output and behavior of a custom logger, you have to use the Formatter and Handler classes provided by the logging module.

The Formatter does what you’d expect; it helps with formatting the output of the logs, while the Handler specifies the log destination which could be the console, a file, an HTTP endpoint, and more.

We also have Filter objects which provide sophisticated filtering capabilities for your Loggers and Handlers.

Pending

A Comprehensive Guide to Logging in Python

Tasks

import logging

logging.debug("Debug message")
logging.info("Info message")
logging.warning("Warning message")
logging.error("Error message")
logging.critical("Critical message")

import logging

# Show INFO and above
logging.basicConfig(level=logging.INFO)

logging.debug("Debug")      # hidden
logging.info("Info")        # shown
logging.warning("Warning")  # shown
logging.error("Error")      # shown
logging.critical("Critical")# shown

# To show only ERROR and above, run a fresh process with:
# logging.basicConfig(level=logging.ERROR)

import logging

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s | %(levelname)s | %(message)s",
)

logging.info("Application started")
logging.warning("Low disk space")
logging.error("Failed to open file")

number = int(input("Enter a number: "))
result = 100 / number
print("Result is:", result)

import logging

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s | %(levelname)s | %(message)s",
)

logging.info("Program started")

try:
    number = int(input("Enter a number: "))
    logging.info("User entered number=%s", number)

    result = 100 / number
    logging.info("Computation successful, result=%s", result)
except ZeroDivisionError:
    logging.error("Division by zero is not allowed")
except ValueError:
    logging.error("Input was not a valid integer")

import logging

logging.basicConfig(
    filename="app.log",
    filemode="a",  # change to "w" to overwrite each run
    level=logging.DEBUG,
    format="%(asctime)s - %(levelname)s - %(message)s",
)

logging.debug("Application starting")
logging.info("User logged in")
logging.warning("Password attempt failed")
logging.error("Database not reachable")

# main.py
import logging
import utils

logger = logging.getLogger("main")
logger.setLevel(logging.INFO)

handler = logging.StreamHandler()
handler.setLevel(logging.INFO)
formatter = logging.Formatter("%(name)s - %(levelname)s - %(message)s")
handler.setFormatter(formatter)

logger.addHandler(handler)

logger.info("Starting main script")
utils.do_work()

# utils.py
import logging

logger = logging.getLogger("utils")

def do_work() -> None:
    logger.debug("utils: starting helper function")
    logger.info("utils: helper completed")

import logging

logger = logging.getLogger("shop")
logger.setLevel(logging.DEBUG)

console_handler = logging.StreamHandler()
console_handler.setLevel(logging.INFO)
console_fmt = logging.Formatter("%(levelname)s - %(message)s")
console_handler.setFormatter(console_fmt)

file_handler = logging.FileHandler("shop.log", mode="a")
file_handler.setLevel(logging.DEBUG)
file_fmt = logging.Formatter(
    "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
)
file_handler.setFormatter(file_fmt)

logger.addHandler(console_handler)
logger.addHandler(file_handler)

logger.debug("Cart total recalculated")
logger.info("User checked out")
logger.warning("Payment took too long")

import logging

logging.basicConfig(
    level=logging.INFO,
    format="%(levelname)s | %(message)s",
)

filename = input("File to open: ")

try:
    with open(filename, "r", encoding="utf-8") as f:
        content = f.read()
        logging.info("Successfully opened file %s", filename)
        print(content)
except Exception:
    # Logs ERROR level plus full traceback
    logging.exception("Failed to open file %s", filename)