Python - Using logging for Debugging and Error tracking

Using Logging for Debugging and Error Tracking in Python

Debugging and error tracking are fundamental aspects of software development. While small-scale Python projects might get by with simple print() statements, professional-grade applications require a more robust and scalable method. Python provides a powerful and flexible logging system through its built-in logging module. This logging system is capable of tracking errors, debugging issues, monitoring runtime behavior, and facilitating long-term maintenance and auditing.

Why Use Logging Instead of Print?

Limitations of print()

• No severity levels: All messages look the same, whether it's critical or informational.
• No configurability: You can't easily turn logging off or redirect it to files or other systems.
• Lack of context: Messages lack timestamps, file names, or line numbers unless manually added.
• No log rotation: If using print, logs cannot be archived or rotated automatically.

Benefits of the Logging Module

• Supports multiple severity levels.
• Flexible output configurations: console, file, email, network, etc.
• Highly customizable message formatting.
• Can be extended with handlers, formatters, filters.
• Supports log rotation, persistence, and structured logging.

Logging Basics

Importing the Module

import logging

Simple Logging Example

import logging
logging.basicConfig(level=logging.INFO)
logging.info("Application started")

Output Explanation

The above command will print:

INFO:root:Application started

Here, "INFO" is the severity level, and "root" is the logger name.

Logging Levels

Logging levels define the severity of the message. They help filter out unnecessary information and focus on important events.

Setting Log Level

logging.basicConfig(level=logging.WARNING)

Only logs with level WARNING and above will be shown.

Customizing Log Messages

Using Format Strings

logging.basicConfig(format='%(levelname)s:%(message)s', level=logging.INFO)

Example output:

INFO:Application started

Advanced Formatting

logging.basicConfig(
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    level=logging.DEBUG
)

Output:

2025-06-23 12:00:00,123 - root - DEBUG - Application started

Logging to a File

logging.basicConfig(filename='app.log', level=logging.DEBUG)

Now all logs are saved to app.log  instead of being displayed in the console.

Appending vs Overwriting

filemode='a'  # Append (default)
filemode='w'  # Overwrite

Using Handlers

Handlers define where the logs go: file, console, or other outputs.

logger = logging.getLogger('my_logger')
logger.setLevel(logging.DEBUG)

console_handler = logging.StreamHandler()
file_handler = logging.FileHandler('my_app.log')

formatter = logging.Formatter('%(name)s - %(levelname)s - %(message)s')
console_handler.setFormatter(formatter)
file_handler.setFormatter(formatter)

logger.addHandler(console_handler)
logger.addHandler(file_handler)

logger.debug("Debug message")

Exception Logging

Logging Exceptions

try:
    1 / 0
except ZeroDivisionError:
    logging.exception("Exception occurred")

Equivalent to:

logging.error("Exception occurred", exc_info=True)

Output Example

ERROR:root:Exception occurred
Traceback (most recent call last):
  File "script.py", line 2, in <module>
    1 / 0
ZeroDivisionError: division by zero

Rotating Log Files

Using RotatingFileHandler

from logging.handlers import RotatingFileHandler

handler = RotatingFileHandler('app.log', maxBytes=2000, backupCount=3)
logger = logging.getLogger()
logger.addHandler(handler)

Creates new log files after 2 KB and keeps 3 old files.

Timed Rotating Log Files

from logging.handlers import TimedRotatingFileHandler

handler = TimedRotatingFileHandler('timed.log', when='midnight', interval=1, backupCount=7)

Creates a new log file at midnight and keeps 7 days of logs.

Custom Loggers

logger = logging.getLogger('my_app')
logger.setLevel(logging.INFO)
logger.info("Custom logger message")

Filters

class InfoFilter(logging.Filter):
    def filter(self, record):
        return record.levelno == logging.INFO

handler.addFilter(InfoFilter())

Structured Logging

Useful for log aggregation tools:

formatter = logging.Formatter(
    '{"time": "%(asctime)s", "level": "%(levelname)s", "message": "%(message)s"}'
)

Using Logging in Modules

Each module should have its own logger:

logger = logging.getLogger(__name__)

Logging in Classes

class Example:
    def __init__(self):
        self.logger = logging.getLogger(self.__class__.__name__)

    def run(self):
        self.logger.info("Running method")

Best Practices

• Use appropriate logging levels.
• Do not log sensitive data (e.g., passwords).
• Use exception logging in try-except blocks.
• Log to files in production, not just console.
• Set up log rotation to manage disk usage.
• Use different loggers for different modules or components.

Logging Configuration via Dictionary

import logging.config

LOGGING_CONFIG = {
    'version': 1,
    'formatters': {
        'simple': {
            'format': '%(asctime)s - %(levelname)s - %(message)s',
        },
    },
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
            'formatter': 'simple',
        },
    },
    'root': {
        'handlers': ['console'],
        'level': 'INFO',
    },
}

logging.config.dictConfig(LOGGING_CONFIG)
logger = logging.getLogger(__name__)
logger.info("Message with dict config")

Disabling Logging

logging.disable(logging.CRITICAL)

Disables all messages at CRITICAL level and below.

Integration with External Tools

• Fluentd: Centralized log aggregation.
• Logstash: Used in ELK stack.
• Sentry: For error tracking and alerting.

Using Logging in Flask and Django

Flask Example

from flask import Flask
import logging

app = Flask(__name__)
app.logger.setLevel(logging.INFO)
app.logger.info("Flask app started")

Django Logging Settings

LOGGING = {
    'version': 1,
    'handlers': {
        'file': {
            'level': 'DEBUG',
            'class': 'logging.FileHandler',
            'filename': 'django.log',
        },
    },
    'loggers': {
        'django': {
            'handlers': ['file'],
            'level': 'DEBUG',
            'propagate': True,
        },
    },
}

Effective logging is a cornerstone of robust software development. Python’s logging module offers a powerful toolkit for developers to monitor applications, diagnose issues, and track errors. By using structured, leveled, and flexible log configurations, developers can ensure smooth operations and quick issue resolution. Whether you are writing a small script or developing a large web application, incorporating proper logging practices from the beginning will make your code more maintainable, traceable, and professional.

Start using logging in all your Python applications—it is an essential skill for any serious Python developer.