Logging in Python: A Comprehensive Guide

Introduction to Logging

Logging is a crucial aspect of software development that helps developers track events, debug issues, and monitor application behavior. Instead of using print() statements for debugging, Python's logging module provides a structured, efficient, and flexible way to record messages.

Why Use Logging Instead of print()?

1. Severity Levels: Logs allow categorizing messages into different levels (INFO, WARNING, ERROR, etc.).

2. Better Debugging: Logs persist even after execution, unlike print(), which disappears.

3. Custom Formatting: Logging provides timestamps, log levels, and message customization.

4. Multiple Output Streams: Logs can be stored in files, databases, or external monitoring services.

Basic Logging in Python

Python provides a built-in logging module that allows different log levels:

1. Importing the logging Module

import logging

2. Setting Up Basic Logging Configuration

logging.basicConfig(level=logging.INFO)  # Set log level to INFO
logging.info("This is an INFO message")  # Output: INFO:root:This is an INFO message

3. Logging Different Levels

Python defines multiple logging levels:

logging.debug("Debugging details")    # Used for debugging information
logging.info("General information")   # Used for informational messages
logging.warning("Something looks off") # Used for warnings
logging.error("An error occurred")    # Used for error reporting
logging.critical("System failure!")   # Used for serious failures

4. Custom Log Formatting

Customizing logs with timestamp, level, and message format:

logging.basicConfig(
    format="%(asctime)s - %(levelname)s - %(message)s",
    level=logging.INFO
)
logging.info("Custom formatted log message")

Example Output:

2025-05-05 12:45:30 - INFO - Custom formatted log message

Writing Logs to a File

Instead of displaying logs in the console, logs can be stored in a file:

logging.basicConfig(
    filename="app.log",
    format="%(asctime)s - %(levelname)s - %(message)s",
    level=logging.INFO
)

logging.info("This log is saved in a file!")

The log will be written to app.log.

Using Loggers for Modular Logging

Creating a Custom Logger

logger = logging.getLogger("custom_logger")  # Create a named logger
logger.setLevel(logging.DEBUG)

# Create a handler for logging to a file
file_handler = logging.FileHandler("custom.log")
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")

file_handler.setFormatter(formatter)
logger.addHandler(file_handler)

logger.debug("This debug log is recorded in custom.log")

This method ensures modular logging across different parts of a program.

Using Logging in a Real-World Application

Example: API Request Logging

import logging
import requests

logging.basicConfig(format="%(asctime)s - %(levelname)s - %(message)s", level=logging.INFO)

def fetch_data(url):
    logging.info(f"Fetching data from {url}")
    try:
        response = requests.get(url)
        response.raise_for_status()  # Raise error for failed requests
        logging.info(f"Received response: {response.status_code}")
        return response.json()
    except requests.exceptions.RequestException as e:
        logging.error(f"Request failed: {e}")

data = fetch_data("https://api.github.com")

Example Output

2025-05-05 12:50:15 - INFO - Fetching data from https://api.github.com
2025-05-05 12:50:15 - INFO - Received response: 200