Python’s structlog: Modern Structured Logging for Clean, JSON-Ready Logs

Introduction

Logging is one of the most overlooked yet critical aspects of building reliable and scalable applications. In traditional Python applications, developers commonly rely on the built-in logging module, sprinkling simple log messages like print() or logger.info() throughout their code.

However, as systems become distributed, containerized, cloud-native, or integrated into microservices, structured logging has become a necessity. Structured logs — logs with machine-readable key-value data instead of plain text — allow powerful querying, filtering, and visualization in modern observability tools like ELK (Elasticsearch-Logstash-Kibana), Loki, Datadog, and AWS CloudWatch.

This is where structlog comes in. It’s a modern, high-performance Python library for structured logging that integrates cleanly with the existing logging module but transforms your logs into JSON-ready, structured messages.

This article offers a deep dive into structlog, covering:

• Why structured logging matters

• How structlog works under the hood

• Integrating it with Python’s logging

• Custom processors and contextual logging

• JSON logs for modern deployments

• Performance considerations

• Production-ready logging setups with structlog

Why Structured Logging?

In traditional applications, logs look like this:

[INFO] User Vinay logged in from IP 192.168.1.1

In modern, distributed systems, this is insufficient. Instead, you'd want:

{
  "level": "info",
  "event": "user_logged_in",
  "user": "Vinay",
  "ip": "192.168.1.1",
  "timestamp": "2025-06-20T08:00:00Z"
}

Advantages:

• Easily query logs by fields (user, IP, timestamp)

• Structured, consistent logs for dashboards

• Machine-readable (JSON/YAML/Key-Value pairs)

• Better correlation in distributed tracing

• Simplifies debugging, alerting, and anomaly detection

What is structlog?

structlog is a Python logging library designed for structured logging.

Key features:

• Lightweight and fast

• Integrates with logging or works independently

• Supports context-aware, thread-safe logging

• Serializes logs into JSON or custom formats

• Pluggable processor pipeline to enrich or transform log events

• Production-ready for microservices and cloud apps

Installing structlog

Install using pip:

pip install structlog

For JSON logging:

pip install structlog[json]

Basic Usage

A simple example:

import structlog

log = structlog.get_logger()
log.info("user_logged_in", user="Vinay", ip="192.168.1.1")

Output:

2025-06-20 08:00.00 [info     ] user_logged_in          user=Vinay ip=192.168.1.1

This is human-readable, but you can swap the processor chain to output JSON as well.

How structlog Works

At its core, structlog separates concerns:

• Logger binding: Attach context (key-value data) to logger instances

• Event processors: Modify, filter, or enrich log events

• Renderer: Transform the final log event into a string or JSON before emitting it

This pipeline design makes it extremely flexible.

Integrating structlog with Python’s logging Module

To capture logs from third-party libraries or use existing handlers like RotatingFileHandler, integrate with logging.

Example setup:

import logging
import structlog

logging.basicConfig(
    format="%(message)s",
    stream=sys.stdout,
    level=logging.INFO
)

structlog.configure(
    processors=[
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.add_log_level,
        structlog.processors.KeyValueRenderer(key_order=["timestamp", "level", "event"]),
    ],
    context_class=dict,
    logger_factory=structlog.stdlib.LoggerFactory(),
    wrapper_class=structlog.stdlib.BoundLogger,
    cache_logger_on_first_use=True
)

log = structlog.get_logger()

log.info("user_logged_in", user="Vinay", ip="192.168.1.1")

Output:

timestamp=2025-06-20T08:00:00Z level=info event=user_logged_in user=Vinay ip=192.168.1.1

Structured JSON Logs

In modern deployments, JSON logs are preferred for log aggregators.

structlog.configure(
    processors=[
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.add_log_level,
        structlog.processors.JSONRenderer()
    ],
    ...
)

log = structlog.get_logger()

log.info("user_logged_in", user="Vinay", ip="192.168.1.1")

Output:

{
  "timestamp": "2025-06-20T08:00:00Z",
  "level": "info",
  "event": "user_logged_in",
  "user": "Vinay",
  "ip": "192.168.1.1"
}

Perfect for ELK stack or CloudWatch ingestion.

Adding Contextual Metadata

Attach global context to loggers, avoiding repeated parameters.

log = structlog.get_logger().bind(service="authentication")

log.info("user_logged_in", user="Vinay")
log.error("invalid_token", user="Kumar")

Every log will now include service=authentication.

Dynamic context via threadlocal is possible using structlog.threadlocal.wrap_dict.

Custom Event Processors

Create processors to modify events at runtime.

Example: Add a UUID to every log.

import uuid

def add_request_id(logger, method_name, event_dict):
    event_dict["request_id"] = str(uuid.uuid4())
    return event_dict

structlog.configure(
    processors=[
        add_request_id,
        structlog.processors.JSONRenderer()
    ],
    ...
)

Now every log has a unique request_id.

Production Logging Setup Example

A complete logging pipeline for a FastAPI microservice or API.

import logging
import structlog
from structlog.stdlib import LoggerFactory
from structlog.processors import JSONRenderer, TimeStamper, add_log_level
from structlog.threadlocal import wrap_dict

logging.basicConfig(
    format="%(message)s",
    level=logging.INFO
)

structlog.configure(
    processors=[
        TimeStamper(fmt="iso"),
        add_log_level,
        JSONRenderer()
    ],
    context_class=wrap_dict(dict),
    logger_factory=LoggerFactory(),
    wrapper_class=structlog.stdlib.BoundLogger,
    cache_logger_on_first_use=True
)

log = structlog.get_logger().bind(service="api")

log.info("server_started", port=8080)

This setup:

• Emits JSON logs

• Includes timestamps and levels

• Supports dynamic context for API endpoints and job workers

Performance Considerations

• structlog is fast and efficient for JSON logging.

• Disabling logs in production:
  Use structlog.dev.ConsoleRenderer() in dev and JSONRenderer() in prod.

• Can integrate with lru_cache or third-party logging transports.

Migration From logging

Convert existing logging calls easily:

Before:

logger.info("User %s logged in from %s", user, ip)

After:

log.info("user_logged_in", user=user, ip=ip)

Fewer format errors, better structure, no positional argument issues.

Testing Logs

Unit test log outputs via:

from structlog.testing import capture_logs

with capture_logs() as logs:
    log.info("test_event", status="ok")

assert logs[0]["status"] == "ok"

Supports clean, testable logging codebases.

Conclusion

structlog is one of Python’s most valuable libraries for modern, structured, JSON-ready logging — yet surprisingly underused in mainstream projects.

It’s a no-brainer choice for:

• Microservices

• FastAPI / Django apps

• Serverless functions

• Asynchronous job queues

• Cloud-native APIs

• Data pipelines and ETL jobs