Architecture and Core Concepts

This guide explains why Pragmatic.Logging exists, how its pieces fit together, and how to choose the right configuration for each situation. Read this before diving into the individual feature guides.

---

The Problem

Logging sits on the hot path of every request. A production application needs structured log output, correlation IDs for distributed tracing, PII redaction for compliance, rate limiting to prevent log spam, and multiple output targets — all without adding measurable latency. The standard .NET logging infrastructure provides the foundation, but leaves these concerns to the developer.

Manual formatting wastes allocations

// Every call allocates a new string, even when the level is disabled
logger.LogInformation($"Order {orderId} placed by {customerId} for {total:C}");

// Even the recommended structured approach allocates parameter arrays
logger.LogInformation("Order {OrderId} placed by {CustomerId} for {Total}",
    orderId, customerId, total);

Under high throughput, string allocations from logging alone can create GC pressure that causes visible latency spikes. The standard LoggerMessage.Define pattern helps, but requires boilerplate for every message.

No built-in privacy protection

// This logs the full connection string, including the password
logger.LogError("Failed to connect: {ConnectionString}", connectionString);

// This leaks the JWT token into your log aggregator
logger.LogDebug("Auth header: {AuthHeader}", request.Headers.Authorization);

// This writes PII to disk in a GDPR-regulated environment
logger.LogInformation("User {Email} placed order {OrderId}", user.Email, orderId);

There is no standard mechanism to detect and redact secrets, API keys, or personally identifiable information before they reach the log output. Developers must remember to sanitize every call site — and they will forget.

No structured context enrichment

// Without enrichment, you pass context manually at every call site
logger.LogInformation("Order placed. CorrelationId={CorrelationId}, UserId={UserId}, " +
    "Machine={Machine}, RequestPath={Path}",
    correlationId, userId, Environment.MachineName, httpContext.Request.Path);

When debugging a production incident across dozens of service replicas, you need every log entry to carry ambient context — correlation IDs, user identifiers, request paths, machine names. Passing these values explicitly at every call site is error-prone and clutters the business code.

No unified configuration

Each concern — output targets, formatting, filtering, rate limiting, compliance — requires its own configuration surface. Developers cobble together Serilog enrichers, NLog targets, custom middleware, and manual redaction code. There is no single place to configure “production-ready logging” with one call.

---

The Solution

Pragmatic.Logging wraps the standard Microsoft.Extensions.Logging infrastructure with a fluent PragmaticLoggingBuilder API. It provides:

• Zero-allocation formatting — stack-allocated buffers and pooled StringBuilder instances eliminate GC pressure on hot paths
• Automatic secret detection and PII redaction — pre-compiled regex patterns scan log content before it reaches any output provider
• Context enrichment — ambient properties (correlation IDs, user identity, machine name) are attached to every log entry without changes to business code
• Multiple providers — console, file, JSON, NDJSON, memory, debug, Windows Event Log, and custom providers
• Configuration presets — one-call setup for Development, Production, High Performance, and Compliance scenarios
• Rate limiting — lock-free token bucket, sliding window, and fixed window strategies prevent log spam
• Audit trail — compliance-grade recording of every redaction event and sensitive data access
• Standard interface — everything works through ILogger<T>. No proprietary abstractions.

// With Pragmatic: one builder, all concerns configured
builder.Services.AddLogging(logging =>
{
    logging.ClearProviders();

    var pragmatic = logging.AddPragmaticLogging();
    pragmatic
        .UseProductionPreset()           // Sensible defaults for production
        .AddConsole()                    // Colored terminal output
        .AddFile("logs/app-{Date}.log") // Rolling file with retention
        .AddJson(Console.OpenStandardOutput()); // Structured JSON for log aggregators
});

Or integrated with the Pragmatic.Composition host:

await PragmaticApp.RunAsync(args, app =>
{
    app.UseLogging(log =>
    {
        log.AddConsole(PragmaticConsoleConfiguration.ForDevelopment());
        log.AddFile("logs/app-{Date}.log");
    });
});

Performance

Benchmarked against Serilog and NLog on .NET 10 (BenchmarkDotNet 0.14.0):

Scenario	Pragmatic	NLog	Serilog
Simple logging	21.87 ns / 64 B	138.52 ns / 488 B	155.06 ns / 440 B
Structured logging	46.98 ns / 216 B	279.84 ns / 696 B	557.59 ns / 2016 B
Exception logging	21.48 ns / 64 B	134.99 ns / 504 B	163.39 ns / 440 B
High volume (1000/iter)	18.88 us / 64 KB	134.75 us / 476 KB	154.71 us / 430 KB

Pragmatic is 5.9x faster than NLog and 11.9x faster than Serilog for structured logging.

---

How It Works

Pragmatic.Logging is built on three layers: the builder that configures the pipeline, the provider base that enforces the pipeline for every output target, and the providers that write log entries to their destinations.

Layer 1: PragmaticLoggingBuilder

The PragmaticLoggingBuilder is the entry point for all configuration. It extends ILoggingBuilder and provides a fluent API for adding providers, applying presets, enabling privacy features, and configuring performance options.

var pragmatic = logging.AddPragmaticLogging();

// Presets configure all knobs at once
pragmatic.UseProductionPreset();

// Then override individual settings
pragmatic.Configure(opts => opts.MinimumLevel = LogLevel.Debug);

// Add providers
pragmatic.AddConsole();
pragmatic.AddFile("logs/app-{Date}.log");

// Enable privacy features
pragmatic.EnableDataRedaction(r => r.RedactionPlaceholder = "[REDACTED]");
pragmatic.EnableAuditTrail(a => a.StorageType = "FileSystem");

Layer 2: PragmaticLoggerProviderBase

Every provider — built-in or custom — extends PragmaticLoggerProviderBase. This base class enforces a consistent pipeline for every log entry:

Log Entry
  |
  v
[IsEnabled check] --no--> (dropped, zero work)
  |
  yes
  v
[Advanced filter chain] --filtered--> (dropped, counter incremented)
  |
  pass
  v
[Context enrichment] -- adds CorrelationId, UserId, MachineName, etc.
  |
  v
[Structured property filter] -- applies ContextFilterMode
  |
  v
[Data redaction]
  |-- SecretDetector scans message + property values
  |-- PragmaticDataRedactor checks property names + patterns
  |-- Audit trail records each redaction event
  |
  v
[WriteLogCore] -- provider-specific output (console, file, JSON, etc.)

This means every provider automatically benefits from the full privacy stack, context enrichment, and filtering. You never need to implement redaction in a custom provider.

Layer 3: Providers

Each provider implements WriteLogCore and handles the specific output format and destination. Pragmatic.Logging ships with eight built-in providers:

Provider	Class	Output
Console	PragmaticConsoleProvider	Colored terminal output with structured data
File	PragmaticFileProvider	File-based with rolling, retention, async I/O
JSON	PragmaticJsonProvider	Structured JSON for log aggregators
Enhanced JSON	PragmaticEnhancedJsonProvider	NDJSON with async buffering
Memory	PragmaticMemoryProvider	In-memory buffer for testing
Debug	PragmaticDebugProvider	Debug output window (Visual Studio, Rider)
Null	PragmaticNullProvider	No-op for benchmarking
Windows Event Log	PragmaticWindowsEventLogProvider	Windows Event Log

---

The LoggerMessage Pattern

.NET provides [LoggerMessage] for high-performance logging with source-generated methods. Pragmatic.Logging builds on this foundation and extends it.

Standard LoggerMessage (recommended for hot paths)

public partial class OrderService(ILogger<OrderService> logger)
{
    [LoggerMessage(Level = LogLevel.Information, Message = "Order {OrderId} placed by {CustomerId}")]
    partial void LogOrderPlaced(Guid orderId, string customerId);

    public void PlaceOrder(Guid orderId, string customerId)
    {
        LogOrderPlaced(orderId, customerId);
    }
}

The [LoggerMessage] attribute generates a partial method that:

• Performs an IsEnabled check before any work
• Uses cached delegates for zero-allocation parameter passing
• Avoids string interpolation entirely

When to use which approach

Approach	Allocation	Use Case
[LoggerMessage] partial method	Zero	Hot paths, high-throughput code
logger.LogInformation("...", args)	Low (parameter array)	Business logic, moderate throughput
String interpolation $"..."	High (string + boxing)	Never in production code

---

Privacy and Redaction

Pragmatic.Logging provides a layered defense against accidental exposure of sensitive data.

Secret Detection

The SecretDetector scans log messages and property values against a curated library of pre-compiled regex patterns. Each detection carries a confidence score and severity level.

Category	Examples	Severity
Crypto	RSA/EC/PGP private keys	Critical
API Keys	AWS (AKIA...), GitHub PAT (ghp_...), Stripe (sk_live_...)	Critical
Tokens	JWT (eyJ...), Bearer tokens, OAuth refresh tokens	High
Database	Connection strings with Password=, MongoDB URIs	Critical
Cloud	Azure Storage Key, GCP Service Account Key	Critical
Application	Encryption keys, hash salts	Medium

The SecretDetectionOptions.MinimumConfidenceForRedaction threshold (default: 0.7) controls which detections trigger automatic redaction. Detections below the threshold are logged for review but not redacted.

Data Redaction

Beyond secret detection, the PragmaticDataRedactor handles PII and business-sensitive data based on property names and message content patterns:

1. Explicit flag — Properties marked with PropertyCharacteristics.Redacted are always redacted
2. Name matching — Property names checked against a HashSet<string> of sensitive names (case-insensitive) and compiled regex patterns
3. Message scanning — Log message text scanned for emails, credit card numbers, SSNs, and custom patterns

pragmatic.EnableDataRedaction(redaction =>
{
    redaction.RedactionPlaceholder = "[REDACTED]";
    redaction.SensitivePropertyNames = ["password", "apiKey", "email", "ssn"];
    redaction.PropertyNamePatterns = [@".*password.*", @".*secret.*"];
    redaction.MessageRedactionPatterns = [@"[\w.+-]+@[\w-]+\.[\w.]+"];  // Email
});

Redaction Styles

Style	Output	Description
Placeholder	[API_KEY_REDACTED]	Descriptive placeholder with secret type
PreserveLength	********************************	Asterisks matching original length
PreserveStructure	eyJ***.***.***	Keeps format markers (JWT segments)
Minimal	[REDACTED]	Simple constant replacement

Compliance Standards

Pre-built compliance templates configure the correct sensitive property lists, message patterns, redaction placeholders, audit requirements, and data retention periods:

Standard	Placeholder	Deep Redaction	Audit Trail	Retention
GDPR	[GDPR-REDACTED]	Yes	Yes	2 years
HIPAA	[PHI-REDACTED]	Yes	Yes	6 years
PCI-DSS	[CARD-DATA-REDACTED]	Yes	Yes	1 year
CCPA	[REDACTED]	Yes	Yes	90 days
SOX	[REDACTED]	Yes	Yes	7 years

// GDPR-compliant logging
pragmatic.UseCompliancePreset(ComplianceStandard.Gdpr);

// Multi-compliance (uses most restrictive settings)
var config = ComplianceTemplates.CreateMultiCompliance(
    includeGdpr: true, includeHipaa: true, includePciDss: false);

---

Zero-Allocation Performance

For hot paths, Pragmatic.Logging provides infrastructure that eliminates heap allocations.

StackAllocatedBuffer

Small messages (up to 1,024 characters) are formatted into a stackalloc buffer. Larger messages fall back to ArrayPool<char>.Shared so the rented array is reused across calls.

ZeroAllocMessageFormatter

The core formatting engine uses a ThreadLocal<StringBuilder> pool and ArrayPool<object?> for parameter arrays, minimizing allocations on every call. It has specialized handlers for common types:

Type	Allocation	Method
int, long, float, double, decimal	Zero	TryFormat(Span<char>)
DateTime, DateTimeOffset	Zero	TryFormat with ISO 8601
TimeSpan, Guid	Zero	TryFormat
bool	Zero	Literal copy
string	Zero	AsSpan().CopyTo()
ISpanFormattable	Zero	Interface TryFormat
Other	1 allocation	ToString() fallback

Object Pooling

Pool	Purpose
StringBuilderPool	Reuses StringBuilder instances with automatic capacity management
DefaultObjectPool<T>	Generic pool for LogEntry objects, formatter state, serialization buffers
PropertyPool	Reuses Dictionary<string, object?> instances for structured log properties

Early-Exit IsEnabled Checks

Every provider performs an IsEnabled check with [MethodImpl(AggressiveInlining)] before any formatting work begins. If the log level is below the configured minimum, the call returns immediately with zero work done.

---

Context Enrichment

Context enrichment automatically attaches ambient information to every log entry without requiring the developer to pass values explicitly at every call site.

Built-in Context Providers

Provider	Priority	Scope	Key Properties
HttpContextProvider	100	Per-request	RequestPath, RequestMethod, UserId, RemoteIpAddress
CorrelationIdProvider	50	Per-request	CorrelationId, TraceId, SpanId
ThreadContextProvider	500	Per-call	ThreadId, ThreadName, IsBackground
ProcessContextProvider	900	Per-process	ProcessId, ProcessName, AppVersion
MachineContextProvider	1000	Per-machine	MachineName, OSVersion, CLRVersion

Custom Providers

Extend ContextProviderBase for domain-specific context:

public sealed class TenantContextProvider : ContextProviderBase
{
    private readonly ITenantContext _tenantContext;

    public TenantContextProvider(ITenantContext tenantContext)
        : base("Tenant", priority: 80) { _tenantContext = tenantContext; }

    public override bool IsAvailable() => _tenantContext.CurrentTenantId is not null;

    public override IReadOnlyDictionary<string, object?> GetContextProperties()
        => CreatePropertiesDictionary(
            ("TenantId", _tenantContext.CurrentTenantId),
            ("TenantName", _tenantContext.CurrentTenantName));
}

ASP.NET Core Middleware

Two middleware components wire enrichment into the HTTP pipeline:

• LoggingEnrichmentMiddleware — Creates a logging scope with correlation ID, request path, request method, and user identity. Optionally logs request start/completion with timing and maps HTTP status codes to log levels (5xx = Error, 4xx = Warning, 2xx/3xx = Information).

• BaggagePropagationMiddleware — Propagates selected context values into Activity.Baggage for automatic forwarding to downstream services via W3C Baggage headers.

app.UseMiddleware<BaggagePropagationMiddleware>();
app.UseMiddleware<LoggingEnrichmentMiddleware>();

Per-Provider Context Filtering

Each provider can independently control which context properties it includes via ContextFilterConfiguration. This lets you add full context to file logs while keeping console output compact.

logging.AddConsole(config =>
{
    config.ContextFilter = new ContextFilterConfiguration
    {
        Mode = ContextFilterMode.Include,
        PropertyNames = new HashSet<string> { "CorrelationId", "UserId" }
    };
});

Mode	Behavior
All	Include all context properties (default)
Include	Include only listed properties
Exclude	Include all except listed properties
None	Include no context properties

---

Configuration Presets

PragmaticLoggingBuilder provides four built-in presets that configure all knobs at once.

Development

pragmatic.UseDevelopmentPreset();

• MinimumLevel: Trace
• HighPerformanceMode: disabled
• Telemetry: disabled
• Machine context: enabled, user context: disabled

Production

pragmatic.UseProductionPreset();

• MinimumLevel: Information
• Background processing: enabled, buffer size 5000
• Rate limiting: enabled (1000 msg/sec)
• Correlation IDs and user context: enabled

High Performance

pragmatic.UseHighPerformancePreset();

• MinimumLevel: Information (skips Debug/Trace)
• HighPerformanceMode: enabled
• Zero-allocation optimizations: enabled
• Buffer size: 10000, flush threshold: 1000

Compliance

pragmatic.UseCompliancePreset(ComplianceStandard.Gdpr);

• Redaction: enabled
• Secret detection: enabled
• Audit trail: enabled with database storage
• Confidence thresholds vary by standard (GDPR: 0.8, HIPAA: 0.9, PCI-DSS: 0.95)

Override After Preset

Presets set sensible defaults. Override individual settings after applying a preset:

pragmatic
    .UseProductionPreset()
    .Configure(opts => opts.MinimumLevel = LogLevel.Debug);

---

Rate Limiting

Under sustained load, a noisy log category can produce thousands of duplicate messages per second. The HighPerformanceRateLimiter throttles log output using one of three strategies without blocking the caller.

Strategy	Algorithm	Behavior
TokenBucket	Token bucket	Allows bursts up to bucket size, then rate-limits. Best for most scenarios.
SlidingWindow	Sliding window	Precise rate over a moving window. Best for strict compliance.
FixedWindow	Fixed window	Simple counter reset at boundary. Fastest, but allows edge bursts.

pragmatic.EnableRateLimiting(rl =>
{
    rl.MaxMessagesPerSecond = 1000;
    rl.BurstSize = 100;
    rl.Strategy = "TokenBucket";
});

---

Background Processing and Batching

Synchronous writes to disk or network block the calling thread. Pragmatic.Logging decouples log production from log output.

Background Processing

When enabled, log entries are enqueued into a bounded channel. A background consumer thread drains the queue and writes entries in batches.

Caller thread              Background thread
     |                           |
     |-- Enqueue(entry) -->      |
     |   (non-blocking)          |
     |                      [Wait for batch/timer]
     |                           |
     |                      [WriteLogCore(batch)]

Queue Overflow Strategies

Strategy	Data Loss	Blocking
DropOldest	Oldest entries	No
DropNewest	Newest entries	No
Block	None	Yes
Expand	None	No

For production, DropOldest is recommended. Use Block only in compliance-critical scenarios.

Batching

Individual disk writes are expensive. Batching amortizes the cost by collecting entries until either the BatchSize threshold or FlushTimeout is reached.

Scenario	Recommended Batch Size	Flush Interval
Web API (moderate traffic)	50-100	1-2 seconds
High-throughput service	200-500	5-10 seconds
Real-time streaming	10-20	100 ms
Compliance (every entry matters)	25-50	500 ms

---

Audit Trail

The PragmaticAuditService records every redaction event, sensitive data access, and compliance violation into a durable audit store. It runs as an IHostedService with batch processing and configurable flush intervals.

Storage Options

Storage	Class	Persistence	Best For
Memory	MemoryAuditStorage	In-process	Testing, development
File System	FileSystemAuditStorage	Disk	Single-server deployments

Audit Policies

Policy	Description
Default	Records all redaction and access events
GDPR	Records everything, forces immediate flush for violations
HighPerformance	Only records high-severity events
Development	Minimal auditing for fast iteration

pragmatic.EnableAuditTrail(audit =>
{
    audit.StorageType = "FileSystem";
    audit.PolicyType = "GDPR";
    audit.BatchSize = 100;
    audit.FlushIntervalSeconds = 30;
});

---

Bootstrap Logger

For logging before the DI container is built (during startup configuration):

using var bootstrap = BootstrapLogger.Create();
bootstrap.LogInformation("Starting application...");

var builder = WebApplication.CreateBuilder(args);
// ... configure services ...

---

Configuration via appsettings.json

All options bind to the PragmaticLogging configuration section:

{
  "PragmaticLogging": {
    "Enabled": true,
    "MinimumLevel": "Information",
    "HighPerformanceMode": false,
    "Privacy": {
      "EnableRedaction": true,
      "EnableSecretDetection": true,
      "ComplianceStandard": "General",
      "SecretDetection": {
        "MinimumSecretLength": 8,
        "PrecompilePatterns": true,
        "MinimumConfidenceForRedaction": 0.7
      },
      "DataRedaction": {
        "RedactionPlaceholder": "[REDACTED]",
        "SensitivePropertyNames": ["password", "secret"]
      }
    },
    "Performance": {
      "BufferSize": 1000,
      "FlushThreshold": 100,
      "EnableZeroAllocation": true,
      "UseBackgroundProcessing": true
    },
    "RateLimiting": {
      "Enabled": false,
      "MaxMessagesPerSecond": 1000,
      "Strategy": "TokenBucket"
    },
    "Providers": {
      "Console": { "Enabled": true, "UseColors": true },
      "File": { "Enabled": true, "BasePath": "./logs", "MaxFileSizeMB": 100 }
    }
  }
}

---

Ecosystem Integration

IPragmaticBuilder

Pragmatic.Logging integrates with the Pragmatic.Composition host via UseLogging():

await PragmaticApp.RunAsync(args, app =>
{
    app.UseLogging(log =>
    {
        log.AddConsole(PragmaticConsoleConfiguration.ForDevelopment());
        log.AddFile("logs/app-{Date}.log");
    });
});

Standard ILogger

After registration, inject and use ILogger<T> as usual. Pragmatic.Logging handles enrichment, redaction, and routing behind the scenes:

public class OrderService(ILogger<OrderService> logger)
{
    public void ProcessOrder(int orderId)
    {
        logger.LogInformation("Processing order {OrderId}", orderId);
    }
}

Custom Providers

Implement IPragmaticLoggerProvider or extend PragmaticLoggerProviderBase for custom output targets. The base class handles configuration, metrics, health checks, enrichment, and redaction — you only implement WriteLogCore.

public sealed class SlackAlertProvider : PragmaticLoggerProviderBase
{
    protected override void WriteLogCore(LogEntry logEntry)
    {
        if (logEntry.LogLevel < LogLevel.Error) return;
        // Send to Slack webhook
    }
}

---

Key Types

Type	Namespace	Purpose
PragmaticLoggingBuilder	Pragmatic.Logging	Fluent configuration API
PragmaticLoggingOptions	Pragmatic.Logging.Configuration	Main options class
SecretDetector	Pragmatic.Logging.Privacy	Secret detection engine
PragmaticDataRedactor	Pragmatic.Logging.Privacy	Data redaction service
PragmaticAuditService	Pragmatic.Logging.Privacy.Audit	Audit trail service
PragmaticFileProvider	Pragmatic.Logging.Providers	High-performance file provider
PragmaticConsoleProvider	Pragmatic.Logging.Providers	Console provider
PragmaticJsonProvider	Pragmatic.Logging.Providers	JSON/NDJSON provider
PragmaticMemoryProvider	Pragmatic.Logging.Providers	In-memory provider for testing
LoggingEnrichmentMiddleware	Pragmatic.Logging.AspNetCore	ASP.NET Core enrichment middleware
BaggagePropagationMiddleware	Pragmatic.Logging.AspNetCore	Distributed tracing baggage propagation
BootstrapLogger	Pragmatic.Logging.AspNetCore	Pre-DI logger
ContextManager	Pragmatic.Logging.Context	Context provider registry
HighPerformanceRateLimiter	Pragmatic.Logging	Lock-free rate limiter
ZeroAllocMessageFormatter	Pragmatic.Logging.ZeroAllocation	Zero-allocation message formatting
StringBuilderPool	Pragmatic.Logging.ObjectPooling	Pooled StringBuilder instances

---

See Also

• Getting Started — Installation, registration, first provider
• Log Providers — Detailed configuration for each provider
• Privacy and Security — Secret detection, data redaction, compliance
• Performance — Zero-allocation, batching, rate limiting, background processing
• Context Enrichment — Built-in and custom context providers
• Common Mistakes — Pitfalls and how to avoid them
• Troubleshooting — Problem/solution guide