loki

Grafana Loki

Grafana Loki is a horizontally-scalable, highly-available log aggregation system inspired by Prometheus. Unlike other logging systems, Loki is designed to be cost-effective and easy to operate, focusing on storing and querying logs with minimal indexing.

Features

• Label-based indexing (like Prometheus)
• LogQL, a powerful query language for logs
• Seamless integration with Grafana
• Multi-tenancy support
• Scalability via microservice architecture
• Efficient storage (logs are compressed and stored in object stores or local filesystems)

Architecture Overview

Loki is built with a microservices design:

• Alloy: Log shipping agent that tags, scrapes, and forwards logs to Loki
• Distributor: Handles ingestion and load balancing
• Ingester: Temporarily stores and processes incoming logs
• Querier: Executes LogQL queries
• Compactor: Merges log segments for better storage performance
• Index Gateway: Handles index writes and queries in scalable deployments
• Storage: Backend storage using object stores (S3, GCS, etc.) or local filesystems

LogQL

LogQL is Loki’s query language, combining filtering and aggregation capabilities:

Log stream selectors

{job="nginx", level="error"}

Filtering log content

{job="nginx"} |= "timeout"
{job="nginx"} != "healthcheck"

Aggregation queries

sum(rate({job="nginx"} |= "error" [5m])) by (instance)

Compose

version: '3'
services:
  loki:
    image: grafana/loki:latest
    ports:
      - "3100:3100"
    command: -config.file=/etc/loki/local-config.yaml