Loki 3 + Vector + Grafana — Test Stand

Test environment for centralized log collection using Grafana Loki in Simple Scalable mode with Vector as the log collector and MinIO as S3-compatible object storage.

Two deployment options: Docker Compose (local testing with log-generator) and Helm chart (Kubernetes with DaemonSet log collection from all pods).

Architecture

Components

Service	Image	Role
minio	minio/minio:latest	S3-compatible object storage for chunks and index
loki-write	grafana/loki:3.4.2	Ingester — accepts and stores log streams
loki-read	grafana/loki:3.4.2	Querier — executes LogQL queries
loki-backend	grafana/loki:3.4.2	Compactor, ruler, index gateway
gateway	nginx:1.27-alpine	Reverse proxy — Loki write/read/ruler + Grafana UI
vector	timberio/vector:0.44.0-alpine	Log collector — Loki sink
grafana	grafana/grafana:12.4	Visualization — Loki datasource pre-provisioned

Data Flow

1. Vector collects container logs (docker_logs / kubernetes_logs source)
2. Vector transforms logs (extracts metadata, removes raw labels) and pushes to gateway
3. Gateway (nginx) routes /loki/api/v1/push to loki-write, all other /loki/api/* to loki-read
4. loki-write ingests logs and stores chunks in MinIO (loki-chunks bucket)
5. loki-backend runs compactor on a 10-minute interval
6. Grafana queries logs via gateway → loki-read

Loki Configuration

• Deployment mode: Simple Scalable (write / read / backend targets)
• Schema: v13 with TSDB store
• Storage: S3 (MinIO) — bucket loki-chunks
• Service discovery: memberlist (port 7946)
• Replication factor: 1 (test stand)
• Chunk encoding: snappy
• Retention: enabled via compactor, old samples rejected after 168h

---

Option 1: Docker Compose

Local test stand with a built-in log-generator container (flog). Vector collects logs from containers with label vector.collect=true via Docker socket.

Quick Start

docker compose up -d

Wait for all services to become healthy:

docker compose ps

Access Points

Service	URL	Credentials
Grafana (via gateway)	http://localhost:3000	admin / admin (anonymous access enabled)
Loki API (via gateway)	http://localhost:3100	—
MinIO Console	http://localhost:9001	loki / supersecret
MinIO API	http://localhost:9000	loki / supersecret

Network

All containers run on a static network 192.168.97.0/24:

Container	IP
loki-minio	192.168.97.10
loki-write	192.168.97.11
loki-read	192.168.97.12
loki-backend	192.168.97.13
loki-gateway	192.168.97.14
loki-vector	192.168.97.15
loki-grafana	192.168.97.16
loki-log-generator	192.168.97.17

Adding Your Own Containers

To collect logs from any container, add the label:

services:
  my-app:
    image: my-app:latest
    labels:
      vector.collect: "true"

Logs will appear in Grafana with label {container="my-app"}.

Useful Commands

# Start
docker compose up -d

# Stop
docker compose down

# Stop and remove volumes
docker compose down -v

# View Vector logs
docker compose logs vector -f

# View Loki write logs
docker compose logs loki-write -f

# Check Loki readiness
curl -s http://localhost:3100/ready

# Query logs via API
curl -s http://localhost:3100/loki/api/v1/query_range \
  --data-urlencode 'query={container="log-generator"}' \
  --data-urlencode 'limit=10' | jq .

# Check MinIO bucket
docker compose exec minio mc ls local/loki-chunks/

---

Option 2: Helm Chart (Kubernetes)

Production-oriented deployment. Vector runs as a DaemonSet collecting logs from all pods in the cluster via kubernetes_logs source. No log-generator — real workload logs are collected.

Prerequisites

• Kubernetes cluster (1.24+)
• Helm 3
• kubectl configured

Install

helm install loki-stack ./helm/loki-stack -n loki --create-namespace

Install with custom values

helm install loki-stack ./helm/loki-stack -n loki --create-namespace \
  -f my-values.yaml

Upgrade

helm upgrade loki-stack ./helm/loki-stack -n loki

Uninstall

helm uninstall loki-stack -n loki

Key Differences from Docker Compose

Feature	Docker Compose	Helm
Vector source	docker_logs (label filter)	kubernetes_logs (all pods)
Vector mode	container	DaemonSet (one per node)
Log labels	container, project	namespace, pod, container, node
Loki targets	Deployment	StatefulSet with PVC
Service discovery	static IPs + memberlist	headless Service + memberlist
Gateway (nginx)	yes — proxies Loki + Grafana	no — direct Service routing
Ingress	—	Grafana ingress (disabled by default)

Configuration (values.yaml)

Key parameters:

# Scale Loki components
loki:
  write:
    replicas: 1        # Increase for HA
  read:
    replicas: 1        # Increase for query throughput
  backend:
    replicas: 1

# MinIO storage
minio:
  rootUser: loki
  rootPassword: supersecret
  storage:
    size: 10Gi

# Vector tolerations (to run on tainted nodes)
vector:
  tolerations: []

# Grafana
grafana:
  adminUser: admin
  adminPassword: admin
  anonymousAccess: true
  service:
    type: ClusterIP
  ingress:
    enabled: false
    className: nginx
    hosts:
      - host: grafana.example.com
        paths:
          - path: /
            pathType: Prefix
    tls: []

Access in Kubernetes

Port-forward to services:

# Grafana
kubectl port-forward -n loki svc/loki-stack-grafana 3000:3000

# Loki API (read)
kubectl port-forward -n loki svc/loki-stack-loki-read 3100:3100

# Loki API (write / push)
kubectl port-forward -n loki svc/loki-stack-loki-write 3100:3100

# MinIO Console
kubectl port-forward -n loki svc/loki-stack-minio 9001:9001

Viewing Logs

Open Grafana and query:

# All logs from a namespace
{namespace="default"}

# Specific pod
{pod="my-app-7d4b8c6f5-x2k9p"}

# By container name across all namespaces
{container="nginx"}

# By node
{node="worker-01"}

Helm Chart Structure

helm/loki-stack/
├── Chart.yaml
├── values.yaml
└── templates/
    ├── _helpers.tpl
    ├── minio-deployment.yaml
    ├── minio-pvc.yaml
    ├── minio-service.yaml
    ├── loki-configmap.yaml
    ├── loki-write-statefulset.yaml
    ├── loki-read-statefulset.yaml
    ├── loki-backend-statefulset.yaml
    ├── loki-services.yaml
    ├── vector-rbac.yaml
    ├── vector-configmap.yaml
    ├── vector-daemonset.yaml
    ├── grafana-configmap.yaml
    ├── grafana-deployment.yaml
    ├── grafana-ingress.yaml
    ├── grafana-pvc.yaml
    └── grafana-service.yaml

Kubernetes Resources Created

Kind	Count	Description
StatefulSet	3	loki-write, loki-read, loki-backend
Deployment	2	minio, grafana
DaemonSet	1	vector (one per node)
Service	9	ClusterIP + headless for each Loki target + memberlist + minio + grafana
ConfigMap	3	loki config, vector config, grafana datasource
PVC	2 + 3	minio + grafana (static) + loki targets (via volumeClaimTemplates)
ServiceAccount	1	vector
ClusterRole	1	vector (pods, namespaces, nodes read access)
ClusterRoleBinding	1	vector
Ingress	0/1	grafana (disabled by default)

---

File Structure

.
├── docker-compose.yaml              # Docker Compose deployment
├── loki-config.yaml                 # Loki configuration (docker-compose)
├── nginx.conf                       # Gateway routing rules (docker-compose)
├── vector.yaml                      # Vector pipeline (docker-compose)
├── provisioning/
│   └── datasources/
│       └── loki.yaml                # Grafana datasource provisioning (docker-compose)
├── docs/
│   └── architecture.excalidraw.png  # Architecture diagram
├── helm/
│   └── loki-stack/                  # Helm chart for Kubernetes
│       ├── Chart.yaml
│       ├── values.yaml
│       └── templates/
└── README.md