init
This commit is contained in:
2
.gitignore
vendored
Normal file
2
.gitignore
vendored
Normal file
@@ -0,0 +1,2 @@
|
||||
.claude
|
||||
.playwright-mcp
|
||||
318
README.md
Normal file
318
README.md
Normal file
@@ -0,0 +1,318 @@
|
||||
# Loki 3 + Vector + Grafana — Test Stand
|
||||
|
||||
Test environment for centralized log collection using Grafana Loki in Simple Scalable mode with Vector as the log collector and MinIO as S3-compatible object storage.
|
||||
|
||||
Two deployment options: **Docker Compose** (local testing with log-generator) and **Helm chart** (Kubernetes with DaemonSet log collection from all pods).
|
||||
|
||||
## Architecture
|
||||
|
||||

|
||||
|
||||
## Components
|
||||
|
||||
| Service | Image | Role |
|
||||
|---|---|---|
|
||||
| **minio** | `minio/minio:latest` | S3-compatible object storage for chunks and index |
|
||||
| **loki-write** | `grafana/loki:3.4.2` | Ingester — accepts and stores log streams |
|
||||
| **loki-read** | `grafana/loki:3.4.2` | Querier — executes LogQL queries |
|
||||
| **loki-backend** | `grafana/loki:3.4.2` | Compactor, ruler, index gateway |
|
||||
| **gateway** | `nginx:1.27-alpine` | Reverse proxy — Loki write/read/ruler + Grafana UI |
|
||||
| **vector** | `timberio/vector:0.44.0-alpine` | Log collector — Loki sink |
|
||||
| **grafana** | `grafana/grafana:12.4` | Visualization — Loki datasource pre-provisioned |
|
||||
|
||||
## Data Flow
|
||||
|
||||
1. **Vector** collects container logs (docker_logs / kubernetes_logs source)
|
||||
2. Vector transforms logs (extracts metadata, removes raw labels) and pushes to **gateway**
|
||||
3. **Gateway** (nginx) routes `/loki/api/v1/push` to **loki-write**, all other `/loki/api/*` to **loki-read**
|
||||
4. **loki-write** ingests logs and stores chunks in **MinIO** (`loki-chunks` bucket)
|
||||
5. **loki-backend** runs compactor on a 10-minute interval
|
||||
6. **Grafana** queries logs via gateway → loki-read
|
||||
|
||||
## Loki Configuration
|
||||
|
||||
- **Deployment mode**: Simple Scalable (write / read / backend targets)
|
||||
- **Schema**: v13 with TSDB store
|
||||
- **Storage**: S3 (MinIO) — bucket `loki-chunks`
|
||||
- **Service discovery**: memberlist (port 7946)
|
||||
- **Replication factor**: 1 (test stand)
|
||||
- **Chunk encoding**: snappy
|
||||
- **Retention**: enabled via compactor, old samples rejected after 168h
|
||||
|
||||
---
|
||||
|
||||
## Option 1: Docker Compose
|
||||
|
||||
Local test stand with a built-in log-generator container (`flog`). Vector collects logs from containers with label `vector.collect=true` via Docker socket.
|
||||
|
||||
### Quick Start
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
Wait for all services to become healthy:
|
||||
|
||||
```bash
|
||||
docker compose ps
|
||||
```
|
||||
|
||||
### Access Points
|
||||
|
||||
| Service | URL | Credentials |
|
||||
|---|---|---|
|
||||
| Grafana (via gateway) | http://localhost:3000 | admin / admin (anonymous access enabled) |
|
||||
| Loki API (via gateway) | http://localhost:3100 | — |
|
||||
| MinIO Console | http://localhost:9001 | loki / supersecret |
|
||||
| MinIO API | http://localhost:9000 | loki / supersecret |
|
||||
|
||||
### Network
|
||||
|
||||
All containers run on a static network `192.168.97.0/24`:
|
||||
|
||||
| Container | IP |
|
||||
|---|---|
|
||||
| `loki-minio` | 192.168.97.10 |
|
||||
| `loki-write` | 192.168.97.11 |
|
||||
| `loki-read` | 192.168.97.12 |
|
||||
| `loki-backend` | 192.168.97.13 |
|
||||
| `loki-gateway` | 192.168.97.14 |
|
||||
| `loki-vector` | 192.168.97.15 |
|
||||
| `loki-grafana` | 192.168.97.16 |
|
||||
| `loki-log-generator` | 192.168.97.17 |
|
||||
|
||||
### Adding Your Own Containers
|
||||
|
||||
To collect logs from any container, add the label:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
my-app:
|
||||
image: my-app:latest
|
||||
labels:
|
||||
vector.collect: "true"
|
||||
```
|
||||
|
||||
Logs will appear in Grafana with label `{container="my-app"}`.
|
||||
|
||||
### Useful Commands
|
||||
|
||||
```bash
|
||||
# Start
|
||||
docker compose up -d
|
||||
|
||||
# Stop
|
||||
docker compose down
|
||||
|
||||
# Stop and remove volumes
|
||||
docker compose down -v
|
||||
|
||||
# View Vector logs
|
||||
docker compose logs vector -f
|
||||
|
||||
# View Loki write logs
|
||||
docker compose logs loki-write -f
|
||||
|
||||
# Check Loki readiness
|
||||
curl -s http://localhost:3100/ready
|
||||
|
||||
# Query logs via API
|
||||
curl -s http://localhost:3100/loki/api/v1/query_range \
|
||||
--data-urlencode 'query={container="log-generator"}' \
|
||||
--data-urlencode 'limit=10' | jq .
|
||||
|
||||
# Check MinIO bucket
|
||||
docker compose exec minio mc ls local/loki-chunks/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Option 2: Helm Chart (Kubernetes)
|
||||
|
||||
Production-oriented deployment. Vector runs as a **DaemonSet** collecting logs from all pods in the cluster via `kubernetes_logs` source. No log-generator — real workload logs are collected.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Kubernetes cluster (1.24+)
|
||||
- Helm 3
|
||||
- `kubectl` configured
|
||||
|
||||
### Install
|
||||
|
||||
```bash
|
||||
helm install loki-stack ./helm/loki-stack -n loki --create-namespace
|
||||
```
|
||||
|
||||
### Install with custom values
|
||||
|
||||
```bash
|
||||
helm install loki-stack ./helm/loki-stack -n loki --create-namespace \
|
||||
-f my-values.yaml
|
||||
```
|
||||
|
||||
### Upgrade
|
||||
|
||||
```bash
|
||||
helm upgrade loki-stack ./helm/loki-stack -n loki
|
||||
```
|
||||
|
||||
### Uninstall
|
||||
|
||||
```bash
|
||||
helm uninstall loki-stack -n loki
|
||||
```
|
||||
|
||||
### Key Differences from Docker Compose
|
||||
|
||||
| Feature | Docker Compose | Helm |
|
||||
|---|---|---|
|
||||
| Vector source | `docker_logs` (label filter) | `kubernetes_logs` (all pods) |
|
||||
| Vector mode | container | DaemonSet (one per node) |
|
||||
| Log labels | `container`, `project` | `namespace`, `pod`, `container`, `node` |
|
||||
| Loki targets | Deployment | StatefulSet with PVC |
|
||||
| Service discovery | static IPs + memberlist | headless Service + memberlist |
|
||||
| Gateway (nginx) | yes — proxies Loki + Grafana | no — direct Service routing |
|
||||
| Ingress | — | Grafana ingress (disabled by default) |
|
||||
|
||||
### Configuration (values.yaml)
|
||||
|
||||
Key parameters:
|
||||
|
||||
```yaml
|
||||
# Scale Loki components
|
||||
loki:
|
||||
write:
|
||||
replicas: 1 # Increase for HA
|
||||
read:
|
||||
replicas: 1 # Increase for query throughput
|
||||
backend:
|
||||
replicas: 1
|
||||
|
||||
# MinIO storage
|
||||
minio:
|
||||
rootUser: loki
|
||||
rootPassword: supersecret
|
||||
storage:
|
||||
size: 10Gi
|
||||
|
||||
# Vector tolerations (to run on tainted nodes)
|
||||
vector:
|
||||
tolerations: []
|
||||
|
||||
# Grafana
|
||||
grafana:
|
||||
adminUser: admin
|
||||
adminPassword: admin
|
||||
anonymousAccess: true
|
||||
service:
|
||||
type: ClusterIP
|
||||
ingress:
|
||||
enabled: false
|
||||
className: nginx
|
||||
hosts:
|
||||
- host: grafana.example.com
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
tls: []
|
||||
```
|
||||
|
||||
### Access in Kubernetes
|
||||
|
||||
Port-forward to services:
|
||||
|
||||
```bash
|
||||
# Grafana
|
||||
kubectl port-forward -n loki svc/loki-stack-grafana 3000:3000
|
||||
|
||||
# Loki API (read)
|
||||
kubectl port-forward -n loki svc/loki-stack-loki-read 3100:3100
|
||||
|
||||
# Loki API (write / push)
|
||||
kubectl port-forward -n loki svc/loki-stack-loki-write 3100:3100
|
||||
|
||||
# MinIO Console
|
||||
kubectl port-forward -n loki svc/loki-stack-minio 9001:9001
|
||||
```
|
||||
|
||||
### Viewing Logs
|
||||
|
||||
Open Grafana and query:
|
||||
|
||||
```logql
|
||||
# All logs from a namespace
|
||||
{namespace="default"}
|
||||
|
||||
# Specific pod
|
||||
{pod="my-app-7d4b8c6f5-x2k9p"}
|
||||
|
||||
# By container name across all namespaces
|
||||
{container="nginx"}
|
||||
|
||||
# By node
|
||||
{node="worker-01"}
|
||||
```
|
||||
|
||||
### Helm Chart Structure
|
||||
|
||||
```
|
||||
helm/loki-stack/
|
||||
├── Chart.yaml
|
||||
├── values.yaml
|
||||
└── templates/
|
||||
├── _helpers.tpl
|
||||
├── minio-deployment.yaml
|
||||
├── minio-pvc.yaml
|
||||
├── minio-service.yaml
|
||||
├── loki-configmap.yaml
|
||||
├── loki-write-statefulset.yaml
|
||||
├── loki-read-statefulset.yaml
|
||||
├── loki-backend-statefulset.yaml
|
||||
├── loki-services.yaml
|
||||
├── vector-rbac.yaml
|
||||
├── vector-configmap.yaml
|
||||
├── vector-daemonset.yaml
|
||||
├── grafana-configmap.yaml
|
||||
├── grafana-deployment.yaml
|
||||
├── grafana-ingress.yaml
|
||||
├── grafana-pvc.yaml
|
||||
└── grafana-service.yaml
|
||||
```
|
||||
|
||||
### Kubernetes Resources Created
|
||||
|
||||
| Kind | Count | Description |
|
||||
|---|---|---|
|
||||
| StatefulSet | 3 | loki-write, loki-read, loki-backend |
|
||||
| Deployment | 2 | minio, grafana |
|
||||
| DaemonSet | 1 | vector (one per node) |
|
||||
| Service | 9 | ClusterIP + headless for each Loki target + memberlist + minio + grafana |
|
||||
| ConfigMap | 3 | loki config, vector config, grafana datasource |
|
||||
| PVC | 2 + 3 | minio + grafana (static) + loki targets (via volumeClaimTemplates) |
|
||||
| ServiceAccount | 1 | vector |
|
||||
| ClusterRole | 1 | vector (pods, namespaces, nodes read access) |
|
||||
| ClusterRoleBinding | 1 | vector |
|
||||
| Ingress | 0/1 | grafana (disabled by default) |
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
.
|
||||
├── docker-compose.yaml # Docker Compose deployment
|
||||
├── loki-config.yaml # Loki configuration (docker-compose)
|
||||
├── nginx.conf # Gateway routing rules (docker-compose)
|
||||
├── vector.yaml # Vector pipeline (docker-compose)
|
||||
├── provisioning/
|
||||
│ └── datasources/
|
||||
│ └── loki.yaml # Grafana datasource provisioning (docker-compose)
|
||||
├── docs/
|
||||
│ └── architecture.excalidraw.png # Architecture diagram
|
||||
├── helm/
|
||||
│ └── loki-stack/ # Helm chart for Kubernetes
|
||||
│ ├── Chart.yaml
|
||||
│ ├── values.yaml
|
||||
│ └── templates/
|
||||
└── README.md
|
||||
```
|
||||
159
docker-compose.yaml
Normal file
159
docker-compose.yaml
Normal file
@@ -0,0 +1,159 @@
|
||||
services:
|
||||
# --- MinIO ---
|
||||
minio:
|
||||
container_name: loki-minio
|
||||
image: minio/minio:latest
|
||||
entrypoint:
|
||||
- sh
|
||||
- -euc
|
||||
- |
|
||||
mkdir -p /data/loki-chunks && \
|
||||
minio server /data --console-address ":9001"
|
||||
environment:
|
||||
MINIO_ROOT_USER: loki
|
||||
MINIO_ROOT_PASSWORD: supersecret
|
||||
ports:
|
||||
- "9000:9000"
|
||||
- "9001:9001"
|
||||
volumes:
|
||||
- minio-data:/data
|
||||
healthcheck:
|
||||
test: ["CMD", "mc", "ready", "local"]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
retries: 10
|
||||
networks:
|
||||
loki-net:
|
||||
ipv4_address: 192.168.97.10
|
||||
|
||||
# --- Loki Write ---
|
||||
loki-write:
|
||||
container_name: loki-write
|
||||
image: grafana/loki:3.4.2
|
||||
hostname: loki-write
|
||||
command: -config.file=/etc/loki/loki-config.yaml -config.expand-env=true -target=write
|
||||
volumes:
|
||||
- ./loki-config.yaml:/etc/loki/loki-config.yaml:ro
|
||||
- loki-write-data:/loki
|
||||
depends_on:
|
||||
minio:
|
||||
condition: service_healthy
|
||||
networks:
|
||||
loki-net:
|
||||
ipv4_address: 192.168.97.11
|
||||
|
||||
# --- Loki Read ---
|
||||
loki-read:
|
||||
container_name: loki-read
|
||||
image: grafana/loki:3.4.2
|
||||
hostname: loki-read
|
||||
command: -config.file=/etc/loki/loki-config.yaml -config.expand-env=true -target=read
|
||||
volumes:
|
||||
- ./loki-config.yaml:/etc/loki/loki-config.yaml:ro
|
||||
- loki-read-data:/loki
|
||||
depends_on:
|
||||
minio:
|
||||
condition: service_healthy
|
||||
networks:
|
||||
loki-net:
|
||||
ipv4_address: 192.168.97.12
|
||||
|
||||
# --- Loki Backend ---
|
||||
loki-backend:
|
||||
container_name: loki-backend
|
||||
image: grafana/loki:3.4.2
|
||||
hostname: loki-backend
|
||||
command: -config.file=/etc/loki/loki-config.yaml -config.expand-env=true -target=backend
|
||||
volumes:
|
||||
- ./loki-config.yaml:/etc/loki/loki-config.yaml:ro
|
||||
- loki-backend-data:/loki
|
||||
depends_on:
|
||||
minio:
|
||||
condition: service_healthy
|
||||
networks:
|
||||
loki-net:
|
||||
ipv4_address: 192.168.97.13
|
||||
|
||||
# --- Gateway (nginx) ---
|
||||
gateway:
|
||||
container_name: loki-gateway
|
||||
image: nginx:1.27-alpine
|
||||
volumes:
|
||||
- ./nginx.conf:/etc/nginx/nginx.conf:ro
|
||||
ports:
|
||||
- "3100:3100"
|
||||
- "3000:3000"
|
||||
depends_on:
|
||||
- loki-write
|
||||
- loki-read
|
||||
- loki-backend
|
||||
- grafana
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "wget --quiet --tries=1 --output-document=- http://192.168.97.14:3100/ready | grep -q 'ready'"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 15
|
||||
networks:
|
||||
loki-net:
|
||||
ipv4_address: 192.168.97.14
|
||||
|
||||
# --- Vector ---
|
||||
vector:
|
||||
container_name: loki-vector
|
||||
image: timberio/vector:0.44.0-alpine
|
||||
volumes:
|
||||
- ./vector.yaml:/etc/vector/vector.yaml:ro
|
||||
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||
depends_on:
|
||||
gateway:
|
||||
condition: service_healthy
|
||||
networks:
|
||||
loki-net:
|
||||
ipv4_address: 192.168.97.15
|
||||
|
||||
# --- Grafana ---
|
||||
grafana:
|
||||
container_name: loki-grafana
|
||||
image: grafana/grafana:12.4
|
||||
environment:
|
||||
GF_SECURITY_ADMIN_USER: admin
|
||||
GF_SECURITY_ADMIN_PASSWORD: admin
|
||||
GF_AUTH_ANONYMOUS_ENABLED: "true"
|
||||
GF_AUTH_ANONYMOUS_ORG_ROLE: Admin
|
||||
volumes:
|
||||
- ./provisioning:/etc/grafana/provisioning:ro
|
||||
- grafana-data:/var/lib/grafana
|
||||
depends_on:
|
||||
loki-read:
|
||||
condition: service_started
|
||||
networks:
|
||||
loki-net:
|
||||
ipv4_address: 192.168.97.16
|
||||
|
||||
# --- Log Generator ---
|
||||
log-generator:
|
||||
container_name: loki-log-generator
|
||||
image: mingrammer/flog:0.4.3
|
||||
command: -f json -d 2s -l
|
||||
labels:
|
||||
vector.collect: "true"
|
||||
depends_on:
|
||||
- vector
|
||||
networks:
|
||||
loki-net:
|
||||
ipv4_address: 192.168.97.17
|
||||
|
||||
networks:
|
||||
loki-net:
|
||||
driver: bridge
|
||||
ipam:
|
||||
config:
|
||||
- subnet: 192.168.97.0/24
|
||||
gateway: 192.168.97.1
|
||||
|
||||
volumes:
|
||||
minio-data:
|
||||
loki-write-data:
|
||||
loki-read-data:
|
||||
loki-backend-data:
|
||||
grafana-data:
|
||||
BIN
docs/architecture.excalidraw.png
Normal file
BIN
docs/architecture.excalidraw.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 96 KiB |
6
helm/loki-stack/Chart.yaml
Normal file
6
helm/loki-stack/Chart.yaml
Normal file
@@ -0,0 +1,6 @@
|
||||
apiVersion: v2
|
||||
name: loki-stack
|
||||
description: Loki 3 Simple Scalable + Vector DaemonSet + Grafana + MinIO
|
||||
version: 0.1.0
|
||||
appVersion: "3.4.2"
|
||||
type: application
|
||||
40
helm/loki-stack/templates/_helpers.tpl
Normal file
40
helm/loki-stack/templates/_helpers.tpl
Normal file
@@ -0,0 +1,40 @@
|
||||
{{/*
|
||||
Expand the name of the chart.
|
||||
*/}}
|
||||
{{- define "loki-stack.name" -}}
|
||||
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
Create a default fully qualified app name.
|
||||
*/}}
|
||||
{{- define "loki-stack.fullname" -}}
|
||||
{{- if .Values.fullnameOverride }}
|
||||
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
|
||||
{{- else }}
|
||||
{{- $name := default .Chart.Name .Values.nameOverride }}
|
||||
{{- if contains $name .Release.Name }}
|
||||
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
|
||||
{{- else }}
|
||||
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
Common labels
|
||||
*/}}
|
||||
{{- define "loki-stack.labels" -}}
|
||||
helm.sh/chart: {{ include "loki-stack.name" . }}
|
||||
app.kubernetes.io/managed-by: {{ .Release.Service }}
|
||||
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
|
||||
app.kubernetes.io/part-of: {{ include "loki-stack.name" . }}
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
Selector labels for a component
|
||||
*/}}
|
||||
{{- define "loki-stack.selectorLabels" -}}
|
||||
app.kubernetes.io/name: {{ include "loki-stack.name" . }}
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
{{- end }}
|
||||
17
helm/loki-stack/templates/grafana-configmap.yaml
Normal file
17
helm/loki-stack/templates/grafana-configmap.yaml
Normal file
@@ -0,0 +1,17 @@
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-grafana-datasources
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: grafana
|
||||
data:
|
||||
loki.yaml: |
|
||||
apiVersion: 1
|
||||
datasources:
|
||||
- name: Loki
|
||||
type: loki
|
||||
access: proxy
|
||||
url: http://{{ include "loki-stack.fullname" . }}-loki-read:3100
|
||||
isDefault: true
|
||||
editable: true
|
||||
56
helm/loki-stack/templates/grafana-deployment.yaml
Normal file
56
helm/loki-stack/templates/grafana-deployment.yaml
Normal file
@@ -0,0 +1,56 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-grafana
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: grafana
|
||||
spec:
|
||||
replicas: {{ .Values.grafana.replicas }}
|
||||
selector:
|
||||
matchLabels:
|
||||
{{- include "loki-stack.selectorLabels" . | nindent 6 }}
|
||||
app.kubernetes.io/component: grafana
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
{{- include "loki-stack.selectorLabels" . | nindent 8 }}
|
||||
app.kubernetes.io/component: grafana
|
||||
spec:
|
||||
securityContext:
|
||||
fsGroup: 472
|
||||
containers:
|
||||
- name: grafana
|
||||
image: {{ .Values.grafana.image.repository }}:{{ .Values.grafana.image.tag }}
|
||||
env:
|
||||
- name: GF_SECURITY_ADMIN_USER
|
||||
value: {{ .Values.grafana.adminUser }}
|
||||
- name: GF_SECURITY_ADMIN_PASSWORD
|
||||
value: {{ .Values.grafana.adminPassword }}
|
||||
- name: GF_AUTH_ANONYMOUS_ENABLED
|
||||
value: {{ .Values.grafana.anonymousAccess | quote }}
|
||||
- name: GF_AUTH_ANONYMOUS_ORG_ROLE
|
||||
value: Admin
|
||||
ports:
|
||||
- name: http
|
||||
containerPort: 3000
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /api/health
|
||||
port: http
|
||||
initialDelaySeconds: 10
|
||||
periodSeconds: 10
|
||||
volumeMounts:
|
||||
- name: datasources
|
||||
mountPath: /etc/grafana/provisioning/datasources
|
||||
- name: data
|
||||
mountPath: /var/lib/grafana
|
||||
resources:
|
||||
{{- toYaml .Values.grafana.resources | nindent 12 }}
|
||||
volumes:
|
||||
- name: datasources
|
||||
configMap:
|
||||
name: {{ include "loki-stack.fullname" . }}-grafana-datasources
|
||||
- name: data
|
||||
persistentVolumeClaim:
|
||||
claimName: {{ include "loki-stack.fullname" . }}-grafana
|
||||
36
helm/loki-stack/templates/grafana-ingress.yaml
Normal file
36
helm/loki-stack/templates/grafana-ingress.yaml
Normal file
@@ -0,0 +1,36 @@
|
||||
{{- if .Values.grafana.ingress.enabled }}
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-grafana
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: grafana
|
||||
{{- with .Values.grafana.ingress.annotations }}
|
||||
annotations:
|
||||
{{- toYaml . | nindent 4 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
{{- if .Values.grafana.ingress.className }}
|
||||
ingressClassName: {{ .Values.grafana.ingress.className }}
|
||||
{{- end }}
|
||||
{{- if .Values.grafana.ingress.tls }}
|
||||
tls:
|
||||
{{- toYaml .Values.grafana.ingress.tls | nindent 4 }}
|
||||
{{- end }}
|
||||
rules:
|
||||
{{- range .Values.grafana.ingress.hosts }}
|
||||
- host: {{ .host | quote }}
|
||||
http:
|
||||
paths:
|
||||
{{- range .paths }}
|
||||
- path: {{ .path }}
|
||||
pathType: {{ .pathType }}
|
||||
backend:
|
||||
service:
|
||||
name: {{ include "loki-stack.fullname" $ }}-grafana
|
||||
port:
|
||||
name: http
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
16
helm/loki-stack/templates/grafana-pvc.yaml
Normal file
16
helm/loki-stack/templates/grafana-pvc.yaml
Normal file
@@ -0,0 +1,16 @@
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-grafana
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: grafana
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
{{- if .Values.grafana.storage.storageClassName }}
|
||||
storageClassName: {{ .Values.grafana.storage.storageClassName }}
|
||||
{{- end }}
|
||||
resources:
|
||||
requests:
|
||||
storage: {{ .Values.grafana.storage.size }}
|
||||
16
helm/loki-stack/templates/grafana-service.yaml
Normal file
16
helm/loki-stack/templates/grafana-service.yaml
Normal file
@@ -0,0 +1,16 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-grafana
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: grafana
|
||||
spec:
|
||||
type: ClusterIP
|
||||
ports:
|
||||
- name: http
|
||||
port: 3000
|
||||
targetPort: http
|
||||
selector:
|
||||
{{- include "loki-stack.selectorLabels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: grafana
|
||||
69
helm/loki-stack/templates/loki-backend-statefulset.yaml
Normal file
69
helm/loki-stack/templates/loki-backend-statefulset.yaml
Normal file
@@ -0,0 +1,69 @@
|
||||
apiVersion: apps/v1
|
||||
kind: StatefulSet
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-loki-backend
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: loki-backend
|
||||
spec:
|
||||
replicas: {{ .Values.loki.backend.replicas }}
|
||||
serviceName: {{ include "loki-stack.fullname" . }}-loki-backend-headless
|
||||
selector:
|
||||
matchLabels:
|
||||
{{- include "loki-stack.selectorLabels" . | nindent 6 }}
|
||||
app.kubernetes.io/component: loki-backend
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
{{- include "loki-stack.selectorLabels" . | nindent 8 }}
|
||||
app.kubernetes.io/component: loki-backend
|
||||
loki.grafana.com/memberlist: "true"
|
||||
spec:
|
||||
containers:
|
||||
- name: loki
|
||||
image: {{ .Values.loki.image.repository }}:{{ .Values.loki.image.tag }}
|
||||
args:
|
||||
- -config.file=/etc/loki/loki-config.yaml
|
||||
- -config.expand-env=true
|
||||
- -target=backend
|
||||
env:
|
||||
- name: POD_IP
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: status.podIP
|
||||
ports:
|
||||
- name: http
|
||||
containerPort: 3100
|
||||
- name: grpc
|
||||
containerPort: 9095
|
||||
- name: memberlist
|
||||
containerPort: 7946
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /ready
|
||||
port: http
|
||||
initialDelaySeconds: 15
|
||||
periodSeconds: 10
|
||||
volumeMounts:
|
||||
- name: config
|
||||
mountPath: /etc/loki
|
||||
- name: data
|
||||
mountPath: /loki
|
||||
resources:
|
||||
{{- toYaml .Values.loki.backend.resources | nindent 12 }}
|
||||
volumes:
|
||||
- name: config
|
||||
configMap:
|
||||
name: {{ include "loki-stack.fullname" . }}-loki
|
||||
volumeClaimTemplates:
|
||||
- metadata:
|
||||
name: data
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
{{- if .Values.loki.backend.storage.storageClassName }}
|
||||
storageClassName: {{ .Values.loki.backend.storage.storageClassName }}
|
||||
{{- end }}
|
||||
resources:
|
||||
requests:
|
||||
storage: {{ .Values.loki.backend.storage.size }}
|
||||
74
helm/loki-stack/templates/loki-configmap.yaml
Normal file
74
helm/loki-stack/templates/loki-configmap.yaml
Normal file
@@ -0,0 +1,74 @@
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-loki
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: loki
|
||||
data:
|
||||
loki-config.yaml: |
|
||||
auth_enabled: false
|
||||
|
||||
server:
|
||||
http_listen_port: 3100
|
||||
grpc_listen_port: 9095
|
||||
log_level: info
|
||||
|
||||
common:
|
||||
compactor_address: http://{{ include "loki-stack.fullname" . }}-loki-backend:3100
|
||||
ring:
|
||||
instance_addr: ${POD_IP}
|
||||
kvstore:
|
||||
store: memberlist
|
||||
replication_factor: 1
|
||||
path_prefix: /loki
|
||||
|
||||
memberlist:
|
||||
join_members:
|
||||
- {{ include "loki-stack.fullname" . }}-loki-memberlist:7946
|
||||
|
||||
schema_config:
|
||||
configs:
|
||||
- from: "2024-01-01"
|
||||
store: tsdb
|
||||
object_store: s3
|
||||
schema: v13
|
||||
index:
|
||||
prefix: index_
|
||||
period: 24h
|
||||
|
||||
storage_config:
|
||||
aws:
|
||||
endpoint: {{ include "loki-stack.fullname" . }}-minio:9000
|
||||
insecure: true
|
||||
bucketnames: {{ .Values.minio.bucketName }}
|
||||
access_key_id: {{ .Values.minio.rootUser }}
|
||||
secret_access_key: {{ .Values.minio.rootPassword }}
|
||||
s3forcepathstyle: true
|
||||
tsdb_shipper:
|
||||
active_index_directory: /loki/index
|
||||
cache_location: /loki/index_cache
|
||||
|
||||
ingester:
|
||||
chunk_encoding: snappy
|
||||
|
||||
querier:
|
||||
max_concurrent: 4
|
||||
|
||||
frontend_worker:
|
||||
frontend_address: 127.0.0.1:9095
|
||||
|
||||
limits_config:
|
||||
reject_old_samples: true
|
||||
reject_old_samples_max_age: 168h
|
||||
allow_structured_metadata: true
|
||||
volume_enabled: true
|
||||
|
||||
compactor:
|
||||
working_directory: /loki/compactor
|
||||
compaction_interval: 10m
|
||||
retention_enabled: true
|
||||
delete_request_store: s3
|
||||
|
||||
pattern_ingester:
|
||||
enabled: true
|
||||
69
helm/loki-stack/templates/loki-read-statefulset.yaml
Normal file
69
helm/loki-stack/templates/loki-read-statefulset.yaml
Normal file
@@ -0,0 +1,69 @@
|
||||
apiVersion: apps/v1
|
||||
kind: StatefulSet
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-loki-read
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: loki-read
|
||||
spec:
|
||||
replicas: {{ .Values.loki.read.replicas }}
|
||||
serviceName: {{ include "loki-stack.fullname" . }}-loki-read-headless
|
||||
selector:
|
||||
matchLabels:
|
||||
{{- include "loki-stack.selectorLabels" . | nindent 6 }}
|
||||
app.kubernetes.io/component: loki-read
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
{{- include "loki-stack.selectorLabels" . | nindent 8 }}
|
||||
app.kubernetes.io/component: loki-read
|
||||
loki.grafana.com/memberlist: "true"
|
||||
spec:
|
||||
containers:
|
||||
- name: loki
|
||||
image: {{ .Values.loki.image.repository }}:{{ .Values.loki.image.tag }}
|
||||
args:
|
||||
- -config.file=/etc/loki/loki-config.yaml
|
||||
- -config.expand-env=true
|
||||
- -target=read
|
||||
env:
|
||||
- name: POD_IP
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: status.podIP
|
||||
ports:
|
||||
- name: http
|
||||
containerPort: 3100
|
||||
- name: grpc
|
||||
containerPort: 9095
|
||||
- name: memberlist
|
||||
containerPort: 7946
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /ready
|
||||
port: http
|
||||
initialDelaySeconds: 15
|
||||
periodSeconds: 10
|
||||
volumeMounts:
|
||||
- name: config
|
||||
mountPath: /etc/loki
|
||||
- name: data
|
||||
mountPath: /loki
|
||||
resources:
|
||||
{{- toYaml .Values.loki.read.resources | nindent 12 }}
|
||||
volumes:
|
||||
- name: config
|
||||
configMap:
|
||||
name: {{ include "loki-stack.fullname" . }}-loki
|
||||
volumeClaimTemplates:
|
||||
- metadata:
|
||||
name: data
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
{{- if .Values.loki.read.storage.storageClassName }}
|
||||
storageClassName: {{ .Values.loki.read.storage.storageClassName }}
|
||||
{{- end }}
|
||||
resources:
|
||||
requests:
|
||||
storage: {{ .Values.loki.read.storage.size }}
|
||||
133
helm/loki-stack/templates/loki-services.yaml
Normal file
133
helm/loki-stack/templates/loki-services.yaml
Normal file
@@ -0,0 +1,133 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-loki-write
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: loki-write
|
||||
spec:
|
||||
type: ClusterIP
|
||||
ports:
|
||||
- name: http
|
||||
port: 3100
|
||||
targetPort: http
|
||||
- name: grpc
|
||||
port: 9095
|
||||
targetPort: grpc
|
||||
selector:
|
||||
{{- include "loki-stack.selectorLabels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: loki-write
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-loki-write-headless
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: loki-write
|
||||
spec:
|
||||
type: ClusterIP
|
||||
clusterIP: None
|
||||
ports:
|
||||
- name: http
|
||||
port: 3100
|
||||
targetPort: http
|
||||
selector:
|
||||
{{- include "loki-stack.selectorLabels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: loki-write
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-loki-read
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: loki-read
|
||||
spec:
|
||||
type: ClusterIP
|
||||
ports:
|
||||
- name: http
|
||||
port: 3100
|
||||
targetPort: http
|
||||
- name: grpc
|
||||
port: 9095
|
||||
targetPort: grpc
|
||||
selector:
|
||||
{{- include "loki-stack.selectorLabels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: loki-read
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-loki-read-headless
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: loki-read
|
||||
spec:
|
||||
type: ClusterIP
|
||||
clusterIP: None
|
||||
ports:
|
||||
- name: http
|
||||
port: 3100
|
||||
targetPort: http
|
||||
selector:
|
||||
{{- include "loki-stack.selectorLabels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: loki-read
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-loki-backend
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: loki-backend
|
||||
spec:
|
||||
type: ClusterIP
|
||||
ports:
|
||||
- name: http
|
||||
port: 3100
|
||||
targetPort: http
|
||||
- name: grpc
|
||||
port: 9095
|
||||
targetPort: grpc
|
||||
selector:
|
||||
{{- include "loki-stack.selectorLabels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: loki-backend
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-loki-backend-headless
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: loki-backend
|
||||
spec:
|
||||
type: ClusterIP
|
||||
clusterIP: None
|
||||
ports:
|
||||
- name: http
|
||||
port: 3100
|
||||
targetPort: http
|
||||
selector:
|
||||
{{- include "loki-stack.selectorLabels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: loki-backend
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-loki-memberlist
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: loki-memberlist
|
||||
spec:
|
||||
type: ClusterIP
|
||||
clusterIP: None
|
||||
publishNotReadyAddresses: true
|
||||
ports:
|
||||
- name: memberlist
|
||||
port: 7946
|
||||
targetPort: memberlist
|
||||
protocol: TCP
|
||||
selector:
|
||||
{{- include "loki-stack.selectorLabels" . | nindent 4 }}
|
||||
loki.grafana.com/memberlist: "true"
|
||||
69
helm/loki-stack/templates/loki-write-statefulset.yaml
Normal file
69
helm/loki-stack/templates/loki-write-statefulset.yaml
Normal file
@@ -0,0 +1,69 @@
|
||||
apiVersion: apps/v1
|
||||
kind: StatefulSet
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-loki-write
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: loki-write
|
||||
spec:
|
||||
replicas: {{ .Values.loki.write.replicas }}
|
||||
serviceName: {{ include "loki-stack.fullname" . }}-loki-write-headless
|
||||
selector:
|
||||
matchLabels:
|
||||
{{- include "loki-stack.selectorLabels" . | nindent 6 }}
|
||||
app.kubernetes.io/component: loki-write
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
{{- include "loki-stack.selectorLabels" . | nindent 8 }}
|
||||
app.kubernetes.io/component: loki-write
|
||||
loki.grafana.com/memberlist: "true"
|
||||
spec:
|
||||
containers:
|
||||
- name: loki
|
||||
image: {{ .Values.loki.image.repository }}:{{ .Values.loki.image.tag }}
|
||||
args:
|
||||
- -config.file=/etc/loki/loki-config.yaml
|
||||
- -config.expand-env=true
|
||||
- -target=write
|
||||
env:
|
||||
- name: POD_IP
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: status.podIP
|
||||
ports:
|
||||
- name: http
|
||||
containerPort: 3100
|
||||
- name: grpc
|
||||
containerPort: 9095
|
||||
- name: memberlist
|
||||
containerPort: 7946
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /ready
|
||||
port: http
|
||||
initialDelaySeconds: 15
|
||||
periodSeconds: 10
|
||||
volumeMounts:
|
||||
- name: config
|
||||
mountPath: /etc/loki
|
||||
- name: data
|
||||
mountPath: /loki
|
||||
resources:
|
||||
{{- toYaml .Values.loki.write.resources | nindent 12 }}
|
||||
volumes:
|
||||
- name: config
|
||||
configMap:
|
||||
name: {{ include "loki-stack.fullname" . }}-loki
|
||||
volumeClaimTemplates:
|
||||
- metadata:
|
||||
name: data
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
{{- if .Values.loki.write.storage.storageClassName }}
|
||||
storageClassName: {{ .Values.loki.write.storage.storageClassName }}
|
||||
{{- end }}
|
||||
resources:
|
||||
requests:
|
||||
storage: {{ .Values.loki.write.storage.size }}
|
||||
52
helm/loki-stack/templates/minio-deployment.yaml
Normal file
52
helm/loki-stack/templates/minio-deployment.yaml
Normal file
@@ -0,0 +1,52 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-minio
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: minio
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
{{- include "loki-stack.selectorLabels" . | nindent 6 }}
|
||||
app.kubernetes.io/component: minio
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
{{- include "loki-stack.selectorLabels" . | nindent 8 }}
|
||||
app.kubernetes.io/component: minio
|
||||
spec:
|
||||
containers:
|
||||
- name: minio
|
||||
image: {{ .Values.minio.image.repository }}:{{ .Values.minio.image.tag }}
|
||||
command:
|
||||
- sh
|
||||
- -euc
|
||||
- |
|
||||
mkdir -p /data/{{ .Values.minio.bucketName }} && \
|
||||
minio server /data --console-address ":9001"
|
||||
env:
|
||||
- name: MINIO_ROOT_USER
|
||||
value: {{ .Values.minio.rootUser }}
|
||||
- name: MINIO_ROOT_PASSWORD
|
||||
value: {{ .Values.minio.rootPassword }}
|
||||
ports:
|
||||
- name: api
|
||||
containerPort: 9000
|
||||
- name: console
|
||||
containerPort: 9001
|
||||
readinessProbe:
|
||||
exec:
|
||||
command: ["mc", "ready", "local"]
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 5
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /data
|
||||
resources:
|
||||
{{- toYaml .Values.minio.resources | nindent 12 }}
|
||||
volumes:
|
||||
- name: data
|
||||
persistentVolumeClaim:
|
||||
claimName: {{ include "loki-stack.fullname" . }}-minio
|
||||
16
helm/loki-stack/templates/minio-pvc.yaml
Normal file
16
helm/loki-stack/templates/minio-pvc.yaml
Normal file
@@ -0,0 +1,16 @@
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-minio
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: minio
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
{{- if .Values.minio.storage.storageClassName }}
|
||||
storageClassName: {{ .Values.minio.storage.storageClassName }}
|
||||
{{- end }}
|
||||
resources:
|
||||
requests:
|
||||
storage: {{ .Values.minio.storage.size }}
|
||||
19
helm/loki-stack/templates/minio-service.yaml
Normal file
19
helm/loki-stack/templates/minio-service.yaml
Normal file
@@ -0,0 +1,19 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-minio
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: minio
|
||||
spec:
|
||||
type: ClusterIP
|
||||
ports:
|
||||
- name: api
|
||||
port: 9000
|
||||
targetPort: api
|
||||
- name: console
|
||||
port: 9001
|
||||
targetPort: console
|
||||
selector:
|
||||
{{- include "loki-stack.selectorLabels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: minio
|
||||
10
helm/loki-stack/templates/namespace.yaml
Normal file
10
helm/loki-stack/templates/namespace.yaml
Normal file
@@ -0,0 +1,10 @@
|
||||
{{- if .Values.createNamespace }}
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: {{ .Release.Namespace }}
|
||||
labels:
|
||||
pod-security.kubernetes.io/audit: privileged
|
||||
pod-security.kubernetes.io/enforce: privileged
|
||||
pod-security.kubernetes.io/warn: privileged
|
||||
{{- end }}
|
||||
33
helm/loki-stack/templates/networkpolicy.yaml
Normal file
33
helm/loki-stack/templates/networkpolicy.yaml
Normal file
@@ -0,0 +1,33 @@
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-allow-internal
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
spec:
|
||||
podSelector:
|
||||
matchLabels:
|
||||
{{- include "loki-stack.selectorLabels" . | nindent 6 }}
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
ingress:
|
||||
# Allow all traffic within the namespace
|
||||
- from:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: {{ .Release.Namespace }}
|
||||
egress:
|
||||
# Allow all traffic within the namespace
|
||||
- to:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: {{ .Release.Namespace }}
|
||||
# Allow DNS
|
||||
- to:
|
||||
- namespaceSelector: {}
|
||||
ports:
|
||||
- protocol: UDP
|
||||
port: 53
|
||||
- protocol: TCP
|
||||
port: 53
|
||||
43
helm/loki-stack/templates/vector-configmap.yaml
Normal file
43
helm/loki-stack/templates/vector-configmap.yaml
Normal file
@@ -0,0 +1,43 @@
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-vector
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: vector
|
||||
data:
|
||||
vector.yaml: |
|
||||
sources:
|
||||
kubernetes_logs:
|
||||
type: kubernetes_logs
|
||||
self_node_name: ${VECTOR_SELF_NODE_NAME}
|
||||
|
||||
transforms:
|
||||
parse_logs:
|
||||
type: remap
|
||||
inputs:
|
||||
- kubernetes_logs
|
||||
source: |
|
||||
.namespace = .kubernetes.pod_namespace
|
||||
.pod = .kubernetes.pod_name
|
||||
.container = .kubernetes.container_name
|
||||
.node = .kubernetes.pod_node_name
|
||||
del(.kubernetes)
|
||||
del(.file)
|
||||
del(.source_type)
|
||||
|
||||
sinks:
|
||||
loki:
|
||||
type: loki
|
||||
inputs:
|
||||
- parse_logs
|
||||
endpoint: http://{{ include "loki-stack.fullname" . }}-loki-write:3100
|
||||
encoding:
|
||||
codec: text
|
||||
labels:
|
||||
source: vector
|
||||
namespace: "{{`{{ namespace }}`}}"
|
||||
pod: "{{`{{ pod }}`}}"
|
||||
container: "{{`{{ container }}`}}"
|
||||
node: "{{`{{ node }}`}}"
|
||||
remove_label_fields: true
|
||||
60
helm/loki-stack/templates/vector-daemonset.yaml
Normal file
60
helm/loki-stack/templates/vector-daemonset.yaml
Normal file
@@ -0,0 +1,60 @@
|
||||
apiVersion: apps/v1
|
||||
kind: DaemonSet
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-vector
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: vector
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
{{- include "loki-stack.selectorLabels" . | nindent 6 }}
|
||||
app.kubernetes.io/component: vector
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
{{- include "loki-stack.selectorLabels" . | nindent 8 }}
|
||||
app.kubernetes.io/component: vector
|
||||
spec:
|
||||
serviceAccountName: {{ include "loki-stack.fullname" . }}-vector
|
||||
containers:
|
||||
- name: vector
|
||||
image: {{ .Values.vector.image.repository }}:{{ .Values.vector.image.tag }}
|
||||
securityContext:
|
||||
privileged: true
|
||||
env:
|
||||
- name: VECTOR_SELF_NODE_NAME
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: spec.nodeName
|
||||
volumeMounts:
|
||||
- name: config
|
||||
mountPath: /etc/vector/vector.yaml
|
||||
subPath: vector.yaml
|
||||
- name: var-log
|
||||
mountPath: /var/log
|
||||
readOnly: true
|
||||
- name: var-lib-containers
|
||||
mountPath: /var/log/pods
|
||||
readOnly: true
|
||||
- name: data
|
||||
mountPath: /vector-data-dir
|
||||
resources:
|
||||
{{- toYaml .Values.vector.resources | nindent 12 }}
|
||||
{{- with .Values.vector.tolerations }}
|
||||
tolerations:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
volumes:
|
||||
- name: config
|
||||
configMap:
|
||||
name: {{ include "loki-stack.fullname" . }}-vector
|
||||
- name: var-log
|
||||
hostPath:
|
||||
path: /var/log
|
||||
- name: var-lib-containers
|
||||
hostPath:
|
||||
path: /var/log/pods
|
||||
- name: data
|
||||
hostPath:
|
||||
path: /var/lib/vector
|
||||
35
helm/loki-stack/templates/vector-rbac.yaml
Normal file
35
helm/loki-stack/templates/vector-rbac.yaml
Normal file
@@ -0,0 +1,35 @@
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-vector
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: vector
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-vector
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: vector
|
||||
rules:
|
||||
- apiGroups: [""]
|
||||
resources: ["namespaces", "nodes", "pods"]
|
||||
verbs: ["get", "list", "watch"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: {{ include "loki-stack.fullname" . }}-vector
|
||||
labels:
|
||||
{{- include "loki-stack.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: vector
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: {{ include "loki-stack.fullname" . }}-vector
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: {{ include "loki-stack.fullname" . }}-vector
|
||||
namespace: {{ .Release.Namespace }}
|
||||
112
helm/loki-stack/values.yaml
Normal file
112
helm/loki-stack/values.yaml
Normal file
@@ -0,0 +1,112 @@
|
||||
nameOverride: ""
|
||||
fullnameOverride: ""
|
||||
|
||||
# Create namespace with PSA labels (privileged) for Talos
|
||||
createNamespace: false
|
||||
|
||||
# --- MinIO ---
|
||||
minio:
|
||||
image:
|
||||
repository: minio/minio
|
||||
tag: latest
|
||||
rootUser: loki
|
||||
rootPassword: supersecret
|
||||
bucketName: loki-chunks
|
||||
storage:
|
||||
size: 10Gi
|
||||
storageClassName: ""
|
||||
resources:
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 256Mi
|
||||
limits:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
|
||||
# --- Loki ---
|
||||
loki:
|
||||
image:
|
||||
repository: grafana/loki
|
||||
tag: "3.4.2"
|
||||
write:
|
||||
replicas: 1
|
||||
resources:
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 256Mi
|
||||
limits:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
storage:
|
||||
size: 10Gi
|
||||
storageClassName: ""
|
||||
read:
|
||||
replicas: 1
|
||||
resources:
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 256Mi
|
||||
limits:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
storage:
|
||||
size: 10Gi
|
||||
storageClassName: ""
|
||||
backend:
|
||||
replicas: 1
|
||||
resources:
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 256Mi
|
||||
limits:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
storage:
|
||||
size: 10Gi
|
||||
storageClassName: ""
|
||||
|
||||
# --- Vector ---
|
||||
vector:
|
||||
image:
|
||||
repository: timberio/vector
|
||||
tag: 0.44.0-alpine
|
||||
resources:
|
||||
requests:
|
||||
cpu: 50m
|
||||
memory: 64Mi
|
||||
limits:
|
||||
cpu: 200m
|
||||
memory: 256Mi
|
||||
tolerations: []
|
||||
|
||||
# --- Grafana ---
|
||||
grafana:
|
||||
image:
|
||||
repository: grafana/grafana
|
||||
tag: "12.4"
|
||||
adminUser: admin
|
||||
adminPassword: admin
|
||||
anonymousAccess: true
|
||||
replicas: 1
|
||||
resources:
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 128Mi
|
||||
limits:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
service:
|
||||
type: ClusterIP
|
||||
storage:
|
||||
size: 5Gi
|
||||
storageClassName: ""
|
||||
ingress:
|
||||
enabled: false
|
||||
className: ""
|
||||
annotations: {}
|
||||
hosts:
|
||||
- host: grafana.local
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
tls: []
|
||||
67
loki-config.yaml
Normal file
67
loki-config.yaml
Normal file
@@ -0,0 +1,67 @@
|
||||
auth_enabled: false
|
||||
|
||||
server:
|
||||
http_listen_port: 3100
|
||||
grpc_listen_port: 9095
|
||||
log_level: info
|
||||
|
||||
common:
|
||||
compactor_address: http://loki-backend:3100
|
||||
ring:
|
||||
instance_addr: ${HOSTNAME}
|
||||
kvstore:
|
||||
store: memberlist
|
||||
replication_factor: 1
|
||||
path_prefix: /loki
|
||||
|
||||
memberlist:
|
||||
join_members:
|
||||
- loki-write:7946
|
||||
- loki-read:7946
|
||||
- loki-backend:7946
|
||||
|
||||
schema_config:
|
||||
configs:
|
||||
- from: "2024-01-01"
|
||||
store: tsdb
|
||||
object_store: s3
|
||||
schema: v13
|
||||
index:
|
||||
prefix: index_
|
||||
period: 24h
|
||||
|
||||
storage_config:
|
||||
aws:
|
||||
endpoint: minio:9000
|
||||
insecure: true
|
||||
bucketnames: loki-chunks
|
||||
access_key_id: loki
|
||||
secret_access_key: supersecret
|
||||
s3forcepathstyle: true
|
||||
tsdb_shipper:
|
||||
active_index_directory: /loki/index
|
||||
cache_location: /loki/index_cache
|
||||
|
||||
ingester:
|
||||
chunk_encoding: snappy
|
||||
|
||||
querier:
|
||||
max_concurrent: 4
|
||||
|
||||
frontend_worker:
|
||||
frontend_address: loki-read:9095
|
||||
|
||||
limits_config:
|
||||
reject_old_samples: true
|
||||
reject_old_samples_max_age: 168h
|
||||
allow_structured_metadata: true
|
||||
volume_enabled: true
|
||||
|
||||
compactor:
|
||||
working_directory: /loki/compactor
|
||||
compaction_interval: 10m
|
||||
retention_enabled: true
|
||||
delete_request_store: s3
|
||||
|
||||
pattern_ingester:
|
||||
enabled: true
|
||||
59
nginx.conf
Normal file
59
nginx.conf
Normal file
@@ -0,0 +1,59 @@
|
||||
worker_processes 1;
|
||||
|
||||
events {
|
||||
worker_connections 1024;
|
||||
}
|
||||
|
||||
http {
|
||||
resolver 127.0.0.11 valid=10s;
|
||||
|
||||
# Loki gateway
|
||||
server {
|
||||
listen 3100;
|
||||
|
||||
location = /ready {
|
||||
proxy_pass http://loki-read:3100$request_uri;
|
||||
}
|
||||
|
||||
# Write path
|
||||
location = /loki/api/v1/push {
|
||||
proxy_pass http://loki-write:3100$request_uri;
|
||||
}
|
||||
|
||||
# Read path
|
||||
location ~ /loki/api/.* {
|
||||
proxy_pass http://loki-read:3100$request_uri;
|
||||
}
|
||||
|
||||
# Ruler
|
||||
location ~ /api/prom/rules.* {
|
||||
proxy_pass http://loki-backend:3100$request_uri;
|
||||
}
|
||||
|
||||
location ~ /prometheus/api/v1/rules.* {
|
||||
proxy_pass http://loki-backend:3100$request_uri;
|
||||
}
|
||||
}
|
||||
|
||||
# Grafana proxy
|
||||
server {
|
||||
listen 3000;
|
||||
|
||||
location / {
|
||||
proxy_pass http://grafana:3000;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
|
||||
# Grafana WebSocket (live features)
|
||||
location /api/live/ {
|
||||
proxy_pass http://grafana:3000;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
proxy_set_header Host $host;
|
||||
}
|
||||
}
|
||||
}
|
||||
9
provisioning/datasources/loki.yaml
Normal file
9
provisioning/datasources/loki.yaml
Normal file
@@ -0,0 +1,9 @@
|
||||
apiVersion: 1
|
||||
|
||||
datasources:
|
||||
- name: Loki
|
||||
type: loki
|
||||
access: proxy
|
||||
url: http://gateway:3100
|
||||
isDefault: true
|
||||
editable: true
|
||||
30
vector.yaml
Normal file
30
vector.yaml
Normal file
@@ -0,0 +1,30 @@
|
||||
sources:
|
||||
docker_logs:
|
||||
type: docker_logs
|
||||
include_labels:
|
||||
- vector.collect=true
|
||||
|
||||
transforms:
|
||||
parse_logs:
|
||||
type: remap
|
||||
inputs:
|
||||
- docker_logs
|
||||
source: |
|
||||
.container_name = del(.label."com.docker.compose.service")
|
||||
.compose_project = del(.label."com.docker.compose.project")
|
||||
# Remove unnecessary labels
|
||||
del(.label)
|
||||
|
||||
sinks:
|
||||
loki:
|
||||
type: loki
|
||||
inputs:
|
||||
- parse_logs
|
||||
endpoint: http://gateway:3100
|
||||
encoding:
|
||||
codec: text
|
||||
labels:
|
||||
source: vector
|
||||
container: "{{ container_name }}"
|
||||
project: "{{ compose_project }}"
|
||||
remove_label_fields: true
|
||||
Reference in New Issue
Block a user