pub/elevenlabs.io
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor: switch nginx to Caddy with automatic Let's Encrypt
Build Admin / Build image (push) Successful in 13s
Details

Browse Source 
Nginx required manually managed certs on port 8443; TLS was effectively
unused. Caddy issues and renews certificates for DOMAIN on its own.

- Caddyfile replaces nginx.conf + locations.conf; upstreams are one
  `import api <prefix> <host> <path-prefix>` line each
- routes wrapped in `route {}` so guards run before handle_path
- access control (IP allowlist, X-Proxy-Token) generated from env into
  matchers.caddy / guard.caddy by the entrypoint, as before
- listeners: 443 (+udp, HTTP/3) and 80 for ACME on DOMAIN, plus the
  unchanged plain-HTTP 8082 -> 8080 path for existing clients
- caddy_data volume added: losing it re-issues certs and burns LE limits
- scripts/migrate-host-nginx.sh backs up and retires the host nginx
- deploy.sh points at the real prod layout (/srv/elevenlabs, compose v1)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
vasyanskandClaude Opus 5 committed 2026-09-20 13:35:49 +07:00
1 parent c31f6d8d90
commit d71001a2bf
15 files changed
+289 -466
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

README.md
+81 -83
View File
@@ -1,12 +1,16 @@
# API Proxy — HTTPS Reverse Proxy
Production-ready nginx reverse proxy in Docker for forwarding requests from an internal Kubernetes cluster to external APIs (ElevenLabs, OpenAI, Telegram Bot API).
Production-ready Caddy reverse proxy in Docker for forwarding requests from an internal Kubernetes cluster to external APIs (ElevenLabs, OpenAI, Telegram Bot API, OpenRouter, DeepSeek).
TLS certificates are issued and renewed automatically by Let's Encrypt — no certbot, no manual cert files.
## Quick Start
### Prerequisites
- Ubuntu 24.04 VM
- Port 8080 (HTTP) and optionally 8443 (HTTPS) open **only** to k8s cluster CIDR
- Ports **80** and **443** free and reachable from the internet (required for the ACME HTTP-01 challenge)
- A DNS `A` record pointing your domain at the VM (e.g. `proxy.bildme.ru → 62.84.98.250`)
- Port **8082** (plain HTTP) open **only** to the k8s cluster CIDR
### Installation
@@ -14,26 +18,36 @@ Production-ready nginx reverse proxy in Docker for forwarding requests from an i
sudo bash scripts/install.sh
```
If the VM already runs a host nginx on 80/443, free those ports first (backs up the config, then disables the service):
```bash
sudo bash scripts/migrate-host-nginx.sh
```
### Configuration
```bash
vi .env
# Set your secret and cluster CIDR:
PROXY_SECRET=<generated-token>
DOMAIN=proxy.bildme.ru
ACME_EMAIL=you@example.com
PROXY_SECRET=<openssl rand -hex 32>
ALLOWED_CIDR=10.0.0.0/8,172.16.0.0/12
docker compose up -d --build
docker-compose up -d
```
`DOMAIN` and `ACME_EMAIL` are mandatory — the entrypoint refuses to start without them.
### Verify
```bash
# Health check (no auth):
curl http://localhost:8080/health
curl http://localhost:8082/health
curl https://proxy.bildme.ru/health
# Test with auth:
curl -H "X-Proxy-Token: YOUR_SECRET" http://localhost:8080/elevenlabs/v1/voices
curl -H "X-Proxy-Token: YOUR_SECRET" https://proxy.bildme.ru/elevenlabs/v1/voices
```
---
@@ -49,29 +63,40 @@ curl -H "X-Proxy-Token: YOUR_SECRET" http://localhost:8080/elevenlabs/v1/voices
| /deepseek/ | https://api.deepseek.com |
| /health | local health check (no auth)|
## Listeners
| Host port | Container port | Purpose |
|-----------|----------------|------------------------------------------------------|
| 443 (tcp+udp) | 443 | Public HTTPS endpoint for `DOMAIN`, HTTP/3 enabled |
| 80 | 80 | ACME HTTP-01 challenge + redirect to HTTPS |
| 8082 | 8080 | Plain HTTP for legacy in-cluster clients |
Both listeners serve the same routes and the same access control.
---
## TLS / HTTPS
To enable HTTPS on port 8443, place your certificate and key in the `ssl/` directory:
Caddy obtains a certificate for `DOMAIN` on first start and renews it automatically (~30 days before expiry).
```bash
mkdir -p ssl
Certificates and the ACME account key live in the `caddy_data` volume mounted at `/data`. **Never drop this volume** — recreating it re-issues certificates and burns the Let's Encrypt rate limit (5 failures/hour, 50 certs/week per domain).
# Option A: self-signed cert
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
-keyout ssl/tls.key \
-out ssl/tls.crt \
-subj "/CN=api-proxy"
To rehearse against the staging CA, add to the global block of `Caddyfile`:
# Option B: copy existing certs
cp /path/to/your/cert.pem ssl/tls.crt
cp /path/to/your/key.pem ssl/tls.key
docker compose up -d --build
```caddyfile
{
email {$ACME_EMAIL}
acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
}
```
The entrypoint auto-detects `ssl/tls.crt` and `ssl/tls.key`. If present — HTTPS is enabled on port 8443. If absent — only HTTP on 8080.
Check the certificate state:
```bash
docker exec api-proxy ls /data/caddy/certificates/acme-v02.api.letsencrypt.org-directory/
echo | openssl s_client -connect proxy.bildme.ru:443 -servername proxy.bildme.ru 2>/dev/null \
| openssl x509 -noout -subject -issuer -dates
```
---
@@ -79,47 +104,19 @@ The entrypoint auto-detects `ssl/tls.crt` and `ssl/tls.key`. If present — HTTP
Example: adding `api.anthropic.com` → `/anthropic/*`
Add this block to `nginx/locations.conf`:
Add one line to the `(routes)` snippet in `Caddyfile`:
```nginx
location /anthropic/ {
if ($allowed_ip = 0) {
return 403 '{"error":"ip_not_allowed"}';
}
if ($auth_ok = 0) {
return 403 '{"error":"invalid_token"}';
}
set $anthropic_upstream https://api.anthropic.com;
rewrite ^/anthropic/(.*) /$1 break;
proxy_pass $anthropic_upstream;
proxy_ssl_server_name on;
proxy_ssl_name api.anthropic.com;
proxy_ssl_protocols TLSv1.2 TLSv1.3;
proxy_set_header Host api.anthropic.com;
proxy_set_header Connection "";
proxy_set_header X-Forwarded-For "";
proxy_set_header X-Real-IP "";
proxy_set_header True-Client-IP "";
proxy_set_header CF-Connecting-IP "";
proxy_set_header X-Client-IP "";
proxy_set_header Forwarded "";
proxy_set_header Via "";
proxy_set_header X-Proxy-Token "";
proxy_http_version 1.1;
proxy_buffering off;
proxy_request_buffering off;
proxy_read_timeout 120s;
proxy_send_timeout 120s;
}
```caddyfile
import api anthropic api.anthropic.com ""
```
Then rebuild: `docker compose up -d --build`
Arguments: `<url-prefix> <upstream-host> <upstream-path-prefix>`. The third argument is prepended to the proxied path — `""` for a plain passthrough, `/api/v1` for OpenRouter-style bases.
Update the catch-all hint in the same snippet, then rebuild:
```bash
docker-compose up -d --build
```
---
@@ -130,7 +127,7 @@ Then rebuild: `docker compose up -d --build`
```yaml
env:
- name: PROXY_BASE_URL
value: "http://10.0.1.50:8080" # or https://10.0.1.50:8443
value: "https://proxy.bildme.ru" # or http://<vm-ip>:8082 inside the private network
- name: PROXY_SECRET
valueFrom:
secretKeyRef:
@@ -156,30 +153,30 @@ const response = await fetch(
);
const buffer = Buffer.from(await response.arrayBuffer());
// Telegram Bot API
const tgResponse = await fetch(
`${process.env.PROXY_BASE_URL}/telegram/bot${process.env.TELEGRAM_BOT_TOKEN}/sendMessage`,
{
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-Proxy-Token': process.env.PROXY_SECRET,
},
body: JSON.stringify({ chat_id: chatId, text: message }),
}
);
// DeepSeek (OpenAI-compatible)
const ds = await fetch(`${process.env.PROXY_BASE_URL}/deepseek/chat/completions`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${process.env.DEEPSEEK_API_KEY}`,
'X-Proxy-Token': process.env.PROXY_SECRET,
},
body: JSON.stringify({ model: 'deepseek-chat', messages }),
});
```
For OpenAI-compatible SDKs set `baseURL` to `https://proxy.bildme.ru/deepseek` and pass `X-Proxy-Token` as a default header.
---
## Security Checklist
- [ ] `PROXY_SECRET` generated via `openssl rand -hex 32`
- [ ] `ALLOWED_CIDR` restricted to cluster CIDR only
- [ ] Port 8080/8443 closed to the public (only k8s CIDR in firewall/NSG)
- [ ] VM has no public IP or is behind NAT
- [ ] Port 8082 closed to the public (only k8s CIDR in firewall/NSG) — it carries API keys in cleartext
- [ ] Clients migrated to `https://DOMAIN`, then port 8082 retired
- [ ] `caddy_data` volume backed up (contains the ACME account key)
- [ ] Logs rotate (Docker logging driver configured: 50m x 5 files)
- [ ] TLS certs have restricted permissions (`chmod 600 ssl/tls.key`)
---
@@ -187,11 +184,11 @@ const tgResponse = await fetch(
```bash
# Tail logs
docker compose logs -f proxy | jq .
docker-compose logs -f proxy | jq .
# Requests per upstream per minute
docker compose logs --since=1m proxy --no-log-prefix \
| jq -r '.uri' \
docker-compose logs --since=1m proxy --no-log-prefix \
| jq -r 'select(.logger=="http.log.access") | .request.uri' \
| cut -d'/' -f2 \
| sort | uniq -c | sort -rn
@@ -203,8 +200,9 @@ docker inspect --format='{{.State.Health.Status}}' api-proxy
## How It Works
1. A k8s pod sends an HTTP request to `http://<proxy-vm>:8080/elevenlabs/v1/text-to-speech/...` with the `X-Proxy-Token` header and the original API key.
2. Nginx checks the source IP against `ALLOWED_CIDR` and validates `X-Proxy-Token` — rejecting with 403 if either fails.
3. The prefix (`/elevenlabs/`, `/openai/`, `/telegram/`, `/openrouter/`, `/deepseek/`) is stripped via rewrite, and the request is forwarded over HTTPS to the upstream with SNI enabled.
4. `X-Proxy-Token` and all IP-leaking headers are removed before forwarding; API keys pass through unchanged.
5. The response streams back unbuffered to the pod — critical for binary audio data and long-polling (Telegram).
1. A k8s pod sends a request to `https://proxy.bildme.ru/elevenlabs/v1/text-to-speech/...` (or `http://<vm-ip>:8082/...`) with the `X-Proxy-Token` header and the original API key.
2. Caddy checks the source IP against `ALLOWED_CIDR` and validates `X-Proxy-Token` — rejecting with 403 if either fails. `/health` is answered before both guards.
3. `handle_path` strips the prefix (`/elevenlabs/`, `/openai/`, `/telegram/`, `/openrouter/`, `/deepseek/`) and the request is forwarded over HTTPS with the upstream host as both `Host` header and TLS SNI.
4. `X-Proxy-Token` and all IP-leaking headers (`X-Forwarded-*`, `X-Real-IP`, `True-Client-IP`, `CF-Connecting-IP`, `X-Client-IP`, `Forwarded`, `Via`) are removed before forwarding; API keys pass through unchanged.
5. Responses stream back unbuffered (`flush_interval -1`) — critical for binary audio, SSE token streams and Telegram long-polling.
6. Access control fragments (`matchers.caddy`, `guard.caddy`) are generated from environment variables at container start by `docker-entrypoint.sh`.