mirror of
https://github.com/wahyd4/home-docker.git
synced 2026-08-08 20:15:03 +10:00
Add k3s Skills.md
This commit is contained in:
@@ -0,0 +1,344 @@
|
||||
---
|
||||
name: home-k3s-deploy
|
||||
description: Deploy and manage apps on junv's home K3s cluster at 192.168.1.2 (server-3). Use this skill when creating new Kubernetes manifests, adding services, configuring ingress/TLS, managing storage, or modifying existing deployments in this homelab cluster. Covers namespace conventions, storage patterns, ingress annotations, TLS, OAuth2 auth, node affinity, and apply workflow.
|
||||
compatibility: Requires kubectl configured for 192.168.1.2; manifests live in /Users/junv/code/home-docker
|
||||
metadata:
|
||||
author: junv
|
||||
cluster: server-3 (192.168.1.2)
|
||||
domain: "*.junv.cc"
|
||||
---
|
||||
|
||||
# Home K3s Deployment Skill
|
||||
|
||||
## Cluster Overview
|
||||
|
||||
- **Node**: `server-3` at `192.168.1.2` (single-node K3s cluster)
|
||||
- **K3s version**: v1.33.3+k3s1, Ubuntu 24.04.3
|
||||
- **SSH**: `ssh -p 22422 junv@192.168.1.2` (Ed25519 key)
|
||||
- **Primary domain**: `*.junv.cc` (Cloudflare DNS + cert-manager Let's Encrypt)
|
||||
- **Local network**: `192.168.1.0/24` (trusted, never block)
|
||||
- **Kubeconfig**: `scp -P 22422 junv@192.168.1.2:/etc/rancher/k3s/k3s.yaml ~/.kube/config && sed -i '' 's/127.0.0.1/192.168.1.2/g' ~/.kube/config`
|
||||
|
||||
## Namespace Organization
|
||||
|
||||
| Namespace | Purpose |
|
||||
|-----------|---------|
|
||||
| `home-apps` | Home automation, productivity, personal tools |
|
||||
| `media` | Media servers (alist, static files) |
|
||||
| `db` | Databases: Redis (`db` ns), Qdrant vector DB |
|
||||
| `ai` | AI/ML workloads |
|
||||
| `argocd` | GitOps deployment (ArgoCD) |
|
||||
| `ingress-nginx` | NGINX Ingress Controller |
|
||||
| `cert-manager` | TLS certificate management |
|
||||
| `crowdsec` | Security monitoring |
|
||||
| `default` | Prometheus/Grafana monitoring |
|
||||
|
||||
## Repository File Layout
|
||||
|
||||
New manifests go in the directory matching the app's namespace:
|
||||
|
||||
```
|
||||
home-docker/
|
||||
├── home-apps/ → namespace: home-apps
|
||||
├── media/ → namespace: media
|
||||
├── db/ → namespace: db
|
||||
├── ai/ → namespace: ai
|
||||
├── adhoc-config/ → one-off PVs, config patches
|
||||
├── terraform/ → infra-as-code (prefer for cluster-level changes)
|
||||
└── archive/ → deprecated configs (do not edit)
|
||||
```
|
||||
|
||||
## Adding a New App: Step-by-Step
|
||||
|
||||
1. Choose the correct directory and namespace (see table above).
|
||||
2. Create a single YAML file `{app-name}.yaml` with all resources separated by `---`.
|
||||
3. Resource order in the file: PV → PVC → Deployment → Service → Ingress.
|
||||
4. If local storage is needed, also create `adhoc-config/{app-name}-local-pv.yaml`.
|
||||
5. Apply: `kubectl apply -f {file}.yaml` (or `kubectl apply -f adhoc-config/{app-name}-local-pv.yaml` first).
|
||||
|
||||
## Storage Patterns
|
||||
|
||||
### Local Storage (most common for new apps)
|
||||
|
||||
Use `local-storage` storageClass with node affinity pinned to `server-3`. Data path: `/mnt/k8s/{app-name}/`.
|
||||
|
||||
Create the directory first on the node: `ssh -p 22422 junv@192.168.1.2 "sudo mkdir -p /mnt/k8s/{app-name}"`
|
||||
|
||||
```yaml
|
||||
# adhoc-config/{app-name}-local-pv.yaml
|
||||
apiVersion: v1
|
||||
kind: PersistentVolume
|
||||
metadata:
|
||||
name: {app-name}-local-pv
|
||||
labels:
|
||||
type: local
|
||||
name: {app-name}
|
||||
spec:
|
||||
capacity:
|
||||
storage: 5Gi
|
||||
volumeMode: Filesystem
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
persistentVolumeReclaimPolicy: Delete
|
||||
storageClassName: local-storage
|
||||
local:
|
||||
path: /mnt/k8s/{app-name}
|
||||
nodeAffinity:
|
||||
required:
|
||||
nodeSelectorTerms:
|
||||
- matchExpressions:
|
||||
- key: kubernetes.io/hostname
|
||||
operator: In
|
||||
values:
|
||||
- server-3
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
namespace: {namespace}
|
||||
name: {app-name}-local-pvc
|
||||
spec:
|
||||
volumeName: {app-name}-local-pv
|
||||
storageClassName: "local-storage"
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 5Gi
|
||||
```
|
||||
|
||||
### NFS Storage (for large or shared data)
|
||||
|
||||
NFS server: `192.168.1.5`, base path: `/fs/1000/nfs/k8s/`. Use `storageClassName: ""` (empty) with no volumeName binding.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: PersistentVolume
|
||||
metadata:
|
||||
name: {app-name}-nfs-pv
|
||||
spec:
|
||||
capacity:
|
||||
storage: 10Gi
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
nfs:
|
||||
server: 192.168.1.5
|
||||
path: "/fs/1000/nfs/k8s/{app-name}"
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
namespace: {namespace}
|
||||
name: {app-name}-nfs-pvc
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
storageClassName: ""
|
||||
resources:
|
||||
requests:
|
||||
storage: 10Gi
|
||||
```
|
||||
|
||||
## Deployment Pattern
|
||||
|
||||
Always set `revisionHistoryLimit: 2`. Pin the timezone. For images from Docker Hub that may be rate-limited, use `mirror.gcr.io/` prefix (e.g., `mirror.gcr.io/library/nginx:latest`).
|
||||
|
||||
```yaml
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: {app-name}
|
||||
namespace: {namespace}
|
||||
labels:
|
||||
app: {app-name}
|
||||
spec:
|
||||
replicas: 1
|
||||
revisionHistoryLimit: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: {app-name}
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: {app-name}
|
||||
spec:
|
||||
nodeSelector:
|
||||
kubernetes.io/hostname: server-3
|
||||
# If pulling from GitHub Container Registry:
|
||||
imagePullSecrets:
|
||||
- name: github-image-pull-secret
|
||||
containers:
|
||||
- name: {app-name}
|
||||
image: mirror.gcr.io/{org}/{image}:{tag}
|
||||
imagePullPolicy: IfNotPresent
|
||||
ports:
|
||||
- containerPort: {port}
|
||||
env:
|
||||
- name: TZ
|
||||
value: Australia/Melbourne
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /data
|
||||
resources:
|
||||
requests:
|
||||
cpu: 200m
|
||||
memory: 256Mi
|
||||
limits:
|
||||
cpu: "1"
|
||||
memory: 512Mi
|
||||
volumes:
|
||||
- name: data
|
||||
persistentVolumeClaim:
|
||||
claimName: {app-name}-local-pvc
|
||||
```
|
||||
|
||||
## Service Pattern
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: {app-name}
|
||||
namespace: {namespace}
|
||||
spec:
|
||||
type: ClusterIP
|
||||
selector:
|
||||
app: {app-name}
|
||||
ports:
|
||||
- name: web
|
||||
port: 80
|
||||
targetPort: {container-port}
|
||||
protocol: TCP
|
||||
```
|
||||
|
||||
For direct LAN access (no ingress), use `LoadBalancer` type with a static MetalLB IP:
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
type: LoadBalancer
|
||||
annotations:
|
||||
metallb.universe.tf/loadBalancerIPs: 192.168.1.2XX
|
||||
```
|
||||
|
||||
## Ingress Patterns
|
||||
|
||||
Always use `ingressClassName: nginx` and `cert-manager.io/cluster-issuer: "letsencrypt-prod"` for TLS.
|
||||
|
||||
### Public HTTPS (anyone on the internet)
|
||||
|
||||
```yaml
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: {app-name}-ingress
|
||||
namespace: {namespace}
|
||||
annotations:
|
||||
cert-manager.io/cluster-issuer: "letsencrypt-prod"
|
||||
spec:
|
||||
ingressClassName: nginx
|
||||
tls:
|
||||
- hosts:
|
||||
- {app-name}.junv.cc
|
||||
secretName: {app-name}-tls
|
||||
rules:
|
||||
- host: {app-name}.junv.cc
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: {app-name}
|
||||
port:
|
||||
number: 80
|
||||
```
|
||||
|
||||
### LAN-Only (IP whitelisted to home network)
|
||||
|
||||
Add the whitelist annotation — no OAuth needed:
|
||||
|
||||
```yaml
|
||||
annotations:
|
||||
cert-manager.io/cluster-issuer: "letsencrypt-prod"
|
||||
nginx.ingress.kubernetes.io/whitelist-source-range: "192.168.1.0/24"
|
||||
```
|
||||
|
||||
### OAuth2-Protected (via Pocket ID / passkey-auth)
|
||||
|
||||
Add these two annotations to require login via `pass.junv.cc`:
|
||||
|
||||
```yaml
|
||||
annotations:
|
||||
cert-manager.io/cluster-issuer: "letsencrypt-prod"
|
||||
nginx.ingress.kubernetes.io/auth-url: "https://pass.junv.cc/oauth2/auth"
|
||||
nginx.ingress.kubernetes.io/auth-signin: "https://pass.junv.cc/oauth2/start?rd=https://$host$escaped_request_uri"
|
||||
```
|
||||
|
||||
## Existing Secrets (reference, do not recreate)
|
||||
|
||||
| Secret Name | Namespace | Keys | Purpose |
|
||||
|-------------|-----------|------|---------|
|
||||
| `postgres-credential` | home-apps | `username`, `password` | PostgreSQL at 192.168.1.2:5432 |
|
||||
| `qdrant-api-key` | db | `key` | Qdrant vector DB API key |
|
||||
| `github-image-pull-secret` | home-apps | — | Pull from ghcr.io |
|
||||
|
||||
Use secrets via `secretKeyRef`:
|
||||
```yaml
|
||||
env:
|
||||
- name: DB_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: postgres-credential
|
||||
key: password
|
||||
```
|
||||
|
||||
## Infrastructure Services (in-cluster)
|
||||
|
||||
| Service | Address | Notes |
|
||||
|---------|---------|-------|
|
||||
| PostgreSQL | `192.168.1.2:5432` | Host-level install |
|
||||
| Redis | `192.168.1.2:6379` | Bitnami Helm chart, `db` namespace |
|
||||
| Qdrant | `qdrant.db.svc.cluster.local:6333` | Vector DB, `db` namespace |
|
||||
| ArgoCD | `https://argo.junv.cc` | GitOps; add apps via ArgoCD UI or Application CR |
|
||||
|
||||
## Common Conventions
|
||||
|
||||
- **Timezone**: Always set `TZ: Australia/Melbourne`
|
||||
- **Node pin**: `nodeSelector: kubernetes.io/hostname: server-3` for stateful workloads
|
||||
- **Image mirror**: Prefix Docker Hub images with `mirror.gcr.io/` to avoid rate limits
|
||||
- **Ingress class**: `ingressClassName: nginx` (not the legacy annotation)
|
||||
- **TLS issuer**: `letsencrypt-prod` ClusterIssuer (email: me@junv.cc)
|
||||
- **Namespace on PVC/Ingress**: Always specify `namespace:` — it's easy to forget
|
||||
- **Namespace on PV**: PersistentVolumes are cluster-scoped — do NOT add `namespace:`
|
||||
- **revisionHistoryLimit: 2**: Keep rollout history small
|
||||
|
||||
## Apply Workflow
|
||||
|
||||
```bash
|
||||
# Apply a new manifest
|
||||
kubectl apply -f home-apps/{app-name}.yaml
|
||||
|
||||
# Apply a local-PV config first (before the main manifest)
|
||||
kubectl apply -f adhoc-config/{app-name}-local-pv.yaml
|
||||
kubectl apply -f home-apps/{app-name}.yaml
|
||||
|
||||
# Check status
|
||||
kubectl get pods -n {namespace}
|
||||
kubectl describe pod -n {namespace} {pod-name}
|
||||
kubectl logs -n {namespace} -l app={app-name} --tail=50
|
||||
|
||||
# Restart a deployment
|
||||
kubectl rollout restart deployment/{app-name} -n {namespace}
|
||||
|
||||
# Get events (useful for debugging PVC bind issues)
|
||||
kubectl get events -n {namespace} --sort-by='.lastTimestamp'
|
||||
```
|
||||
|
||||
## Do NOT
|
||||
|
||||
- Delete files without user confirmation
|
||||
- Use `iptables` integration in CrowdSec (breaks kubectl access to port 6443)
|
||||
- Use inotify for container log acquisition in CrowdSec (use `poll_without_inotify: true`)
|
||||
- Hardcode secrets in manifests — always use `secretKeyRef`
|
||||
- Block `192.168.1.0/24` — it's the trusted local network
|
||||
- Use SSH port 22 — the correct port is `22422`
|
||||
Reference in New Issue
Block a user