K8s部署jumpserver

AI 摘要

文章讲述在Kubernetes集群中部署JumpServer的两种方法，使用Helm安装，涉及Traefik配置、Metallb负载均衡、依赖的PostgreSQL与Redis，并解决GitLab占用22端口后改用2222的SSH访问，以及外部数据库场景下的配置与验证。

一、环境准备

1. kubenetes
2. k8s
3. harbor
4. dnsmasq
5. helm
6. cert-manager
7. traefik
8. metallb

gitlab 占用了 ssh 22 端口，这里为了图方便，不换 ip 了，就改用默认的 2222 端口。

二、安装【法一】

2.1 traefik 配置

1. 添加 helm 仓库

helm repo add jumpserver https://jumpserver.github.io/helm-charts
helm repo update jumpserver

1. 给 traefik 添加端口的配置

vim /opt/selfhost/infra-deployment/traefik/v3-traefik-ssh.yaml

# v3-traefik-ssh.yaml
ingressClass:
  enabled: true
  isDefaultClass: true       # 设为默认 ingressClass，Ingress 不写 class 也走 Traefik

api:
  dashboard: true            # 打开 dashboard（下一步用我们自己的 IngressRoute 暴露它）

ingressRoute:
  dashboard:
    enabled: false           # 关掉自带的 dashboard 路由，避免和我们带 TLS 的那个冲突

# service 默认就是 LoadBalancer 类型，会自动向 MetalLB 要一个 IP
ports:                       # 新增的顶层块，顶格写
  ssh:
    port: 2222               # Traefik 容器内监听端口
    exposedPort: 22          # 对外(MetalLB VIP)端口
    protocol: TCP
    expose:
      default: true          # 新版 chart 写法
  jms-ssh:
    port: 2223               # 容器内端口，必须和上面 ssh 的 2222 错开！
    exposedPort: 2222        # 对外 VIP 端口
    protocol: TCP
    expose:
      default: true

1. 更新 traefik

helm upgrade traefik traefik/traefik -n traefik -f /opt/selfhost/infra-deployment/traefik/v3-traefik-ssh.yaml

1. 验证

kubectl -n traefik get svc traefik -o wide

2.2 jms 依赖的组件

主要是 pgsql、redis、命名空间、ingress 等

1. 编辑资源清单文件

vim jumpserver-stack.yaml

# ============================================================
# jumpserver-stack.yaml
# 集群内依赖 (PG + Redis) + Web Ingress + SSH 的 TCP 路由
#
# 改动点：
#   - nfs-client  -> 你的 StorageClass (k3s: local-path)
#   - jms.qx.lab            -> 你的域名
#   - qx-ca-issuer          -> 你的 cert-manager ClusterIssuer 名（已按你给的填好）
# 注意 PG/Redis 密码要和 values.yaml 完全一致（已预填好）。
#
# 应用顺序见聊天里的 runbook，不要一次性全 apply。
# ============================================================
apiVersion: v1
kind: Namespace
metadata:
  name: jumpserver
---
# ========================= PostgreSQL =========================
apiVersion: v1
kind: Secret
metadata:
  name: jms-postgresql
  namespace: jumpserver
type: Opaque
stringData:
  POSTGRES_DB: jumpserver
  POSTGRES_USER: jumpserver
  POSTGRES_PASSWORD: "u4bWW0RTZMFeqVPGI315"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jms-postgresql-data
  namespace: jumpserver
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: nfs-client
  resources:
    requests:
      storage: 10Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: jms-postgresql
  namespace: jumpserver
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: jms-postgresql
  template:
    metadata:
      labels:
        app: jms-postgresql
    spec:
      containers:
        - name: postgresql
          image: postgres:16-alpine
          envFrom:
            - secretRef:
                name: jms-postgresql
          env:
            # 数据放到子目录，避免挂载卷里的 lost+found 干扰初始化
            - name: PGDATA
              value: /var/lib/postgresql/data/pgdata
          ports:
            - containerPort: 5432
          readinessProbe:
            exec:
              command: ["pg_isready", "-U", "jumpserver", "-d", "jumpserver"]
            initialDelaySeconds: 10
            periodSeconds: 5
          volumeMounts:
            - name: data
              mountPath: /var/lib/postgresql/data
      volumes:
        - name: data
          persistentVolumeClaim:
            claimName: jms-postgresql-data
---
apiVersion: v1
kind: Service
metadata:
  name: jms-postgresql
  namespace: jumpserver
spec:
  selector:
    app: jms-postgresql
  ports:
    - port: 5432
      targetPort: 5432
---
# =========================== Redis ===========================
apiVersion: v1
kind: Secret
metadata:
  name: jms-redis
  namespace: jumpserver
type: Opaque
stringData:
  REDIS_PASSWORD: "62rnrIY03ilA8TZF3vI3"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: jms-redis
  namespace: jumpserver
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jms-redis
  template:
    metadata:
      labels:
        app: jms-redis
    spec:
      containers:
        - name: redis
          image: redis:7-alpine
          args: ["--requirepass", "$(REDIS_PASSWORD)"]
          env:
            - name: REDIS_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: jms-redis
                  key: REDIS_PASSWORD
          ports:
            - containerPort: 6379
          readinessProbe:
            exec:
              command: ["sh", "-c", "redis-cli -a \"$REDIS_PASSWORD\" ping"]
            initialDelaySeconds: 5
            periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: jms-redis
  namespace: jumpserver
spec:
  selector:
    app: jms-redis
  ports:
    - port: 6379
      targetPort: 6379
---
# ===================== Web Ingress =====================
# 标准 Ingress：Traefik 接管 + cert-manager 按注解自动签发证书到 jms-qx-lab-tls
# Traefik 原生支持 WebSocket，JumpServer 终端无需额外注解
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: jms-web
  namespace: jumpserver
  annotations:
    cert-manager.io/cluster-issuer: qx-ca-issuer
spec:
  ingressClassName: traefik
  tls:
    - hosts:
        - jms.qx.lab
      secretName: jms-qx-lab-tls
  rules:
    - host: jms.qx.lab
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: jms-jumpserver-jms-web
                port:
                  number: 80
---
# ===================== SSH 路由 =====================
# 普通 Ingress 转不了 SSH，这里用 Traefik 的 IngressRouteTCP
# 走自定义 TCP entrypoint jms-ssh(2222) -> koko 的 2222
# 非 TLS 的 TCP 必须 HostSNI(`*`)
apiVersion: traefik.io/v1alpha1
kind: IngressRouteTCP
metadata:
  name: jms-ssh
  namespace: jumpserver
spec:
  entryPoints:
    - jms-ssh
  routes:
    - match: HostSNI(`*`)
      services:
        - name: jms-jumpserver-jms-koko
          port: 2222

1. 创建资源

kubectl apply -f jumpserver-stack.yaml

1. 验证

kubectl -n jumpserver rollout status deploy/jms-postgresql
kubectl -n jumpserver rollout status deploy/jms-redis

2.3 helm 安装 jumpserver

1. 创建 values.yaml

vim values.yaml

# ============================================================
# JumpServer Helm values
#   helm repo add jumpserver https://jumpserver.github.io/helm-charts
#   helm install jms jumpserver/jumpserver -n jumpserver -f values.yaml
#
# 需要你改的地方只有两处：
#   1) 把所有 nfs-client 换成你集群的 StorageClass
#      (k3s 默认是 local-path，用 `kubectl get sc` 查看)
#   2) 域名 jms.qx.lab 换成你自己的
# 密码/密钥已随机生成，和 jumpserver-stack.yaml 里的保持一致即可，不用改。
# ============================================================

# 关闭 chart 自带的 ingress（它是给 nginx-ingress 用的），改用 Traefik IngressRoute
ingress:
  enabled: false

# ---- 集群内 PostgreSQL（由 jumpserver-stack.yaml 部署）----
externalDatabase:
  engine: postgresql
  host: jms-postgresql          # 同 namespace 内的 Service 名
  port: 5432
  user: jumpserver
  password: "u4bWW0RTZMFeqVPGI315"
  database: jumpserver

# ---- 集群内 Redis（由 jumpserver-stack.yaml 部署）----
externalRedis:
  host: jms-redis
  port: 6379
  password: "62rnrIY03ilA8TZF3vI3"

core:
  config:
    secretKey: "E4TbGLMRiSLSORtZpDaOeAFcZE4oppgKcsx1EZb55l169AY9WV"
    bootstrapToken: "E0BQP33Vm6GQ2NNpwf037R1i"
    log:
      level: ERROR
  env:
    SESSION_EXPIRE_AT_BROWSER_CLOSE: true
    # v3/v4 必须设置可信域名，否则登录报 CSRF 403
    DOMAINS: "jms.qx.lab:443"
  persistence:
    storageClassName: nfs-client
    accessModes:
      - ReadWriteMany
    size: 10Gi

koko:
  service:
    type: ClusterIP             # 不直接暴露，交给 Traefik TCP 转发
    web:
      port: 5000
    ssh:
      port: 2222
  persistence:
    storageClassName: nfs-client
    accessModes:
      - ReadWriteMany
    size: 10Gi

lion:
  persistence:
    storageClassName: nfs-client
    accessModes:
      - ReadWriteMany
    size: 10Gi

chen:
  persistence:
    storageClassName: nfs-client
    accessModes:
      - ReadWriteMany
    size: 5Gi

web:
  persistence:
    storageClassName: nfs-client
    accessModes:
      - ReadWriteMany
    size: 1Gi

# 企业版组件，社区版保持关闭
xpack:
  enabled: false

1. 安装

helm install jms jumpserver/jumpserver -n jumpserver -f values.yaml

1. 验证

kubectl -n jumpserver get pods -w

二、安装【法二】

2026年8月27日补充，第二次部署，集群里没有 gitlab，数据库、redis 都放到了集群外.

2.1 更新 traefik

1. 复制配置文件

cp traefik-values.yaml traefik-values-with-ssh.yaml

1. 编辑配置文件

vim traefik-values-with-ssh.yaml

# traefik-values-with-ssh.yaml
# 安装时指定的配置
ingressClass:
  enabled: true
  isDefaultClass: true       # 设为默认 ingressClass，Ingress 不写 class 也走 Traefik

api:
  dashboard: true            # 打开 dashboard（下一步用我们自己的 IngressRoute 暴露它）

ingressRoute:
  dashboard:
    enabled: false           # 关掉自带的 dashboard 路由，避免和我们带 TLS 的那个冲突

# service 默认就是 LoadBalancer 类型，会自动向 MetalLB 要一个 IP
# 新增暴露 22 端口
ports:
  ssh:
    port: 2222          # 容器内监听（非 root 用户不能绑 <1024）
    exposedPort: 22     # Service 对外暴露的端口
    protocol: TCP
    expose:
      default: true

1. 更新 traefik

helm upgrade traefik traefik/traefik \
  -n traefik --version 41.3.0 -f  traefik-values-with-ssh.yaml

1. 验证 22 端口

kubectl get svc -n traefik traefik -o wide

2.2 创建数据库

1. 创建用户，限制连接数，创建数据库

CREATE ROLE jumpserver WITH LOGIN PASSWORD 'jq3lW54MgXQFl7MXOkoY';

ALTER ROLE jumpserver CONNECTION LIMIT 20;

CREATE DATABASE jumpserver
    OWNER = jumpserver
    ENCODING = 'UTF8'
    TEMPLATE = template0;

1. 数据库层面 pg_hba.conf 放行

host  jumpserver  jumpserver  10.244.0.0/16  scram-sha-256

1. 验证 pgsql

kubectl run pgtest --rm -it --image=harbor.qx.lab/dockerhub/library/postgres:17 --restart=Never --   psql "postgresql://jumpserver:jq3lW54MgXQFl7MXOkoY@192.168.10.1:5432/jumpserver" -c '\conninfo'

1. 验证 redis

kubectl run redistest --rm -it --image=harbor.qx.lab/dockerhub/library/redis:7 --restart=Never -- \
  redis-cli -h 192.168.10.1 -a '123456' ping

2.3 准备 secrets

这里的 charts 不支持使用密钥。

1. 创建命名空间

kubectl create ns jumpserver

1. 创建 secrets

kubectl create secret generic jms-secret -n jumpserver \
  --from-literal=db-password='jq3lW54MgXQFl7MXOkoY' \
  --from-literal=redis-password='123456' \
  --from-literal=secret-key="$(openssl rand -hex 32)" \
  --from-literal=bootstrap-token="$(openssl rand -hex 16)"

2.4 查看 helm

1. 添加仓库，查看仓库

helm repo add jumpserver https://jumpserver.github.io/helm-charts
helm repo update
helm search repo jumpserver --versions | head

1. 查看字段

helm show values jumpserver/jumpserver --version v4.10.19 > values-ref.yaml

1. 筛选数据库相关的字段

grep -nE 'external|postgres|redis|Database|persistence' values-ref.yaml

2.5 安装

1. 设置字段

vim jumpserver-values.yaml

# jumpserver-values.yaml
postgresql:
  enabled: false
redis:
  enabled: false

externalDatabase:
  engine: postgresql
  host: 192.168.10.1
  port: 5432
  user: jumpserver
  password: "jq3lW54MgXQFl7MXOkoY"
  database: jumpserver

externalRedis:
  host: 192.168.10.1
  port: 6379
  password: "123456"

core:
  config:
    secretKey: "bedfe1ca79f5e1daf62c14ff6554443730a117edacc94f95760baa1a3b6eab82"
    bootstrapToken: "a6e6f1f55313b8dd77f19be7b1dc49b5"
  persistence:
    storageClassName: nfs-client

koko:
  persistence:
    storageClassName: nfs-client

lion:
  persistence:
    storageClassName: nfs-client

chen:
  persistence:
    storageClassName: nfs-client

web:
  persistence:
    storageClassName: nfs-client

ingress:
  enabled: false

1. 渲染检查

helm template jumpserver jumpserver/jumpserver \
  -n jumpserver -f jumpserver-values.yaml | grep -A2 'DB_HOST\|REDIS_HOST'

1. 安装

helm install jumpserver jumpserver/jumpserver \
  -n jumpserver --version v4.10.19 -f jumpserver-values.yaml

1. 验证

kubectl get pods -n jumpserver -w

2.6 添加域名访问、证书、ssh

1. 查看 svc

kubectl get svc -n jumpserver

1. 添加路由，需要和 svc 的信息保持一致

vim jms-routes.yaml

# jms-routes.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: jumpserver
  namespace: jumpserver
  annotations:
    cert-manager.io/cluster-issuer: qx.lab-ca-issuer
    traefik.ingress.kubernetes.io/router.entrypoints: websecure
spec:
  ingressClassName: traefik
  tls:
    - hosts: [jms.qx.lab]
      secretName: jumpserver-tls
  rules:
    - host: jms.qx.lab
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: jumpserver-jms-web
                port:
                  number: 80
---
apiVersion: traefik.io/v1alpha1
kind: IngressRouteTCP
metadata:
  name: jumpserver-ssh
  namespace: jumpserver
spec:
  entryPoints: [ssh]
  routes:
    - match: HostSNI(`*`)          # SSH 没有 SNI，只能通配
      services:
        - name: jumpserver-jms-koko
          port: 2222

1. 创建资源并验证

kubectl apply -f jms-routes.yaml
kubectl get certificate -n jumpserver     # READY=True

三、验证

1. 浏览器访问验证

地址: https://jms.qx.lab
用户名: admin
密码: ChangeMe

1. ssh验证

ssh -p 2222 admin@jms.qx.lab