2

我一直在尝试设置一个面向外部的 GRPC 支付微服务客户端,并使用 tls 自动更新证书。

到目前为止,我已经正确设置了带有证书续订的 certmanager,但是我的网关似乎没有正确转发流量,因为这kubectl -n istio-system describe challenge payments-cert表明挑战由于HTTP 404被退回而出错。

我曾经grpcurl测试过服务的活跃度,结果是这样的:

Failed to dial target host "payments.mywebsite.com:443": read tcp 192.168.0.16:58849 ->xx.xx.xxx.xx:443: read: connection reset by peer

这表明ip已经被dns正确注册,并且地址确实到达了443,所以我的Gateway -> VirtualService -> Service -> Deployment设置肯定有问题。

在描述 istio ingress ( kubectl get svc -n istio-system istio-ingressgateway) 时,我得到:

istio-ingressgateway LoadBalancer 10.0.13.184 xx.xx.xx.xx ...443:31390/TCP...

哪个似乎没有转发443到预期的端口?

我想让流量进入“payments.mywebsite.com:443”(使用 let-encrypt 自动更新证书)以路由到容器端口上的我的 grpc 后端50051。我一直很难破译Istio文档和资源来完成这项工作!

任何帮助都将受到热烈欢迎!

网关

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: payments-gateway
  namespace: default
spec:
  selector:
    istio: ingressgateway
  servers:
  - hosts:
    - payments.mywebsite.com
    port:
      name: payments-gateway-https
      number: 443
      protocol: HTTPS
    tls:
      credentialName: payments-cert
      mode: SIMPLE
      privateKey: sds
      serverCertificate: sds

虚拟服务

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: payments-virtual
spec:
  hosts:
  - "payments.mywebsite.com"
  gateways:
  - payments-gateway
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        port:
          number: 50051
        host: payments-service

服务

apiVersion: v1
kind: Service
metadata:
  name: payments-service
  labels:
    app: payments-service
spec:
  ports:
  - port: 50051
    name: grpc
  selector:
    app: payments-server

部署

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app: payments-server
  name: payments-server
spec:
  selector:
    matchLabels:
      app: payments-server
  replicas: 1
  strategy: {}
  template:
    metadata:
      labels:
        app: payments-server
        version: v1
    spec:
      containers:
      - image: gcr.io/mywebsite/payments_server:v1
        name: payments-server
        ports:
        - containerPort: 50051
        resources: {}
      restartPolicy: Always
status: {}

编辑 1:从网关添加日志

2020-01-21T14:31:48.841505Z     error   k8s.io/client-go@v11.0.1-0.20190409021438-1a26190bd76a+incompatible/tools/cache/reflector.go:98: Failed to list *v1.Secret: Get https://10.0.0.1:443/api/v1/namespaces/istio-system/secrets?limit=500&resourceVersion=0: net/http: TLS handshake timeout
2020-01-21T14:32:13.173045Z     info    secretFetcherLog        scrtUpdated is called on kubernetes secret payments-cert
2020-01-21T14:32:13.173124Z     warn    secretFetcherLog        failed load server cert/key pair from secret payments-cert: server cert or private key is empty
2020-01-21T14:32:13.173131Z     warn    secretFetcherLog        failed load server cert/key pair from secret payments-cert: server cert or private key is empty
2020-01-21T14:32:13.173137Z     info    secretFetcherLog        secret payments-cert does not change, skip update
2020-01-21T14:32:13.173257Z     info    Trace[1103410]: "Reflector k8s.io/client-go@v11.0.1-0.20190409021438-1a26190bd76a+incompatible/tools/cache/reflector.go:98 ListAndWatch" (started: 2020-01-21 14:31:49.841728972 +0000 UTC m=+44333.359224094) (total time: 23.331269811s):`

编辑 2:添加了代理边车日志(省略了不记名令牌)

2020-01-21T14:47:51.784617Z     info    curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: sidecar-injector/v0.0.0 (linux/amd64) kubernetes/$Format" -H "Authorization: Bearer ***" 'https://10.0.0.1:443/apis/admissionregistration.k8s.io/v1beta1/mutatingwebhookconfigurations?fieldSelector=metadata.name%3Distio-sidecar-injector&resourceVersion=299045&timeoutSeconds=449&watch=true'
2020-01-21T14:47:51.787107Z     info    GET https://10.0.0.1:443/apis/admissionregistration.k8s.io/v1beta1/mutatingwebhookconfigurations?fieldSelector=metadata.name%3Distio-sidecar-injector&resourceVersion=299045&timeoutSeconds=449&watch=true 200 OK in 2 milliseconds
2020-01-21T14:47:51.787126Z     info    Response Headers:
2020-01-21T14:47:51.787132Z     info        Audit-Id: 21862701-0185-4e25-b26e-ff78ddb6de1e
2020-01-21T14:47:51.787136Z     info        Content-Type: application/json
2020-01-21T14:47:51.787140Z     info        Date: Tue, 21 Jan 2020 14:47:51 GMT

编辑 3:支付证书结构和集群发行者

apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
  name: payments-cert
  namespace: istio-system
spec:
  commonName: payments.mywebsite.com
  dnsNames:
  - payments.mywebsite.com
  issuerRef:
    kind: ClusterIssuer
    name: letsencrypt-prod
  secretName: payments-cert

---

apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
 labels:
   name: letsencrypt-prod
 name: letsencrypt-prod
 namespace: istio-system
spec:
 acme:
   email: mywebsite@gmail.com
   solvers:
    - http01:
        ingress:
          class:  istioIngress
   privateKeySecretRef:
     name: letsencrypt-staging
   server: https://acme-v02.api.letsencrypt.org/directory

编辑 4:describe payment-cert 的输出(+ istio sidecar injector 的附加日志)

Name:         payments-cert
Namespace:    istio-system
Labels:       <none>
Annotations:  cert-manager.io/certificate-name: payments-cert
              cert-manager.io/issuer-kind: ClusterIssuer
              cert-manager.io/issuer-name: letsencrypt-prod

Type:  kubernetes.io/tls

Data
====
tls.crt:  0 bytes
tls.key:  1679 bytes
ca.crt:   0 bytes

我还查看了谷歌云上 istio sidecar 注入器发出的事件,记录了这些日志:

(Message): No matching pods found   
(Reason): NoPods    
(First Seen): Jan 21, 2020, 2:32:47 PM  
(Last Seen): Jan 21, 2020, 3:28:17 PM   
(counts): 112 
4

1 回答 1

1

istio 网关日志出错的原因failed load server cert/key pair from secret payments-cert: server cert or private key is emptytls.crt秘密为空 payments-cert。证书管理器生成证书时存在一些问题。

另一件事是您在证书颁发者中混合了letsencrypt-staging 和letsencrypt-prod。

这里还有一个您可以遵循的示例。

于 2020-01-21T15:52:32.280 回答