Kibana не доверяет Let's Encrypt CA в настройке Kubernetes Helm.

У меня есть кластер k8s, настроенный с использованием calico в качестве сетевой системы. Я использую немного настроенные версии helm elastic / elasticsearch и elastic / kibana с включенной безопасностью. В целях безопасности я использую сертификаты Let's Encrypt. При запуске kibana происходит сбой подключения к экземпляру elasticsearch с этой ошибкой

{"type":"log","@timestamp":"2021-06-01T13:09:55+00:00","tags":["debug","elasticsearch","query","data"],"pid":952,"message":"[ConnectionError]: unable to get issuer certificate"}

Я могу решить эту проблему, отключив проверку сертификата в kibana. Можно. кто-нибудь понимает, почему это не удается?

создание хранилища ключей

cat cert1.pem > store.pem
cat privkey1.pem >> store.pem
cat chain1.pem >> store.pem
cat fullchain1.pem >> store.pem
openssl pkcs12 -export -in store.pem -out keystore.pkcs12

values_elastic.yaml

replicas: 1
minimumMasterNodes: 1

esConfig:
   elasticsearch.yml: |
     xpack.security.transport.ssl.enabled: true
     xpack.security.transport.ssl.verification_mode: certificate
     xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/certs-gen/keystore.pkcs12
     xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/certs-gen/keystore.pkcs12
     xpack.security.http.ssl.enabled: true
     xpack.security.http.ssl.truststore.path: /usr/share/elasticsearch/config/certs-gen/keystore.pkcs12
     xpack.security.http.ssl.keystore.path: /usr/share/elasticsearch/config/certs-gen/keystore.pkcs12
     xpack.security.enabled: true
extraEnvs:
  - name: ELASTIC_PASSWORD
    valueFrom:
      secretKeyRef:
        name: elastic-credentials
        key: password
  - name: ELASTIC_USERNAME
    valueFrom:
      secretKeyRef:
        name: elastic-credentials
        key: username
secretMounts:
  - name: elastic-certificates
    secretName: elastic-certificates
    path: /usr/share/elasticsearch/config/certs-gen/
protocol: https
service:
  labels: {}
  labelsHeadless: {}
  type: NodePort
  nodePort: 30001
  annotations: {}
  httpPortName: http
  transportPortName: transport
  loadBalancerIP: ""
  loadBalancerSourceRanges: []
  externalTrafficPolicy: ""
clusterHealthCheckParams: "wait_for_status=yellow&timeout=1s"

values_kibana.yaml

elasticsearchHosts: "redacted its a TLD with appropriate port"

extraEnvs:
  - name: "NODE_OPTIONS"
    value: "--max-old-space-size=1800"
  - name: 'ELASTICSEARCH_USERNAME'
    valueFrom:
      secretKeyRef:
        name: elastic-credentials
        key: username
  - name: 'ELASTICSEARCH_PASSWORD'
    valueFrom:
      secretKeyRef:
        name: elastic-credentials
        key: password
  - name: 'KIBANA_ENCRYPTION_KEY'
    valueFrom:
      secretKeyRef:
        name: kibana
        key: encryptionkey
  - name: 'LOGGING_VERBOSE'
    value: "true"


secretMounts:
  - name: elastic-certificates
    secretName: elastic-certificates
    path: /usr/share/kibana/config/certs-gen/


kibanaConfig:
  kibana.yml: |
    server.ssl:
      enabled: true
      key: /usr/share/kibana/config/certs-gen/privkey1.pem
      certificate: /usr/share/kibana/config/certs-gen/fullchain1.pem
    elasticsearch.ssl:
      certificateAuthorities: /usr/share/kibana/config/certs-gen/fullchain1.pem
      verificationMode: certificate
    xpack.reporting.encryptionKey: ${KIBANA_ENCRYPTION_KEY}
    xpack.security.encryptionKey: ${KIBANA_ENCRYPTION_KEY}
    xpack.encryptedSavedObjects.encryptionKey: ${KIBANA_ENCRYPTION_KEY}

protocol: https

service:
  type: NodePort
  loadBalancerIP: ""
  port: 5601
  nodePort: 30002
  labels: {}
  annotations: {}
    # cloud.google.com/load-balancer-type: "Internal"
    # service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0
    # service.beta.kubernetes.io/azure-load-balancer-internal: "true"
    # service.beta.kubernetes.io/openstack-internal-load-balancer: "true"
    # service.beta.kubernetes.io/cce-load-balancer-internal-vpc: "true"
  loadBalancerSourceRanges: []
    # 0.0.0.0/0
  httpPortName: HTTP

kubectl get pv, pvc, nodes, pods, svc

NAME                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                                 STORAGECLASS   REASON   AGE
persistentvolume/elk-data   30Gi       RWO            Retain           Bound    default/elasticsearch-master-elasticsearch-master-0                           40m

NAME                                                                STATUS   VOLUME     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/elasticsearch-master-elasticsearch-master-0   Bound    elk-data   30Gi       RWO                           32m

NAME                 STATUS   ROLES                  AGE   VERSION
node/kubeloadbalan   Ready    control-plane,master   28h   v1.21.1

NAME                                    READY   STATUS    RESTARTS   AGE
pod/elasticsearch-master-0              1/1     Running   0          13m
pod/kibana-kibana-7fdbd7c66d-bg5xb      0/1     Running   0          7m1s
pod/nginx-deployment-868c6bb874-tsbg4   1/1     Running   0          40m

NAME                                    TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                         AGE
service/elasticsearch-master            NodePort    10.97.47.66     <none>        9200:30001/TCP,9300:32185/TCP   13m
service/elasticsearch-master-headless   ClusterIP   None            <none>        9200/TCP,9300/TCP               13m
service/kibana-kibana                   NodePort    10.101.78.90    <none>        5601:30002/TCP                  7m1s
service/kubernetes                      ClusterIP   10.96.0.1       <none>        443/TCP                         28h
service/nginx-service                   NodePort    10.100.231.27   <none>        80:30000/TCP                    40m

Изнутри контейнера:

kubectl exec pod/kibana-kibana-7fdbd7c66d-bg5xb -it bash

curl -k -u redacted:redacted https://redacted:30001

{
  "name" : "elasticsearch-master-0",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "iXEuqB7iQ9abptIZ_Gp1yg",
  "version" : {
    "number" : "7.13.0",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "5ca8591c6fcdb1260ce95b08a8e023559635c6f3",
    "build_date" : "2021-05-19T22:22:26.081971330Z",
    "build_snapshot" : false,
    "lucene_version" : "8.8.2",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}


curl  -u redacted:redacted https://redacted:30001
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

Это показывает, что даже curl не может проверить легитимность изнутри контейнера. Это может указывать на то, что проблема также находится в elasticsearch. Полный журнал Kibana представляет собой полный беспорядок из-за повышенной многословности. Могу выложить по запросу.

Я провел в этом несколько дней. Я выгорел и не знаю, что дальше.