Отладка соединения между Puppet и PuppetDB (в отдельных контейнерах)

Недавно серверы, на которых размещен наш puppet-сервер, вышли из строя.

После повторного развертывания контейнеров, похоже, возникла проблема с SSL.

2018-01-16T14:36:49.770274413Z Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem

2018-01-16T14:36:49.770278010Z  at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)

2018-01-16T14:36:49.770281700Z  at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1728)

2018-01-16T14:36:49.770285230Z  at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:304)

2018-01-16T14:36:49.770288860Z  at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296)

2018-01-16T14:36:49.770292535Z  at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1514)

2018-01-16T14:36:49.770296037Z  at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)

2018-01-16T14:36:49.770299517Z  at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026)

2018-01-16T14:36:49.770303285Z  at sun.security.ssl.Handshaker$1.run(Handshaker.java:966)

2018-01-16T14:36:49.770306850Z  at sun.security.ssl.Handshaker$1.run(Handshaker.java:963)

2018-01-16T14:36:49.770310430Z  at java.security.AccessController.doPrivileged(Native Method)

2018-01-16T14:36:49.770314068Z  at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1416)

2018-01-16T14:36:49.770317603Z  at org.apache.http.nio.reactor.ssl.SSLIOSession.doRunTask(SSLIOSession.java:283)

2018-01-16T14:36:49.770321175Z  at org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:353)

2018-01-16T14:36:49.770324797Z  ... 9 common frames omitted

2018-01-16T14:36:49.770328925Z Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed

2018-01-16T14:36:49.770336317Z  at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:352)

2018-01-16T14:36:49.770340178Z  at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:260)

2018-01-16T14:36:49.770344615Z  at sun.security.validator.Validator.validate(Validator.java:260)

2018-01-16T14:36:49.770350867Z  at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)

2018-01-16T14:36:49.770355767Z  at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:281)

2018-01-16T14:36:49.770359543Z  at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136)

2018-01-16T14:36:49.770363103Z  at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1501)

2018-01-16T14:36:49.770366760Z  ... 17 common frames omitted

2018-01-16T14:36:49.770370253Z Caused by: java.security.cert.CertPathValidatorException: timestamp check failed

2018-01-16T14:36:49.770373823Z  at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:135)

2018-01-16T14:36:49.770377522Z  at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:219)

2018-01-16T14:36:49.770381140Z  at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:140)

2018-01-16T14:36:49.770384758Z  at sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:79)

2018-01-16T14:36:49.770388458Z  at java.security.cert.CertPathValidator.validate(CertPathValidator.java:292)

2018-01-16T14:36:49.770392038Z  at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:347)

2018-01-16T14:36:49.770395575Z  ... 23 common frames omitted

2018-01-16T14:36:49.770399060Z Caused by: java.security.cert.CertificateExpiredException: NotAfter: Mon Jan 15 18:12:18 UTC 2018

2018-01-16T14:36:49.770402708Z  at sun.security.x509.CertificateValidity.valid(CertificateValidity.java:274)

2018-01-16T14:36:49.770408587Z  at sun.security.x509.X509CertImpl.checkValidity(X509CertImpl.java:629)

2018-01-16T14:36:49.770413647Z  at sun.security.provider.certpath.BasicChecker.verifyTimestamp(BasicChecker.java:190)

2018-01-16T14:36:49.770419840Z  at sun.security.provider.certpath.BasicChecker.check(BasicChecker.java:144)

2018-01-16T14:36:49.770429403Z  at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:125)

2018-01-16T14:36:49.770443412Z  ... 28 common frames omitted

2018-01-16T14:36:49.774570269Z 2018-01-16 14:36:49,774 WARN  [puppetserver] Puppet Error connecting to puppetdb on 8081 at route /pdb/cmd/v1?checksum=6a40b1127a0e8c1dee4fdd40cd45c9a9b4478dc6&version=8&certname=2klic-dev-596e89d2fe5e08410003f2e6&command=store_report&producer-timestamp=1516113409, error message received was 'Error executing http request'. Failing over to the next PuppetDB server_url in the 'server_urls' list

2018-01-16T14:36:49.776385101Z 2018-01-16 14:36:49,776 ERROR [puppetserver] Puppet Failed to execute '/pdb/cmd/v1?checksum=6a40b1127a0e8c1dee4fdd40cd45c9a9b4478dc6&version=8&certname=2klic-dev-596e89d2fe5e08410003f2e6&command=store_report&producer-timestamp=1516113409' on at least 1 of the following 'server_urls': https://puppetdb:8081

2018-01-16T14:36:49.777516859Z 74.57.127.213 - - - 16/Jan/2018:14:36:49 +0000 "PUT /puppet/v3/report/2klic-dev-596e89d2fe5e08410003f2e6?environment=2klic_smart_controller_ws1_2_beta& HTTP/1.1" 200 12 74.57.127.213 74.57.127.213 8140 246

Я удалил puppetdb как узел и заставил его снова зарегистрироваться, и процесс SSL прошел нормально. Но PuppetDB по-прежнему не записывает данные, полученные при регистрации.

Я также посмотрел на файл ca в PuppetDb, чтобы проверить, не истек ли срок его действия:

openssl x509 -enddate -noout -in /etc/puppetlabs/puppetdb/ssl/ca

output notAfter=Jun 25 20:16:09 2022 GMT.

Аналогичный вывод на сервере puppet ca:

openssl x509 -enddate -noout -in /etc/puppetlabs/puppet/ssl/ca/ca_crt.pem

Структура проекта основана на: https://github.com/puppetlabs/puppet-in-docker-examples/blob/master/compose/docker-compose.yml