PuppetDB：无法提交“替换事实”命令

Question

我最近撤销/清理了 Puppet 代理证书，这似乎对 PuppetDB 产生了负面影响。我看到这里已经提交了一个错误，其中包含一些关于修复问题的说明。一个用户在这里遇到了类似的问题，但这些都不适合我。

服务器运行 CentOS 6.2、Puppet 2.7.13 和 Puppet DB 0.9。错误是：

root@harp:/etc/puppetdb/ssl> puppet agent --test
err: Cached facts for harp failed: Failed to find facts from PuppetDB at harp.mydomain.com:8081: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed.  This is often because the time is out of sync on the server or client
info: Loading facts in /etc/puppet/modules/dns/lib/facter/datacenter.rb
info: Caching facts for harp
err: Could not send report: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed.  This is often because the time is out of sync on the server or client
err: Could not run Puppet configuration client: Could not retrieve local facts: Failed to submit 'replace facts' command for harp to PuppetDB at harp.mydomain.com:8081: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed.  This is often because the time is out of sync on the server or client

从我所见，NTP 工作正常，日期时间看起来不错。“harp”实际上是傀儡主服务器，所以这里的代理和服务器之间应该没有时间问题，因为它们是相同的。

旧证书：

root@harp:/etc/puppetdb/ssl> puppet cert list --all
+ harp  (DF:8F:65:36:58:4C:DE:66:2B:65:D1:E6:18:B7:F2:33)

为代理清理并生成新证书：

root@harp:/etc/puppetdb/ssl> puppet cert clean harp
notice: Revoked certificate with serial 18
notice: Removing file Puppet::SSL::Certificate harp at '/var/lib/puppet/ssl/ca/signed/harp.pem'
notice: Removing file Puppet::SSL::Certificate harp at '/var/lib/puppet/ssl/certs/harp.pem'
notice: Removing file Puppet::SSL::CertificateRequest harp at '/var/lib/puppet/ssl/certificate_requests/harp.pem'
notice: Removing file Puppet::SSL::Key harp at '/var/lib/puppet/ssl/private_keys/harp.pem'

root@harp:/etc/puppetdb/ssl> puppet agent --test
info: Creating a new SSL key for harp
warning: peer certificate won't be verified in this SSL session
warning: peer certificate won't be verified in this SSL session
info: Creating a new SSL certificate request for harp
info: Certificate Request fingerprint (md5): 72:5E:99:6A:DE:B0:76:BD:1A:7D:FD:DC:A9:E8:71:AD
warning: peer certificate won't be verified in this SSL session
warning: peer certificate won't be verified in this SSL session
warning: peer certificate won't be verified in this SSL session
Exiting; no certificate found and waitforcert is disabled

root@harp:/etc/puppetdb/ssl> puppet cert list
  harp (72:5E:99:6A:DE:B0:76:BD:1A:7D:FD:DC:A9:E8:71:AD)

root@harp:/etc/puppetdb/ssl> puppet cert sign harp
notice: Signed certificate request for harp
notice: Removing file Puppet::SSL::CertificateRequest harp at '/var/lib/puppet/ssl/ca/requests/harp.pem'

root@harp:/etc/puppetdb/ssl> puppet cert list --all
+ harp  (4A:D4:90:87:15:1B:D3:FD:A8:15:D9:C0:FB:08:5C:79)

root@harp:/etc/puppetdb/ssl> service puppetdb restart
Stopping puppetdb: /etc/init.d/puppetdb: line 77: kill: (8623) - No such process
                                                           [FAILED]
Starting puppetdb:                                         [  OK  ]

好的，然后再次重新启动以进行良好的测量：

root@harp:/etc/puppetdb/ssl> service puppetdb restart
Stopping puppetdb:                                         [  OK  ]
Starting puppetdb:                                         [  OK  ]

运行SSL 配置脚本

root@harp:/etc/puppetdb/ssl> /usr/sbin/puppetdb-ssl-setup
cp: cannot stat `/var/lib/puppet/ssl/certs/harp.pem': No such file or directory

root@harp:/etc/puppetdb/ssl> ls -la /var/lib/puppet/ssl/certs
total 12
drwxr-xr-x 2 puppet root 4096 Jun 19 07:19 ./
drwxrwx--x 8 puppet root 4096 Apr 24 10:04 ../
-rw-r--r-- 1 puppet root 1854 Apr 24 10:04 ca.pem

好的，然后再试一次：

root@harp:/etc/puppetdb/ssl> /usr/sbin/puppetdb-ssl-setup
Certificate was added to keystore
Usage: pkcs12 [options]
where options are
-export       output PKCS12 file
-chain        add certificate chain
-inkey file   private key if not infile
-certfile f   add all certs in f
-CApath arg   - PEM format directory of CA's
-CAfile arg   - PEM format file of CA's
-name "name"  use name as friendly name
-caname "nm"  use nm as CA friendly name (can be used more than once).
-in  infile   input filename
...snip...
-CSP name     Microsoft CSP name
-LMK          Add local machine keyset attribute to private key

/etc/puppetdb/ssl 中的密钥库似乎没有更改/重新生成。此时，运行puppet agent --test会出现同样的错误，重启 puppet 和 puppetdb 也无济于事。

密钥库信息：

root@harp:/etc/puppetdb/ssl> keytool -list -keystore /etc/puppetdb/ssl/keystore.jks
Enter keystore password:

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 1 entry

harp.mydomain.com, May 25, 2012, PrivateKeyEntry,
Certificate fingerprint (MD5): 06:A8:D3:2A:70:F3:6D:34:62:91:45:22:8A:C4:A8:86
root@harp:/etc/puppetdb/ssl> keytool -list -keystore /etc/puppetdb/ssl/truststore.jks
Enter keystore password:

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 1 entry

puppetdb ca, May 25, 2012, trustedCertEntry,
Certificate fingerprint (MD5): 13:AD:D8:BC:42:40:47:BB:D2:5C:ED:3C:D1:78:26:88
root@harp:/etc/puppetdb/ssl> puppet cert --fingerprint ca harp.mydomain.com
ca 13:AD:D8:BC:42:40:47:BB:D2:5C:ED:3C:D1:78:26:88
err: Could not call fingerprint: Could not find a certificate or csr for harp.mydomain.com

root@harp:/etc/puppetdb/ssl> puppet cert --fingerprint ca harp
ca 13:AD:D8:BC:42:40:47:BB:D2:5C:ED:3C:D1:78:26:88
harp 4A:D4:90:87:15:1B:D3:FD:A8:15:D9:C0:FB:08:5C:79

我怎样才能让 puppetdb 密钥库真正重新生成？我尝试删除 /etc/puppetdb/ssl/ 中的文件，但没有成功。

Answer 1

我明白了，但不能确切地说出哪些步骤是必要的或不需要的。

此问题的出现是因为多个主机上的身份验证速度缓慢或挂起，并且似乎与域控制器/DNS 缓存问题有关。从木偶大师和代理中删除domain mydomain.com条目解决了问题，但这给现有木偶证书带来了问题。/etc/resolv.conf我puppet cert clean --all在 master 上运行尝试重新创建所有证书，但这在 PuppetDB 上运行得不太好。

解决方案

清除 master 上的旧证书：

puppet cert clean --all

清除所有代理上的旧证书：

rm -rf /var/lib/puppet/ssl

重新创建 PuppetDB 密钥库：

facter fqdndomain foo.com从 中删除后不可用/etc/resolv.conf。这会导致puppetdb-ssl-setup无声地失败。

编辑/usr/sbin/puppetdb-ssl-setup，添加一段代码，以便在facter hostname为facter fqdn空时使用：

# near line 10
fqdn=`facter fqdn`
# add this "if" section
if [ ! -n "$fqdn" ] ; then
  fqdn=`facter hostname`
fi

权限修复：

chown -R puppetdb:puppetdb /etc/puppetdb/ssl

使用新的密钥库/信任库密码（相同的密码）更新 /etc/puppetdb/conf.d/jetty.ini 中的密码，您可以从以下位置获取该密码：

cat /etc/puppetdb/ssl/puppetdb_keystore_pw.txt

重启puppetdb

service puppetdb restart

然后转到每个代理并请求新证书并在主服务器上签署每个证书。