Securely store Public Certificates in Vault, generated by acme.sh

6 min readSep 28, 2022

Automate your Certificates Lifecycle with Vault — Part 4

Photo by Anton Maksimov 5642.su on Unsplash

This article is a part of a series about how to
Automate your Certificates Lifecycle with Vault.

Introduction

This article presents a solution to manage your public certificates and securely store your public certificates in Vault, generated by acme.sh.

Then, as soon as the public certificates are stored in Vault, consul-template (or other similar solutions) can be used to deploy and automatically update the deployed certificate when ACME.sh renews certificates.

Let’s Encrypt among other providers is compatible with ACME.sh as well. Here are the differences between ZeroSSL and Let’s Encrypt.

Pre-Requisites

Launch Vault Server in Dev mode using root as a root token id. The root token is used on purpose to focus only on the KV secret engine features.

Disclaimer: On production
- Don’t use the root token
- Create auth methods, entities, groups and policies to grant or forbid access to KV features

$ vault server -dev -dev-root-token-id=root

Create ZeroSSL credentials

First, let’s create a ZeroSSL account on https://zerossl.com
Then, go to Developer page

In EAB Credentials for ACME Clients section, click on the Generate button, then copy EAB KID and EAB HMAC Key fields.

Install and Configure ACME.sh

For further details, please read the ACME.sh documentation.

ACME.sh is a pure Unix shell script implementing ACME client protocol

To install and configure ACME.sh, follow these instructions.

Install the script and the crontab entry to renew certificate automatically

$ curl https://get.acme.sh | sh -s email=me@example.com

$ acme.sh --register-account \
    --server zerossl \
    --eab-kid <EAB KID> \
    --eab-hmac-key <EAB HMAC Key>

Enable KV secret engine on Vault to store the certificate, key and CA chain

$ vault secrets enable -path=acme kv

Issue and Deploy Certificate

Let’s issue an ECDSA certificate using the DNS Challenge on CloudFlare, then verify it.

$ export CF_Token=xxxxxxxxx
$ acme.sh --issue -k ec-256 --dns dns_cf \
    -d example.com --always-force-new-domain-key
...$ openssl x509 -in ~/.acme.sh/example.com_ecc/example.com.cer \
    -issuer -subject -startdate -enddate -noout
issuer=C = AT, O = ZeroSSL, CN = ZeroSSL ECC Domain Secure Site CA
subject=CN = example.com
notBefore=Sep 20 00:00:00 2022 GMT
notAfter=Dec 19 23:59:59 2022 GMT

Next, update .acme.sh/deploy/vault_cli.shto add ttl=0 in Vault commands at the end of the script. By doing so, the secret polling period is managed by consul-template configuration (see default_lease_duration in Consul Template section below)

if [ -n "$FABIO" ]; then
  $VAULT_CMD kv put ... key=@"$_ckey" || return 1
else
  $VAULT_CMD kv put ... value=@"$_ccert" ttl=0 || return 1
  $VAULT_CMD kv put ... value=@"$_ckey" ttl=0 || return 1
  $VAULT_CMD kv put ... value=@"$_cca" ttl=0 || return 1
  $VAULT_CMD kv put ... value=@"$_cfullchain" ttl=0 || return 1
fi

Finally, deploy the certificate to Vault

$ export VAULT_ADDR=http://127.0.0.1:8200
$ export VAULT_TOKEN=root
$ acme.sh --deploy --ecc -d example.com --deploy-hook vault_cli
Success! Data written to: acme/example.com/cert.pem
Success! Data written to: acme/example.com/cert.key
Success! Data written to: acme/example.com/chain.pem
Success! Data written to: acme/example.com/fullchain.pem$ vault kv list acme/example.com
Keys
----
cert.key
cert.pem
chain.pem
fullchain.pem
$ vault kv get acme/example.com/cert.pem
$ vault kv get acme/example.com/cert.key

Consul Template

The following templates query Vault to get respectively the private key, the certificate, the CA chain and the fullchain.

{{- with secret "acme/example.com/cert.key" -}}
{{ .Data.value }}
{{- end -}}{{- with secret "acme/example.com/cert.pem" -}}
{{ .Data.value }}
{{- end -}}{{- with secret "acme/example.com/chain.pem" -}}
{{ .Data.value }}
{{- end -}}{{- with secret "acme/example.com/fullchain.pem" -}}
{{ .Data.value }}
{{- end -}}

First of all, the consul-template daemon should be configured with a dedicated configuration file:

to connect to the Vault server
to generate files from specified templates

Read the configuration documentation for further information

Here is a configuration file that queries cert.key and cert.crt from KV secret engine on acme mount for example.com domain.

Disclaimer: On production
- Don’t use the root token
- Create auth methods, entities, groups and policies to grant or forbid access to KV features

$ cat <<EOF > consul-template-acme.hcl
vault {
  address     = "http://127.0.0.1:8200"
  token       = "root"
  renew_token = false  default_lease_duration = "10s"
}template {
  contents = <<EOH
{{- with secret "acme/example.com/cert.key" -}}
{{ .Data.value }}
{{- end }}
EOH
  destination = "cert.key"
  exec {
    command = [ "cat", "cert.key" ]
  }
}template {
  contents = <<EOH
{{- with secret "acme/example.com/cert.pem" -}}
{{ .Data.value }}
{{- end }}
EOH
  destination = "cert.crt"
  exec {
    command = [ "openssl", "x509", "-in", "cert.crt", "-text", "-noout" ]
  }
}
EOF

Start consul-template to check the generated files.

$ consul-template -config=consul-template-acme.hcl -log-level info
[INFO] consul-template v0.29.2 (06389a3d)
[INFO] (runner) creating new runner (dry: false, once: false)
[INFO] (runner) creating watcher
[INFO] (runner) starting
[INFO] (runner) rendered "(dynamic)" => "cert.key"
[INFO] (runner) executing command "[\"cat\" \"cert.key\"]" from "(dynamic)" => "cert.key"
[INFO] (child) spawning: cat cert.key
-----BEGIN EC PRIVATE KEY-----
MHcCAQEEIKk4jGLCfwcLyB913K+5bQDEOURBv2Ex+P1cnpxNvvcKoAoGCCqGSM49
AwEHoUQDQgAEoZNWBcOP4lABJdiz9CTeXW9FDirBD7mNPzsvGIJ69Y1vEby871iD
30nz2iKVrvx2/XUslneUdWCv77TYYfwtoA==
-----END EC PRIVATE KEY-----[INFO] (runner) rendered "(dynamic)" => "cert.crt"
[INFO] (runner) executing command "[\"openssl\" \"x509\" \"-in\" \"cert.crt\" \"-text\" \"-noout\"]" from "(dynamic)" => "cert.crt"
[INFO] (child) spawning: openssl x509 -in cert.crt -text -noout
Certificate:
...
       Issuer: C = AT, O = ZeroSSL, CN = ZeroSSL ECC...
        Validity
            Not Before: Sep 20 00:00:00 2022 GMT
            Not After : Dec 19 23:59:59 2022 GMT
        Subject: CN = example.com

Renew Certificate

The renewal is managed by ACME.sh using the crontab entry. By default, ACME.sh renews the certificate at 2/3 of the TTL (i.e. every 60 days).

To force the renewal, let’s launch the related ACME.sh command.

$ acme.sh --renew -d example.com --ecc --force

Now, let’s check the terminal where consul-template is running. After the renewal of the certificate, ACME.sh deploys it into Vault, then consul-template renders the updated certificate and key.

[INFO] (runner) rendered "(dynamic)" => "cert.key"
[INFO] (runner) executing command "[\"cat\" \"cert.key\"]" from "(dynamic)" => "cert.key"
[INFO] (child) spawning: cat cert.key
-----BEGIN EC PRIVATE KEY-----
MHcCAQEEIKk4jGLCfwcLyB913K+5bQDEOURBv2Ex+P1cnpxNvvcKoAoGCCqGSM49
AwEHoUQDQgAEoZNWBcOP4lABJdiz9CTeXW9FDirBD7mNPzsvGIJ69Y1vEby871iD
30nz2iKVrvx2/XUslneUdWCv77TYYfwtoA==
-----END EC PRIVATE KEY-----[INFO] (runner) rendered "(dynamic)" => "cert.crt"
[INFO] (runner) executing command "[\"openssl\" \"x509\" \"-in\" \"cert.crt\" \"-text\" \"-noout\"]" from "(dynamic)" => "cert.crt"
[INFO] (child) spawning: openssl x509 -in cert.crt -text -noout
Certificate:
...
       Issuer: C = AT, O = ZeroSSL, CN = ZeroSSL ECC...
        Validity
            Not Before: Sep 20 00:00:00 2022 GMT
            Not After : Dec 19 23:59:59 2022 GMT
        Subject: CN = example.com

Conclusion

This article shows a solution to manage your public certificates and securely store your public certificates in Vault, generated by acme.sh.

References

GitHub - acmesh-official/acme.sh: A pure Unix shell script implementing ACME client protocol

An ACME protocol client written purely in Shell (Unix shell) language. Full ACME protocol implementation. Support ECDSA…

acme.sh