Track your VM Templates in VMware vSphere with HCP Packer

8 min readAug 30, 2022

Using HCP Packer with VMware vSphere

The HCP Packer registry enables development, operations and security teams to collaborate to generate, maintain, and consume images in a centralized manner by bridging the gap between image factories and image deployments.

This article describes:

how to create the VM Templates in VMware vSphere and track them using HCP Packer registry.
how to create the VM Clone Templates in VMware vSphere and track them and their dependencies using HCP Packer registry.
how to create your VMs in VMware vSphere using Terraform and HCP Packer registry.

Photo by Christopher Izquierdo on Unsplash

Introduction

First of all, HashiCorp Packer is a tool that automates the creation of your golden images for multiple platforms.

Then, the HashiCorp Cloud Platform (HCP) is a fully managed platform for HashiCorp tools. Unlike other tools in HCP like Terraform, Vault, or Consul, HCP Packer is not a managed platform for Packer, it’s a metadata registry to track images, their artifacts, their iterations, and their build artifacts across platforms. Only VMware vSphere platform is considered in this article.

HCP Packer offers a Standard Free plan (see HCP Packer Pricing for further details); this plan is suitable for this article.

Prerequisites

VMware vSphere 7.0 platform with adequate privileges (see Configure Service Account Privileges in vSphere to configure least privilege access to the infrastructure)
HashiCorp Packer 1.8.2 or higher
HashiCorp Terraform 1.2.8 or higher
HCP account
(Optional) Packer Examples for vSphere v22.08 or higher

Packer Templates

Before using HCP Packer with VMware vSphere, a Packer template for VMware vSphere should be created.

If you already have a ready-to-use Packer template for VMware vSphere, go directly to Create HCP Packer registry section.

Otherwise, Packer Examples for vSphere project (maintained by Ryan Johnson) could be utilized to have one ready-to-use template.

Packer Example for vSphere — Project presentation

In your terminal, clone the project and run the config script

$ git clone https://github.com/vmware-samples/packer-examples-for-vsphere.git
$ cd packer-examples-for-vsphere
$ ./config.sh

Then, configure the variables to adapt to your configuration.

Create HCP Packer registry

This section is an excerpt from the official learning tutorial Get Started with HCP.

Go to the HashiCorp Cloud Platform portal. After logging in, you will find Packer under Services in the left navigation menu.

Screenshot by HashiCorp from Get Started with HCP Packer learning tutorial

HCP Packer registry must be enabled before Packer can publish build metadata to it.

Click the Create a registry button. This only needs to be done once.

Then, go to Access control (IAM) in the left navigation menu, then select the Service principals tab.

Create a service principal named packer with the Contributor role.

Once you create the service principal, click the service principal name to view its details. From the detail page, click + Generate key to create a Client ID and Client Secret.

Copy and save the client ID and secret; you will not be able to retrieve the secret later. You will use these credentials in the next step.

Once you generate the keys for the service principal, set the Client ID and Client Secret as environment variables so that Packer can authenticate with HCP.

In your terminal, set an environment variable for the Client ID and Client secret.

$  export HCP_CLIENT_ID=<your client ID>
$  export HCP_CLIENT_SECRET=<your client secret>

Publish metadata to HCP Packer registry

The hcp_packer_registry block should be added to an existingbuild block to enable HCP Packer mode and customize the metadata Packer sends to HCP Packer registry (see official documentation page for futher details).

build {
  ...
  hcp_packer_registry {
    bucket_name = "my-first-bucket"
    description = "My first bucket"    bucket_labels = { ... }    build_labels = { ... }
  }
  ...
}

If the proper HCP credentials are provided (i.e. environment variables HCP_CLIENT_ID and HCP_CLIENT_SECRET are set), Packer will publish all builds metadata contained within that build block to the HCP Packer registry. To prevent any potential artifact drift across the specified builders, Packer will immediatly fail the build if no HCP credentials are set.

If you use Packer Examples for vSphere project, edit config/common.pkvars.hcl configuration file to enable HCP Packer.

// HCP Packer
common_hcp_packer_registry_enabled = true

Then, launch the build of your Packer image(s).

When each image build is finished, explore the new image bucket that has been created on HCP Packer registry (here only linux-ubuntu-2204lts).

Click on an image bucket to get an overview of it. You find details about the selected image bucket, including its description and labels: settings specified in the hcp_packer_registry block of your Packer template file,description and bucket_labels arguments, respectively.

Click on Iterations in the left navigation menu to explore image iterations.

An image iteration, which is an immutable record of the build, is produced each time Packer builds a template. Each iteration has a unique identifier that maps to the Git SHA or the value set in HCP_PACKER_BUILD_FINGERPRINT environement variable.

Click on an image iteration to get an overview of it. You find details about the selected image iteration, including its labels: from Packer, Packer plugin and settings specified in the hcp_packer_registry block of your Packer template file, inbuild_labels argument.

Create a VM clone template using HCP Packer registry

You can link an image iteration to a permanent slug using channels.

Using data sources, this channel may then be used to query for images in Packer or Terraform.

Create a channel

In a created bucket image, select Channels in the left navigation menu and click Create a channel.

Create a new channel named stable and select an image iteration.

Query image iteration

Let’s begin to use the stable channel with Packer (the next section will focus on Terraform).

hcp-packer-iteration andhcp-packer-image data sources are used in the Packer template to query an image iteration from HCP Packer registry (see official documentation for further details).

data “hcp-packer-iteration” “base” {
  bucket_name = "my-first-bucket"
  channel     = "stable"
}data “hcp-packer-image” “image” {
  bucket_name    = "my-first-bucket"
  iteration_id   = data.hcp-packer-iteration.base.id
  cloud_provider = “vsphere”.
  region         = "Datacenter"
}

The cloud_provider argument should be set to vsphere to retrieve image metadata for VMware vSphere.

Then, cloud_provider argument is equal to the related datacenter where the VM Template resource is located.

Clone

To clone a VM Template, the vsphere-clone builder is used. After cloning the VM Template, it modifies it (in source block) and saves it as a new VM template (in build block).

source “vsphere-clone” “clone” {
...
}build {
  ...
  hcp_packer_registry {
    bucket_name = "my-clone-bucket"
    description = "My clone bucket"    bucket_labels = { ... }    build_labels = { ... }
  }
  ...
}

Like the Packer template for the base image, the hcp_packer_registry block should be added to thebuild block to enable HCP Packer mode and customize the metadata Packer sends to HCP Packer registry.

Then, launch the build of your Packer image. When the build is finished, explore the new image bucket that has been created on HCP Packer.
Like previously, you can create a new channel and select an image iteration.

This new channel may then be used to query for images in Packer (for another level of VM Template clone) or in Terraform.

Create a VM using Terraform and HCP Packer registry

Using data sources, a channel may be used to query for images in Terraform as well.

Query image iteration

hcp_packer_image data source is used in the Terraform configuration to query an image iteration from HCP Packer registry (see official documentation for further details).

Then, the related artifact (i.e. VM Template) can be retrieved from the given datacenter.

data "vsphere_datacenter" "datacenter" {
  name = "Datacenter"
}data “hcp_packer_image” “clone” {
  bucket_name    = "my-clone-bucket"
  cloud_provider = “vsphere”
  channel        = "stable"
  region         = "Datacenter"
}data “vsphere_virtual_machine” “template” {
  name          = data.hcp_packer_image.image.cloud_image_id
  datacenter_id = data.vsphere_datacenter.datacenter.id
}

Clone the VM Template

So, the regular Terraform life-cycle could be applied to manage your VMs using the VM Template pointed by HCP Packer registry from stable channel of the desired image bucket.

Conclusion

The HCP Packer registry enables development, operations and security teams to collaborate to generate, maintain, and consume images in a centralized manner by bridging the gap between image factories and image deployments

The image iteration metadata are stored on a dedicated bucket. Using channels, you can identify image iterations to indicate the level of build quality and stability. So, you have control over how your images are delivered thanks to channels.

Based on the quality and stability they desire, other teams can dynamically query images. This process requires less human work than manual referencing VM Template and keeps your infrastructure more stable and secure.