1. Introducing configuration management using Puppet

You can use Puppet to manage and automate configurations of hosts. Puppet uses a declarative language to describe the desired state of hosts.

Puppet increases your productivity as you can administer multiple hosts simultaneously. At the same time, it decreases your configuration effort as Puppet makes it easy to verify and possibly correct the state of the hosts.

Additional resources

1.1. How Puppet integrates with Foreman

Puppet uses a server-agent architecture. The Puppet server is the central component that stores configuration definitions. Foreman server or any Smart Proxies are typically deployed with the Puppet server and Foreman acts as an External Node Classifier (ENC) for such Puppet server. Hosts run the Puppet agent that communicates with the Puppet server.

The Puppet agent collects facts about a host and reports them to the Puppet server on each run. You can display the Puppet facts in JSON format by running puppet facts on a host.

The Puppet server forwards facts to Foreman and Foreman stores them for later use. Based on the facts and other definitions, Foreman constructs the ENC answer to the Puppet server. The Puppet server compiles a catalog based on the ENC answer and sends the catalog to the Puppet agent.

The Puppet agent evaluates the system state on the host. If the Puppet agent finds differences, known as drifts, between the desired state defined in the catalog and the actual state, it enforces correction of the state of the host. The Puppet agent then reports correction results back to the Puppet server, which reports them to Foreman.

Puppet modules

The desired state of a host is defined in a catalog. The catalog is compiled from Puppet manifests of one or more Puppet modules assigned to the host. A Puppet module is a collection of classes, manifests, resources, files, and templates. The Puppet modules work as components of host configuration definitions.

Smart Class parameters

You can override parameters of a Puppet module using Smart Class parameters if the module supports the use of parameters. You can define the parameters in your Foreman as key-value pairs, which behave similar to host parameters or Ansible variables.

Puppet environments

You can also create multiple Puppet environments to control versions of configuration definitions or to manage variants of the definitions, and to test the definitions before you deploy them on production.

High-level integration steps

Puppet integration with Foreman involves the following high-level steps:

  1. Enable Puppet integration.

  2. Import Puppet agent packages into Foreman. Puppet agent packages can be managed like any other content with Foreman by syncing repositories in custom products and by using activation keys and content views.

  3. Install Puppet agent on hosts during provisioning, registration, manually, or by remote job execution.

Additional resources

The following procedures outline how to use a Puppet module to install, configure, and manage the ntp service to provide examples.

1.2. Supported Puppet versions and system requirements

Before you begin with the Puppet integration, review the supported Puppet versions and system requirements.

Supported Puppet Versions

Foreman supports Puppet 7. Ensure that the Puppet modules used to configure your hosts are compatible with Puppet 7.

System Requirements

Before you begin integrating Puppet with your Foreman, ensure that you meet the system requirements. For more information, see System Requirements for Puppet 7 in the Open Source Puppet documentation.

1.3. Enabling Puppet integration with Foreman

By default, Foreman does not have any Puppet integration configured. You need to enable the integration as is appropriate for your situation. This means that you can configure Foreman to manage and deploy Puppet server on Foreman server or Smart Proxy servers. Additionally, you can deploy Puppet server to Foreman externally and integrate it with Foreman for reporting, facts, and external node classification (ENC).

Procedure

  1. Enable Puppet integration and install Puppet server on Foreman server:

    # foreman-installer \
--enable-foreman-cli-puppet \
--enable-foreman-plugin-puppet \
--enable-puppet \
--foreman-proxy-puppet true \
--foreman-proxy-puppetca true \
--puppet-server true

  2. If you want to use Puppet integration on Smart Proxy servers, enable Puppet integration and install Puppet server on Smart Proxy servers:

    # foreman-installer \
--enable-puppet \
--foreman-proxy-puppet true \
--foreman-proxy-puppetca true \
--puppet-server true

1.4. Installing and configuring Puppet agent during host provisioning

You can install and configure the Puppet agent on a host during the provisioning process. A configured Puppet agent is required on the host for Puppet integration with your Foreman.

Prerequisites

  • You created a product and repository for the upstream Puppet agent, such as https://yum.puppet.com or https://apt.puppet.com, and synchronized the repository to Foreman. For more information, see Importing Content in Managing content.

  • You created an activation key that enables the Puppet agent repository for hosts. For more information, see Managing Activation Keys in Managing content.

Procedure

  1. Navigate to Hosts > Templates > Provisioning Templates.

  2. Select a provisioning template depending on your host provisioning method. For more information, see Kinds of Provisioning Templates in Provisioning hosts.

  3. Ensure the puppet_setup snippet is included as follows:

    <%= snippet 'puppet_setup' %>

    Note that this snippet is already included in the templates shipped with Foreman, such as Kickstart default or Preseed default.

  4. Enable the Puppet agent using a host parameter in global parameters, a host group, or for a single host. Add a host parameter named enable-puppet7, select the boolean type, and set the value to true.

  5. Set configuration for the Puppet agent.

    • If you use an integrated Puppet server, ensure that you select a Puppet Smart Proxy, Puppet CA Smart Proxy, and Puppet environment when you create a host.

    • If you use a non-integrated Puppet server, either set the following host parameters in global parameters, or a host group, or when you create a host:

      • Add a host parameter named puppet_server, select the string type, and set the value to the hostname of your Puppet server, such as puppet.example.com.

      • Optional: Add a host parameter named puppet_ca_server, select the string type, and set the value to the hostname of your Puppet CA server, such as puppet-ca.example.com. If puppet_ca_server is not set, the Puppet agent will use the same server as puppet_server.

      • Optional: Add a host parameter named puppet_environment, select the string type, and set the value to the Puppet environment you want the host to use.

  6. Ensure your host has access to the Puppet agent packages from Foreman server by using an appropriate activation key.

1.5. Installing and configuring Puppet agent during host registration

You can install and configure the Puppet agent on the host during registration. A configured Puppet agent is required on the host for Puppet integration with your Foreman.

Prerequisites

  • You created a product and repository for the upstream Puppet agent, such as https://yum.puppet.com or https://apt.puppet.com, and synchronized the repository to Foreman. For more information, see Importing Content in Managing content.

  • You created an activation key that enables the Puppet agent repository for hosts. For more information, see Managing Activation Keys in Managing content.

Procedure

  1. In the Foreman web UI, navigate to Configure > Global Parameters to add host parameters globally. Alternatively, you can navigate to Configure > Host Groups and edit or create a host group to add host parameters only to a host group.

  2. Enable the Puppet agent using a host parameter in global parameters or a host group. Add a host parameter named enable-puppet7, select the boolean type, and set the value to true.

  3. Specify configuration for the Puppet agent using the following host parameters in global parameters or a host group:

    • Add a host parameter named puppet_server, select the string type, and set the value to the hostname of your Puppet server, such as puppet.example.com.

    • Optional: Add a host parameter named puppet_ca_server, select the string type, and set the value to the hostname of your Puppet CA server, such as puppet-ca.example.com. If puppet_ca_server is not set, the Puppet agent will use the same server as puppet_server.

    • Optional: Add a host parameter named puppet_environment, select the string type, and set the value to the Puppet environment you want the host to use.

    Until the BZ2177730 is resolved, you must use host parameters to specify the Puppet agent configuration even in integrated setups where the Puppet server is a Smart Proxy server.

  4. Navigate to Hosts > Register Host and register your host using an appropriate activation key. For more information, see Registering Hosts in Managing hosts.

  5. Navigate to Infrastructure > Smart Proxies.

  6. From the list in the Actions column for the required Smart Proxy server, select Certificates.

  7. Click Sign to the right of the required host to sign the SSL certificate for the Puppet agent.

1.6. Installing and configuring Puppet agent manually

You can install and configure the Puppet agent on a host manually. A configured Puppet agent is required on the host for Puppet integration with your Foreman.

Prerequisites

  • The host must have a Puppet environment assigned to it.

  • Ensure a repository containing the Puppet agent is enabled on the host, for example apt.puppet.com or yum.puppet.com.

Procedure

  1. Log in to the host as the root user.

  2. Install the Puppet agent package.

    • On hosts running Enterprise Linux 8 and above:

      # dnf install puppet-agent

    • On hosts running Debian:

      # apt install puppet-agent

    • On hosts running SUSE Linux Enterprise Server:

      # zypper install puppet-agent

  3. Add the Puppet agent to PATH in your current shell using the following script:

    . /etc/profile.d/puppet-agent.sh

  4. Configure the Puppet agent. Set the environment parameter to the name of the Puppet environment to which the host belongs:

    # puppet config set server foreman.example.com --section agent
# puppet config set environment My_Puppet_Environment --section agent

  5. Start the Puppet agent service:

    # puppet resource service puppet ensure=running enable=true

  6. Create a certificate for the host:

    # puppet ssl bootstrap

  7. In the Foreman web UI, navigate to Infrastructure > Smart Proxies.

  8. From the list in the Actions column for the required Smart Proxy server, select Certificates.

  9. Click Sign to the right of the required host to sign the SSL certificate for the Puppet agent.

  10. On the host, run the Puppet agent again:

    # puppet ssl bootstrap

1.7. Performing configuration management

After you deploy Puppet agent on a host, you can start performing configuration management with Puppet. This involves the following high-level steps:

  1. Managing Puppet modules on the Puppet server, that is installing and updating them.

  2. Importing Puppet classes and environments from Puppet modules into Foreman.

  3. Optional: Creating config groups from Puppet classes.

  4. Configuring overrides of Smart Class parameters on various levels.

  5. Assigning Puppet classes or config groups to host groups or individual hosts.

  6. Configuring intervals for runs of the Puppet agent on hosts and for configuration enforcement runs of the Puppet server.

  7. Monitoring configuration management using reports in the Foreman web UI. For more information, see Monitoring Resources in Administering Foreman.

  8. Configuring email notifications. For more information, see Configuring Email Notification Preferences in Administering Foreman.

After assigning Puppet classes or config groups, Foreman runs configuration management automatically in the configured intervals to enforce Puppet configuration on your hosts, or you can initiate it manually on demand with the Run Puppet Once feature. For more information, see Running Puppet once using SSH.

1.8. Disabling Puppet integration with Foreman

To discontinue using Puppet in your Foreman, follow this procedure.

Note that the command without the --remove-all-data argument removes all Puppet-related data in Foreman database. With the --remove-all-data argument, the command additionally removes Puppet server data files, including Puppet environments.

Warning

If you disable Puppet with the --remove-all-data argument, you will not be able to re-enable Puppet afterwards. This is a known issue, see the Bug 2087067.
Prerequisites

  • Puppet is enabled on Foreman.

Procedure

  1. If you have used Puppet server on any Smart Proxies, disable Puppet server on all Smart Proxies:

    # foreman-maintain plugin purge-puppet --remove-all-data

  2. Disable Puppet server on Foreman server:

    # foreman-maintain plugin purge-puppet --remove-all-data

2. Managing Puppet modules

2.1. Installing a Puppet module on Foreman server

You can install a pre-built Puppet module from the Puppet Forge. The Puppet Forge is a repository that provides Puppet modules contributed by the community. Puppet modules flagged as supported are officially supported and tested by Puppet Inc.

This example shows how to add the ntp module to hosts.

Procedure

  1. Navigate to forge.puppet.com and search for ntp. One of the first modules is puppetlabs/ntp.

  2. Connect to your Foreman server using SSH and install the Puppet module:

    # puppet module install puppetlabs-ntp -i /etc/puppetlabs/code/environments/production/modules

    Use the -i parameter to specify the path and Puppet environment, for example production.

    Once the installation is completed, the output looks as follows:

    Notice: Preparing to install into /etc/puppetlabs/code/environments/production/modules ...
Notice: Created target directory /etc/puppetlabs/code/environments/production/modules
Notice: Downloading from https://forgeapi.puppet.com ...
Notice: Installing -- do not interrupt ...
/etc/puppetlabs/code/environments/production/modules
|-| puppetlabs-ntp (v8.3.0)
  |-- puppetlabs-stdlib (v4.25.1) [/etc/puppetlabs/code/environments/production/modules]

An alternative way to install a Puppet module is to copy a folder containing the Puppet module to the module path as mentioned above. Ensure to resolve its dependencies manually.

2.2. Updating a Puppet module

You can update an existing Puppet module using the puppet command.

Procedure

  1. Connect to your Puppet server using SSH and find out where the Puppet modules are located:

    # puppet config print modulepath

    This returns output as follows:

    /etc/puppetlabs/code/environments/production/modules:/etc/puppetlabs/code/environments/common:/etc/puppetlabs/code/modules:/opt/puppetlabs/puppet/modules:/usr/share/puppet/modules

  2. If the module is located in the path as displayed above, the following command updates a module:

    # puppet module upgrade module name

2.3. Managing Puppet modules with r10k

You can manage Puppet modules and environments using r10k. r10k uses a list of Puppet roles from a Puppetfile to download Puppet modules from the Puppet Forge or git repositories. It does not handle dependencies between Puppet modules.

A Puppetfile looks as follows:

forge "https://forge.puppet.com"

mod "puppet-nginx",
  :git => "https://github.com/voxpupuli/puppet-nginx.git",
  :ref => "master"

mod "puppetlabs/apache"
mod "puppetlabs/ntp", "8.3.0"

3. Importing Puppet classes and environments into Foreman

Import Puppet classes and environments from the installed Puppet modules to Foreman server or any attached Smart Proxy server before you assign any of the classes to hosts.

Prerequisites

  • Ensure to select Any Organization and Any Location as context, otherwise the import might fail.

Procedure

  1. In the Foreman web UI, navigate to Configure > Puppet ENC > Classes or Configure > Puppet ENC > Environments.

  2. Click Import in the upper right corner and select which Smart Proxy you want to import modules from. You may typically choose between your Foreman server or any attached Smart Proxy server.

  3. Select the Puppet environments to import using checkboxes on the left.

  4. Click Update to import the Puppet environments and classes to Foreman.

  5. The import should result in a notification as follows:

    Successfully updated environments and Puppet classes from the on-disk Puppet installation

4. Creating a custom Puppet environment

You can create a Puppet environment within your Foreman.

Procedure

  1. In the Foreman web UI, navigate to Configure > Puppet Environments.

  2. Click Create Puppet Environment to create a Puppet environment.

  3. Enter a Name, alphanumeric characters and underscores are allowed, such as example_environment.

  4. Optional: Set a location context.

  5. Optional: Set an organization context.

  6. Click Submit to create the Puppet environment.

Note that before you run an import of Puppet modules into Foreman, the environment must already exist as the folder /etc/puppetlabs/code/environments/example_environment on the Puppet server and contain installed Puppet modules.

5. Creating a Puppet config group

A Puppet config group is a named list of Puppet classes that allows you to combine their capabilities and assign them to hosts at a click. This is equivalent to the concept of profiles in pure Puppet.

Procedure

  1. In the Foreman web UI, navigate to Configure > Puppet ENC > Config Groups.

  2. Click Create Config Group.

  3. Select the classes you want to add to the config group.

    1. Choose a meaningful Name for the Puppet config group.

    2. Add selected Puppet classes to the Included Classes field.

  4. Click Submit to save the changes.

6. Configuring Puppet smart class parameters

6.1. Puppet parameter hierarchy

Puppet parameters are structured hierarchically. Parameters at a lower level override parameters of the higher levels:

  1. Global parameters

  2. Organization parameters

  3. Location parameters

  4. Host group parameters

  5. Host parameters

For example, host specific parameters override the parameter at any higher level, and location parameters only override parameters at the organization or global level. This feature is especially useful when you use locations or organizations to group hosts.

6.2. Overriding a smart class parameter globally

You can configure a Puppet class after you have imported it to Foreman server. This example overrides the default list of ntp servers.

Procedure

  1. In the Foreman web UI, navigate to Configure > Puppet ENC > Classes.

  2. Select the ntp Puppet class to change its configuration.

  3. Select the Smart Class Parameter tab and search for servers.

  4. Ensure the Override checkbox is selected.

  5. Set the Parameter Type drop down menu to array.

  6. Insert a list of ntp servers as Default Value:

    ["0.de.pool.ntp.org","1.de.pool.ntp.org","2.de.pool.ntp.org","3.de.pool.ntp.org"]

    An alternative way to describe the array is the yaml syntax:

    - 0.de.pool.ntp.org
- 1.de.pool.ntp.org
- 2.de.pool.ntp.org
- 3.de.pool.ntp.org

  7. Click Submit to change the default configuration of the Puppet module ntp.

6.3. Overriding a smart class parameter for an organization

You can use groups of hosts to override Puppet parameters for multiple hosts at once. The following example chooses the organization context to illustrate setting context based parameters.

Note that organization-level Puppet parameters are overridden by location-level Puppet parameters.

Procedure

  1. In the Foreman web UI, navigate to Configure > Puppet ENC > Classes.

  2. Click a class name to select a class.

  3. On the Smart Class Parameter tab, select a parameter.

  4. Use the Order list to define the hierarchy of the Puppet parameters. The individual host (fqdn) marks the most and the organization context (organization) the least relevant.

  5. Check Merge Overrides if you want to add all further matched parameters after finding the first match.

  6. Check Merge Default if you want to also include the default value even if there are more specific values defined.

  7. Check Avoid Duplicates if you want to create a list of unique values for the selected parameter.

  8. The matcher field requires an attribute type from the order list.

  9. Optional: Click Add Matcher to add more matchers.

  10. Click Submit to save the changes.

6.4. Overriding a smart class parameter for a location

You can use groups of hosts to override Puppet parameters for multiple hosts at once. The following examples chooses the location context to illustrate setting context based parameters.

Procedure

  1. In the Foreman web UI, navigate to Configure > Puppet ENC > Classes.

  2. Click a class name to select a class.

  3. On the Smart Class Parameter tab, select a parameter.

  4. Use the Order list to define the hierarchy of the Puppet parameters. The individual host (fqdn) marks the most and the location context (location) the least relevant.

  5. Check Merge Overrides if you want to add all further matched parameters after finding the first match.

  6. Check Merge Default if you want to also include the default value even if there are more specific values defined.

  7. Check Avoid Duplicates if you want to create a list of unique values for the selected parameter.

  8. The matcher field requires an attribute type from the order list. For example, you can choose Paris as location context and set the value to French ntp servers.

  9. Optional: Click Add Matcher to add more matchers.

  10. Click Submit to save the changes.

6.5. Overriding a smart class parameter on an individual host

You can override parameters on individual hosts. This is recommended if you have multiple hosts and only want to make changes to a single one.

Procedure

  1. In the Foreman web UI, navigate to Hosts > All Hosts.

  2. Click a host name to select a host.

  3. Click Edit.

  4. On the Host tab, select a Puppet Environment.

  5. Select the Puppet ENC tab.

  6. Click Override to edit the Puppet parameter.

  7. Click Submit to save the changes.

7. Assigning a Puppet class to a host group

Use a host group to assign the ntp Puppet class to multiple hosts at once. Every host you deploy based on this host group has this Puppet class installed.

Procedure

  1. In the Foreman web UI, navigate to Configure > Host Groups to create a host group or edit an existing one.

  2. On the Host Group tab, set the following parameters:

    1. The Lifecycle Environment describes the stage in which certain versions of content are available to hosts.

    2. The Content View is comprised of products and allows for version control of content repositories.

    3. The Environment allows you to supply a group of hosts with their own dedicated configuration.

  3. Navigate to the Puppet ENC tab.

  4. Add the Puppet class to the Included Classes or to the Included Config Groups if a Puppet config group is configured.

  5. Click Submit to save the changes.

8. Assigning a Puppet class to an individual host

Procedure

  1. In the Foreman web UI, navigate to Hosts > All Hosts.

  2. Locate the host you want to add the ntp Puppet class to and click Edit.

  3. Select the Puppet ENC tab and look for the ntp class.

  4. Click the + symbol next to ntp to add the ntp submodule to the list of included classes.

  5. Click Submit to save your changes.

    Tip

    If the Puppet classes tab of an individual host is empty, check if it is assigned to the proper Puppet environment.

  6. Verify the Puppet configuration.

    1. Navigate to Hosts > All Hosts and select the host.

    2. From the top overflow menu, select Legacy UI.

    3. Under Details, click Puppet YAML. This produces output similar as follows:

      ---
parameters:
  // shortened YAML output
classes:
  ntp:
    servers: '["0.de.pool.ntp.org","1.de.pool.ntp.org","2.de.pool.ntp.org","3.de.pool.ntp.org"]'
environment: production
...

  7. Verify the ntp configuration.

    Connect to your host using SSH and check the content of /etc/ntp.conf.

    This example assumes your host is running CentOS 7. Other operating systems may store the ntp config file in a different path.

    Tip

    You may need to run the Puppet agent on your host by executing the following command:

    # puppet agent -t

  8. Running the following command on the host checks which ntp servers are used for clock synchronization:

    # cat /etc/ntp.conf

    This returns output similar as follows:

    # ntp.conf: Managed by puppet.
server 0.de.pool.ntp.org
server 1.de.pool.ntp.org
server 2.de.pool.ntp.org
server 3.de.pool.ntp.org

You now have a working ntp module which you can add to a host or group of hosts to roll out your ntp configuration automatically.

9. Enforcing Puppet configuration on hosts

You can enforce configuration from Foreman either manually on demand (run once) or automatically in configurable intervals.

9.1. Running Puppet once using SSH

Assign the proper job template to the Run Puppet Once feature to run Puppet on hosts.

Procedure

  1. In the Foreman web UI, navigate to Administer > Remote Execution Features.

  2. Select the puppet_run_host remote execution feature.

  3. Assign the Run Puppet Once – SSH Default job template.

Run Puppet on hosts by running a job and selecting category Puppet and template Run Puppet Once - SSH Default. Alternatively, click Run Puppet Once in the Schedule Remote Job drop down menu on the host details page.

9.2. Understanding intervals of automatic enforcement

Foreman considers hosts to be out of sync if the last Puppet report is older than the combined values of outofsync_interval and puppet_interval set in minutes. By default, the Puppet agent on your hosts runs every 30 minutes, the puppet_interval is set to 35 minutes and the global outofsync_interval is set to 30 minutes.

The effective time after which hosts are considered out of sync is the sum of outofsync_interval and puppet_interval. For example, setting the global outofsync_interval to 30 and the puppet_interval to 60 results in a total of 90 minutes after which the host status changes to out of sync.

9.3. Setting the Puppet agent run interval on a host

Set the interval when the Puppet agent runs and sends reports to Foreman.

Procedure

  1. Connect to your host using SSH.

  2. Add the Puppet agent run interval to /etc/puppetlabs/puppet/puppet.conf, for example runinterval = 1h.

9.4. Setting the global out-of-sync interval

Procedure

  1. In the Foreman web UI, navigate to Administer > Settings.

  2. On the General tab, edit Out of sync interval. Set a duration, in minutes, after which hosts are considered to be out of sync.

    You can also override this interval on host groups or individual hosts by adding the outofsync_interval parameter.

9.5. Setting the Puppet out-of-sync interval

Procedure

  1. In the Foreman web UI, navigate to Administer > Settings, and click the Config Management tab.

  2. In the Puppet interval field, set the value to the duration, in minutes, after which hosts reporting using Puppet are considered to be out of sync.

9.6. Overriding out-of-sync interval for a host group

Procedure

  1. In the Foreman web UI, navigate to Configure > Host Groups.

  2. Select a host group.

  3. On the Parameters tab, click Add Parameter.

  4. In the Name field, enter outofsync_interval.

  5. From the Type dropdown menu, select integer.

  6. In the Value field, enter the new interval in minutes.

  7. Click Submit.

9.7. Overriding out-of-sync interval for an individual host

Procedure

  1. In the Foreman web UI, navigate to Hosts > All Hosts.

  2. Click Edit for a selected host.

  3. On the Parameters tab, click Add Parameter.

  4. In the Name field, enter outofsync_interval.

  5. From the Type dropdown menu, select integer.

  6. In the Value field, enter the new interval in minutes.

  7. Click Submit.