1. Introducing Configuration Management Using Puppet
You can use Puppet to manage and automate configurations of hosts. Puppet uses a declarative language to describe the desired state of managed hosts.
Puppet increases your productivity as you can administer multiple hosts simultaneously. At the same time, it decreases your configuration effort as Puppet makes it easy to verify and possibly correct the state of the hosts.
-
Puppet Forge — a repository of pre-built Puppet modules
1.1. How Puppet Integrates with Foreman
Puppet uses a server-agent architecture. The Puppet server is the central component that stores configuration definitions. Foreman server or any Smart Proxies are typically deployed with the Puppet server and Foreman acts as an External Node Classifier (ENC) for such Puppet server. Managed hosts run the Puppet agent that communicates with the Puppet server.
The Puppet agent collects facts about a host and reports them to the Puppet server on each run.
You can display the Puppet facts in JSON format by running puppet facts
on a host.
The Puppet server forwards facts to Foreman and Foreman stores them for later use. Based on the facts and other definitions, Foreman constructs the ENC answer to the Puppet server. The Puppet server compiles a catalog based on the ENC answer and sends the catalog to the Puppet agent.
The Puppet agent evaluates the system state on the host. If the Puppet agent finds differences, known as drifts, between the desired state defined in the catalog and the actual state, it enforces correction of the state of the host. The Puppet agent then reports correction results back to the Puppet server, which reports them to Foreman.
The desired state of a host is defined in a catalog. The catalog is compiled from Puppet manifests of one or more Puppet modules assigned to the host. A Puppet module is a collection of classes, manifests, resources, files, and templates. The Puppet modules work as components of host configuration definitions.
You can override parameters of a Puppet module using Smart Class parameters if the module supports the use of parameters. You can define the parameters in your Foreman as key-value pairs, which behave similar to host parameters or Ansible variables.
You can also create multiple Puppet environments to control versions of configuration definitions or to manage variants of the definitions, and to test the definitions before you deploy them on production.
Puppet integration with Foreman involves the following high-level steps:
-
Import Puppet agent packages into Foreman. Puppet agent packages can be managed like any other content with Foreman by syncing repositories in custom products and by using Activation Keys and Content Views.
-
Install Puppet agent on hosts during provisioning, registration, manually, or by remote job execution.
-
Registering Hosts in the Managing Hosts Guide
-
Configuring and Setting Up Remote Jobs in the Managing Hosts Guide
The following procedures outline how to use a Puppet module to install, configure, and manage the ntp service to provide examples.
1.2. Supported Puppet Versions And System Requirements
Before you begin with the Puppet integration, review the supported Puppet versions and system requirements.
- Supported Puppet Versions
-
Foreman supports the following Puppet versions:
-
Puppet 7
-
Puppet 6
-
- System Requirements
-
Before you begin integrating Puppet with your Foreman, ensure that you meet the system requirements. For details, see System Requirements for Puppet 7 or System Requirements for Puppet 6 in the Open Source Puppet documentation.
1.3. Enabling Puppet Integration with Foreman
By default, Foreman does not have any Puppet integration configured. You need to enable the integration as is appropriate for your situation. This means that you can configure Foreman to manage and deploy Puppet server on Foreman server or on Smart Proxy. Additionally, you can deploy Puppet server to Foreman externally and integrate it with Foreman for reporting, facts, and external node classification (ENC).
-
Enable Puppet integration and install Puppet server on Foreman server:
# foreman-installer --enable-foreman-plugin-puppet \ --enable-foreman-cli-puppet \ --foreman-proxy-puppet true \ --foreman-proxy-puppetca true \ --foreman-proxy-content-puppet true \ --enable-puppet \ --puppet-server true \ --puppet-server-foreman-ssl-ca /etc/pki/katello/puppet/puppet_client_ca.crt \ --puppet-server-foreman-ssl-cert /etc/pki/katello/puppet/puppet_client.crt \ --puppet-server-foreman-ssl-key /etc/pki/katello/puppet/puppet_client.key
-
If you want to use Puppet integration on Smart Proxies, enable Puppet integration and install Puppet server on Smart Proxies:
# foreman-installer --foreman-proxy-puppet true \ --foreman-proxy-puppetca true \ --foreman-proxy-content-puppet true \ --enable-puppet \ --puppet-server true \ --puppet-server-foreman-ssl-ca /etc/pki/katello/puppet/puppet_client_ca.crt \ --puppet-server-foreman-ssl-cert /etc/pki/katello/puppet/puppet_client.crt \ --puppet-server-foreman-ssl-key /etc/pki/katello/puppet/puppet_client.key \ --puppet-server-foreman-url "https://foreman.example.com"
Enter the URL of your Foreman server as the value of the
--puppet-server-foreman-url
argument.
1.4. Installing and Configuring Puppet Agent during Host Provisioning
You can install and configure the Puppet agent on a host during the provisioning process. A configured Puppet agent is required on the host for Puppet integration with your Foreman.
-
You created a Product and repository for the upstream Puppet agent, such as
https://yum.puppet.com
orhttps://apt.puppet.com
, and synchronized the repository to Foreman. For more information, see Importing Content in Managing Content. -
You created an activation key that enables the Puppet agent repository for hosts. For more information, see Managing Activation Keys in Managing Content.
-
Navigate to Hosts > Provisioning Templates.
-
Select a provisioning template depending on your host provisioning method. For more information, see Kinds of Provisioning Templates in Provisioning Hosts.
-
Ensure the
puppet_setup
snippet is included as follows:<%= snippet 'puppet_setup' %>
Note that this snippet is already included in the templates shipped with Foreman, such as
Kickstart default
orPreseed default
. -
Enable the Puppet agent using a host parameter in global parameters, a host group, or for a single host. Add a host parameter named
enable-puppet7
, select the boolean type, and set the value totrue
. -
Set configuration for the Puppet agent.
-
If you use an integrated Puppet server, ensure that you select a Puppet Smart Proxy, Puppet CA Smart Proxy, and Puppet environment when you create a host.
-
If you use a non-integrated Puppet server, either set the following host parameters in global parameters, or a host group, or when you create a host:
-
Add a host parameter named
puppet_server
, select the string type, and set the value to the hostname of your Puppet server, such aspuppet.example.com
. -
Optional: Add a host parameter named
puppet_ca_server
, select the string type, and set the value to the hostname of your Puppet CA server, such aspuppet-ca.example.com
. Ifpuppet_ca_server
is not set, the Puppet agent will use the same server aspuppet_server
. -
Optional: Add a host parameter named
puppet_environment
, select the string type, and set the value to the Puppet environment you want the host to use.
-
-
-
Ensure your host has access to the Puppet agent packages from Foreman server by using an appropriate activation key.
1.5. Installing and Configuring Puppet Agent during Host Registration
You can install and configure the Puppet agent on the host during registration. A configured Puppet agent is required on the host for Puppet integration with your Foreman.
-
You created a Product and repository for the upstream Puppet agent, such as
https://yum.puppet.com
orhttps://apt.puppet.com
, and synchronized the repository to Foreman. For more information, see Importing Content in Managing Content. -
You created an activation key that enables the Puppet agent repository for hosts. For more information, see Managing Activation Keys in Managing Content.
-
In the Foreman web UI, navigate to Configure > Global Parameters to add host parameters globally. Alternatively, you can navigate to Configure > Host Groups and edit or create a host group to add host parameters only to a host group.
-
Enable the Puppet agent using a host parameter in global parameters or a host group. Add a host parameter named
enable-puppet7
, select the boolean type, and set the value totrue
. -
Specify configuration for the Puppet agent using the following host parameters in global parameters or a host group:
-
Add a host parameter named
puppet_server
, select the string type, and set the value to the hostname of your Puppet server, such aspuppet.example.com
. -
Optional: Add a host parameter named
puppet_ca_server
, select the string type, and set the value to the hostname of your Puppet CA server, such aspuppet-ca.example.com
. Ifpuppet_ca_server
is not set, the Puppet agent will use the same server aspuppet_server
. -
Optional: Add a host parameter named
puppet_environment
, select the string type, and set the value to the Puppet environment you want the host to use.
Until the BZ2177730 is resolved, you must use host parameters to specify the Puppet agent configuration even in integrated setups where the Puppet server is a Smart Proxy server.
-
-
Navigate to Hosts > Register Host and register your host using an appropriate activation key. For more information, see Registering Hosts in Managing Hosts.
-
Navigate to Infrastructure > Smart Proxies.
-
From the list in the Actions column for the required Smart Proxy server, select Certificates.
-
Click Sign to the right of the required host to sign the SSL certificate for the Puppet agent.
1.6. Installing and Configuring Puppet Agent Manually
You can install and configure the Puppet agent on a host manually. A configured Puppet agent is required on the host for Puppet integration with your Foreman.
-
The host must have a Puppet environment assigned to it.
-
Ensure a repository containing the Puppet agent is enabled on the host, for example apt.puppet.com or yum.puppet.com.
-
Log in to the host as the
root
user. -
Install the Puppet agent package.
-
On hosts running Enterprise Linux 8 and above:
# dnf install puppet-agent
-
On hosts running Enterprise Linux 7 and below:
# yum install puppet-agent
-
On hosts running Debian:
# apt-get install puppet-agent
-
On hosts running SUSE Linux Enterprise Server:
# zypper install puppet-agent
-
-
Add the Puppet agent to
PATH
in your current shell using the following script:. /etc/profile.d/puppet-agent.sh
-
Configure the Puppet agent. Set the
environment
parameter to the name of the Puppet environment to which the host belongs:# puppet config set server foreman.example.com --section agent # puppet config set environment My_Puppet_Environment --section agent
-
Start the Puppet agent service:
# puppet resource service puppet ensure=running enable=true
-
Create a certificate for the host:
# puppet ssl bootstrap
-
In the Foreman web UI, navigate to Infrastructure > Smart Proxies.
-
From the list in the Actions column for the required Smart Proxy server, select Certificates.
-
Click Sign to the right of the required host to sign the SSL certificate for the Puppet agent.
-
On the host, run the Puppet agent again:
# puppet ssl bootstrap
1.7. Performing Configuration Management
After you deploy Puppet agent on a host, you can start performing configuration management with Puppet. This involves the following high-level steps:
-
Managing Puppet modules on the Puppet server, that is installing and updating them.
-
Importing Puppet classes and environments from Puppet modules into Foreman.
-
Optional: Creating config groups from Puppet classes.
-
Configuring overrides of Smart Class parameters on various levels.
-
Assigning Puppet classes or config groups to host groups or individual hosts.
-
Configuring intervals for runs of the Puppet agent on hosts and for configuration enforcement runs of the Puppet server.
-
Monitoring configuration management using reports in the Foreman web UI. For more information, see Monitoring Resources in Administering Foreman.
-
Configuring email notifications. For more information, see Configuring Email Notification Preferences in Administering Foreman.
After assigning Puppet classes or config groups, Foreman runs configuration management automatically in the configured intervals to enforce Puppet configuration on the managed hosts, or you can initiate it manually on demand with the Run Puppet Once feature. For more information, see Running Puppet Once Using SSH.
1.8. Disabling Puppet Integration with Foreman
To discontinue using Puppet in your Foreman, follow this procedure.
Note that the command without the --remove-all-data
argument removes all Puppet-related data in Foreman database.
With the --remove-all-data
argument, the command additionally removes Puppet server data files, including Puppet environments.
Warning
|
If you disable Puppet with the |
-
Puppet is enabled on Foreman.
-
If you have used Puppet server on any Smart Proxies, disable Puppet server on all Smart Proxies:
# foreman-maintain plugin purge-puppet --remove-all-data
-
Disable Puppet server on Foreman server:
# foreman-maintain plugin purge-puppet --remove-all-data
2. Managing Puppet Modules
2.1. Installing a Puppet Module on Foreman Server
You can install a pre-built Puppet module from the Puppet Forge. The Puppet Forge is a repository that provides Puppet modules contributed by the community. Puppet modules flagged as supported are officially supported and tested by Puppet Inc.
This example shows how to add the ntp module to managed hosts.
-
Navigate to forge.puppet.com and search for
ntp
. One of the first modules is puppetlabs/ntp. -
Connect to your Foreman server using SSH and install the Puppet module:
# puppet module install puppetlabs-ntp -i /etc/puppetlabs/code/environments/production/modules
Use the
-i
parameter to specify the path and Puppet environment, for exampleproduction
.Once the installation is completed, the output looks as follows:
Notice: Preparing to install into /etc/puppetlabs/code/environments/production/modules ... Notice: Created target directory /etc/puppetlabs/code/environments/production/modules Notice: Downloading from https://forgeapi.puppet.com ... Notice: Installing -- do not interrupt ... /etc/puppetlabs/code/environments/production/modules |-| puppetlabs-ntp (v8.3.0) |-- puppetlabs-stdlib (v4.25.1) [/etc/puppetlabs/code/environments/production/modules]
An alternative way to install a Puppet module is to copy a folder containing the Puppet module to the module path as mentioned above. Ensure to resolve its dependencies manually.
2.2. Updating a Puppet Module
You can update an existing Puppet module using the puppet
command.
-
Connect to your Puppet server using SSH and find out where the Puppet modules are located:
# puppet config print modulepath
This returns output as follows:
/etc/puppetlabs/code/environments/production/modules:/etc/puppetlabs/code/environments/common:/etc/puppetlabs/code/modules:/opt/puppetlabs/puppet/modules:/usr/share/puppet/modules
-
If the module is located in the path as displayed above, the following command updates a module:
# puppet module upgrade module name
2.3. Managing Puppet Modules with r10k
You can manage Puppet modules and environments using r10k.
r10k uses a list of Puppet roles from a Puppetfile
to download Puppet modules from the Puppet Forge or git repositories.
It does not handle dependencies between Puppet modules.
A Puppetfile looks as follows:
forge "https://forge.puppet.com"
mod "puppet-nginx",
:git => "https://github.com/voxpupuli/puppet-nginx.git",
:ref => "master"
mod "puppetlabs/apache"
mod "puppetlabs/ntp", "8.3.0"
3. Importing Puppet Classes and Environments into Foreman
Import Puppet classes and environments from the installed Puppet modules to Foreman server or any attached Smart Proxy server before you assign any of the classes to managed hosts.
-
Ensure to select Any Organization and Any Location as context, otherwise the import might fail.
-
In the Foreman web UI, navigate to Configure > Classes or Configure > Environments.
-
Click the Import button in the upper right corner and select which Smart Proxy you want to import modules from. You may typically choose between your Foreman server or any attached Smart Proxy server.
-
Select the Puppet environments to import using checkboxes on the left.
-
Click the Update button to import the Puppet environments and classes to Foreman.
-
The import should result in a notification as follows:
Successfully updated environments and Puppet classes from the on-disk Puppet installation
4. Creating a Custom Puppet Environment
You can create a Puppet environment within your Foreman.
-
In the Foreman web UI, navigate to Configure > Puppet Environments.
-
Click Create Puppet Environment to create a Puppet environment.
-
Enter a Name, alphanumeric characters and underscores are allowed, such as
example_environment
. -
Optional: Set a location context.
-
Optional: Set an organization context.
-
Click Submit to create the Puppet environment.
Note that before you run an import of Puppet modules into Foreman, the environment must already exist as the folder /etc/puppetlabs/code/environments/example_environment
on the Puppet server and contain installed Puppet modules.
5. Creating a Puppet Config Group
A Puppet config group is a named list of Puppet classes that allows you to combine their capabilities and assign them to managed hosts at a click. This is equivalent to the concept of profiles in pure Puppet.
-
In the Foreman web UI, navigate to Configure > Config Groups.
-
Click the Create Config Group button.
-
Select the classes you want to add to the config group.
-
Choose a meaningful Name for the Puppet config group.
-
Add selected Puppet classes to the Included Classes field.
-
-
Click Submit to save the changes.
6. Configuring Puppet Smart Class Parameters
6.1. Puppet Parameter Hierarchy
Puppet parameters are structured hierarchically. Parameters at a lower level override parameters of the higher levels:
-
Global parameters
-
Organization parameters
-
Location parameters
-
Host group parameters
-
Host parameters
For example, host specific parameters override the parameter at any higher level, and location parameters only override parameters at the organization or global level. This feature is especially useful when you use locations or organizations to group hosts.
6.2. Overriding a Smart Class Parameter Globally
You can configure a Puppet class after you have imported it to Foreman server. This example overrides the default list of ntp servers.
-
In the Foreman web UI, navigate to Configure > Classes.
-
Select the ntp Puppet class to change its configuration.
-
Select the Smart Class Parameter tab and search for servers.
-
Ensure the Override checkbox is selected.
-
Set the Parameter Type drop down menu to array.
-
Insert a list of ntp servers as Default Value:
["0.de.pool.ntp.org","1.de.pool.ntp.org","2.de.pool.ntp.org","3.de.pool.ntp.org"]
An alternative way to describe the array is the
yaml
syntax:- 0.de.pool.ntp.org - 1.de.pool.ntp.org - 2.de.pool.ntp.org - 3.de.pool.ntp.org
-
Click the Submit button after adding the values. This changes the default configuration of the Puppet module
ntp
.
6.3. Overriding a Smart Class Parameter for an Organization
You can use groups of hosts to override Puppet parameters for multiple hosts at once. The following example chooses the organization context to illustrate setting context based parameters.
Note that organization-level Puppet parameters are overridden by location-level Puppet parameters.
-
In the Foreman web UI, navigate to Configure > Classes.
-
Click a class name to select a class.
-
On the Smart Class Parameter tab, select a parameter.
-
Use the Order list to define the hierarchy of the Puppet parameters. The individual host (
fqdn
) marks the most and the organization context (organization
) the least relevant. -
Check Merge Overrides if you want to add all further matched parameters after finding the first match.
-
Check Merge Default if you want to also include the default value even if there are more specific values defined.
-
Check Avoid Duplicates if you want to create a list of unique values for the selected parameter.
-
The matcher field requires an attribute type from the order list.
-
Use the Add Matcher button to add more matchers.
-
Click Submit to save the changes.
6.4. Overriding a Smart Class Parameter for a Location
You can use groups of hosts to override Puppet parameters for multiple hosts at once. The following examples chooses the location context to illustrate setting context based parameters.
-
In the Foreman web UI, navigate to Configure > Classes.
-
Click a class name to select a class.
-
On the Smart Class Parameter tab, select a parameter.
-
Use the Order list to define the hierarchy of the Puppet parameters. The individual host (
fqdn
) marks the most and the location context (location
) the least relevant. -
Check Merge Overrides if you want to add all further matched parameters after finding the first match.
-
Check Merge Default if you want to also include the default value even if there are more specific values defined.
-
Check Avoid Duplicates if you want to create a list of unique values for the selected parameter.
-
The matcher field requires an attribute type from the order list. For example, you can choose
Paris
as location context and set the value to French ntp servers. -
Use the Add Matcher button to add more matchers.
-
Click Submit to save the changes.
6.5. Overriding a Smart Class Parameter on an Individual Host
You can override parameters on individual hosts. This is recommended if you have multiple hosts and only want to make changes to a single one.
-
In the Foreman web UI, navigate to Hosts > All Hosts.
-
Click a host name to select a host.
-
Click Edit.
-
On the Host tab, select a Puppet Environment.
-
Select the Puppet ENC tab.
-
Click the Override button to edit the Puppet parameter.
-
Click Submit to save the changes.
7. Assigning a Puppet Class to a Host Group
Use a host group to assign the ntp Puppet class to multiple hosts at once. Every host you deploy based on this host group has this Puppet class installed.
-
In the Foreman web UI, navigate to Configure > Host Groups to create a host group or edit an existing one.
-
On the Host Group tab, set the following parameters:
-
The Lifecycle Environment describes the stage in which certain versions of content are available to hosts.
-
The Content View is comprised of products and allows for version control of content repositories.
-
The Environment allows you to supply a group of hosts with their own dedicated configuration.
-
-
Navigate to the Puppet ENC tab.
-
Add the Puppet class to the Included Classes or to the Included Config Groups if a Puppet config group is configured.
-
Click Submit to save the changes.
8. Assigning a Puppet Class to an Individual Host
-
In the Foreman web UI, navigate to Hosts > All hosts.
-
Click on the Edit button of the host you want to add the
ntp
Puppet class to. -
Select the Puppet ENC tab and look for the ntp class.
-
Click the + symbol next to
ntp
to add the ntp submodule to the list of included classes. -
Click the Submit button at the bottom to save your changes.
TipIf the Puppet classes tab of an individual host is empty, check if it is assigned to the proper Puppet environment.
-
Verify the Puppet configuration.
-
Navigate to Hosts > All Hosts and select the host.
-
From the top overflow menu, select Legacy UI.
-
Under Details, click the Puppet YAML button. This produces output similar as follows:
--- parameters: // shortened YAML output classes: ntp: servers: '["0.de.pool.ntp.org","1.de.pool.ntp.org","2.de.pool.ntp.org","3.de.pool.ntp.org"]' environment: production ...
-
-
Verify the ntp configuration.
Connect to your host using SSH and check the content of
/etc/ntp.conf
.This example assumes your host is running CentOS 7. Other operating systems may store the ntp config file in a different path.
TipYou may need to run the Puppet agent on your host by executing the following command:
# puppet agent -t
-
Running the following command on the host checks which ntp servers are used for clock synchronization:
# cat /etc/ntp.conf
This returns output similar as follows:
# ntp.conf: Managed by puppet. server 0.de.pool.ntp.org server 1.de.pool.ntp.org server 2.de.pool.ntp.org server 3.de.pool.ntp.org
You now have a working ntp module which you can add to a host or group of hosts to roll out your ntp configuration automatically.
9. Enforcing Puppet Configuration on Managed Hosts
You can enforce configuration from Foreman either manually on demand (run once) or automatically in configurable intervals.
9.1. Running Puppet Once Using SSH
Assign the proper job template to the Run Puppet Once feature to run Puppet on managed hosts.
-
In the Foreman web UI, navigate to Administer > Remote Execution Features.
-
Select the
puppet_run_host
remote execution feature. -
Assign the
Run Puppet Once - SSH Default
job template.
Run Puppet on managed hosts by running a job and selecting category Puppet
and template Run Puppet Once - SSH Default
.
Alternatively, click the Run Puppet Once button in the Schedule Remote Job drop down menu on the host details page.
9.2. Understanding Intervals of Automatic Enforcement
Foreman considers hosts to be out of sync if the last Puppet report is older than the combined values of outofsync_interval
and puppet_interval
set in minutes.
By default, the Puppet agent on managed hosts runs every 30 minutes,
the puppet_interval
is set to 35 minutes and the global outofsync_interval
is set to 30 minutes.
The effective time after which hosts are considered out of sync is the sum of outofsync_interval
and puppet_interval
.
For example, setting the global outofsync_interval
to 30 and the puppet_interval
to 60 results in a total of 90 minutes after which the host status changes to out of sync
.
9.3. Setting the Puppet Agent Run Interval on a Host
Set the interval when the Puppet agent runs and sends reports to Foreman.
-
Connect to your managed host using SSH.
-
Add the Puppet agent run interval to
/etc/puppetlabs/puppet/puppet.conf
, for exampleruninterval = 1h
.
9.4. Setting the Global Out-of-Sync Interval
-
In the Foreman web UI, navigate to Administer > Settings.
-
On the General tab, edit Out of sync interval. Set a duration, in minutes, after which hosts are considered to be out of sync.
You can also override this interval on host groups or individual hosts by adding the
outofsync_interval
parameter.
9.5. Setting the Puppet Out-of-Sync Interval
-
In the Foreman web UI, navigate to Administer > Settings, and click the Config Management tab.
-
In the Puppet interval field, set the value to the duration, in minutes, after which hosts reporting using Puppet are considered to be out of sync.
9.6. Overriding Out-of-Sync Interval for a Host Group
-
In the Foreman web UI, navigate to Configure > Host Groups.
-
Select a host group.
-
On the Parameters tab, click Add Parameter.
-
In the Name field, enter
outofsync_interval
. -
From the Type dropdown menu, select
integer
. -
In the Value field, enter the new interval in minutes.
-
Click the Submit button.
9.7. Overriding Out-of-Sync Interval for an Individual Host
-
In the Foreman web UI, navigate to Hosts > All Hosts.
-
Click Edit for a selected host.
-
On the Parameters tab, click Add Parameter.
-
In the Name field, enter
outofsync_interval
. -
From the Type dropdown menu, select
integer
. -
In the Value field, enter the new interval in minutes.
-
Click the Submit button.