Configuring Hosts Using Salt

1. Using Salt for Configuration Management

Use this section as a guide to configuring Foreman to use Salt for configuration management on managed hosts.

1.1. Introduction to Salt

Salt can be used as a configuration management tool similar to Ansible or Puppet.

This guide describes how to use Salt for configuration management in Foreman. This guide contains information about how to install the Salt plugin, how to integrate Foreman with an existing Salt Master, and how to configure managed hosts with Salt.

Note	Salt offers two distinct modes of operation: Clientless using SSH or the Salt Minion client software.

1.2. Salt Architecture

You need a Salt Master that either runs on your Foreman server or Smart Proxy server with the Salt plugin enabled. You can also use an existing Salt Master by installing and configuring the relevant Smart Proxy features on the existing Salt Master host.

For more information on installing a Salt Master, consult the official Salt documentation. Alternatively, use the bootstrap instructions found in the official Salt package repository.

Terminology

Managed hosts are referred to as Salt Minions.
Information in form of key-value pairs gathered from Salt Minions is referred to as Salt Grains.
Configuration templates are referred to as Salt States.
Bundles of Salt States are referred to as Salt Environments.

Use the same Salt version on the Salt Master as you are using on your Salt Minions. You can use Foreman’s content management to provide managed hosts with the correct version of the Salt Minion client software.

Table 1. Required Ports from Salt Master to Salt Minions
Port	Protocol	Service	Required For
4505 and 4506	TCP	HTTPS	Salt Master to Salt Minions
9191	TCP	HTTPS	Salt API

1.3. Installing Salt Plugin

To configure managed hosts with Salt, you need to install the Salt plugin.

Procedure

Connect to your Foreman using SSH:
```
# ssh root@foreman.example.com
```

Install the Salt plugin:

# foreman-installer \
--enable-foreman-plugin-salt \
--enable-foreman-proxy-plugin-salt

1.4. Configuring Salt

After you have installed the Salt plugin, you need to connect it to a Salt Master. This is required when adding Salt support to an existing Foreman installation, when adding an existing Salt Master to Foreman, or when setting up Salt on Smart Proxy.

Perform all actions on your Salt Master unless noted otherwise. This is either your Foreman server or Smart Proxy server with Salt enabled.

1.4.1. Configuring Foreman

You need to configure Foreman to use the Salt plugin.

Procedure

Connect to your Foreman using SSH:
```
# ssh root@foreman.example.com
```

Extend the /etc/sudoers file to allow the foreman-proxy user to run Salt:

Cmd_Alias SALT = /usr/bin/salt, /usr/bin/salt-key
foreman-proxy ALL = NOPASSWD: SALT
Defaults:foreman-proxy !requiretty

Add a user called saltuser to access the Salt API:
```
# adduser --no-create-home --shell /bin/false --home-dir / saltuser
# passwd saltuser
```
Enter the password for the Salt user twice.

Note

The command adduser saltuser -p password does not work. Using it prevents you from importing Salt States.

1.4.2. Configuring Salt Master

Configure your Salt Master to configure managed hosts using Salt.

Procedure

Connect to your Salt Master using SSH:
```
# ssh root@salt-master.example.com
```
Set Foreman as an external node classifier in /etc/salt/master for Salt:
```
master_tops:
  ext_nodes: /usr/bin/foreman-node
```

Enable Salt Pillar data for use with Foreman:

ext_pillar:
  - puppet: /usr/bin/foreman-node

Add a Salt Environment called base that is associated with the /srv/salt directory:
```
file_roots:
  base:
    - /srv/salt
```

Use the saltuser user for the Salt API and specify the connection settings in /etc/salt/master:

external_auth:
  pam:
    saltuser:
      - '@runner'

rest_cherrypy:
  port: 9191
  host: 0.0.0.0
  ssl_key: /etc/puppetlabs/puppet/ssl/private_keys/foreman.example.com.pem
  ssl_crt: /etc/puppetlabs/puppet/ssl/certs/foreman.example.com.pem

Optional: Use Salt as a remote execution provider.

You can run arbitrary commands on managed hosts by using the existing connection from your Salt Master to Salt Minions. Configure the foreman-proxy user to run Salt commands in /etc/salt/master:
```
publisher_acl:
  foreman-proxy:
    - state.template_str
```

1.4.3. Authenticating Salt Minions Using Salt Autosign Grains

Configure Foreman to automatically accept Salt Minions using Salt autosign Grains.

Procedure

Configure the Grains key file on the Salt Master and add a reactor in the /etc/salt/master file:
```
autosign_grains_dir: /var/lib/foreman-proxy/salt/grains

reactor:
  - 'salt/auth':
    - /usr/share/foreman-proxy/salt/reactors/foreman_minion_auth.sls
```
The Grains file holds the acceptable keys when you deploy Salt Minions. The reactor initiates an interaction with the Salt plugin if the Salt Minion is successfully authenticated.
Add the reactor to the salt/auth event.
Copy the Salt runners into your file_roots runners directory. The directory depends on your /etc/salt/master config. If it is configured to use /srv/salt, create the runners folder /srv/salt/_runners and copy the Salt runners into it.
```
# mkdir -p /srv/salt/_runners
# cp /usr/share/foreman-proxy/salt/runners/* /srv/salt/_runners/
```
Restart the Salt Master service:
```
# systemctl restart salt-master
```
Enable the Salt reactors and runners in your Salt Environment:
```
# salt-run saltutil.sync_all
```

1.4.4. Authenticating Salt Minions Using Host Names

Configure Foreman to authenticate Salt Minions based on their host names. This relies on the autosign.conf file that stores the host names of Salt Minions the Salt Master accepts.

Procedure

Connect to your Salt Master using SSH:
```
# ssh root@salt-master.example.com
```
Add the foreman-proxy user that is running Salt to the root user group:
```
# usermod -a -G foreman-proxy root
```

Enable the autosign.conf file in /etc/salt/master:

autosign_file: /etc/salt/autosign.conf
permissive_pki_access: True

Create the /etc/salt/autosign.conf file and set appropriate ownership and permissions:

# touch /etc/salt/autosign.conf
# chown root:foreman-proxy /etc/salt/autosign.conf
# chmod 660 /etc/salt/autosign.conf

1.4.5. Enabling Salt Grains Upload

Managed hosts running the Salt Minion client software can upload Salt Grains to Foreman server or Smart Proxy server. Salt Grains are collected system properties, for example the operating system or IP address of a Salt Minion.

Procedure

Connect to your Salt Master using SSH:
```
# ssh root@salt-master.example.com
```

Edit /etc/salt/foreman.yaml on your Salt Master:

:proto: https
:host: foreman.example.com
:port: 443
:ssl_ca: "/etc/puppetlabs/puppet/ssl/ssl_ca.pem"
:ssl_cert: "/etc/puppetlabs/puppet/ssl/client_cert.pem"
:ssl_key: "/etc/puppetlabs/puppet/ssl/client_key.pem"
:timeout: 10
:salt: /usr/bin/salt
:upload_grains: true

1.4.6. Configuring Salt API

Configure the Salt API in /etc/foreman-proxy/settings.d/salt.yml.

Procedure

Connect to your Salt Master using SSH:
```
# ssh root@salt-master.example.com
```

Edit /etc/foreman-proxy/settings.d/salt.yml:

:use_api: true
:api_auth: pam
:api_url: https://foreman.example.com:9191
:api_username: saltuser
:api_password: password

Ensure to use the password of the previously created saltuser.

1.4.7. Activating Salt

Procedure

Connect to your Salt Master using SSH:
```
# ssh root@salt-master.example.com
```

Restart the Salt services:

# systemctl restart salt-master salt-api

Connect to your Foreman using SSH:
```
# ssh root@foreman.example.com
```
Restart Foreman services:
```
# foreman-maintain service restart
```
Refresh your Smart Proxy features.
1. In the Foreman web UI, navigate to Infrastructure > Smart Proxies.
2. Click Refresh for the relevant Smart Proxy.

1.5. Setting up Salt Minions

Salt Minions require the Salt Minion client software to interact with your Salt Master.

1.5.1. Using Foremans Content Management

Provide managed hosts with the required Salt Minion client software using Foreman.

Procedure

Create a product called Salt. For more information, see Creating a Custom Product.
Create a repository within the Salt product for each operating system supported by Foreman that you want to install the Salt Minion client software on. For more information, see Adding RPM Repositories or Adding DEB Repositories.

Add the operating system to the name of the repository, for example Salt for Ubuntu 20.04.

You can find the upstream URL for the Salt packages on the official Salt package repository. The URL depends on both the Salt version and the operating system, for example https://repo.saltproject.io/py3/ubuntu/20.04/amd64/3003/.
Synchronize the previously created products. For more information, see Synchronizing Repositories.
Create a Content View for each repository. For more information, see Creating a Content View.
Create a Composite Content View for each major version of each operating system to make the new content available. For more information, see Create a Composite Content View.
Add each of your operating system specific Salt Content Views to your main Composite Content View for that operating system and version.
Publish a new version of the Composite Content View from the previous step.
Promote the Content View from the previous step to your lifecycle environments as appropriate. For more information, see Promoting a Content View.
Optional: Create activation keys for your Composite Content View and lifecycle environment combinations.

1.5.2. Creating a Host Group With Salt

To bundle provisioning and configuration settings, you can create a host group with Salt enabled for managed hosts.

Procedure

In the Foreman web UI, navigate to Configure > Host Groups.
Click Create Host Group.
Click the Host Group tab and select a Salt Environment and a Salt Master.
Click the Salt States tab and assign Salt States to your host group.
Click the Activation Keys tab and select an activation key containing the Salt Minion client software.
Click Submit to save your host group.

Managed hosts deployed using this host group automatically install and configure the required Salt Minion client software and register with your Salt Master. For more information, see Creating a Host Group Managing Hosts.

1.5.3. Deploying Minion Hosts

Deploy managed hosts that are fully provisioned and configured for Salt usage.

Prerequisites to deploy Salt Minions

A Salt Master
A Salt Environment
A Content View containing the required Salt Minion client software
An activation key
A lifecycle environment

Procedure

In the Foreman web UI, navigate to Hosts > Create Host.
Select the previously created host group with Salt enabled.
Click Submit to deploy a host.

1.5.4. Verifying the Connection between Salt Master and Salt Minions

Verify the connection between your Salt Master and Salt Minions.

Procedure

Connect to your Salt Master using SSH:
```
# ssh root@salt-master.example.com
```
Ping your Salt Minions:
```
# salt "*" test.ping
```
Display all Salt Grains of all connected Salt Minions:
```
# salt "*" grains.items
```

1.6. Salt Usage

Salt Minions managed by Foreman are associated with a Salt Master and a Salt Environment. The associated Salt Environment within Foreman must match the actual Salt Environment from the file_roots option in the /etc/salt/master file. You can configure managed hosts with Salt once they are associated with your Foreman server or Smart Proxy server and the Salt Minion client software is installed.

1.6.1. Using the Salt Hammer CLI

You can use Hammer CLI to configure managed hosts using Salt. Run hammer --help for more information.

Prerequisites

Install hammer_cli_foreman_salt on your Foreman server

Examples

Creating a Salt State:

# hammer salt-state create \
--name My_Salt_State

Viewing information about a Salt Minion:

# hammer salt-minion info \
--name salt-minion.example.com

Adding Salt States to a Salt Minion:

# hammer salt-minion update \
--name salt-minion.example.com \
--salt-states My_Salt_State

1.6.2. Using the Salt API

Foreman Salt extends the Foreman REST API with Salt-specific features. View the full API documentation on your Foreman server at http://foreman.example.com/apidoc/v2.html.

Example

Use curl to get a list of keys from smartproxy.example.com:

# curl -u My_User_Name:My_Password \
-H "Accept: version=2,application/json" \
https://foreman.example.com/salt/api/v2/salt_keys/smartproxy.example.com

1.6.3. Importing Salt States

A Salt State configures parts of a host, for example, a service or the installation of a package. You can import Salt States from your Salt Master to Foreman. The Salt Master configuration in this guide uses a Salt Environment called base that includes the Salt States stored in /srv/salt/.

Procedure

In the Foreman web UI, navigate to Configure > Salt States and click Import from FQDN.
Optional: Click Edit to assign Salt States to Salt Environments.
Optional: Click Delete to remove a Salt State from your Foreman. This only removes the Salt State from Foreman, not from the disk of your Salt Master.
Click Submit to import the Salt States.

After you have imported Salt States, you can assign them to hosts or Host Groups. Salt applies these Salt States to any hosts they are assigned to every time you run state.highstate. For more information, see Running Salt.

Note	Configure the paths for Salt States and Salt Pillars in `/etc/salt/master`. By default, Salt States are located in `/srv/salt` and and Salt Pillars in `/srv/pillar`.

1.6.4. Viewing Salt Autosign Keys

The Salt Keys page lists hosts and their Salt keys. You can manually accept, reject, or delete keys.

Use the Salt autosign feature to automatically accept signing requests from managed hosts. By default, managed hosts are supplied with a Salt key during host provisioning.

Note	This feature only covers the autosign via `autosign.conf` authentication procedure.

Procedure

In the Foreman web UI, navigate to Infrastructure > Smart Proxies.
Select a Smart Proxy.
In the Actions drop down menu, click Salt Keys.

1.6.5. Viewing Salt Reports

Salt reports are uploaded to Foreman every ten minutes using the /etc/cron.d/smart_proxy_salt cron job. You can trigger this action manually on your Salt Master:

# /usr/sbin/upload-salt-reports

Procedure

To view Salt reports, in the Foreman web UI, navigate to Monitor > Config Management Reports.
To view Salt reports associated with an individual host, in the Foreman web UI, navigate to Hosts > All Hosts and select a single host.

1.6.6. Enabling Salt Report Uploads

The Salt Master can directly upload Salt reports to Foreman.

Procedure

Connect to your Salt Master using SSH:
```
# ssh root@salt-master.example.com
```

Ensure the reactor is present:

# file /usr/share/foreman-proxy/salt/reactors/foreman_report_upload.sls

Copy report upload script:

# cp /usr/share/foreman-proxy/salt/runners/foreman_report_upload.py /srv/salt/_runners/

Add the reactor to /etc/salt/master:

reactor:
  - 'salt/auth':
    - /usr/share/foreman-proxy/salt/reactors/foreman_minion_auth.sls
  - 'salt/job/*/ret/*':
    - /usr/share/foreman-proxy/salt/reactors/foreman_report_upload.sls

Restart the Salt Master:
```
# systemctl restart salt-master
```
Enable the new runner:
```
# salt-run saltutil.sync_all
```
If you use a cron job to upload facts from your Salt Master to Foreman, disable the cron job:
```
# rm -f /etc/cron.d/smart_proxy_salt
```

Alternatively, you can upload Salt reports from your Salt Master to Foreman manually:

# /usr/sbin/upload-salt-reports

1.6.7. Salt Variables

You can configure Salt Variables within Foreman. The configured values are available as Salt Pillar data.

1.6.8. Viewing ENC Parameters

You can use Foreman as an external node classifier for Salt. Click Salt ENC on the host overview page to view assigned Salt States. This shows a list of parameters that are made available for Salt usage as Salt Pillar data.

You can check what parameters are truly available on the Salt side by completing the following procedure.

Procedure

Connect to your Salt Master using SSH:
```
# ssh root@salt-master.example.com
```
View available ENC parameters:
```
# salt '*' pillar.items
```
Optional: Refresh the Salt Pillar data if a parameter is missing:
```
# salt '*' saltutil.refresh_pillar
```

1.6.9. Running Salt

You can run arbitrary Salt functions using Foreman’s remote execution features.

Most commonly, you can run salt.highstate on one or more Salt Minions. This applies all relevant Salt States on your managed hosts.

Procedure

In the Foreman web UI, navigate to Hosts > All Hosts.
Select one or multiple hosts.
Click Run Salt from the actions drop down menu.

Alternatively, you can click Run Salt from the Schedule Remote Job drop down menu on the host overview page.

Triggering a remote execution job takes you to the corresponding job overview page.

If you select a host from the list, you can see the output of the remote job in real time.

To view remote jobs, in the Foreman web UI, navigate to Monitor > Jobs.

Running state.highstate generates a Salt report. Note that reports are uploaded to Foreman every ten minutes. For more information, see Viewing Salt Reports.

You can use Foreman’s remote execution features to run arbitrary Salt functions. This includes an optional test mode that tells you what would happen without actually changing anything.

Procedure

In the Foreman web UI, navigate to Hosts > All Hosts
Select a host.
Click Schedule Remote Job.
For the Job category field, select Salt-Call from the drop down menu.
For the Job template field, select Salt Run function from the drop down menu.
In the Function field, enter the name of the function that you want to trigger, for example, pillar.items or test.ping.
Optional: Click Display advanced fields and enable test-run.
Click Submit to run the job.

You can also schedule remote jobs or run them recurrently. For more information, see Configuring and Setting up Remote Jobs in Managing Hosts.

Alternatively, you can define recurrent actions using the native Salt way. For example, you can schedule hourly state.highstate runs on individual Salt Minions by extending /etc/salt/minion:

schedule:
  highstate:
    function: state.highstate
    minutes: 60

1.7. Salt Example

This example uses a Salt State to manage the /etc/motd file on one or more Salt Minions. It demonstrates the use of Foreman as an external node classifier and the use of Salt Grains.

Procedure

Create a global parameter called vendor_name with the string Foreman as its value.
Add a new Salt State called motd to your Salt Master.
Create the /srv/salt/motd/ directory:
```
# mkdir -p /srv/salt/motd/
```

Create /srv/salt/motd/init.sls as a Salt State file:

/etc/motd:
  file.managed:
    - user: root
    - group: root
    - mode: 0644
    - source: salt://motd/motd.template
    - template: jinja

Create /srv/salt/motd/motd.template as a template referenced by the Salt State file:
```
Welcome to {{ grains['fqdn'] }} Powered by {{ salt['pillar.get']('vendor_name') }}
```
Access the fqdn Salt Grain from within this template and retrieve the vendor_name parameter from the Salt Pillar.
Import the motd Salt State into Foreman. For more information, see Importing Salt States.
Verify that Salt has been given access to the vendor_name parameter by running either of the following commands on your Salt Master:
```
# salt '*' pillar.items | grep -A 1 vendor_name
# salt '*' pillar.get vendor_name
```
If the output does not include the value of the vendor_name parameter, you must refresh the Salt Pillar data first:
```
# salt '*' saltutil.refresh_pillar
```
For information about how to refresh Salt Pillar data, see Viewing ENC Parameters.
Add the motd Salt State to your Salt Minions or a host group.
Run state.highstate to apply the Salt State. For more information, see Running Salt.
Optional: Verify the contents of /etc/motd on a Salt Minion.

1.8. Additional Resources for Salt

The official Salt documentation is a good entry point when starting with Salt.
You can download the Salt packages from the official Salt package repository.