1. Upgrade Overview

This chapter details the prerequisites and available upgrade paths to Foreman nightly. Review this information before upgrading your current Foreman installation.

In this guide, the terms update, upgrade, and migrate have the following meanings:

Upgrading

The process of advancing your Foreman server and Smart Proxy server installations from a y-stream release to the next, for example Foreman 3.1 to Foreman nightly.

Updating

The process of advancing your Foreman server and Smart Proxy server installations from a z-stream release to the next, for example Foreman nightly.0 to Foreman nightly.1.

Migrating

The process of moving an existing Foreman installation to another Red Hat Enterprise Linux server.

Note that you can upgrade Smart Proxies separately from Foreman. For more information, see Upgrading Smart Proxies Separately from Foreman.

1.1. Prerequisites

Upgrading to Foreman nightly affects your entire Foreman infrastructure. Before proceeding, complete the following:

  • Read the Foreman nightly Release Notes.

  • Review this guide so that you are aware of the upgrade process and its impact.

  • Plan your upgrade path. For more information, see Upgrade Paths.

  • Plan for the required downtime. Foreman services are shut down during the upgrade. The upgrade process duration might vary depending on your hardware configuration, network speed, and the amount of data that is stored on the server.

    Upgrading Foreman takes approximately 1 - 2 hours.

    Upgrading Smart Proxy takes approximately 10 - 30 minutes.

  • Ensure that you have sufficient storage space on your server. For more information, see Preparing your Environment for Installation in Installing Foreman server from a Connected Network and Preparing your Environment for Installation in Installing Smart Proxy server.

  • Back up your Foreman server and all Smart Proxy servers. For more information, see Backing Up Foreman server and Smart Proxy server in the Administering Foreman guide.

  • Plan for updating any scripts you use that contain Foreman API commands because some API commands differ between versions of Foreman. For more information about changes in the API, see the Red Hat Knowledgebase article API Changes Between Foreman Versions on the Red Hat Customer Portal.

Warning
If you customize configuration files, manually or use a tool such as Hiera, these customizations are overwritten when the installation script runs during upgrading or updating. You can use the --noop option with the foreman-installer script to test for changes. For more information, see the Red Hat Knowledgebase solution How to use the noop option to check for changes in Foreman config files during an upgrade.

1.2. Upgrade Paths

You can upgrade to Foreman nightly from Foreman 3.1. Foreman servers and Smart Proxy servers on earlier versions must first be upgraded to Foreman 3.1. For more information, see the Foreman 3.1 Upgrade documentation.

Overview of Foreman nightly Upgrade Paths

The high level steps in upgrading to Foreman nightly are as follows.

  1. Upgrade Foreman server to nightly. For more information, see Upgrading Foreman server.

  2. Upgrade all Smart Proxy servers to nightly. For more information, see Upgrading Smart Proxy servers.

1.3. Following the Progress of the Upgrade

Because of the lengthy upgrade time, use a utility such as screen to suspend and reattach a communication session. You can then check the upgrade progress without staying connected to the command shell continuously. For more information about using the screen command, see How do I use the screen command? article in the Red Hat Knowledge Base. For more information, see the screen manual page.

If you lose connection to the command shell where the upgrade command is running you can see the logs in /var/log/foreman-installer/foreman.log to check if the process completed successfully.

1.4. Upgrading Smart Proxies Separately from Foreman

You can upgrade Foreman to version nightly and keep Smart Proxies at version 3.1 until you have the capacity to upgrade them too.

All the functionality that worked previously works on 3.1 Smart Proxies. However, the functionality added in the nightly release will not work until you upgrade Smart Proxies to nightly.

Upgrading Smart Proxies after upgrading Foreman can be useful in the following example scenarios:

  1. If you want to have several smaller outage windows instead of one larger window.

  2. If Smart Proxies in your organization are managed by several teams and are located in different locations.

  3. If you use a load-balanced configuration, you can upgrade one load-balanced Smart Proxy and keep other load-balanced Smart Proxies at one version lower. This allows you to upgrade all Smart Proxies one after another without any outage.

2. Upgrading Foreman

Use the following procedures to upgrade your existing Foreman to Foreman nightly:

Before upgrading, see Prerequisites.

2.1. Upgrading Foreman server

This section describes how to upgrade Foreman server from 3.1 to nightly. You can upgrade from any minor version of Foreman server 3.1.

Before You Begin
  • Note that you can upgrade Smart Proxies separately from Foreman. For more information, see Upgrading Smart Proxies Separately from Foreman.

  • Review and update your firewall configuration prior to upgrading your Foreman server. For more information, see Preparing your environment for installation in Installing Foreman server.

  • Back up and remove all Foreman hooks before upgrading. Restore any hooks only after Foreman is known to be working after the upgrade is complete.

  • If you have edited any of the default templates, back up the files either by cloning or exporting them. Cloning is the recommended method because that prevents them being overwritten in future updates or upgrades. To confirm if a template has been edited, you can view its History before you upgrade or view the changes in the audit log after an upgrade. In the Foreman web UI, navigate to Monitor > Audits and search for the template to see a record of changes made. If you use the export method, restore your changes by comparing the exported template and the default template, manually applying your changes.

Smart Proxy Considerations
  • Note that Foreman server upgraded from 3.1 to nightly can use Smart Proxy servers still at 3.1.

FIPS mode

You cannot upgrade Foreman server from a RHEL base system that is not operating in FIPS mode to a RHEL base system that is operating in FIPS mode.

To run Foreman server on a Red Hat Enterprise Linux base system operating in FIPS mode, you must install Foreman on a freshly provisioned RHEL base system operating in FIPS mode. For more information, see Preparing your environment for installation in Installing Foreman server.

2.1.1. Upgrading a Connected Foreman server

Use this procedure for a Foreman server with access to the public internet

Warning
If you customize configuration files, manually or using a tool such as Hiera, these changes are overwritten when the installation script runs during upgrading or updating. You can use the --noop option with the foreman-installer script to test for changes. For more information, see the Red Hat Knowledgebase solution How to use the noop option to check for changes in Foreman config files during an upgrade.
Upgrade Foreman server
  1. Create a backup.

  2. Optional: If you made manual edits to DNS or DHCP configuration in the /etc/zones.conf or /etc/dhcp/dhcpd.conf files, back up the configuration files because the installer only supports one domain or subnet, and therefore restoring changes from these backups might be required.

  3. Optional: If you made manual edits to DNS or DHCP configuration files and do not want to overwrite the changes, enter the following command:

    # foreman-installer --foreman-proxy-dns-managed=false \
    --foreman-proxy-dhcp-managed=false
  4. In the Foreman web UI, navigate to Hosts > Discovered hosts. On the Discovered Hosts page, power off and then delete the discovered hosts. From the Select an Organization menu, select each organization in turn and repeat the process to power off and delete the discovered hosts. Make a note to reboot these hosts when the upgrade is complete.

  5. Check when the kernel packages were last updated:

    # rpm -qa --last | grep kernel
  6. Optional: If a kernel update occurred since the last reboot, reboot the system:

    # reboot
  7. If using a BASH shell, after a successful or failed upgrade, enter:

    # hash -d foreman-maintain service 2> /dev/null

2.2. Upgrading Smart Proxy servers

This section describes how to upgrade Smart Proxy servers from 3.1 to nightly.

Before You Begin
  • You must upgrade Foreman server before you can upgrade any Smart Proxy servers. Note that you can upgrade Smart Proxies separately from Foreman. For more information, see Upgrading Smart Proxies Separately from Foreman.

  • If you use Content Views to control updates to the base operating system of Smart Proxy server, update those Content Views with new repositories and publish their updated versions. For more information, see Managing Content Views in the Content Management Guide.

  • Ensure the Smart Proxy’s base system is registered to the newly upgraded Foreman server.

  • Ensure the Smart Proxy has the correct organization and location settings in the newly upgraded Foreman server.

  • Review and update your firewall configuration prior to upgrading your Smart Proxy server. For more information, see Preparing Your Environment for Smart Proxy Installation in Installing Smart Proxy server.

Upgrading Smart Proxy servers
  1. Create a backup.

  2. Check when the kernel packages were last updated:

    # rpm -qa --last | grep kernel
  3. Optional: If a kernel update occurred since the last reboot, reboot the system:

    # reboot
  4. Optional: If you made manual edits to DNS or DHCP configuration files, check and restore any changes required to the DNS and DHCP configuration files using the backups made earlier.

  5. Optional: If you use custom repositories, ensure that you enable these custom repositories after the upgrade completes.

2.2.1. Post-Upgrade Tasks

Some of the procedures in this section are optional. You can choose to perform only those procedures that are relevant to your installation.

If you use the PXE-based discovery process, then you must complete the discovery upgrade procedure on Foreman and on any Smart Proxy server with hosts that you want to be listed in Foreman on the Hosts > Discovered hosts page.

2.3. Upgrading Discovery

This section describes updating the PXELinux template and the boot image passed to hosts that use PXE booting to register themselves with Foreman server.

From Foreman 1.22, provisioning templates now have a separate association with a subnet, and do not default to using the TFTP Smart Proxy for that subnet. If you create subnets after the upgrade, you must specifically enable the Foreman or a Smart Proxy to provide a proxy service for discovery templates and then configure all subnets with discovered hosts to use a specific template Smart Proxy.

During the upgrade, for every subnet with a TFTP proxy enabled, the template Smart Proxy is set to be the same as the TFTP Smart Proxy. After the upgrade, check all subnets to verify this was set correctly.

These procedures are not required if you do not use PXE booting of hosts to enable Foreman to discover new hosts.

2.3.1. Upgrading Discovery on Foreman server

  1. Update the Discovery template in the Foreman web UI:

    1. In the Foreman web UI, navigate to Hosts > Provisioning templates.

    2. On the PXELinux global default line, click Clone.

    3. Enter a new name for the template in the Name field, for example ACME PXE global default.

    4. In the template editor field, change the line ONTIMEOUT local to ONTIMEOUT discovery and click Submit.

    5. In the Foreman web UI, navigate to Administer > Settings.

    6. Locate Global default PXELinux template and click on its Value.

    7. Select the name of the newly created template from the menu and click the tick button.

    8. In the Foreman web UI, navigate to Hosts > Provisioning templates.

    9. Click Build PXE Default, then click OK.

  2. In the Foreman web UI, go to Configure > Discovery Rules and associate selected organizations and locations with discovery rules.

2.3.2. Upgrading Discovery on Smart Proxy servers

  1. Verify that the Foreman Discovery package is current on Foreman server.

    # yum install tfm-rubygem-foreman_discovery
  2. If an update occurred in the previous step, restart the foreman-maintain services.

    # foreman-maintain service restart
  3. Upgrade the Discovery image on the Foreman Smart Proxy that is either connected to the provisioning network with discovered hosts or provides TFTP services for discovered hosts.

    # yum install foreman-discovery-image
  4. On the same instance, install the package which provides the Proxy service, and then restart foreman-proxy service.

    # yum install tfm-rubygem-smart_proxy_discovery
    # service foreman-proxy restart
  5. In the Foreman web UI, go to Infrastructure > Smart Proxies and verify that the relevant Smart Proxy lists Discovery in the features column. Select Refresh from the Actions drop-down menu if necessary.

  6. Go to Infrastructure > Subnets and for each subnet on which you want to use discovery:

    1. Click the subnet name.

    2. On the Smart Proxies tab, ensure the Discovery Smart Proxy is set to a Smart Proxy you configured above.

2.3.3. Verifying Subnets have a Template Smart Proxy

Ensure all subnets with discovered hosts have a template Smart Proxy:
  1. In the Foreman web UI, navigate to Infrastructure > Subnets.

  2. Select the subnet you want to check.

  3. On the Smart Proxies tab, ensure a Template Smart Proxy has been set for this subnet.

For more information about configuring subnets with template Smart Proxies, see Configuring the Discovery Service in the Provisioning guide.

2.4. Upgrading virt-who

If virt-who is installed on Foreman server or a Smart Proxy server, it will be upgraded when they are upgraded. No further action is required. If virt-who is installed elsewhere, it must be upgraded manually.

Before You Begin

If virt-who is installed on a host registered to Foreman server or a Smart Proxy server, first upgrade the host to the latest packages available in the https://yum.theforeman.org/client/nightly/ repository.

Upgrade virt-who Manually
  1. Upgrade virt-who.

    # yum upgrade virt-who
  2. Restart the virt-who service so the new version is activated.

    # systemctl restart virt-who.service

2.5. Reclaiming PostgreSQL Space

The PostgreSQL database can use a large amount of disk space especially in heavily loaded deployments. Use this procedure to reclaim some of this disk space on Foreman.

Procedure
  1. Stop all services, except for the postgresql service:

    # foreman-maintain service stop --exclude postgresql
  2. Switch to the postgres user and reclaim space on the database:

    # su - postgres -c 'vacuumdb --full --dbname=foreman'
  3. Start the other services when the vacuum completes:

    # foreman-maintain service start

3. Updating Foreman server and Content Hosts

Use this chapter to update your existing Foreman server, Smart Proxy server, and Content Hosts to a new minor version, for example, from nightly.0 to nightly.1.

Updates patch security vulnerabilities and minor issues discovered after code is released, and are often fast and non-disruptive to your operating environment.

Before updating, back up your Foreman server and all Smart Proxy servers. For more information, see Backing Up Foreman server and Smart Proxy server in the Administering Foreman guide.

3.1. Updating Foreman server

Prerequisites
  • Ensure that you have synchronized Foreman server repositories for Foreman, Smart Proxy, and https://yum.theforeman.org/client/nightly/.

  • Ensure each external Smart Proxy and Content Host can be updated by promoting the updated repositories to all relevant Content Views.

Warning
If you customize configuration files, manually or use a tool such as Hiera, these customizations are overwritten when the installation script runs during upgrading or updating. You can use the --noop option with the foreman-installer script to test for changes. For more information, see the Red Hat Knowledgebase solution How to use the noop option to check for changes in Foreman config files during an upgrade.

Updating Foreman server to the Next Minor Version

To Update Foreman server:
  1. Ensure the Foreman Maintenance repository is enabled:

    # subscription-manager repos --enable \
    rhel-7-server-satellite-maintenance-6-rpms
  2. Check the available versions to confirm the next minor version is listed:

    # foreman-maintain upgrade list-versions
  3. Use the health check option to determine if the system is ready for upgrade. On first use of this command, foreman-maintain prompts you to enter the hammer admin user credentials and saves them in the /etc/foreman-maintain/foreman-maintain-hammer.yml file.

    # foreman-maintain upgrade check --target-version nightly.z

    Review the results and address any highlighted error conditions before performing the upgrade.

  4. Because of the lengthy update time, use a utility such as screen to suspend and reattach a communication session. You can then check the upgrade progress without staying connected to the command shell continuously. For more information about using the screen command, see How do I use the screen command? article in the Red Hat Knowledge Base.

    If you lose connection to the command shell where the upgrade command is running, you can see the logged messages in the /var/log/foreman-installer/foreman.log file to check if the process completed successfully.

  5. Perform the upgrade:

    # foreman-maintain upgrade run --target-version nightly.z
  6. Check when the kernel packages were last updated:

    # rpm -qa --last | grep kernel
  7. Optional: If a kernel update occurred since the last reboot, stop the foreman-maintain services and reboot the system:

    # foreman-maintain service stop
    # reboot