Overview
I had Tenable Security Center running on a RHEL6 VM which scans around 1000 hosts several times a week. REHL6 is getting a bit long-in-the-tooth and also there was a need to upgrade to the latest and greatest version of SC and RHEL, those being at the time SecurityCenter-5.8.0-el7.x86_64 and RHEL 7.6.
In theory, this should work with other Linux distros as well, there will be some subtle changes, but conceptually you should be good to go.
The user of SC wanted to keep all the custom dashboards, plugins and scan data, so this meant that the entire SC configuration and associated database needed to be migrated, there was only a 24hr window to do this as the scans are used almost daily.
Plan (Simplified)
- Stand-up and new RHEL7 VM with a temporary IP and name that is different to the original SC, I named it SC-new
- Stop all SC services on the current SC (named SC-old) and use rsync over SSH to copy everything over
- Once data has been copied, power off SC-old
- Change the identity of SC-new to be the same as SC-old, so, same IP host name etc – this is crucial for the migration to work, the host name is tied to the license keys.
- Install the same version of SC as it had on the old one, once this is done and configured SC should work just like the old one
- Next, upgrade the SC to the new version
- Finally patch the latest version
- Test
Assumptions
I’m assuming you have full access to both hosts and have the firewall etc all configured, have stood-up a new RHEL7 host, and have some Linux command line experience. I have successfully migrated using this process but use at your own risk, if you have never done anything like this before I would seek professional advice before going any further.
I also had the luxury of taking VM snapshots for the entire process, so could revert back if anything what wrong.
Environment
- VMs on VSphere, 12 cores 8GB RAM
- Separate 500GB logical volume for /opt – which is the location SC installs
- Migrating from RHEL6.10 to RHEL7.6
- Security Center from SecurityCenter-5.6.2.1-es7.x86_64 to SecurityCenter-5.8.0-el7.x86_64
Ports
Incoming TCP Port 443 – User Interface
Incoming TCP Port 22 – Remote Repository Sync with other Tenable.sc(s)
Outgoing TCP Port 443 – Plugins update and Nessus Cloud Manager Communication
Outgoing TCP Port 25 – SMTP Email Notification
Outgoing UDP Port 53 – DNS Resolving
Tools used
- Ensure you have rsync installed on both hosts. I used rsync to transfer data. The advantage of using rsync is that if your transfer gets interrupted you can resume it, also you can transfer the bulk of the data, say in the evening, and then just copy over the changes another day. For me, the entire sc directory was 200GB in size and took around 1hr to transfer, a few days later, when I ran another sync it only had to copy 6GB of data over so took minutes, this is a huge time saver.
- On the new host for SC, check you have the following installed; zip, unzip and java-11-openjdk.x86_64, at this point you DO NOT INSTALL any Security Center products.
Prep
- Ensure new host has been stood-up, with a different name and IP, is ready to accept transfer, has zip, unzip and java-11-openjdk.x86_64 installed
-
yum install zip unzip java-11-openjdk.x86_64
- You have configured enough disk for the transfer
- Inform users there maybe some downtime
The process
Current hosts – SC-old
- Stop all Security Center services before rsyncing the data over
service SecurityCenter stop
Check it has stopped, there should no output
ps -fu tns
- On sc-old – copy/rsync the entire contents of /opt/sc over to the new host, it may take several hours so plan accordingly.
rsync -avr --stats --progress -e ssh /opt/sc user1@SC-new:/opt/
- You are now done with the old Security Center, so power if off, if all goes well, you never need this again.
New host – SC-new
- Check that you now have a /opt/sc directory and all looks good, currently, as Security Center is not installed, the directory and contents are owned by the user used to copy the data over, we need to fix this later, the user and group will eventually be tns.
pwd /opt ls -ltr drwxr-x--- 16 user1 users 4096 Jan 23 12:20 sc
- Now we are going to change the identity of the new host to be the same as the old, this is crucial, without doing it will complain about licensing and fail to work.
hostnamectl set-hostname sc-old
- Now edit /etc/host
vi /etc/hosts 10.1.2.3 SC-old SC-old.mydomain
- Change IP by editing ifcfg-xxx file, or by what ever method you are used to.
vi /etc/sysconfig/network-scripts/ifcfg-ens192
- For me, this was all I had to change, but YMMV, so check.
- Reboot for the identity change to kick-in.
- Now install the same version of Security Center that was on the old on, for me it was SecurityCenter-5.6.2.1-es7.x86.
rpm ivh SecurityCenter-5.6.2.1-es7.x86_64.rpm Preparing... ################################# [100%] /usr/bin/chown: invalid user: ‘tns:tns’ Backing up previous application files ... complete. Updating / installing...
- You can see it checked and found that the tns user was not there, don’t worry, it creates this for you during the re-install, a long with the new systemd type service start-up instead of the old initd.
- Now cd into /opt, when installing I noticed that it didn’t recursively set all the permissions up, to fix do
chown -R tns:tns sc
- At this point you are ready to test and see if you old version of SC works on the new OS, before upgrading to the new version of SC.
- I started the SecurityCenter service, it started but there were a few errors, so I rebooted and checked
systemctl status SecurityCenter
….and all was good
[user@SC-old opt]# systemctl status SecurityCenter ● SecurityCenter.service - SYSV: Starts and stops the Lightning daemons Loaded: loaded (/etc/rc.d/init.d/SecurityCenter; bad; vendor preset: disabled) Active: active (running) since Tue 2019-01-22 10:50:42 CST; 4s ago Docs: man:systemd-sysv-generator(8) Process: 7119 ExecStop=/etc/rc.d/init.d/SecurityCenter stop (code=exited, status=0/SUCCESS) Process: 7158 ExecStart=/etc/rc.d/init.d/SecurityCenter start (code=exited, status=0/SUCCESS) CGroup: /system.slice/SecurityCenter.service ├─7169 /opt/sc/support/bin/php -f /opt/sc/daemons/Jobd.php ├─7175 /opt/sc/support/bin/httpd -k start ├─7182 /opt/sc/support/bin/httpd -k start ├─7183 /opt/sc/support/bin/httpd -k start └─7184 /opt/sc/support/bin/httpd -k start Jan 22 10:50:41 SC-old systemd[1]: Starting SYSV: Starts and stops the Lightning daemons... Jan 22 10:50:42 SC-old SecurityCenter[7158]: Starting SecurityCenter services: [ OK ] Jan 22 10:50:42 SC-old systemd[1]: Started SYSV: Starts and stops the Lightning daemons.
- Next I got the SC user to login to the admin web GUI and runs some scans and jobs etc to test that the old version was working before upgrading to the new version of SC, after a day of use they reported it was working great, so now we can upgrade and patch.
- To upgrade, it only took a few minutes
rpm Uvh SecurityCenter-5.8.0-el7.x86_64.rpm
- Restart SC for good measure and get the SC user to test again
systemctl start SecurityCenter
- Now the last thing to do is patch
tar zxf SC-201812.1-5.8.0.tgz cd SC-201812.1-5.8.0 ./install.sh
- That’s it all done!