Introduction:
- systemd manages services and daemons on your machine
- Before that job was done by init, now all distros use systemd
- Systemd is the first process, and it has process ID of 1
- All processes have systemd as a parent
- Centos adopted systemd with version 7
- Ubuntu adopted systemd with version 15.04
- Debian adopted systemd with Jessie 8
1. Determining running services:
- Good idea to check what software is running on each machine
- To find out, modern systems will use systemctl to do that.
- systemctl stands for system control
- It is a software that manages all other software
- systemctl is the main control mechanism for any systemd system.
- systemctl is used to query systemd
Show systemd units:
# systemctl
- Shows every unit systemd is aware of
- Very noisy, not clean
Show only services and their status:
# systemctl list-units --type service
Show only running services and their status:
# systemctl list-units --type service --state running
Show only running services and their status: (cleaner output)
# systemctl list-units --type service --state running --no-legend
Show only services that were loaded, done the job, and exited:
# systemctl list-units --type service --state exited
Show only services that have failed:
# systemctl list-units --type service --state failed
Option –no-pager shows stdout in clear ASCII. An option used for scripting
2. List installed services:
# systemctl list-unit-files --type service
- Options are: static, enabled, disabled.
- This is a list of all services systemd knows about on your machine!
- Enabled services are running
- Disabled services are not running
- Static services exist, they are not enabled, and cannot be enabled because they do not have [Install] section of its unit file.
Service files, and all types of unit file, exist in /usr/lib/systemd/system
From this location, files are symlinked into runlevel directories at /etc/systemd/system/
The most used services are inside multi-user.target.wants directory
In there we have services required for a multi-user session
List static services:
# systemctl --no-pager list-unit-files --type service --state static
Look at service config files:
# systemctl cat sshd-keygen.service
- Insert any service name
- The following image shows output of cat command
- Since sshd-keygen.service is PartOf=sshd.service sshd.socket, it does not need to be enabled, and therefore is STATIC service, waiting for parent service to run.
# systemctl cat sshd.service
- Output is showing that sshd.service wants sshd-keygen.service mentioned above.
- Therefore, sshd-keygen.service is static, it depends on sshd.service and does not need to be enabled on its own (it is static, cannot be enabled by itself)
3. Starting and stopping services:
Stop a service:
# systemctl stop servicename
Start a service:
# systemctl start servicename
Check service status:
# systemctl status servicename
What systemd does when start/stop/reload command is used:
- When we instruct systemd to stop/start/reload a service, it actually runs the ExecStart, ExecReload or ExecStop portions of its unit file:
Check unit file for a service:
# systemctl cat httpd
See service status:
# systemctl status httpd
To run service at each boot:
# systemctl enable servicename
Disable a service:
# systemctl disable servicename
- By enabling/disabling service, we create symlink from default location of service’s unit file to the multi-user.target.wants directory.
- On boot, systemd will check multi-user.target.wants directory and start the services it finds there
Start and Enable service at the same time:
# systemctl enable --now servicename
- Command equivalent to systemctl start servicename and systemctl enable servicename.
4. Common services you might see:
# less systemctl list-unit-files --type service
- Search with “/” to search the list.
auditd.service
- “The Linux Audit Daemon”
- Used to write audit records of your system
- Stores logs at: /var/log/audit/ directory
chronyd.service
- Responsible for keeping your system clock correct
crond.service
- Responsible for scheduled commands and their run-time
lvm2-*.service
- Collection of lvm2 service files
- Used to maintain and monitor Logical volume Manager (LVM) setups.
- Most servers will use LVM for their filesystem and volume management needs.
NetworkManager.service
- Service responsible for networking on your system.
nfs.service
- Suite of services for management of Network File System (NFS) interfaces and devices
- Well maintained by distribution producers
prostifx.service
- Default service on Red Hat, CentOS and Fedora.
- On others it might be Exim mail server
rsyslog.service
- rsyslog or syslog-ng will be probably installed on your system.
- These are system log daemons
- Responsible for writing logs to text files (/var/log
sshd.service
- Service for SSH
systemd-journald.service
- Competitor to syslog
- journald is the logging daemon managed by systemd
systemd-logind.service
- logind manages user logins to the systems of which you are an administrator
These services and others you are running make up the OS
Default services are disabled as part of server hardening tasks.
See how long system boots:
# systemd-analyze
See which process takes most of the booot time:
# systemd-analyze blame
5. Understanding service unit files:
Throughout this example, we will use sshd service.
Several options are available and more are being added with every systemd release
Read systemd.service and systemd.unit man pages
To understand this file, break it into parts [Unit] [Service] and [Install]
[Unit]
- Area where generic entries live
- It is the Main section of the unit file.
Description=OpenSSH server daemon
- Short brief or description of a service
Documentation=man:sshd man:sshd_config
- Shows where to look for documentation man pages
After= and Before=network.target sshd-keygen.service
- Shows which services need to start after or before this service.
- In this example, network service and sshd-keygen needs to start before sshd.service
Wants=sshd-keygen.service
- Services which would be nice to trigger, prior to this service
- Wants= is less-mandated version of Requires=.
- If Wants= service fails to start, the parent service will give it a try to start
- If Requires= service fails to start, the parent service will not start either
[Service]
- This block is specific to service options
Type=notify
- Explains process start up type – here it says notify
- Notify means systemd expects the daemon to send a notification message once its finished starting up. Only when this notification is received, systemd continues starting services that depend on this one.
EnvironmentFile=/etc/sysconfig/sshd
- This setting is used to point the service to the file that contains its environment variables.
- For sshd, this contains one option/argument shown in ExecStart setting.
ExecStart=/usr/bin/sshd -D $OPTIONS
- It gives us the command thats run
- $OPTION value read from the EnvironmentFile line!
ExecReload=/bin/kill -HUP $MAINPID
- Tells systemd what should be run if the systemctl reload sshd command is run
- -HUP (hangup) = we are sending HUP signal to the sshd PID
KillMode=process
- Setting specifies how processes of this unit should be killed.
- Here, process means only the main process itself is killed by systemd.
Restart=on-failure
- How systemd will restart the process – on failure.
- Here, sshd will be restarted on an unclean exit code, unclean signal, a timeout…
RestartSec=42s
- Time to take before restarting the service following the Restart= criteria is met.
[Install]
WantedBy=multi-user.target
- This service is wanted by multi-user.target directory.
- In multi-user mode, sshd will be started.
6. Customizing systemd unit files:
- There’s no guarantee that your changes will be persistent across updates of your system.
- Package managers might take offence to your mangling of their service files, and simply replace them without warning.
- Correct way to modify the unit file is to write a whole other file with your custom changes
- systemd’s method for doing this is called a snippet
# systemctl edit sshd.service
- Empty file will be created
- /etc/systemd/system/sshd.service.d/override.conf
- Inside empty file, add your entries.
- Save and exit
# systemctl show sshd
- Test if the config is saved and loaded correctly
# systemctl daemon-reload
# systemctl restart sshd
7. Testing running services:
- Three ways to see if a service is running: systemctl, ps, telnet
8. Working with systemd timers (and cron)
- Timers act as the instruction for when another unit is to trigger
- In old distros, such events would be controlled with cron
Let’s take a look what this timer does:
Let’s take a look what this service does:
Type=oneshot
- Means it is expected to exit after running
9. Other init systems:
We have two systems at play:
- CentOS 6 uses Upstart init
- Debian 7 uses SysV init
CentOS 6 and Upstart init:
- systemctl wont work
- Instead of systemctl, we use service –status-all
# service --status-all
- List services and their status
# /etc/init.d/sshd status
- Show service status
Start/stop service:
# service servicename stop
# service servicename start
Disable/enable service:
# chkconfig --list servicename
# chkconfig servicename off
# chkconfig servicename on
Debian 7 and SysV init:
# /etc/init.d/ssh status
# update-rc.d servicename disable
# update-rc.d servicename enable
# chkconfig ssh on
# chkconfig ssh off
One thought on “Chapter 4 – Services and Daemons”