Chapter 4 – Services and Daemons

Introduction:

  • systemd manages services and daemons on your machine
  • Before that job was done by init, now all distros use systemd
  • Systemd is the first process, and it has process ID of 1
  • All processes have systemd as a parent
  • Centos adopted systemd with version 7
  • Ubuntu adopted systemd with version 15.04
  • Debian adopted systemd with Jessie 8

1. Determining running services:

  • Good idea to check what software is running on each machine
  • To find out, modern systems will use systemctl to do that.
  • systemctl stands for system control
  • It is a software that manages all other software
  • systemctl is the main control mechanism for any systemd system.
  • systemctl is used to query systemd

Show systemd units:

# systemctl
  • Shows every unit systemd is aware of
  • Very noisy, not clean

Show only services and their status:

# systemctl list-units --type service

Show only running services and their status:

# systemctl list-units --type service --state running

Show only running services and their status: (cleaner output)

# systemctl list-units --type service --state running --no-legend

Show only services that were loaded, done the job, and exited:

# systemctl list-units --type service --state exited

Show only services that have failed:

# systemctl list-units --type service --state failed 

Option –no-pager shows stdout in clear ASCII. An option used for scripting


2. List installed services:

# systemctl list-unit-files --type service
  • Options are: static, enabled, disabled.
  • This is a list of all services systemd knows about on your machine!
  • Enabled services are running
  • Disabled services are not running
  • Static services exist, they are not enabled, and cannot be enabled because they do not have [Install] section of its unit file.

Service files, and all types of unit file, exist in /usr/lib/systemd/system

From this location, files are symlinked into runlevel directories at /etc/systemd/system/

targets are basically different runlevels of a system
The most used services are inside multi-user.target.wants directory
In there we have services required for a multi-user session
Each line represents a service, and their symlink to location on disk.

List static services:

# systemctl --no-pager list-unit-files --type service --state static

Look at service config files:

# systemctl cat sshd-keygen.service
  • Insert any service name 
  • The following image shows output of cat command
  • Since sshd-keygen.service is PartOf=sshd.service sshd.socket, it does not need to be enabled, and therefore is STATIC service, waiting for parent service to run.
# systemctl cat sshd.service
  • Output is showing that sshd.service wants sshd-keygen.service mentioned above.
  • Therefore, sshd-keygen.service is static, it depends on sshd.service and does not need to be enabled on its own (it is static, cannot be enabled by itself)

3. Starting and stopping services:

Stop a service:

# systemctl stop servicename

Start a service:

# systemctl start servicename

Check service status:

# systemctl status servicename

What systemd does when start/stop/reload command is used:

  • When we instruct systemd to stop/start/reload a service, it actually runs the ExecStart, ExecReload or ExecStop portions of its unit file:

Check unit file for a service:

# systemctl cat httpd

See service status:

# systemctl status httpd

To run service at each boot:

# systemctl enable servicename

Disable a service:

# systemctl disable servicename
  • By enabling/disabling service, we create symlink from default location of service’s unit file to the multi-user.target.wants directory.
  • On boot, systemd will check multi-user.target.wants directory and start the services it finds there

Start and Enable service at the same time:

# systemctl enable --now servicename
  • Command equivalent to systemctl start servicename and systemctl enable servicename.

4. Common services you might see:

# less systemctl list-unit-files --type service 
  • Search with “/” to search the list.

auditd.service

  • “The Linux Audit Daemon”
  • Used to write audit records of your system
  • Stores logs at: /var/log/audit/ directory

chronyd.service

  • Responsible for keeping your system clock correct

crond.service

  • Responsible for scheduled commands and their run-time

lvm2-*.service

  • Collection of lvm2 service files
  • Used to maintain and monitor Logical volume Manager (LVM) setups.
  • Most servers will use LVM for their filesystem and volume management needs.

NetworkManager.service

  • Service responsible for networking on your system.

nfs.service

  • Suite of services for management of Network File System (NFS) interfaces and devices
  • Well maintained by distribution producers

prostifx.service

  • Default service on Red Hat, CentOS and Fedora. 
  • On others it might be Exim mail server

rsyslog.service

  • rsyslog or syslog-ng will be probably installed on your system.
  • These are system log daemons
  • Responsible for writing logs to text files (/var/log

sshd.service

  • Service for SSH

systemd-journald.service

  • Competitor to syslog
  • journald is the logging daemon managed by systemd

systemd-logind.service

  • logind manages user logins to the systems of which you are an administrator

These services and others you are running make up the OS

Default services are disabled as part of server hardening tasks.

See how long system boots:

# systemd-analyze

See which process takes most of the booot time:

# systemd-analyze blame

5. Understanding service unit files:

Throughout this example, we will use sshd service.

Several options are available and more are being added with every systemd release

Read systemd.service and systemd.unit man pages

To understand this file, break it into parts [Unit] [Service] and [Install]

[Unit]

  • Area where generic entries live
  • It is the Main section of the unit file.

Description=OpenSSH server daemon

  • Short brief or description of a service

Documentation=man:sshd man:sshd_config

  • Shows where to look for documentation man pages

After= and Before=network.target sshd-keygen.service

  • Shows which services need to start after or before this service.
  • In this example, network service and sshd-keygen needs to start before sshd.service

Wants=sshd-keygen.service

  • Services which would be nice to trigger, prior to this service
  • Wants= is less-mandated version of Requires=. 
  • If Wants= service fails to start, the parent service will give it a try to start
  • If Requires= service fails to start, the parent service will not start either

[Service]

  • This block is specific to service options

Type=notify

  • Explains process start up type – here it says notify 
  • Notify means systemd expects the daemon to send a notification message once its finished starting up. Only when this notification is received, systemd continues starting services that depend on this one.

EnvironmentFile=/etc/sysconfig/sshd

  • This setting is used to point the service to the file that contains its environment variables. 
  • For sshd, this contains one option/argument shown in ExecStart setting.

ExecStart=/usr/bin/sshd -D $OPTIONS

  • It gives us the command thats run
  • $OPTION value read from the EnvironmentFile line!

ExecReload=/bin/kill -HUP $MAINPID

  • Tells systemd what should be run if the systemctl reload sshd command is run
  • -HUP (hangup) = we are sending HUP signal to the sshd PID

KillMode=process

  • Setting specifies how processes of this unit should be killed.
  • Here, process means only the main process itself is killed by systemd.

Restart=on-failure

  • How systemd will restart the process – on failure.
  • Here, sshd will be restarted on an unclean exit code, unclean signal, a timeout…

RestartSec=42s

  • Time to take before restarting the service following the Restart= criteria is met.

[Install]

WantedBy=multi-user.target

  • This service is wanted by multi-user.target directory.
  • In multi-user mode, sshd will be started.

6. Customizing systemd unit files:

  • There’s no guarantee that your changes will be persistent across updates of your system. 
  • Package managers might take offence to your mangling of their service files, and simply replace them without warning.
  • Correct way to modify the unit file is to write a whole other file with your custom changes
  • systemd’s method for doing this is called a snippet
# systemctl edit sshd.service
  • Empty file will be created
  • /etc/systemd/system/sshd.service.d/override.conf
  • Inside empty file, add your entries.
  • Save and exit
# systemctl show sshd
  • Test if the config is saved and loaded correctly
# systemctl daemon-reload
# systemctl restart sshd

7. Testing running services:

  • Three ways to see if a service is running: systemctl, ps, telnet

8. Working with systemd timers (and cron)

  • Timers act as the instruction for when another unit is to trigger
  • In old distros, such events would be controlled with cron

Let’s take a look what this timer does:

Let’s take a look what this service does:

Type=oneshot

  • Means it is expected to exit after running

9. Other init systems:

We have two systems at play:

  • CentOS 6 uses Upstart init
  • Debian 7 uses SysV init

CentOS 6 and Upstart init:

  • systemctl wont work
  • Instead of systemctl, we use service –status-all
# service --status-all
  • List services and their status
# /etc/init.d/sshd status
  • Show service status

Start/stop service:

# service servicename stop
# service servicename start

Disable/enable service:

# chkconfig --list servicename
# chkconfig servicename off
# chkconfig servicename on

Debian 7 and SysV init:

# /etc/init.d/ssh status
# update-rc.d servicename disable
# update-rc.d servicename enable
# chkconfig ssh on
# chkconfig ssh off

Almost 90% of the content was used from the Linux System Administration Cookbook by Packt which I highly recommend!

This Post Has One Comment

Leave a Reply