Understanding the Role of Firmware

What is firmware?

Firmware controls how operating system starts. On older systems, firmware is called Basic Input/Output System (BIOS). Later on, newer systems are using new firmware called Unified Extensible Firmware Interface (UEFI). They are responsible for maintaining the hardware status and launching an operating system. Both of them have the same goal, but they do it in different ways. BIOS and UEFI are interfaces offered on a PC to access disks. They are present on a system to provide a platform that sits between hardware/firmware and the operating system/kernel

What is BIOS?

Huge limitation of BIOS firmware is that it can read only one sector worth of data from a hard drive into the RAM memory. Unfortunately, on newer systems this is not enough to load entire operating system. However, on older systems this was more than enough since operating systems were smaller in size. The way around this is to use boot loader.

What is boot loader?

Boot loader is a small program that initializes the hardware to find and run full operating system. As we know, operating system is stored on the system because otherwise it cannot start. Boot loader has a job to find it on several locations: inside internal or external hard drive, CD or DVD drive, USB memory stick, ISO file, or from a network using NFS, HTTP, or FTP. In one of those places, boot loader will find OS, otherwise system will not boot up. Since boot loader has minimum filesystem knowledge it can go through it and be smart enough to find the kernel inside. In this phase, when boot loader finds kernel, the kernel will not have drivers installed, yet. For that, it will call initramfs which will load drivers to the kernel, and then the kernel can load init/systemd which runs all other services, users, and applications.

If operating system is stored inside physical drive, you must designate which hard drive and partition has OS inside it. To do that, define this information inside Master Boot Record (MBR). On a BIOS system, the MBR is read from disk and at stage 1 boot loader is activated. The stage 1 bootloader is 446 bytes, and its task is to load the stage 2 boot loader that resides in the first MB of the disk. This stage 2 area of 1 MB is metadata area. Inside, there must be enough information to load the kernel. There is only one MBR for the entire system. Once BIOS finds the MBR, it runs whatever is inside that sector of physical drive, usually a boot loader. The MBR table does not belong to a filesystem. The boot loader’s job is to run actual kernel file from the physical drive.

What is UEFI?

As operating systems became more complex, new firmware was needed. Intel created Extensible Firmware Interface (EFI) in 1998, and in 2005 Unified Extensible Firmware Interface (UEFI) was created. When UEFI has GUID Partition Table (GPT), then the system can boot from large disks. However, MBR is still present and is used for backward compatibility. This GPT partition table contains an EFI System Partition (ESP) which contains a directory with the name /boot/efi. Inside /boot/efi/ directory all boot loader files are stored with .efi filename extension.

With ESP, you can store any size boot loader or multiple boot loaders and because of that, inside /boot/efi directory each boot loader has its own subdirectory (/boot/efi/microsoft, /boot/efi/grub). The boot loader also resides inside each of those subdirectories, ready to take action when needed.

After the firmware finds boot loader, UEFI firmware’s job is done. Lastly, boot manager is small built-in mini boot loader that allows you to configure which boot loader to run.

Examples

With gdisk partitioning tool, let’s see what /dev/sda holds. It says disk uses MBR as protective, and GPT which is present

[aldin@arch ~]$ sudo gdisk /dev/sda
GPT fdisk (gdisk) version 1.0.5

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): 

With “p” we can print partitions. There are three of them labeled 1,2 and 3. Each partition has a start and end sector, and their size. The “Code” represents type of the partition.

Command (? for help): p
Disk /dev/sda: 468862128 sectors, 223.6 GiB
Model: KINGSTON SUV5002
Sector size (logical/physical): 512/4096 bytes
Disk identifier (GUID): F64AEA71-E097-634D-A060-F6AD601357E4
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 2048, last usable sector is 468862094
Partitions will be aligned on 2048-sector boundaries
Total free space is 0 sectors (0 bytes)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048          206847   100.0 MiB   EF00  
   2          206848       209922047   100.0 GiB   8300  
   3       209922048       468862094   123.5 GiB   8300  

Now, let’s see the contents of the /boot directory.

[aldin@arch boot]$ ls -lA
drwxr-xr-x 3 root root      512 Sep 30 13:51 EFI
drwxr-xr-x 6 root root      512 Sep 30 13:52 grub
-rwxr-xr-x 1 root root 32130653 Oct 25 02:50 initramfs-linux-fallback.img
-rwxr-xr-x 1 root root  9994134 Oct 25 02:50 initramfs-linux.img
-rwxr-xr-x 1 root root  3161088 Jun 16 19:50 intel-ucode.img
-rwxr-xr-x 1 root root  8958080 Oct 25 02:50 vmlinuz-linux

[aldin@arch boot]$ ls -lA ./EFI/GRUB/grubx64.efi 
-rwxr-xr-x 1 root root 131072 Sep 30 13:51 ./EFI/GRUB/grubx64.efi

[aldin@arch boot]$ ls -lA ./grub/
drwxr-xr-x 2 root root   512 Sep 30 13:51 fonts
-rwxr-xr-x 1 root root  5149 Sep 30 13:52 grub.cfg
-rwxr-xr-x 1 root root  1024 Sep 30 13:51 grubenv
drwxr-xr-x 2 root root  3072 Sep 30 13:51 locale
drwxr-xr-x 3 root root   512 Sep 30 13:51 themes
drwxr-xr-x 2 root root 19968 Sep 30 13:51 x86_64-efi

Inside /boot directory there are initarmfs files and compressed kernel (vmlinuz-linux). Furthermore, there is /EFI directory which holds grubx64.efi (this is GRUB basically), and /grub directory which holds grub.cfg main configuration file which holds instructions on how grubx64.efi will behave.

Let’s take a look at first 512 bytes of /dev/sda. Normally this would hold MBR table for BIOS systems. However, since we are using UEFI firmware these first 512 bytes should be almost empty:

[aldin@arch ~]$ sudo xxd -l 512 /dev/sda
00000000: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000020: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000030: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000040: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000050: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000060: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000070: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000080: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000090: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000100: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000110: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000120: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000130: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000140: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000150: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000160: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000170: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000180: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000190: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001c0: 0200 eeff ffff 0100 0000 af44 f21b 0000  ...........D....
000001d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001f0: 0000 0000 0000 0000 0000 0000 0000 55aa  ..............U.

It is quite empty except this little part of bytes that hold backward compatible MBR. The last 55aa is magic hex that tells when first sector ends.

Device Interfaces

Your system is using devices such as keyboard, mouse, speakers, network cards and tons of USB external peripherals. In order to use them, kernel must communicate with that hardware devices. Furthermore, kernel must know how to send data to and receive (read) data from those devices. The way it does so is by using protocols. Currently, there are three standards to connect devices with the kernel:

PCI boards: Peripheral Component Interconnect (PCI) is a method for connecting hardware devices with the motherboard. PCI provides interface for external hardware devices to communicate with the kernel. Lots of devices use PCI to connect to a server or desktop: Internal hard drives, network interface cards, bluetooth devices, audio cards…

USB interface: USB standard has three versions so far: v1.0 supports transfer speeds up to 12Mbps, v2.0 supports transfer speeds up to 480Mbps, and v3.0 supports transfer speeds up to 20Gbps. Hard drives, printers, digital cameras, keyboard, mice, network cards use USB to connect with the system.

GPIO interface: General Purpose Input/Output (GPIO) is used to support communicating with external devices such as relays, lights, sensors, motors, etc.

/dev Directory

After the Linux kernel communicates with a device on a certain interface, it must be able to transfer data to and from the device. This process is done with device files. Device files are files that kernel creates in /dev directory to communicate with hardware devices. To retrieve data from the device, any program needs to read device file for that device. To send data to the device, any program needs to write to device file for that device.

There are two types of device files:

  1. Character device files: they transfer data one character at a time. They are often used for serial devices (terminals, USB devices)
  2. Block device files: they transfer data in large blocks of data. They are used for transfer devices such as hard drives, solid state drives and network cards.

Device mapper: It’s job is to create virtual block devices and associate them with real block devices. Virtual block devices intercept data and perform some type of operation on them. For example, it is used to encrypt data before writing to the real block device. The device mapper creates virtual devices inside /dev/mapper/ directory. All files there are links to physical block devices in /dev/ directory.

/proc Directory

/proc directory is not physical directory on the system. On contrary, it is virtual directory that kernel dynamically populates. The kernel changes files and data in /proc directory as it monitors the status of hardware on the system. Files important to manage connected devices are:

  1. Interrupt requests: tells CPU if any device has to send data to it. Each device inside this file needs to have IRQ address to uniquely identify them. Some IRQs are reserved by the system for hardware devices. Other IRQs are assigned by the system as devices are detected.
  2. I/O ports: they are locations in memory where CPU can send data to and receive data from the hardware device. System assigns unique I/O port, usually handled by PnP (Plug and Play). Port conflicts aren’t common if ports are assigned by PnP.
  3. Direct Memory Access: I/O Ports are somewhat slow. To speed this many devices use DMA channels. They send data to memory directly, without waiting for CPU. From there, CPU can read those memory locations and access data when it’s ready. Each device that uses DMA has unique channel number.

/sys Directory

Yet another virtual directory is /sys. It is created by the kernel and it provides additional information about hardware devices. Each device has dedicated file broken down into subdirectories based on the device and its function.

[aldin@arch ~]$ ls -al /sys
total 4
dr-xr-xr-x  13 root root    0 Oct 18 12:27 .
drwxr-xr-x  17 root root 4096 Oct 18 00:10 ..
drwxr-xr-x   2 root root    0 Oct 18 12:27 block
drwxr-xr-x  42 root root    0 Oct 18 12:27 bus
drwxr-xr-x  65 root root    0 Oct 18 12:27 class
drwxr-xr-x   4 root root    0 Oct 18 12:27 dev
drwxr-xr-x  29 root root    0 Oct 18 12:27 devices
drwxr-xr-x   7 root root    0 Oct 18 12:27 firmware
drwxr-xr-x   6 root root    0 Oct 18 12:27 fs
drwxr-xr-x   2 root root    0 Oct 18 12:27 hypervisor
drwxr-xr-x  14 root root    0 Oct 18 12:27 kernel
drwxr-xr-x 179 root root    0 Oct 18 12:27 module
drwxr-xr-x   3 root root    0 Oct 18 12:27 power
[aldin@arch ~]$ 

Working with Devices

  • lsdev: displays information about the hardware devices installed on the system. lsdev retrieves information from /proc/interrupts, /proc/ioports, and /proc/dma virtual files.
  • lsblk: displays information about all block devices installed on the system.
  • lspci: displays information about installed and recognized PCI and PCIe cards.
  • lsusb: displays information about USB devices connected.

Leave a Reply