Sakshi Education
D. Balakrishna Research Associate, IIITH
File System
File System is a container used to store the data and programs in the system and this data can be accessed by the kernel and applications. File systems are used to organize data in files and directories on the storage devices in a hierarchy manner. File systems are mounted to a specific location in the hierarchy in order to access the data. A Linux directory that is used for mounting the file system is known as mounting point and the content of this directory reflects the content of the storage device. When the file systems are unmounted then the mount point will be empty. This allows applications to access files and directories easily regardless of their storage location. The small file systems can also created in system memory but do not support persistent data because they are recreated in memory after system restart. File systems that must support persistent user and configuration data are created on persistence storage devices such as formatted NAND/NOR flash or any disk based storage devices regardless of whether that storage is local to the device or over some network.
The Linux system requires one or more file systems at the time of boot process that contain the executables for the basic services. The most Linux systems first load an in-memory file system which contains relatively small number of system level executables that start basic services like load device drivers that are required later in the boot process. This file system during boot process is common but not mandatory. Most of the embedded devices do not need the initialization capabilities that are provided by these file systems and they can directly mount the root file system. In-memory file systems are typically used during boot process are knows as initial RAM disk or initial RAM file system. Most of the Linux systems use initial RAM file system because of the optimizations that are provided by the initial RAM file system. This file system can be built into the kernel or can be loaded via kernel boot option when loading a kernel and beginning its execution.
Root File System
A File system is mounted at the root of the hierarchy is known as root file system and identified by /. The root file system contains user applications which Linux system can execute. The mount and unmount commands are programs and they are files inside a file system. The root file system is the first mounted file system so it cannot be mounted using mount command; it is directly mounted by the kernel according kernel boot options. When there is no root file system is available then kernel gives kernel panic error. Root file system can be mounted from different locations.
Figure1: Overall booting process
Booting process with initramfs:
Figure2: Booting process with initramfs
Root File system Organization:
The organization of a Linux root file system in terms of directories is well defined by the file system hierarchy standard. The lists of directories are following
/bin – basic programs
/boot – kernel image
/dev – device files
/etc – system wide configuration
/home – directory for the user home directories
/lib – library files
/media – mount point for removable media
/mnt – mount point for static media
/proc – mount point for the proc virtual file system
/root – home directory for the root user
/sbin – system programs
/tmp – temporary files
/usr - Contain all other programs and libraries
/use/bin
/usr/lib
/usr/sbin
/var – variable data files
The basic programs are installed in /bin directories for example ls, ifconfig, bash, cp, mv etc. The /lib directory contain all c libraries. All other programs and libraries are installed in /usr/bin, /usr/lib directories.
Device files:
In Linux everything is a file so the devices also treated as special files that represent the physical or virtual devices attached to the Linux system. These device files are used by the operating system to enable the access to the physical devices in order to access the standard input/output operations. Every device must have its own specific device driver so that the Linux kernel module can interact with device. There are generally three types of devices in Linux.
Types of File systems
Linux supports many different types of traditional file systems. The most common file systems for Linux are ext2, ext3 and ext4 and for windows vfat file system. The ext2 file system is fast, efficient and robust file system but data corruption or data loss from unexpected power loss or shutdown. The above problem is overcome in journaling file systems with additional functionalities. The journaling file systems are designed such that stay in a correct state even after system crashes due to sudden power off. A journaling file system contains a special file often hidden from the user that is used to store file system metadata and file data itself. This special file referred as the journal, whenever the file system subject to change, the changes are first written to the journal and then to storage medium. The Journaling file system is shown figure3.The ext3 file system is an extension of ext2 file system with added journaling capability and it is more powerful, high performance and robust journaling file system most of the Linux distributions are using. The ext4 file system builds on the success of the ext3 file system. The ext4 file system removed 16 terabyte limits for file system and increasing the size to 1 exbibyte and supports individual file sizes up to 1024 gigabytes. There are other types of Linux file systems such as JFS, ReiserFS and XFS are designed and optimized for the use on traditional storage media and each of them have their own characteristics. Except ext2 all file systems are a journaling file system which supports fast restarts.
Figure3: Journaling file system
Pseudo File systems
There are two File systems fall under pseudo file systems in the kernel and they are /proc file system and sysfs file system.
Proc filesystem – It is special file system in Linux operating system which represents information about processes and other system information in a hierarchical structure. The proc virtual File system exists since the beginning of Linux. It allows the kernel to expose statistics about running processes in the system. The proc file system is used by many user space applications and they expect it to be mounted in /proc.
Sysfs filesystem – It is a virtual file system provided by kernel. This file system exports the information about various kernel subsystems, hardware devices and associated device drivers from the kernel to user space. This feature is integrated in the 2.6 Linux kernels and above. It is useful for various user space applications that need to list the available hardware. All applications using sysfs can be mounted in the /sys directory.
Minimal filesystem – To start a Linux system working we need some basic applications are running.
File systems for Embedded Systems
The file systems used in embedded Linux are more concentrate on minimizing file system size and providing media specific capabilities. For example flash devices can only be written to a finite number of times and therefore usually provide internal mechanism such as wear leveling which try to evenly distribute writes to the underlying media to maximize its usable lifespan. The different types flash file systems that are commonly used in embedded linux are following.
Jffs2:
Jffs2 (Journaling file system version2) is a log based file system is designed for use on NAND/NOR flash devices with special attention to flash oriented issues such as wear leveling. To use jffs2 file system need mtd-utils package from the MTD project or their embedded variants from busybox. The limitation with jffs2 image is several chunks if bigger than the RAM size.
Yaffs2:
Yaffs2 (Yet another flash file system version 2) provides fast and robust file system for large NAND/NOR flash devices. It requires less RAM to hold file system state information than the jffs2 file system.
Cramfs:
The compressed RAM file system is a simple read only file system is designed to minimize the size by maximizing the efficient use of underlying storage. The cramfs file system is very useful for embedded system that contains small ROM or flash memory that holds static data and programs. Files on cramfs are compressed in units that match Linux page size to provide efficient random access to file contents.
Ubifs:
The unsorted block image file system is advanced version of jffs2 file system. This file system provide better performance compare to jffs2 file system on larger NAND flash devices and also supports write caching to provide additional performance improvements.
Squashfs:
Squashfs file system is a compressed read only file system designed for low memory or limited size environments.
MTD (Memory Technology Device)
A MTD is a device file in Linux distribution used for interacting with flash memories. The MTD subsystem was created to provide an abstraction layer between the low level device and higher level applications. The objective of the system is to make it simple to provide a driver for new hardware, by providing a generic interface between the hardware drivers and the upper layers of the system. Hardware drivers need to know nothing about the storage formats used, but will only need to provide simple operations such as read write and erase. The MTD is not a block device. MTD deals with devices in units of erase blocks that are not always a uniform size, whereas block devices operate on fixed size of read and write blocks. The architecture of MTD is shown in the below figure.
Figure4: Linux MTD architecture
MTD Characteristics:
File System is a container used to store the data and programs in the system and this data can be accessed by the kernel and applications. File systems are used to organize data in files and directories on the storage devices in a hierarchy manner. File systems are mounted to a specific location in the hierarchy in order to access the data. A Linux directory that is used for mounting the file system is known as mounting point and the content of this directory reflects the content of the storage device. When the file systems are unmounted then the mount point will be empty. This allows applications to access files and directories easily regardless of their storage location. The small file systems can also created in system memory but do not support persistent data because they are recreated in memory after system restart. File systems that must support persistent user and configuration data are created on persistence storage devices such as formatted NAND/NOR flash or any disk based storage devices regardless of whether that storage is local to the device or over some network.
The Linux system requires one or more file systems at the time of boot process that contain the executables for the basic services. The most Linux systems first load an in-memory file system which contains relatively small number of system level executables that start basic services like load device drivers that are required later in the boot process. This file system during boot process is common but not mandatory. Most of the embedded devices do not need the initialization capabilities that are provided by these file systems and they can directly mount the root file system. In-memory file systems are typically used during boot process are knows as initial RAM disk or initial RAM file system. Most of the Linux systems use initial RAM file system because of the optimizations that are provided by the initial RAM file system. This file system can be built into the kernel or can be loaded via kernel boot option when loading a kernel and beginning its execution.
- Initial RAM disk (initrd) -- A compressed fixed size file system that is loaded into memory, requires a specific driver for this type of file system and requires some amount of memory and processing time to copy programs from the file system into memory so that they can be executed.
- Initial RAM filesystem (initramfs) -- Directly expanding a compressed file into the page cache used by the kernel and does not require any driver. It is very fast and does not have a fixed size because the size of the page cache can be varied as per the requirement.
Root File System
A File system is mounted at the root of the hierarchy is known as root file system and identified by /. The root file system contains user applications which Linux system can execute. The mount and unmount commands are programs and they are files inside a file system. The root file system is the first mounted file system so it cannot be mounted using mount command; it is directly mounted by the kernel according kernel boot options. When there is no root file system is available then kernel gives kernel panic error. Root file system can be mounted from different locations.
- From the partition of NAND/NOR flash devices.
- From the partition of SD card.
- From the partition of USB devices.
- From the network using NFS protocol.
- From memory using pre-loaded file systems.
Figure1: Overall booting process
Booting process with initramfs:
Figure2: Booting process with initramfs
Root File system Organization:
The organization of a Linux root file system in terms of directories is well defined by the file system hierarchy standard. The lists of directories are following
/bin – basic programs
/boot – kernel image
/dev – device files
/etc – system wide configuration
/home – directory for the user home directories
/lib – library files
/media – mount point for removable media
/mnt – mount point for static media
/proc – mount point for the proc virtual file system
/root – home directory for the root user
/sbin – system programs
/tmp – temporary files
/usr - Contain all other programs and libraries
/use/bin
/usr/lib
/usr/sbin
/var – variable data files
The basic programs are installed in /bin directories for example ls, ifconfig, bash, cp, mv etc. The /lib directory contain all c libraries. All other programs and libraries are installed in /usr/bin, /usr/lib directories.
Device files:
In Linux everything is a file so the devices also treated as special files that represent the physical or virtual devices attached to the Linux system. These device files are used by the operating system to enable the access to the physical devices in order to access the standard input/output operations. Every device must have its own specific device driver so that the Linux kernel module can interact with device. There are generally three types of devices in Linux.
- Character Devices
- Block devices
- Network Devices
Types of File systems
Linux supports many different types of traditional file systems. The most common file systems for Linux are ext2, ext3 and ext4 and for windows vfat file system. The ext2 file system is fast, efficient and robust file system but data corruption or data loss from unexpected power loss or shutdown. The above problem is overcome in journaling file systems with additional functionalities. The journaling file systems are designed such that stay in a correct state even after system crashes due to sudden power off. A journaling file system contains a special file often hidden from the user that is used to store file system metadata and file data itself. This special file referred as the journal, whenever the file system subject to change, the changes are first written to the journal and then to storage medium. The Journaling file system is shown figure3.The ext3 file system is an extension of ext2 file system with added journaling capability and it is more powerful, high performance and robust journaling file system most of the Linux distributions are using. The ext4 file system builds on the success of the ext3 file system. The ext4 file system removed 16 terabyte limits for file system and increasing the size to 1 exbibyte and supports individual file sizes up to 1024 gigabytes. There are other types of Linux file systems such as JFS, ReiserFS and XFS are designed and optimized for the use on traditional storage media and each of them have their own characteristics. Except ext2 all file systems are a journaling file system which supports fast restarts.
Figure3: Journaling file system
Pseudo File systems
There are two File systems fall under pseudo file systems in the kernel and they are /proc file system and sysfs file system.
Proc filesystem – It is special file system in Linux operating system which represents information about processes and other system information in a hierarchical structure. The proc virtual File system exists since the beginning of Linux. It allows the kernel to expose statistics about running processes in the system. The proc file system is used by many user space applications and they expect it to be mounted in /proc.
Sysfs filesystem – It is a virtual file system provided by kernel. This file system exports the information about various kernel subsystems, hardware devices and associated device drivers from the kernel to user space. This feature is integrated in the 2.6 Linux kernels and above. It is useful for various user space applications that need to list the available hardware. All applications using sysfs can be mounted in the /sys directory.
Minimal filesystem – To start a Linux system working we need some basic applications are running.
- An init application which is the first user space application started by the kernel after mounting the root file system.
- The init application is responsible for starting all other user space applications.
- In the case of root file system the kernel runs the services /sbin/init, /bin/init, /etc/init, /bin/sh.
- In the case of an initramfs the kernel runs the services /init and another path can be given as rdinit kernel argument.
- If none of them are found then the kernel panic error and boot process stops.
- A shell to allow user to interact with system.
- The basic unix commands like to list files, move files and copy files.
File systems for Embedded Systems
The file systems used in embedded Linux are more concentrate on minimizing file system size and providing media specific capabilities. For example flash devices can only be written to a finite number of times and therefore usually provide internal mechanism such as wear leveling which try to evenly distribute writes to the underlying media to maximize its usable lifespan. The different types flash file systems that are commonly used in embedded linux are following.
Jffs2:
Jffs2 (Journaling file system version2) is a log based file system is designed for use on NAND/NOR flash devices with special attention to flash oriented issues such as wear leveling. To use jffs2 file system need mtd-utils package from the MTD project or their embedded variants from busybox. The limitation with jffs2 image is several chunks if bigger than the RAM size.
Yaffs2:
Yaffs2 (Yet another flash file system version 2) provides fast and robust file system for large NAND/NOR flash devices. It requires less RAM to hold file system state information than the jffs2 file system.
Cramfs:
The compressed RAM file system is a simple read only file system is designed to minimize the size by maximizing the efficient use of underlying storage. The cramfs file system is very useful for embedded system that contains small ROM or flash memory that holds static data and programs. Files on cramfs are compressed in units that match Linux page size to provide efficient random access to file contents.
Ubifs:
The unsorted block image file system is advanced version of jffs2 file system. This file system provide better performance compare to jffs2 file system on larger NAND flash devices and also supports write caching to provide additional performance improvements.
Squashfs:
Squashfs file system is a compressed read only file system designed for low memory or limited size environments.
MTD (Memory Technology Device)
A MTD is a device file in Linux distribution used for interacting with flash memories. The MTD subsystem was created to provide an abstraction layer between the low level device and higher level applications. The objective of the system is to make it simple to provide a driver for new hardware, by providing a generic interface between the hardware drivers and the upper layers of the system. Hardware drivers need to know nothing about the storage formats used, but will only need to provide simple operations such as read write and erase. The MTD is not a block device. MTD deals with devices in units of erase blocks that are not always a uniform size, whereas block devices operate on fixed size of read and write blocks. The architecture of MTD is shown in the below figure.
Figure4: Linux MTD architecture
MTD Characteristics:
- To use MTD services the kernel has to be configured with MTD enabled.
- MTD devices are visible in /proc/mtd.
- MTD devices are usually portioned to use different areas of the flash for different purposes like read only filesystem, read/write file system and etc.
- The mtdchar driver creates a character device for each MTD device of the system and named as /dev/mtedX. Provides ioctl() to erase and manage the flash.
- The mtdblock driver creates a block device for each MTD device of the system and named as /dev/mtdblockX. Allows read/write block level access.
- Each partition becomes a separate MTD device.
- The device tree is the standard place to device MTD partition for boards with device tree support.
- The boards which do not have device tree support, the MTD partitions can be defined by the kernel.
- “mtdinfo” command gives detailed information about MTD device.
- “flash_eraseall” command to completely erase the information.
- “flashcp” command to write into NOR flash.
- “nandwrite” command to write into NAND flash.
- File system image creation tools: mkfs.jffs2
- The above commands are available through mtd-utils package in GNU Linux distribution.
Published date : 28 Jul 2015 12:28PM
