Files and the Filesystem

Files and the Filesystem

Files are the part of the operating system most visible to user. Files are used for storing data received from outside the system, generated by the user or generated by applications. Stored data is then sent out outside the system or processed by the user or by applications. Most applications directly or indirectly use the file API provided by the operating system.

In this chapter we present the common operations part of the file API focusing on POSIX. We show the internals of files, key to understanding I/O redirection, file pointers, sparse files and UNIX universal I/O.

The main goal is to get you accustomed to the file I/O APIs: ANSI and POSIX and their similarities and differences. You will see I/O specifics such as blocking I/O, buffering, partial data. To get a better grasp on files and their interface you will take a look into file implementation internals.

Contents

storing data, files, byte streams, data and metadata directories as list of file names opening files, file descriptors, file descriptor table, file pointer file descriptors as universal interface: sockets, pipes file API: system I/O vs buffered I/O

Files are stored as inodes (index nodes): metadata with pointers to actual data. Use stat command to list file information. Then use stat()/lstat() library call. 03-files/stat.

Demos

Implement cp using ANSI C API. 03-files/copy

Using lsof to monitor open files and file cursor update. 03-files/c-file-ops

ANSI C API is buffered, POSIX API is unbuffered: show printf() vs write. 03-files/buffered-system-io

Implement ls like functionality. 03-files/dirent

Redirect standard input, output, see file descriptor table. Use touch in; sleep > out 2>& err < in

Show file descriptor table update for sockets and for pipes. 03-files/universal-io

Find out file descriptor table size. 03-files/fdtsize

Tasks

1. Implement recursive ls. Make a copy of the 03-files/dirent/ folder and implement there.

2. Implement cp using POSIX C API. Make a copy of the 03-files/copy/ folder and implement there.

3. Implement no buffering for stdout. Use setvbuf. Make a copy of the 03-files/buffered-system-io/ folder and implement there.

4. Redirect standard output using close() and open(). Redirect output using dup(). Why is dup2() required.

5. Check if a file exists or is accessible using access().

6. Use getdtablesize() to get file descriptor table size. Make a copy of the 03-files/fdtsize/ folder and implement there.

7. Use fstat on file descriptors. Work on 03-files/universal-io. Print the device id and the file type/mode for each file; see fields in the stat structure.