Protection and Security

Protection and security become an important topic as we use computer to accomplish a lot of personal tasks such as online banking and personal emails. We definitely want absolute privacy and protection on those data. The goals of protection and security:

Why list sharing explicitly? Because without sharing, we can achieve isolation trivially: unplug your computer from the Internet and only run one process at a time! Computer systems that try to achieve isolation at the expense of sharing are too inflexible for modern use. But computer systems that achieve sharing at the expense of isolation (*cough* Windows [until recently]) are too dangerous for modern use! We need a happy medium. Computer system security researchers are searching for that medium.

Fundamental abstraction: access control

Some definitions:

State entity, example: file, process, memory pages, disk, keyboard, mouse, network, ...
Process "identity" which specifies accessible resources, unit of access granting or restriction. UNIX specify domain by defining effective userID and groupID for each process
Operation on an object that might or night not be allowed

Example:          /tmp/x = object

                        Open, read, write, execute = right

                        Network = object

                        Read, write = right

Example: strawman, 2 domains: root and nobody, 1 right: allowall

Access matrix


Everything else







Here, the rows are domains, the columns are objects, and rights are written in each cell. For instance, root has "Allowall" rights for /bin/echo, and for everything else.

There are several problems with this scheme:

1)      Nobody has too many right on echo, in which nobody can change the /bin/echo file to do whatever he likes -- such as explode the computer, or execute a lot of privileged operations. Then, when root runs echo, all hell will break loose. This can be a security hole.

2)      Nobody has too few rights with respect to everything else!

3)      Coarse-grained domain, if you need any privilege in the system, you need to login as root in order to get all privilege, this lead to not flexible

This scheme has one advantage: it is easy to understand!

But it doesn't follow the...

Principle of least privilege (POLP): every program should run with only the minimal privilege required to do its job.

If our operating system followed POLP, then the /bin/echo attack above wouldn't matter as much. When root ran echo, it would only have privilege to write to the terminal! So it could not do very much mischief.

So let's try another strawman design. The last one had only one access right; that was too coarse-grained. Instead, let's have huge numbers of access rights. In fact, our system will have different rights for writing any sequence of bytes to a file. That is, a process might have permission to write "Hello", but not "Goodbye". This has one advantage, but many disadvantages.

+ least privilege possible

− overhead

− hard to understand

− In practice, it will be too hard to keep track of privilege, so processes will take all privilege! Not clear this scheme will do much better than "All access".

Unix Access Matrices






r w x o

r w x “search” o

r w o

r w x o



r w x

r w o




r w x


r w x o

What gets stored in /tmp?

In Unix, tmp is temporary storage running on the machine.


Bob’s processes

Raw disk access


r w x o


r w o


r w x o




r: read

w: write

x: execute

o: owner (applies to all object)

c: control


if M(d, o) contains “owner,” then d can add or remove rights to the column containing object o.


control lets one process control all the rights another process have.

If control is in the matrix of d1 and d2, then d1 can control d2 access rights.

Trojan horse: program that looks like it does its job and additionally leaves a hole for others to exploit. 

Unix Domain

Users and groups

Root is a special user, called "superuser". Root has all privilege on the system. Root can really do anything -- including, sometimes, changing the running kernel. (Modern Unixes allow you to constrain root in some important ways, but the principle stands.)

Why do we need root? Well, imagine login. Login can turn into any user! => must run as root

Checks password to determine rights.

Password file: 1-way hash of physical password

UNIX’s Domain

Each domain is associated with a particular user within this operation system. There are groups and there are users (root, a special kind of user is the most privileged). A group is a set of user ids.

Each user on a system must have a login account identified by a unique username and UID number.

Each user in a system belongs to at least one group. Users may belong to multiple groups, up to a limit of eight or 16. A list of all valid groups for a system are kept in /etc/group.

The /etc/passswd file contains a list of users that the system recognizes. Each user's login name must be in this file.

Creating a new account involves:

Every file or directory in a UNIX file system has three types of permissions (or protections) that define whether certain actions can be carried out. The permissions are:

read ( r ) A user who has read permission for a file may look at its contents or make a copy of it. For a directory, read permission enables a user to find out what files are in that directory.

write ( w ) A user who has write permission for a file can alter or remove the contents of that file. For a directory, the user can create and delete files in that directory.

execute ( x ) A user who has execute permission for a file can cause the contents of that file to be executed (provided that it is executable). For a directory, execute permission allows a user to change to that directory.

The UNIX operating system maintains information, known as permissions, for every file and directory on the system.

For each file and directory, the read, write, and execute permissions may be set separately for each of the following classes of users:

User ( u ) The user who owns the file or directory.

Group ( g ) Several users purposely lumped together so that they can share access to each other's files.

Others ( o ) The remainder of the authorized users of the system.

The superuser is a privileged user who has unrestricted access to the whole system; all commands and all files regardless of their permissions. By convention the username for the superuser account is root.

The root account is necessary as many system administration files and programs need to be kept separate from the executables available to non-privileged users. Unix allow users to set permissions on the files they own. A system administrator may need to override those permissions.

Access to the root account is restricted by a password and there are several ways to log into the root account. If a system comes up in single user mode whoever is logged in automatically has root privileges. When a system is already up in multi user mode, a user can log in directly as root. This is not recommended, as it is easy to take superuser privileges for granted and perform mundane tasks in this mode. When a user is already logged in, issuing the su command, without options, will cause the system to prompt for the root password. Once it is given the user becomes root.

Commands and programs that a system administrator will need as root are kept in /etc to decrease the chances of a user trying them by accident. For example, /etc contains /etc/passwd, which holds a list of all users who have permission to use the system.

Because the root account has extensive privileges it has an equal potential for destruction. This is amplified by the fact that safeguards built into some commands do not apply to this account. For example, the superuser may change another user's password without knowing the old password. The superuser can also mount and unmount file systems, remove any file or directory, and shut down the entire system. The root account should be used with caution and only when necessary to perform a given task. A misplaced keystroke in this mode can have disastrous results.

Further protection can be obtained by placing restrictions on the root account.


Represents one element of the access matrix (object-right pair). For each process, the operating system stores a list of capabilities it has. Those are transferable between processes.

Buffer Overflow:

This is an exploitation of a system bug—such as poor programming practice—by using trial and error or actually inspecting the code. The attacker—this may be the most common way for a hacker outside of the system (network or even dial-up connection) to attempt to gain unauthorized access—generates a script/program that results in one of the following:

Overflow of an input field/buffer.

Return address is being overwritten in the stack.

Place a desired code to next space in the stack.

One way to prevent this is to disallow the execution of code in a stack section of memory.

Time-of-check-to-time-of use: an attack arising due to synchronization flaws. Such attacks can occur when a system utility program checks that a certain condition, say C, is true at time t1 and performs certain actions subsequently, say at time t2 assuming C is still true. A malicious program can act between t1 and t2 and ensure that condition C is not true at t2. Clearly, the interval [t1, t2] is a “vulnerable” period. The malicious program exploits non-atomicity across various statements of a system utility and acts in a window of opportunity. Hence, the malicious program can cause the system utility to perform an unintended action.

A common TOCTTOU flaw in the Unix environment occurs when a setuid1 system utility is run with the effective userid (euid) root. If this setuid system utility performs an update on the user’s file, an attacker (who has privileges of the user) can use the interval between checking the access permissions (using the real userid) of the file, and updating (as root) the file to make the system utility perform an unintended action (e.g. root ends up overwriting the /etc/passwd file).

Trojan horse:

It is a program that appears to have some useful or benign purpose, but really masks some hidden malicious code.

As you might expect, the term refers to the act of planting a Trojan. If you recall your ancient Greek history, you'll remember that the original Trojan horse allowed an army to sneak right through a highly fortified gate. Amazingly, the attacking army hid inside a giant wooden horse offered as a gift to the unsuspecting victims. It worked like a charm, a gift (Kenny Thompson offered his Trojan horse for practically free—so is UNIX). In a similar fashion, today's Trojan horses try to sneak past computer security fortifications, such as firewalls, by employing like-minded trickery. By looking like normal, happy software, Trojan horse programs are used for the following goals: Duping a user or system administrator into installing the Trojan horse in the first place. In this case, the Trojan horse and the unsuspecting user becomes the entry vehicle for the malicious software on the system.

[Along with Dennis Ritchie, Ken Thompson received the ACM Turing Award in 1983, for "for their development of generic operating systems theory and specifically for the implementation of the UNIX operating system.". In his Turing award lecture, Reflections on trusting Trust, Ken Thompson described a hack that he placed into early UNIX systems: the C compiler would insert a back door whenever it compiled the login program, allowing Ken Thompson to access any UNIX system. The scheme was so evil that if you tried remove the back-door generating code from the source code and recompile the compiler, the compiler would reintroduce the back door generation into the source code!]