DOMAIN AND TYPE ENFORCEMENT

7.6. SUMMARY 101

is FreeBSD Jails [ 156 ], which implement a lightweight virtualization similar to Solaris Containers see Chapter 8 where processes running in a jail are isolated from processes outside that jail. The security focus for OpenBSD [ 235 ] is on correct coding and configuration of systems to minimize its attack surface. OpenBSD does not enforce mandatory access control, but instead focuses on the correctness of its trusted programs and limiting the amount of code in trusted programs. Rigorous code reviews are required for all trusted programs to reduce the possibility of vulnerabilities. Privilege separation is often employed to re-engineer trusted programs to remove code that does not require root privileges to execute, such as for OpenSSH [ 251 ]. This approach reduces the trusted computing base of the system, by ensuring that less code has the privileges necessary to compromise the system. In addition to code review and privilege separation, other system hardening techniques, such as buffer overflow protection and least privilege configurations, are employed to prevent system compromise. As a mandatory protection system requires that a tamperproof trusted computing base, such attention to the trusted computing base programs is necessary for a secure system in general. Other systems can, and have, leveraged the re-engineered programs developed for OpenBSD. NetBSD [ 224 ] contains many of the security features of modern UNIX systems to prevent buffer overflows, but it additionally provides in-kernel authentication and verification of file execu- tion. In UNIX, user authentication is traditionally performed by trusted programs running outside the kernel. These programs are vulnerable to compromised root programs e.g., network facing dae- mons, so the system’s security may depend on programs that cannot be protected from tampering. As NetBSD’s Kauth framework is deployed inside the kernel, it is not susceptible to compromised user-space processes, so trust in authentication is improved. NetBSD also defines a Veriexec mech- anism which can be used to verify the integrity of a file prior to its use. Veriexec ensures that only files whose contents correspond to an authorized hash may be accessed. It defines different modes of permissible access for a file: 1 DIRECT for executables; 2 INDIRECT for interpreters run indirectly e.g., via binsh; and 3 FILE for data files that may not be executed. The NetBSD kernel checks the integrity i.e., hash of the file before its is accessed in the specified manner to detect unauthorized modification.

7.6 SUMMARY

Adding security features to an existing operating system, with its existing customer base and appli- cations, has been a popular approach for building secure systems. Unfortunately, retrofitting security into existing, insecure systems leads to a variety of issues. Many programs are designed and con- figured such that they will not work in the more restrictive environment of a secure system. The operating systems themselves have complex interfaces that may be difficult to mediate. In this chapter, we surveyed a variety of systems where security is retrofitted. We describe the security features that are added to these systems, the challenges in ensuring that the reference monitor concept is achieved, and the decisions that were taken to address these challenges. In general, these efforts show that it is practical to add a reference monitor interface to an existing system, but that it difficult to ensure the reference monitor guarantees are actually achieved. The complexity 102 CHAPTER 7. SECURING COMMERCIAL OPERATING SYSTEMS and dynamics of these commercial systems prevent security professionals from developing models necessary to verifying mediation, tamperproofing, or correctness. We examine these challenges in detail for Solaris Trusted Extensions in Chapter 8 and for Linux in Chapter 9.