My scroll-wheel stops. Under my mouse, the typical HN comment:

Docker’s just about packaging! Containers aren’t secure! Unlike like jails *strokes unix beard*

The world’s full of interesting partial truths.

Docker did clue a lot of developers into the Linux kernel’s isolation primitives by providing a compelling developer experience. It didn’t take long for others to catch on, though. In my semi-recent foray with FreeBSD, I noticed that managing jails with iocage doesn’t feel too far off managing containers with docker. I’m not going to dive into those details, though, because this post’s title doesn’t contain the words “user experience”.

This perception of security when it comes to jails and containers is a common one. Understanding how someone could come to this conclusion requires a bit of history. Let’s take a look at the abstract for “Jails: Confining the omnipotent root”, which is the original paper on FreeBSD jails:

The traditional UNIX security model is simple but inexpressive. Adding fine-grained access control improves the expressiveness, but often dramatically increases both the cost of system management and implementation complexity. In environments with a more complex management model, with delegation of some management functions to parties under varying degrees of trust, the base UNIX model and most natural extensions are inappropriate at best. Where multiple mutually un-trusting parties are introduced, ‘inappropriate’ rapidly transitions to ’nightmarish’, especially with regards to data integrity and privacy protection.

The FreeBSD ‘Jail’ facility provides the ability to partition the operating system environment, while maintaining the simplicity of the UNIX ‘root’ model. In Jail, users with privilege find that the scope of their requests is limited to the jail, allowing system administrators to delegate management capabilities for each virtual machine environment. Creating virtual machines in this manner has many potential uses; the most popular thus far has been for providing virtual machine services in Internet Service Provider environments.

The first paragraph argues the infeasibility of adding many fine-grained controls for all the different aspects of isolation. The second argues for the compromise that led to the 200LOC patch that was initially jails in FreeBSD. Jails carved out what at the time seemed like a reasonable subset of functionality (but didn’t feel the need to define all restrictions within the paper) and called it a day.

In the usual fashion of ignoring the experiences of the BSDs, the isolation primitives that came to Linux, mostly in the form of cgroups and namespaces, attempted to add the very same fine-grained controls that the Jails paper argues against. This design decision compounded with being “late to market” leaves people with an impression: containers are less mature and playing a game of whack-a-mole. As time passes, though, this becomes less true and jails begin to show their age.

Immediately obvious to anyone that has ran a jail: the networking stack isn’t isolated. Nowadays, you can use VIMAGE for virtualized networking, but it’s not totally stable. And, as one might expect, there are more subtle boundaries to jail isolation, too. System V IPC is shared across all jails, meaning you can clobber the shared memory of another jail. This is not a hypothetical problem and needs to be paid attention to when running PostgreSQL. However, now’s not the time to build a comprehensive list of jail inadequacies; I’m sure that can be found on a mailing list somewhere.

Getting back to the point, it makes sense from a historical perspective that people would consider jails more secure than containers. The Linux kernel came after jails and gave the appearance of more opportunities for things to go wrong. No software ships finished and there absolutely were container escapes as it matured over the past few years. However, as we’ve seen, there are an equal number of opportunities for things to go wrong with jails, too. And with less traction than Linux, many of them have been and will be there for years.

Sources / Interesting Follow-Ups: