This week I learned: 2023-05-21

Note: I have posted basically nothing¹ since setting this site up nearly two years ago. It’s probably a bit ambitious to set an explicit goal of posting every week, but I’m going to try to create a summary or recap of things I’ve learned on a semi-regular basis.

Song of the week: Esperanza by Hermanos Gutiérrez

This week’s topics:

• Writing a Container in Rust
• Redux Fundamentals

Writing a Container in Rust

Containerization (a.k.a. OS-level virtualization) is part and parcel of modern backend software development. A not-wholly-wrong, if pithy, explanation is that this is the result of a happy confluence of “it works on my machine, so we’ll ship the whole machine” and “servers as cattle, not pets” type thinking. (Yes, mechanisms for process isolation and resource sharing are about as old as modern computing, but we’re taking a high-level view here.)

This has brought with it some nice conveniences (stable build artifacts, even for interpreted languages; “simple” horizontal scalability; environment-variable driven behavior/separation of configuration from code) and sometimes not-so-nice challenges (container orchestration; bloated container images²; security implications). For many developers, Docker is synonymous with containerized applications, though technologies like LXC³ and OpenVZ on Linux or FreeBSD jails predate it by years. Regardless, the industry standard is now “OCI images on a containerd runtime” (managed by Kubernetes or the like).

Musings on containerized development aside, containers on Linux are really “just” a clever combination of a whole slew of kernel-provided isolation mechanisms, security primitives, and resource restriction functions. Lizzie Dixon published the original “Linux containers in 500 lines of code” in 2016. Late last year, I found an update of that article by Litchi Pi, implementing the same basic container architecture using Rust, which (like the original) ended up being a really great tutorial on the essential ingredients of containers on Linux, while providing exposure to some of the Rust crates one can use to interface with kernel mechanisms and system calls. This year I’ve found time here and there to follow along with the article, and finally finished it this week.

The end result handles the following:

• spawning the child process in its own namespace for mounts, PIDs, IPC, hostname, etc.
• creating Unix sockets to drive the container
• mounting several directories and performing a pivot_root
• setting a random hostname like small-blue-violin
• handling user namespace UID/GID mapping
• limiting resource usage via cgroups and rlimit/ulimit
• restricting capabilities and limiting syscalls
• and more

As mentioned at the end of Litchi Pi’s series, there are a few interesting directions to take it in from here: actual cleanup of the temporary directories, adding proper network isolation, implementing a container format spec, etc. When I have time I’d like to attempt some of these.

In the end, it’s probably worth reading both tutorials (and the second naturally references the first): together they’re packed with references to various manpages, historical LWN articles, discussions of security considerations, and overall a ton of information. In one word, the amount of material you could branch out to by reading these articles is: sprawling⁴. The story of containers on Linux is the culmination of the last 20+ years of isolation and security developments in the kernel. Oh, and I had a good chuckle at

Note: this capability is overloaded

on the description of CAP_SYS_ADMIN. My journey with Rust and learning systems programming continues (we’ll see how long it takes to work through Rust in Action), but this was fun deep dive into a technology I use daily.

Redux Fundamentals

I expended a lot of words on the above, so I’ll keep this shorter. Early this year I made it a goal to pick up more modern frontend web development and remain relevant ascend to (in modern parlance) full-stack greatness, having cut my teeth on desktop GUI development years ago with Qt and PyQt. We use a variety of state management techniques in our (React) frontend codebases at work, but this week I went through a Frontend Masters course on using Redux in React applications.

Steve Kinney is a thoroughly entertaining instructor, and the course demystifies what Redux is doing under-the-hood. Indeed we actually use Redux Toolkit (RTK)⁵, and without knowing the basics of pure Redux (and related core libraries like Immer) I think one would be hard-pressed to claim a deep understanding of it.