8.3 KiB
_This is a draft for a high level documentation of Youki. After it is finished this is intended to explain how control flow and high level functioning of Youki happens for development purposes.
Some reference links
These are references to various documentations and specifications, which can be useful to understand commands and constraints.
- OCI runtime specification : The specification for a container runtime. Any OCI complaisant runtime must follow this.
- runc man pages : Has description on commands and their options in runc.
- cgroups man page : Contains information about cgroups, their creation, deletion etc.
- pseudoterminal man page : Information about the pseudoterminal system, useful to understand console_socket parameter in create subcommand
- Unix Sockets man page : Useful to understand sockets
- prctl man page : Process control man pages
- OCI Linux spec : Linux specific section of OCI Spec
- pipe2 man page : Definition and usage of pipe2
- systemd resource control: Options offered by systemd for resource control
Control flow diagram
This is diagram as given in #14, which is not actually how this works, but helpful to understand overall flow. Someone needs to check and correct.
sequenceDiagram
participant U as User
participant D as Docker
participant Y_Main as Youki(Main Process)
participant Y_Intermediate as Youki(Intermeidate Process)
participant Y_init as Youki(Init Process)
U ->> D : $ docker run --rm -it --runtime youki $image
D ->> Y_Main : youki create $container_id
Y_Main ->> Y_Intermediate : fork(2) to create new intermediate process, entering into user and pid namespaces.
Y_Intermediate ->> Y_Main : set user id mapping if entering into usernamespaces
Y_Intermediate ->> Y_Init: fork(2) to create the container init process.
Y_Init ->> Y_Init : configure resource limits, mount the devices, entering into rest of namespaces, and etc.
Y_Init ->> Y_Intermediate : ready message (Unix domain socket)
Y_Intermediate ->> Y_Main : ready message (Unix domain socket)
Y_Main ->> Y_Main: set cgroup configuration for Y_Init
Y_Main ->> D : exit $code
D ->> Y_Main : $ youki start $container_id
Y_Main -->> Y_Init : start message through notify listener (Unix domain socket)
Y_Init ->> Y_Init : run the commands in dockerfile, using `execv`
D ->> D : monitor pid written in pid file
D ->> U : exit $code
Control flow
main invocation
On invoking Youki, main function parses args passed to it, which contains directory path to store container state (check runc . 8 . md in runc man pages), optional log path and log format string and a subcommand such as create, delete etc.
From there it matches subcommand arg with possible subcommand and takes appropriate actions, such as creating a new container, deleting a container, etc.
create container
One thing to note is that in the end, a container is just another process in Linux, which has control groups, namespaces, pivot_root and other mechanisms applied to it. The program executing has the impression that is is running on a complete system, but from the host system's perspective, it is just another process, and has attributes such as pid, file descriptors, etc. associated with it like any other process.
When given the create command, Youki will load the specification, configuration, sockets etc., use clone syscall to create the container process (init process), applies the limits, namespaces, and etc. to the cloned container process. The container process will wait on a unix domain socket before executing the command/program.
The main youki process will setup pipes to communicate and synchronize with the intermediate and init process. The init process will notify the intermediate process, and then intermediate process to the main youki process that it is ready and start to wait on a unix domain socket. The youki process will then write the container state and exit.
Process
This handles creation of the container process. The main youki process creates the intermediate process and the intermediate process creates the container process (init process). The hierarchy is: main youki process -> intermediate process -> init process
The main youki process will set up pipes used as message passing and synchronization mechanism with the init process. The reason youki needs to create/fork two process instead of one is due to the user and pid namespaces. In rootless container, we need to first enter user namespace, since all other namespaces requires CAP_SYSADMIN. When unshare or set_ns into pid namespace, only the children of the current process will enter into a different pid namespace. As a result, we must first fork a process to enter into user namespace, call unshare or set_ns for pid namespace, then fork again to enter into the correct pid namespace.
Note: clone(2) offers us the ability to enter into user and pid namespace by creatng only one process. However, clone(2) can only create new pid namespace, but cannot enter into existing pid namespaces. Therefore, to enter into existing pid namespaces, we would need to fork twice. Currently, there is no getting around this limitation.
Container
This structure contains functions related to container process and its state and status.
Command
This contains a trait to wrap commonly required syscalls, so that they can be abstracted from implementation details for rest of Youki. This also provides implementation for Linux syscalls for the trait.
Capabilities
This has functions related to set and reset specific capabilities, as well as to drop extra privileges
Info
This is primarily for printing info about system running youki, such as OS release, architecture, cpu info, cgroups info etc. , as this info can be helpful when reporting issues.
Namespaces
This has functions related to setting of namespaces to the calling process
Pause and Resume
This contains functionality regarding pausing and resuming container. Pausing a container indicates suspending all processes in it. This can be done with signals SIGSTOP and SIGCONT, but these can be intercepted. Using cgroups to suspend and resume processes without letting tasks know.