Setup Initial MdBook, User documentation

docs/.gitignore

@@ -0,0 +1 @@
+book

docs/book.toml

@@ -0,0 +1,6 @@
+[book]
+authors = ["Yashodhan Joshi"]
+language = "en"
+multilingual = false
+src = "src"
+title = "Youki User and Developer Documentation"

docs/src/SUMMARY.md

@@ -0,0 +1,18 @@
+# Summary
+
+[Youki](./youki.md)
+
+---
+
+- [User Documentation](./user/introduction.md)
+  - [Basic Setup](./user/basic_setup.md)
+  - [Basic Usage](./user/basic_usage.md)
+  - [Crates provided](./user/crates.md)
+    - [libcgroups](./user/libcgroups.md)
+    - [libcontainer](./user/libcontainer.md)
+    - [liboci-cli](./user/liboci_cli.md)
+    - [libseccomp](./user/libseccomp.md)
+
+---
+
+- [Developer Documentation](./developer/introduction.md)

docs/src/developer/introduction.md

@@ -0,0 +1 @@
+# Developer Documentation

docs/src/user/basic_setup.md

@@ -0,0 +1,99 @@
+# Basic Setup
+
+This section will explain how to setup youki for use. Currently the only way to get youki is to compile it from the source. Currently youki only supports Linux systems, so this documentation assumes you are using a Linux system. For running youki on other platforms, you can use Vagrant, which is explained in the last part of this section.
+
+### Requirements
+
+youki is written in rust, so you will need rust toolchain installed to compile it. Also if you want to use it with a higher level container engine, such as docker, you will need to install that as well. In case you want to use one of the sub-crates of youki as a dependency, you may not need docker.
+
+The rest of document uses docker as an example when required.
+
+- Rust(See [here](https://www.rust-lang.org/tools/install)), edition 2021
+- Docker(See [here](https://docs.docker.com/engine/install))
+
+Apart from these basic requirements, some other libraries are also required to compile and run youki. To install them on :
+
+#### Debian, Ubuntu and related distributions
+
+```console
+$ sudo apt-get install   \
+      pkg-config         \
+      libsystemd-dev     \
+      libdbus-glib-1-dev \
+      build-essential    \
+      libelf-dev \
+      libseccomp-dev
+```
+
+#### Fedora, Centos, RHEL and related distributions
+
+```console
+$ sudo dnf install   \
+      pkg-config     \
+      systemd-devel  \
+      dbus-devel     \
+      elfutils-libelf-devel \
+      libseccomp-devel
+```
+
+#### Getting the source
+
+After installing the dependencies you will need to clone the youki repository if you want to use it directly :
+
+```console
+git clone git@github.com:containers/youki.git
+```
+
+Or if you want ot use it as a dependency in a Rust project, you can specify it in your Cargo.toml :
+
+```toml
+[dependencies]
+...
+liboci-cli = { git = "https://github.com/containers/youki.git" }
+...
+```
+
+You can specify the crate that you need as a dependency in place of `liboci-cli`
+
+#### Installing the source
+
+If you have cloned the source, you can build it using
+
+```console
+# go into the cloned directory
+cd youki
+
+# build
+./build.sh
+```
+
+This will build the youki, and put the binary at the root level of the cloned directory.
+
+When using it as a dependency, you can use it in your source as :
+
+```
+use liboci_cli::{...}
+```
+
+You can specify the crate that you need as a dependency in place of `liboci-cli`
+
+#### Using Vagrant to run Youki on non-Linux Platform
+
+you can use the vagrantfile provided with the source, to setup vagrant and run youki inside it. You can see [vagrant installation](https://www.vagrantup.com/docs/installation) for how to download and setup vagrant.
+
+Once done, you can run vagrant commands in the cloned directory to run youki inside the VM created by vagrant :
+
+```console
+# for rootless mode, which is default
+vagrant up
+vagrant ssh
+
+# or if you want to develop in rootful mode
+VAGRANT_VAGRANTFILE=Vagrantfile.root vagrant up
+VAGRANT_VAGRANTFILE=Vagrantfile.root vagrant ssh
+
+# in virtual machine
+cd youki
+./build.sh
+
+```

docs/src/user/basic_usage.md

@@ -0,0 +1,110 @@
+# Basic Usage
+
+This section will explain how to use youki as a low-level container runtime with some other high-level container runtime such as docker.
+
+Youki can also be used with other runtimes such as podman, but for this section, we will use docker as an example.
+
+#### Using youki with a high-level runtime
+
+This explains how to use youki as a lowe level runtime along with some high-level runtime. For this example we use Docker.
+
+1. If you have the docker daemon running (which you probably have if you have installed docker), first you will need to stop it. For example, if you use systemd as your init system, you can use `systemctl stop docker`, with root permissions.
+2. Start the docker daemon with you as the runtime :
+
+   ```console
+   dockerd --experimental --add-runtime="youki=$(pwd)/youki" # run in the root directory
+   ```
+
+   This will start the docker daemon with youki as the low-level runtime. This will keep the terminal busy, so you can start it as a background process or open a new terminal for next steps.
+
+   In case you get an error message such as :
+
+   ```
+   failed to start daemon: pid file found, ensure docker is not running or delete /var/run/docker.pid
+   ```
+
+   This means you docker daemon is already running, and you need to stop is as explained in step 1.
+
+3. Now you can use docker normally, and give youki as the runtime in the arguments :
+   ```console
+   docker run -it --rm --runtime youki busybox   # run a container
+   ```
+4. After you are done, you can stop the docker daemon process that was started in step 2, and restart the normal docker daemon, by using your init system, for example using `systemctl start docker` with root permissions, for systems using systemd.
+
+#### Using Youki Standalone
+
+You can use youki directly without a higher level runtime, but it will be tedious. This example shows how to create and run a container using only youki.
+
+1. run
+
+   ```console
+   mkdir -p tutorial/rootfs
+   ```
+
+   This will create a directory which will be used as the root directory for the container.
+
+2. run
+   ```console
+   cd tutorial
+   docker export $(docker create busybox) | tar -C rootfs -xvf -
+   ```
+   This will export the basic file system structure needed to run the container. You can manually create all the directories, but it will be tedious.
+3. run
+   ```console
+   ../youki spec
+   ```
+   This will generate the config.json file needed to setup the permissions and configuration for the container process.
+   You can manually edit the file to customize the behavior of the container process. For example, you can edit the process.args to specify command to run in the process :
+   ```json
+   "process": {
+   ...
+   "args": [
+     "sleep", "30"
+   ],
+   ...
+    }
+   ```
+   Then go back to the root directory :
+   ```console
+   cd ..
+   ```
+4. After this you can use youki to :
+   - create the container
+   ```console
+    # create a container with name `tutorial_container`
+    sudo ./youki create -b tutorial tutorial_container
+   ```
+   - get the state of the container
+   ```console
+    # you can see the state the container is `created`
+    sudo ./youki state tutorial_container
+   ```
+   - start the container
+   ```console
+    # start the container
+    sudo ./youki start tutorial_container
+   ```
+   - list all the containers
+   ```console
+   # will show the list of containers, the container is `running`
+   sudo ./youki list
+   ```
+   - delete the specific container
+   ```console
+   # delete the container
+   sudo ./youki delete tutorial_container
+   ```
+5. The above step created the containers with root permissions, but youki can also create rootless containers,which does not need root permissions. for that, after exporting the rootfs from docker, when creating spec, use `--rootless` flag :
+   ```console
+   ../youki spec --rootless
+   ```
+   This will generate the spec needed for rootless containers.
+   After this the steps are same, except you can run them without sudo and root access :
+   ```console
+   cd ..
+   sudo ./youki create -b tutorial rootless_container
+   sudo ./youki state rootless_container
+   sudo ./youki start rootless_container
+   sudo ./youki list
+   sudo ./youki delete rootless_container
+   ```

docs/src/user/crates.md

@@ -0,0 +1,3 @@
+# Crates provided
+
+The github repo of youki is a Cargo workspace containing several crates, which provide various functionalities : for example, libcgroups provide means to work with Linux cgroups. Youki itself is one of the crates, and uses functionalities from the other crates. The following sections explain the crates and public interfaces provided by them, as well as commandline interface of youki itself.

docs/src/user/introduction.md

@@ -0,0 +1,13 @@
+# User Documentation
+
+This section is the user documentation for youki, for using it as a low-level container runtime.
+This section will explain :
+
+- Basic Setup : Explains how to setup youki for use
+- Basic Usage : Using it as a low level runtime
+- crates
+- libcgroups
+- libcontainer
+- liboci-cli
+- libseccomp
+- youki

docs/src/user/libcgroups.md

@@ -0,0 +1,66 @@
+# libcgroups
+
+`libcgroups` provide a Rust interface over cgroups v1 and v2.
+
+It exposes the modules :
+
+- common
+- stats
+- systemd
+- test_manager
+- v1
+- v2
+
+### common
+
+This module contains common features for cgroups v1 and v2, such as
+
+- trait CgroupManager, which gives public interface to
+
+  - add a task to a cgroup
+  - apply resource restriction
+  - remove a cgroup
+  - freezer cgroup state control
+  - get stats from a cgroup
+  - get pids belonging to the cgroup
+
+- functions `write_cgroup_file_str` and `write_cgroup_file` which writes data to a cgroup file
+- function `read_cgroup_file` which reads data from given cgroup file
+- function `get_cgroup_setup` which returns setup of cgroups (v1,v2, hybrid) on the system
+
+### stats
+
+This module contains structs and functions to work with statistics for a cgroup, such as
+
+- struct Stats which contains all following individual stat structs
+  - CpuStats : stores usage and throttling information
+  - MemoryStats : stores usage of memory, swap and memory combined, kernel memory, kernel tcp memory and other memory stats
+  - PidStats : contains current number of active pids and allowed number of pids
+  - BlkioStats : contains block io related stats such as number of bytes transferred from/to by a device in cgroup, number of io operations done by a device in cgroup, device access and queue information
+  - HugeTlbStats : stats for huge TLB , containing usage, max_usage and fail count
+- function `supported_page_size` which returns hugepage size supported by the system
+- functions to operate with data in cgroups files such as
+  - `parse_single_value` : reads file expecting it to have a single value, and returns the value
+  - `parse_flat_keyed_data` : parses cgroup file data which is in flat keyed format (key value)
+  - `parse_nested_keyed_data` : parses cgroup file data which is in nested keyed format (key list of values)
+  - `parse_device_number` : parses major and minor number of device
+
+### systemd
+
+This module contains functions and modules to deal with systemd, as currently youki depends on systemd. This exposes
+
+- function `booted` to check if the system was booted with systemd or not
+- module controller_type, which contains
+  - enum ControllerType which is used to specify controller types
+- module manager, which contains
+  - struct Manager, which is the cgroup manager, and contain information about the root cgroups path, path for the specific cgroups, client to communicate with systemd etc. This also implements CgroupManager trait.
+
+### test_manager
+
+This exposes a TestManager struct which can be used as dummy for testing purposes, which also implements CgroupManager.
+
+### v1 and v2
+
+These modules contains modules and fucntions related to working with cgroups v1 and v2. They expose respective cgroups version specific mangers, and some utility functions to get mount points (for v1 and v2), get subsystem mount points (for v1) and get available controllers (for v2) etc.
+
+For cgroups v2, it also exposes devices module, which gives functions for working with bpf such as load a bpf program, query info of a bpf program, attach and detach a bpf program to a cgroup, etc.

docs/src/user/libcontainer.md

@@ -0,0 +1,35 @@
+# libcontainer
+
+This is a library that provides utilities to setup and run containers. Youki itself uses this crate to manage and control the containers.
+
+This exposes several modules, each dealing with a specific aspect of working with containers.
+
+- apparmor : functions that deal with apparmor, which is a Linux Kernel security module to control program capabilities with per program profiles.
+
+- capabilities : this has functions related to set and reset specific capabilities, as well as to drop extra privileges.
+
+- config : this exposes YoukiConfig struct, which contains a subset of the data in the config.json. This subset is needed when starting or managing containers after creation, and rather than parsing and passing around whole config.json, this smaller YoukiConfig is passed, which is comparatively faster.
+
+- container : This is the core of the container module, and contains modules and structs that deal with the container lifecycle including creating, starting , stopping and deleting containers.
+
+- hooks : exposes function run_hooks, which is used to run various container lifecycle hooks as specified in oci-spec
+
+- namespaces : exposes Namespaces struct, which deals with applying namespaces to a container process
+
+- notify_socket : this has NotifyListener struct, which is used internally to communicate between the main youki process and the forked container process
+
+- process : a module which exposes functions related to forking the process, starting the container process with correct namespaces and setting up the namespaces
+
+- rootfs : this contains modules which deal with rootfs, which is minimal filesystem
+
+- rootless : this deals with running containers rootless, that is without needing root privileges on the host system
+
+- seccomp : this deals with setting up seccomp for container process, this uses libseccomp.
+
+- signal : this provide simple wrappers for unix signal's ascii names/numbers.
+
+- syscall : this provides a trail Syscall, which is used to abstract over several functions which need to call libc functions, so that other parts of library can use them without having to deal with implementation details.
+
+- tty : this deals with setting up the tty for the container process
+
+- utils : provides various utility functions such as parse_env to parse the env variables, do_exec to do an exec syscall and execute a binary, get_cgroups_path, create_dir_all_with_mode etc.

docs/src/user/liboci_cli.md

@@ -0,0 +1,24 @@
+# liboci-cli
+
+This is a crate to parse command line arguments for OCI container runtimes as specified in the OCI Runtime Command Line Interface. This exposes structures with Clap Parser derived, so that they can be directly used for parsing oci-commandline arguments.
+
+### Implemented subcommands
+
+|  Command   | liboci-cli | CLI Specification | runc | crun | youki |
+| :--------: | :--------: | :---------------: | :--: | :--: | :---: |
+|   create   |     ✅     |        ✅         |  ✅  |  ✅  |  ✅   |
+|   start    |     ✅     |        ✅         |  ✅  |  ✅  |  ✅   |
+|   state    |     ✅     |        ✅         |  ✅  |  ✅  |  ✅   |
+|    kill    |     ✅     |        ✅         |  ✅  |  ✅  |  ✅   |
+|   delete   |     ✅     |        ✅         |  ✅  |  ✅  |  ✅   |
+| checkpoint |            |                   |  ✅  |  ✅  |       |
+|   events   |     ✅     |                   |  ✅  |      |  ✅   |
+|    exec    |     ✅     |                   |  ✅  |  ✅  |  ✅   |
+|    list    |     ✅     |                   |  ✅  |  ✅  |  ✅   |
+|   pause    |     ✅     |                   |  ✅  |  ✅  |  ✅   |
+|     ps     |     ✅     |                   |  ✅  |  ✅  |  ✅   |
+|  restore   |            |                   |  ✅  |  ✅  |       |
+|   resume   |     ✅     |                   |  ✅  |  ✅  |  ✅   |
+|    run     |     ✅     |                   |  ✅  |  ✅  |  ✅   |
+|    spec    |     ✅     |                   |  ✅  |  ✅  |  ✅   |
+|   update   |            |                   |  ✅  |  ✅  |       |

docs/src/user/libseccomp.md

@@ -0,0 +1,3 @@
+# libseccomp
+
+This crate provides Rust FFI bindings to [libseccomp](https://github.com/seccomp/libseccomp). This is adapted from code generated using rust-bindgen from a C header file. This also manually fixes some of the issues that occur as rust-bindgen has some issues when dealing with C function macros.