Fossil SCM

Expanded the runc section of the container doc to cover "bundle" terminology and to show a method for rsyncing the bundle across to a remote host. Also explained why this is a bad idea unless you've got a rather constrained use case, lest people avoid using podman/docker in places where they could provide real value.

wyoung 2022-09-04 06:02 trunk
Commit f9f13ce7a9637a3e84ea59fe5b143ce220de553d4258aef6045210ff522e20f5
1 file changed +111 -40
+111 -40
--- www/containers.md
+++ www/containers.md
@@ -456,33 +456,50 @@
456456
rest. `runc` alone is about 18 MiB, and you can do without `containerd`
457457
entirely, if you want.
458458
459459
The method isn’t complicated, but it *is* cryptic enough to want a shell
460460
script:
461
+
462
+----
461463
462464
```shell
463465
#!/bin/sh
464466
c=fossil
465
-r=/containers/$c/rootfs
466
-sudo mkdir -p $r
467
-docker container export $c | sudo tar -C $r -xf -
468
-id=$(docker inspect --format="{{.Id}}" $c)
469
-sudo cat /run/containerd/io.containerd.runtime.v2.task/moby/$id/config.json |
470
- jq '.root.path = "'$r'"' |
471
- jq '.linux.cgroupsPath = ""' > $r/../config.json
467
+b=$HOME/containers/$c
468
+r=$b/rootfs
469
+m=/run/containerd/io.containerd.runtime.v2.task/moby
470
+
471
+if [ -d "$t" ] && mkdir -p $r
472
+then
473
+ docker container export $c | sudo tar -C $r -xf -
474
+ id=$(docker inspect --format="{{.Id}}" $c)
475
+ sudo cat $m/$id/config.json |
476
+ jq '.root.path = "'$r'"' |
477
+ jq '.linux.cgroupsPath = ""' > $b/config.json
478
+fi
472479
```
473480
474
-The first two lines set configurables: the name of the Docker container
475
-you’re exporting for use with `runc` and the path where you want its
476
-file system root to live. They can be anything you like.
477
-
478
-The rest is generic, but you’re welcome to freestyle here. For instance,
479
-you could untar the rootfs through SSH in order to transfer the
480
-container to a remote system. Or, you could get really clever: unpack it
481
-in a local temp directory, then `rsync` it to the remote system to avoid
482
-transferring elements of the rootfs that haven’t changed since the last
483
-update.
481
+----
482
+
483
+The first several lines list configurables:
484
+
485
+* **b**: the path of the exported container, called the “bundle” in OCI
486
+ jargon
487
+* **c**: the name of the Docker container you’re bundling up for use
488
+ with `runc`
489
+* **m**: the [moby] directory, both because it’s long and because it’s
490
+ been known to change from one version of Docker to the next
491
+* **r**: the path of the directory containing the bundle’s root file
492
+ system.
493
+
494
+That last doesn’t have to be called `rootfs/`, and it doesn’t have to
495
+live in the same directory as `config.json`, but it is conventional.
496
+Because some OCI tools use those names as defaults, it’s best to follow
497
+suit.
498
+
499
+The rest is generic, but you’re welcome to freestyle here. We’ll show an
500
+example of this below.
484501
485502
We’re using [jq] for two separate purposes:
486503
487504
1. To change the container configuration for `runc`:
488505
@@ -496,11 +513,11 @@
496513
a value like a mount point, the kernel capability set, and so forth.
497514
498515
With the container exported like this, you can start it as:
499516
500517
```
501
- $ cd /path/to/rootfs
518
+ $ cd /path/to/bundle
502519
$ c=any-name-you-like
503520
$ sudo runc create $c
504521
$ sudo runc start $c
505522
$ sudo runc exec $c -t sh -l
506523
~ $ ls museum
@@ -511,19 +528,15 @@
511528
~ $ exit
512529
$ sudo runc kill fossil-runc
513530
$ sudo runc delete fossil-runc
514531
```
515532
516
-The first command refers to wherever the rootfs ended up on the `runc`
517
-host. If it’s the same as the export host, then this is simply `$r`
518
-from the shell script above. If instead you’re doing something like the
519
-SSH/rsync trickery suggested above, the remote rootfs directory might be
520
-named differently on the `runc` host.
521
-
522
-There’s nothing that says the container name on the build host has to be
523
-the same as that on the runc host, so we’ve defined a separate `c`
524
-variable here to keep the commands short.
533
+If you’re doing this on the export host, the first command is “`cd $b`”
534
+if we’re using the variables from the shell script above. We do this
535
+because `runc` assumes you’re running it from the bundle directory. If
536
+you prefer, the `runc` commands that care about this take a
537
+`--bundle/-b` flag to let you avoid switching directories.
525538
526539
The rest should be straightforward: create and start the container as
527540
root so the `chroot(2)` call inside the container will succeed, then get
528541
into it with a login shell and poke around to prove to ourselves that
529542
everything is working properly. It is. Yay!
@@ -531,25 +544,78 @@
531544
The remaining commands show shutting the container down and destroying
532545
it, simply to show how these commands change relative to using the
533546
Docker Engine commands. It’s “kill,” not “stop,” and it’s “delete,” not
534547
“rm.”
535548
536
-[ctrd]: https://containerd.io/
537
-[ecg]: https://github.com/opencontainers/runc/pull/3131
538
-[jq]: https://stedolan.github.io/jq/
539
-[runc]: https://github.com/opencontainers/runc
549
+Beware that if you’re doing this on a remote host, your bundle export
550
+directory on the build host might not be the same as where it ended up
551
+on the remote host. If so, the shell script above will create a broken
552
+bundle because it’s assuming the `mkdir` command should go to the same
553
+directory as the “`rootfs`” value it set in the `config.json` value.
554
+This is a more realistic shell script for that case:
555
+
556
+----
557
+
558
+```shell
559
+#!/bin/sh
560
+c=fossil
561
+b=/var/lib/machines/$c
562
+m=/run/containerd/io.containerd.runtime.v2.task/moby
563
+t=$(mktemp -d /tmp/$c-bundle.XXXXXX)
564
+r=$t/rootfs
565
+
566
+if [ -d "$t" ] && mkdir -p $r
567
+then
568
+ docker container export $c | sudo tar -C $r -xf -
569
+ id=$(docker inspect --format="{{.Id}}" $c)
570
+ sudo cat $m/$id/config.json |
571
+ jq '.root.path = "'$r'"' |
572
+ jq '.linux.cgroupsPath = ""' > $t/config.json
573
+ rsync -av $t/* remotehost:$b
574
+ sudo rm -rf $t
575
+fi
576
+```
577
+
578
+----
579
+
580
+We’ve introduced the “`t`” variable, a temporary directory we populate
581
+locally, then `rsync` across to the remote machine, updating a
582
+*different* bundle directory, `$b`. We’re using the convention for
583
+systemd based machines here, which will play into the [`nspawn`][sdnsp]
584
+alternative below. Even if you aren’t using `nspawn`, it’s a reasonable
585
+place to put containers under the [Linux FHS rules][LFHS].
586
+
587
+[ctrd]: https://containerd.io/
588
+[ecg]: https://github.com/opencontainers/runc/pull/3131
589
+[LFHS]: https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard
590
+[jq]: https://stedolan.github.io/jq/
591
+[moby]: https://github.com/moby/moby
592
+[sdnsp]: #nspawn
593
+[runc]: https://github.com/opencontainers/runc
540594
541595
542596
### <a id="podman"></a>Podman
543597
544
-Although your humble author claims the `runc` method above is not
545
-complicated, you might be recollecting the carefree commands at the top
546
-of this document, pondering whether you can live without the
547
-abstractions a proper container runtime system provides.
598
+Although your humble author claims the `runc` methods above are not
599
+complicated, merely cryptic, you might be fondly recollecting the
600
+carefree commands at the top of this document, pondering whether you can
601
+live without the abstractions a proper container runtime system
602
+provides.
603
+
604
+More than that, there’s a hidden cost to the `runc` method: there is no
605
+layer sharing among containers. If you have multiple Fossil containers
606
+on a single host — perhaps because each serves an independent section of
607
+the overall web site — and you export them to a remote host using the
608
+shell script above, you’ll end up with redundant copies of the `rootfs`
609
+in each. A proper OCI container runtime knows they’re all derived from
610
+the same base image, differing only in minor configuration details,
611
+giving us one of the major advantages of containerization: if none of
612
+the running containers can change these immutable base layers, it
613
+doesn’t have to copy them.
548614
549615
A lighter-weight alternative to Docker Engine that doesn’t give up so
550
-much of its administrator affordances is [Podman], initially created by
616
+many of its administrator affordances is [Podman], initially created by
551617
Red Hat and thus popular on that family of OSes, although it will run on
552618
any flavor of Linux. It can even be made to run [on macOS via Homebrew][pmmac]
553619
or [on Windows via WSL2][pmwin].
554620
555621
On Ubuntu 22.04, it’s about a quarter the size of Docker Engine. That
@@ -620,15 +686,20 @@
620686
#### <a id="crun"></a>`crun`
621687
622688
In the same way that [Docker Engine is based on `runc`](#runc), Podman’s
623689
engine is based on [`crun`][crun], a lighter-weight alternative to
624690
`runc`. It’s only 1.4 MiB on the system I tested it on, yet it will run
625
-the same container bundles as in my `runc` examples above. This makes it
626
-a great option for tiny remote hosts. Above, we saved more than that by
627
-compressing the container’s Fossil executable with UPX!
691
+Above, we saved more than that by compressing the container’s Fossil
692
+executable with UPX! the same container bundles as in my `runc`
693
+examples above.
694
+
695
+This makes `crun` a great option for tiny remote hosts with a single
696
+container, or at least where none of the containers share base layers,
697
+so that there is no effective cost to duplicating the immutable base
698
+layers of the containers’ source images.
628699
629
-This suggests a method around the problem of rootless Podman containers:
700
+This suggests one method around the problem of rootless Podman containers:
630701
`sudo crun`, following the examples above.
631702
632703
[crun]: https://github.com/containers/crun
633704
634705
<div style="height:50em" id="this-space-intentionally-left-blank"></div>
635706
--- www/containers.md
+++ www/containers.md
@@ -456,33 +456,50 @@
456 rest. `runc` alone is about 18 MiB, and you can do without `containerd`
457 entirely, if you want.
458
459 The method isn’t complicated, but it *is* cryptic enough to want a shell
460 script:
 
 
461
462 ```shell
463 #!/bin/sh
464 c=fossil
465 r=/containers/$c/rootfs
466 sudo mkdir -p $r
467 docker container export $c | sudo tar -C $r -xf -
468 id=$(docker inspect --format="{{.Id}}" $c)
469 sudo cat /run/containerd/io.containerd.runtime.v2.task/moby/$id/config.json |
470 jq '.root.path = "'$r'"' |
471 jq '.linux.cgroupsPath = ""' > $r/../config.json
 
 
 
 
 
472 ```
473
474 The first two lines set configurables: the name of the Docker container
475 you’re exporting for use with `runc` and the path where you want its
476 file system root to live. They can be anything you like.
477
478 The rest is generic, but you’re welcome to freestyle here. For instance,
479 you could untar the rootfs through SSH in order to transfer the
480 container to a remote system. Or, you could get really clever: unpack it
481 in a local temp directory, then `rsync` it to the remote system to avoid
482 transferring elements of the rootfs that haven’t changed since the last
483 update.
 
 
 
 
 
 
 
 
 
 
484
485 We’re using [jq] for two separate purposes:
486
487 1. To change the container configuration for `runc`:
488
@@ -496,11 +513,11 @@
496 a value like a mount point, the kernel capability set, and so forth.
497
498 With the container exported like this, you can start it as:
499
500 ```
501 $ cd /path/to/rootfs
502 $ c=any-name-you-like
503 $ sudo runc create $c
504 $ sudo runc start $c
505 $ sudo runc exec $c -t sh -l
506 ~ $ ls museum
@@ -511,19 +528,15 @@
511 ~ $ exit
512 $ sudo runc kill fossil-runc
513 $ sudo runc delete fossil-runc
514 ```
515
516 The first command refers to wherever the rootfs ended up on the `runc`
517 host. If it’s the same as the export host, then this is simply `$r`
518 from the shell script above. If instead you’re doing something like the
519 SSH/rsync trickery suggested above, the remote rootfs directory might be
520 named differently on the `runc` host.
521
522 There’s nothing that says the container name on the build host has to be
523 the same as that on the runc host, so we’ve defined a separate `c`
524 variable here to keep the commands short.
525
526 The rest should be straightforward: create and start the container as
527 root so the `chroot(2)` call inside the container will succeed, then get
528 into it with a login shell and poke around to prove to ourselves that
529 everything is working properly. It is. Yay!
@@ -531,25 +544,78 @@
531 The remaining commands show shutting the container down and destroying
532 it, simply to show how these commands change relative to using the
533 Docker Engine commands. It’s “kill,” not “stop,” and it’s “delete,” not
534 “rm.”
535
536 [ctrd]: https://containerd.io/
537 [ecg]: https://github.com/opencontainers/runc/pull/3131
538 [jq]: https://stedolan.github.io/jq/
539 [runc]: https://github.com/opencontainers/runc
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
540
541
542 ### <a id="podman"></a>Podman
543
544 Although your humble author claims the `runc` method above is not
545 complicated, you might be recollecting the carefree commands at the top
546 of this document, pondering whether you can live without the
547 abstractions a proper container runtime system provides.
 
 
 
 
 
 
 
 
 
 
 
 
548
549 A lighter-weight alternative to Docker Engine that doesn’t give up so
550 much of its administrator affordances is [Podman], initially created by
551 Red Hat and thus popular on that family of OSes, although it will run on
552 any flavor of Linux. It can even be made to run [on macOS via Homebrew][pmmac]
553 or [on Windows via WSL2][pmwin].
554
555 On Ubuntu 22.04, it’s about a quarter the size of Docker Engine. That
@@ -620,15 +686,20 @@
620 #### <a id="crun"></a>`crun`
621
622 In the same way that [Docker Engine is based on `runc`](#runc), Podman’s
623 engine is based on [`crun`][crun], a lighter-weight alternative to
624 `runc`. It’s only 1.4 MiB on the system I tested it on, yet it will run
625 the same container bundles as in my `runc` examples above. This makes it
626 a great option for tiny remote hosts. Above, we saved more than that by
627 compressing the container’s Fossil executable with UPX!
 
 
 
 
 
628
629 This suggests a method around the problem of rootless Podman containers:
630 `sudo crun`, following the examples above.
631
632 [crun]: https://github.com/containers/crun
633
634 <div style="height:50em" id="this-space-intentionally-left-blank"></div>
635
--- www/containers.md
+++ www/containers.md
@@ -456,33 +456,50 @@
456 rest. `runc` alone is about 18 MiB, and you can do without `containerd`
457 entirely, if you want.
458
459 The method isn’t complicated, but it *is* cryptic enough to want a shell
460 script:
461
462 ----
463
464 ```shell
465 #!/bin/sh
466 c=fossil
467 b=$HOME/containers/$c
468 r=$b/rootfs
469 m=/run/containerd/io.containerd.runtime.v2.task/moby
470
471 if [ -d "$t" ] && mkdir -p $r
472 then
473 docker container export $c | sudo tar -C $r -xf -
474 id=$(docker inspect --format="{{.Id}}" $c)
475 sudo cat $m/$id/config.json |
476 jq '.root.path = "'$r'"' |
477 jq '.linux.cgroupsPath = ""' > $b/config.json
478 fi
479 ```
480
481 ----
482
483 The first several lines list configurables:
484
485 * **b**: the path of the exported container, called the “bundle” in OCI
486 jargon
487 * **c**: the name of the Docker container you’re bundling up for use
488 with `runc`
489 * **m**: the [moby] directory, both because it’s long and because it’s
490 been known to change from one version of Docker to the next
491 * **r**: the path of the directory containing the bundle’s root file
492 system.
493
494 That last doesn’t have to be called `rootfs/`, and it doesn’t have to
495 live in the same directory as `config.json`, but it is conventional.
496 Because some OCI tools use those names as defaults, it’s best to follow
497 suit.
498
499 The rest is generic, but you’re welcome to freestyle here. We’ll show an
500 example of this below.
501
502 We’re using [jq] for two separate purposes:
503
504 1. To change the container configuration for `runc`:
505
@@ -496,11 +513,11 @@
513 a value like a mount point, the kernel capability set, and so forth.
514
515 With the container exported like this, you can start it as:
516
517 ```
518 $ cd /path/to/bundle
519 $ c=any-name-you-like
520 $ sudo runc create $c
521 $ sudo runc start $c
522 $ sudo runc exec $c -t sh -l
523 ~ $ ls museum
@@ -511,19 +528,15 @@
528 ~ $ exit
529 $ sudo runc kill fossil-runc
530 $ sudo runc delete fossil-runc
531 ```
532
533 If you’re doing this on the export host, the first command is “`cd $b`”
534 if we’re using the variables from the shell script above. We do this
535 because `runc` assumes you’re running it from the bundle directory. If
536 you prefer, the `runc` commands that care about this take a
537 `--bundle/-b` flag to let you avoid switching directories.
 
 
 
 
538
539 The rest should be straightforward: create and start the container as
540 root so the `chroot(2)` call inside the container will succeed, then get
541 into it with a login shell and poke around to prove to ourselves that
542 everything is working properly. It is. Yay!
@@ -531,25 +544,78 @@
544 The remaining commands show shutting the container down and destroying
545 it, simply to show how these commands change relative to using the
546 Docker Engine commands. It’s “kill,” not “stop,” and it’s “delete,” not
547 “rm.”
548
549 Beware that if you’re doing this on a remote host, your bundle export
550 directory on the build host might not be the same as where it ended up
551 on the remote host. If so, the shell script above will create a broken
552 bundle because it’s assuming the `mkdir` command should go to the same
553 directory as the “`rootfs`” value it set in the `config.json` value.
554 This is a more realistic shell script for that case:
555
556 ----
557
558 ```shell
559 #!/bin/sh
560 c=fossil
561 b=/var/lib/machines/$c
562 m=/run/containerd/io.containerd.runtime.v2.task/moby
563 t=$(mktemp -d /tmp/$c-bundle.XXXXXX)
564 r=$t/rootfs
565
566 if [ -d "$t" ] && mkdir -p $r
567 then
568 docker container export $c | sudo tar -C $r -xf -
569 id=$(docker inspect --format="{{.Id}}" $c)
570 sudo cat $m/$id/config.json |
571 jq '.root.path = "'$r'"' |
572 jq '.linux.cgroupsPath = ""' > $t/config.json
573 rsync -av $t/* remotehost:$b
574 sudo rm -rf $t
575 fi
576 ```
577
578 ----
579
580 We’ve introduced the “`t`” variable, a temporary directory we populate
581 locally, then `rsync` across to the remote machine, updating a
582 *different* bundle directory, `$b`. We’re using the convention for
583 systemd based machines here, which will play into the [`nspawn`][sdnsp]
584 alternative below. Even if you aren’t using `nspawn`, it’s a reasonable
585 place to put containers under the [Linux FHS rules][LFHS].
586
587 [ctrd]: https://containerd.io/
588 [ecg]: https://github.com/opencontainers/runc/pull/3131
589 [LFHS]: https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard
590 [jq]: https://stedolan.github.io/jq/
591 [moby]: https://github.com/moby/moby
592 [sdnsp]: #nspawn
593 [runc]: https://github.com/opencontainers/runc
594
595
596 ### <a id="podman"></a>Podman
597
598 Although your humble author claims the `runc` methods above are not
599 complicated, merely cryptic, you might be fondly recollecting the
600 carefree commands at the top of this document, pondering whether you can
601 live without the abstractions a proper container runtime system
602 provides.
603
604 More than that, there’s a hidden cost to the `runc` method: there is no
605 layer sharing among containers. If you have multiple Fossil containers
606 on a single host — perhaps because each serves an independent section of
607 the overall web site — and you export them to a remote host using the
608 shell script above, you’ll end up with redundant copies of the `rootfs`
609 in each. A proper OCI container runtime knows they’re all derived from
610 the same base image, differing only in minor configuration details,
611 giving us one of the major advantages of containerization: if none of
612 the running containers can change these immutable base layers, it
613 doesn’t have to copy them.
614
615 A lighter-weight alternative to Docker Engine that doesn’t give up so
616 many of its administrator affordances is [Podman], initially created by
617 Red Hat and thus popular on that family of OSes, although it will run on
618 any flavor of Linux. It can even be made to run [on macOS via Homebrew][pmmac]
619 or [on Windows via WSL2][pmwin].
620
621 On Ubuntu 22.04, it’s about a quarter the size of Docker Engine. That
@@ -620,15 +686,20 @@
686 #### <a id="crun"></a>`crun`
687
688 In the same way that [Docker Engine is based on `runc`](#runc), Podman’s
689 engine is based on [`crun`][crun], a lighter-weight alternative to
690 `runc`. It’s only 1.4 MiB on the system I tested it on, yet it will run
691 Above, we saved more than that by compressing the container’s Fossil
692 executable with UPX! the same container bundles as in my `runc`
693 examples above.
694
695 This makes `crun` a great option for tiny remote hosts with a single
696 container, or at least where none of the containers share base layers,
697 so that there is no effective cost to duplicating the immutable base
698 layers of the containers’ source images.
699
700 This suggests one method around the problem of rootless Podman containers:
701 `sudo crun`, following the examples above.
702
703 [crun]: https://github.com/containers/crun
704
705 <div style="height:50em" id="this-space-intentionally-left-blank"></div>
706

Keyboard Shortcuts

Open search /
Next entry (timeline) j
Previous entry (timeline) k
Open focused entry Enter
Show this help ?
Toggle theme Top nav button