Fossil SCM
Extracted the Docker containers material from www/build.wiki and moved it into a new document dedicated to the topic, containers.md. It was already pushing the bounds of how much info we want to provide in a single section of that doc, and it's about to get bigger. As part of the conversion from wiki format to Markdown, did another edit pass on the doc, improving a few things along the way. Dropped the "docker-" prefix from all internal IDs, as we no longer need them to disambiguate references to other parts of the build doc.
Commit
7129dc986832851da748453d180110c440421c354d33390219224cd17f4a4a5a
Parent
76c9bbb3bd2d708…
2 files changed
+9
-363
+12
+9
-363
| --- www/build.wiki | ||
| +++ www/build.wiki | ||
| @@ -250,374 +250,19 @@ | ||
| 250 | 250 | TCC += -DSQLITE_MAX_MMAP_SIZE=0 |
| 251 | 251 | </pre></blockquote> |
| 252 | 252 | </ul> |
| 253 | 253 | |
| 254 | 254 | |
| 255 | -<h2 id="docker">5.0 Building a Docker Container</h2> | |
| 256 | - | |
| 257 | -Fossil ships a <tt>Dockerfile</tt> at the top of its source tree which | |
| 258 | -you can build like so: | |
| 259 | - | |
| 260 | -<pre><code> $ docker build -t fossil .</code></pre> | |
| 261 | - | |
| 262 | -If the image built successfully, you can create a container from it and | |
| 263 | -test that it runs: | |
| 264 | - | |
| 265 | -<pre><code> $ docker run --name fossil -p 9999:8080/tcp fossil</code></pre> | |
| 266 | - | |
| 267 | -This shows us remapping the internal TCP listening port as 9999 on the | |
| 268 | -host. As a rule, there's little point to using the "<tt>fossil server | |
| ---port</tt>" feature inside a Docker container. Let it default to 8080 | ||
| 269 | -internally, then remap it to wherever you want it on the host instead. | |
| 270 | - | |
| 271 | -Our stock <tt>Dockerfile</tt> configures Fossil with the default feature | |
| 272 | -set, so you may wish to modify the <tt>Dockerfile</tt> to add | |
| 273 | -configuration options, add APK packages to support those options, and so | |
| 274 | -forth. It also strips out all but the default and darkmode skins to save | |
| 275 | -executable space. | |
| 276 | - | |
| 277 | -There are two convenience targets for you, <tt>make container-image</tt> | |
| 278 | -and <tt>make container-run</tt>. The first creates a versioned container | |
| 279 | -image, and the second does that and then launches a fresh container | |
| 280 | -based on that image. You can pass extra arguments to the first command | |
| 281 | -via the Makefile's <tt>DBFLAGS</tt> variable and to the second with the | |
| 282 | -<tt>DRFLAGS</tt> variable. To get the custom port setting as in second | |
| 283 | -command above, say: | |
| 284 | - | |
| 285 | -<pre><code> $ make container-run DRFLAGS='-p 9999:8080/tcp'</code></pre> | |
| 286 | - | |
| 287 | -Contrast the raw "<tt>docker</tt>" commands above, which create an | |
| 288 | -<i>unversioned</i> image called <tt>fossil:latest</tt> and from that a | |
| 289 | -container simply called <tt>fossil</tt>. The unversioned names are more | |
| 290 | -convenient for interactive use, while the versioned ones are good for | |
| 291 | -CI/CD type applications since they avoid a conflict with past versions; | |
| 292 | -it lets you keep old containers around for quick roll-backs while | |
| 293 | -replacing them with fresh ones. | |
| 294 | - | |
| 295 | - | |
| 296 | -<h3 id="docker-storage">5.1 Repository Storage Options</h3> | |
| 297 | - | |
| 298 | -If you want the container to serve an existing repository, there are at | |
| 299 | -least two right ways to do it. | |
| 300 | - | |
| 301 | -The wrong way is to use the <tt>Dockerfile COPY</tt> command to bake it | |
| 302 | -into the image at build time. It will become one of the image's base | |
| 303 | -layers, so that each time you build a container from that image, the | |
| 304 | -repo gets reset to its build-time state. Worse, restarting the container | |
| 305 | -will do the same thing, since the base image layers are immutable in | |
| 306 | -Docker. This is almost certainly not what you want. | |
| 307 | - | |
| 308 | -The correct ways put the repo into the <i>container</i> created from the | |
| 309 | -<i>image</i>, not in the image itself. | |
| 310 | - | |
| 311 | - | |
| 312 | -<h4>5.1.1 Storing the Repo Inside the Container</h4> | |
| 313 | - | |
| 314 | -The simplest method is to stop the container if it was running, | |
| 315 | -then say: | |
| 316 | - | |
| 317 | -<pre><code> $ docker cp /path/to/my-project.fossil fossil:/jail/museum/repo.fossil | |
| 318 | - $ docker start fossil | |
| 319 | - $ docker exec fossil chown -R 499 /jail/museum</code></pre> | |
| 320 | - | |
| 321 | -That copies the local Fossil repo into the container where the server | |
| 322 | -expects to find it, so that the "start" command causes it to serve from | |
| 323 | -that copied-in file instead. Since it lives atop the immutable base layers, it | |
| 324 | -persists as part of the container proper, surviving restarts. | |
| 325 | - | |
| 326 | -Notice that the copy command changes the name of the repository database. | |
| 327 | -The container configuration expects it to be called | |
| 328 | -<tt>repo.fossil</tt>, which it almost certainly was not out on the host | |
| 329 | -system. This is because there is only one repository inside this | |
| 330 | -container, so we don't have to name it after the project it contains, as | |
| 331 | -is traditional. A generic name lets us hard-code the server start | |
| 332 | -command. | |
| 333 | - | |
| 334 | -If you skip the "chown" command above and put "http://localhost:9999/" into | |
| 335 | -your browser, expecting to see the copied-in repo's home page, you will | |
| 336 | -get an opaque "Not Found" error. This is because the user and | |
| 337 | -group ID of the file will be that of your local user on the container's | |
| 338 | -host machine, which is unlikely to map to anything in the container's | |
| 339 | -<tt>/etc/passwd</tt> and <tt>/etc/group</tt> files, effectively | |
| 340 | -preventing the server from reading the copied-in repository file. | |
| 341 | -499 is the default "fossil" user ID inside the container, causing Fossil to run | |
| 342 | -with that user's privileges after it enters the chroot. | |
| 343 | -(See [#docker-args | below] for how to change this default.) | |
| 344 | -You don't have to restart the server after fixing this with | |
| 345 | -<tt>chmod</tt>: simply reload the browser, and Fossil will try again. | |
| 346 | - | |
| 347 | - | |
| 348 | -<h4 id="docker-bind-mount">5.1.2 Storing the Repo Outside the Container</h4> | |
| 349 | - | |
| 350 | -The simple storage method above has a problem: Docker containers are designed to be | |
| 351 | -killed off at the slightest cause, rebuilt, and redeployed. If you do | |
| 352 | -that with the repo inside the container, it gets destroyed, too. The | |
| 353 | -solution is to replace the "run" command above with the following: | |
| 354 | - | |
| 355 | -<pre><code> $ docker run \ | |
| 356 | - --name fossil-bind-mount -p 9999:8080 \ | |
| 357 | - -v ~/museum:/jail/museum fossil | |
| 358 | -</code></pre> | |
| 359 | - | |
| 360 | -Because this bind mount maps a host-side directory (<tt>~/museum</tt>) | |
| 361 | -into the container, you don't need to <tt>docker cp</tt> the repo into | |
| 362 | -the container at all. It still expects to find the repository as | |
| 363 | -<tt>repo.fossil</tt> under that directory, but now both the host and the | |
| 364 | -container can see that file. (Beware: This may create a | |
| 365 | -[https://www.sqlite.org/howtocorrupt.html | risk of data corruption] due | |
| 366 | -to SQLite locking issues if you try to modify the DB from both sides at | |
| 367 | -once.) | |
| 368 | - | |
| 369 | -Instead of a bind mount, you could instead set up | |
| 370 | -a separate [https://docs.docker.com/storage/volumes/ | Docker volume], | |
| 371 | -at which point you <i>would</i> need to <tt>docker cp</tt> the repo file | |
| 372 | -into the container. | |
| 373 | - | |
| 374 | -Either way, files in these mounted directories have a lifetime independent | |
| 375 | -of the container(s) they're mounted into. When you need to | |
| 376 | -rebuild the container or its underlying image — such as to upgrade to a newer version of Fossil | |
| 377 | -— the external directory remains behind and gets remapped into the new container | |
| 378 | -when you recreate it with <tt>-v</tt>. | |
| 379 | - | |
| 380 | - | |
| 381 | -<h3 id="docker-security">5.2 Security</h3> | |
| 382 | - | |
| 383 | -<h4 id="docker-chroot">5.2.1 Why Chroot?</h4> | |
| 384 | - | |
| 385 | -A potentially surprising feature of this container is that it runs | |
| 386 | -Fossil as root. Since that causes [./chroot.md | Fossil's chroot jail | |
| 387 | -feature] to kick in, and a Docker container is a type of über-jail | |
| 388 | -already, you may be wondering why we bother. Instead, why not either: | |
| 389 | - | |
| 390 | - # run <tt>fossil server --nojail</tt> to skip the internal chroot; or | |
| 391 | - # set "<tt>USER fossil</tt>" in the Dockerfile so it starts Fossil as | |
| 392 | - that user instead | |
| 393 | - | |
| 394 | -The reason is, although this container is quite stripped-down by today's | |
| 395 | -standards, it's based on the [https://www.busybox.net/BusyBox.html | | |
| 396 | -surprisingly powerful Busybox project]. (This author made a living for | |
| 397 | -years in the early 1990s using Unix systems that were less powerful than | |
| 398 | -this container.) If someone ever figured out how to make a Fossil binary | |
| 399 | -execute arbitrary commands on the host or to open up a remote shell, | |
| 400 | -the power available to them at that point would make it likely that | |
| 401 | -they'd be able to island-hop from there into the rest of your network. | |
| 402 | -That power is there for you as the system administrator, to let you | |
| 403 | -inspect the container's runtime behavior, change things on the fly, and | |
| 404 | -so forth. Fossil proper doesn't need that power; if we take it away via | |
| 405 | -this cute double-jail dance, we keep any potential attacker from making | |
| 406 | -use of it should they ever get in. | |
| 407 | - | |
| 408 | -Having said this, know that | |
| 409 | -we deem this risk low since a) it's never happened, that we know of; | |
| 410 | -and b) we haven't enabled any of the risky features of Fossil such as | |
| 411 | -[https://fossil-scm.org/forum/forumpost/42e0c16544 | TH1 docs]. | |
| 412 | -Nevertheless, we believe in defense-in-depth. | |
| 413 | - | |
| 414 | -If you say something like "<tt>docker exec fossil ps</tt>" while the | |
| 415 | -system is idle, it's likely to report a single <tt>fossil</tt> process | |
| 416 | -running as <tt>root</tt> even though the chroot feature is documented as | |
| 417 | -causing Fossil to drop its privileges in favor of the owner of the | |
| 418 | -repository database or its containing folder. If the repo file is owned | |
| 419 | -by the in-container user "<tt>fossil</tt>", why is the server still | |
| 420 | -running as root? | |
| 421 | - | |
| 422 | -It's because you're seeing only the parent | |
| 423 | -process, which assumes it's running on bare metal or a VM and thus | |
| 424 | -may need to do rootly things like listening on port 80 or | |
| 425 | -443 before forking off any children to handle HTTP hits. | |
| 426 | -Fossil's chroot feature only takes effect in these child processes. | |
| 427 | -This is why you can fix broken | |
| 428 | -permissions with <tt>chown</tt> after the container is already running, | |
| 429 | -without restarting it: each hit reevaluates the repository file | |
| 430 | -permissions when deciding what user to become when dropping root | |
| 431 | -privileges. | |
| 432 | - | |
| 433 | - | |
| 434 | -<h4 id="docker-caps">5.2.2 Dropping Unnecessary Capabilities</h4> | |
| 435 | - | |
| 436 | -The example commands given in this section create the container with | |
| 437 | -[https://docs.docker.com/engine/security/#linux-kernel-capabilities | a | |
| 438 | -default set of Linux kernel capabilities]. Although Docker strips almost | |
| 439 | -all of the traditional root capabilities away by default, and Fossil | |
| 440 | -doesn't need any of those it does take away, Docker does leave some | |
| 441 | -enabled that Fossil doesn't actually need. You can tighten the scope of | |
| 442 | -capabilities by adding a "<tt>--cap-drop LIST</tt>" option to your | |
| 443 | -container creation commands. Specifically: | |
| 444 | - | |
| 445 | - * <b><tt>AUDIT_WRITE</tt></b>: Fossil doesn't write to the kernel's | |
| 446 | - auditing log, and we can't see any reason you'd want to be able to | |
| 447 | - do that as an administrator shelled into the container, either. | |
| 448 | - Auditing is something done on the host, not from inside each | |
| 449 | - individual container.<p> | |
| 450 | - * <b><tt>CHOWN</tt></b>: The Fossil server never even calls | |
| 451 | - <tt>chown(2)</tt>, and our image build process sets up all file | |
| 452 | - ownership properly, to the extent that this is possible under the | |
| 453 | - limitations of our automation.<p> | |
| 454 | - Curiously, stripping this capability doesn't affect your ability to | |
| 455 | - run commands like "<tt>chown -R fossil:fossil /jail/museum</tt>" | |
| 456 | - when you're using bind mounts or external volumes — as we recommend | |
| 457 | - [#docker-bind-mount | above] — because it's the host OS's kernel | |
| 458 | - capabilities that affect the underlying <tt>chown(2)</tt> call in | |
| 459 | - that case, not those of the container.<p> | |
| 460 | - If for some reason you did have to change file ownership of | |
| 461 | - in-container files, it's best to do that by changing the | |
| 462 | - <tt>Dockerfile</tt> to suit, then rebuilding the container, since | |
| 463 | - that bakes the need for the change into your reproducible build | |
| 464 | - process. If you had to do it without rebuilding the container, | |
| 465 | - [https://stackoverflow.com/a/45752205/142454 | there's a | |
| 466 | - workaround] for the fact that capabilities are a create-time | |
| 467 | - change, baked semi-indelibly into the container configuration.<p> | |
| 468 | - * <b><tt>FSETID</tt></b>: Fossil doesn't use the SUID and SGID bits | |
| 469 | - itself, and our build process doesn't set those flags on any of the | |
| 470 | - files. Although the second fact means we can't see any harm from | |
| 471 | - leaving this enabled, we also can't see any good reason to allow | |
| 472 | - it, so we strip it.<p> | |
| 473 | - * <b><tt>KILL</tt></b>: The only place Fossil calls <tt>kill(2)</tt> | |
| 474 | - is in the [./backoffice.md | backoffice], and then only for | |
| 475 | - processes it created on earlier runs; it doesn't need the ability | |
| 476 | - to kill processes created by other users. You might wish for this | |
| 477 | - ability as an administrator shelled into the container, but you can | |
| 478 | - pass the "<tt>docker exec --user</tt>" option to run commands | |
| 479 | - within your container as the legitimate owner of the process, | |
| 480 | - removing the need for this capability.<p> | |
| 481 | - * <b><tt>MKNOD</tt></b>: All device nodes are created at build time | |
| 482 | - and are never changed at run time. Realize that the virtualized | |
| 483 | - device nodes inside the container get mapped onto real devices on | |
| 484 | - the host, so if an attacker ever got a root shell on the container, | |
| 485 | - they might be able to do actual damage to the host if we didn't | |
| 486 | - preemptively strip this capability away.<p> | |
| 487 | - * <b><tt>NET_BIND_SERVICE</tt></b>: With containerized deployment, | |
| 488 | - Fossil never needs the ability to bind the server to low-numbered | |
| 489 | - TCP ports, not even if you're running the server in production with | |
| 490 | - TLS enabled and want the service bound to port 443. It's perfectly | |
| 491 | - fine to let the Fossil instance inside the container bind to its | |
| 492 | - default port (8080) because you can rebind it on the host with the | |
| 493 | - "<tt>docker create --publish 443:8080</tt>" option. It's the | |
| 494 | - container's <i>host</i> that needs this ability, not the container | |
| 495 | - itself.<p> (Even the container runtime might not need that | |
| 496 | - capability if you're [./ssl.wiki#server | terminating TLS with a | |
| 497 | - front-end proxy]. You're more likely to say something like "<tt>-p | |
| 498 | - localhost:12345:8080</tt>", then configure the reverse proxy to | |
| 499 | - translate external HTTPS calls into HTTP directed at this internal | |
| 500 | - port 12345.)<p> | |
| 501 | - * <b><tt>NET_RAW</tt></b>: Fossil itself doesn't use raw sockets, and | |
| 502 | - our build process leaves out all the Busybox utilities that require | |
| 503 | - them. Although that set includes common tools like <tt>ping</tt>, | |
| 504 | - we foresee no compelling reason to use that or any of these other | |
| 505 | - elided utilities — <tt>ether-wake</tt>, <tt>netstat</tt>, | |
| 506 | - <tt>traceroute</tt>, and <tt>udhcp</tt> — inside the container. If | |
| 507 | - you need to ping something, do it on the host.<p> | |
| 508 | - If we did not take this hard-line stance, an attacker that broke | |
| 509 | - into the container and gained root privileges might use raw sockets | |
| 510 | - to do a wide array of bad things to any network the container is | |
| 511 | - bound to.<p> | |
| 512 | - * <b><tt>SETFCAP, SETPCAP</tt></b>: There isn't much call for file | |
| 513 | - permission granularity beyond the classic Unix ones inside the | |
| 514 | - container, so we drop root's ability to change them. | |
| 515 | - | |
| 516 | -All together, we recommend adding the following options to your | |
| 517 | -"<tt>docker run</tt>" commands, as well as to any "<tt>docker | |
| 518 | -create</tt>" command that will be followed by "<tt>docker start</tt>": | |
| 519 | - | |
| 520 | -<pre><code> --cap-drop AUDIT_WRITE \ | |
| 521 | - --cap-drop CHOWN \ | |
| 522 | - --cap-drop FSETID \ | |
| 523 | - --cap-drop KILL \ | |
| 524 | - --cap-drop MKNOD \ | |
| 525 | - --cap-drop NET_BIND_SERVICE \ | |
| 526 | - --cap-drop NET_RAW \ | |
| 527 | - --cap-drop SETFCAP \ | |
| 528 | - --cap-drop SETPCAP | |
| 529 | -</code></pre> | |
| 530 | - | |
| 531 | -In the next section, we'll show a case where you create a container | |
| 532 | -without ever running it, making these options pointless. | |
| 533 | - | |
| 534 | - | |
| 535 | - | |
| 536 | -<h3 id="docker-static">5.3 Extracting a Static Binary</h3> | |
| 537 | - | |
| 538 | -Our 2-stage build process uses Alpine Linux only as a build host. Once | |
| 539 | -we've got everything reduced to the two key static binaries — Fossil and | |
| 540 | -Busybox — we throw all the rest of it away. | |
| 541 | - | |
| 542 | -A secondary benefit falls out of this process for free: it's arguably | |
| 543 | -the easiest way to build a purely static Fossil binary for Linux. Most | |
| 544 | -modern Linux distros make this surprisingly difficult, but Alpine's | |
| 545 | -back-to-basics nature makes static builds work the way they used to, | |
| 546 | -back in the day. If that's what you're after, you can skip the "run" | |
| 547 | -command above and create a temporary container from the image, then | |
| 548 | -extract the executable from it instead: | |
| 549 | - | |
| 550 | -<pre><code> $ docker create --name fossil-static-tmp fossil | |
| 551 | - $ docker cp fossil-static-tmp:/jail/bin/fossil . | |
| 552 | - $ docker container rm fossil-static-tmp | |
| 553 | -</code></pre> | |
| 554 | - | |
| 555 | -The resulting binary is the single largest file inside that container, | |
| 556 | -at about 6 MiB. (It's built stripped.) | |
| 557 | - | |
| 558 | - | |
| 559 | -<h3 id="docker-args">5.4 Container Build Arguments</h3> | |
| 560 | - | |
| 561 | -<h4>5.4.1 Package Versions</h4> | |
| 562 | - | |
| 563 | -You can override the default versions of Fossil and BusyBox that get | |
| 564 | -fetched in the build step. To get the latest-and-greatest of | |
| 565 | -everything, you could say: | |
| 566 | - | |
| 567 | -<pre><code> $ docker build -t fossil \ | |
| 568 | - --build-arg FSLVER=trunk \ | |
| 569 | - --build-arg BBXVER=master .</code></pre> | |
| 570 | - | |
| 571 | -(But don't, for reasons we will get to.) | |
| 572 | - | |
| 573 | -Because the BusyBox configuration file we ship was created with and | |
| 574 | -tested against a specific stable release, that's the version we pull by | |
| 575 | -default. It does try to merge the defaults for any new configuration | |
| 576 | -settings into the stock set, but since it's possible this will fail, we | |
| 577 | -don't blindly update the BusyBox version merely because a new release | |
| 578 | -came out. Someone needs to get around to vetting it against our stock | |
| 579 | -configuration first. | |
| 580 | - | |
| 581 | -As for Fossil, it defaults to fetching the same version as the checkout | |
| 582 | -you're running the build command from, based on checkin ID. The most | |
| 583 | -common reason to override this is to get a release version: | |
| 584 | - | |
| 585 | -<pre><code> $ docker build -t fossil \ | |
| 586 | - --build-arg FSLVER=version-2.19 .</code></pre> | |
| 587 | - | |
| 588 | -It's best to use a specific version number rather than the generic | |
| 589 | -"release" tag because Docker caches downloaded files and tries to reuse | |
| 590 | -them across builds. If you ask for "release" before a new version is | |
| 591 | -tagged and then immediately after, you expect to get two different | |
| 592 | -tarballs, but because the URL hasn't changed, if you have an old release | |
| 593 | -tarball in your Docker cache, you'll get the old version even if you | |
| 594 | -pass the "docker build --no-cache" option. | |
| 595 | - | |
| 596 | -This is why we default to a checkin ID rather than a generic tag like | |
| 597 | -"trunk": so the URL will change each time you update your Fossil source | |
| 598 | -tree, forcing Docker to pull a fresh tarball. | |
| 599 | - | |
| 600 | -<h4>5.4.2 User & Group IDs</h4> | |
| 601 | - | |
| 602 | -The "fossil" user and group IDs inside the container default to 499. | |
| 603 | -Why? Regular user IDs start at 500 or 1000 on most Unix type systems, | |
| 604 | -leaving those below it for "system" users like this Fossil daemon owner. | |
| 605 | -Since standard OS system users start at 0 and go upward, we started at | |
| 606 | -500 and went down one to reduce the chance of a conflict to as close to | |
| 607 | -zero as we can manage. | |
| 608 | - | |
| 609 | -To change it to something else, say: | |
| 610 | - | |
| 611 | -<pre><code> $ docker build -t fossil --build-arg UID=501 .</code></pre> | |
| 612 | - | |
| 613 | -This is particularly useful if you're putting your repository on a | |
| 614 | -Docker volume since the IDs "leak" out into the host environment via | |
| 615 | -file permissions. You may therefore wish them to mean something on both | |
| 616 | -sides of the container barrier rather than have "499" appear on the host | |
| 617 | -in "<tt>ls -l</tt>" output. | |
| 255 | +<h2 id="docker" name="oci">5.0 Building a Docker Container</h2> | |
| 256 | + | |
| 257 | +The information on building Fossil inside an | |
| 258 | +[https://opencontainers.org/ | OCI container] is now in | |
| 259 | +[./containers.md | a separate document]. | |
| 260 | + | |
| 261 | +This includes the instructions on using the OCI container as an | |
| 262 | +expedient intermediary for building a statically-linked Fossil binary on | |
| 263 | +modern Linux platforms, which otherwise make this difficult. | |
| 618 | 264 | |
| 619 | 265 | |
| 620 | 266 | <h2>6.0 Building on/for Android</h2> |
| 621 | 267 | |
| 622 | 268 | <h3>6.1 Cross-compiling from Linux</h3> |
| 623 | 269 | |
| 624 | 270 | ADDED www/containers.md |
| --- www/build.wiki | |
| +++ www/build.wiki | |
| @@ -250,374 +250,19 @@ | |
| 250 | TCC += -DSQLITE_MAX_MMAP_SIZE=0 |
| 251 | </pre></blockquote> |
| 252 | </ul> |
| 253 | |
| 254 | |
| 255 | <h2 id="docker">5.0 Building a Docker Container</h2> |
| 256 | |
| 257 | Fossil ships a <tt>Dockerfile</tt> at the top of its source tree which |
| 258 | you can build like so: |
| 259 | |
| 260 | <pre><code> $ docker build -t fossil .</code></pre> |
| 261 | |
| 262 | If the image built successfully, you can create a container from it and |
| 263 | test that it runs: |
| 264 | |
| 265 | <pre><code> $ docker run --name fossil -p 9999:8080/tcp fossil</code></pre> |
| 266 | |
| 267 | This shows us remapping the internal TCP listening port as 9999 on the |
| 268 | host. As a rule, there's little point to using the "<tt>fossil server |
| ---port</tt>" feature inside a Docker container. Let it default to 8080 | |
| 269 | internally, then remap it to wherever you want it on the host instead. |
| 270 | |
| 271 | Our stock <tt>Dockerfile</tt> configures Fossil with the default feature |
| 272 | set, so you may wish to modify the <tt>Dockerfile</tt> to add |
| 273 | configuration options, add APK packages to support those options, and so |
| 274 | forth. It also strips out all but the default and darkmode skins to save |
| 275 | executable space. |
| 276 | |
| 277 | There are two convenience targets for you, <tt>make container-image</tt> |
| 278 | and <tt>make container-run</tt>. The first creates a versioned container |
| 279 | image, and the second does that and then launches a fresh container |
| 280 | based on that image. You can pass extra arguments to the first command |
| 281 | via the Makefile's <tt>DBFLAGS</tt> variable and to the second with the |
| 282 | <tt>DRFLAGS</tt> variable. To get the custom port setting as in second |
| 283 | command above, say: |
| 284 | |
| 285 | <pre><code> $ make container-run DRFLAGS='-p 9999:8080/tcp'</code></pre> |
| 286 | |
| 287 | Contrast the raw "<tt>docker</tt>" commands above, which create an |
| 288 | <i>unversioned</i> image called <tt>fossil:latest</tt> and from that a |
| 289 | container simply called <tt>fossil</tt>. The unversioned names are more |
| 290 | convenient for interactive use, while the versioned ones are good for |
| 291 | CI/CD type applications since they avoid a conflict with past versions; |
| 292 | it lets you keep old containers around for quick roll-backs while |
| 293 | replacing them with fresh ones. |
| 294 | |
| 295 | |
| 296 | <h3 id="docker-storage">5.1 Repository Storage Options</h3> |
| 297 | |
| 298 | If you want the container to serve an existing repository, there are at |
| 299 | least two right ways to do it. |
| 300 | |
| 301 | The wrong way is to use the <tt>Dockerfile COPY</tt> command to bake it |
| 302 | into the image at build time. It will become one of the image's base |
| 303 | layers, so that each time you build a container from that image, the |
| 304 | repo gets reset to its build-time state. Worse, restarting the container |
| 305 | will do the same thing, since the base image layers are immutable in |
| 306 | Docker. This is almost certainly not what you want. |
| 307 | |
| 308 | The correct ways put the repo into the <i>container</i> created from the |
| 309 | <i>image</i>, not in the image itself. |
| 310 | |
| 311 | |
| 312 | <h4>5.1.1 Storing the Repo Inside the Container</h4> |
| 313 | |
| 314 | The simplest method is to stop the container if it was running, |
| 315 | then say: |
| 316 | |
| 317 | <pre><code> $ docker cp /path/to/my-project.fossil fossil:/jail/museum/repo.fossil |
| 318 | $ docker start fossil |
| 319 | $ docker exec fossil chown -R 499 /jail/museum</code></pre> |
| 320 | |
| 321 | That copies the local Fossil repo into the container where the server |
| 322 | expects to find it, so that the "start" command causes it to serve from |
| 323 | that copied-in file instead. Since it lives atop the immutable base layers, it |
| 324 | persists as part of the container proper, surviving restarts. |
| 325 | |
| 326 | Notice that the copy command changes the name of the repository database. |
| 327 | The container configuration expects it to be called |
| 328 | <tt>repo.fossil</tt>, which it almost certainly was not out on the host |
| 329 | system. This is because there is only one repository inside this |
| 330 | container, so we don't have to name it after the project it contains, as |
| 331 | is traditional. A generic name lets us hard-code the server start |
| 332 | command. |
| 333 | |
| 334 | If you skip the "chown" command above and put "http://localhost:9999/" into |
| 335 | your browser, expecting to see the copied-in repo's home page, you will |
| 336 | get an opaque "Not Found" error. This is because the user and |
| 337 | group ID of the file will be that of your local user on the container's |
| 338 | host machine, which is unlikely to map to anything in the container's |
| 339 | <tt>/etc/passwd</tt> and <tt>/etc/group</tt> files, effectively |
| 340 | preventing the server from reading the copied-in repository file. |
| 341 | 499 is the default "fossil" user ID inside the container, causing Fossil to run |
| 342 | with that user's privileges after it enters the chroot. |
| 343 | (See [#docker-args | below] for how to change this default.) |
| 344 | You don't have to restart the server after fixing this with |
| 345 | <tt>chmod</tt>: simply reload the browser, and Fossil will try again. |
| 346 | |
| 347 | |
| 348 | <h4 id="docker-bind-mount">5.1.2 Storing the Repo Outside the Container</h4> |
| 349 | |
| 350 | The simple storage method above has a problem: Docker containers are designed to be |
| 351 | killed off at the slightest cause, rebuilt, and redeployed. If you do |
| 352 | that with the repo inside the container, it gets destroyed, too. The |
| 353 | solution is to replace the "run" command above with the following: |
| 354 | |
| 355 | <pre><code> $ docker run \ |
| 356 | --name fossil-bind-mount -p 9999:8080 \ |
| 357 | -v ~/museum:/jail/museum fossil |
| 358 | </code></pre> |
| 359 | |
| 360 | Because this bind mount maps a host-side directory (<tt>~/museum</tt>) |
| 361 | into the container, you don't need to <tt>docker cp</tt> the repo into |
| 362 | the container at all. It still expects to find the repository as |
| 363 | <tt>repo.fossil</tt> under that directory, but now both the host and the |
| 364 | container can see that file. (Beware: This may create a |
| 365 | [https://www.sqlite.org/howtocorrupt.html | risk of data corruption] due |
| 366 | to SQLite locking issues if you try to modify the DB from both sides at |
| 367 | once.) |
| 368 | |
| 369 | Instead of a bind mount, you could instead set up |
| 370 | a separate [https://docs.docker.com/storage/volumes/ | Docker volume], |
| 371 | at which point you <i>would</i> need to <tt>docker cp</tt> the repo file |
| 372 | into the container. |
| 373 | |
| 374 | Either way, files in these mounted directories have a lifetime independent |
| 375 | of the container(s) they're mounted into. When you need to |
| 376 | rebuild the container or its underlying image — such as to upgrade to a newer version of Fossil |
| 377 | — the external directory remains behind and gets remapped into the new container |
| 378 | when you recreate it with <tt>-v</tt>. |
| 379 | |
| 380 | |
| 381 | <h3 id="docker-security">5.2 Security</h3> |
| 382 | |
| 383 | <h4 id="docker-chroot">5.2.1 Why Chroot?</h4> |
| 384 | |
| 385 | A potentially surprising feature of this container is that it runs |
| 386 | Fossil as root. Since that causes [./chroot.md | Fossil's chroot jail |
| 387 | feature] to kick in, and a Docker container is a type of über-jail |
| 388 | already, you may be wondering why we bother. Instead, why not either: |
| 389 | |
| 390 | # run <tt>fossil server --nojail</tt> to skip the internal chroot; or |
| 391 | # set "<tt>USER fossil</tt>" in the Dockerfile so it starts Fossil as |
| 392 | that user instead |
| 393 | |
| 394 | The reason is, although this container is quite stripped-down by today's |
| 395 | standards, it's based on the [https://www.busybox.net/BusyBox.html | |
| 396 | surprisingly powerful Busybox project]. (This author made a living for |
| 397 | years in the early 1990s using Unix systems that were less powerful than |
| 398 | this container.) If someone ever figured out how to make a Fossil binary |
| 399 | execute arbitrary commands on the host or to open up a remote shell, |
| 400 | the power available to them at that point would make it likely that |
| 401 | they'd be able to island-hop from there into the rest of your network. |
| 402 | That power is there for you as the system administrator, to let you |
| 403 | inspect the container's runtime behavior, change things on the fly, and |
| 404 | so forth. Fossil proper doesn't need that power; if we take it away via |
| 405 | this cute double-jail dance, we keep any potential attacker from making |
| 406 | use of it should they ever get in. |
| 407 | |
| 408 | Having said this, know that |
| 409 | we deem this risk low since a) it's never happened, that we know of; |
| 410 | and b) we haven't enabled any of the risky features of Fossil such as |
| 411 | [https://fossil-scm.org/forum/forumpost/42e0c16544 | TH1 docs]. |
| 412 | Nevertheless, we believe in defense-in-depth. |
| 413 | |
| 414 | If you say something like "<tt>docker exec fossil ps</tt>" while the |
| 415 | system is idle, it's likely to report a single <tt>fossil</tt> process |
| 416 | running as <tt>root</tt> even though the chroot feature is documented as |
| 417 | causing Fossil to drop its privileges in favor of the owner of the |
| 418 | repository database or its containing folder. If the repo file is owned |
| 419 | by the in-container user "<tt>fossil</tt>", why is the server still |
| 420 | running as root? |
| 421 | |
| 422 | It's because you're seeing only the parent |
| 423 | process, which assumes it's running on bare metal or a VM and thus |
| 424 | may need to do rootly things like listening on port 80 or |
| 425 | 443 before forking off any children to handle HTTP hits. |
| 426 | Fossil's chroot feature only takes effect in these child processes. |
| 427 | This is why you can fix broken |
| 428 | permissions with <tt>chown</tt> after the container is already running, |
| 429 | without restarting it: each hit reevaluates the repository file |
| 430 | permissions when deciding what user to become when dropping root |
| 431 | privileges. |
| 432 | |
| 433 | |
| 434 | <h4 id="docker-caps">5.2.2 Dropping Unnecessary Capabilities</h4> |
| 435 | |
| 436 | The example commands given in this section create the container with |
| 437 | [https://docs.docker.com/engine/security/#linux-kernel-capabilities | a |
| 438 | default set of Linux kernel capabilities]. Although Docker strips almost |
| 439 | all of the traditional root capabilities away by default, and Fossil |
| 440 | doesn't need any of those it does take away, Docker does leave some |
| 441 | enabled that Fossil doesn't actually need. You can tighten the scope of |
| 442 | capabilities by adding a "<tt>--cap-drop LIST</tt>" option to your |
| 443 | container creation commands. Specifically: |
| 444 | |
| 445 | * <b><tt>AUDIT_WRITE</tt></b>: Fossil doesn't write to the kernel's |
| 446 | auditing log, and we can't see any reason you'd want to be able to |
| 447 | do that as an administrator shelled into the container, either. |
| 448 | Auditing is something done on the host, not from inside each |
| 449 | individual container.<p> |
| 450 | * <b><tt>CHOWN</tt></b>: The Fossil server never even calls |
| 451 | <tt>chown(2)</tt>, and our image build process sets up all file |
| 452 | ownership properly, to the extent that this is possible under the |
| 453 | limitations of our automation.<p> |
| 454 | Curiously, stripping this capability doesn't affect your ability to |
| 455 | run commands like "<tt>chown -R fossil:fossil /jail/museum</tt>" |
| 456 | when you're using bind mounts or external volumes — as we recommend |
| 457 | [#docker-bind-mount | above] — because it's the host OS's kernel |
| 458 | capabilities that affect the underlying <tt>chown(2)</tt> call in |
| 459 | that case, not those of the container.<p> |
| 460 | If for some reason you did have to change file ownership of |
| 461 | in-container files, it's best to do that by changing the |
| 462 | <tt>Dockerfile</tt> to suit, then rebuilding the container, since |
| 463 | that bakes the need for the change into your reproducible build |
| 464 | process. If you had to do it without rebuilding the container, |
| 465 | [https://stackoverflow.com/a/45752205/142454 | there's a |
| 466 | workaround] for the fact that capabilities are a create-time |
| 467 | change, baked semi-indelibly into the container configuration.<p> |
| 468 | * <b><tt>FSETID</tt></b>: Fossil doesn't use the SUID and SGID bits |
| 469 | itself, and our build process doesn't set those flags on any of the |
| 470 | files. Although the second fact means we can't see any harm from |
| 471 | leaving this enabled, we also can't see any good reason to allow |
| 472 | it, so we strip it.<p> |
| 473 | * <b><tt>KILL</tt></b>: The only place Fossil calls <tt>kill(2)</tt> |
| 474 | is in the [./backoffice.md | backoffice], and then only for |
| 475 | processes it created on earlier runs; it doesn't need the ability |
| 476 | to kill processes created by other users. You might wish for this |
| 477 | ability as an administrator shelled into the container, but you can |
| 478 | pass the "<tt>docker exec --user</tt>" option to run commands |
| 479 | within your container as the legitimate owner of the process, |
| 480 | removing the need for this capability.<p> |
| 481 | * <b><tt>MKNOD</tt></b>: All device nodes are created at build time |
| 482 | and are never changed at run time. Realize that the virtualized |
| 483 | device nodes inside the container get mapped onto real devices on |
| 484 | the host, so if an attacker ever got a root shell on the container, |
| 485 | they might be able to do actual damage to the host if we didn't |
| 486 | preemptively strip this capability away.<p> |
| 487 | * <b><tt>NET_BIND_SERVICE</tt></b>: With containerized deployment, |
| 488 | Fossil never needs the ability to bind the server to low-numbered |
| 489 | TCP ports, not even if you're running the server in production with |
| 490 | TLS enabled and want the service bound to port 443. It's perfectly |
| 491 | fine to let the Fossil instance inside the container bind to its |
| 492 | default port (8080) because you can rebind it on the host with the |
| 493 | "<tt>docker create --publish 443:8080</tt>" option. It's the |
| 494 | container's <i>host</i> that needs this ability, not the container |
| 495 | itself.<p> (Even the container runtime might not need that |
| 496 | capability if you're [./ssl.wiki#server | terminating TLS with a |
| 497 | front-end proxy]. You're more likely to say something like "<tt>-p |
| 498 | localhost:12345:8080</tt>", then configure the reverse proxy to |
| 499 | translate external HTTPS calls into HTTP directed at this internal |
| 500 | port 12345.)<p> |
| 501 | * <b><tt>NET_RAW</tt></b>: Fossil itself doesn't use raw sockets, and |
| 502 | our build process leaves out all the Busybox utilities that require |
| 503 | them. Although that set includes common tools like <tt>ping</tt>, |
| 504 | we foresee no compelling reason to use that or any of these other |
| 505 | elided utilities — <tt>ether-wake</tt>, <tt>netstat</tt>, |
| 506 | <tt>traceroute</tt>, and <tt>udhcp</tt> — inside the container. If |
| 507 | you need to ping something, do it on the host.<p> |
| 508 | If we did not take this hard-line stance, an attacker that broke |
| 509 | into the container and gained root privileges might use raw sockets |
| 510 | to do a wide array of bad things to any network the container is |
| 511 | bound to.<p> |
| 512 | * <b><tt>SETFCAP, SETPCAP</tt></b>: There isn't much call for file |
| 513 | permission granularity beyond the classic Unix ones inside the |
| 514 | container, so we drop root's ability to change them. |
| 515 | |
| 516 | All together, we recommend adding the following options to your |
| 517 | "<tt>docker run</tt>" commands, as well as to any "<tt>docker |
| 518 | create</tt>" command that will be followed by "<tt>docker start</tt>": |
| 519 | |
| 520 | <pre><code> --cap-drop AUDIT_WRITE \ |
| 521 | --cap-drop CHOWN \ |
| 522 | --cap-drop FSETID \ |
| 523 | --cap-drop KILL \ |
| 524 | --cap-drop MKNOD \ |
| 525 | --cap-drop NET_BIND_SERVICE \ |
| 526 | --cap-drop NET_RAW \ |
| 527 | --cap-drop SETFCAP \ |
| 528 | --cap-drop SETPCAP |
| 529 | </code></pre> |
| 530 | |
| 531 | In the next section, we'll show a case where you create a container |
| 532 | without ever running it, making these options pointless. |
| 533 | |
| 534 | |
| 535 | |
| 536 | <h3 id="docker-static">5.3 Extracting a Static Binary</h3> |
| 537 | |
| 538 | Our 2-stage build process uses Alpine Linux only as a build host. Once |
| 539 | we've got everything reduced to the two key static binaries — Fossil and |
| 540 | Busybox — we throw all the rest of it away. |
| 541 | |
| 542 | A secondary benefit falls out of this process for free: it's arguably |
| 543 | the easiest way to build a purely static Fossil binary for Linux. Most |
| 544 | modern Linux distros make this surprisingly difficult, but Alpine's |
| 545 | back-to-basics nature makes static builds work the way they used to, |
| 546 | back in the day. If that's what you're after, you can skip the "run" |
| 547 | command above and create a temporary container from the image, then |
| 548 | extract the executable from it instead: |
| 549 | |
| 550 | <pre><code> $ docker create --name fossil-static-tmp fossil |
| 551 | $ docker cp fossil-static-tmp:/jail/bin/fossil . |
| 552 | $ docker container rm fossil-static-tmp |
| 553 | </code></pre> |
| 554 | |
| 555 | The resulting binary is the single largest file inside that container, |
| 556 | at about 6 MiB. (It's built stripped.) |
| 557 | |
| 558 | |
| 559 | <h3 id="docker-args">5.4 Container Build Arguments</h3> |
| 560 | |
| 561 | <h4>5.4.1 Package Versions</h4> |
| 562 | |
| 563 | You can override the default versions of Fossil and BusyBox that get |
| 564 | fetched in the build step. To get the latest-and-greatest of |
| 565 | everything, you could say: |
| 566 | |
| 567 | <pre><code> $ docker build -t fossil \ |
| 568 | --build-arg FSLVER=trunk \ |
| 569 | --build-arg BBXVER=master .</code></pre> |
| 570 | |
| 571 | (But don't, for reasons we will get to.) |
| 572 | |
| 573 | Because the BusyBox configuration file we ship was created with and |
| 574 | tested against a specific stable release, that's the version we pull by |
| 575 | default. It does try to merge the defaults for any new configuration |
| 576 | settings into the stock set, but since it's possible this will fail, we |
| 577 | don't blindly update the BusyBox version merely because a new release |
| 578 | came out. Someone needs to get around to vetting it against our stock |
| 579 | configuration first. |
| 580 | |
| 581 | As for Fossil, it defaults to fetching the same version as the checkout |
| 582 | you're running the build command from, based on checkin ID. The most |
| 583 | common reason to override this is to get a release version: |
| 584 | |
| 585 | <pre><code> $ docker build -t fossil \ |
| 586 | --build-arg FSLVER=version-2.19 .</code></pre> |
| 587 | |
| 588 | It's best to use a specific version number rather than the generic |
| 589 | "release" tag because Docker caches downloaded files and tries to reuse |
| 590 | them across builds. If you ask for "release" before a new version is |
| 591 | tagged and then immediately after, you expect to get two different |
| 592 | tarballs, but because the URL hasn't changed, if you have an old release |
| 593 | tarball in your Docker cache, you'll get the old version even if you |
| 594 | pass the "docker build --no-cache" option. |
| 595 | |
| 596 | This is why we default to a checkin ID rather than a generic tag like |
| 597 | "trunk": so the URL will change each time you update your Fossil source |
| 598 | tree, forcing Docker to pull a fresh tarball. |
| 599 | |
| 600 | <h4>5.4.2 User & Group IDs</h4> |
| 601 | |
| 602 | The "fossil" user and group IDs inside the container default to 499. |
| 603 | Why? Regular user IDs start at 500 or 1000 on most Unix type systems, |
| 604 | leaving those below it for "system" users like this Fossil daemon owner. |
| 605 | Since standard OS system users start at 0 and go upward, we started at |
| 606 | 500 and went down one to reduce the chance of a conflict to as close to |
| 607 | zero as we can manage. |
| 608 | |
| 609 | To change it to something else, say: |
| 610 | |
| 611 | <pre><code> $ docker build -t fossil --build-arg UID=501 .</code></pre> |
| 612 | |
| 613 | This is particularly useful if you're putting your repository on a |
| 614 | Docker volume since the IDs "leak" out into the host environment via |
| 615 | file permissions. You may therefore wish them to mean something on both |
| 616 | sides of the container barrier rather than have "499" appear on the host |
| 617 | in "<tt>ls -l</tt>" output. |
| 618 | |
| 619 | |
| 620 | <h2>6.0 Building on/for Android</h2> |
| 621 | |
| 622 | <h3>6.1 Cross-compiling from Linux</h3> |
| 623 | |
| 624 | DDED www/containers.md |
| --- www/build.wiki | |
| +++ www/build.wiki | |
| @@ -250,374 +250,19 @@ | |
| 250 | TCC += -DSQLITE_MAX_MMAP_SIZE=0 |
| 251 | </pre></blockquote> |
| 252 | </ul> |
| 253 | |
| 254 | |
| ---port</tt>" feature inside a Docker container. Let it default to 8080 | |
| 255 | <h2 id="docker" name="oci">5.0 Building a Docker Container</h2> |
| 256 | |
| 257 | The information on building Fossil inside an |
| 258 | [https://opencontainers.org/ | OCI container] is now in |
| 259 | [./containers.md | a separate document]. |
| 260 | |
| 261 | This includes the instructions on using the OCI container as an |
| 262 | expedient intermediary for building a statically-linked Fossil binary on |
| 263 | modern Linux platforms, which otherwise make this difficult. |
| 264 | |
| 265 | |
| 266 | <h2>6.0 Building on/for Android</h2> |
| 267 | |
| 268 | <h3>6.1 Cross-compiling from Linux</h3> |
| 269 | |
| 270 | DDED www/containers.md |
+12
| --- a/www/containers.md | ||
| +++ b/www/containers.md | ||
| @@ -0,0 +1,12 @@ | ||
| 1 | +surpr | |
| 2 | +### <a id="runc" name="containerd"></a>Stripping Docker Engine Down | |
| 3 | + | |
| 4 | +The core of Docker Engine is its [containerd] daemon and the [runc] | |
| 5 | +container run using only | |
| 6 | +these tools, leaving out all the rest. Those two pieces come to about a | |
| 7 | +tenth the size of Docker Engine on the system where I tested this. | |
| 8 | + | |
| 9 | +**TODO:** Work out how to do this and document it. | |
| 10 | + | |
| 11 | +[containerd]: https://containerd.io/ | |
| 12 | +[runc]: https://github.com/opencontainers/runc |
| --- a/www/containers.md | |
| +++ b/www/containers.md | |
| @@ -0,0 +1,12 @@ | |
| --- a/www/containers.md | |
| +++ b/www/containers.md | |
| @@ -0,0 +1,12 @@ | |
| 1 | surpr |
| 2 | ### <a id="runc" name="containerd"></a>Stripping Docker Engine Down |
| 3 | |
| 4 | The core of Docker Engine is its [containerd] daemon and the [runc] |
| 5 | container run using only |
| 6 | these tools, leaving out all the rest. Those two pieces come to about a |
| 7 | tenth the size of Docker Engine on the system where I tested this. |
| 8 | |
| 9 | **TODO:** Work out how to do this and document it. |
| 10 | |
| 11 | [containerd]: https://containerd.io/ |
| 12 | [runc]: https://github.com/opencontainers/runc |