Fossil SCM
Using the preceding --chroot fixes to make the Docker container serve the repo from /jail/museum/repo.fossil rather than from the chroot dir, /jail. This then allows us to mount a Docker volume at /jail/museum, which has an independent persistence from the container proper, so we can now rebuild the container without destroying the presumably precious repo. Updated build.wiki to track this change and document the lessons gleaned from doing all of this.
Commit
f76e762fb7008283e88686c4adc031bbd8eef4e607ad178cc7d4677363636787
Parent
e94621186f71c5f…
2 files changed
+5
-5
+58
-4
+5
-5
| --- Dockerfile | ||
| +++ Dockerfile | ||
| @@ -21,24 +21,24 @@ | ||
| 21 | 21 | && make -j |
| 22 | 22 | |
| 23 | 23 | # STAGE 2: Pare that back to the bare essentials. |
| 24 | 24 | |
| 25 | 25 | FROM scratch |
| 26 | -ENV JAIL=/jail | |
| 27 | -WORKDIR ${JAIL} | |
| 28 | -COPY --from=builder /tmp/fossil ${JAIL}/bin/ | |
| 26 | +WORKDIR /jail | |
| 27 | +COPY --from=builder /tmp/fossil /jail/bin/ | |
| 29 | 28 | COPY --from=builder /bin/busybox.static /bin/busybox |
| 30 | 29 | RUN [ "/bin/busybox", "--install", "/bin" ] |
| 31 | -RUN mkdir -m 700 dev \ | |
| 30 | +RUN mkdir -m 700 dev museum \ | |
| 32 | 31 | && mknod -m 600 dev/null c 1 3 \ |
| 33 | 32 | && mknod -m 600 dev/urandom c 1 9 |
| 34 | 33 | |
| 35 | 34 | # Now we can run the stripped-down environment in a chroot jail, while |
| 36 | 35 | # leaving open the option to debug it live via the Busybox shell. |
| 37 | 36 | |
| 38 | 37 | EXPOSE 8080/tcp |
| 39 | 38 | CMD [ \ |
| 40 | 39 | "bin/fossil", "server", \ |
| 40 | + "--chroot", "/jail", \ | |
| 41 | 41 | "--create", \ |
| 42 | 42 | "--jsmode", "bundled", \ |
| 43 | 43 | "--user", "admin", \ |
| 44 | - "repo.fossil"] | |
| 44 | + "museum/repo.fossil"] | |
| 45 | 45 |
| --- Dockerfile | |
| +++ Dockerfile | |
| @@ -21,24 +21,24 @@ | |
| 21 | && make -j |
| 22 | |
| 23 | # STAGE 2: Pare that back to the bare essentials. |
| 24 | |
| 25 | FROM scratch |
| 26 | ENV JAIL=/jail |
| 27 | WORKDIR ${JAIL} |
| 28 | COPY --from=builder /tmp/fossil ${JAIL}/bin/ |
| 29 | COPY --from=builder /bin/busybox.static /bin/busybox |
| 30 | RUN [ "/bin/busybox", "--install", "/bin" ] |
| 31 | RUN mkdir -m 700 dev \ |
| 32 | && mknod -m 600 dev/null c 1 3 \ |
| 33 | && mknod -m 600 dev/urandom c 1 9 |
| 34 | |
| 35 | # Now we can run the stripped-down environment in a chroot jail, while |
| 36 | # leaving open the option to debug it live via the Busybox shell. |
| 37 | |
| 38 | EXPOSE 8080/tcp |
| 39 | CMD [ \ |
| 40 | "bin/fossil", "server", \ |
| 41 | "--create", \ |
| 42 | "--jsmode", "bundled", \ |
| 43 | "--user", "admin", \ |
| 44 | "repo.fossil"] |
| 45 |
| --- Dockerfile | |
| +++ Dockerfile | |
| @@ -21,24 +21,24 @@ | |
| 21 | && make -j |
| 22 | |
| 23 | # STAGE 2: Pare that back to the bare essentials. |
| 24 | |
| 25 | FROM scratch |
| 26 | WORKDIR /jail |
| 27 | COPY --from=builder /tmp/fossil /jail/bin/ |
| 28 | COPY --from=builder /bin/busybox.static /bin/busybox |
| 29 | RUN [ "/bin/busybox", "--install", "/bin" ] |
| 30 | RUN mkdir -m 700 dev museum \ |
| 31 | && mknod -m 600 dev/null c 1 3 \ |
| 32 | && mknod -m 600 dev/urandom c 1 9 |
| 33 | |
| 34 | # Now we can run the stripped-down environment in a chroot jail, while |
| 35 | # leaving open the option to debug it live via the Busybox shell. |
| 36 | |
| 37 | EXPOSE 8080/tcp |
| 38 | CMD [ \ |
| 39 | "bin/fossil", "server", \ |
| 40 | "--chroot", "/jail", \ |
| 41 | "--create", \ |
| 42 | "--jsmode", "bundled", \ |
| 43 | "--user", "admin", \ |
| 44 | "museum/repo.fossil"] |
| 45 |
+58
-4
| --- www/build.wiki | ||
| +++ www/build.wiki | ||
| @@ -276,15 +276,66 @@ | ||
| 276 | 276 | |
| 277 | 277 | It builds tip-of-trunk. To get a release version instead, append |
| 278 | 278 | "<tt>?r=release</tt>" to the URL in the <tt>Dockerfile</tt>, then |
| 279 | 279 | (re)build it. |
| 280 | 280 | |
| 281 | -You may wish to direct Docker to copy an existing repo into the image at | |
| 282 | -build time, rather than let it create a blank one automatically. Simply | |
| 283 | -add this to the <tt>Dockerfile</tt> before the first "RUN" directive: | |
| 284 | 281 | |
| 285 | -<pre><code> COPY /local/path/to/my-project.fossil /jail/repo.fossil</code></pre> | |
| 282 | +<h3>5.1 Running It in Production</h3> | |
| 283 | + | |
| 284 | +If you want the container to serve an existing repository, there are at | |
| 285 | +least two right ways to do it. | |
| 286 | + | |
| 287 | +The wrong way is to use the <tt>Dockerfile COPY</tt> command to bake it | |
| 288 | +into the container at build time. It will become one of the container's | |
| 289 | +base layers, so that each time you restart the container, it gets reset | |
| 290 | +to its prior state. This almost certainly is not what you want. | |
| 291 | + | |
| 292 | +The simplest correct method is to stop the container if it was running, | |
| 293 | +then say: | |
| 294 | + | |
| 295 | +<pre><code> $ docker cp /path/to/repo.fossil fossil:/jail/museum/repo.fossil | |
| 296 | + $ docker start fossil | |
| 297 | + $ docker exec fossil chown 0 /jail/museum/repo.fossil</code></pre> | |
| 298 | + | |
| 299 | +That copies the local Fossil repo into the container where the server | |
| 300 | +expects to find it, so that the "start" command causes it to serve from | |
| 301 | +that copied-in file instead. Since it lives atop the base layers, it | |
| 302 | +persists as part of the container proper, surviving restarts. | |
| 303 | + | |
| 304 | +(The same is true of the default mode of operation: the <tt>fossil | |
| 305 | +server --create</tt> flag initializes a fresh Fossil repo atop the base | |
| 306 | +layer.) | |
| 307 | + | |
| 308 | +If you skip the "chown" command and put "http://localhost:9999/" into | |
| 309 | +your browser, expecting to see the copied-in repo's home page, you will | |
| 310 | +get an opaque "Not Found" error instead. This is because the user and | |
| 311 | +group ID of the file will be that of your local user on the container's | |
| 312 | +host machine, which won't map to anything in the container's | |
| 313 | +<tt>/etc/passwd</tt> and <tt>/etc/group</tt> files, effectively | |
| 314 | +preventing the server from reading the copied-in repository file. You | |
| 315 | +don't have to restart the server after fixing this: simply reload the | |
| 316 | +browser, and Fossil will try again. | |
| 317 | + | |
| 318 | +This simple method has a problem: Docker containers are designed to be | |
| 319 | +killed off at the slightest cause, rebuilt, and redeployed. If you do | |
| 320 | +that with the repo inside the container, it gets destroyed, too. The | |
| 321 | +solution is to replace the "run" command above with the following: | |
| 322 | + | |
| 323 | +<pre><code> $ docker create \ | |
| 324 | + --name fossil -p 9999:8080/tcp \ | |
| 325 | + -v museum:/jail/museum fossil | |
| 326 | +</code></pre> | |
| 327 | + | |
| 328 | +Now when you "docker cp" the local repo into the container, it lands on | |
| 329 | +a separate [https://docs.docker.com/storage/volumes/ | Docker volume] | |
| 330 | +mounted inside it, which has an independent lifetime. When you need to | |
| 331 | +rebuild the container — such as to upgrade to a newer version of Fossil | |
| 332 | +— the volume remains behind and gets remapped into the new contanier | |
| 333 | +when you recreate it by giving the above command again. | |
| 334 | + | |
| 335 | + | |
| 336 | +<h3>5.2 Why Chroot?</h3> | |
| 286 | 337 | |
| 287 | 338 | A potentially surprising feature of this container is that it runs |
| 288 | 339 | Fossil as root, which causes [./chroot.md | Fossil's chroot jail |
| 289 | 340 | feature] to kick in. Since a Docker container is a type of über-jail |
| 290 | 341 | already, you may be wondering why we don't either: |
| @@ -304,10 +355,13 @@ | ||
| 304 | 355 | |
| 305 | 356 | We deem this risk low since a) it's never happened, that we know of; |
| 306 | 357 | and b) we've turned off all of the risky features like TH1 docs. |
| 307 | 358 | Nevertheless, we believe in defense-in-depth. |
| 308 | 359 | |
| 360 | + | |
| 361 | +<h3>5.3 Extracting a Static Binary</h3> | |
| 362 | + | |
| 309 | 363 | Our 2-stage build process uses Alpine Linux only as a build host. Once |
| 310 | 364 | we've got everything reduced to the two key static binaries — Fossil and |
| 311 | 365 | Busybox — we throw all the rest of it away. |
| 312 | 366 | |
| 313 | 367 | A secondary benefit falls out of this process for free: it's arguably |
| 314 | 368 |
| --- www/build.wiki | |
| +++ www/build.wiki | |
| @@ -276,15 +276,66 @@ | |
| 276 | |
| 277 | It builds tip-of-trunk. To get a release version instead, append |
| 278 | "<tt>?r=release</tt>" to the URL in the <tt>Dockerfile</tt>, then |
| 279 | (re)build it. |
| 280 | |
| 281 | You may wish to direct Docker to copy an existing repo into the image at |
| 282 | build time, rather than let it create a blank one automatically. Simply |
| 283 | add this to the <tt>Dockerfile</tt> before the first "RUN" directive: |
| 284 | |
| 285 | <pre><code> COPY /local/path/to/my-project.fossil /jail/repo.fossil</code></pre> |
| 286 | |
| 287 | A potentially surprising feature of this container is that it runs |
| 288 | Fossil as root, which causes [./chroot.md | Fossil's chroot jail |
| 289 | feature] to kick in. Since a Docker container is a type of über-jail |
| 290 | already, you may be wondering why we don't either: |
| @@ -304,10 +355,13 @@ | |
| 304 | |
| 305 | We deem this risk low since a) it's never happened, that we know of; |
| 306 | and b) we've turned off all of the risky features like TH1 docs. |
| 307 | Nevertheless, we believe in defense-in-depth. |
| 308 | |
| 309 | Our 2-stage build process uses Alpine Linux only as a build host. Once |
| 310 | we've got everything reduced to the two key static binaries — Fossil and |
| 311 | Busybox — we throw all the rest of it away. |
| 312 | |
| 313 | A secondary benefit falls out of this process for free: it's arguably |
| 314 |
| --- www/build.wiki | |
| +++ www/build.wiki | |
| @@ -276,15 +276,66 @@ | |
| 276 | |
| 277 | It builds tip-of-trunk. To get a release version instead, append |
| 278 | "<tt>?r=release</tt>" to the URL in the <tt>Dockerfile</tt>, then |
| 279 | (re)build it. |
| 280 | |
| 281 | |
| 282 | <h3>5.1 Running It in Production</h3> |
| 283 | |
| 284 | If you want the container to serve an existing repository, there are at |
| 285 | least two right ways to do it. |
| 286 | |
| 287 | The wrong way is to use the <tt>Dockerfile COPY</tt> command to bake it |
| 288 | into the container at build time. It will become one of the container's |
| 289 | base layers, so that each time you restart the container, it gets reset |
| 290 | to its prior state. This almost certainly is not what you want. |
| 291 | |
| 292 | The simplest correct method is to stop the container if it was running, |
| 293 | then say: |
| 294 | |
| 295 | <pre><code> $ docker cp /path/to/repo.fossil fossil:/jail/museum/repo.fossil |
| 296 | $ docker start fossil |
| 297 | $ docker exec fossil chown 0 /jail/museum/repo.fossil</code></pre> |
| 298 | |
| 299 | That copies the local Fossil repo into the container where the server |
| 300 | expects to find it, so that the "start" command causes it to serve from |
| 301 | that copied-in file instead. Since it lives atop the base layers, it |
| 302 | persists as part of the container proper, surviving restarts. |
| 303 | |
| 304 | (The same is true of the default mode of operation: the <tt>fossil |
| 305 | server --create</tt> flag initializes a fresh Fossil repo atop the base |
| 306 | layer.) |
| 307 | |
| 308 | If you skip the "chown" command and put "http://localhost:9999/" into |
| 309 | your browser, expecting to see the copied-in repo's home page, you will |
| 310 | get an opaque "Not Found" error instead. This is because the user and |
| 311 | group ID of the file will be that of your local user on the container's |
| 312 | host machine, which won't map to anything in the container's |
| 313 | <tt>/etc/passwd</tt> and <tt>/etc/group</tt> files, effectively |
| 314 | preventing the server from reading the copied-in repository file. You |
| 315 | don't have to restart the server after fixing this: simply reload the |
| 316 | browser, and Fossil will try again. |
| 317 | |
| 318 | This simple method has a problem: Docker containers are designed to be |
| 319 | killed off at the slightest cause, rebuilt, and redeployed. If you do |
| 320 | that with the repo inside the container, it gets destroyed, too. The |
| 321 | solution is to replace the "run" command above with the following: |
| 322 | |
| 323 | <pre><code> $ docker create \ |
| 324 | --name fossil -p 9999:8080/tcp \ |
| 325 | -v museum:/jail/museum fossil |
| 326 | </code></pre> |
| 327 | |
| 328 | Now when you "docker cp" the local repo into the container, it lands on |
| 329 | a separate [https://docs.docker.com/storage/volumes/ | Docker volume] |
| 330 | mounted inside it, which has an independent lifetime. When you need to |
| 331 | rebuild the container — such as to upgrade to a newer version of Fossil |
| 332 | — the volume remains behind and gets remapped into the new contanier |
| 333 | when you recreate it by giving the above command again. |
| 334 | |
| 335 | |
| 336 | <h3>5.2 Why Chroot?</h3> |
| 337 | |
| 338 | A potentially surprising feature of this container is that it runs |
| 339 | Fossil as root, which causes [./chroot.md | Fossil's chroot jail |
| 340 | feature] to kick in. Since a Docker container is a type of über-jail |
| 341 | already, you may be wondering why we don't either: |
| @@ -304,10 +355,13 @@ | |
| 355 | |
| 356 | We deem this risk low since a) it's never happened, that we know of; |
| 357 | and b) we've turned off all of the risky features like TH1 docs. |
| 358 | Nevertheless, we believe in defense-in-depth. |
| 359 | |
| 360 | |
| 361 | <h3>5.3 Extracting a Static Binary</h3> |
| 362 | |
| 363 | Our 2-stage build process uses Alpine Linux only as a build host. Once |
| 364 | we've got everything reduced to the two key static binaries — Fossil and |
| 365 | Busybox — we throw all the rest of it away. |
| 366 | |
| 367 | A secondary benefit falls out of this process for free: it's arguably |
| 368 |