Fossil SCM
Replaced most of the speculation in the walmode section of the containers doc with a link to the walbanger project, where we'll be answering this question.
Commit
96633067d561ad4c5fec87cbee49425e1e3e92f28ad5ea05c8ff221c3f504ef1
Parent
558cef7c6d40932…
1 file changed
+8
-20
+8
-20
| --- www/containers.md | ||
| +++ www/containers.md | ||
| @@ -155,12 +155,12 @@ | ||
| 155 | 155 | --volume ~/museum/my-project.fossil:/jail/museum/repo.fossil |
| 156 | 156 | ``` |
| 157 | 157 | |
| 158 | 158 | That lets us have a convenient file name for the project outside the |
| 159 | 159 | container while letting the configuration inside the container refer to |
| 160 | -the generic “`/museum/repo.fossil`” name. Why should we have to rename | |
| 161 | -the container generically on the outside just to placate the container? | |
| 160 | +the generic “`/museum/repo.fossil`” name. Why should we have to name | |
| 161 | +the repo generically on the outside merely to placate the container? | |
| 162 | 162 | |
| 163 | 163 | The reason is, you might be serving that repo with [WAL mode][wal] |
| 164 | 164 | enabled. If you map the repo DB alone into the container, the Fossil |
| 165 | 165 | instance inside the container will write the `-journal` and `-wal` files |
| 166 | 166 | alongside the mapped-in repository inside the container. That’s fine as |
| @@ -174,28 +174,16 @@ | ||
| 174 | 174 | so there is at least a *hope* that WAL will work properly across that |
| 175 | 175 | boundary. The success of the scheme depends on the `mmap()` and shared |
| 176 | 176 | memory system calls being coordinated properly by the OS kernel the two |
| 177 | 177 | worlds share. |
| 178 | 178 | |
| 179 | -At some point, someone should perform tests in the hopes of *failing* to | |
| 180 | -create database corruption in this scenario. | |
| 181 | - | |
| 182 | -Why the tortured grammar? Because you cannot prove a negative, being in | |
| 183 | -this case “SQLite will not corrupt the database in WAL mode if there’s a | |
| 184 | -container barrier in the way.” All you can prove is that a given test | |
| 185 | -didn’t cause corruption. With enough tests of sufficient power, you can | |
| 186 | -begin to make definitive statements, but even then, science is always | |
| 187 | -provisional, awaiting a single disproving experiment. Atop that, OCI | |
| 188 | -container runtimes give the sysadmin freedom to impose barriers between | |
| 189 | -the two worlds, so even if you convince yourself that WAL mode is safe | |
| 190 | -in a given setup, it’s possible to configure it to fail. As if that | |
| 191 | -weren’t enough, different container runtimes have different defaults, | |
| 192 | -including details like whether shared memory is truly shared between | |
| 193 | -the host and its containers. | |
| 194 | - | |
| 195 | -Until someone gets around to establishing this ground truth and scoping | |
| 196 | -its applicable range, my advice to those who want to use WAL mode on | |
| 179 | +There is [a plan](https://tangentsoft.com/sqlite/dir/walbanger?ci=trunk) | |
| 180 | +for proving to a reasonable level of confidence that using WAL across a | |
| 181 | +container boundary is safe, but this effort is still in the early stages | |
| 182 | +as of this writing. | |
| 183 | + | |
| 184 | +Until that’s settled, my advice to those who want to use WAL mode on | |
| 197 | 185 | containerized servers is to map the whole directory as shown in these |
| 198 | 186 | examples, but then isolate the two sides with a secondary clone. On the |
| 199 | 187 | outside, you say something like this: |
| 200 | 188 | |
| 201 | 189 | ``` |
| 202 | 190 |
| --- www/containers.md | |
| +++ www/containers.md | |
| @@ -155,12 +155,12 @@ | |
| 155 | --volume ~/museum/my-project.fossil:/jail/museum/repo.fossil |
| 156 | ``` |
| 157 | |
| 158 | That lets us have a convenient file name for the project outside the |
| 159 | container while letting the configuration inside the container refer to |
| 160 | the generic “`/museum/repo.fossil`” name. Why should we have to rename |
| 161 | the container generically on the outside just to placate the container? |
| 162 | |
| 163 | The reason is, you might be serving that repo with [WAL mode][wal] |
| 164 | enabled. If you map the repo DB alone into the container, the Fossil |
| 165 | instance inside the container will write the `-journal` and `-wal` files |
| 166 | alongside the mapped-in repository inside the container. That’s fine as |
| @@ -174,28 +174,16 @@ | |
| 174 | so there is at least a *hope* that WAL will work properly across that |
| 175 | boundary. The success of the scheme depends on the `mmap()` and shared |
| 176 | memory system calls being coordinated properly by the OS kernel the two |
| 177 | worlds share. |
| 178 | |
| 179 | At some point, someone should perform tests in the hopes of *failing* to |
| 180 | create database corruption in this scenario. |
| 181 | |
| 182 | Why the tortured grammar? Because you cannot prove a negative, being in |
| 183 | this case “SQLite will not corrupt the database in WAL mode if there’s a |
| 184 | container barrier in the way.” All you can prove is that a given test |
| 185 | didn’t cause corruption. With enough tests of sufficient power, you can |
| 186 | begin to make definitive statements, but even then, science is always |
| 187 | provisional, awaiting a single disproving experiment. Atop that, OCI |
| 188 | container runtimes give the sysadmin freedom to impose barriers between |
| 189 | the two worlds, so even if you convince yourself that WAL mode is safe |
| 190 | in a given setup, it’s possible to configure it to fail. As if that |
| 191 | weren’t enough, different container runtimes have different defaults, |
| 192 | including details like whether shared memory is truly shared between |
| 193 | the host and its containers. |
| 194 | |
| 195 | Until someone gets around to establishing this ground truth and scoping |
| 196 | its applicable range, my advice to those who want to use WAL mode on |
| 197 | containerized servers is to map the whole directory as shown in these |
| 198 | examples, but then isolate the two sides with a secondary clone. On the |
| 199 | outside, you say something like this: |
| 200 | |
| 201 | ``` |
| 202 |
| --- www/containers.md | |
| +++ www/containers.md | |
| @@ -155,12 +155,12 @@ | |
| 155 | --volume ~/museum/my-project.fossil:/jail/museum/repo.fossil |
| 156 | ``` |
| 157 | |
| 158 | That lets us have a convenient file name for the project outside the |
| 159 | container while letting the configuration inside the container refer to |
| 160 | the generic “`/museum/repo.fossil`” name. Why should we have to name |
| 161 | the repo generically on the outside merely to placate the container? |
| 162 | |
| 163 | The reason is, you might be serving that repo with [WAL mode][wal] |
| 164 | enabled. If you map the repo DB alone into the container, the Fossil |
| 165 | instance inside the container will write the `-journal` and `-wal` files |
| 166 | alongside the mapped-in repository inside the container. That’s fine as |
| @@ -174,28 +174,16 @@ | |
| 174 | so there is at least a *hope* that WAL will work properly across that |
| 175 | boundary. The success of the scheme depends on the `mmap()` and shared |
| 176 | memory system calls being coordinated properly by the OS kernel the two |
| 177 | worlds share. |
| 178 | |
| 179 | There is [a plan](https://tangentsoft.com/sqlite/dir/walbanger?ci=trunk) |
| 180 | for proving to a reasonable level of confidence that using WAL across a |
| 181 | container boundary is safe, but this effort is still in the early stages |
| 182 | as of this writing. |
| 183 | |
| 184 | Until that’s settled, my advice to those who want to use WAL mode on |
| 185 | containerized servers is to map the whole directory as shown in these |
| 186 | examples, but then isolate the two sides with a secondary clone. On the |
| 187 | outside, you say something like this: |
| 188 | |
| 189 | ``` |
| 190 |