Fossil SCM

fossil-scm / www / backoffice.md
Source Blame History 225 lines
c6522a0… drh 1 Backoffice
c6522a0… drh 2 ==========
c6522a0… drh 3
c6522a0… drh 4 This is technical documentation about the internal workings of Fossil.
c6522a0… drh 5 Ordinary Fossil users do not need to know about anything covered by this
c6522a0… drh 6 document. The information here is intended for people who want to enhance
e2751b1… jan.nijtmans 7 or extend the Fossil code, or who just want a deeper understanding of
c6522a0… drh 8 the internal workings of Fossil.
c6522a0… drh 9
c6522a0… drh 10 What Is The Backoffice
c6522a0… drh 11 ----------------------
c6522a0… drh 12
e2751b1… jan.nijtmans 13 The backoffice is a mechanism used by a
f146e21… drh 14 [Fossil server](./server/) to do low-priority
c6522a0… drh 15 background work that is not directly related to the user interface. Here
c6522a0… drh 16 are some examples of the kinds of work that backoffice performs:
c6522a0… drh 17
c6522a0… drh 18 1. Sending email alerts and notifications
c6522a0… drh 19 2. Sending out daily digests of email notifications
c6522a0… drh 20 3. Other background email handling chores
c6522a0… drh 21 4. Automatic syncing of peer repositories
c6522a0… drh 22 5. Repository maintenance and optimization
c6522a0… drh 23
c6522a0… drh 24 (As of 2018-08-07, only items 1 and 2 have actually been implemented.)
c6522a0… drh 25 The idea is that the backoffice handles behind-the-scenes work that does
c6522a0… drh 26 not have tight latency requirements.
c6522a0… drh 27
c6522a0… drh 28 When Backoffice Runs
c6522a0… drh 29 --------------------
c6522a0… drh 30
c6522a0… drh 31 A backoffice process is usually launched automatically by a webpage
c6522a0… drh 32 request. After each webpage is generated, Fossil checks to see if any
c6522a0… drh 33 backoffice work needs to be done. If there is work to do, and no other
c6522a0… drh 34 process is already assigned to do the work, then a new backoffice process
c6522a0… drh 35 is started to do the work.
c6522a0… drh 36
c6522a0… drh 37 This happens for every webpage, regardless of how that webpage is launched,
c6522a0… drh 38 and regardless of the purpose of the webpage. This also happens on the
c64f28d… drh 39 server for "[fossil sync](/help/sync)" and
c500fa6… brickviking 40 "[fossil clone](/help/clone)" commands which are implemented as
c6522a0… drh 41 web requests - albeit requests that the human user never sees.
c6522a0… drh 42 Web requests can arrive at the Fossil server via direct TCP/IP (for example
c64f28d… drh 43 when Fossil is started using commands like "[fossil server](/help/server)")
f146e21… drh 44 or via [CGI](./server/any/cgi.md) or
f146e21… drh 45 [SCGI](./server/any/scgi.md) or via SSH.
c6522a0… drh 46 A backoffice process might be started regardless of the origin of the
c6522a0… drh 47 request.
c6522a0… drh 48
c6522a0… drh 49 The backoffice is not a daemon. Each backoffice process runs for a short
c6522a0… drh 50 while and then exits. This helps keep Fossil easy to manage, since there
c6522a0… drh 51 are no daemons to start and stop. To upgrade Fossil to a new version,
c6522a0… drh 52 you simply replace the older "fossil" executable with the newer one, and
c6522a0… drh 53 the backoffice processes will (within a minute or so) start using the new
e2751b1… jan.nijtmans 54 one. (Upgrading the executable on Windows is more complicated, since on
e2751b1… jan.nijtmans 55 Windows it is not possible to replace an executable file that is in active
c6522a0… drh 56 use. But Windows users probably already know this.)
c6522a0… drh 57
c6522a0… drh 58 The backoffice is serialized and rate limited. No more than a single
c6522a0… drh 59 backoffice process will be running at once, and backoffice runs will not
8a4b5c4… drh 60 occur more frequently than once every 60 seconds. (The 60-second spacing
8a4b5c4… drh 61 is controlled by the BKOFCE_LEASE_TIME macro in the
8a4b5c4… drh 62 [backoffice.c](/file/src/backoffice.c) source file.)
c6522a0… drh 63
c6522a0… drh 64 If a Fossil server is idle, then no backoffice processes will be running.
e2751b1… jan.nijtmans 65 That means there are no extra processes sitting around taking up memory
c6522a0… drh 66 and process table slots for seldom accessed repositories.
c6522a0… drh 67 The backoffice is an on-demand system.
c6522a0… drh 68 A busy repository will usually have a backoffice
c6522a0… drh 69 running at all times. But an infrequently accessed repository will only have
c6522a0… drh 70 backoffice processes running for a minute or two following the most recent
c6522a0… drh 71 access.
c6522a0… drh 72
c6522a0… drh 73 Manually Running The Backoffice
c6522a0… drh 74 -------------------------------
c6522a0… drh 75
c6522a0… drh 76 The automatic backoffice runs are sufficient for most installations.
c6522a0… drh 77 However, the daily digest of email notifications is handled by the
c6522a0… drh 78 backoffice. If a Fossil server can sometimes go more than a day without
c6522a0… drh 79 being accessed, then the automatic backoffice will never run, and the
c6522a0… drh 80 daily digest might not go out until somebody does visit a webpage.
79c2cb0… wyoung 81 If this is a problem, an administrator can set up a cron job to
c6522a0… drh 82 periodically run:
c6522a0… drh 83
8a1ba49… wyoung 84 fossil backoffice _REPOSITORY_
c6522a0… drh 85
c6522a0… drh 86 That command will cause backoffice processing to occur immediately.
c6522a0… drh 87 Note that this is almost never necessary for an internet-facing
c6522a0… drh 88 Fossil repository, since most repositories will get multiple accesses
c6522a0… drh 89 per day from random robots, which will be sufficient to kick off the
c6522a0… drh 90 daily digest emails. And even for a private server, if there is very
c6522a0… drh 91 little traffic, then the daily digests are probably a no-op anyhow
c6522a0… drh 92 and won't be missed.
c6522a0… drh 93
8a4b5c4… drh 94 Automatic Backoffice Does Not Work On Some Systems
8a4b5c4… drh 95 --------------------------------------------------
8a4b5c4… drh 96
8a4b5c4… drh 97 We have observed that the automatic backoffice does not work on
c500fa6… brickviking 98 some systems - OpenBSD in particular. We still do not understand why
8a4b5c4… drh 99 this is. (If you have insights, please share them on the
8a4b5c4… drh 100 [Fossil Forum](https://fossil-scm.org/forum) so that we can perhaps
8a4b5c4… drh 101 fix the problem.) For now, the backoffice must be run manually
8a4b5c4… drh 102 on OpenBSD systems.
8a4b5c4… drh 103
c500fa6… brickviking 104 To set up a fully-manual backoffice, first disable the automatic backoffice
c64f28d… drh 105 using the "[backoffice-disable](/help/backoffice-disable)" setting.
8a4b5c4… drh 106
8a1ba49… wyoung 107 fossil setting backoffice-disable on
8a4b5c4… drh 108
8a4b5c4… drh 109 Then arrange to invoke the backoffice separately using a command
8a4b5c4… drh 110 like this:
8a4b5c4… drh 111
8a1ba49… wyoung 112 fossil backoffice --poll 30 _REPOSITORY-LIST_
8a4b5c4… drh 113
8a4b5c4… drh 114 Multiple repositories can be named. This one command will handle
8a4b5c4… drh 115 launching the backoffice for all of them. There are additional useful
c64f28d… drh 116 command-line options. See the "[fossil backoffice](/help/backoffice)"
8a4b5c4… drh 117 documentation for details.
8a4b5c4… drh 118
c500fa6… brickviking 119 The backoffice processes that are run manually using the "fossil backoffice"
efd79f8… drh 120 command do not normally use a lease. That means that if you run the
8a4b5c4… drh 121 "fossil backoffice" command with --poll and you forget to disable
8a4b5c4… drh 122 automatic backoffice by setting the "backoffice-disable" flag, then
efd79f8… drh 123 you might have one backoffice running due to a command and another due
8a4b5c4… drh 124 to a webpage access, both at the same time. This is harmless. The
8a4b5c4… drh 125 only downside is that it uses extra CPU time.
8a4b5c4… drh 126
c6522a0… drh 127 How Backoffice Is Implemented
c6522a0… drh 128 -----------------------------
c6522a0… drh 129
8a4b5c4… drh 130 The backoffice is implemented by the
8a4b5c4… drh 131 "[backoffice.c](/file/src/backoffice.c)" source file.
c6522a0… drh 132
c6522a0… drh 133 Serialization and rate limiting is handled by a single entry in the
c6522a0… drh 134 repository database CONFIG table named "backoffice". This entry is
c6522a0… drh 135 called "the lease". The value of the lease
c6522a0… drh 136 is a text string representing four integers, which
c6522a0… drh 137 are respectively:
c6522a0… drh 138
c6522a0… drh 139 1. The process id of the "current" backoffice process
c6522a0… drh 140 2. The lease expiration time of the current backoffice process
c6522a0… drh 141 3. The process id of the "next" backoffice process
c6522a0… drh 142 4. The lease expiration time for the next backoffice process
c6522a0… drh 143
c6522a0… drh 144 Times are expressed in seconds since 1970. A process id of zero means
c6522a0… drh 145 "no process". Sometimes the process id will be non-zero even if there
c6522a0… drh 146 is no corresponding process. Fossil knows how to figure out whether or
c6522a0… drh 147 not a process still exists.
c6522a0… drh 148
c6522a0… drh 149 You can print out a decoded copy of the current backoffice lease using
c6522a0… drh 150 this command:
c6522a0… drh 151
8a1ba49… wyoung 152 fossil test-backoffice-lease -R _REPOSITORY_
c6522a0… drh 153
c6522a0… drh 154 If a system has been idle for a long time, then there will be no
c6522a0… drh 155 backoffice processes. (Either the process id entries in the lease
c6522a0… drh 156 will be zero, or there will exist no process associated with the
c6522a0… drh 157 process id.) When a new web request comes in, the system
c6522a0… drh 158 sees that no backoffice process is active and so it kicks off a separate
e2751b1… jan.nijtmans 159 process to run backoffice.
c6522a0… drh 160
e2751b1… jan.nijtmans 161 The new backoffice process becomes the "current" process. It sets a
c6522a0… drh 162 lease expiration time for itself to be 60 seconds in the future.
c6522a0… drh 163 Then it does the backoffice processing and exits. Note that usually
c6522a0… drh 164 the backoffice process will exit long before its lease expires. That
c6522a0… drh 165 is ok. The lease is there to limit the rate at which backoffice processes
c6522a0… drh 166 run.
c6522a0… drh 167
c6522a0… drh 168 If a new backoffice process starts up and sees that the "current" lease has
c6522a0… drh 169 yet to expire, the new process makes itself the "next" backoffice process
c6522a0… drh 170 and sets its expiration time to be 60 seconds past the expiration time of
c6522a0… drh 171 the "current" backoffice process. The "next" process then puts itself to
c6522a0… drh 172 sleep until the "current" lease expires. After the "current"
c6522a0… drh 173 lease expires and the "current" process has itself exited, then
c6522a0… drh 174 the "next" process promotes itself to the new "current" process. It
c6522a0… drh 175 sets the current lease expiration to be 60 seconds in the future, runs
c6522a0… drh 176 whatever backoffice work is needed, then exits.
c6522a0… drh 177
c6522a0… drh 178 If a new backoffice process starts up and finds that there is already
c6522a0… drh 179 a "current" lease and a "next" process, it exits without doing anything.
c6522a0… drh 180 This should happen only rarely, since the lease information is checked
c6522a0… drh 181 prior to spawning the backoffice process, so a conflict will only happen
c6522a0… drh 182 in a race.
c6522a0… drh 183
c6522a0… drh 184 Because the "backoffice" entry of the CONFIG table is in the repository
c6522a0… drh 185 database, access to the lease is serialized. The lease prevents more
c6522a0… drh 186 than one backoffice process from running at a time. It prevents
c6522a0… drh 187 backoffice processes from running more frequently than once every 60 seconds.
c6522a0… drh 188 And, it guarantees (assuming processes are not killed out-of-band) that
c6522a0… drh 189 every web request will be followed within 60 seconds by a backoffice
c6522a0… drh 190 run.
c6522a0… drh 191
c6522a0… drh 192 Debugging The Backoffice
c6522a0… drh 193 ------------------------
c6522a0… drh 194
c6522a0… drh 195 The backoffice should "just work". It should not require administrator
c6522a0… drh 196 attention. However, if you suspect that something is not working right,
c6522a0… drh 197 there are some debugging aids.
c6522a0… drh 198
c6522a0… drh 199 We have already mentioned the command that shows the backoffice lease
c6522a0… drh 200 for a repository:
c6522a0… drh 201
8a1ba49… wyoung 202 fossil test-backoffice-lease -R _REPOSITORY_
c6522a0… drh 203
c6522a0… drh 204 Running that command every few seconds should show what is going on with
c6522a0… drh 205 backoffice processing in a particular repository.
c6522a0… drh 206
2467a35… drh 207 There are also settings that control backoffice behavior. The
c6522a0… drh 208 "backoffice-nodelay" setting prevents the "next" process from taking a
c6522a0… drh 209 lease and sleeping. If "backoffice-nodelay" is set, that causes all
c6522a0… drh 210 backoffice processes to exit either immediately or after doing whatever
c6522a0… drh 211 backoffice works needs to be done. If something is going wrong and
c6522a0… drh 212 backoffice leases are causing delays in webpage processing, then setting
c6522a0… drh 213 "backoffice-nodelay" to true can work around the problem until the bug
c6522a0… drh 214 can be fixed. The "backoffice-logfile" setting is the name of a log
efd79f8… drh 215 file onto which is appended a short message every time a backoffice
c6522a0… drh 216 process actually starts to do the backoffice work. This log file can
c6522a0… drh 217 be used to verify that backoffice really is running, if there is any
2467a35… drh 218 doubt. The "backoffice-disable" setting prevents automatic backoffice
2467a35… drh 219 processing, if true. Use this to completely disable backoffice processing
2467a35… drh 220 that occurs automatically after each HTTP request. The "backoffice-disable"
e2751b1… jan.nijtmans 221 setting does not affect the operation of the manual
2467a35… drh 222 "fossil backoffice" command.
2467a35… drh 223 Most installations should leave "backoffice-nodelay" and "backoffice-disable"
2467a35… drh 224 set to their default values of off and
c6522a0… drh 225 leave "backoffice-logfile" unset or set to an empty string.

Keyboard Shortcuts

Open search /
Next entry (timeline) j
Previous entry (timeline) k
Open focused entry Enter
Show this help ?
Toggle theme Top nav button