Fossil SCM

fossil-scm / www / alerts.md
Source Blame History 727 lines
2ef70be… wyoung 1 # Email Alerts
2ef70be… wyoung 2
2ef70be… wyoung 3 ## Overview
2ef70be… wyoung 4
2ef70be… wyoung 5 Beginning with version 2.7, Fossil can send email messages to
2ef70be… wyoung 6 subscribers to alert them to changes in the repository:
2ef70be… wyoung 7
c64f28d… drh 8 * New [checkins](/help/ci)
2ef70be… wyoung 9 * [Ticket](./tickets.wiki) changes
2ef70be… wyoung 10 * [Wiki](./wikitheory.wiki) page changes
2ef70be… wyoung 11 * New and edited [forum](./forum.wiki) posts
d96055c… stephan 12 * Users receiving [new permissions](./caps/index.md) (admins only)
62411f3… drh 13 * Announcements
62411f3… drh 14
2ef70be… wyoung 15 Subscribers can elect to receive emails as soon as these events happen,
3e75a68… wyoung 16 or they can receive a daily digest of the events instead.
3e75a68… wyoung 17
f146e21… drh 18 Email alerts are sent by a [Fossil server](./server/), which must be
2ef70be… wyoung 19 [set up](#quick) by the Fossil administrator to send email.
2ef70be… wyoung 20
2ef70be… wyoung 21 Email alerts do not currently work if you are only using Fossil from the
3e75a68… wyoung 22 command line.
3e75a68… wyoung 23
2ef70be… wyoung 24 A bit of terminology: Fossil uses the terms "email alerts" and
2ef70be… wyoung 25 "notifications" interchangeably. We stick to the former term in this
2ef70be… wyoung 26 document except when referring to parts of the Fossil UI still using the
2ef70be… wyoung 27 latter term.
2ef70be… wyoung 28
2ef70be… wyoung 29
2ef70be… wyoung 30 ## Setup Prerequisites
2ef70be… wyoung 31
2ef70be… wyoung 32 Much of this document describes how to set up Fossil's email alert
2ef70be… wyoung 33 system. To follow this guide, you will need a Fossil UI browser window
2ef70be… wyoung 34 open to the [Admin → Notification](/setup_notification) Fossil UI screen
0996347… wyoung 35 on the Fossil server that will be sending these email alerts, logged
779ddef… wyoung 36 in as a user with [**Admin** capability](./caps/ref.html#a). It is not possible to work on a
2ef70be… wyoung 37 clone of the server's repository and push the configuration changes up
2ef70be… wyoung 38 to that repo as an Admin user, [on purpose](#backup).
2ef70be… wyoung 39
2ef70be… wyoung 40 <a id="cd"></a>
2ef70be… wyoung 41 You will also need a CLI window open with its working directory changed
2ef70be… wyoung 42 to a checkout directory of the Fossil repository you are setting up to
2ef70be… wyoung 43 send email. If you don't `cd` to such a checkout directory first,
2ef70be… wyoung 44 you'll need to add `-R /path/to/repo.fossil` to each `fossil` command
2ef70be… wyoung 45 below to tell Fossil which repository you mean it to apply the command
2ef70be… wyoung 46 to.
2ef70be… wyoung 47
2ef70be… wyoung 48 There are other prerequisites for email service, but since they vary
2ef70be… wyoung 49 depending on the configuration you choose, we'll cover these inline
2ef70be… wyoung 50 below.
2ef70be… wyoung 51
2ef70be… wyoung 52
2ef70be… wyoung 53 <a id="quick"></a>
2ef70be… wyoung 54 ## Quick Email Service Setup
2ef70be… wyoung 55
5209582… drh 56 If you've already got a working OpenSMTPD, Postfix, Exim, Sendmail,
5209582… drh 57 or similar server on the machine running your Fossil instance(s),
5209582… drh 58 and you aren't using Fossil's [chroot jail feature](./chroot.md)
5209582… drh 59 to wall Fossil off from the rest of the machine, it's
2ef70be… wyoung 60 fairly simple to set up email alerts.
2ef70be… wyoung 61
2ef70be… wyoung 62 (Otherwise, skip [ahead](#advanced) to the sections on advanced email
2ef70be… wyoung 63 service setup.)
2ef70be… wyoung 64
2ef70be… wyoung 65 This is our "quick setup" option even though setting up an SMTP mail
79c2cb0… wyoung 66 server is not trivial, because there are many other reasons to have such
2ef70be… wyoung 67 a server set up already: internal project email service, `cron`
2ef70be… wyoung 68 notifications, server status monitoring notifications...
2ef70be… wyoung 69
2ef70be… wyoung 70 With that out of the way, the Fossil-specific steps are easy:
2ef70be… wyoung 71
2ef70be… wyoung 72 1. Go to [Admin → Notification](/setup_notification) and fill out all
2ef70be… wyoung 73 of the **Required** fields:
2ef70be… wyoung 74
2ef70be… wyoung 75 * **Canonical server URL** — Use the suggested URL
2ef70be… wyoung 76
2ef70be… wyoung 77 * **"From" email address** — `[email protected]` is
2ef70be… wyoung 78 traditional, but suit yourself
2ef70be… wyoung 79
2ef70be… wyoung 80 * **Repository nickname** — See the suggested examples on the web page.
2ef70be… wyoung 81
2ef70be… wyoung 82 2. Set "Email Send Method" to "Pipe to a command"
2ef70be… wyoung 83
2ef70be… wyoung 84 3. Set the "Administrator email address" to a suitable valid email
2ef70be… wyoung 85 address on that machine. It could be the same value you used for
2ef70be… wyoung 86 the "From" address above, or it could be a different value like
2ef70be… wyoung 87 `[email protected]`.
2ef70be… wyoung 88
2ef70be… wyoung 89 Save your changes.
2ef70be… wyoung 90
2ef70be… wyoung 91 At the command line, say
2ef70be… wyoung 92
8a1ba49… wyoung 93 $ fossil set email-send-command
2ef70be… wyoung 94
2ef70be… wyoung 95 If that gives a blank value instead of `sendmail -ti`, say
2ef70be… wyoung 96
8a1ba49… wyoung 97 $ fossil set email-send-command "sendmail -ti"
2ef70be… wyoung 98
2ef70be… wyoung 99 to force the setting. That works around a [known
2ef70be… wyoung 100 bug](https://fossil-scm.org/forum/forumpost/840b676410) which may be
2ef70be… wyoung 101 squished by the time you read this.
2ef70be… wyoung 102
2ef70be… wyoung 103 If you're running Postfix or Exim, you might think that command is
2ef70be… wyoung 104 wrong, since you aren't running Sendmail. These mail servers provide a
2ef70be… wyoung 105 `sendmail` command for compatibility with software like Fossil that has
2ef70be… wyoung 106 no good reason to care exactly which SMTP server implementation is
2ef70be… wyoung 107 running at a given site. There may be other SMTP servers that also
2ef70be… wyoung 108 provide a compatible `sendmail` command, in which case they may work
2ef70be… wyoung 109 with Fossil using the same steps as above.
2ef70be… wyoung 110
2ef70be… wyoung 111 <a id="status"></a>
2ef70be… wyoung 112 If you reload the Admin → Notification page, the Status section at the
2ef70be… wyoung 113 top should show:
2ef70be… wyoung 114
8a1ba49… wyoung 115 Outgoing Email: Piped to command "sendmail -ti"
8a1ba49… wyoung 116 Pending Alerts: 0 normal, 0 digest
8a1ba49… wyoung 117 Subscribers: 0 active, 0 total
2ef70be… wyoung 118
2ef70be… wyoung 119 Before you move on to the next section, you might like to read up on
2ef70be… wyoung 120 [some subtleties](#pipe) with the "pipe to a command" method that we did
2ef70be… wyoung 121 not cover above.
2ef70be… wyoung 122
2ef70be… wyoung 123
2ef70be… wyoung 124 <a id="usage"></a>
2ef70be… wyoung 125 ## Usage and Testing
2ef70be… wyoung 126
2ef70be… wyoung 127 Now that email service from Fossil is set up, you can test it and begin
2ef70be… wyoung 128 using it.
2ef70be… wyoung 129
2ef70be… wyoung 130
2ef70be… wyoung 131 <a id="sub" name="subscribe"></a>
2ef70be… wyoung 132 ### Subscribing to Alerts
2ef70be… wyoung 133
2ef70be… wyoung 134 In the Status output above, we saw that there are no subscribers, so the
2ef70be… wyoung 135 next step is to add the first one.
2ef70be… wyoung 136
2ef70be… wyoung 137 Go to the `/subscribe` page on your Fossil instance to sign up for email
2ef70be… wyoung 138 alerts. At the very least, you will need to sign up for "Forum Posts"
2ef70be… wyoung 139 and "Announcements" to complete the testing steps below.
2ef70be… wyoung 140
2ef70be… wyoung 141 If you're logged in with a Fossil repository user account and put the
2ef70be… wyoung 142 same user name and email address into this forum as you used for your
2ef70be… wyoung 143 user information under Admin → Users, Fossil will simply tie your alert
2ef70be… wyoung 144 preferences to your login record, and the email address in your user's
2ef70be… wyoung 145 Contact Info field will be considered already-verified. Otherwise,
2ef70be… wyoung 146 Fossil will create an alert-only record, and you will have to verify the
2ef70be… wyoung 147 email address before Fossil will send alerts to it.
2ef70be… wyoung 148
2ef70be… wyoung 149 This shows a key aspect of the way Fossil's email alerts system works,
2ef70be… wyoung 150 by the way: a user can be signed up for email alerts without having a
2ef70be… wyoung 151 full-fledged Fossil user account. Only when both user names are the same
2ef70be… wyoung 152 are the two records tied together under the hood. For more on this, see
2ef70be… wyoung 153 [Users vs Subscribers below](#uvs).
2ef70be… wyoung 154
2ef70be… wyoung 155 If you are seeing the following complaint from Fossil:
2ef70be… wyoung 156
8a1ba49… wyoung 157 > Use a different login with greater privilege than FOO to access /subscribe
2ef70be… wyoung 158
779ddef… wyoung 159 ...then the repository's administrator forgot to give the
779ddef… wyoung 160 [**EmailAlert** capability][cap7]
2ef70be… wyoung 161 to that user or to a user category that the user is a member of.
2ef70be… wyoung 162
2ef70be… wyoung 163 After a subscriber signs up for alerts for the first time, a single
2ef70be… wyoung 164 verification email is sent to that subscriber's given email address.
2ef70be… wyoung 165 The new subscriber must click a link in that email in order to activate
2ef70be… wyoung 166 the subscription.
62411f3… drh 167
62411f3… drh 168 Subscription verification emails are only sent once. This is a defense
2ef70be… wyoung 169 against malicious robots that try to harass innocent Internet users by
2ef70be… wyoung 170 having subscription pages send multiple verification emails. If the
2ef70be… wyoung 171 initial subscription verification does not go through correctly, an
2ef70be… wyoung 172 administrator must [intervene](#admin) to reset the subscription.
2ef70be… wyoung 173
2ef70be… wyoung 174 Every subscriber-only email address has a [long random hexadecimal
2ef70be… wyoung 175 security code](#scode) that serves in place of a password. All email
2ef70be… wyoung 176 alerts contain a link in their footer back to the Fossil server,
2ef70be… wyoung 177 incorporating this security code, which allows the subscriber to adjust
2ef70be… wyoung 178 their subscription options. If a user doesn't have any of those emails,
2ef70be… wyoung 179 they can request a link via email by visiting the `/alerts` or
2ef70be… wyoung 180 `/unsubscribe` page on the repository.
2ef70be… wyoung 181
2ef70be… wyoung 182 Those with Fossil repository logins can adjust their email alert
2ef70be… wyoung 183 settings by visiting the `/alerts` page on the repository. With the
2ef70be… wyoung 184 default skin, you can get there by clicking the "Logout" link in the
2ef70be… wyoung 185 upper right corner of any Fossil UI page then clicking the "Email
2ef70be… wyoung 186 Alerts" link. That link is also available via the Sitemap (`/sitemap`)
2ef70be… wyoung 187 and via the default skin's hamburger menu (&#9776;).
2ef70be… wyoung 188
779ddef… wyoung 189 [cap7]: ./caps/ref.html#7
779ddef… wyoung 190
2ef70be… wyoung 191
2ef70be… wyoung 192 <a id="unsub" name="unsubscribe"></a>
2ef70be… wyoung 193 ### Unsubscribing
2ef70be… wyoung 194
2ef70be… wyoung 195 To unsubscribe from alerts, visit the `/alerts` page on the repository,
79c2cb0… wyoung 196 click the "Unsubscribe" button, then check the "Unsubscribe" checkbox to
2ef70be… wyoung 197 verify your action and press the "Unsubscribe" button a second time.
2ef70be… wyoung 198
2ef70be… wyoung 199 This interlock is intended to prevent accidental unsubscription.
2ef70be… wyoung 200
2ef70be… wyoung 201
2ef70be… wyoung 202 <a id="test"></a>
2ef70be… wyoung 203 ### Test Email Service
2ef70be… wyoung 204
2ef70be… wyoung 205 The easiest way to test email sending from Fossil is via the "[Send
2ef70be… wyoung 206 Announcement](/announce)" link at the top of the "Email Notification
2ef70be… wyoung 207 Setup" page. Put your email address in the "To:" line and a test
2ef70be… wyoung 208 message below, then press "Send Message" to verify that outgoing email
2ef70be… wyoung 209 is working.
2ef70be… wyoung 210
2ef70be… wyoung 211 Another method is from the command line:
2ef70be… wyoung 212
8a1ba49… wyoung 213 $ fossil alerts test-message [email protected] --body README.md --subject Test
2ef70be… wyoung 214
2ef70be… wyoung 215 That should send you an email with "Test" in the subject line and the
2ef70be… wyoung 216 contents of your project's `README.md` file in the body.
2ef70be… wyoung 217
2ef70be… wyoung 218 That command assumes that your project contains a "readme" file, but of
2ef70be… wyoung 219 course it does, because you have followed the [Programming Style Guide
2ef70be… wyoung 220 Checklist][cl], right? Right.
2ef70be… wyoung 221
2ef70be… wyoung 222 [cl]: https://sendgrid.com/blog/programming-style-guide-checklist/
2ef70be… wyoung 223
2ef70be… wyoung 224
779ddef… wyoung 225 <a id="cap7" name="ucap"></a>
2ef70be… wyoung 226 ### User Capabilities
2ef70be… wyoung 227
779ddef… wyoung 228 Once email alerts are working, you may need to [adjust the default user
779ddef… wyoung 229 capabilities](./caps/) to give "[Email Alerts][cap7]" capability to any
779ddef… wyoung 230 [user category](./caps/#ucat) or [individual user](./caps/#ucap) that
779ddef… wyoung 231 needs to use the subscription setup pages, `/subscribe` and `/alerts`.
779ddef… wyoung 232 [**Admin**][capa] and [**Setup**][caps] users always have this
779ddef… wyoung 233 capability.
2ef70be… wyoung 234
2ef70be… wyoung 235 To allow any passer-by on the Internet to subscribe, give the "Email
2ef70be… wyoung 236 Alerts" capability to the "nobody" user category. To require that a
2ef70be… wyoung 237 person solve a simple CAPTCHA first, give that capability to the
2ef70be… wyoung 238 "anonymous" user category instead.
779ddef… wyoung 239
779ddef… wyoung 240 [capa]: ./caps/ref.html#a
779ddef… wyoung 241 [caps]: ./caps/ref.html#s
2ef70be… wyoung 242
2ef70be… wyoung 243
2ef70be… wyoung 244 <a id="first" name="frist"></a>
2ef70be… wyoung 245 ### First Post
2ef70be… wyoung 246
2ef70be… wyoung 247 I suggest taking the time to compose a suitable introductory message
2ef70be… wyoung 248 especially for your project's forum, one which a new user would find
2ef70be… wyoung 249 helpful.
2ef70be… wyoung 250
2ef70be… wyoung 251 Wait a few seconds, and you should receive an email alert with the
2ef70be… wyoung 252 post's subject and body text in the email.
2ef70be… wyoung 253
2ef70be… wyoung 254
2ef70be… wyoung 255 <a id="trouble"></a>
2ef70be… wyoung 256 ### Troubleshooting
2ef70be… wyoung 257
2ef70be… wyoung 258 If email alerts aren't working, there are several useful commands you
2ef70be… wyoung 259 can give to figure out why.
2ef70be… wyoung 260
2ef70be… wyoung 261 (Be sure to [`cd` into a repo checkout directory](#cd) first!)
2ef70be… wyoung 262
8a1ba49… wyoung 263 $ fossil alerts status
2ef70be… wyoung 264
2ef70be… wyoung 265 This should give much the same information as you saw [above](#status).
2ef70be… wyoung 266 One difference is that, since you've created a forum post, the
2ef70be… wyoung 267 `pending-alerts` value should only be zero if you did in fact get the
2ef70be… wyoung 268 requested email alert. If it's zero, check your mailer's spam folder. If
2ef70be… wyoung 269 it's nonzero, continue with these troubleshooting steps.
2ef70be… wyoung 270
8a1ba49… wyoung 271 $ fossil backoffice
2ef70be… wyoung 272
2ef70be… wyoung 273 That forces Fossil to run its ["back office" process](./backoffice.md).
2ef70be… wyoung 274 Its only purpose at the time of this writing is to push out alert
2ef70be… wyoung 275 emails, but it might do other things later. Sometimes it can get stuck
2ef70be… wyoung 276 and needs to be kicked. For that reason, you might want to set up a
2ef70be… wyoung 277 crontab entry to make sure it runs occasionally.
2ef70be… wyoung 278
8a1ba49… wyoung 279 $ fossil alerts send
2ef70be… wyoung 280
2ef70be… wyoung 281 This should also kick off the backoffice processing, if there are any
2ef70be… wyoung 282 pending alerts to send out.
2ef70be… wyoung 283
8a1ba49… wyoung 284 $ fossil alert pending
2ef70be… wyoung 285
2ef70be… wyoung 286 Show any pending alerts. The number of lines output here should equal
2ef70be… wyoung 287 the [status output above](#status).
2ef70be… wyoung 288
8a1ba49… wyoung 289 $ fossil test-add-alerts f5900
8a1ba49… wyoung 290 $ fossil alert send
2ef70be… wyoung 291
2ef70be… wyoung 292 Manually create an email alert and push it out immediately.
2ef70be… wyoung 293
2ef70be… wyoung 294 The `f` in the first command's final parameter means you're scheduling a
2ef70be… wyoung 295 "forum" alert. The integer is the ID of a forum post, which you can find
2ef70be… wyoung 296 by visiting `/timeline?showid` on your Fossil instance.
2ef70be… wyoung 297
2ef70be… wyoung 298 The second command above is necessary because the `test-add-alerts`
2ef70be… wyoung 299 command doesn't kick off a backoffice run.
2ef70be… wyoung 300
8a1ba49… wyoung 301 $ fossil ale send
2ef70be… wyoung 302
2ef70be… wyoung 303 This only does the same thing as the final command above, rather than
2ef70be… wyoung 304 send you an ale, as you might be hoping. Sorry.
2ef70be… wyoung 305
2ef70be… wyoung 306
2ef70be… wyoung 307 <a id="advanced"></a>
2ef70be… wyoung 308 ## Advanced Email Setups
2ef70be… wyoung 309
2ef70be… wyoung 310 Fossil offers several methods of sending email:
2ef70be… wyoung 311
2ef70be… wyoung 312 1. Pipe the email message text into a command.
c0654b1… brickviking 313 2. Store email messages as entries in an SQLite database.
2ef70be… wyoung 314 3. Store email messages as individual files in a directory.
2ef70be… wyoung 315 4. Send emails to an SMTP relay.
2ef70be… wyoung 316 5. Send emails directly to the recipients via SMTP.
2ef70be… wyoung 317
2ef70be… wyoung 318 This wide range of options allows Fossil to talk to pretty much any
2ef70be… wyoung 319 SMTP setup.
2ef70be… wyoung 320
2ef70be… wyoung 321 The first four options let Fossil delegate email handling to an existing
897bd90… wyoung 322 [MTA] so that Fossil does not need to implement the [roughly two
2ef70be… wyoung 323 dozen][mprotos] separate [RFCs][rfcs] required in order to properly
2ef70be… wyoung 324 support SMTP email in this complex world we've built. As well, this
2ef70be… wyoung 325 design choice means you do not need to do duplicate configuration, such
2ef70be… wyoung 326 as to point Fossil at your server's TLS certificate in order to support
2ef70be… wyoung 327 users behind mail servers that require STARTTLS encryption.
2ef70be… wyoung 328
2ef70be… wyoung 329 [mprotos]: http://sqlite.1065341.n5.nabble.com/Many-ML-emails-going-to-GMail-s-SPAM-tp98685p98722.html
2ef70be… wyoung 330 [rfcs]: https://en.wikipedia.org/wiki/Request_for_Comments
2ef70be… wyoung 331
2ef70be… wyoung 332
2ef70be… wyoung 333 <a id="pipe"></a>
2ef70be… wyoung 334 ### Method 1: Pipe to a Command
2ef70be… wyoung 335
2ef70be… wyoung 336 This is our ["quick setup" option](#quick) above, but there are some
2ef70be… wyoung 337 details we ignored which we'll cover now.
2ef70be… wyoung 338
2ef70be… wyoung 339 Fossil pipes the email message in [RFC 822 format][rfc822] to the
2ef70be… wyoung 340 standard input of the command you gave as the "Email Send Method",
2ef70be… wyoung 341 defaulting to `sendmail -ti`. This constitutes a protocol between Fossil
897bd90… wyoung 342 and the SMTP [message transfer agent (MTA)][MTA]. Any other MTA which
2ef70be… wyoung 343 speaks the same protocol can be used in place of the most common
2ef70be… wyoung 344 options: Sendmail, Exim, and Postfix.
2ef70be… wyoung 345
2ef70be… wyoung 346 The `-t` option tells the command to expect the list of email recipients
2ef70be… wyoung 347 in a `To` header in the RFC 822 message presented on its standard input.
2ef70be… wyoung 348 Without this option, the `sendmail` command expects to receive the
2ef70be… wyoung 349 recipient list on the command line, but that's not possible with the
2ef70be… wyoung 350 current design of this email sending method. Therefore, if you're
2ef70be… wyoung 351 attempting to use a less common MTA which cannot parse the recipient
2ef70be… wyoung 352 list from the `To` header in the email message, you might need to look
2ef70be… wyoung 353 for a different MTA.
2ef70be… wyoung 354
2ef70be… wyoung 355 The `-i` option is only needed for MTAs that take a dot/period at the
2ef70be… wyoung 356 beginning of a line of standard input text as "end of message." Fossil
2ef70be… wyoung 357 doesn't attempt to escape such dots, so if the line wrapping happens to
2ef70be… wyoung 358 occur such that a dot or period in an alert message is at the beginning
2ef70be… wyoung 359 of a line, you'll get a truncated email message without this option.
2ef70be… wyoung 360 Statistically, this will happen about once every 70 or so messages, so
2ef70be… wyoung 361 it is important to give this option if your MTA treats leading dots on a
2ef70be… wyoung 362 line this way.
2ef70be… wyoung 363
2ef70be… wyoung 364 <a id="msmtp"></a>
ea74c83… wyoung 365 The [`msmtp`][msmtp] SMTP client is compatible with this
ea74c83… wyoung 366 protocol if you give it the `-t` option. It’s a useful option on a server
2ef70be… wyoung 367 hosting a Fossil repository which doesn't otherwise require a separate
ea74c83… wyoung 368 SMTP server for other purposes, such as because you’ve got a separate
ea74c83… wyoung 369 provider for your email and merely need a way to let Fossil feed
ea74c83… wyoung 370 messages into it.
2ef70be… wyoung 371
2ef70be… wyoung 372 It is probably also possible to configure [`procmail`][pmdoc] to work
2ef70be… wyoung 373 with this protocol. If you know how to do it, a patch to this document
2ef70be… wyoung 374 or a how-to on [the Fossil forum][ff] would be appreciated.
2ef70be… wyoung 375
2ef70be… wyoung 376 [ff]: https://fossil-scm.org/forum/
2ef70be… wyoung 377 [msmtp]: https://marlam.de/msmtp/
897bd90… wyoung 378 [MTA]: https://en.wikipedia.org/wiki/Message_transfer_agent
2ef70be… wyoung 379 [pmdoc]: http://pm-doc.sourceforge.net/doc/
2ef70be… wyoung 380 [rfc822]: https://www.w3.org/Protocols/rfc822/
2ef70be… wyoung 381
2ef70be… wyoung 382
2ef70be… wyoung 383 <a id="db"></a>
2ef70be… wyoung 384 ### Method 2: Store in a Database
2ef70be… wyoung 385
09908ab… drh 386 The self-hosting Fossil repository at <https://fossil-scm.org/>
2ef70be… wyoung 387 currently uses this method rather than [the pipe method](#pipe) because
2ef70be… wyoung 388 it is running inside of a restrictive [chroot jail][cj] which is unable
2ef70be… wyoung 389 to hand off messages to the local MTA directly.
2ef70be… wyoung 390
2ef70be… wyoung 391 When you configure a Fossil server this way, it adds outgoing email
c0654b1… brickviking 392 messages to an SQLite database file. A separate daemon process can then
2ef70be… wyoung 393 extract those messages for further disposition.
2ef70be… wyoung 394
4163ef0… drh 395 Fossil uses a short TCL script (seen at [](/file/tools/email-sender.tcl))
4163ef0… drh 396 that continuously monitors this database for new messages and hands any that
2ef70be… wyoung 397 it finds off to a local MTA using the same [pipe to MTA protocol](#pipe)
2ef70be… wyoung 398 as above.
2ef70be… wyoung 399
2ef70be… wyoung 400 In this way, outbound email alerts escape the chroot jail without
2ef70be… wyoung 401 requiring that we insert a separate MTA configuration inside that jail.
2ef70be… wyoung 402 We only need to arrange that the same SQLite DB file be visible both
2ef70be… wyoung 403 inside and outside the chroot jail, which we do by naming the database
2ef70be… wyoung 404 file in the "Store Emails In This Database" setting under Admin →
2ef70be… wyoung 405 Notification. The Tcl script has this path hard-coded as
2ef70be… wyoung 406 `/home/www/fossil/emailqueue.db`, but you will probably need to adjust
2ef70be… wyoung 407 that for your local purposes.
2ef70be… wyoung 408
2ef70be… wyoung 409 This method may work with other similar technologies besides `chroot`:
2ef70be… wyoung 410 Docker containers, LXC containers, BSD jails, Solaris zones, etc.
2ef70be… wyoung 411
2ef70be… wyoung 412 With suitable file share mappings, this method may even work with
2ef70be… wyoung 413 virtual machine or distributed computing setups where the MTA and Fossil
2ef70be… wyoung 414 servers are not on the same machine, though beware the [risk of DB
2ef70be… wyoung 415 corruption][rdbc] if used with a file sharing technology that doesn't
2ef70be… wyoung 416 use proper file locking.
2ef70be… wyoung 417
2ef70be… wyoung 418 You can start this Tcl script as a daemon automatically on most Unix and
2ef70be… wyoung 419 Unix-like systems by adding the following line to the `/etc/rc.local`
2ef70be… wyoung 420 file of the server that hosts the repository sending email alerts:
2ef70be… wyoung 421
8a1ba49… wyoung 422 /usr/bin/tclsh /home/www/fossil/email-sender.tcl &
2ef70be… wyoung 423
2ef70be… wyoung 424 [cj]: https://en.wikipedia.org/wiki/Chroot
2ef70be… wyoung 425 [rdbc]: https://www.sqlite.org/howtocorrupt.html#_filesystems_with_broken_or_missing_lock_implementations
2ef70be… wyoung 426
2ef70be… wyoung 427
2ef70be… wyoung 428 <a id="dir"></a>
2ef70be… wyoung 429 ### Method 3: Store in a Directory
2ef70be… wyoung 430
2ef70be… wyoung 431 This method is functionally very similar to [the DB method](#db),
2ef70be… wyoung 432 differing only in that messages are written to a directory in the
2ef70be… wyoung 433 filesystem. You should therefore read that section and make the minor
2ef70be… wyoung 434 adjustments required by the storage method.
2ef70be… wyoung 435
2ef70be… wyoung 436 This method may work over a file sharing mechanism that doesn't do file
2ef70be… wyoung 437 locking properly, as long as the reading process is somehow restricted
2ef70be… wyoung 438 from reading a message file as it's being written.
2ef70be… wyoung 439
2ef70be… wyoung 440 It might be useful in testing and debugging to temporarily switch to
2ef70be… wyoung 441 this method, since you can easily read the generated email messages
897bd90… wyoung 442 without needing to involve an [MTA].
2ef70be… wyoung 443
2ef70be… wyoung 444
2ef70be… wyoung 445 <a id="relay"></a>
2ef70be… wyoung 446 ### Method 4: SMTP Relay
2ef70be… wyoung 447
2ef70be… wyoung 448 In this configuration, the Fossil server contacts an open SMTP relay and
2ef70be… wyoung 449 sends the messages to it. This method is only appropriate when:
2ef70be… wyoung 450
2ef70be… wyoung 451 1. You have a local MTA that doesn't accept [the pipe
2ef70be… wyoung 452 protocol](#pipe).
2ef70be… wyoung 453
2ef70be… wyoung 454 2. The MTA is willing to accept anonymous submissions, since Fossil
2ef70be… wyoung 455 currently has no way to authenticate itself to the MTA. This is [an
2ef70be… wyoung 456 unsafe configuration][omr] in most cases, but some SMTP servers make
2ef70be… wyoung 457 an exception for connections coming from a `localhost` or LAN
2ef70be… wyoung 458 address, choosing to accept such submissions as inherently safe.
2ef70be… wyoung 459
2ef70be… wyoung 460 If you have a local MTA meeting criterion #1 but not #2, we'd suggest
2ef70be… wyoung 461 using a more powerful SMTP client such as [msmtp](#msmtp) along with one
2ef70be… wyoung 462 of the other methods above.
2ef70be… wyoung 463
2ef70be… wyoung 464 [omr]: https://en.wikipedia.org/wiki/Open_mail_relay
2ef70be… wyoung 465
2ef70be… wyoung 466
2ef70be… wyoung 467 <a id="direct"></a>
2ef70be… wyoung 468 ### Method 5: Direct SMTP Send
2ef70be… wyoung 469
2ef70be… wyoung 470 As of Fossil 2.7, the code to support this method is incomplete, so you
2ef70be… wyoung 471 cannot currently select it as an option in Admin → Notification.
2ef70be… wyoung 472
2ef70be… wyoung 473
2ef70be… wyoung 474 <a id="uvs"></a>
2ef70be… wyoung 475 ## Users vs Subscribers
2ef70be… wyoung 476
2ef70be… wyoung 477 Fossil makes a distinction between "users" and "subscribers". A user is
2ef70be… wyoung 478 someone with a username and password: that is, someone who can log into
91e8c12… wyoung 479 the Fossil repository. A subscriber is someone who receives email
2ef70be… wyoung 480 alerts. Users can also be subscribers and subscribers can be users, but
2ef70be… wyoung 481 that does not have to be the case. It is possible to be a user without
2ef70be… wyoung 482 being a subscriber and to be a subscriber without being a user.
2ef70be… wyoung 483
2ef70be… wyoung 484 In the repository database file, users are tracked with the `user` table
2ef70be… wyoung 485 and subscribers are tracked via the `subscriber` table.
2ef70be… wyoung 486
2ef70be… wyoung 487
2ef70be… wyoung 488 <a id="admin"></a>
2ef70be… wyoung 489 ## Administrator Activities
2ef70be… wyoung 490
2ef70be… wyoung 491 The "[List Subscribers](/subscribers)" button at the top of the Admin →
2ef70be… wyoung 492 Notification screen gives a list of subscribers, which gives a Fossil
2ef70be… wyoung 493 server administrator a lot of power over those subscriptions.
2ef70be… wyoung 494
2ef70be… wyoung 495 Clicking an email address in this subscriber list opens the same
2ef70be… wyoung 496 `/alerts` page that the user can see for their own subscription, but
2ef70be… wyoung 497 with more information and functionality than normal users get:
2ef70be… wyoung 498
2ef70be… wyoung 499 * Subscription creation and modification timestamps.
2ef70be… wyoung 500
2ef70be… wyoung 501 * The IP address the user had when they last made a change via either
2ef70be… wyoung 502 `/subscribe` or `/alert`.
2ef70be… wyoung 503
2ef70be… wyoung 504 * The user's login name, if they are not [a mere subscriber](#uvs). A
2ef70be… wyoung 505 Fossil Admin user is allowed to modify this, either to tie a
2ef70be… wyoung 506 subscription-only record to an existing Fossil user account or to
2ef70be… wyoung 507 break that tie.
2ef70be… wyoung 508
2ef70be… wyoung 509 * The "Do not call" checkbox allows a Fossil Admin user to mark a given
2ef70be… wyoung 510 email address so that Fossil never sends email to that address. This
2ef70be… wyoung 511 is distinct from unsubscribing that email address because it prevents
2ef70be… wyoung 512 Fossil from accepting a new subscription for that address.
2ef70be… wyoung 513
2ef70be… wyoung 514 * The Verified checkbox is initially unchecked for subscriber-only
2ef70be… wyoung 515 email addresses until the user clicks the link in the verification
2ef70be… wyoung 516 email. This checkbox lets the Fossil Admin user manually verify the
2ef70be… wyoung 517 user, such as in the case where the verification email message got
2ef70be… wyoung 518 lost. Unchecking this box does not cause another verification email
2ef70be… wyoung 519 to be sent.
2ef70be… wyoung 520
d96055c… stephan 521 * Admin users (only) may activate the "user elevation" subscription,
d96055c… stephan 522 which sends a notification when a user is created or is explicitly
d96055c… stephan 523 assigned permission they did not formerly have.
d96055c… stephan 524
2ef70be… wyoung 525 This screen also allows a Fossil Admin user to perform other activities
2ef70be… wyoung 526 on behalf of a subscriber which they could do themselves, such as to
2ef70be… wyoung 527 [unsubscribe](#unsub) them.
2ef70be… wyoung 528
2ef70be… wyoung 529
2ef70be… wyoung 530 <a id="backup"></a>
2ef70be… wyoung 531 ## Cloning, Syncing, and Backups
2ef70be… wyoung 532
2b1c5bc… wyoung 533 That’s [covered elsewhere](./backup.md#alerts).
2ef70be… wyoung 534
2ef70be… wyoung 535
2ef70be… wyoung 536 <a id="pages" name="commands"></a>
2ef70be… wyoung 537 ## Controlling the Email Alert System
2ef70be… wyoung 538
2ef70be… wyoung 539 This section collects the list of Fossil UI pages and CLI commands that
2ef70be… wyoung 540 control the email alert system, some of which have not been mentioned so
2ef70be… wyoung 541 far:
2ef70be… wyoung 542
2ef70be… wyoung 543 Commands:
2ef70be… wyoung 544
c64f28d… drh 545 * The [`alerts`](/help/alerts) command
c64f28d… drh 546 * The [`test-alert`](/help/test-alert) command
c64f28d… drh 547 * The [`test-add-alerts`](/help/test-add-alerts) command
2ef70be… wyoung 548
2ef70be… wyoung 549 Web pages available to users and subscribers:
2ef70be… wyoung 550
c64f28d… drh 551 * The [`/subscribe`](/help/www/subscribe) page
c64f28d… drh 552 * The [`/alerts`](/help/www/alerts) page
c64f28d… drh 553 * The [`/unsubscribe`](/help/www/unsubscribe) page
c64f28d… drh 554 * The [`/renew`](/help/www/renew) page
c64f28d… drh 555 * The [`/contact_admin`](/help/www/contact_admin) page
2ef70be… wyoung 556
2ef70be… wyoung 557 Administrator-only web pages:
2ef70be… wyoung 558
c64f28d… drh 559 * The [`/setup_notification`](/help/www/setup_notification) page
c64f28d… drh 560 * The [`/subscribers`](/help/www/subscribers) page
2ef70be… wyoung 561
2ef70be… wyoung 562
2ef70be… wyoung 563 <a id="design"></a>
2ef70be… wyoung 564 ## Design of Email Alerts
2ef70be… wyoung 565
2ef70be… wyoung 566 This section describes the low-level design of the email alert system in
79c2cb0… wyoung 567 Fossil. This expands on the high-level administration focused material
2ef70be… wyoung 568 above with minimal repetition.
2ef70be… wyoung 569
2ef70be… wyoung 570 This section assumes expert-level systems knowledge. If the material
2ef70be… wyoung 571 above sufficed for your purposes, feel free to skip this section, which
2ef70be… wyoung 572 runs to the end of this document.
2ef70be… wyoung 573
2ef70be… wyoung 574
2ef70be… wyoung 575 <a id="datades"></a>
2ef70be… wyoung 576 ### Data Design
2ef70be… wyoung 577
34d45c5… drh 578 There are two new tables in the repository database, starting with
2ef70be… wyoung 579 Fossil 2.7. These tables are not created in new repositories by
2ef70be… wyoung 580 default. The tables only come into existence as needed when email
2ef70be… wyoung 581 alerts are configured and used.
2ef70be… wyoung 582
2ef70be… wyoung 583 * <b>SUBSCRIBER</b> →
2ef70be… wyoung 584 The subscriber table records the email address for people who
2ef70be… wyoung 585 want to receive email notifications. Each subscriber has a
2ef70be… wyoung 586 `subscriberCode` which is a random 32-byte blob that uniquely
2ef70be… wyoung 587 identifies the subscriber. There are also fields to indicate
2ef70be… wyoung 588 what kinds of notifications the subscriber wishes to receive,
2ef70be… wyoung 589 whether or not the email address of the subscriber has been
2ef70be… wyoung 590 verified, etc.
2ef70be… wyoung 591
2ef70be… wyoung 592 * <b>PENDING\_ALERT</b> →
2ef70be… wyoung 593 The PENDING\_ALERT table contains records that define events
2ef70be… wyoung 594 about which alert emails might need to be sent.
2ef70be… wyoung 595 A pending\_alert always refers to an entry in the
2ef70be… wyoung 596 EVENT table. The EVENT table is part of the standard schema
2ef70be… wyoung 597 and records timeline entries. In other words, there is one
2ef70be… wyoung 598 row in the EVENT table for each possible timeline entry. The
2ef70be… wyoung 599 PENDING\_ALERT table refers to EVENT table entries for which
2ef70be… wyoung 600 we might need to send alert emails.
2ef70be… wyoung 601
2ef70be… wyoung 602 As pointed out above, ["subscribers" are distinct from "users"](#uvs).
2ef70be… wyoung 603 The SUBSCRIBER.SUNAME field is the optional linkage between users and
2ef70be… wyoung 604 subscribers.
2ef70be… wyoung 605
2ef70be… wyoung 606
2ef70be… wyoung 607 <a id="stdout"></a>
2ef70be… wyoung 608 ### The "stdout" Method
2ef70be… wyoung 609
2ef70be… wyoung 610 The [list of mail sending methods](#advanced) above left out an
2ef70be… wyoung 611 internal-only method called "stdout" which simply writes the text of the
2ef70be… wyoung 612 email message on standard output. The "stdout" method is used for
2ef70be… wyoung 613 testing and debugging. If you need something similar and can't modify
2ef70be… wyoung 614 your local Fossil instance to use this method, you might temporarily
2ef70be… wyoung 615 switch to [the "dir" method](#dir) instead.
2ef70be… wyoung 616
2ef70be… wyoung 617
2ef70be… wyoung 618 <a id="msgfmt"></a>
2ef70be… wyoung 619 ### Message Format
2ef70be… wyoung 620
2ef70be… wyoung 621 The email messages generated by Fossil have a [well-formed
2ef70be… wyoung 622 header][rfc822]. The downstream processing is expected to extract the
2ef70be… wyoung 623 "To:", "From:", "Subject:" and whatever other attributes it needs from
2ef70be… wyoung 624 the email header text.
2ef70be… wyoung 625
2ef70be… wyoung 626 These emails use the `text/plain` MIME type with the UTF-8 character
2ef70be… wyoung 627 set. We currently use a transfer encoding of `quoted-printable`, but
2ef70be… wyoung 628 there is commented-out code in Fossil to switch to `base64` encoding,
2ef70be… wyoung 629 which Fossil used in the early days leading up to the 2.7 release.
2ef70be… wyoung 630
2ef70be… wyoung 631 If you switch Fossil back to `base64` mode, you may want to build a
2ef70be… wyoung 632 utility program that ships in the Fossil source tree named
2ef70be… wyoung 633 ["tools/decode-email.c"](/file/tools/decode-email.c) which can decode
2ef70be… wyoung 634 these messages into a human-readable format.
2ef70be… wyoung 635
2ef70be… wyoung 636
2ef70be… wyoung 637 <a id="inbound" name="bounces"></a>
2ef70be… wyoung 638 ### Dealing with Inbound Email
2ef70be… wyoung 639
2ef70be… wyoung 640 Inbound email messages — for example, bounces from failed alert emails —
2ef70be… wyoung 641 should be relayed to the `fossil email inbound` command. That command
2ef70be… wyoung 642 is currently a no-op place-holder. At some point, we will need to
2ef70be… wyoung 643 design and write a bounce-message processing system for Fossil.
2ef70be… wyoung 644
2ef70be… wyoung 645
2ef70be… wyoung 646 <a id="password" name="scode" name="verification"></a>
2ef70be… wyoung 647 ### Passwords vs Subscriber Codes
2ef70be… wyoung 648
2ef70be… wyoung 649 When anonymous passers-by on the Internet sign up for email alerts,
2ef70be… wyoung 650 their email address must first be verified. An email message is sent to
2ef70be… wyoung 651 the address supplied inviting the user to click on a link. The link
2ef70be… wyoung 652 includes a pseudorandom 128-bit blob encoded as 32 hexadecimal digits,
2ef70be… wyoung 653 which serves in place of a password for that email address. (This is
2ef70be… wyoung 654 stored in the database as `subscriber.subscriberCode`.) If anyone visits
2ef70be… wyoung 655 the link, the email address is verified.
2ef70be… wyoung 656
2ef70be… wyoung 657 Knowledge of the `subscriberCode` is sufficient to control a
2ef70be… wyoung 658 subscription.
2ef70be… wyoung 659
2ef70be… wyoung 660 Because this code is included in plain text in email alert messages, it
2ef70be… wyoung 661 is not as secure as a separate password, but it has several virtues:
2ef70be… wyoung 662
2ef70be… wyoung 663 * It is easier for the average subscriber to deal with in that they
2ef70be… wyoung 664 don't have to come up with yet another password and store it safely.
2ef70be… wyoung 665
2ef70be… wyoung 666 * If the `subscriberCode` is stolen, the worst that can happen is that
2ef70be… wyoung 667 the thief can change that email address's subscription settings.
2ef70be… wyoung 668 Contrast a password which may be shared with other services, which
2ef70be… wyoung 669 then compromises those other services.
2ef70be… wyoung 670
2ef70be… wyoung 671 * No PII other than the subscriber's email address is available to an
2ef70be… wyoung 672 attacker with the `subscriberCode`. Nor can knowledge of the
99a319b… wyoung 673 `subscriberCode` lead to an email flood or other annoyance attack, as
2ef70be… wyoung 674 far as I can see.
2ef70be… wyoung 675
2ef70be… wyoung 676 If the `subscriberCodes` for a Fossil repository are ever compromised,
2ef70be… wyoung 677 new ones can be generated as follows:
2ef70be… wyoung 678
8a1ba49… wyoung 679 UPDATE subscriber SET subscriberCode=randomblob(32);
2ef70be… wyoung 680
2ef70be… wyoung 681 Since this then affects all new email alerts going out from Fossil, your
2ef70be… wyoung 682 end users may never even realize that they're getting new codes, as long
2ef70be… wyoung 683 as they don't click on the URLs in the footer of old alert messages.
2ef70be… wyoung 684
2ef70be… wyoung 685 With that in mind, a Fossil server administrator could choose to
2ef70be… wyoung 686 randomize the `subscriberCodes` periodically, such as just before the
2ef70be… wyoung 687 daily digest emails are sent out each day.
2ef70be… wyoung 688
2ef70be… wyoung 689 **Important:** All of the above is distinct from the passwords for users
2ef70be… wyoung 690 with a Fossil repository login. Such users also have subscriber codes,
2ef70be… wyoung 691 but those codes can only be used to modify the user's email alert
2ef70be… wyoung 692 settings. That code cannot allow a user to log into the user's Fossil
2ef70be… wyoung 693 repository account.
2ef70be… wyoung 694
2ef70be… wyoung 695
2ef70be… wyoung 696 <a id="processing"></a>
2ef70be… wyoung 697 ### Internal Processing Flow
2ef70be… wyoung 698
2ef70be… wyoung 699 Almost all of the email alert code is found in the
2ef70be… wyoung 700 [`src/alerts.c`](/file/src/alerts.c) source file.
2ef70be… wyoung 701
2ef70be… wyoung 702 When email alerts are enabled, a trigger is created in the schema
2ef70be… wyoung 703 (`email_trigger1`) that adds a new entry to the `PENDING_ALERT` table
6ec931a… drh 704 every time a row is added to the `EVENT` table. During a
6ec931a… drh 705 `fossil rebuild`, the `EVENT` table is rebuilt from scratch; since we do not
2ef70be… wyoung 706 want users to get alerts for every historical check-in, the trigger is
2ef70be… wyoung 707 disabled during `rebuild`.
2ef70be… wyoung 708
2ef70be… wyoung 709 Email alerts are sent out by the `alert_send_alerts()` function, which
2ef70be… wyoung 710 is normally called automatically due to the `email-autoexec` setting,
2ef70be… wyoung 711 which defaults to enabled. If that setting is disabled or if the user
2ef70be… wyoung 712 simply wants to force email alerts to be sent immediately, they can give
2ef70be… wyoung 713 a `fossil alert send` command, such as via a `cron` script. Each time
2ef70be… wyoung 714 this function is called, the alert messages are moved further down the
2ef70be… wyoung 715 chain, so you cannot cause duplicate alerts by calling it too often.
2ef70be… wyoung 716
2ef70be… wyoung 717 Digests are handled by recording the time of the last digest in the
2ef70be… wyoung 718 `email-last-digest` setting, and only sending a new digest if the
2ef70be… wyoung 719 current time is one day or later after the last digest.
2ef70be… wyoung 720
2ef70be… wyoung 721 Individual emails are sent to each subscriber. I (drh) ran tests and
2ef70be… wyoung 722 found that I could send about 1200 emails/second, which is fast enough
2ef70be… wyoung 723 that I do not need to resort to trying to notify multiple subscribers
2ef70be… wyoung 724 with a single email. Because each subscriber gets a separate email, the
2ef70be… wyoung 725 system can include information in the email that is unique to the
2ef70be… wyoung 726 subscriber, such as a link to the page to edit their subscription. That
2ef70be… wyoung 727 link includes the `subscriberCode`.

Keyboard Shortcuts

Open search /
Next entry (timeline) j
Previous entry (timeline) k
Open focused entry Enter
Show this help ?
Toggle theme Top nav button