Fossil SCM

Split the HTTP-only parts out of www/tls-nginx.md into a new document discussing only the reverse-proxying of `fossil --scgi` to HTTP using nginx on Debian type OSes. That material is now in www/server/debian/nginx.md, which is referred to from www/server.wiki. While in there, did a bit of prose polishing on this old guide.

wyoung 2019-08-16 09:15 server-docs
Commit 2baa8151d7d79e3eef3a20fbcaaa0c97d89fd3ed697d05516240141822993254
Parent fad82714aab2c5b…
3 files changed +1 -1 +108 +50 -242
~ www/server.wiki + www/server/debian/nginx.md ~ www/tls-nginx.md
M www/server.wiki
+1 -1
--- www/server.wiki
+++ www/server.wiki
@@ -26,11 +26,11 @@
2626
 information on the underlying CGI technology.
2727
 See "[./sync.wiki|The Fossil Sync Protocol]" for information on the
2828
 wire protocol used for client/server communication.
2929
 </blockquote>
3030
31
-<h2>Methods</h2>
31
+<h2 id="methods">Methods</h2>
3232
3333
 <blockquote>
3434
 There are basically four ways to set up a Fossil server:
3535
3636
 <ol>
3737
3838
ADDED www/server/debian/nginx.md
--- www/server.wiki
+++ www/server.wiki
@@ -26,11 +26,11 @@
26 information on the underlying CGI technology.
27 See "[./sync.wiki|The Fossil Sync Protocol]" for information on the
28 wire protocol used for client/server communication.
29 </blockquote>
30
31 <h2>Methods</h2>
32
33 <blockquote>
34 There are basically four ways to set up a Fossil server:
35
36 <ol>
37
38 DDED www/server/debian/nginx.md
--- www/server.wiki
+++ www/server.wiki
@@ -26,11 +26,11 @@
26 information on the underlying CGI technology.
27 See "[./sync.wiki|The Fossil Sync Protocol]" for information on the
28 wire protocol used for client/server communication.
29 </blockquote>
30
31 <h2 id="methods">Methods</h2>
32
33 <blockquote>
34 There are basically four ways to set up a Fossil server:
35
36 <ol>
37
38 DDED www/server/debian/nginx.md
A www/server/debian/nginx.md
+108
--- a/www/server/debian/nginx.md
+++ b/www/server/debian/nginx.md
@@ -0,0 +1,108 @@
1
+# S generic;
2
+
3
+ S# S# S }
4
+
5
+
6
+ S# S# S }
7
+ # Redirect everything els }
8
+ }T is
9
+We focus
10
+Ubuntu 20.04 here, which a
11
+private servers][vpadding TLS access
12
+ supports
13
+TLS. One such option is nginx on Debian, so we show the details of that
14
+here.
15
+
16
+You can extend this guide to other operating systems by following the
17
+instructions fou18 via [the front Certbot web page][c;
18
+
19
+ S# S# S }
20
+ # Redirect everything els ls }
21
+ }T is
22
+# S generic;
23
+
24
+ S# S# S }
25
+ # Redirect everything If you want to add TLS to this configuration, that is covered [in a
26
+separate document][tls] which was written with the assumption that
27
+you’ve read this firstnfused Certbot, so we had totls]: ../../tls-nginxs manually](#lehw),
28
+else the Let’s Encrypt [ACME] exchange failed in thenameecessary domain
29
+validation steps.
30
+
31
+At this point, if# S generic;
32
+
33
+ S# S# S }
34
+ # Redirect everything els }
35
+ }T is
36
+
37
+
38
+ $ sudo snap install --classic certbot
39
+ $ sudo systemctl disable certbot.timer
40
+
41
+Next, edit `H@3AF,50:renewal/example.com.conf` to disable the
42
+nginx plugins. You’re looking for two lines setting the “install” and
43
+“auth” plugins to “nginx”. You can comment them out or remove them
44
+entirely.
45
+
46
+
47
+#### Step 2: Configuring nginx
48
+
49
+This is a straightforward extension to the HTTP/help/serveration
50
+[above](#config)e@1X4,9:foo.net;
51
+P@1Y9,_:tls-common;
52
+
53
+ charset utf-8c@this
54
+ option is overkill for our purposes. nginx is itself a fully
55
+ featured HTTP server, so we will choose in this guide not to make
56
+ S generic;
57
+
58
+ S# S# S }
59
+ ic;
60
+
61
+ S# S# S }
62
+ #.I run my Fossil SCGI server instances with a variant of [the `fslsrv`
63
+shell currently hosted in the Fossil source
64
+code repository. You’ll want to download that and make a copy of it, so
65
+you can customize it to your particular needSCGI servers, one per
66
+repository, each bound to a different high-numbered `localhost` port, so
67
+that only nginx can see and proxy them out to the public. The
68
+“`example`” repo is on TCP port localhost:12345, and the “`foo`” repo is
69
+on localhost:12346.
70
+
71
+As written, the `fslsrv` script expects repositories to be stored in the
72
+calling user’s home directory under `~/museum`, because where else do
73
+you keep Fossils?
74
+
75
+That home directory also needs to have a directory to hold log files,
76
+`~/log/fossil/*.log`. Fossil doesn’t put out much logging, but when it
77
+does, it’s better to have it captured than to need to re-create the
78
+problem after the fact.
79
+
80
+The use of `--baseurl` in this script lets us have each Fossil
81
+repository mounted in a different location in the URL scheme. Here, for
82
+example, we’re saying that the “`example`” repository is hosted under
83
+the `/code` URI on its domains, but that the “`foo`” repo is hosted at
84
+the top level of its domain. You’ll want to do something like the
85
+former for a Fossil repo that’s just one piece of a larger site, but the
86
+latter for a repo that is basically the whole point of the site.
87
+
88
+You might also want another script to automate the update, build, and
89
+deployment steps for new Fossil versions:
90
+
91
+ #!/bin/sh
92
+ fossil up
93
+ make -j11
94
+ killall fossil
95
+ sudo make install
96
+ fslsrv
97
+
98
+The `killall fossil` step is needed only on OSes that refuse to let you
99
+replace a running binary on disk.
100
+
101
+As written, the `fslsrv` script assumes a Linux environment. It expects
102
+`/bin/bash` to exist, and it depends on non-POSIX tools like `pgrep`.
103
+It should not be difficult to port to systems like macOS or the BSDs` contains thfoo.netT is
104
+We focus
105
+Ubuntu 20.04 # Sfoo.net;
106
+foo.netfoo.netDoxygen docsfoo.net scgi_param HTTPS "on"";
107
+ simpleedirect everything he public. The
108
+“`example`”
--- a/www/server/debian/nginx.md
+++ b/www/server/debian/nginx.md
@@ -0,0 +1,108 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--- a/www/server/debian/nginx.md
+++ b/www/server/debian/nginx.md
@@ -0,0 +1,108 @@
1 # S generic;
2
3 S# S# S }
4
5
6 S# S# S }
7 # Redirect everything els }
8 }T is
9 We focus
10 Ubuntu 20.04 here, which a
11 private servers][vpadding TLS access
12 supports
13 TLS. One such option is nginx on Debian, so we show the details of that
14 here.
15
16 You can extend this guide to other operating systems by following the
17 instructions fou18 via [the front Certbot web page][c;
18
19 S# S# S }
20 # Redirect everything els ls }
21 }T is
22 # S generic;
23
24 S# S# S }
25 # Redirect everything If you want to add TLS to this configuration, that is covered [in a
26 separate document][tls] which was written with the assumption that
27 you’ve read this firstnfused Certbot, so we had totls]: ../../tls-nginxs manually](#lehw),
28 else the Let’s Encrypt [ACME] exchange failed in thenameecessary domain
29 validation steps.
30
31 At this point, if# S generic;
32
33 S# S# S }
34 # Redirect everything els }
35 }T is
36
37
38 $ sudo snap install --classic certbot
39 $ sudo systemctl disable certbot.timer
40
41 Next, edit `H@3AF,50:renewal/example.com.conf` to disable the
42 nginx plugins. You’re looking for two lines setting the “install” and
43 “auth” plugins to “nginx”. You can comment them out or remove them
44 entirely.
45
46
47 #### Step 2: Configuring nginx
48
49 This is a straightforward extension to the HTTP/help/serveration
50 [above](#config)e@1X4,9:foo.net;
51 P@1Y9,_:tls-common;
52
53 charset utf-8c@this
54 option is overkill for our purposes. nginx is itself a fully
55 featured HTTP server, so we will choose in this guide not to make
56 S generic;
57
58 S# S# S }
59 ic;
60
61 S# S# S }
62 #.I run my Fossil SCGI server instances with a variant of [the `fslsrv`
63 shell currently hosted in the Fossil source
64 code repository. You’ll want to download that and make a copy of it, so
65 you can customize it to your particular needSCGI servers, one per
66 repository, each bound to a different high-numbered `localhost` port, so
67 that only nginx can see and proxy them out to the public. The
68 “`example`” repo is on TCP port localhost:12345, and the “`foo`” repo is
69 on localhost:12346.
70
71 As written, the `fslsrv` script expects repositories to be stored in the
72 calling user’s home directory under `~/museum`, because where else do
73 you keep Fossils?
74
75 That home directory also needs to have a directory to hold log files,
76 `~/log/fossil/*.log`. Fossil doesn’t put out much logging, but when it
77 does, it’s better to have it captured than to need to re-create the
78 problem after the fact.
79
80 The use of `--baseurl` in this script lets us have each Fossil
81 repository mounted in a different location in the URL scheme. Here, for
82 example, we’re saying that the “`example`” repository is hosted under
83 the `/code` URI on its domains, but that the “`foo`” repo is hosted at
84 the top level of its domain. You’ll want to do something like the
85 former for a Fossil repo that’s just one piece of a larger site, but the
86 latter for a repo that is basically the whole point of the site.
87
88 You might also want another script to automate the update, build, and
89 deployment steps for new Fossil versions:
90
91 #!/bin/sh
92 fossil up
93 make -j11
94 killall fossil
95 sudo make install
96 fslsrv
97
98 The `killall fossil` step is needed only on OSes that refuse to let you
99 replace a running binary on disk.
100
101 As written, the `fslsrv` script assumes a Linux environment. It expects
102 `/bin/bash` to exist, and it depends on non-POSIX tools like `pgrep`.
103 It should not be difficult to port to systems like macOS or the BSDs` contains thfoo.netT is
104 We focus
105 Ubuntu 20.04 # Sfoo.net;
106 foo.netfoo.netDoxygen docsfoo.net scgi_param HTTPS "on"";
107 simpleedirect everything he public. The
108 “`example`”
M www/tls-nginx.md
+50 -242
--- www/tls-nginx.md
+++ www/tls-nginx.md
@@ -3,183 +3,45 @@
33
 One of the [many ways](./ssl.wiki) to provide TLS-encrypted HTTP access
44
 (a.k.a. HTTPS) to Fossil is to run it behind a web proxy that supports
55
 TLS. This document explains how to use the powerful [nginx web
66
 server](http://nginx.org/) to do that.
77
8
-
9
-## Benefits
10
-
11
-This scheme is complicated, even with the benefit of this guide and
12
-pre-built binary packages. Why should you put up with this complexity?
13
-Because it gives many benefits that are difficult or impossible to get
14
-with the less complicated options:
15
-
16
-* **Power** — nginx is one of the most powerful web servers in the
17
- world. The chance that you will run into a web serving wall that you
18
- can’t scale with nginx is very low.
19
-
20
- To give you some idea of the sort of thing you can readily
21
- accomplish with nginx, your author runs a single public web server
22
- that provides transparent name-based virtual hosting for four
23
- separate domains:
24
-
25
- * One is entirely static, not involving any dynamic content or
26
- Fossil integration at all.
27
-
28
- * Another is served almost entirely by Fossil, with a few select
29
- static content exceptions punched past Fossil, which are handled
30
- entirely via nginx.
31
-
32
- * The other two domains are aliases for one another — e.g.
33
- `example.com` and `example.net` — with most of the content being
34
- static. This pair of domains has three different Fossil repo
35
- proxies attached to various sections of the URI hierarchy.
36
-
37
- All of this is done with minimal configuration repetition between
38
- the site configurations.
39
-
40
-* **Integration** — Because nginx is so popular, it integrates with
41
-many different technologies, and many other systems integrate with it in
42
-turn. This makes it great middleware, sitting between the outer web
43
-world and interior site services like Fossil. It allows Fossil to
44
-participate seamlessly as part of a larger web stack.
45
-
46
-* **Availability** — nginx is already in most operating system binary
47
-package repositories, so you don’t need to go out of your way to get it.
48
-
49
-
50
-## Fossil Remote Access Methods
51
-
52
-Fossil provides four major ways to access a repository it’s serving
53
-remotely, three of which are straightforward to use with nginx:
54
-
55
-* **HTTP** — Fossil has a built-in HTTP server: `fossil server`.
56
- While this method is efficient and it’s possible to use nginx to
57
- proxy access to another HTTP server, this option is overkill for our
58
- purposes. nginx is itself a fully featured HTTP server, so we will
59
- choose in this guide not to make nginx reinterpret Fossil’s
60
- implementation of HTTP.
61
-
62
-* **CGI** — This method is simple but inefficient, because it launches
63
- a separate Fossil instance on every HTTP hit.
64
-
65
- Since Fossil is a relatively small self-contained program, and it’s
66
- designed to start up quickly, this method can work well in a
67
- surprisingly large number of cases.
68
-
69
- Nevertheless, we will avoid this option in this document because
70
- we’re already buying into a certain amount of complexity here in
71
- order to gain power. There’s no sense in throwing away any of that
72
- hard-won performance on CGI overhead.
73
-
74
-* **SCGI** — The [SCGI protocol][scgi] provides the simplicity of CGI
75
- without its performance problems.
76
-
77
-* **SSH** — This method exists primarily to avoid the need for HTTPS
78
- in the first place. There is probably a way to get nginx to proxy
79
- Fossil to HTTPS via SSH, but it would be pointlessly complicated.
80
-
81
-SCGI it is, then.
82
-
83
-
84
-# Installing
85
-
86
-The first step is to install the pieces we’ll be working with. This
87
-varies on different operating systems, so to avoid overcomplicating this
88
-guide, we’re going to assume you’re using Ubuntu Server 18.04 LTS, a
89
-common Tier 1 offering for [virtual private servers][vps].
90
-
91
-SSH into your server, then say:
92
-
93
- $ sudo apt install certbot fossil nginx
94
-
95
-For other operating systems, simply visit [the front Certbot web
96
-page][cb] and tell it what OS and web stack you’re using. Chances are
97
-good that they’ve got a good guide for you already.
98
-
99
-
100
-# Running Fossil in SCGI Mode
101
-
102
-You presumably already have a working Fossil configuration on the public
103
-server you’re trying to set up and are just following this guide to
104
-replace HTTP service with HTTPS.
105
-
106
-(You can adjust the advice in this guide to get both HTTP *and* HTTPS
107
-service on the same site, but I strongly recommend that you do not do
108
-that: the good excuses remaining for continuing to allow HTTP on public
109
-web servers are running thin these days.)
110
-
111
-I run my Fossil SCGI server instances with a variant of [the `fslsrv`
112
-shell script](/file/tools/fslsrv) currently hosted in the Fossil source
113
-code repository. You’ll want to download that and make a copy of it, so
114
-you can customize it to your particular needs.
115
-
116
-This script allows running multiple Fossil SCGI servers, one per
117
-repository, each bound to a different high-numbered `localhost` port, so
118
-that only nginx can see and proxy them out to the public. The
119
-“`example`” repo is on TCP port localhost:12345, and the “`foo`” repo is
120
-on localhost:12346.
121
-
122
-As written, the `fslsrv` script expects repositories to be stored in the
123
-calling user’s home directory under `~/museum`, because where else do
124
-you keep Fossils?
125
-
126
-That home directory also needs to have a directory to hold log files,
127
-`~/log/fossil/*.log`. Fossil doesn’t put out much logging, but when it
128
-does, it’s better to have it captured than to need to re-create the
129
-problem after the fact.
130
-
131
-The use of `--baseurl` in this script lets us have each Fossil
132
-repository mounted in a different location in the URL scheme. Here, for
133
-example, we’re saying that the “`example`” repository is hosted under
134
-the `/code` URI on its domains, but that the “`foo`” repo is hosted at
135
-the top level of its domain. You’ll want to do something like the
136
-former for a Fossil repo that’s just one piece of a larger site, but the
137
-latter for a repo that is basically the whole point of the site.
138
-
139
-You might also want another script to automate the update, build, and
140
-deployment steps for new Fossil versions:
141
-
142
- #!/bin/sh
143
- cd $HOME/src/fossil/trunk
144
- fossil up
145
- make -j11
146
- killall fossil
147
- sudo make install
148
- fslsrv
149
-
150
-The `killall fossil` step is needed only on OSes that refuse to let you
151
-replace a running binary on disk.
152
-
153
-As written, the `fslsrv` script assumes a Linux environment. It expects
154
-`/bin/bash` to exist, and it depends on non-POSIX tools like `pgrep`.
155
-It should not be difficult to port to systems like macOS or the BSDs.
8
+This document is an extension of the [Serving via nginx on Debian][nod]
9
+document. Please read that first, then come back here to extend its
10
+configuration with TLS.
11
+
12
+[nod]: ./server/debian/nginx.md
13
+
14
+
15
+## Install Certbot
16
+
17
+The [nginx-on-Debian document][nod] had you install a few non-default
18
+packages to the system, but there’s one more you need for this guide:
19
+
20
+ $ sudo apt install certbot
21
+
22
+You can extend this guide to other operating systems by following the
23
+instructions found via [the front Certbot web page][cb] instead, telling
24
+it what OS and web stack you’re using. Chances are good that they’ve got
25
+a good guide for you already.
15626
15727
15828
 # Configuring Let’s Encrypt, the Easy Way
15929
16030
 If your web serving needs are simple, [Certbot][cb] can configure nginx
161
-for you and keep its certificates up to date. You can follow the Certbot
162
-documentation for [nginx on Ubuntu 18.04 LTS guide][cbnu] as-is, though
163
-we’d recommend one small change: to use the version of Certbot in the
164
-Ubuntu package repository rather than the first-party Certbot package
165
-that the guide recommends.
166
-
167
-The primary local configuration you need is to tell nginx how to proxy
168
-certain URLs down to the Fossil instance you started above with the
169
-`fslsrv` script:
170
-
171
- location / {
172
- include scgi_params;
173
- scgi_pass 127.0.0.1:12345;
174
- scgi_param HTTPS "on";
175
- scgi_param SCRIPT_NAME "";
176
- }
177
-
178
-The TCP port number in that snippet is the key: it has to match the port
179
-number generated by `fslsrv` from the base port number passed to the
180
-`start_one` function.
31
+for you and keep its certificates up to date. Simply follow Certbot’s
32
+[nginx on Ubuntu 18.04 LTS guide][cbnu]. We’d recommend one small
33
+change: to use the version of Certbot in the Ubuntu package repository
34
+rather than download it from the Certbot site.
35
+
36
+You should be able to use the nginx configuration given in our [Serving
37
+via nginx on Debian][nod] guide with little to no change. The main thing
38
+to watch out for is that the TCP port number in the nginx configuration
39
+needs to match the value you gave when starting Fossil. If you followed
40
+that guide’s advice, it will be 9000. Another option is to use [the
41
+`fslsrv` script](/file/tools/fslsrv), in which case the TCP port number
42
+will be 12345 or higher.
18143
18244
18345
 # Configuring Let’s Encrypt, the Hard Way
18446
18547
 If you’re finding that you can’t get certificates to be issued or
@@ -193,11 +55,12 @@
19355
 Environment][acme] protocol (ACME) to determine whether a given client
19456
 actually has control over the domain(s) for which it wants a certificate
19557
 minted. Let’s Encrypt will not blithely let you mint certificates for
19658
 `google.com` and `paypal.com` just because you ask for it!
19759
198
-Your author’s configuration, glossed above, is complicated enough that
60
+Your author’s configuration, glossed [in the HTTP-only guide][nod],
61
+is complicated enough that
19962
 the current version of Certbot (0.28 at the time of this writing) can’t
20063
 cope with it. That’s the primary motivation for me to write this guide:
20164
 I’m addressing the “me” years hence who needs to upgrade to Ubuntu 20.04
20265
 or 22.04 LTS and has forgotten all of this stuff. 😉
20366
@@ -216,24 +79,12 @@
21679
 entirely.
21780
21881
21982
 ## Step 2: Configuring nginx
22083
221
-On Ubuntu systems, at least, the primary user-level configuration file
222
-is `/etc/nginx/sites-enabled/default`. For a configuration like I
223
-described at the top of this article, I recommend that this file contain
224
-only a list of include statements, one for each site that server hosts:
225
-
226
- include local/example
227
- include local/foo
228
-
229
-Those files then each define one domain’s configuration. Here,
230
-`/etc/nginx/local/example` contains the configuration for
231
-`*.example.com` and `*.example.net`; and `local/foo` contains the
232
-configuration for `*.foo.net`.
233
-
234
-Here’s an example configuration:
84
+This is a straightforward extension to [the HTTP-only
85
+configuration](./server/debian/nginx.md#config):
23586
23687
 server {
23788
 server_name .foo.net;
23889
23990
 include local/tls-common;
@@ -268,12 +119,13 @@
268119
 include local/http-certbot-only;
269120
 access_log /var/log/nginx/foo.net-http-access.log;
270121
 error_log /var/log/nginx/foo.net-http-error.log;
271122
 }
272123
273
-Notice that we need two `server { }` blocks: one for HTTPS service, and
274
-one for HTTP-only service:
124
+One big difference between this and the HTTP-only case is
125
+that we need two `server { }` blocks: one for HTTPS service, and
126
+one for HTTP-only service.
275127
276128
277129
 ### HTTP over TLS (HTTPS) Service
278130
279131
 The first `server { }` block includes this file, `local/tls-common`:
@@ -385,43 +237,17 @@
385237
 # Force everything else to HTTPS with a permanent redirect.
386238
 #return 301 https://$host$request_uri;
387239
388240
 As written above, this configuration does nothing other than to tell
389241
 nginx that it’s allowed to serve content via HTTP on port 80 as well.
390
-
391242
 We’ll uncomment the `rewrite` and `return` directives below, when we’re
392243
 ready to begin testing.
393244
394
-
395
-#### Why the Repetition?
396
-
397
-These `server { }` blocks contain several directives that have to be
398
-either completely repeated or copied with only trivial changes when
399
-you’re hosting multiple domains from a single server.
400
-
401
-You might then wonder, why haven’t I factored some of those directives
402
-into the included files `local/tls-common` and
403
-`local/http-certbot-only`? Why can’t the HTTP-only `server { }` block
404
-above be just two lines? That is, why can I not say:
405
-
406
- server_name .foo.net;
407
- include local/http-certbot-only;
408
-
409
-Then in `local/http-certbot-only` say:
410
-
411
- root /var/www/$host;
412
- access_log /var/log/nginx/$host-http-access.log;
413
- error_log /var/log/nginx/$host-http-error.log;
414
-
415
-Sadly, nginx doesn’t allow variable substitution into these particular
416
-directives. As I understand it, allowing that would make nginx slower,
417
-so we must largely repeat these directives in each HTTP `server { }`
418
-block.
419
-
420
-These configurations are, as shown, as small as I know how to get them.
421
-If you know of a way to reduce some of this repetition, [I solicit your
422
-advice][fd].
245
+Notice that this configuration is very different from that in the
246
+[HTTP-only nginx on Debian][nod] guide. Most of that guide’s nginx
247
+directives moved up into the TLS `server { }` block, because we
248
+eventually want this site to be as close to HTTPS-only as we can get it.
423249
424250
425251
 ## Step 3: Dry Run
426252
427253
 We want to first request a dry run, because Let’s Encrypt puts some
@@ -541,29 +367,19 @@
541367
 it would actually [cause an infinite redirect loop if
542368
 enabled](./ssl.wiki#rloop).
543369
544370
545371
546
-## Step 6: Re-Sync Your Repositories
372
+## Step 6: Re-Point Fossil at Your Repositories
547373
548
-Now that the repositories hosted by this server are available via HTTPS,
549
-you need to tell Fossil about it:
374
+As of Fossil 2.9, the permanent HTTP-to-HTTPS redirect we enabled above
375
+causes Fossil to remember the new URL automatically the first time it’s
376
+redirected to it. All you need to do to switch your syncs to HTTPS is:
550377
551378
 $ cd ~/path/to/checkout
552
- $ fossil sync https://example.com/code
553
-
554
-Once that’s done per repository file, all checkouts of that repo will
555
-from that point on use the HTTPS URI to sync.
556
-
557
-You might wonder if that’s necessary, since we have the automatic
558
-HTTP-to-HTTPS redirect on this site now. If you clone or sync one of
559
-these nginx-hosted Fossil repositories over an untrustworthy network
560
-that allows [MITM attacks][mitm], that redirect won’t protect you from a
561
-sufficiently capable and motivated attacker unless you’ve also gone
562
-ahead and [enabled HSTS](#hsts). You can put off the need to enable
563
-HSTS by explicitly using HTTPS URIs.
564
-
379
+ $ fossil sync
380
+
565381
566382
 ## Step 7: Renewing Automatically
567383
568384
 Now that the configuration is solid, you can renew the LE cert with the
569385
 `certbot` command from above without the `--dry-run` flag plus a restart
@@ -588,27 +404,21 @@
588404
 -----------
589405
590406
 <a id=”evolution”></a>
591407
 **Document Evolution**
592408
593
-TLS and web proxying are a constantly evolving technology. This article
594
-replaces my [earlier effort][2016], which had whole sections that were
595
-basically obsolete within about a year of posting it. Two years on, and
596
-I was encouraging readers to ignore about half of that HOWTO. I am now
597
-writing this document about 3 years later because Let’s Encrypt
598
-deprecated key technology that HOWTO depended on, to the point that
599
-following that old HOWTO is more likely to confuse than enlighten.
409
+Large parts of this article have been rewritten several times now due to
410
+shifting technology in the TLS and proxying spheres.
600411
601412
 There is no particularly good reason to expect that this sort of thing
602
-will not continue to happen, so this effort is expected to be a living
413
+will not continue to happen, so we consider this to be a living
603414
 document. If you do not have commit access on the `fossil-scm.org`
604415
 repository to update this document as the world changes around it, you
605416
 can discuss this document [on the forum][fd]. This document’s author
606417
 keeps an eye on the forum and expects to keep this document updated with
607418
 ideas that appear in that thread.
608419
609
-[2016]: https://www.mail-archive.com/[email protected]/msg22907.html
610420
 [acme]: https://en.wikipedia.org/wiki/Automated_Certificate_Management_Environment
611421
 [cb]: https://certbot.eff.org/
612422
 [cbnu]: https://certbot.eff.org/lets-encrypt/ubuntubionic-nginx
613423
 [fd]: https://fossil-scm.org/forum/forumpost/ae6a4ee157
614424
 [hsts]: https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security
@@ -616,7 +426,5 @@
616426
 [mitm]: https://en.wikipedia.org/wiki/Man-in-the-middle_attack
617427
 [nest]: https://www.nginx.com/blog/http-strict-transport-security-hsts-and-nginx/
618428
 [ocsp]: https://en.wikipedia.org/wiki/OCSP_stapling
619429
 [qslc]: https://github.com/ssllabs/research/wiki/SSL-and-TLS-Deployment-Best-Practices
620430
 [qslt]: https://www.ssllabs.com/ssltest/
621
-[scgi]: https://en.wikipedia.org/wiki/Simple_Common_Gateway_Interface
622
-[vps]: https://en.wikipedia.org/wiki/Virtual_private_server
623431
--- www/tls-nginx.md
+++ www/tls-nginx.md
@@ -3,183 +3,45 @@
3 One of the [many ways](./ssl.wiki) to provide TLS-encrypted HTTP access
4 (a.k.a. HTTPS) to Fossil is to run it behind a web proxy that supports
5 TLS. This document explains how to use the powerful [nginx web
6 server](http://nginx.org/) to do that.
7
8
9 ## Benefits
10
11 This scheme is complicated, even with the benefit of this guide and
12 pre-built binary packages. Why should you put up with this complexity?
13 Because it gives many benefits that are difficult or impossible to get
14 with the less complicated options:
15
16 * **Power** — nginx is one of the most powerful web servers in the
17 world. The chance that you will run into a web serving wall that you
18 can’t scale with nginx is very low.
19
20 To give you some idea of the sort of thing you can readily
21 accomplish with nginx, your author runs a single public web server
22 that provides transparent name-based virtual hosting for four
23 separate domains:
24
25 * One is entirely static, not involving any dynamic content or
26 Fossil integration at all.
27
28 * Another is served almost entirely by Fossil, with a few select
29 static content exceptions punched past Fossil, which are handled
30 entirely via nginx.
31
32 * The other two domains are aliases for one another — e.g.
33 `example.com` and `example.net` — with most of the content being
34 static. This pair of domains has three different Fossil repo
35 proxies attached to various sections of the URI hierarchy.
36
37 All of this is done with minimal configuration repetition between
38 the site configurations.
39
40 * **Integration** — Because nginx is so popular, it integrates with
41 many different technologies, and many other systems integrate with it in
42 turn. This makes it great middleware, sitting between the outer web
43 world and interior site services like Fossil. It allows Fossil to
44 participate seamlessly as part of a larger web stack.
45
46 * **Availability** — nginx is already in most operating system binary
47 package repositories, so you don’t need to go out of your way to get it.
48
49
50 ## Fossil Remote Access Methods
51
52 Fossil provides four major ways to access a repository it’s serving
53 remotely, three of which are straightforward to use with nginx:
54
55 * **HTTP** — Fossil has a built-in HTTP server: `fossil server`.
56 While this method is efficient and it’s possible to use nginx to
57 proxy access to another HTTP server, this option is overkill for our
58 purposes. nginx is itself a fully featured HTTP server, so we will
59 choose in this guide not to make nginx reinterpret Fossil’s
60 implementation of HTTP.
61
62 * **CGI** — This method is simple but inefficient, because it launches
63 a separate Fossil instance on every HTTP hit.
64
65 Since Fossil is a relatively small self-contained program, and it’s
66 designed to start up quickly, this method can work well in a
67 surprisingly large number of cases.
68
69 Nevertheless, we will avoid this option in this document because
70 we’re already buying into a certain amount of complexity here in
71 order to gain power. There’s no sense in throwing away any of that
72 hard-won performance on CGI overhead.
73
74 * **SCGI** — The [SCGI protocol][scgi] provides the simplicity of CGI
75 without its performance problems.
76
77 * **SSH** — This method exists primarily to avoid the need for HTTPS
78 in the first place. There is probably a way to get nginx to proxy
79 Fossil to HTTPS via SSH, but it would be pointlessly complicated.
80
81 SCGI it is, then.
82
83
84 # Installing
85
86 The first step is to install the pieces we’ll be working with. This
87 varies on different operating systems, so to avoid overcomplicating this
88 guide, we’re going to assume you’re using Ubuntu Server 18.04 LTS, a
89 common Tier 1 offering for [virtual private servers][vps].
90
91 SSH into your server, then say:
92
93 $ sudo apt install certbot fossil nginx
94
95 For other operating systems, simply visit [the front Certbot web
96 page][cb] and tell it what OS and web stack you’re using. Chances are
97 good that they’ve got a good guide for you already.
98
99
100 # Running Fossil in SCGI Mode
101
102 You presumably already have a working Fossil configuration on the public
103 server you’re trying to set up and are just following this guide to
104 replace HTTP service with HTTPS.
105
106 (You can adjust the advice in this guide to get both HTTP *and* HTTPS
107 service on the same site, but I strongly recommend that you do not do
108 that: the good excuses remaining for continuing to allow HTTP on public
109 web servers are running thin these days.)
110
111 I run my Fossil SCGI server instances with a variant of [the `fslsrv`
112 shell script](/file/tools/fslsrv) currently hosted in the Fossil source
113 code repository. You’ll want to download that and make a copy of it, so
114 you can customize it to your particular needs.
115
116 This script allows running multiple Fossil SCGI servers, one per
117 repository, each bound to a different high-numbered `localhost` port, so
118 that only nginx can see and proxy them out to the public. The
119 “`example`” repo is on TCP port localhost:12345, and the “`foo`” repo is
120 on localhost:12346.
121
122 As written, the `fslsrv` script expects repositories to be stored in the
123 calling user’s home directory under `~/museum`, because where else do
124 you keep Fossils?
125
126 That home directory also needs to have a directory to hold log files,
127 `~/log/fossil/*.log`. Fossil doesn’t put out much logging, but when it
128 does, it’s better to have it captured than to need to re-create the
129 problem after the fact.
130
131 The use of `--baseurl` in this script lets us have each Fossil
132 repository mounted in a different location in the URL scheme. Here, for
133 example, we’re saying that the “`example`” repository is hosted under
134 the `/code` URI on its domains, but that the “`foo`” repo is hosted at
135 the top level of its domain. You’ll want to do something like the
136 former for a Fossil repo that’s just one piece of a larger site, but the
137 latter for a repo that is basically the whole point of the site.
138
139 You might also want another script to automate the update, build, and
140 deployment steps for new Fossil versions:
141
142 #!/bin/sh
143 cd $HOME/src/fossil/trunk
144 fossil up
145 make -j11
146 killall fossil
147 sudo make install
148 fslsrv
149
150 The `killall fossil` step is needed only on OSes that refuse to let you
151 replace a running binary on disk.
152
153 As written, the `fslsrv` script assumes a Linux environment. It expects
154 `/bin/bash` to exist, and it depends on non-POSIX tools like `pgrep`.
155 It should not be difficult to port to systems like macOS or the BSDs.
156
157
158 # Configuring Let’s Encrypt, the Easy Way
159
160 If your web serving needs are simple, [Certbot][cb] can configure nginx
161 for you and keep its certificates up to date. You can follow the Certbot
162 documentation for [nginx on Ubuntu 18.04 LTS guide][cbnu] as-is, though
163 we’d recommend one small change: to use the version of Certbot in the
164 Ubuntu package repository rather than the first-party Certbot package
165 that the guide recommends.
166
167 The primary local configuration you need is to tell nginx how to proxy
168 certain URLs down to the Fossil instance you started above with the
169 `fslsrv` script:
170
171 location / {
172 include scgi_params;
173 scgi_pass 127.0.0.1:12345;
174 scgi_param HTTPS "on";
175 scgi_param SCRIPT_NAME "";
176 }
177
178 The TCP port number in that snippet is the key: it has to match the port
179 number generated by `fslsrv` from the base port number passed to the
180 `start_one` function.
181
182
183 # Configuring Let’s Encrypt, the Hard Way
184
185 If you’re finding that you can’t get certificates to be issued or
@@ -193,11 +55,12 @@
193 Environment][acme] protocol (ACME) to determine whether a given client
194 actually has control over the domain(s) for which it wants a certificate
195 minted. Let’s Encrypt will not blithely let you mint certificates for
196 `google.com` and `paypal.com` just because you ask for it!
197
198 Your author’s configuration, glossed above, is complicated enough that
 
199 the current version of Certbot (0.28 at the time of this writing) can’t
200 cope with it. That’s the primary motivation for me to write this guide:
201 I’m addressing the “me” years hence who needs to upgrade to Ubuntu 20.04
202 or 22.04 LTS and has forgotten all of this stuff. 😉
203
@@ -216,24 +79,12 @@
216 entirely.
217
218
219 ## Step 2: Configuring nginx
220
221 On Ubuntu systems, at least, the primary user-level configuration file
222 is `/etc/nginx/sites-enabled/default`. For a configuration like I
223 described at the top of this article, I recommend that this file contain
224 only a list of include statements, one for each site that server hosts:
225
226 include local/example
227 include local/foo
228
229 Those files then each define one domain’s configuration. Here,
230 `/etc/nginx/local/example` contains the configuration for
231 `*.example.com` and `*.example.net`; and `local/foo` contains the
232 configuration for `*.foo.net`.
233
234 Here’s an example configuration:
235
236 server {
237 server_name .foo.net;
238
239 include local/tls-common;
@@ -268,12 +119,13 @@
268 include local/http-certbot-only;
269 access_log /var/log/nginx/foo.net-http-access.log;
270 error_log /var/log/nginx/foo.net-http-error.log;
271 }
272
273 Notice that we need two `server { }` blocks: one for HTTPS service, and
274 one for HTTP-only service:
 
275
276
277 ### HTTP over TLS (HTTPS) Service
278
279 The first `server { }` block includes this file, `local/tls-common`:
@@ -385,43 +237,17 @@
385 # Force everything else to HTTPS with a permanent redirect.
386 #return 301 https://$host$request_uri;
387
388 As written above, this configuration does nothing other than to tell
389 nginx that it’s allowed to serve content via HTTP on port 80 as well.
390
391 We’ll uncomment the `rewrite` and `return` directives below, when we’re
392 ready to begin testing.
393
394
395 #### Why the Repetition?
396
397 These `server { }` blocks contain several directives that have to be
398 either completely repeated or copied with only trivial changes when
399 you’re hosting multiple domains from a single server.
400
401 You might then wonder, why haven’t I factored some of those directives
402 into the included files `local/tls-common` and
403 `local/http-certbot-only`? Why can’t the HTTP-only `server { }` block
404 above be just two lines? That is, why can I not say:
405
406 server_name .foo.net;
407 include local/http-certbot-only;
408
409 Then in `local/http-certbot-only` say:
410
411 root /var/www/$host;
412 access_log /var/log/nginx/$host-http-access.log;
413 error_log /var/log/nginx/$host-http-error.log;
414
415 Sadly, nginx doesn’t allow variable substitution into these particular
416 directives. As I understand it, allowing that would make nginx slower,
417 so we must largely repeat these directives in each HTTP `server { }`
418 block.
419
420 These configurations are, as shown, as small as I know how to get them.
421 If you know of a way to reduce some of this repetition, [I solicit your
422 advice][fd].
423
424
425 ## Step 3: Dry Run
426
427 We want to first request a dry run, because Let’s Encrypt puts some
@@ -541,29 +367,19 @@
541 it would actually [cause an infinite redirect loop if
542 enabled](./ssl.wiki#rloop).
543
544
545
546 ## Step 6: Re-Sync Your Repositories
547
548 Now that the repositories hosted by this server are available via HTTPS,
549 you need to tell Fossil about it:
 
550
551 $ cd ~/path/to/checkout
552 $ fossil sync https://example.com/code
553
554 Once that’s done per repository file, all checkouts of that repo will
555 from that point on use the HTTPS URI to sync.
556
557 You might wonder if that’s necessary, since we have the automatic
558 HTTP-to-HTTPS redirect on this site now. If you clone or sync one of
559 these nginx-hosted Fossil repositories over an untrustworthy network
560 that allows [MITM attacks][mitm], that redirect won’t protect you from a
561 sufficiently capable and motivated attacker unless you’ve also gone
562 ahead and [enabled HSTS](#hsts). You can put off the need to enable
563 HSTS by explicitly using HTTPS URIs.
564
565
566 ## Step 7: Renewing Automatically
567
568 Now that the configuration is solid, you can renew the LE cert with the
569 `certbot` command from above without the `--dry-run` flag plus a restart
@@ -588,27 +404,21 @@
588 -----------
589
590 <a id=”evolution”></a>
591 **Document Evolution**
592
593 TLS and web proxying are a constantly evolving technology. This article
594 replaces my [earlier effort][2016], which had whole sections that were
595 basically obsolete within about a year of posting it. Two years on, and
596 I was encouraging readers to ignore about half of that HOWTO. I am now
597 writing this document about 3 years later because Let’s Encrypt
598 deprecated key technology that HOWTO depended on, to the point that
599 following that old HOWTO is more likely to confuse than enlighten.
600
601 There is no particularly good reason to expect that this sort of thing
602 will not continue to happen, so this effort is expected to be a living
603 document. If you do not have commit access on the `fossil-scm.org`
604 repository to update this document as the world changes around it, you
605 can discuss this document [on the forum][fd]. This document’s author
606 keeps an eye on the forum and expects to keep this document updated with
607 ideas that appear in that thread.
608
609 [2016]: https://www.mail-archive.com/[email protected]/msg22907.html
610 [acme]: https://en.wikipedia.org/wiki/Automated_Certificate_Management_Environment
611 [cb]: https://certbot.eff.org/
612 [cbnu]: https://certbot.eff.org/lets-encrypt/ubuntubionic-nginx
613 [fd]: https://fossil-scm.org/forum/forumpost/ae6a4ee157
614 [hsts]: https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security
@@ -616,7 +426,5 @@
616 [mitm]: https://en.wikipedia.org/wiki/Man-in-the-middle_attack
617 [nest]: https://www.nginx.com/blog/http-strict-transport-security-hsts-and-nginx/
618 [ocsp]: https://en.wikipedia.org/wiki/OCSP_stapling
619 [qslc]: https://github.com/ssllabs/research/wiki/SSL-and-TLS-Deployment-Best-Practices
620 [qslt]: https://www.ssllabs.com/ssltest/
621 [scgi]: https://en.wikipedia.org/wiki/Simple_Common_Gateway_Interface
622 [vps]: https://en.wikipedia.org/wiki/Virtual_private_server
623
--- www/tls-nginx.md
+++ www/tls-nginx.md
@@ -3,183 +3,45 @@
3 One of the [many ways](./ssl.wiki) to provide TLS-encrypted HTTP access
4 (a.k.a. HTTPS) to Fossil is to run it behind a web proxy that supports
5 TLS. This document explains how to use the powerful [nginx web
6 server](http://nginx.org/) to do that.
7
8 This document is an extension of the [Serving via nginx on Debian][nod]
9 document. Please read that first, then come back here to extend its
10 configuration with TLS.
11
12 [nod]: ./server/debian/nginx.md
13
14
15 ## Install Certbot
16
17 The [nginx-on-Debian document][nod] had you install a few non-default
18 packages to the system, but there’s one more you need for this guide:
19
20 $ sudo apt install certbot
21
22 You can extend this guide to other operating systems by following the
23 instructions found via [the front Certbot web page][cb] instead, telling
24 it what OS and web stack you’re using. Chances are good that they’ve got
25 a good guide for you already.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
26
27
28 # Configuring Let’s Encrypt, the Easy Way
29
30 If your web serving needs are simple, [Certbot][cb] can configure nginx
31 for you and keep its certificates up to date. Simply follow Certbot’s
32 [nginx on Ubuntu 18.04 LTS guide][cbnu]. We’d recommend one small
33 change: to use the version of Certbot in the Ubuntu package repository
34 rather than download it from the Certbot site.
35
36 You should be able to use the nginx configuration given in our [Serving
37 via nginx on Debian][nod] guide with little to no change. The main thing
38 to watch out for is that the TCP port number in the nginx configuration
39 needs to match the value you gave when starting Fossil. If you followed
40 that guide’s advice, it will be 9000. Another option is to use [the
41 `fslsrv` script](/file/tools/fslsrv), in which case the TCP port number
42 will be 12345 or higher.
 
 
 
 
 
 
 
 
43
44
45 # Configuring Let’s Encrypt, the Hard Way
46
47 If you’re finding that you can’t get certificates to be issued or
@@ -193,11 +55,12 @@
55 Environment][acme] protocol (ACME) to determine whether a given client
56 actually has control over the domain(s) for which it wants a certificate
57 minted. Let’s Encrypt will not blithely let you mint certificates for
58 `google.com` and `paypal.com` just because you ask for it!
59
60 Your author’s configuration, glossed [in the HTTP-only guide][nod],
61 is complicated enough that
62 the current version of Certbot (0.28 at the time of this writing) can’t
63 cope with it. That’s the primary motivation for me to write this guide:
64 I’m addressing the “me” years hence who needs to upgrade to Ubuntu 20.04
65 or 22.04 LTS and has forgotten all of this stuff. 😉
66
@@ -216,24 +79,12 @@
79 entirely.
80
81
82 ## Step 2: Configuring nginx
83
84 This is a straightforward extension to [the HTTP-only
85 configuration](./server/debian/nginx.md#config):
 
 
 
 
 
 
 
 
 
 
 
 
86
87 server {
88 server_name .foo.net;
89
90 include local/tls-common;
@@ -268,12 +119,13 @@
119 include local/http-certbot-only;
120 access_log /var/log/nginx/foo.net-http-access.log;
121 error_log /var/log/nginx/foo.net-http-error.log;
122 }
123
124 One big difference between this and the HTTP-only case is
125 that we need two `server { }` blocks: one for HTTPS service, and
126 one for HTTP-only service.
127
128
129 ### HTTP over TLS (HTTPS) Service
130
131 The first `server { }` block includes this file, `local/tls-common`:
@@ -385,43 +237,17 @@
237 # Force everything else to HTTPS with a permanent redirect.
238 #return 301 https://$host$request_uri;
239
240 As written above, this configuration does nothing other than to tell
241 nginx that it’s allowed to serve content via HTTP on port 80 as well.
 
242 We’ll uncomment the `rewrite` and `return` directives below, when we’re
243 ready to begin testing.
244
245 Notice that this configuration is very different from that in the
246 [HTTP-only nginx on Debian][nod] guide. Most of that guide’s nginx
247 directives moved up into the TLS `server { }` block, because we
248 eventually want this site to be as close to HTTPS-only as we can get it.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
249
250
251 ## Step 3: Dry Run
252
253 We want to first request a dry run, because Let’s Encrypt puts some
@@ -541,29 +367,19 @@
367 it would actually [cause an infinite redirect loop if
368 enabled](./ssl.wiki#rloop).
369
370
371
372 ## Step 6: Re-Point Fossil at Your Repositories
373
374 As of Fossil 2.9, the permanent HTTP-to-HTTPS redirect we enabled above
375 causes Fossil to remember the new URL automatically the first time it’s
376 redirected to it. All you need to do to switch your syncs to HTTPS is:
377
378 $ cd ~/path/to/checkout
379 $ fossil sync
380
 
 
 
 
 
 
 
 
 
 
 
381
382 ## Step 7: Renewing Automatically
383
384 Now that the configuration is solid, you can renew the LE cert with the
385 `certbot` command from above without the `--dry-run` flag plus a restart
@@ -588,27 +404,21 @@
404 -----------
405
406 <a id=”evolution”></a>
407 **Document Evolution**
408
409 Large parts of this article have been rewritten several times now due to
410 shifting technology in the TLS and proxying spheres.
 
 
 
 
 
411
412 There is no particularly good reason to expect that this sort of thing
413 will not continue to happen, so we consider this to be a living
414 document. If you do not have commit access on the `fossil-scm.org`
415 repository to update this document as the world changes around it, you
416 can discuss this document [on the forum][fd]. This document’s author
417 keeps an eye on the forum and expects to keep this document updated with
418 ideas that appear in that thread.
419
 
420 [acme]: https://en.wikipedia.org/wiki/Automated_Certificate_Management_Environment
421 [cb]: https://certbot.eff.org/
422 [cbnu]: https://certbot.eff.org/lets-encrypt/ubuntubionic-nginx
423 [fd]: https://fossil-scm.org/forum/forumpost/ae6a4ee157
424 [hsts]: https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security
@@ -616,7 +426,5 @@
426 [mitm]: https://en.wikipedia.org/wiki/Man-in-the-middle_attack
427 [nest]: https://www.nginx.com/blog/http-strict-transport-security-hsts-and-nginx/
428 [ocsp]: https://en.wikipedia.org/wiki/OCSP_stapling
429 [qslc]: https://github.com/ssllabs/research/wiki/SSL-and-TLS-Deployment-Best-Practices
430 [qslt]: https://www.ssllabs.com/ssltest/
 
 
431

Keyboard Shortcuts

Open search /
Next entry (timeline) j
Previous entry (timeline) k
Open focused entry Enter
Show this help ?
Toggle theme Top nav button