|
1
|
# Serving via nginx on Debian and Ubuntu |
|
2
|
|
|
3
|
This document is an extension of [the platform-independent SCGI |
|
4
|
instructions][scgii], which may suffice for your purposes if your needs |
|
5
|
are simple. |
|
6
|
|
|
7
|
Here, we add more detailed information on nginx itself, plus details |
|
8
|
about running it on Debian type OSes. This document was originally |
|
9
|
written for and tested on Debian 10 (Buster) and Ubuntu 20.04, which |
|
10
|
were common Tier 1 OS offerings for [virtual private servers][vps] |
|
11
|
at the time. The same configuration appears to run on Ubuntu 22.04 |
|
12
|
LTS without change. This material may not work for older OSes. It is |
|
13
|
known in particular to not work as given for Debian 9 and older! |
|
14
|
|
|
15
|
We also cover [adding TLS](#tls) to the basic configuration, because several |
|
16
|
details depend on the host OS and web stack details. Besides, TLS is |
|
17
|
widely considered part of the baseline configuration these days. |
|
18
|
|
|
19
|
[scgii]: ../any/scgi.md |
|
20
|
[vps]: https://en.wikipedia.org/wiki/Virtual_private_server |
|
21
|
|
|
22
|
|
|
23
|
## <a id="benefits"></a>Benefits |
|
24
|
|
|
25
|
This scheme is considerably more complicated than the [standalone HTTP |
|
26
|
server](../any/none.md) and [CGI options](../any/cgi.md). Even with the |
|
27
|
benefit of this guide and pre-built binary packages, it requires quite a |
|
28
|
bit of work to set it up. Why should you put up with this complexity? |
|
29
|
Because it gives many benefits that are difficult or impossible to get |
|
30
|
with the less complicated options: |
|
31
|
|
|
32
|
* **Power** — nginx is one of the most powerful web servers in the |
|
33
|
world. The chance that you will run into a web serving wall that you |
|
34
|
can’t scale with nginx is very low. |
|
35
|
|
|
36
|
To give you some idea of the sort of thing you can readily |
|
37
|
accomplish with nginx, your author runs a single public web server |
|
38
|
that provides transparent name-based virtual hosting for four |
|
39
|
separate domains: |
|
40
|
|
|
41
|
* <p>One is entirely static, not involving any dynamic content or |
|
42
|
Fossil integration at all.</p> |
|
43
|
|
|
44
|
* <p>Another is served almost entirely by Fossil, with a few select |
|
45
|
static content exceptions punched past Fossil, which are handled |
|
46
|
entirely via nginx.</p> |
|
47
|
|
|
48
|
* <p>The other two domains are aliases for one another — e.g. |
|
49
|
`example.com` and `example.net` — with most of the content being |
|
50
|
static. This pair of domains has several unrelated Fossil repo |
|
51
|
proxies attached to various sections of the URI hierarchy.</p> |
|
52
|
|
|
53
|
By using nginx, I was able to do all of the above with minimal |
|
54
|
repetition between the site configurations. |
|
55
|
|
|
56
|
* **Integration** — Because nginx is so popular, it integrates with |
|
57
|
many different technologies, and many other systems integrate with it in |
|
58
|
turn. This makes it great middleware, sitting between the outer web |
|
59
|
world and interior site services like Fossil. It allows Fossil to |
|
60
|
participate seamlessly as part of a larger web stack. |
|
61
|
|
|
62
|
* **Availability** — nginx is already in most operating system binary |
|
63
|
package repositories, so you don’t need to go out of your way to get it. |
|
64
|
|
|
65
|
|
|
66
|
## <a id="modes"></a>Fossil Service Modes |
|
67
|
|
|
68
|
Fossil provides four major ways to access a repository it’s serving |
|
69
|
remotely, three of which are straightforward to use with nginx: |
|
70
|
|
|
71
|
* **HTTP** — Fossil has [a built-in HTTP server](../any/none.md). |
|
72
|
While this method is efficient and it’s |
|
73
|
possible to use nginx to proxy access to another HTTP server, we |
|
74
|
don’t see any particularly good reason to make nginx reinterpret |
|
75
|
Fossil’s own implementation of HTTP when we have a better option. |
|
76
|
(But see [below](#http).) |
|
77
|
|
|
78
|
* **SSH** — This method exists primarily to avoid the need for HTTPS, |
|
79
|
but we *want* HTTPS. (We’ll get to that [below](#tls).) |
|
80
|
There is probably a way to get nginx to proxy Fossil to HTTPS via |
|
81
|
SSH, but it would be pointlessly complicated. |
|
82
|
|
|
83
|
* **CGI** — This method is simple but inefficient, because it launches |
|
84
|
a separate Fossil instance on every HTTP hit. |
|
85
|
Since Fossil is a relatively small self-contained program, and it’s |
|
86
|
designed to start up quickly, this method can work well in a |
|
87
|
surprisingly large number of cases. |
|
88
|
Nevertheless, we will avoid this option in this document because |
|
89
|
we’re already buying into a certain amount of complexity here in |
|
90
|
order to gain power. There’s no sense in throwing away any of that |
|
91
|
hard-won performance on CGI overhead. |
|
92
|
|
|
93
|
* **SCGI** — The [SCGI protocol][scgip] provides the simplicity of CGI |
|
94
|
without its performance problems. |
|
95
|
|
|
96
|
SCGI it is, then. |
|
97
|
|
|
98
|
[scgip]: https://en.wikipedia.org/wiki/Simple_Common_Gateway_Interface |
|
99
|
|
|
100
|
|
|
101
|
## <a id="deps"></a>Installing the Dependencies |
|
102
|
|
|
103
|
The first step is to install some non-default packages we’ll need. SSH into |
|
104
|
your server, then say: |
|
105
|
|
|
106
|
$ sudo apt install fossil nginx |
|
107
|
|
|
108
|
You can leave “`fossil`” out of that if you’re building Fossil from |
|
109
|
source to get a more up-to-date version than is shipped with the host |
|
110
|
OS. |
|
111
|
|
|
112
|
|
|
113
|
## <a id="scgi"></a>Running Fossil in SCGI Mode |
|
114
|
|
|
115
|
For the following nginx configuration to work, it needs to contact a |
|
116
|
background Fossil instance speaking the SCGI protocol. There are |
|
117
|
[many ways](../) to set that up, such as [with `systemd`](./service.md) |
|
118
|
on mainstream Linux distros. Another way is to [containerize][ctz] your |
|
119
|
repository servers, then use the [`fslsrv` wrapper for Podman][fspm] to |
|
120
|
generate `systemd` units for use by the front-end proxy. |
|
121
|
|
|
122
|
However you do it, you need to match up the TCP port numbers between it |
|
123
|
and those in the nginx configuration below. |
|
124
|
|
|
125
|
[ctz]: ../../containers.md |
|
126
|
[fspm]: https://tangentsoft.com/fossil/dir/bin |
|
127
|
|
|
128
|
|
|
129
|
## <a id="config"></a>Configuration |
|
130
|
|
|
131
|
On Debian and Ubuntu systems the primary user-level configuration file |
|
132
|
for nginx is `/etc/nginx/sites-enabled/default`. I recommend that this |
|
133
|
file contain only a list of include statements, one for each site that |
|
134
|
server hosts: |
|
135
|
|
|
136
|
include local/example.com |
|
137
|
include local/foo.net |
|
138
|
|
|
139
|
Those files then each define one domain’s configuration. Here, |
|
140
|
`/etc/nginx/local/example.com` contains the configuration for |
|
141
|
`*.example.com` and its alias `*.example.net`; and `local/foo.net` |
|
142
|
contains the configuration for `*.foo.net`. |
|
143
|
|
|
144
|
The configuration for our `example.com` web site, stored in |
|
145
|
`/etc/nginx/sites-enabled/local/example.com` is: |
|
146
|
|
|
147
|
---- |
|
148
|
|
|
149
|
server { |
|
150
|
server_name .example.com .example.net ""; |
|
151
|
include local/generic; |
|
152
|
include local/code; |
|
153
|
|
|
154
|
access_log /var/log/nginx/example.com-https-access.log; |
|
155
|
error_log /var/log/nginx/example.com-https-error.log; |
|
156
|
|
|
157
|
# Bypass Fossil for the static documentation generated from |
|
158
|
# our source code by Doxygen, so it merges into the embedded |
|
159
|
# doc URL hierarchy at Fossil’s $ROOT/doc without requiring that |
|
160
|
# these generated files actually be stored in the repo. This |
|
161
|
# also lets us set aggressive caching on these docs, since |
|
162
|
# they rarely change. |
|
163
|
location /code/doc/html { |
|
164
|
root /var/www/example.com/code/doc/html; |
|
165
|
|
|
166
|
location ~* \.(html|ico|css|js|gif|jpg|png)$ { |
|
167
|
add_header Vary Accept-Encoding; |
|
168
|
access_log off; |
|
169
|
expires 7d; |
|
170
|
} |
|
171
|
} |
|
172
|
|
|
173
|
# Redirect everything under /code to the Fossil instance |
|
174
|
location /code { |
|
175
|
include local/code; |
|
176
|
|
|
177
|
# Extended caching for URLs that include unique IDs |
|
178
|
location ~ "/(artifact|doc|file|raw)/[0-9a-f]{40,64}" { |
|
179
|
add_header Cache-Control "public, max-age=31536000, immutable"; |
|
180
|
include local/code; |
|
181
|
access_log off; |
|
182
|
} |
|
183
|
|
|
184
|
# Lesser caching for URLs likely to be quasi-static |
|
185
|
location ~* \.(css|gif|ico|js|jpg|png)$ { |
|
186
|
add_header Vary Accept-Encoding; |
|
187
|
include local/code; |
|
188
|
access_log off; |
|
189
|
expires 7d; |
|
190
|
} |
|
191
|
} |
|
192
|
} |
|
193
|
|
|
194
|
---- |
|
195
|
|
|
196
|
As you can see, this is a pure extension of [the basic nginx service |
|
197
|
configuration for SCGI][scgii], showing off a few ideas you might want to |
|
198
|
try on your own site, such as static asset proxying. |
|
199
|
|
|
200
|
You also need a `local/code` file containing: |
|
201
|
|
|
202
|
include scgi_params; |
|
203
|
scgi_pass 127.0.0.1:12345; |
|
204
|
scgi_param SCRIPT_NAME "/code"; |
|
205
|
|
|
206
|
We separate that out because nginx refuses to inherit certain settings |
|
207
|
between nested location blocks, so rather than repeat them, we extract |
|
208
|
them to this separate file and include it from both locations where it’s |
|
209
|
needed. You see this above where we set far-future expiration dates on |
|
210
|
files served by Fossil via URLs that contain hashes that change when the |
|
211
|
content changes. It tells your browser that the content of these URLs |
|
212
|
can never change without the URL itself changing, which makes your |
|
213
|
Fossil-based site considerably faster. |
|
214
|
|
|
215
|
Similarly, the `local/generic` file referenced above helps us reduce unnecessary |
|
216
|
repetition among the multiple sites this configuration hosts: |
|
217
|
|
|
218
|
root /var/www/$host; |
|
219
|
|
|
220
|
listen 80; |
|
221
|
listen [::]:80; |
|
222
|
|
|
223
|
charset utf-8; |
|
224
|
|
|
225
|
There are some configuration directives that nginx refuses to substitute |
|
226
|
variables into, citing performance considerations, so there is a limit |
|
227
|
to how much repetition you can squeeze out this way. One such example |
|
228
|
are the `access_log` and `error_log` directives, which follow an obvious |
|
229
|
pattern from one host to the next. Sadly, you must tolerate some |
|
230
|
repetition across `server { }` blocks when setting up multiple domains |
|
231
|
on a single server. |
|
232
|
|
|
233
|
The configuration for `foo.net` is similar. |
|
234
|
|
|
235
|
See [the nginx docs](https://nginx.org/en/docs/) for more ideas. |
|
236
|
|
|
237
|
|
|
238
|
## <a id="http"></a>Proxying HTTP Anyway |
|
239
|
|
|
240
|
[Above](#modes), we argued that proxying SCGI is a better option than |
|
241
|
making nginx reinterpret Fossil’s own implementation of HTTP. If you |
|
242
|
want Fossil to speak HTTP, just [set Fossil up as a standalone |
|
243
|
server](../any/none.md). And if you want nginx to [provide TLS |
|
244
|
encryption for Fossil](#tls), proxying HTTP instead of SCGI provides no |
|
245
|
benefit. |
|
246
|
|
|
247
|
However, it is still worth showing the proper method of proxying |
|
248
|
Fossil’s HTTP server through nginx if only to make reading nginx |
|
249
|
documentation on other sites easier: |
|
250
|
|
|
251
|
location /code { |
|
252
|
rewrite ^/code(/.*) $1 break; |
|
253
|
proxy_pass http://127.0.0.1:12345; |
|
254
|
} |
|
255
|
|
|
256
|
The most common thing people get wrong when hand-rolling a configuration |
|
257
|
like this is to get the slashes wrong. Fossil is sensitive to this. For |
|
258
|
instance, Fossil will not collapse double slashes down to a single |
|
259
|
slash, as some other HTTP servers will. |
|
260
|
|
|
261
|
|
|
262
|
## <a id="large-uv"></a> Allowing Large Unversioned Files |
|
263
|
|
|
264
|
By default, nginx only accepts HTTP messages [up to a |
|
265
|
meg](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size) |
|
266
|
in size. Fossil chunks its sync protocol such that this is not normally |
|
267
|
a problem, but when sending [unversioned content][uv], it uses a single |
|
268
|
message for the entire file. Therefore, if you will be storing files |
|
269
|
larger than this limit as unversioned content, you need to raise the |
|
270
|
limit. Within the `location` block: |
|
271
|
|
|
272
|
# Allow large unversioned file uploads, such as PDFs |
|
273
|
client_max_body_size 20M; |
|
274
|
|
|
275
|
[uv]: ../../unvers.wiki |
|
276
|
|
|
277
|
|
|
278
|
## <a id="fail2ban"></a> Integrating `fail2ban` |
|
279
|
|
|
280
|
One of the nice things that falls out of proxying Fossil behind nginx is |
|
281
|
that it makes it easier to configure `fail2ban` to recognize attacks on |
|
282
|
Fossil and automatically block them. Fossil logs the sorts of errors we |
|
283
|
want to detect, but it does so in places like the repository’s admin |
|
284
|
log, a SQL table, which `fail2ban` doesn’t know how to query. By putting |
|
285
|
Fossil behind an nginx proxy, we convert these failures to log file |
|
286
|
form, which `fail2ban` is designed to handle. |
|
287
|
|
|
288
|
First, install `fail2ban`, if you haven’t already: |
|
289
|
|
|
290
|
sudo apt install fail2ban |
|
291
|
|
|
292
|
We’d like `fail2ban` to react to Fossil `/login` failures. The stock |
|
293
|
configuration of `fail2ban` only detects a few common sorts of SSH |
|
294
|
attacks by default, and its included (but disabled) nginx attack |
|
295
|
detectors don’t include one that knows how to detect an attack on |
|
296
|
Fossil. We have to teach it by putting the following into |
|
297
|
`/etc/fail2ban/filter.d/nginx-fossil-login.conf`: |
|
298
|
|
|
299
|
[Definition] |
|
300
|
failregex = ^<HOST> - .*POST .*/login HTTP/..." 401 |
|
301
|
|
|
302
|
That teaches `fail2ban` how to recognize the errors logged by Fossil. |
|
303
|
|
|
304
|
Then in `/etc/fail2ban/jail.local`, add this section: |
|
305
|
|
|
306
|
[nginx-fossil-login] |
|
307
|
enabled = true |
|
308
|
logpath = /var/log/nginx/*-https-access.log |
|
309
|
|
|
310
|
The last line is the key: it tells `fail2ban` where we’ve put all of our |
|
311
|
per-repo access logs in the nginx config above. |
|
312
|
|
|
313
|
There’s a [lot more you can do][dof2b], but that gets us out of scope of |
|
314
|
this guide. |
|
315
|
|
|
316
|
|
|
317
|
[dof2b]: https://www.digitalocean.com/community/tutorials/how-to-protect-an-nginx-server-with-fail2ban-on-ubuntu-14-04 |
|
318
|
|
|
319
|
|
|
320
|
## <a id="tls"></a> Adding TLS (HTTPS) Support |
|
321
|
|
|
322
|
One of the [many ways](../../ssl.wiki) to provide TLS-encrypted HTTP |
|
323
|
access (a.k.a. HTTPS) to Fossil is to run it behind a web proxy that |
|
324
|
supports TLS. Because one such option is nginx, it’s best to delegate |
|
325
|
TLS to it if you were already using nginx for some other reason, such as |
|
326
|
static content serving, with only part of the site being served by |
|
327
|
Fossil. |
|
328
|
|
|
329
|
The simplest way by far to do this is to use [Let’s Encrypt][LE]’s |
|
330
|
[Certbot][CB], which can configure nginx for you and keep its |
|
331
|
certificates up to date. You need but follow their [nginx on Ubuntu 20 |
|
332
|
guide][CBU]. We had trouble with this in the past, but either Certbot |
|
333
|
has gotten smarter or our nginx configurations have gotten simpler, so |
|
334
|
we have removed the manual instructions we used to have here. |
|
335
|
|
|
336
|
You may wish to include something like this from each `server { }` |
|
337
|
block in your configuration to enable TLS in a common, secure way: |
|
338
|
|
|
339
|
``` |
|
340
|
# Tell nginx to accept TLS-encrypted HTTPS on the standard TCP port. |
|
341
|
listen 443 ssl; |
|
342
|
listen [::]:443 ssl; |
|
343
|
|
|
344
|
# Reference the TLS cert files produced by Certbot. |
|
345
|
ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; |
|
346
|
ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; |
|
347
|
|
|
348
|
# Load the Let's Encrypt Diffie-Hellman parameters generated for |
|
349
|
# this server. Without this, the server is vulnerable to Logjam. |
|
350
|
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; |
|
351
|
|
|
352
|
# Tighten things down further, per Qualys’ and Certbot’s advice. |
|
353
|
ssl_session_cache shared:le_nginx_SSL:1m; |
|
354
|
ssl_protocols TLSv1.2 TLSv1.3; |
|
355
|
ssl_prefer_server_ciphers on; |
|
356
|
ssl_session_timeout 1440m; |
|
357
|
|
|
358
|
# Offer OCSP certificate stapling. |
|
359
|
ssl_stapling on; |
|
360
|
ssl_stapling_verify on; |
|
361
|
|
|
362
|
# Enable HSTS. |
|
363
|
include local/enable-hsts; |
|
364
|
``` |
|
365
|
|
|
366
|
The [HSTS] step is optional and should be applied only after due |
|
367
|
consideration, since it has the potential to lock users out of your |
|
368
|
site if you later change your mind on the TLS configuration. |
|
369
|
The `local/enable-hsts` file it references is simply: |
|
370
|
|
|
371
|
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always; |
|
372
|
|
|
373
|
It’s a separate file because nginx requires that headers like this be |
|
374
|
applied separately for each `location { }` block. We’ve therefore |
|
375
|
factored this out so you can `include` it everywhere you need it. |
|
376
|
|
|
377
|
The [OCSP] step is optional, but recommended. |
|
378
|
|
|
379
|
You may find [Qualys’ SSL Server Test][QSLT] helpful in verifying that |
|
380
|
you have set all this up correctly, and that the configuration is |
|
381
|
strong. We’ve found their [best practices doc][QSLC] to be helpful. As |
|
382
|
of this writing, the above configuration yields an A+ rating when run on |
|
383
|
Ubuntu 22.04.01 LTS. |
|
384
|
|
|
385
|
[CB]: https://certbot.eff.org/ |
|
386
|
[CBU]: https://certbot.eff.org/instructions?ws=nginx&os=ubuntufocal |
|
387
|
[LE]: https://letsencrypt.org/ |
|
388
|
[HSTS]: https://www.nginx.com/blog/http-strict-transport-security-hsts-and-nginx/ |
|
389
|
[OCSP]: https://en.wikipedia.org/wiki/OCSP_stapling |
|
390
|
[QSLC]: https://github.com/ssllabs/research/wiki/SSL-and-TLS-Deployment-Best-Practices |
|
391
|
[QSLT]: https://www.ssllabs.com/ssltest/ |
|
392
|
|
|
393
|
<div style="height:50em" id="this-space-intentionally-left-blank"></div> |
|
394
|
|
|
395
|
*[Return to the top-level Fossil server article.](../)* |
|
396
|
|