Fossil SCM

Updated the debian/nginx.md doc for Ubuntu 22.04. The biggie is simplifying the TLS configuration, since the manual method we used to have no longer seems to be required with current versions of Certbot.

wyoung 2022-10-07 23:06 trunk
Commit 716ae7c06994b8ee3f31e90a7128762687f2d88d38729ca6e2b91e193a2fcf45
1 file changed +80 -384
--- www/server/debian/nginx.md
+++ www/server/debian/nginx.md
@@ -3,16 +3,18 @@
33
This document is an extension of [the platform-independent SCGI
44
instructions][scgii], which may suffice for your purposes if your needs
55
are simple.
66
77
Here, we add more detailed information on nginx itself, plus details
8
-about running it on Debian type OSes. We focus on Debian 10 (Buster) and
9
-Ubuntu 20.04 here, which are common Tier 1 OS offerings for [virtual
10
-private servers][vps] at the time of writing. This material may not work for older OSes. It is
8
+about running it on Debian type OSes. This document was originally
9
+written for and tested on Debian 10 (Buster) and Ubuntu 20.04, which
10
+were common Tier 1 OS offerings for [virtual private servers][vps]
11
+at the time. The same configuration appears to run on Ubuntu 22.04
12
+LTS without change. This material may not work for older OSes. It is
1113
known in particular to not work as given for Debian 9 and older!
1214
13
-We also cover adding TLS to the basic configuration, because several
15
+We also cover [adding TLS](#tls) to the basic configuration, because several
1416
details depend on the host OS and web stack details. Besides, TLS is
1517
widely considered part of the baseline configuration these days.
1618
1719
[scgii]: ../any/scgi.md
1820
[vps]: https://en.wikipedia.org/wiki/Virtual_private_server
@@ -285,387 +287,81 @@
285287
[dof2b]: https://www.digitalocean.com/community/tutorials/how-to-protect-an-nginx-server-with-fail2ban-on-ubuntu-14-04
286288
287289
288290
## <a id="tls"></a> Adding TLS (HTTPS) Support
289291
290
-One of the [many ways](../../ssl.wiki) to provide TLS-encrypted HTTP access
291
-(a.k.a. HTTPS) to Fossil is to run it behind a web proxy that supports
292
-TLS. One such option is nginx on Debian, so we show the details of that
293
-here.
294
-
295
-You can extend this guide to other operating systems by following the
296
-instructions found via [the front Certbot web page][cb] instead, telling
297
-it what OS and web stack you’re using. Chances are good that they’ve got
298
-a good guide for you already.
299
-
300
-
301
-### <a id="leew"></a> Configuring Let’s Encrypt, the Easy Way
302
-
303
-If your web serving needs are simple, [Certbot][cb] can configure nginx
304
-for you and keep its certificates up to date. Simply follow Certbot’s
305
-[nginx on Ubuntu 20.04 LTS guide][cbnu].
306
-
307
-Unfortunately, the setup above was beyond Certbot’s ability to cope the
308
-last time we tried it. The use of per-subdomain files in particular
309
-confused Certbot, so we had to [arrange these details manually](#lehw),
310
-else the Let’s Encrypt [ACME] exchange failed in the necessary domain
311
-validation steps.
312
-
313
-At this point, if your configuration needs are simple, needing only a
314
-single Internet domain and a single Fossil repo, you might wish to try
315
-to reduce the above configuration to a more typical single-file nginx
316
-config, which Certbot might then cope with out of the box.
317
-
318
-
319
-
320
-### <a id="lehw"></a> Configuring Let’s Encrypt, the Hard Way
321
-
322
-The primary motivation for this section is that it documents the manual
323
-Certbot configuration on my public Fossil-based site. I’m addressing
324
-the “me” years hence who needs to upgrade to Ubuntu 22.04 or 24.04 LTS
325
-and has forgotten all of this stuff. 😉
326
-
327
-
328
-#### Step 1: Shifting into Manual
329
-
330
-The first thing we’ll do is install Certbot in the normal way, but we’ll
331
-turn off all of the Certbot automation and won’t follow through with use
332
-of the `--nginx` plugin:
333
-
334
- $ sudo snap install --classic certbot
335
- $ sudo systemctl disable certbot.timer
336
-
337
-Next, edit `/etc/letsencrypt/renewal/example.com.conf` to disable the
338
-nginx plugins. You’re looking for two lines setting the “install” and
339
-“auth” plugins to “nginx”. You can comment them out or remove them
340
-entirely.
341
-
342
-
343
-#### Step 2: Configuring nginx
344
-
345
-This is a straightforward extension to the HTTP-only configuration
346
-[above](#config):
347
-
348
- server {
349
- server_name .foo.net;
350
-
351
- include local/tls-common;
352
-
353
- charset utf-8;
354
-
355
- access_log /var/log/nginx/foo.net-https-access.log;
356
- error_log /var/log/nginx/foo.net-https-error.log;
357
-
358
- # Bypass Fossil for the static Doxygen docs
359
- location /doc/html {
360
- root /var/www/foo.net;
361
-
362
- location ~* \.(html|ico|css|js|gif|jpg|png)$ {
363
- expires 7d;
364
- add_header Vary Accept-Encoding;
365
- access_log off;
366
- }
367
- }
368
-
369
- # Redirect everything else to the Fossil instance
370
- location / {
371
- include scgi_params;
372
- scgi_pass 127.0.0.1:12345;
373
- scgi_param HTTPS "on";
374
- scgi_param SCRIPT_NAME "";
375
- }
376
- }
377
- server {
378
- server_name .foo.net;
379
- root /var/www/foo.net;
380
- include local/http-certbot-only;
381
- access_log /var/log/nginx/foo.net-http-access.log;
382
- error_log /var/log/nginx/foo.net-http-error.log;
383
- }
384
-
385
-One big difference between this and the HTTP-only case is
386
-that we need two `server { }` blocks: one for HTTPS service, and
387
-one for HTTP-only service.
388
-
389
-
390
-##### HTTP over TLS (HTTPS) Service
391
-
392
-The first `server { }` block includes this file, `local/tls-common`:
393
-
394
- listen 443 ssl;
395
-
396
- ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
397
- ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
398
-
399
- ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
400
-
401
- ssl_stapling on;
402
- ssl_stapling_verify on;
403
-
404
- ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
405
- ssl_ciphers "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-CBC-SHA:ECDHE-ECDSA-AES256-CBC-SHA:ECDHE-ECDSA-AES128-CBC-SHA256:ECDHE-ECDSA-AES256-CBC-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-CBC-SHA:ECDHE-RSA-AES256-CBC-SHA:ECDHE-RSA-AES128-CBC-SHA256:ECDHE-RSA-AES256-CBC-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-CBC-SHA:DHE-RSA-AES256-CBC-SHA:DHE-RSA-AES128-CBC-SHA256:DHE-RSA-AES256-CBC-SHA256";
406
- ssl_session_cache shared:le_nginx_SSL:1m;
407
- ssl_prefer_server_ciphers on;
408
- ssl_session_timeout 1440m;
409
-
410
-These are the common TLS configuration parameters used by all domains
411
-hosted by this server.
412
-
413
-The first line tells nginx to accept TLS-encrypted HTTP connections on
414
-the standard HTTPS port. It is the same as `listen 443; ssl on;` in
415
-older versions of nginx.
416
-
417
-Since all of those domains share a single TLS certificate, we reference
418
-the same `example.com/*.pem` files written out by Certbot with the
419
-`ssl_certificate*` lines.
420
-
421
-The `ssl_dhparam` directive isn’t strictly required, but without it, the
422
-server becomes vulnerable to the [Logjam attack][lja] because some of
423
-the cryptography steps are precomputed, making the attacker’s job much
424
-easier. The parameter file this directive references should be
425
-generated automatically by the Let’s Encrypt package upon installation,
426
-making those parameters unique to your server and thus unguessable. If
427
-the file doesn’t exist on your system, you can create it manually, so:
428
-
429
- $ sudo openssl dhparam -out /etc/letsencrypt/dhparams.pem 2048
430
-
431
-Beware, this can take a long time. On a shared Linux host I tried it on
432
-running OpenSSL 1.1.0g, it took about 21 seconds, but on a fast, idle
433
-iMac running LibreSSL 2.6.5, it took 8 minutes and 4 seconds!
434
-
435
-The next section is also optional. It enables [OCSP stapling][ocsp], a
436
-protocol that improves the speed and security of the TLS connection
437
-negotiation.
438
-
439
-The next section containing the `ssl_protocols` and `ssl_ciphers` lines
440
-restricts the TLS implementation to only those protocols and ciphers
441
-that are currently believed to be safe and secure. This section is the
442
-one most prone to bit-rot: as new attacks on TLS and its associated
443
-technologies are discovered, this configuration is likely to need to
444
-change. Even if we fully succeed in keeping this document up-to-date in
445
-the face of the evolving security landscape, we’re recommending static
446
-configurations for your server: it will thus be up to you to track
447
-changes in this document and others to merge the changes into your local
448
-static configuration.
449
-
450
-Running a TLS certificate checker against your site occasionally is a
451
-good idea. The most thorough service I’m aware of is the [Qualys SSL
452
-Labs Test][qslt], which gives the site I’m basing this guide on an “A+”
453
-rating at the time of this writing. The long `ssl_ciphers` line above is
454
-based on [their advice][qslc]: the default nginx configuration tells
455
-OpenSSL to use whatever ciphersuites it considers “high security,” but
456
-some of those have come to be considered “weak” in the time between that
457
-judgement and the time of this writing. By explicitly giving the list of
458
-ciphersuites we want OpenSSL to use within nginx, we can remove those
459
-that become considered weak in the future.
460
-
461
-<a id=”hsts”></a>There are a few things you can do to get an even better
462
-grade, such as to enable [HSTS][hsts]:
463
-
464
- add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
465
-
466
-This prevents a particular variety of [man in the middle attack][mitm]
467
-where our HTTP-to-HTTPS permanent redirect is intercepted, allowing the
468
-attacker to prevent the automatic upgrade of the connection to a secure
469
-TLS-encrypted one. I didn’t enable that in the configuration above
470
-because it is something a site administrator should enable only after
471
-the configuration is tested and stable, and then only after due
472
-consideration. There are ways to lock your users out of your site by
473
-jumping to HSTS hastily. When you’re ready, there are [guides you can
474
-follow][nest] elsewhere online.
475
-
476
-
477
-##### HTTP-Only Service
478
-
479
-While we’d prefer not to offer HTTP service at all, we need to do so for
480
-two reasons:
481
-
482
-* The temporary reason is that until we get Let’s Encrypt certificates
483
- minted and configured properly, we can’t use HTTPS yet at all.
484
-
485
-* The ongoing reason is that the Certbot [ACME][acme] HTTP-01
486
- challenge used by the Let’s Encrypt service only runs over HTTP. This is
487
- not only because it has to work before HTTPS is first configured,
488
- but also because it might need to work after a certificate is
489
- accidentally allowed to lapse to get that server back into a state
490
- where it can speak HTTPS safely again.
491
-
492
-So, from the second `service { }` block, we include this file to set up
493
-the minimal HTTP service we require, `local/http-certbot-only`:
494
-
495
- listen 80;
496
- listen [::]:80;
497
-
498
- # This is expressed as a rewrite rule instead of an "if" because
499
- # http://wiki.nginx.org/IfIsEvil
500
- #rewrite ^(/.well-known/acme-challenge/.*) $1 break;
501
-
502
- # Force everything else to HTTPS with a permanent redirect.
503
- #return 301 https://$host$request_uri;
504
-
505
-As written above, this configuration does nothing other than to tell
506
-nginx that it’s allowed to serve content via HTTP on port 80 as well.
507
-We’ll uncomment the `rewrite` and `return` directives below, when we’re
508
-ready to begin testing.
509
-
510
-Notice that most of the nginx directives given [above](#config) moved up
511
-into the TLS `server { }` block, because we eventually want this site to
512
-be as close to HTTPS-only as we can get it.
513
-
514
-
515
-#### Step 3: Dry Run
516
-
517
-We want to first request a dry run, because Let’s Encrypt puts some
518
-rather low limits on how often you’re allowed to request an actual
519
-certificate. You want to be sure everything’s working before you do
520
-that. You’ll run a command something like this:
521
-
522
- $ sudo certbot certonly --webroot --dry-run \
523
- --webroot-path /var/www/example.com \
524
- -d example.com -d www.example.com \
525
- -d example.net -d www.example.net \
526
- --webroot-path /var/www/foo.net \
527
- -d foo.net -d www.foo.net
528
-
529
-There are two key options here.
530
-
531
-First, we’re telling Certbot to use its `--webroot` plugin instead of
532
-the automated `--nginx` plugin. With this plugin, Certbot writes the
533
-[ACME][acme] HTTP-01 challenge files to the static web document root
534
-directory behind each domain. For this example, we’ve got two web
535
-roots, one of which holds documents for two different second-level
536
-domains (`example.com` and `example.net`) with `www` at the third level
537
-being optional. This is a common sort of configuration these days, but
538
-you needn’t feel that you must slavishly imitate it. The other web root
539
-is for an entirely different domain, also with `www` being optional.
540
-Since all of these domains are served by a single nginx instance, we
541
-need to give all of this in a single command, because we want to mint a
542
-single certificate that authenticates all of these domains.
543
-
544
-The second key option is `--dry-run`, which tells Certbot not to do
545
-anything permanent. We’re just seeing if everything works as expected,
546
-at this point.
547
-
548
-
549
-##### Troubleshooting the Dry Run
550
-
551
-If that didn’t work, try creating a manual test:
552
-
553
- $ mkdir -p /var/www/example.com/.well-known/acme-challenge
554
- $ echo hi > /var/www/example.com/.well-known/acme-challenge/test
555
-
556
-Then try to pull that file over HTTP — not HTTPS! — as
557
-`http://example.com/.well-known/acme-challenge/test`. I’ve found that
558
-using Firefox or Safari is better for this sort of thing than Chrome,
559
-because Chrome is more aggressive about automatically forwarding URLs to
560
-HTTPS even if you requested “`http`”.
561
-
562
-In extremis, you can do the test manually:
563
-
564
- $ curl -i http://example.com/.well-known/acme-challenge/test
565
- HTTP/1.1 200 OK
566
- Server: nginx/1.14.0 (Ubuntu)
567
- Date: Sat, 19 Jan 2019 19:43:58 GMT
568
- Content-Type: application/octet-stream
569
- Content-Length: 3
570
- Last-Modified: Sat, 19 Jan 2019 18:21:54 GMT
571
- Connection: keep-alive
572
- ETag: "5c436ac2-4"
573
- Accept-Ranges: bytes
574
-
575
- hi
576
-
577
-The key bits you’re looking for here are the “200 OK” response code at
578
-the start and the “hi” line at the end. (Or whatever you wrote in to the
579
-test file.)
580
-
581
-If you get a 301 redirect to an `https://` URI, you either haven’t
582
-uncommented the `rewrite` line for HTTP-only service for this directory,
583
-or there’s some other problem with the “redirect to HTTPS” config.
584
-
585
-If you get a 404 or other error response, you need to look into your web
586
-server logs to find out what’s going wrong.
587
-
588
-If you’re still running into trouble, the log file written by Certbot
589
-can be helpful. It tells you where it’s writing the ACME files early in
590
-each run.
591
-
592
-
593
-
594
-#### Step 4: Getting Your First Certificate
595
-
596
-Once the dry run is working, you can drop the `--dry-run` option and
597
-re-run the long command above. (The one with all the `--webroot*`
598
-flags.) This should now succeed, and it will save all of those flag
599
-values to your Let’s Encrypt configuration file, so you don’t need to
600
-keep giving them.
601
-
602
-
603
-
604
-#### Step 5: Test It
605
-
606
-Edit the `local/http-certbot-only` file and uncomment the `redirect` and
607
-`return` directives, then restart your nginx server and make sure it now
608
-forces everything to HTTPS like it should:
609
-
610
- $ sudo systemctl restart nginx
611
-
612
-Test ideas:
613
-
614
-* Visit both Fossil and non-Fossil URLs
615
-
616
-* Log into the repo, log out, and log back in
617
-
618
-* Clone via `http`: ensure that it redirects to `https`, and that
619
- subsequent `fossil sync` commands go directly to `https` due to the
620
- 301 permanent redirect.
621
-
622
-This forced redirect is why we don’t need the Fossil Admin &rarr; Access
623
-"Redirect to HTTPS on the Login page" setting to be enabled. Not only
624
-is it unnecessary with this HTTPS redirect at the front-end proxy level,
625
-it would actually [cause an infinite redirect loop if
626
-enabled](./ssl.wiki#rloop).
627
-
628
-
629
-
630
-#### Step 6: Switch to HTTPS Sync
631
-
632
-Fossil remembers permanent HTTP-to-HTTPS redirects on sync since version
633
-2.9, so all you need to do to switch your syncs to HTTPS is:
634
-
635
- $ fossil sync -R /path/to/repo.fossil
636
-
637
-
638
-#### Step 7: Renewing Automatically
639
-
640
-Now that the configuration is solid, you can renew the LE cert with the
641
-`certbot` command from above without the `--dry-run` flag plus a restart
642
-of nginx:
643
-
644
- sudo certbot certonly --webroot \
645
- --webroot-path /var/www/example.com \
646
- -d example.com -d www.example.com \
647
- -d example.net -d www.example.net \
648
- --webroot-path /var/www/foo.net \
649
- -d foo.net -d www.foo.net
650
- sudo systemctl restart nginx
651
-
652
-I put those commands in a script in the `PATH`, then arrange to call that
653
-periodically. Let’s Encrypt doesn’t let you renew the certificate very
654
-often unless forced, and when forced there’s a maximum renewal counter.
655
-Nevertheless, some people recommend running this daily and just letting
656
-it fail until the server lets you renew. Others arrange to run it no
657
-more often than it’s known to work without complaint. Suit yourself.
658
-
659
-
660
-[acme]: https://en.wikipedia.org/wiki/Automated_Certificate_Management_Environment
661
-[cb]: https://certbot.eff.org/
662
-[cbnu]: https://certbot.eff.org/lets-encrypt/ubuntufocal-nginx
663
-[hsts]: https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security
664
-[lja]: https://en.wikipedia.org/wiki/Logjam_(computer_security)
665
-[mitm]: https://en.wikipedia.org/wiki/Man-in-the-middle_attack
666
-[nest]: https://www.nginx.com/blog/http-strict-transport-security-hsts-and-nginx/
667
-[ocsp]: https://en.wikipedia.org/wiki/OCSP_stapling
668
-[qslc]: https://github.com/ssllabs/research/wiki/SSL-and-TLS-Deployment-Best-Practices
669
-[qslt]: https://www.ssllabs.com/ssltest/
292
+One of the [many ways](../../ssl.wiki) to provide TLS-encrypted HTTP
293
+access (a.k.a. HTTPS) to Fossil is to run it behind a web proxy that
294
+supports TLS. Because one such option is nginx, it’s best to delegate
295
+TLS to it if you were already using nginx for some other reason, such as
296
+static content serving, with only part of the site being served by
297
+Fossil.
298
+
299
+The simplest way by far to do this is to use [Let’s Encrypt][LE]’s
300
+[Certbot][CB], which can configure nginx for you and keep its
301
+certificates up to date. You need but follow their [nginx on Ubuntu 20
302
+guide][CBU]. We had trouble with this in the past, but either Certbot
303
+has gotten smarter or our nginx configurations have gotten simpler, so
304
+we have removed the manual instructions we used to have here.
305
+
306
+You may wish to include something like this from each `server { }`
307
+block in your configuration to enable TLS in a common, secure way:
308
+
309
+```
310
+ # Tell nginx to accept TLS-encrypted HTTPS on the standard TCP port.
311
+ listen 443 ssl;
312
+ listen [::]:443 ssl;
313
+
314
+ # Reference the TLS cert files produced by Certbot.
315
+ ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
316
+ ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
317
+
318
+ # Load the Let's Encrypt Diffie-Hellman parameters generated for
319
+ # this server. Without this, the server is vulnerable to Logjam.
320
+ ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
321
+
322
+ # Tighten things down further, per Qualys’ and Certbot’s advice.
323
+ ssl_session_cache shared:le_nginx_SSL:1m;
324
+ ssl_protocols TLSv1.2 TLSv1.3;
325
+ ssl_prefer_server_ciphers on;
326
+ ssl_session_timeout 1440m;
327
+
328
+ # Offer OCSP certificate stapling.
329
+ ssl_stapling on;
330
+ ssl_stapling_verify on;
331
+
332
+ # Enable HSTS.
333
+ include local/enable-hsts;
334
+```
335
+
336
+The [HSTS] step is optional and should be applied only after due
337
+consideration, since it has the potential to lock users out of your
338
+site if you later change your mind on the TLS configuration.
339
+The `local/enable-hsts` file it references is simply:
340
+
341
+```
342
+ add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
343
+```
344
+
345
+It’s a separate file because nginx requires that headers like this be
346
+applied separately for each `location { }` block. We’ve therefore
347
+factored this out so you can `include` it everywhere you need it.
348
+
349
+The [OCSP] step is optional, but recommended.
350
+
351
+You may find [Qualys’ SSL Server Test][QSLT] helpful in verifying that
352
+you have set all this up correctly, and that the configuration is
353
+strong. We’ve found their [best practices doc][QSLC] to be helpful. As
354
+of this writing, the above configuration yields an A+ rating when run on
355
+Ubuntu 22.04.01 LTS.
356
+
357
+[CB]: https://certbot.eff.org/
358
+[CBU]: https://certbot.eff.org/instructions?ws=nginx&os=ubuntufocal
359
+[LE]: https://letsencrypt.org/
360
+[HSTS]: https://www.nginx.com/blog/http-strict-transport-security-hsts-and-nginx/
361
+[OCSP]: https://en.wikipedia.org/wiki/OCSP_stapling
362
+[QSLC]: https://github.com/ssllabs/research/wiki/SSL-and-TLS-Deployment-Best-Practices
363
+[QSLT]: https://www.ssllabs.com/ssltest/
364
+
365
+<div style="height:50em" id="this-space-intentionally-left-blank"></div>
670366
671367
*[Return to the top-level Fossil server article.](../)*
672368
--- www/server/debian/nginx.md
+++ www/server/debian/nginx.md
@@ -3,16 +3,18 @@
3 This document is an extension of [the platform-independent SCGI
4 instructions][scgii], which may suffice for your purposes if your needs
5 are simple.
6
7 Here, we add more detailed information on nginx itself, plus details
8 about running it on Debian type OSes. We focus on Debian 10 (Buster) and
9 Ubuntu 20.04 here, which are common Tier 1 OS offerings for [virtual
10 private servers][vps] at the time of writing. This material may not work for older OSes. It is
 
 
11 known in particular to not work as given for Debian 9 and older!
12
13 We also cover adding TLS to the basic configuration, because several
14 details depend on the host OS and web stack details. Besides, TLS is
15 widely considered part of the baseline configuration these days.
16
17 [scgii]: ../any/scgi.md
18 [vps]: https://en.wikipedia.org/wiki/Virtual_private_server
@@ -285,387 +287,81 @@
285 [dof2b]: https://www.digitalocean.com/community/tutorials/how-to-protect-an-nginx-server-with-fail2ban-on-ubuntu-14-04
286
287
288 ## <a id="tls"></a> Adding TLS (HTTPS) Support
289
290 One of the [many ways](../../ssl.wiki) to provide TLS-encrypted HTTP access
291 (a.k.a. HTTPS) to Fossil is to run it behind a web proxy that supports
292 TLS. One such option is nginx on Debian, so we show the details of that
293 here.
294
295 You can extend this guide to other operating systems by following the
296 instructions found via [the front Certbot web page][cb] instead, telling
297 it what OS and web stack you’re using. Chances are good that they’ve got
298 a good guide for you already.
299
300
301 ### <a id="leew"></a> Configuring Let’s Encrypt, the Easy Way
302
303 If your web serving needs are simple, [Certbot][cb] can configure nginx
304 for you and keep its certificates up to date. Simply follow Certbot’s
305 [nginx on Ubuntu 20.04 LTS guide][cbnu].
306
307 Unfortunately, the setup above was beyond Certbot’s ability to cope the
308 last time we tried it. The use of per-subdomain files in particular
309 confused Certbot, so we had to [arrange these details manually](#lehw),
310 else the Let’s Encrypt [ACME] exchange failed in the necessary domain
311 validation steps.
312
313 At this point, if your configuration needs are simple, needing only a
314 single Internet domain and a single Fossil repo, you might wish to try
315 to reduce the above configuration to a more typical single-file nginx
316 config, which Certbot might then cope with out of the box.
317
318
319
320 ### <a id="lehw"></a> Configuring Let’s Encrypt, the Hard Way
321
322 The primary motivation for this section is that it documents the manual
323 Certbot configuration on my public Fossil-based site. I’m addressing
324 the “me” years hence who needs to upgrade to Ubuntu 22.04 or 24.04 LTS
325 and has forgotten all of this stuff. 😉
326
327
328 #### Step 1: Shifting into Manual
329
330 The first thing we’ll do is install Certbot in the normal way, but we’ll
331 turn off all of the Certbot automation and won’t follow through with use
332 of the `--nginx` plugin:
333
334 $ sudo snap install --classic certbot
335 $ sudo systemctl disable certbot.timer
336
337 Next, edit `/etc/letsencrypt/renewal/example.com.conf` to disable the
338 nginx plugins. You’re looking for two lines setting the “install” and
339 “auth” plugins to “nginx”. You can comment them out or remove them
340 entirely.
341
342
343 #### Step 2: Configuring nginx
344
345 This is a straightforward extension to the HTTP-only configuration
346 [above](#config):
347
348 server {
349 server_name .foo.net;
350
351 include local/tls-common;
352
353 charset utf-8;
354
355 access_log /var/log/nginx/foo.net-https-access.log;
356 error_log /var/log/nginx/foo.net-https-error.log;
357
358 # Bypass Fossil for the static Doxygen docs
359 location /doc/html {
360 root /var/www/foo.net;
361
362 location ~* \.(html|ico|css|js|gif|jpg|png)$ {
363 expires 7d;
364 add_header Vary Accept-Encoding;
365 access_log off;
366 }
367 }
368
369 # Redirect everything else to the Fossil instance
370 location / {
371 include scgi_params;
372 scgi_pass 127.0.0.1:12345;
373 scgi_param HTTPS "on";
374 scgi_param SCRIPT_NAME "";
375 }
376 }
377 server {
378 server_name .foo.net;
379 root /var/www/foo.net;
380 include local/http-certbot-only;
381 access_log /var/log/nginx/foo.net-http-access.log;
382 error_log /var/log/nginx/foo.net-http-error.log;
383 }
384
385 One big difference between this and the HTTP-only case is
386 that we need two `server { }` blocks: one for HTTPS service, and
387 one for HTTP-only service.
388
389
390 ##### HTTP over TLS (HTTPS) Service
391
392 The first `server { }` block includes this file, `local/tls-common`:
393
394 listen 443 ssl;
395
396 ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
397 ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
398
399 ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
400
401 ssl_stapling on;
402 ssl_stapling_verify on;
403
404 ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
405 ssl_ciphers "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-CBC-SHA:ECDHE-ECDSA-AES256-CBC-SHA:ECDHE-ECDSA-AES128-CBC-SHA256:ECDHE-ECDSA-AES256-CBC-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-CBC-SHA:ECDHE-RSA-AES256-CBC-SHA:ECDHE-RSA-AES128-CBC-SHA256:ECDHE-RSA-AES256-CBC-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-CBC-SHA:DHE-RSA-AES256-CBC-SHA:DHE-RSA-AES128-CBC-SHA256:DHE-RSA-AES256-CBC-SHA256";
406 ssl_session_cache shared:le_nginx_SSL:1m;
407 ssl_prefer_server_ciphers on;
408 ssl_session_timeout 1440m;
409
410 These are the common TLS configuration parameters used by all domains
411 hosted by this server.
412
413 The first line tells nginx to accept TLS-encrypted HTTP connections on
414 the standard HTTPS port. It is the same as `listen 443; ssl on;` in
415 older versions of nginx.
416
417 Since all of those domains share a single TLS certificate, we reference
418 the same `example.com/*.pem` files written out by Certbot with the
419 `ssl_certificate*` lines.
420
421 The `ssl_dhparam` directive isn’t strictly required, but without it, the
422 server becomes vulnerable to the [Logjam attack][lja] because some of
423 the cryptography steps are precomputed, making the attacker’s job much
424 easier. The parameter file this directive references should be
425 generated automatically by the Let’s Encrypt package upon installation,
426 making those parameters unique to your server and thus unguessable. If
427 the file doesn’t exist on your system, you can create it manually, so:
428
429 $ sudo openssl dhparam -out /etc/letsencrypt/dhparams.pem 2048
430
431 Beware, this can take a long time. On a shared Linux host I tried it on
432 running OpenSSL 1.1.0g, it took about 21 seconds, but on a fast, idle
433 iMac running LibreSSL 2.6.5, it took 8 minutes and 4 seconds!
434
435 The next section is also optional. It enables [OCSP stapling][ocsp], a
436 protocol that improves the speed and security of the TLS connection
437 negotiation.
438
439 The next section containing the `ssl_protocols` and `ssl_ciphers` lines
440 restricts the TLS implementation to only those protocols and ciphers
441 that are currently believed to be safe and secure. This section is the
442 one most prone to bit-rot: as new attacks on TLS and its associated
443 technologies are discovered, this configuration is likely to need to
444 change. Even if we fully succeed in keeping this document up-to-date in
445 the face of the evolving security landscape, we’re recommending static
446 configurations for your server: it will thus be up to you to track
447 changes in this document and others to merge the changes into your local
448 static configuration.
449
450 Running a TLS certificate checker against your site occasionally is a
451 good idea. The most thorough service I’m aware of is the [Qualys SSL
452 Labs Test][qslt], which gives the site I’m basing this guide on an “A+”
453 rating at the time of this writing. The long `ssl_ciphers` line above is
454 based on [their advice][qslc]: the default nginx configuration tells
455 OpenSSL to use whatever ciphersuites it considers “high security,” but
456 some of those have come to be considered “weak” in the time between that
457 judgement and the time of this writing. By explicitly giving the list of
458 ciphersuites we want OpenSSL to use within nginx, we can remove those
459 that become considered weak in the future.
460
461 <a id=”hsts”></a>There are a few things you can do to get an even better
462 grade, such as to enable [HSTS][hsts]:
463
464 add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
465
466 This prevents a particular variety of [man in the middle attack][mitm]
467 where our HTTP-to-HTTPS permanent redirect is intercepted, allowing the
468 attacker to prevent the automatic upgrade of the connection to a secure
469 TLS-encrypted one. I didn’t enable that in the configuration above
470 because it is something a site administrator should enable only after
471 the configuration is tested and stable, and then only after due
472 consideration. There are ways to lock your users out of your site by
473 jumping to HSTS hastily. When you’re ready, there are [guides you can
474 follow][nest] elsewhere online.
475
476
477 ##### HTTP-Only Service
478
479 While we’d prefer not to offer HTTP service at all, we need to do so for
480 two reasons:
481
482 * The temporary reason is that until we get Let’s Encrypt certificates
483 minted and configured properly, we can’t use HTTPS yet at all.
484
485 * The ongoing reason is that the Certbot [ACME][acme] HTTP-01
486 challenge used by the Let’s Encrypt service only runs over HTTP. This is
487 not only because it has to work before HTTPS is first configured,
488 but also because it might need to work after a certificate is
489 accidentally allowed to lapse to get that server back into a state
490 where it can speak HTTPS safely again.
491
492 So, from the second `service { }` block, we include this file to set up
493 the minimal HTTP service we require, `local/http-certbot-only`:
494
495 listen 80;
496 listen [::]:80;
497
498 # This is expressed as a rewrite rule instead of an "if" because
499 # http://wiki.nginx.org/IfIsEvil
500 #rewrite ^(/.well-known/acme-challenge/.*) $1 break;
501
502 # Force everything else to HTTPS with a permanent redirect.
503 #return 301 https://$host$request_uri;
504
505 As written above, this configuration does nothing other than to tell
506 nginx that it’s allowed to serve content via HTTP on port 80 as well.
507 We’ll uncomment the `rewrite` and `return` directives below, when we’re
508 ready to begin testing.
509
510 Notice that most of the nginx directives given [above](#config) moved up
511 into the TLS `server { }` block, because we eventually want this site to
512 be as close to HTTPS-only as we can get it.
513
514
515 #### Step 3: Dry Run
516
517 We want to first request a dry run, because Let’s Encrypt puts some
518 rather low limits on how often you’re allowed to request an actual
519 certificate. You want to be sure everything’s working before you do
520 that. You’ll run a command something like this:
521
522 $ sudo certbot certonly --webroot --dry-run \
523 --webroot-path /var/www/example.com \
524 -d example.com -d www.example.com \
525 -d example.net -d www.example.net \
526 --webroot-path /var/www/foo.net \
527 -d foo.net -d www.foo.net
528
529 There are two key options here.
530
531 First, we’re telling Certbot to use its `--webroot` plugin instead of
532 the automated `--nginx` plugin. With this plugin, Certbot writes the
533 [ACME][acme] HTTP-01 challenge files to the static web document root
534 directory behind each domain. For this example, we’ve got two web
535 roots, one of which holds documents for two different second-level
536 domains (`example.com` and `example.net`) with `www` at the third level
537 being optional. This is a common sort of configuration these days, but
538 you needn’t feel that you must slavishly imitate it. The other web root
539 is for an entirely different domain, also with `www` being optional.
540 Since all of these domains are served by a single nginx instance, we
541 need to give all of this in a single command, because we want to mint a
542 single certificate that authenticates all of these domains.
543
544 The second key option is `--dry-run`, which tells Certbot not to do
545 anything permanent. We’re just seeing if everything works as expected,
546 at this point.
547
548
549 ##### Troubleshooting the Dry Run
550
551 If that didn’t work, try creating a manual test:
552
553 $ mkdir -p /var/www/example.com/.well-known/acme-challenge
554 $ echo hi > /var/www/example.com/.well-known/acme-challenge/test
555
556 Then try to pull that file over HTTP — not HTTPS! — as
557 `http://example.com/.well-known/acme-challenge/test`. I’ve found that
558 using Firefox or Safari is better for this sort of thing than Chrome,
559 because Chrome is more aggressive about automatically forwarding URLs to
560 HTTPS even if you requested “`http`”.
561
562 In extremis, you can do the test manually:
563
564 $ curl -i http://example.com/.well-known/acme-challenge/test
565 HTTP/1.1 200 OK
566 Server: nginx/1.14.0 (Ubuntu)
567 Date: Sat, 19 Jan 2019 19:43:58 GMT
568 Content-Type: application/octet-stream
569 Content-Length: 3
570 Last-Modified: Sat, 19 Jan 2019 18:21:54 GMT
571 Connection: keep-alive
572 ETag: "5c436ac2-4"
573 Accept-Ranges: bytes
574
575 hi
576
577 The key bits you’re looking for here are the “200 OK” response code at
578 the start and the “hi” line at the end. (Or whatever you wrote in to the
579 test file.)
580
581 If you get a 301 redirect to an `https://` URI, you either haven’t
582 uncommented the `rewrite` line for HTTP-only service for this directory,
583 or there’s some other problem with the “redirect to HTTPS” config.
584
585 If you get a 404 or other error response, you need to look into your web
586 server logs to find out what’s going wrong.
587
588 If you’re still running into trouble, the log file written by Certbot
589 can be helpful. It tells you where it’s writing the ACME files early in
590 each run.
591
592
593
594 #### Step 4: Getting Your First Certificate
595
596 Once the dry run is working, you can drop the `--dry-run` option and
597 re-run the long command above. (The one with all the `--webroot*`
598 flags.) This should now succeed, and it will save all of those flag
599 values to your Let’s Encrypt configuration file, so you don’t need to
600 keep giving them.
601
602
603
604 #### Step 5: Test It
605
606 Edit the `local/http-certbot-only` file and uncomment the `redirect` and
607 `return` directives, then restart your nginx server and make sure it now
608 forces everything to HTTPS like it should:
609
610 $ sudo systemctl restart nginx
611
612 Test ideas:
613
614 * Visit both Fossil and non-Fossil URLs
615
616 * Log into the repo, log out, and log back in
617
618 * Clone via `http`: ensure that it redirects to `https`, and that
619 subsequent `fossil sync` commands go directly to `https` due to the
620 301 permanent redirect.
621
622 This forced redirect is why we don’t need the Fossil Admin &rarr; Access
623 "Redirect to HTTPS on the Login page" setting to be enabled. Not only
624 is it unnecessary with this HTTPS redirect at the front-end proxy level,
625 it would actually [cause an infinite redirect loop if
626 enabled](./ssl.wiki#rloop).
627
628
629
630 #### Step 6: Switch to HTTPS Sync
631
632 Fossil remembers permanent HTTP-to-HTTPS redirects on sync since version
633 2.9, so all you need to do to switch your syncs to HTTPS is:
634
635 $ fossil sync -R /path/to/repo.fossil
636
637
638 #### Step 7: Renewing Automatically
639
640 Now that the configuration is solid, you can renew the LE cert with the
641 `certbot` command from above without the `--dry-run` flag plus a restart
642 of nginx:
643
644 sudo certbot certonly --webroot \
645 --webroot-path /var/www/example.com \
646 -d example.com -d www.example.com \
647 -d example.net -d www.example.net \
648 --webroot-path /var/www/foo.net \
649 -d foo.net -d www.foo.net
650 sudo systemctl restart nginx
651
652 I put those commands in a script in the `PATH`, then arrange to call that
653 periodically. Let’s Encrypt doesn’t let you renew the certificate very
654 often unless forced, and when forced there’s a maximum renewal counter.
655 Nevertheless, some people recommend running this daily and just letting
656 it fail until the server lets you renew. Others arrange to run it no
657 more often than it’s known to work without complaint. Suit yourself.
658
659
660 [acme]: https://en.wikipedia.org/wiki/Automated_Certificate_Management_Environment
661 [cb]: https://certbot.eff.org/
662 [cbnu]: https://certbot.eff.org/lets-encrypt/ubuntufocal-nginx
663 [hsts]: https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security
664 [lja]: https://en.wikipedia.org/wiki/Logjam_(computer_security)
665 [mitm]: https://en.wikipedia.org/wiki/Man-in-the-middle_attack
666 [nest]: https://www.nginx.com/blog/http-strict-transport-security-hsts-and-nginx/
667 [ocsp]: https://en.wikipedia.org/wiki/OCSP_stapling
668 [qslc]: https://github.com/ssllabs/research/wiki/SSL-and-TLS-Deployment-Best-Practices
669 [qslt]: https://www.ssllabs.com/ssltest/
670
671 *[Return to the top-level Fossil server article.](../)*
672
--- www/server/debian/nginx.md
+++ www/server/debian/nginx.md
@@ -3,16 +3,18 @@
3 This document is an extension of [the platform-independent SCGI
4 instructions][scgii], which may suffice for your purposes if your needs
5 are simple.
6
7 Here, we add more detailed information on nginx itself, plus details
8 about running it on Debian type OSes. This document was originally
9 written for and tested on Debian 10 (Buster) and Ubuntu 20.04, which
10 were common Tier 1 OS offerings for [virtual private servers][vps]
11 at the time. The same configuration appears to run on Ubuntu 22.04
12 LTS without change. This material may not work for older OSes. It is
13 known in particular to not work as given for Debian 9 and older!
14
15 We also cover [adding TLS](#tls) to the basic configuration, because several
16 details depend on the host OS and web stack details. Besides, TLS is
17 widely considered part of the baseline configuration these days.
18
19 [scgii]: ../any/scgi.md
20 [vps]: https://en.wikipedia.org/wiki/Virtual_private_server
@@ -285,387 +287,81 @@
287 [dof2b]: https://www.digitalocean.com/community/tutorials/how-to-protect-an-nginx-server-with-fail2ban-on-ubuntu-14-04
288
289
290 ## <a id="tls"></a> Adding TLS (HTTPS) Support
291
292 One of the [many ways](../../ssl.wiki) to provide TLS-encrypted HTTP
293 access (a.k.a. HTTPS) to Fossil is to run it behind a web proxy that
294 supports TLS. Because one such option is nginx, it’s best to delegate
295 TLS to it if you were already using nginx for some other reason, such as
296 static content serving, with only part of the site being served by
297 Fossil.
298
299 The simplest way by far to do this is to use [Let’s Encrypt][LE]’s
300 [Certbot][CB], which can configure nginx for you and keep its
301 certificates up to date. You need but follow their [nginx on Ubuntu 20
302 guide][CBU]. We had trouble with this in the past, but either Certbot
303 has gotten smarter or our nginx configurations have gotten simpler, so
304 we have removed the manual instructions we used to have here.
305
306 You may wish to include something like this from each `server { }`
307 block in your configuration to enable TLS in a common, secure way:
308
309 ```
310 # Tell nginx to accept TLS-encrypted HTTPS on the standard TCP port.
311 listen 443 ssl;
312 listen [::]:443 ssl;
313
314 # Reference the TLS cert files produced by Certbot.
315 ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
316 ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
317
318 # Load the Let's Encrypt Diffie-Hellman parameters generated for
319 # this server. Without this, the server is vulnerable to Logjam.
320 ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
321
322 # Tighten things down further, per Qualys’ and Certbot’s advice.
323 ssl_session_cache shared:le_nginx_SSL:1m;
324 ssl_protocols TLSv1.2 TLSv1.3;
325 ssl_prefer_server_ciphers on;
326 ssl_session_timeout 1440m;
327
328 # Offer OCSP certificate stapling.
329 ssl_stapling on;
330 ssl_stapling_verify on;
331
332 # Enable HSTS.
333 include local/enable-hsts;
334 ```
335
336 The [HSTS] step is optional and should be applied only after due
337 consideration, since it has the potential to lock users out of your
338 site if you later change your mind on the TLS configuration.
339 The `local/enable-hsts` file it references is simply:
340
341 ```
342 add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
343 ```
344
345 It’s a separate file because nginx requires that headers like this be
346 applied separately for each `location { }` block. We’ve therefore
347 factored this out so you can `include` it everywhere you need it.
348
349 The [OCSP] step is optional, but recommended.
350
351 You may find [Qualys’ SSL Server Test][QSLT] helpful in verifying that
352 you have set all this up correctly, and that the configuration is
353 strong. We’ve found their [best practices doc][QSLC] to be helpful. As
354 of this writing, the above configuration yields an A+ rating when run on
355 Ubuntu 22.04.01 LTS.
356
357 [CB]: https://certbot.eff.org/
358 [CBU]: https://certbot.eff.org/instructions?ws=nginx&os=ubuntufocal
359 [LE]: https://letsencrypt.org/
360 [HSTS]: https://www.nginx.com/blog/http-strict-transport-security-hsts-and-nginx/
361 [OCSP]: https://en.wikipedia.org/wiki/OCSP_stapling
362 [QSLC]: https://github.com/ssllabs/research/wiki/SSL-and-TLS-Deployment-Best-Practices
363 [QSLT]: https://www.ssllabs.com/ssltest/
364
365 <div style="height:50em" id="this-space-intentionally-left-blank"></div>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
366
367 *[Return to the top-level Fossil server article.](../)*
368

Keyboard Shortcuts

Open search /
Next entry (timeline) j
Previous entry (timeline) k
Open focused entry Enter
Show this help ?
Toggle theme Top nav button