|
1
|
<title>Hash Policy</title> |
|
2
|
|
|
3
|
<h2>Executive Summary</h2> |
|
4
|
|
|
5
|
<b>Or: How To Avoid Reading This Article</b> |
|
6
|
|
|
7
|
There was much angst over the [http://www.shattered.io|SHAttered attack] |
|
8
|
against SHA1 when it was announced in early 2017. If you are concerned |
|
9
|
about this and its implications for Fossil, simply |
|
10
|
[./quickstart.wiki#install|upgrade to Fossil 2.1 or later], and the |
|
11
|
problem will go away. Everything will continue to work as before. |
|
12
|
|
|
13
|
* Legacy repositories will continue working just as |
|
14
|
they always have, without any conversions or upgrades. |
|
15
|
* Historical check-ins will keep their same historical |
|
16
|
SHA1 names. |
|
17
|
* New check-ins will get more secure SHA3-256 hash names. |
|
18
|
* Everything will continue as if nothing happened. |
|
19
|
* Your workflow will be unchanged. |
|
20
|
|
|
21
|
But if you are curious and want a deeper understanding of what is |
|
22
|
going on, read on... |
|
23
|
|
|
24
|
|
|
25
|
<h2>Introduction</h2> |
|
26
|
|
|
27
|
The first snapshot-based distributed version control system |
|
28
|
was [http://www.monotone.ca|Monotone]. Many of the ideas behind the design |
|
29
|
of Fossil were copied from Monotone, including the use of a SHA1 hash to |
|
30
|
assign names to artifacts. Git and Mercurial did the same thing. |
|
31
|
|
|
32
|
The SHA1 hash algorithm is used only to create names for artifacts in Fossil |
|
33
|
(and in Git, Mercurial, and Monotone). It is not used for security. |
|
34
|
Nevertheless, when the [http://www.shattered.io|SHAttered attack] found |
|
35
|
two different PDF files with the same SHA1 hash, many users learned that |
|
36
|
"SHA1 is broken". They see that Fossil (and Git, Mercurial, and Monotone) |
|
37
|
use SHA1 and they therefore conclude that "Fossil is broken". This is |
|
38
|
not true, but it is a public relations problem. So the decision |
|
39
|
was made to migrate Fossil away from SHA1. |
|
40
|
|
|
41
|
This article describes how that migration is occurring. |
|
42
|
|
|
43
|
<h2>Use Of Hardened SHA1</h2> |
|
44
|
|
|
45
|
In Fossil version 2.0 ([/timeline?c=version-2.0|2017-03-03]), |
|
46
|
the internal SHA1 implementation was changed from a generic |
|
47
|
FIPS PUB 180-4 SHA1 implementation to a "Hardened SHA1" |
|
48
|
[[https://github.com/cr-marcstevens/sha1collisiondetection|1]] |
|
49
|
[[https://marc-stevens.nl/research/papers/C13-S.pdf|2]]. |
|
50
|
|
|
51
|
The Hardened SHA1 algorithm automatically detects when the artifact |
|
52
|
being hashed is specifically designed to exploit the known weaknesses |
|
53
|
in the SHA1 algorithm, and when it detects such an attack it changes |
|
54
|
the hash algorithm (by increasing the number of rounds in the compression |
|
55
|
function) to make the algorithm secure again. If the attack detection |
|
56
|
gets a false-positive, that means that Hardened SHA1 will get a different |
|
57
|
answer than the standard FIPS PUB 180-4 SHA1, but the creators of |
|
58
|
Hardened SHA1 (see the second paper |
|
59
|
[[https://marc-stevens.nl/research/papers/C13-S.pdf|2]]) |
|
60
|
report that the probability of a false-positive is vanishingly small - |
|
61
|
less than 1 false positive out of 10<sup><font size=1>27</font></sup> |
|
62
|
hashes. |
|
63
|
|
|
64
|
Hardened SHA1 is slower (and a lot bigger) but Fossil does not do that |
|
65
|
much hashing, so performance is not really an issue. |
|
66
|
|
|
67
|
All versions of Fossil moving forward will use Hardened SHA1. So if |
|
68
|
someone says "SHA1 is broken, and Fossil uses SHA1, therefore Fossil is |
|
69
|
broken," you can rebut the argument by pointing out that Fossil uses |
|
70
|
<em>Hardened SHA1</em>, not generic SHA1, and Hardened SHA1 is <em>not</em> |
|
71
|
broken. |
|
72
|
|
|
73
|
<h2>Support For SHA3-256</h2> |
|
74
|
|
|
75
|
Prior to Fossil version 2.0 ([/timeline?c=version-2.0|2017-03-03]), |
|
76
|
all artifacts in all Fossil repositories were named |
|
77
|
by only a SHA1 hash. |
|
78
|
Version 2.0 extended the [./fileformat.wiki|Fossil file format] |
|
79
|
to allow artifacts to be named by either SHA1 or SHA3-256 hashes. |
|
80
|
(SHA3-256 is the only variant of SHA3 that |
|
81
|
Fossil uses for artifact naming, so for the remainder of this article |
|
82
|
it will be called simply "SHA3". Similarly, "Hardened SHA1" will |
|
83
|
be shortened to "SHA1" in the remaining text.) |
|
84
|
|
|
85
|
To be clear: Fossil (version 2.0 and later) |
|
86
|
allows the SHA1 and SHA3 hashes to be mixed within |
|
87
|
the same repository. Older check-ins, created years ago, |
|
88
|
continue to be named using their legacy SHA1 hashes while |
|
89
|
newer check-ins are named using modern SHA3 hashes. There |
|
90
|
is no need to "convert" a repository from SHA1 over to SHA3. |
|
91
|
You can see this in Fossil itself. The recent |
|
92
|
[9d9ef82234f63758] check-in uses a SHA3 hash whereas the older |
|
93
|
[1669115ab9d05c18] check-in uses a SHA1 hash. |
|
94
|
|
|
95
|
Other than permitting the use of SHA3 in addition to SHA1, there |
|
96
|
were no file format changes in Fossil version 2.0 relative |
|
97
|
to the previous version 1.37. Both Fossil 2.0 and Fossil 1.37 read |
|
98
|
and write all the same repositories and sync with one another, as long |
|
99
|
as none of the repositories contain artifacts named using SHA3. If |
|
100
|
a repository does contain artifacts named using SHA3, Fossil 1.37 will |
|
101
|
not know how to interpret those artifacts and will generate various warnings |
|
102
|
and errors. |
|
103
|
|
|
104
|
<h2>How Fossil Decides Which Hash Algorithm To Use For New Artifacts</h2> |
|
105
|
|
|
106
|
If newer versions of Fossil are able to use either SHA1 or SHA3 to |
|
107
|
name artifacts, which hash algorithm is actually used? That question |
|
108
|
is answered by the "hash policy". These are the supported hash policies: |
|
109
|
|
|
110
|
<table cellpadding=10> |
|
111
|
<tr> |
|
112
|
<td valign='top'>sha1</td> |
|
113
|
<td>Name all new artifacts using the (Hardened) SHA1 hash algorithm.</td> |
|
114
|
</tr> |
|
115
|
<tr> |
|
116
|
<td valign='top'>auto</td> |
|
117
|
<td>Name new artifacts using the SHA1 hash algorithm, but if any |
|
118
|
artifacts are encountered which are already named using SHA3, then |
|
119
|
automatically switch the hash policy to "sha3"</td> |
|
120
|
</tr> |
|
121
|
<tr> |
|
122
|
<td valign='top'>sha3</td> |
|
123
|
<td>Name new artifacts using the SHA3 hash algorithm if the artifact |
|
124
|
does not already have a SHA1 name. If the artifact already has a SHA1 |
|
125
|
name, then continue to use the older SHA1 name. Use SHA3 for new |
|
126
|
artifacts that have never before been encountered.</td> |
|
127
|
</tr> |
|
128
|
<tr> |
|
129
|
<td valign='top'>sha3-only</td> |
|
130
|
<td>Name new artifacts using the SHA3 hash algorithm even if the artifact |
|
131
|
already has a SHA1 name. In other words, force the use of SHA3. This can |
|
132
|
cause some artifacts to be added to the repository twice, once under their |
|
133
|
SHA1 name and again under their SHA3 name, but delta compression will |
|
134
|
prevent that from causing repository size problems.</td> |
|
135
|
</tr> |
|
136
|
<tr> |
|
137
|
<td valign='top'>shun-sha1</td> |
|
138
|
<td>Like "sha3-only" but at this level do not accept a push of SHA1-named |
|
139
|
artifacts. If another Fossil instance tries to push a SHA1-named artifact, |
|
140
|
that artifact is discarded and ignored. |
|
141
|
</tr> |
|
142
|
</table> |
|
143
|
|
|
144
|
For Fossil 2.0, and obviously also for Fossil 1.37 and before, the |
|
145
|
only hash policy supported was the one now called "sha1", meaning that |
|
146
|
all new artifacts were named |
|
147
|
using a SHA1 hash. |
|
148
|
Even though Fossil 2.0 added the capability of understanding SHA3 hashes, it |
|
149
|
never actually generates any SHA3 hashes. |
|
150
|
|
|
151
|
From Fossil 2.1 through 2.9, the default hash policy for legacy repositories |
|
152
|
changed to "auto", meaning that |
|
153
|
Fossil continued to generate only SHA1 hashes until it |
|
154
|
encountered one artifact with a SHA3 hash. Once those older versions of |
|
155
|
Fossil saw a single SHA3 hash, they |
|
156
|
automatically switched to "sha3" mode and thereafter generated |
|
157
|
only SHA3 hashes. |
|
158
|
|
|
159
|
When a new repository is created by cloning, the hash policy is copied |
|
160
|
from the parent. |
|
161
|
|
|
162
|
For new repositories created using the |
|
163
|
[/help/new|fossil new] command the default hash policy is "sha3". |
|
164
|
That means new repositories |
|
165
|
will normally hold nothing except SHA3 hashes. The hash policy for new |
|
166
|
repositories can be overridden using the "--sha1" option to the |
|
167
|
"fossil new" command. |
|
168
|
|
|
169
|
If you are still on Fossil 2.1 through 2.9 but you want Fossil to go |
|
170
|
ahead and start using SHA3 hashes, change the hash policy to |
|
171
|
"sha3" using a command like this: |
|
172
|
|
|
173
|
<verbatim> |
|
174
|
fossil hash-policy sha3 |
|
175
|
</verbatim> |
|
176
|
|
|
177
|
The next check-in will use a SHA3 hash, so that when that check-in is pushed |
|
178
|
to colleagues, their clones will include the new SHA3-named artifact, so |
|
179
|
their local Fossil instances will automatically convert their clones to |
|
180
|
"sha3" mode as well. |
|
181
|
|
|
182
|
Of course, if some members of your team stubbornly refuse to upgrade past |
|
183
|
Fossil 1.37, you should avoid changing the hash policy and creating |
|
184
|
artifacts with SHA3 names, because once you do that your recalcitrant |
|
185
|
coworkers will no longer be able to collaborate. |
|
186
|
|
|
187
|
<h2>A Pure SHA3 Future</h2> |
|
188
|
|
|
189
|
Fossil 2.10 changed the default hash policy to "sha3" mode even for |
|
190
|
legacy repositories, so if you |
|
191
|
upgrade to the latest version of Fossil, all of your new artifacts will |
|
192
|
use a SHA3 hash. Legacy SHA1 artifacts continue to use their original |
|
193
|
names, but new artifacts will use SHA3 names. You might not even notice |
|
194
|
this automatic change over to stronger hashes. |
|
195
|
|
|
196
|
We decided to make the change to pure SHA3 since the last known distributor |
|
197
|
of Fossil 1.x binaries — Debian 9 — was finally replaced in June 2019 |
|
198
|
by Debian 10, which included Fossil 2.8. All other known sources of |
|
199
|
Fossil 1.x binaries upgraded well before that point. |
|
200
|
|