Fossil SCM

fossil-scm / www / build.wiki
1
<title>Compiling and Installing Fossil</title>
2
3
<h2>0.0 Using A Pre-compiled Binary</h2>
4
5
[/uv/download.html|Pre-compiled binaries] are available for recent
6
releases. Just download
7
the appropriate executable for your platform
8
and put it on your $PATH.
9
To uninstall, simply delete the executable.
10
To upgrade from an older release, just overwrite the older binary with
11
the newer one.
12
13
For details about how those binaries are built, see
14
[/wiki?name=Release+Build+How-To | the Release Build How-To wiki page].
15
16
17
<h2>0.1 Executive Summary</h2>
18
19
Building and installing is very simple. Three steps:
20
21
<ol>
22
<li> Download and unpack a source tarball or ZIP.
23
<li> <b>./configure; make</b>
24
<li> Move the resulting "fossil" or "fossil.exe" executable to someplace on
25
your $PATH.
26
</ol>
27
28
<hr>
29
30
<h2>1.0 Obtaining The Source Code</h2>
31
32
Fossil is self-hosting, so you can obtain a ZIP archive or tarball
33
containing a snapshot of the <em>latest</em> version directly from
34
Fossil's own fossil repository. Additionally, source archives of
35
<em>released</em> versions of
36
fossil are available from the [/uv/download.html|downloads page].
37
To obtain a development version of fossil, follow these steps:
38
39
<ol>
40
<li>Point your web browser to [https://fossil-scm.org/]</li>
41
42
<li>Click on the [/timeline|Timeline]
43
link at the top of the page.</li>
44
45
<li>Select a version of Fossil you want to download. The latest
46
version on the trunk branch is usually a good choice. Click on its
47
link.</li>
48
49
<li>Finally, click on one of the
50
"Zip Archive" or "Tarball" links, according to your preference.
51
These links will build a ZIP archive or a gzip-compressed tarball of the
52
complete source code and download it to your computer.</li>
53
</ol>
54
55
<h2>Aside: Is it really safe to use an unreleased development version of
56
the Fossil source code?</h2>
57
58
Yes! Any check-in on the
59
[/timeline?t=trunk | trunk branch] of the Fossil
60
[https://fossil-scm.org/ | self-hosting repository]
61
will work fine. (Dodgy code is always on a branch.) In the unlikely
62
event that you pick a version with a serious bug, it still won't
63
clobber your files. Fossil uses several
64
[./selfcheck.wiki | self-checks] prior to committing any
65
repository change that prevent loss-of-work due to bugs.
66
67
The Fossil [./selfhost.wiki | self-hosting repositories], especially
68
the one at [https://fossil-scm.org/home], usually run a version
69
of trunk that is less than a week or two old. Look at the bottom
70
left-hand corner of this screen (to the right of "This page was
71
generated in...") to see exactly which version of Fossil is
72
rendering this page. It is always safe to use whatever version
73
of the Fossil code you find running on the main Fossil website.
74
75
<h2>2.0 Compiling</h2>
76
77
<ol>
78
<li value="5">
79
Unpack the ZIP or tarball you downloaded then
80
<b>cd</b> into the directory created.</li>
81
82
<li><i>(Optional, Debian-compatible Linux only)</i>
83
Make sure you have all the necessary tools and libraries at hand by running:
84
<b>sudo apt install tcl-dev tk libssl-dev zlib1g-dev</b>.
85
86
<li><i>(Optional, Unix only)</i>
87
Run <b>./configure</b> to construct a makefile.
88
89
<ol type="a">
90
<li>
91
The build system for Fossil on Unix-like systems assumes that the
92
OpenSSL development and runtime files are available on your system,
93
because unprotected repositories are trivial to attack otherwise.
94
Indeed, some public Fossil repositories — including Fossil's own — today
95
run in an HTTPS-only mode, so that you can't even do an anonymous clone
96
from them without using the TLS features added to Fossil by OpenSSL. To
97
weaken that stance could allow a
98
[https://en.wikipedia.org/wiki/Man-in-the-middle_attack|man in the
99
middle attack], such as one that substitutes malicious code into your
100
Fossil repository clone.
101
102
You can force the Fossil build system to avoid searching for, building
103
against, and linking to the OpenSSL library by passing
104
<b>--with-openssl=none</b> to the <tt>configure</tt> script.
105
106
If you do not have the OpenSSL development libraries on your system,
107
we recommend that you install them, typically via your OS's package
108
manager. The Fossil build system goes to a lot of effort to seek these
109
out wherever they may be found, so that is typically all you need to
110
do.
111
112
For more advanced use cases, see the [./ssl.wiki#openssl-bin|OpenSSL
113
discussion in the "TLS and Fossil" document].
114
</li>
115
116
<li>
117
To build a statically linked binary, you can <i>try</i> adding
118
the <b>--static</b> option, but
119
[https://stackoverflow.com/questions/3430400/linux-static-linking-is-dead
120
| it may well not work]. If your platform of choice is affected by this,
121
the simplest workaround we're aware of is to build a Fossil container,
122
then [./containers.md#static | extract the static executable from it].
123
</li>
124
125
<li>
126
To enable the native [./th1.md#tclEval | Tcl integration feature] feature,
127
add the <b>--with-tcl=1</b> and <b>--with-tcl-private-stubs=1</b> options.
128
</li>
129
130
<li>
131
Other configuration options can be seen by running
132
<b>./configure --help</b>
133
</li>
134
</ol>
135
136
<li>Run "<b>make</b>" to build the "fossil" or "fossil.exe" executable.
137
The details depend on your platform and compiler.</li>
138
139
<ol type="a">
140
<li><i>Unix</i> → the configure-generated Makefile should work on
141
all Unix and Unix-like systems. Simply type "<b>make</b>".</li>
142
143
<li><i>Unix without running "configure"</i> → if you prefer to avoid
144
running configure, you can also use: <b>make -f Makefile.classic</b>. You may
145
want to make minor edits to Makefile.classic to configure the build for your
146
system.</li>
147
148
<li><i>MinGW / MinGW-w64</i> → The best-supported path is to build
149
via the MinGW specific Makefile under a POSIX build of GNU make:
150
"<b>make -f win/Makefile.mingw</b>".</li>
151
152
To enable the native [./th1.md#tclEval | Tcl integration feature], use a
153
command line like the following (all on one line):
154
155
<pre>make -f win/Makefile.mingw FOSSIL_ENABLE_TCL=1 FOSSIL_ENABLE_TCL_STUBS=1 FOSSIL_ENABLE_TCL_PRIVATE_STUBS=1</pre>
156
157
<li><i>MSYS2 / Cygwin</i> → This is something of a hybrid between
158
options "a" and "c" above: it configures and builds
159
<code>fossil.exe</code> much as on Linux, but you get a native Windows
160
executable out at the end. The primary downside is that this type of
161
executable can become confused when attempting to interoperate with
162
fully native Windows EXEs by making assumptions that only hold true when
163
all elements are running under the Cygwin/MSYS environment. The MSVC and
164
MinGW options do not have this limitation.
165
166
Even so, there is value to Linux/Unix natives in having this hybrid
167
while off in Windows-land. The simpler of the two paths is MSYS2, since
168
it lets you install the necessary prerequisites in a single command
169
after installing the base environment:
170
171
<pre>pacman -sS gcc make openssl-devel zlib-devel</pre>
172
173
The equivalent in Cygwin's <code>setup.exe</code> requires stepping
174
through the GUI package chooser, and then if you miss one of the
175
prereqs, going all the way back through it again until you get it right.
176
177
<li><i>MSVC</i> → Use the MSVC makefile.</li>
178
179
<em>NB:</em> Run the following <code>nmake</code> commands from a "x64 Native
180
Tools Command Prompt"; <code>buildmsvc.bat</code> is able to automatically load
181
the build tools (x64 by default, pass "x86" as the first argument to use the
182
x86 tools), so it can be called from a normal command prompt.
183
184
First, change to the "win/" subdirectory ("<b>cd win</b>"), then run
185
"<b>nmake /f Makefile.msc</b>".<br><br>Alternatively, the batch
186
file "<b>win\buildmsvc.bat</b>" may be used and it will attempt to
187
detect and use the latest installed version of MSVC.<br><br>To enable
188
the optional <a href="https://www.openssl.org/">OpenSSL</a> support,
189
first <a href="https://www.openssl.org/source/">download the official
190
source code for OpenSSL</a> and extract it to an appropriately named
191
"<b>openssl</b>" subdirectory within the local
192
[/tree?ci=trunk&name=compat | compat] directory then make sure that some recent
193
<a href="http://www.perl.org/">Perl</a> binaries are installed locally,
194
and finally run one of the following commands:
195
<pre>
196
nmake /f Makefile.msc FOSSIL_ENABLE_SSL=1 FOSSIL_BUILD_SSL=1 PERLDIR=C:\full\path\to\Perl\bin
197
</pre>
198
<pre>
199
buildmsvc.bat FOSSIL_ENABLE_SSL=1 FOSSIL_BUILD_SSL=1 PERLDIR=C:\full\path\to\Perl\bin
200
</pre>
201
To enable the optional native [./th1.md#tclEval | Tcl integration feature],
202
run one of the following commands or add the &quot;FOSSIL_ENABLE_TCL=1&quot;
203
argument to one of the other NMAKE command lines:
204
<pre>
205
nmake /f Makefile.msc FOSSIL_ENABLE_TCL=1
206
</pre>
207
<pre>
208
buildmsvc.bat FOSSIL_ENABLE_TCL=1
209
</pre>
210
211
<li><i>Cygwin</i> → The same as other Unix-like systems. It is
212
recommended to configure using: "<b>configure --disable-internal-sqlite</b>",
213
making sure you have the "libsqlite3-devel" , "zlib-devel" and
214
"openssl-devel" packages installed first.</li>
215
</ol>
216
</ol>
217
218
<h2>3.0 Installing</h2>
219
220
<ol>
221
<li value="9">
222
The finished binary is named "fossil" (or "fossil.exe" on Windows).
223
Put this binary in a
224
directory that is somewhere on your PATH environment variable.
225
It does not matter where.
226
</li>
227
228
<li>
229
<b>(Optional:)</b>
230
To uninstall, just delete the binary.
231
</li>
232
</ol>
233
234
<h2>4.0 Additional Considerations</h2>
235
236
<ul>
237
<li>
238
If the makefiles that come with Fossil do not work for
239
you, or for some other reason you want to know how to build
240
Fossil manually, then refer to the
241
[./makefile.wiki | Fossil Build Process] document which describes
242
in detail what the makefiles do behind the scenes.
243
</li>
244
245
<li>
246
The fossil executable is self-contained and stand-alone and usually
247
requires no special libraries or other software to be installed. However,
248
the "--tk" option to the [/help/diff|diff command] requires that Tcl/Tk
249
be installed on the local machine. You can get Tcl/Tk from
250
[http://www.activestate.com/activetcl|ActiveState].
251
</li>
252
253
<li>
254
To build on older Macs (circa 2002, MacOS 10.2) edit the Makefile
255
generated by configure to add the following lines:
256
<pre>
257
TCC += -DSQLITE_WITHOUT_ZONEMALLOC
258
TCC += -D_BSD_SOURCE
259
TCC += -DWITHOUT_ICONV
260
TCC += -Dsocketlen_t=int
261
TCC += -DSQLITE_MAX_MMAP_SIZE=0
262
</pre>
263
</li>
264
</ul>
265
266
267
<h2 id="docker" name="oci">5.0 Building a Docker Container</h2>
268
269
The information on building Fossil inside an
270
[https://opencontainers.org/ | OCI container] is now in
271
[./containers.md | a separate document].
272
273
This includes the instructions on using the OCI container as an
274
expedient intermediary for building a statically-linked Fossil binary on
275
modern Linux platforms, which otherwise make this difficult.
276
277
278
<h2>6.0 Building on/for Android</h2>
279
280
<h3>6.1 Cross-compiling from Linux</h3>
281
282
The following instructions for building Fossil for Android via Linux,
283
without requiring a rooted OS, are adapted from
284
[forum:/forumpost/e0e9de4a7e | a forum post].
285
286
On the development machine, from the fossil source tree:
287
288
<pre><code>export CC=$NDK_PATH/toolchains/llvm/prebuilt/linux-x86_64/bin/armv7a-linux-androideabi21-clang
289
./configure --with-openssl=none
290
make
291
</code></pre>
292
293
294
On the Android device, enable the <em>USB debugging</em> option from
295
Developer menu in Device Options. Connect the device to the development
296
system with USB. If it's configured and connected properly,
297
the device should show up in the output of <code>adb devices</code>:
298
299
<pre><code>sudo adb devices
300
</code></pre>
301
302
Copy the resulting fossil binary onto the device...
303
304
<pre><code>sudo adb push fossil /data/local/tmp
305
</code></pre>
306
307
And run it from an <code>adb</code> shell:
308
309
<pre><code>sudo adb shell
310
&gt; cd /data/local/tmp
311
# Fossil requires a HOME directory to work with:
312
&gt; export HOME=$PWD
313
&gt; export PATH=$PWD:$PATH
314
&gt; fossil version
315
This is fossil version 2.11 &#91;e5653a4ceb] 2020-03-26 18:54:02 UTC
316
</code></pre>
317
318
The output might, or might not, include warnings such as:
319
320
<pre><code>WARNING: linker: ./fossil: unused DT entry: type 0x6ffffef5 arg 0x1464
321
WARNING: linker: ./fossil: unused DT entry: type 0x6ffffffe arg 0x1ba8
322
WARNING: linker: ./fossil: unused DT entry: type 0x6fffffff arg 0x2
323
</code></pre>
324
325
The source of such warnings is not 100% certain.
326
Some information about these (reportedly harmless) warnings can
327
be found
328
[https://stackoverflow.com/a/41900551 | on this StackOverflow post].
329
330
331
<a id='fuzzer'></a>
332
<h2>7.0 Building for Fuzz Testing</h2>
333
334
This feature is primarily intended for fossil's developers and may
335
change at any time. It is only known to work on Linux systems and has
336
been seen to work on x86/64 and ARM.
337
338
Fossil has builtin support for processing specific features using
339
<tt>libfuzzer</tt>. The features which can be tested this way are
340
found in the help text for the [/help/test-fuzz|test-fuzz
341
command].
342
343
Fuzzing requires:
344
345
* The clang C compiler.
346
* libfuzzer. On Ubuntu-derived systems, it can be installed with
347
<tt>apt install libfuzzer-XYZ</tt>, where XYZ is a version number
348
(several versions may be available on any given system)
349
350
351
Compile as follows:
352
353
<pre><code>make clean
354
make TCCFLAGS='-DFOSSIL_FUZZ -fsanitize=fuzzer,address,undefined -O0 -g' CC=clang
355
</code></pre>
356
357
The resulting <tt>fossil</tt> binary differs from the standard
358
one primarily in that it runs the <tt>test-fuzz</tt> command by
359
default. It needs to be told what to fuzz and needs to be given a
360
directory of input files to seed the fuzzer with:
361
362
363
<pre><code>$ mkdir cases
364
# Copy input files into ./cases. e.g. when fuzzing the markdown
365
# processor, copy any to-be-tested .md files into that directory.
366
# Then start the fuzzer:
367
$ ./fossil-fuzz --fuzztype markdown cases
368
</code></pre>
369
370
As it works, it writes its mutated test files into the "cases"
371
directory, each one named in the form of a hash. When it finds a
372
problem it will produce a stack trace for the offending code, will
373
output the name of the file which triggered the crash (named
374
<tt>cases/SOME_HASH</tt>) and may, depending on the nature of the
375
problem, produce a file named <tt>crash-SOMETHING</tt>. In theory the
376
crash file can be fed directly back into the fuzzer to reproduce the
377
problem:
378
379
<pre></code>$ ./fossil-fuzz --fuzztype markdown crash-SOMETHING
380
</code></pre>
381
382
But whether or not it will genuinely crash may depend on static
383
app-level state which might not trigger the crash when running an
384
individual test.
385
386
For a detailed information about the fuzzer's flags and features, see:
387
388
* [https://llvm.org/docs/LibFuzzer.html]
389
* [https://github.com/google/fuzzing/blob/master/tutorial/libFuzzerTutorial.md]
390
391
Flags for the fuzzer can be passed directly to fossil,
392
e.g. <tt>-jobs=4</tt> to start four fuzzer jobs in parallel, but doing
393
so may cause the fuzzer to <em>strip the --fuzztype flag</em>, leading
394
to it testing the wrong thing. When passing on fuzzer-specific flags
395
along with <tt>--fuzztype</tt>, be sure to check your system's process
396
list to ensure that your <tt>--fuzztype</tt> flag is there.
397
398
399
<a id='wasm'></a>
400
<h2>8.0 Building WebAssembly Components</h2>
401
402
Fossil uses one component built as
403
[https://developer.mozilla.org/en-US/docs/WebAssembly | WebAssembly]
404
a.k.a. WASM. Because compiling WASM code requires non-trivial
405
client-side tooling, the repository includes compiled copies of these
406
pieces. Most Fossil hackers should never need to concern themselves
407
with the WASM parts, but this section describes how to for those who
408
want or need to do so.
409
410
<strong>The bits described in this section are necessary when updating
411
<tt>extsrc/pikchr.c</tt></strong> from the upstream source, or the
412
fossil binary will use a different version of pikchr than
413
[/pikchrshow] does (as the latter runs the WASM build of pikchr).
414
415
These instructions have only ever been tested on Linux systems. They
416
"should" work on any Unix-like system supported by Emscripten. The
417
fossil makefiles for Windows builds <em>do not</em> include any of the
418
WASM-related components (patches to add that would be welcomed, of
419
course).
420
421
The first step is to configure the tree with support for
422
[https://emscripten.org/|Emscripten]. This requires that the system
423
has the Emscripten SDK (a.k.a. emsdk) installed, as documented at:
424
425
[https://emscripten.org/docs/getting_started/downloads.html]
426
427
For instructions on keeping the SDK up to date, see:
428
429
[https://emscripten.org/docs/tools_reference/emsdk.html]
430
431
<div class="sidebar">Getting Emscripten up and running is trivial and
432
painless, at least on Linux systems, but the installer downloads
433
many hundreds of megabytes of tools and dependencies, all of which
434
will be installed under the single SDK directory (as opposed to
435
being installed at the system level). It does, however, require
436
that python3 be installed at the system level and it can
437
optionally make use of a system-level cmake for certain tasks
438
unrelated to how fossil uses the SDK.</div>
439
440
After installing the SDK, configure the fossil tree with emsdk
441
support:
442
443
<pre><code>$ ./configure --with-emsdk=/path/to/emsdk \
444
--and-other-options...
445
</code></pre>
446
447
If the <tt>--with-emsdk</tt> flag is not provided, the configure
448
script will check for the environment variable <tt>EMSDK</tt>, which
449
is one of the standard variables the SDK environment uses. If that
450
variable is found, its value will implicitly be used in place of the
451
missing <tt>--with-emsdk</tt> flag. Thus, if the <tt>emsdk_env.sh</tt>
452
script is sourced into the shell before running the configure script,
453
the SDK will be detected even without the config flag.
454
455
The configure script installs some makefile variables which tell the
456
build where to find the SDK and it generates a script named
457
<tt>tools/emcc.sh</tt> (from the template file
458
<tt>[/file/tools/emcc.sh.in|/tools/emcc.sh.in]</tt>), which is a
459
wrapper around the Emscripten C compiler (<tt>emcc</tt>). The wrapper
460
script uses the configure-time state to attempt to set up the various
461
environment variables which are required by <tt>emcc</tt> and will
462
fail if it cannot do so. Once it's set up the environment, it passes
463
on all of its arguments to <tt>emcc</tt>.
464
465
The WASM-related build parts are set up such that none of them should
466
ever trigger implicitly (e.g. via dependencies resolution) in a normal
467
build cycle. They are instead explicitly built as described below.
468
469
From the top of the source tree, all WASM-related components can be
470
built with:
471
472
<pre><code>$ make wasm</code></pre>
473
474
<div class="sidebar">The file
475
<tt>[/file/extsrc/pikcher-worker.js|extsrc/pikcher-worker.js]</tt>
476
is hand-coded and intended to be loaded as a "Worker" in
477
JavaScript. That file loads the main module and provides an
478
interface via which a main JavaScript thread can communicate with
479
pikchr running in a Worker thread. The file
480
<tt>[/file/src/fossil.page.pikchrshowasm.js|src/fossil.page.pikchrshowasm.js]</tt>
481
implements the [/pikchrshow] app and demonstrates how
482
<tt>pikchr-worker.js</tt> is used.</div>
483
484
As of this writing, those parts include:
485
486
* <tt>extsrc/pikchr.wasm</tt> is a WASM-compiled form of
487
<tt>extsrc/pikchr.c</tt>.
488
* <tt>extsrc/pikchr.js</tt> is JS/WASM glue code generated by Emscripten
489
to give JS code access to the API exported by the WASM file.
490
491
When a new version of <tt>extsrc/pikchr.c</tt> is installed, the
492
files <tt>pikchr.{js,wasm}</tt> will need to be recompiled to account
493
for that. Running <tt>make wasm</tt> will, if the build is set up for
494
the emsdk, recompile those:
495
496
<pre><code>$ rm extsrc/pikchr.{js,wasm}
497
# ^^^^ that rm has proven necessary in order to ensure rebuilds
498
$ make wasm
499
./tools/emcc.sh -o extsrc/pikchr.js ...
500
$ ls -la extsrc/pikchr.{js,wasm}
501
-rw-rw-r-- 1 stephan stephan 17263 Jun 8 03:59 extsrc/pikchr.js
502
-rw-rw-r-- 1 stephan stephan 97578 Jun 8 03:59 extsrc/pikchr.wasm
503
</code></pre>
504
505
<div class="sidebar">If that fails with a message along the lines of
506
“<code>setting `EXPORTED_RUNTIME_METHODS` expects `<class 'list'>` but
507
got `<class 'str'>`</code>” then the emcc being invoked is too old: emcc
508
changed the format of list-type arguments at some point. The required
509
minimum version is unknown, but any SDK version from May 2022 or later
510
"should" (as of this writing) suffice. Any older version may or may not
511
work.</div>
512
513
After that succeeds, we need to run the normal build so that those
514
generated files can be compiled in to the fossil binary, accessible
515
via the [/help/www/builtin|/builtin page]:
516
517
<pre><code>$ make</code></pre>
518
519
Before checking in those newly-built files, they need to be tested by
520
running the [/pikchrshow] page. If that page loads, the compilation
521
process fundamentally worked (a load failure will be made obvious to
522
the viewer). If it fails to load then the browser's dev tools console
523
likely provides at least a small hint (and <em>sometimes</em> a useful
524
hint) about the nature of the problem. Don't check those files in
525
until [/pikchrshow] runs, though!
526
527
Should pikchr's C interface ever change, <tt>pikchr-worker.js</tt>
528
will need to be updated to accommodate it, but such modification is
529
typically trivial.
530
531
<h3>8.1 Solutions other than Emscripten?</h3>
532
533
Emscripten is not the only option for building C as WASM, but it
534
provides a complete toolchain which eliminates many other steps which
535
must otherwise be accounted for on a per-project basis. Despite its
536
convenience, it behooves us to explore other build options for the
537
sake of portability and avoiding what amounts to vendor lock-in.
538
539
For later refererence, here are articles specifically covering
540
building WASM projects without using Emscripten:
541
542
* [https://surma.dev/things/c-to-webassembly/]
543
* [https://schellcode.github.io/webassembly-without-emscripten]
544

Keyboard Shortcuts

Open search /
Next entry (timeline) j
Previous entry (timeline) k
Open focused entry Enter
Show this help ?
Toggle theme Top nav button