Fossil SCM

fossil-scm / www / build.wiki
Source Blame History 543 lines
44610be… drh 1 <title>Compiling and Installing Fossil</title>
9eb6ea7… kejoki 2
9eb6ea7… kejoki 3 <h2>0.0 Using A Pre-compiled Binary</h2>
9eb6ea7… kejoki 4
1fd407f… wyoung 5 [/uv/download.html|Pre-compiled binaries] are available for recent
a0f5be9… drh 6 releases. Just download
a0f5be9… drh 7 the appropriate executable for your platform
75ea5ac… drh 8 and put it on your $PATH.
a0f5be9… drh 9 To uninstall, simply delete the executable.
ef0b48d… drh 10 To upgrade from an older release, just overwrite the older binary with
1fd407f… wyoung 11 the newer one.
c763040… stephan 12
1fd407f… wyoung 13 For details about how those binaries are built, see
1fd407f… wyoung 14 [/wiki?name=Release+Build+How-To | the Release Build How-To wiki page].
c763040… stephan 15
ef0b48d… drh 16
ef0b48d… drh 17 <h2>0.1 Executive Summary</h2>
ef0b48d… drh 18
1fd407f… wyoung 19 Building and installing is very simple. Three steps:
ef0b48d… drh 20
ef0b48d… drh 21 <ol>
ef0b48d… drh 22 <li> Download and unpack a source tarball or ZIP.
ef0b48d… drh 23 <li> <b>./configure; make</b>
2ab19a1… mistachkin 24 <li> Move the resulting "fossil" or "fossil.exe" executable to someplace on
2ab19a1… mistachkin 25 your $PATH.
ef0b48d… drh 26 </ol>
ef0b48d… drh 27
1fd407f… wyoung 28 <hr>
f94f7e5… drh 29
f94f7e5… drh 30 <h2>1.0 Obtaining The Source Code</h2>
f94f7e5… drh 31
1fd407f… wyoung 32 Fossil is self-hosting, so you can obtain a ZIP archive or tarball
901427f… mistachkin 33 containing a snapshot of the <em>latest</em> version directly from
901427f… mistachkin 34 Fossil's own fossil repository. Additionally, source archives of
44610be… drh 35 <em>released</em> versions of
f5240a1… drh 36 fossil are available from the [/uv/download.html|downloads page].
1fd407f… wyoung 37 To obtain a development version of fossil, follow these steps:
f94f7e5… drh 38
f94f7e5… drh 39 <ol>
1fd407f… wyoung 40 <li>Point your web browser to [https://fossil-scm.org/]</li>
f5240a1… drh 41
1fd407f… wyoung 42 <li>Click on the [/timeline|Timeline]
1fd407f… wyoung 43 link at the top of the page.</li>
f5240a1… drh 44
3ec73a0… drh 45 <li>Select a version of Fossil you want to download. The latest
44610be… drh 46 version on the trunk branch is usually a good choice. Click on its
1fd407f… wyoung 47 link.</li>
44610be… drh 48
1fd407f… wyoung 49 <li>Finally, click on one of the
2d50893… drh 50 "Zip Archive" or "Tarball" links, according to your preference.
3ec73a0… drh 51 These links will build a ZIP archive or a gzip-compressed tarball of the
1fd407f… wyoung 52 complete source code and download it to your computer.</li>
f94f7e5… drh 53 </ol>
ec12ded… drh 54
ec12ded… drh 55 <h2>Aside: Is it really safe to use an unreleased development version of
ec12ded… drh 56 the Fossil source code?</h2>
ec12ded… drh 57
901427f… mistachkin 58 Yes! Any check-in on the
ec12ded… drh 59 [/timeline?t=trunk | trunk branch] of the Fossil
efd79f8… drh 60 [https://fossil-scm.org/ | self-hosting repository]
ec12ded… drh 61 will work fine. (Dodgy code is always on a branch.) In the unlikely
ec12ded… drh 62 event that you pick a version with a serious bug, it still won't
ec12ded… drh 63 clobber your files. Fossil uses several
ec12ded… drh 64 [./selfcheck.wiki | self-checks] prior to committing any
ec12ded… drh 65 repository change that prevent loss-of-work due to bugs.
ec12ded… drh 66
ec12ded… drh 67 The Fossil [./selfhost.wiki | self-hosting repositories], especially
143f1db… wyoung 68 the one at [https://fossil-scm.org/home], usually run a version
ec12ded… drh 69 of trunk that is less than a week or two old. Look at the bottom
d5ef91f… drh 70 left-hand corner of this screen (to the right of "This page was
ec12ded… drh 71 generated in...") to see exactly which version of Fossil is
ec12ded… drh 72 rendering this page. It is always safe to use whatever version
ec12ded… drh 73 of the Fossil code you find running on the main Fossil website.
f94f7e5… drh 74
f94f7e5… drh 75 <h2>2.0 Compiling</h2>
f94f7e5… drh 76
f94f7e5… drh 77 <ol>
512def0… stephan 78 <li value="5">
1fd407f… wyoung 79 Unpack the ZIP or tarball you downloaded then
1fd407f… wyoung 80 <b>cd</b> into the directory created.</li>
c2cfaa6… drh 81
c2cfaa6… drh 82 <li><i>(Optional, Debian-compatible Linux only)</i>
c2cfaa6… drh 83 Make sure you have all the necessary tools and libraries at hand by running:
12ce928… danield 84 <b>sudo apt install tcl-dev tk libssl-dev zlib1g-dev</b>.
fe38a76… drh 85
fe38a76… drh 86 <li><i>(Optional, Unix only)</i>
ef0b48d… drh 87 Run <b>./configure</b> to construct a makefile.
ef0b48d… drh 88
ef0b48d… drh 89 <ol type="a">
1fd407f… wyoung 90 <li>
4f81027… wyoung 91 The build system for Fossil on Unix-like systems assumes that the
4f81027… wyoung 92 OpenSSL development and runtime files are available on your system,
4f81027… wyoung 93 because unprotected repositories are trivial to attack otherwise.
4f81027… wyoung 94 Indeed, some public Fossil repositories — including Fossil's own — today
4f81027… wyoung 95 run in an HTTPS-only mode, so that you can't even do an anonymous clone
4f81027… wyoung 96 from them without using the TLS features added to Fossil by OpenSSL. To
4f81027… wyoung 97 weaken that stance could allow a
4f81027… wyoung 98 [https://en.wikipedia.org/wiki/Man-in-the-middle_attack|man in the
4f81027… wyoung 99 middle attack], such as one that substitutes malicious code into your
1fd407f… wyoung 100 Fossil repository clone.
4f81027… wyoung 101
1fd407f… wyoung 102 You can force the Fossil build system to avoid searching for, building
4f81027… wyoung 103 against, and linking to the OpenSSL library by passing
1fd407f… wyoung 104 <b>--with-openssl=none</b> to the <tt>configure</tt> script.
4f81027… wyoung 105
1fd407f… wyoung 106 If you do not have the OpenSSL development libraries on your system,
4f81027… wyoung 107 we recommend that you install them, typically via your OS's package
4f81027… wyoung 108 manager. The Fossil build system goes to a lot of effort to seek these
4f81027… wyoung 109 out wherever they may be found, so that is typically all you need to
1fd407f… wyoung 110 do.
d282e42… wyoung 111
1fd407f… wyoung 112 For more advanced use cases, see the [./ssl.wiki#openssl-bin|OpenSSL
1fd407f… wyoung 113 discussion in the "TLS and Fossil" document].
1fd407f… wyoung 114 </li>
d282e42… wyoung 115
1fd407f… wyoung 116 <li>
d282e42… wyoung 117 To build a statically linked binary, you can <i>try</i> adding
d282e42… wyoung 118 the <b>--static</b> option, but
d282e42… wyoung 119 [https://stackoverflow.com/questions/3430400/linux-static-linking-is-dead
d282e42… wyoung 120 | it may well not work]. If your platform of choice is affected by this,
d282e42… wyoung 121 the simplest workaround we're aware of is to build a Fossil container,
d282e42… wyoung 122 then [./containers.md#static | extract the static executable from it].
1fd407f… wyoung 123 </li>
d282e42… wyoung 124
1fd407f… wyoung 125 <li>
901427f… mistachkin 126 To enable the native [./th1.md#tclEval | Tcl integration feature] feature,
901427f… mistachkin 127 add the <b>--with-tcl=1</b> and <b>--with-tcl-private-stubs=1</b> options.
1fd407f… wyoung 128 </li>
901427f… mistachkin 129
1fd407f… wyoung 130 <li>
ef0b48d… drh 131 Other configuration options can be seen by running
ef0b48d… drh 132 <b>./configure --help</b>
1fd407f… wyoung 133 </li>
ef0b48d… drh 134 </ol>
ef0b48d… drh 135
1fd407f… wyoung 136 <li>Run "<b>make</b>" to build the "fossil" or "fossil.exe" executable.
1fd407f… wyoung 137 The details depend on your platform and compiler.</li>
2d50893… drh 138
2d50893… drh 139 <ol type="a">
1fd407f… wyoung 140 <li><i>Unix</i> → the configure-generated Makefile should work on
1fd407f… wyoung 141 all Unix and Unix-like systems. Simply type "<b>make</b>".</li>
901427f… mistachkin 142
1fd407f… wyoung 143 <li><i>Unix without running "configure"</i> → if you prefer to avoid
901427f… mistachkin 144 running configure, you can also use: <b>make -f Makefile.classic</b>. You may
901427f… mistachkin 145 want to make minor edits to Makefile.classic to configure the build for your
1fd407f… wyoung 146 system.</li>
901427f… mistachkin 147
1fd407f… wyoung 148 <li><i>MinGW / MinGW-w64</i> → The best-supported path is to build
2c66a53… wyoung 149 via the MinGW specific Makefile under a POSIX build of GNU make:
1fd407f… wyoung 150 "<b>make -f win/Makefile.mingw</b>".</li>
901427f… mistachkin 151
901427f… mistachkin 152 To enable the native [./th1.md#tclEval | Tcl integration feature], use a
901427f… mistachkin 153 command line like the following (all on one line):
901427f… mistachkin 154
7384043… wyoung 155 <pre>make -f win/Makefile.mingw FOSSIL_ENABLE_TCL=1 FOSSIL_ENABLE_TCL_STUBS=1 FOSSIL_ENABLE_TCL_PRIVATE_STUBS=1</pre>
1fd407f… wyoung 156
7384043… wyoung 157 <li><i>MSYS2 / Cygwin</i> → This is something of a hybrid between
7384043… wyoung 158 options "a" and "c" above: it configures and builds
7384043… wyoung 159 <code>fossil.exe</code> much as on Linux, but you get a native Windows
7384043… wyoung 160 executable out at the end. The primary downside is that this type of
7384043… wyoung 161 executable can become confused when attempting to interoperate with
7384043… wyoung 162 fully native Windows EXEs by making assumptions that only hold true when
7384043… wyoung 163 all elements are running under the Cygwin/MSYS environment. The MSVC and
7384043… wyoung 164 MinGW options do not have this limitation.
7384043… wyoung 165
7384043… wyoung 166 Even so, there is value to Linux/Unix natives in having this hybrid
7384043… wyoung 167 while off in Windows-land. The simpler of the two paths is MSYS2, since
7384043… wyoung 168 it lets you install the necessary prerequisites in a single command
7384043… wyoung 169 after installing the base environment:
7384043… wyoung 170
7384043… wyoung 171 <pre>pacman -sS gcc make openssl-devel zlib-devel</pre>
7384043… wyoung 172
7384043… wyoung 173 The equivalent in Cygwin's <code>setup.exe</code> requires stepping
7384043… wyoung 174 through the GUI package chooser, and then if you miss one of the
7384043… wyoung 175 prereqs, going all the way back through it again until you get it right.
1fd407f… wyoung 176
1fd407f… wyoung 177 <li><i>MSVC</i> → Use the MSVC makefile.</li>
1fd407f… wyoung 178
b7e8b8e… danield 179 <em>NB:</em> Run the following <code>nmake</code> commands from a "x64 Native
b7e8b8e… danield 180 Tools Command Prompt"; <code>buildmsvc.bat</code> is able to automatically load
b7e8b8e… danield 181 the build tools (x64 by default, pass "x86" as the first argument to use the
b7e8b8e… danield 182 x86 tools), so it can be called from a normal command prompt.
1fd407f… wyoung 183
b7e8b8e… danield 184 First, change to the "win/" subdirectory ("<b>cd win</b>"), then run
aea5e3d… mistachkin 185 "<b>nmake /f Makefile.msc</b>".<br><br>Alternatively, the batch
aea5e3d… mistachkin 186 file "<b>win\buildmsvc.bat</b>" may be used and it will attempt to
cf901e5… mistachkin 187 detect and use the latest installed version of MSVC.<br><br>To enable
fe3887b… mistachkin 188 the optional <a href="https://www.openssl.org/">OpenSSL</a> support,
fe3887b… mistachkin 189 first <a href="https://www.openssl.org/source/">download the official
fe3887b… mistachkin 190 source code for OpenSSL</a> and extract it to an appropriately named
8c361b3… danield 191 "<b>openssl</b>" subdirectory within the local
8c361b3… danield 192 [/tree?ci=trunk&name=compat | compat] directory then make sure that some recent
fe3887b… mistachkin 193 <a href="http://www.perl.org/">Perl</a> binaries are installed locally,
fe3887b… mistachkin 194 and finally run one of the following commands:
8a1ba49… wyoung 195 <pre>
80b4add… mistachkin 196 nmake /f Makefile.msc FOSSIL_ENABLE_SSL=1 FOSSIL_BUILD_SSL=1 PERLDIR=C:\full\path\to\Perl\bin
8a1ba49… wyoung 197 </pre>
8a1ba49… wyoung 198 <pre>
80b4add… mistachkin 199 buildmsvc.bat FOSSIL_ENABLE_SSL=1 FOSSIL_BUILD_SSL=1 PERLDIR=C:\full\path\to\Perl\bin
8a1ba49… wyoung 200 </pre>
901427f… mistachkin 201 To enable the optional native [./th1.md#tclEval | Tcl integration feature],
901427f… mistachkin 202 run one of the following commands or add the &quot;FOSSIL_ENABLE_TCL=1&quot;
901427f… mistachkin 203 argument to one of the other NMAKE command lines:
8a1ba49… wyoung 204 <pre>
901427f… mistachkin 205 nmake /f Makefile.msc FOSSIL_ENABLE_TCL=1
8a1ba49… wyoung 206 </pre>
8a1ba49… wyoung 207 <pre>
901427f… mistachkin 208 buildmsvc.bat FOSSIL_ENABLE_TCL=1
8a1ba49… wyoung 209 </pre>
80b4add… mistachkin 210
1fd407f… wyoung 211 <li><i>Cygwin</i> → The same as other Unix-like systems. It is
5610dc4… jan.nijtmans 212 recommended to configure using: "<b>configure --disable-internal-sqlite</b>",
5610dc4… jan.nijtmans 213 making sure you have the "libsqlite3-devel" , "zlib-devel" and
1fd407f… wyoung 214 "openssl-devel" packages installed first.</li>
5fb963a… drh 215 </ol>
2d50893… drh 216 </ol>
f94f7e5… drh 217
f94f7e5… drh 218 <h2>3.0 Installing</h2>
f94f7e5… drh 219
f94f7e5… drh 220 <ol>
c2cfaa6… drh 221 <li value="9">
1fd407f… wyoung 222 The finished binary is named "fossil" (or "fossil.exe" on Windows).
901427f… mistachkin 223 Put this binary in a
f94f7e5… drh 224 directory that is somewhere on your PATH environment variable.
1fd407f… wyoung 225 It does not matter where.
1fd407f… wyoung 226 </li>
5fb963a… drh 227
5fb963a… drh 228 <li>
1fd407f… wyoung 229 <b>(Optional:)</b>
1fd407f… wyoung 230 To uninstall, just delete the binary.
1fd407f… wyoung 231 </li>
5fb963a… drh 232 </ol>
5fb963a… drh 233
5fb963a… drh 234 <h2>4.0 Additional Considerations</h2>
5fb963a… drh 235
44610be… drh 236 <ul>
1fd407f… wyoung 237 <li>
44610be… drh 238 If the makefiles that come with Fossil do not work for
44610be… drh 239 you, or for some other reason you want to know how to build
44610be… drh 240 Fossil manually, then refer to the
44610be… drh 241 [./makefile.wiki | Fossil Build Process] document which describes
44610be… drh 242 in detail what the makefiles do behind the scenes.
1fd407f… wyoung 243 </li>
44610be… drh 244
1fd407f… wyoung 245 <li>
901427f… mistachkin 246 The fossil executable is self-contained and stand-alone and usually
901427f… mistachkin 247 requires no special libraries or other software to be installed. However,
e94bef2… drh 248 the "--tk" option to the [/help/diff|diff command] requires that Tcl/Tk
901427f… mistachkin 249 be installed on the local machine. You can get Tcl/Tk from
e94bef2… drh 250 [http://www.activestate.com/activetcl|ActiveState].
1fd407f… wyoung 251 </li>
e94bef2… drh 252
1fd407f… wyoung 253 <li>
44610be… drh 254 To build on older Macs (circa 2002, MacOS 10.2) edit the Makefile
44610be… drh 255 generated by configure to add the following lines:
8a1ba49… wyoung 256 <pre>
44610be… drh 257 TCC += -DSQLITE_WITHOUT_ZONEMALLOC
481741f… drh 258 TCC += -D_BSD_SOURCE
44610be… drh 259 TCC += -DWITHOUT_ICONV
44610be… drh 260 TCC += -Dsocketlen_t=int
bf6a4d3… drh 261 TCC += -DSQLITE_MAX_MMAP_SIZE=0
8a1ba49… wyoung 262 </pre>
1fd407f… wyoung 263 </li>
44610be… drh 264 </ul>
942be4c… stephan 265
77d603c… wyoung 266
7129dc9… wyoung 267 <h2 id="docker" name="oci">5.0 Building a Docker Container</h2>
7129dc9… wyoung 268
7129dc9… wyoung 269 The information on building Fossil inside an
7129dc9… wyoung 270 [https://opencontainers.org/ | OCI container] is now in
7129dc9… wyoung 271 [./containers.md | a separate document].
7129dc9… wyoung 272
7129dc9… wyoung 273 This includes the instructions on using the OCI container as an
7129dc9… wyoung 274 expedient intermediary for building a statically-linked Fossil binary on
7129dc9… wyoung 275 modern Linux platforms, which otherwise make this difficult.
b707445… stephan 276
b707445… stephan 277
b707445… stephan 278 <h2>6.0 Building on/for Android</h2>
b707445… stephan 279
b707445… stephan 280 <h3>6.1 Cross-compiling from Linux</h3>
b707445… stephan 281
7fcb462… stephan 282 The following instructions for building Fossil for Android via Linux,
b707445… stephan 283 without requiring a rooted OS, are adapted from
143f1db… wyoung 284 [forum:/forumpost/e0e9de4a7e | a forum post].
b707445… stephan 285
b707445… stephan 286 On the development machine, from the fossil source tree:
b707445… stephan 287
b707445… stephan 288 <pre><code>export CC=$NDK_PATH/toolchains/llvm/prebuilt/linux-x86_64/bin/armv7a-linux-androideabi21-clang
b707445… stephan 289 ./configure --with-openssl=none
b707445… stephan 290 make
b707445… stephan 291 </code></pre>
b707445… stephan 292
b707445… stephan 293
b707445… stephan 294 On the Android device, enable the <em>USB debugging</em> option from
b707445… stephan 295 Developer menu in Device Options. Connect the device to the development
b707445… stephan 296 system with USB. If it's configured and connected properly,
b707445… stephan 297 the device should show up in the output of <code>adb devices</code>:
b707445… stephan 298
b707445… stephan 299 <pre><code>sudo adb devices
b707445… stephan 300 </code></pre>
b707445… stephan 301
b707445… stephan 302 Copy the resulting fossil binary onto the device...
b707445… stephan 303
b707445… stephan 304 <pre><code>sudo adb push fossil /data/local/tmp
b707445… stephan 305 </code></pre>
b707445… stephan 306
b707445… stephan 307 And run it from an <code>adb</code> shell:
b707445… stephan 308
b707445… stephan 309 <pre><code>sudo adb shell
b707445… stephan 310 &gt; cd /data/local/tmp
b707445… stephan 311 # Fossil requires a HOME directory to work with:
b707445… stephan 312 &gt; export HOME=$PWD
b707445… stephan 313 &gt; export PATH=$PWD:$PATH
b707445… stephan 314 &gt; fossil version
b707445… stephan 315 This is fossil version 2.11 &#91;e5653a4ceb] 2020-03-26 18:54:02 UTC
b707445… stephan 316 </code></pre>
b707445… stephan 317
b707445… stephan 318 The output might, or might not, include warnings such as:
b707445… stephan 319
b707445… stephan 320 <pre><code>WARNING: linker: ./fossil: unused DT entry: type 0x6ffffef5 arg 0x1464
b707445… stephan 321 WARNING: linker: ./fossil: unused DT entry: type 0x6ffffffe arg 0x1ba8
b707445… stephan 322 WARNING: linker: ./fossil: unused DT entry: type 0x6fffffff arg 0x2
b707445… stephan 323 </code></pre>
b707445… stephan 324
b707445… stephan 325 The source of such warnings is not 100% certain.
b707445… stephan 326 Some information about these (reportedly harmless) warnings can
b707445… stephan 327 be found
b707445… stephan 328 [https://stackoverflow.com/a/41900551 | on this StackOverflow post].
a42f5af… stephan 329
a42f5af… stephan 330
a42f5af… stephan 331 <a id='fuzzer'></a>
7fcb462… stephan 332 <h2>7.0 Building for Fuzz Testing</h2>
a42f5af… stephan 333
a42f5af… stephan 334 This feature is primarily intended for fossil's developers and may
dfb1234… stephan 335 change at any time. It is only known to work on Linux systems and has
dfb1234… stephan 336 been seen to work on x86/64 and ARM.
a42f5af… stephan 337
a42f5af… stephan 338 Fossil has builtin support for processing specific features using
dfb1234… stephan 339 <tt>libfuzzer</tt>. The features which can be tested this way are
c64f28d… drh 340 found in the help text for the [/help/test-fuzz|test-fuzz
a42f5af… stephan 341 command].
a42f5af… stephan 342
a42f5af… stephan 343 Fuzzing requires:
a42f5af… stephan 344
dfb1234… stephan 345 * The clang C compiler.
a42f5af… stephan 346 * libfuzzer. On Ubuntu-derived systems, it can be installed with
a42f5af… stephan 347 <tt>apt install libfuzzer-XYZ</tt>, where XYZ is a version number
a42f5af… stephan 348 (several versions may be available on any given system)
a42f5af… stephan 349
a42f5af… stephan 350
16d98b9… drh 351 Compile as follows:
16d98b9… drh 352
16d98b9… drh 353 <pre><code>make clean
16d98b9… drh 354 make TCCFLAGS='-DFOSSIL_FUZZ -fsanitize=fuzzer,address,undefined -O0 -g' CC=clang
a42f5af… stephan 355 </code></pre>
a42f5af… stephan 356
16d98b9… drh 357 The resulting <tt>fossil</tt> binary differs from the standard
a42f5af… stephan 358 one primarily in that it runs the <tt>test-fuzz</tt> command by
a42f5af… stephan 359 default. It needs to be told what to fuzz and needs to be given a
a42f5af… stephan 360 directory of input files to seed the fuzzer with:
a42f5af… stephan 361
a42f5af… stephan 362
16d98b9… drh 363 <pre><code>$ mkdir cases
a42f5af… stephan 364 # Copy input files into ./cases. e.g. when fuzzing the markdown
a42f5af… stephan 365 # processor, copy any to-be-tested .md files into that directory.
a42f5af… stephan 366 # Then start the fuzzer:
a42f5af… stephan 367 $ ./fossil-fuzz --fuzztype markdown cases
a42f5af… stephan 368 </code></pre>
a42f5af… stephan 369
16d98b9… drh 370 As it works, it writes its mutated test files into the "cases"
dfb1234… stephan 371 directory, each one named in the form of a hash. When it finds a
dfb1234… stephan 372 problem it will produce a stack trace for the offending code, will
dfb1234… stephan 373 output the name of the file which triggered the crash (named
a42f5af… stephan 374 <tt>cases/SOME_HASH</tt>) and may, depending on the nature of the
a42f5af… stephan 375 problem, produce a file named <tt>crash-SOMETHING</tt>. In theory the
a42f5af… stephan 376 crash file can be fed directly back into the fuzzer to reproduce the
a42f5af… stephan 377 problem:
a42f5af… stephan 378
a42f5af… stephan 379 <pre></code>$ ./fossil-fuzz --fuzztype markdown crash-SOMETHING
a42f5af… stephan 380 </code></pre>
a42f5af… stephan 381
a42f5af… stephan 382 But whether or not it will genuinely crash may depend on static
a42f5af… stephan 383 app-level state which might not trigger the crash when running an
a42f5af… stephan 384 individual test.
a42f5af… stephan 385
1a397ee… stephan 386 For a detailed information about the fuzzer's flags and features, see:
a42f5af… stephan 387
1a397ee… stephan 388 * [https://llvm.org/docs/LibFuzzer.html]
a42f5af… stephan 389 * [https://github.com/google/fuzzing/blob/master/tutorial/libFuzzerTutorial.md]
a42f5af… stephan 390
df1d256… stephan 391 Flags for the fuzzer can be passed directly to fossil,
df1d256… stephan 392 e.g. <tt>-jobs=4</tt> to start four fuzzer jobs in parallel, but doing
df1d256… stephan 393 so may cause the fuzzer to <em>strip the --fuzztype flag</em>, leading
df1d256… stephan 394 to it testing the wrong thing. When passing on fuzzer-specific flags
df1d256… stephan 395 along with <tt>--fuzztype</tt>, be sure to check your system's process
df1d256… stephan 396 list to ensure that your <tt>--fuzztype</tt> flag is there.
7fcb462… stephan 397
7fcb462… stephan 398
7fcb462… stephan 399 <a id='wasm'></a>
7fcb462… stephan 400 <h2>8.0 Building WebAssembly Components</h2>
7fcb462… stephan 401
ad47a44… wyoung 402 Fossil uses one component built as
7fcb462… stephan 403 [https://developer.mozilla.org/en-US/docs/WebAssembly | WebAssembly]
7fcb462… stephan 404 a.k.a. WASM. Because compiling WASM code requires non-trivial
7fcb462… stephan 405 client-side tooling, the repository includes compiled copies of these
7fcb462… stephan 406 pieces. Most Fossil hackers should never need to concern themselves
7fcb462… stephan 407 with the WASM parts, but this section describes how to for those who
7fcb462… stephan 408 want or need to do so.
7fcb462… stephan 409
7fcb462… stephan 410 <strong>The bits described in this section are necessary when updating
7fcb462… stephan 411 <tt>extsrc/pikchr.c</tt></strong> from the upstream source, or the
7fcb462… stephan 412 fossil binary will use a different version of pikchr than
7fcb462… stephan 413 [/pikchrshow] does (as the latter runs the WASM build of pikchr).
7fcb462… stephan 414
7fcb462… stephan 415 These instructions have only ever been tested on Linux systems. They
7fcb462… stephan 416 "should" work on any Unix-like system supported by Emscripten. The
7fcb462… stephan 417 fossil makefiles for Windows builds <em>do not</em> include any of the
7fcb462… stephan 418 WASM-related components (patches to add that would be welcomed, of
7fcb462… stephan 419 course).
7fcb462… stephan 420
7fcb462… stephan 421 The first step is to configure the tree with support for
7fcb462… stephan 422 [https://emscripten.org/|Emscripten]. This requires that the system
7fcb462… stephan 423 has the Emscripten SDK (a.k.a. emsdk) installed, as documented at:
7fcb462… stephan 424
7fcb462… stephan 425 [https://emscripten.org/docs/getting_started/downloads.html]
7fcb462… stephan 426
7fcb462… stephan 427 For instructions on keeping the SDK up to date, see:
7fcb462… stephan 428
7fcb462… stephan 429 [https://emscripten.org/docs/tools_reference/emsdk.html]
7fcb462… stephan 430
8a1ba49… wyoung 431 <div class="sidebar">Getting Emscripten up and running is trivial and
8a1ba49… wyoung 432 painless, at least on Linux systems, but the installer downloads
8a1ba49… wyoung 433 many hundreds of megabytes of tools and dependencies, all of which
8a1ba49… wyoung 434 will be installed under the single SDK directory (as opposed to
8a1ba49… wyoung 435 being installed at the system level). It does, however, require
8a1ba49… wyoung 436 that python3 be installed at the system level and it can
8a1ba49… wyoung 437 optionally make use of a system-level cmake for certain tasks
8a1ba49… wyoung 438 unrelated to how fossil uses the SDK.</div>
7fcb462… stephan 439
7fcb462… stephan 440 After installing the SDK, configure the fossil tree with emsdk
7fcb462… stephan 441 support:
7fcb462… stephan 442
8a1ba49… wyoung 443 <pre><code>$ ./configure --with-emsdk=/path/to/emsdk \
8a1ba49… wyoung 444 --and-other-options...
7fcb462… stephan 445 </code></pre>
7fcb462… stephan 446
7fcb462… stephan 447 If the <tt>--with-emsdk</tt> flag is not provided, the configure
7fcb462… stephan 448 script will check for the environment variable <tt>EMSDK</tt>, which
7fcb462… stephan 449 is one of the standard variables the SDK environment uses. If that
7fcb462… stephan 450 variable is found, its value will implicitly be used in place of the
7fcb462… stephan 451 missing <tt>--with-emsdk</tt> flag. Thus, if the <tt>emsdk_env.sh</tt>
7fcb462… stephan 452 script is sourced into the shell before running the configure script,
7fcb462… stephan 453 the SDK will be detected even without the config flag.
7fcb462… stephan 454
7fcb462… stephan 455 The configure script installs some makefile variables which tell the
7fcb462… stephan 456 build where to find the SDK and it generates a script named
7fcb462… stephan 457 <tt>tools/emcc.sh</tt> (from the template file
7fcb462… stephan 458 <tt>[/file/tools/emcc.sh.in|/tools/emcc.sh.in]</tt>), which is a
7fcb462… stephan 459 wrapper around the Emscripten C compiler (<tt>emcc</tt>). The wrapper
7fcb462… stephan 460 script uses the configure-time state to attempt to set up the various
7fcb462… stephan 461 environment variables which are required by <tt>emcc</tt> and will
7fcb462… stephan 462 fail if it cannot do so. Once it's set up the environment, it passes
7fcb462… stephan 463 on all of its arguments to <tt>emcc</tt>.
7fcb462… stephan 464
7fcb462… stephan 465 The WASM-related build parts are set up such that none of them should
efd79f8… drh 466 ever trigger implicitly (e.g. via dependencies resolution) in a normal
7fcb462… stephan 467 build cycle. They are instead explicitly built as described below.
7fcb462… stephan 468
7fcb462… stephan 469 From the top of the source tree, all WASM-related components can be
7fcb462… stephan 470 built with:
7fcb462… stephan 471
7fcb462… stephan 472 <pre><code>$ make wasm</code></pre>
8a1ba49… wyoung 473
8a1ba49… wyoung 474 <div class="sidebar">The file
8a1ba49… wyoung 475 <tt>[/file/extsrc/pikcher-worker.js|extsrc/pikcher-worker.js]</tt>
8a1ba49… wyoung 476 is hand-coded and intended to be loaded as a "Worker" in
8a1ba49… wyoung 477 JavaScript. That file loads the main module and provides an
8a1ba49… wyoung 478 interface via which a main JavaScript thread can communicate with
8a1ba49… wyoung 479 pikchr running in a Worker thread. The file
8a1ba49… wyoung 480 <tt>[/file/src/fossil.page.pikchrshowasm.js|src/fossil.page.pikchrshowasm.js]</tt>
8a1ba49… wyoung 481 implements the [/pikchrshow] app and demonstrates how
8a1ba49… wyoung 482 <tt>pikchr-worker.js</tt> is used.</div>
7fcb462… stephan 483
7fcb462… stephan 484 As of this writing, those parts include:
7fcb462… stephan 485
7fcb462… stephan 486 * <tt>extsrc/pikchr.wasm</tt> is a WASM-compiled form of
7fcb462… stephan 487 <tt>extsrc/pikchr.c</tt>.
7fcb462… stephan 488 * <tt>extsrc/pikchr.js</tt> is JS/WASM glue code generated by Emscripten
7fcb462… stephan 489 to give JS code access to the API exported by the WASM file.
7fcb462… stephan 490
7fcb462… stephan 491 When a new version of <tt>extsrc/pikchr.c</tt> is installed, the
7fcb462… stephan 492 files <tt>pikchr.{js,wasm}</tt> will need to be recompiled to account
7fcb462… stephan 493 for that. Running <tt>make wasm</tt> will, if the build is set up for
7fcb462… stephan 494 the emsdk, recompile those:
7fcb462… stephan 495
b7338a5… stephan 496 <pre><code>$ rm extsrc/pikchr.{js,wasm}
b7338a5… stephan 497 # ^^^^ that rm has proven necessary in order to ensure rebuilds
b7338a5… stephan 498 $ make wasm
7fcb462… stephan 499 ./tools/emcc.sh -o extsrc/pikchr.js ...
7fcb462… stephan 500 $ ls -la extsrc/pikchr.{js,wasm}
7fcb462… stephan 501 -rw-rw-r-- 1 stephan stephan 17263 Jun 8 03:59 extsrc/pikchr.js
7fcb462… stephan 502 -rw-rw-r-- 1 stephan stephan 97578 Jun 8 03:59 extsrc/pikchr.wasm
7fcb462… stephan 503 </code></pre>
7fcb462… stephan 504
8a1ba49… wyoung 505 <div class="sidebar">If that fails with a message along the lines of
8a1ba49… wyoung 506 “<code>setting `EXPORTED_RUNTIME_METHODS` expects `<class 'list'>` but
8a1ba49… wyoung 507 got `<class 'str'>`</code>” then the emcc being invoked is too old: emcc
8a1ba49… wyoung 508 changed the format of list-type arguments at some point. The required
8a1ba49… wyoung 509 minimum version is unknown, but any SDK version from May 2022 or later
8a1ba49… wyoung 510 "should" (as of this writing) suffice. Any older version may or may not
8a1ba49… wyoung 511 work.</div>
f7d53fa… stephan 512
f7d53fa… stephan 513 After that succeeds, we need to run the normal build so that those
f7d53fa… stephan 514 generated files can be compiled in to the fossil binary, accessible
c64f28d… drh 515 via the [/help/www/builtin|/builtin page]:
7fcb462… stephan 516
7fcb462… stephan 517 <pre><code>$ make</code></pre>
7fcb462… stephan 518
7fcb462… stephan 519 Before checking in those newly-built files, they need to be tested by
7fcb462… stephan 520 running the [/pikchrshow] page. If that page loads, the compilation
7fcb462… stephan 521 process fundamentally worked (a load failure will be made obvious to
7fcb462… stephan 522 the viewer). If it fails to load then the browser's dev tools console
7fcb462… stephan 523 likely provides at least a small hint (and <em>sometimes</em> a useful
7fcb462… stephan 524 hint) about the nature of the problem. Don't check those files in
7fcb462… stephan 525 until [/pikchrshow] runs, though!
7fcb462… stephan 526
f7d53fa… stephan 527 Should pikchr's C interface ever change, <tt>pikchr-worker.js</tt>
7fcb462… stephan 528 will need to be updated to accommodate it, but such modification is
7fcb462… stephan 529 typically trivial.
f7d53fa… stephan 530
f7d53fa… stephan 531 <h3>8.1 Solutions other than Emscripten?</h3>
f7d53fa… stephan 532
f7d53fa… stephan 533 Emscripten is not the only option for building C as WASM, but it
f7d53fa… stephan 534 provides a complete toolchain which eliminates many other steps which
f7d53fa… stephan 535 must otherwise be accounted for on a per-project basis. Despite its
f7d53fa… stephan 536 convenience, it behooves us to explore other build options for the
f7d53fa… stephan 537 sake of portability and avoiding what amounts to vendor lock-in.
f7d53fa… stephan 538
f7d53fa… stephan 539 For later refererence, here are articles specifically covering
f7d53fa… stephan 540 building WASM projects without using Emscripten:
f7d53fa… stephan 541
f7d53fa… stephan 542 * [https://surma.dev/things/c-to-webassembly/]
f7d53fa… stephan 543 * [https://schellcode.github.io/webassembly-without-emscripten]

Keyboard Shortcuts

Open search /
Next entry (timeline) j
Previous entry (timeline) k
Open focused entry Enter
Show this help ?
Toggle theme Top nav button