Fossil SCM
Added a section to www/build.wiki explaining the pikchr WASM bits and how to rebuild them (required when updating pikchr.c).
Commit
ed51ec4ef6810718ba2329e92bf66d8eff197e009a5e9ae52733f926888b10f8
Parent
11f779039e8959c…
2 files changed
+5
-5
+114
+5
-5
| --- extsrc/pikchr-worker.js | ||
| +++ extsrc/pikchr-worker.js | ||
| @@ -8,12 +8,12 @@ | ||
| 8 | 8 | * May you find forgiveness for yourself and forgive others. |
| 9 | 9 | * May you share freely, never taking more than you give. |
| 10 | 10 | |
| 11 | 11 | *********************************************************************** |
| 12 | 12 | |
| 13 | - This is the JS Worker file for the pikchr fiddle app. It loads the | |
| 14 | - pikchr wasm module and offers access to it via the Worker | |
| 13 | + This is a JS Worker file for use with the pikchr wasm build. It | |
| 14 | + loads the pikchr wasm module and offers access to it via the Worker | |
| 15 | 15 | message-passing interface. |
| 16 | 16 | |
| 17 | 17 | Because we can have only a single message handler, as opposed to an |
| 18 | 18 | arbitrary number of discrete event listeners like with DOM elements, |
| 19 | 19 | we have to define a lower-level message API. Messages abstractly |
| @@ -66,14 +66,14 @@ | ||
| 66 | 66 | |
| 67 | 67 | {type: 'pikchr', |
| 68 | 68 | data:{ |
| 69 | 69 | pikchr: input text, |
| 70 | 70 | result: rendered result (SVG on success, HTML on error), |
| 71 | - isError: bool, true if .pikchr holds an error report, | |
| 71 | + isError: bool, true if .result holds an error report, | |
| 72 | 72 | flags: integer: flags used to configure the pikchr rendering, |
| 73 | - width: if !isError, width (pixels) of the SVG, | |
| 74 | - height: if !isError, height (pixels) of the SVG | |
| 73 | + width: if !isError, width (integer pixels) of the SVG, | |
| 74 | + height: if !isError, height (integer pixels) of the SVG | |
| 75 | 75 | } |
| 76 | 76 | } |
| 77 | 77 | |
| 78 | 78 | */ |
| 79 | 79 | |
| 80 | 80 |
| --- extsrc/pikchr-worker.js | |
| +++ extsrc/pikchr-worker.js | |
| @@ -8,12 +8,12 @@ | |
| 8 | * May you find forgiveness for yourself and forgive others. |
| 9 | * May you share freely, never taking more than you give. |
| 10 | |
| 11 | *********************************************************************** |
| 12 | |
| 13 | This is the JS Worker file for the pikchr fiddle app. It loads the |
| 14 | pikchr wasm module and offers access to it via the Worker |
| 15 | message-passing interface. |
| 16 | |
| 17 | Because we can have only a single message handler, as opposed to an |
| 18 | arbitrary number of discrete event listeners like with DOM elements, |
| 19 | we have to define a lower-level message API. Messages abstractly |
| @@ -66,14 +66,14 @@ | |
| 66 | |
| 67 | {type: 'pikchr', |
| 68 | data:{ |
| 69 | pikchr: input text, |
| 70 | result: rendered result (SVG on success, HTML on error), |
| 71 | isError: bool, true if .pikchr holds an error report, |
| 72 | flags: integer: flags used to configure the pikchr rendering, |
| 73 | width: if !isError, width (pixels) of the SVG, |
| 74 | height: if !isError, height (pixels) of the SVG |
| 75 | } |
| 76 | } |
| 77 | |
| 78 | */ |
| 79 | |
| 80 |
| --- extsrc/pikchr-worker.js | |
| +++ extsrc/pikchr-worker.js | |
| @@ -8,12 +8,12 @@ | |
| 8 | * May you find forgiveness for yourself and forgive others. |
| 9 | * May you share freely, never taking more than you give. |
| 10 | |
| 11 | *********************************************************************** |
| 12 | |
| 13 | This is a JS Worker file for use with the pikchr wasm build. It |
| 14 | loads the pikchr wasm module and offers access to it via the Worker |
| 15 | message-passing interface. |
| 16 | |
| 17 | Because we can have only a single message handler, as opposed to an |
| 18 | arbitrary number of discrete event listeners like with DOM elements, |
| 19 | we have to define a lower-level message API. Messages abstractly |
| @@ -66,14 +66,14 @@ | |
| 66 | |
| 67 | {type: 'pikchr', |
| 68 | data:{ |
| 69 | pikchr: input text, |
| 70 | result: rendered result (SVG on success, HTML on error), |
| 71 | isError: bool, true if .result holds an error report, |
| 72 | flags: integer: flags used to configure the pikchr rendering, |
| 73 | width: if !isError, width (integer pixels) of the SVG, |
| 74 | height: if !isError, height (integer pixels) of the SVG |
| 75 | } |
| 76 | } |
| 77 | |
| 78 | */ |
| 79 | |
| 80 |
+114
| --- www/build.wiki | ||
| +++ www/build.wiki | ||
| @@ -460,5 +460,119 @@ | ||
| 460 | 460 | e.g. <tt>-jobs=4</tt> to start four fuzzer jobs in parallel, but doing |
| 461 | 461 | so may cause the fuzzer to <em>strip the --fuzztype flag</em>, leading |
| 462 | 462 | to it testing the wrong thing. When passing on fuzzer-specific flags |
| 463 | 463 | along with <tt>--fuzztype</tt>, be sure to check your system's process |
| 464 | 464 | list to ensure that your <tt>--fuzztype</tt> flag is there. |
| 465 | + | |
| 466 | + | |
| 467 | +<a id='wasm'></a> | |
| 468 | +<h2>Building WebAssembly Components</h2> | |
| 469 | + | |
| 470 | +As of version 2.19, fossil uses one component built as | |
| 471 | +[https://developer.mozilla.org/en-US/docs/WebAssembly | WebAssembly] | |
| 472 | +a.k.a. WASM. Because compiling WASM code requires non-trivial | |
| 473 | +client-side tooling, the repository includes compiled copies of these | |
| 474 | +pieces. Most Fossil hackers should never need to concern themselves | |
| 475 | +with the WASM parts, but this section describes how to for those who | |
| 476 | +want or need to do so. | |
| 477 | + | |
| 478 | +<strong>The bits described in this section are necessary when updating | |
| 479 | +<tt>extsrc/pikchr.c</tt></strong> from the upstream source, or the | |
| 480 | +fossil binary will use a different version of pikchr than | |
| 481 | +[/pikchrshow] does (as the latter runs the WASM build of pikchr). | |
| 482 | + | |
| 483 | +These instructions have only ever been tested on Linux systems. They | |
| 484 | +"should" work on any Unix-like system supported by Emscripten. The | |
| 485 | +fossil makefiles for Windows builds <em>do not</em> include any of the | |
| 486 | +WASM-related components (patches to add that would be welcomed, of | |
| 487 | +course). | |
| 488 | + | |
| 489 | +The first step is to configure the tree with support for | |
| 490 | +[https://emscripten.org/|Emscripten]. This requires that the system | |
| 491 | +has the Emscripten SDK (a.k.a. emsdk) installed, as documented at: | |
| 492 | + | |
| 493 | +[https://emscripten.org/docs/getting_started/downloads.html] | |
| 494 | + | |
| 495 | + Sidebar: getting Emscripten up and running is trivial and | |
| 496 | + painless, at least on Linux systems, but the installer | |
| 497 | + downloads many hundreds of megabytes of tools and dependencies, | |
| 498 | + all of which will be installed under the single SDK directory (as | |
| 499 | + opposed to being installed at the system level). | |
| 500 | + | |
| 501 | +After installing the SDK, configure the fossil tree with emsdk | |
| 502 | +support: | |
| 503 | + | |
| 504 | +<pre><code>$ ./configure --with-emsdk=/path/to/emsdk ...other options... | |
| 505 | +</code></pre> | |
| 506 | + | |
| 507 | +If the <tt>--with-emsdk</tt> flag is not provided, the configure | |
| 508 | +script will check for the environment variable <tt>EMSDK</tt>, which | |
| 509 | +is one of the standard variables the SDK environment uses. If that | |
| 510 | +variable is found, its value will implicitly be used in place of the | |
| 511 | +missing <tt>--with-emsdk</tt> flag. Thus, if the <tt>emsdk_env.sh</tt> | |
| 512 | +script is sourced into the shell before running the configure script, | |
| 513 | +the SDK will be detected even without the config flag. | |
| 514 | + | |
| 515 | +The configure script installs some makefile variables which tell the | |
| 516 | +build where to find the SDK and it generates a script named | |
| 517 | +<tt>tools/emcc.sh</tt> (from the template file | |
| 518 | +<tt>[/file/tools/emcc.sh.in|/tools/emcc.sh.in]</tt>), which is a | |
| 519 | +wrapper around the Emscripten C compiler (<tt>emcc</tt>). The wrapper | |
| 520 | +script uses the configure-time state to attempt to set up the various | |
| 521 | +environment variables which are required by <tt>emcc</tt> and will | |
| 522 | +fail if it cannot do so. Once it's set up the environment, it passes | |
| 523 | +on all of its arguments to <tt>emcc</tt>. | |
| 524 | + | |
| 525 | +The WASM-related build parts are set up such that none of them should | |
| 526 | +ever trigger implicity (e.g. via dependencies resolution) in a normal | |
| 527 | +build cycle. They are instead explicitly built as described below. | |
| 528 | + | |
| 529 | +From the top of the source tree, all WASM-related components can be | |
| 530 | +built with: | |
| 531 | + | |
| 532 | +<pre><code>$ make wasm</code></pre> | |
| 533 | + | |
| 534 | +As of this writing, those parts include: | |
| 535 | + | |
| 536 | + * <tt>extsrc/pikchr.wasm</tt> is a WASM-compiled form of | |
| 537 | + <tt>extsrc/pikchr.c</tt>. | |
| 538 | + * <tt>extsrc/pikchr.js</tt> is JS/WASM glue code generated by Emscripten | |
| 539 | + to give JS code access to the API exported by the WASM file. | |
| 540 | + | |
| 541 | + Sidebar: The file | |
| 542 | + <tt>[/file/extsrc/pikcher-worker.js|extsrc/pikcher-worker.js]</tt> | |
| 543 | + is hand-coded and intended to be loaded as a "Worker" in | |
| 544 | + JavaScript. That file loads the main module and provides an | |
| 545 | + interface via which a main JavaScript thread can communicate with | |
| 546 | + pikchr running in a Worker thread. The file | |
| 547 | + <tt>[/file/src/fossil.page.pikchrshowasm.js|src/fossil.page.pikchrshowasm.js]</tt> | |
| 548 | + implements the [/pikchrshow] app and demonstrates how | |
| 549 | + <tt>pikchr-worker.js</tt> is used. | |
| 550 | + | |
| 551 | +When a new version of <tt>extsrc/pikchr.c</tt> is installed, the | |
| 552 | +files <tt>pikchr.{js,wasm}</tt> will need to be recompiled to account | |
| 553 | +for that. Running <tt>make wasm</tt> will, if the build is set up for | |
| 554 | +the emsdk, recompile those: | |
| 555 | + | |
| 556 | +<pre><code>$ make wasm | |
| 557 | +./tools/emcc.sh -o extsrc/pikchr.js ... | |
| 558 | +$ ls -la extsrc/pikchr.{js,wasm} | |
| 559 | +-rw-rw-r-- 1 stephan stephan 17263 Jun 8 03:59 extsrc/pikchr.js | |
| 560 | +-rw-rw-r-- 1 stephan stephan 97578 Jun 8 03:59 extsrc/pikchr.wasm | |
| 561 | +</code></pre> | |
| 562 | + | |
| 563 | +Then run the normal build so that those files can be compiled in to | |
| 564 | +the fossil binary, accessible via the [/help?cmd=/builtin|/builtin page]: | |
| 565 | + | |
| 566 | +<pre><code>$ make</code></pre> | |
| 567 | + | |
| 568 | +Before checking in those newly-built files, they need to be tested by | |
| 569 | +running the [/pikchrshow] page. If that page loads, the compilation | |
| 570 | +process fundamentally worked (a load failure will be made obvious to | |
| 571 | +the viewer). If it fails to load then the browser's dev tools console | |
| 572 | +likely provides at least a small hint (and <em>sometimes</em> a useful | |
| 573 | +hint) about the nature of the problem. Don't check those files in | |
| 574 | +until [/pikchrshow] runs, though! | |
| 575 | + | |
| 576 | +Should pikchr's C interface ever changes, <tt>pikchr-worker.js</tt> | |
| 577 | +will need to be updated to accommodate it, but such modification is | |
| 578 | +typically trivial. | |
| 465 | 579 |
| --- www/build.wiki | |
| +++ www/build.wiki | |
| @@ -460,5 +460,119 @@ | |
| 460 | e.g. <tt>-jobs=4</tt> to start four fuzzer jobs in parallel, but doing |
| 461 | so may cause the fuzzer to <em>strip the --fuzztype flag</em>, leading |
| 462 | to it testing the wrong thing. When passing on fuzzer-specific flags |
| 463 | along with <tt>--fuzztype</tt>, be sure to check your system's process |
| 464 | list to ensure that your <tt>--fuzztype</tt> flag is there. |
| 465 |
| --- www/build.wiki | |
| +++ www/build.wiki | |
| @@ -460,5 +460,119 @@ | |
| 460 | e.g. <tt>-jobs=4</tt> to start four fuzzer jobs in parallel, but doing |
| 461 | so may cause the fuzzer to <em>strip the --fuzztype flag</em>, leading |
| 462 | to it testing the wrong thing. When passing on fuzzer-specific flags |
| 463 | along with <tt>--fuzztype</tt>, be sure to check your system's process |
| 464 | list to ensure that your <tt>--fuzztype</tt> flag is there. |
| 465 | |
| 466 | |
| 467 | <a id='wasm'></a> |
| 468 | <h2>Building WebAssembly Components</h2> |
| 469 | |
| 470 | As of version 2.19, fossil uses one component built as |
| 471 | [https://developer.mozilla.org/en-US/docs/WebAssembly | WebAssembly] |
| 472 | a.k.a. WASM. Because compiling WASM code requires non-trivial |
| 473 | client-side tooling, the repository includes compiled copies of these |
| 474 | pieces. Most Fossil hackers should never need to concern themselves |
| 475 | with the WASM parts, but this section describes how to for those who |
| 476 | want or need to do so. |
| 477 | |
| 478 | <strong>The bits described in this section are necessary when updating |
| 479 | <tt>extsrc/pikchr.c</tt></strong> from the upstream source, or the |
| 480 | fossil binary will use a different version of pikchr than |
| 481 | [/pikchrshow] does (as the latter runs the WASM build of pikchr). |
| 482 | |
| 483 | These instructions have only ever been tested on Linux systems. They |
| 484 | "should" work on any Unix-like system supported by Emscripten. The |
| 485 | fossil makefiles for Windows builds <em>do not</em> include any of the |
| 486 | WASM-related components (patches to add that would be welcomed, of |
| 487 | course). |
| 488 | |
| 489 | The first step is to configure the tree with support for |
| 490 | [https://emscripten.org/|Emscripten]. This requires that the system |
| 491 | has the Emscripten SDK (a.k.a. emsdk) installed, as documented at: |
| 492 | |
| 493 | [https://emscripten.org/docs/getting_started/downloads.html] |
| 494 | |
| 495 | Sidebar: getting Emscripten up and running is trivial and |
| 496 | painless, at least on Linux systems, but the installer |
| 497 | downloads many hundreds of megabytes of tools and dependencies, |
| 498 | all of which will be installed under the single SDK directory (as |
| 499 | opposed to being installed at the system level). |
| 500 | |
| 501 | After installing the SDK, configure the fossil tree with emsdk |
| 502 | support: |
| 503 | |
| 504 | <pre><code>$ ./configure --with-emsdk=/path/to/emsdk ...other options... |
| 505 | </code></pre> |
| 506 | |
| 507 | If the <tt>--with-emsdk</tt> flag is not provided, the configure |
| 508 | script will check for the environment variable <tt>EMSDK</tt>, which |
| 509 | is one of the standard variables the SDK environment uses. If that |
| 510 | variable is found, its value will implicitly be used in place of the |
| 511 | missing <tt>--with-emsdk</tt> flag. Thus, if the <tt>emsdk_env.sh</tt> |
| 512 | script is sourced into the shell before running the configure script, |
| 513 | the SDK will be detected even without the config flag. |
| 514 | |
| 515 | The configure script installs some makefile variables which tell the |
| 516 | build where to find the SDK and it generates a script named |
| 517 | <tt>tools/emcc.sh</tt> (from the template file |
| 518 | <tt>[/file/tools/emcc.sh.in|/tools/emcc.sh.in]</tt>), which is a |
| 519 | wrapper around the Emscripten C compiler (<tt>emcc</tt>). The wrapper |
| 520 | script uses the configure-time state to attempt to set up the various |
| 521 | environment variables which are required by <tt>emcc</tt> and will |
| 522 | fail if it cannot do so. Once it's set up the environment, it passes |
| 523 | on all of its arguments to <tt>emcc</tt>. |
| 524 | |
| 525 | The WASM-related build parts are set up such that none of them should |
| 526 | ever trigger implicity (e.g. via dependencies resolution) in a normal |
| 527 | build cycle. They are instead explicitly built as described below. |
| 528 | |
| 529 | From the top of the source tree, all WASM-related components can be |
| 530 | built with: |
| 531 | |
| 532 | <pre><code>$ make wasm</code></pre> |
| 533 | |
| 534 | As of this writing, those parts include: |
| 535 | |
| 536 | * <tt>extsrc/pikchr.wasm</tt> is a WASM-compiled form of |
| 537 | <tt>extsrc/pikchr.c</tt>. |
| 538 | * <tt>extsrc/pikchr.js</tt> is JS/WASM glue code generated by Emscripten |
| 539 | to give JS code access to the API exported by the WASM file. |
| 540 | |
| 541 | Sidebar: The file |
| 542 | <tt>[/file/extsrc/pikcher-worker.js|extsrc/pikcher-worker.js]</tt> |
| 543 | is hand-coded and intended to be loaded as a "Worker" in |
| 544 | JavaScript. That file loads the main module and provides an |
| 545 | interface via which a main JavaScript thread can communicate with |
| 546 | pikchr running in a Worker thread. The file |
| 547 | <tt>[/file/src/fossil.page.pikchrshowasm.js|src/fossil.page.pikchrshowasm.js]</tt> |
| 548 | implements the [/pikchrshow] app and demonstrates how |
| 549 | <tt>pikchr-worker.js</tt> is used. |
| 550 | |
| 551 | When a new version of <tt>extsrc/pikchr.c</tt> is installed, the |
| 552 | files <tt>pikchr.{js,wasm}</tt> will need to be recompiled to account |
| 553 | for that. Running <tt>make wasm</tt> will, if the build is set up for |
| 554 | the emsdk, recompile those: |
| 555 | |
| 556 | <pre><code>$ make wasm |
| 557 | ./tools/emcc.sh -o extsrc/pikchr.js ... |
| 558 | $ ls -la extsrc/pikchr.{js,wasm} |
| 559 | -rw-rw-r-- 1 stephan stephan 17263 Jun 8 03:59 extsrc/pikchr.js |
| 560 | -rw-rw-r-- 1 stephan stephan 97578 Jun 8 03:59 extsrc/pikchr.wasm |
| 561 | </code></pre> |
| 562 | |
| 563 | Then run the normal build so that those files can be compiled in to |
| 564 | the fossil binary, accessible via the [/help?cmd=/builtin|/builtin page]: |
| 565 | |
| 566 | <pre><code>$ make</code></pre> |
| 567 | |
| 568 | Before checking in those newly-built files, they need to be tested by |
| 569 | running the [/pikchrshow] page. If that page loads, the compilation |
| 570 | process fundamentally worked (a load failure will be made obvious to |
| 571 | the viewer). If it fails to load then the browser's dev tools console |
| 572 | likely provides at least a small hint (and <em>sometimes</em> a useful |
| 573 | hint) about the nature of the problem. Don't check those files in |
| 574 | until [/pikchrshow] runs, though! |
| 575 | |
| 576 | Should pikchr's C interface ever changes, <tt>pikchr-worker.js</tt> |
| 577 | will need to be updated to accommodate it, but such modification is |
| 578 | typically trivial. |
| 579 |