Fossil SCM
Do a better job of detecting misformatted structural artifacts and reporting the error. Ticket [15d04de574383d61].
Commit
023fddeec402930662a022c1e4bd2ea63027ed0c8384d9e8bfd1bd0540f47de3
Parent
1cd6c545ca3a9c6…
2 files changed
+14
-1
+43
-2
+14
-1
| --- src/manifest.c | ||
| +++ src/manifest.c | ||
| @@ -534,14 +534,27 @@ | ||
| 534 | 534 | /* Begin parsing, card by card. |
| 535 | 535 | */ |
| 536 | 536 | x.z = z; |
| 537 | 537 | x.zEnd = &z[n]; |
| 538 | 538 | x.atEol = 1; |
| 539 | - while( (cType = next_card(&x))!=0 && cType>=cPrevType ){ | |
| 539 | + while( (cType = next_card(&x))!=0 ){ | |
| 540 | + if( cType<cPrevType ){ | |
| 541 | + /* Cards must be in increasing order. However, out-of-order detection | |
| 542 | + ** was broken prior to 2021-02-10 due to a bug. Furthermore, there | |
| 543 | + ** was a bug in technote generation (prior to 2021-02-10) that caused | |
| 544 | + ** the P card to occur before the N card. Hence, for historical | |
| 545 | + ** compatibility, we do allow the N card of a technote to occur after | |
| 546 | + ** the P card. See tickets 15d04de574383d61 and 5e67a7f4041a36ad. | |
| 547 | + */ | |
| 548 | + if( cType!='N' || cPrevType!='P' || p->zEventId==0 ){ | |
| 549 | + SYNTAX("cards not in lexicographical order"); | |
| 550 | + } | |
| 551 | + } | |
| 540 | 552 | lineNo++; |
| 541 | 553 | if( cType<'A' || cType>'Z' ) SYNTAX("bad card type"); |
| 542 | 554 | seenCard |= 1 << (cType-'A'); |
| 555 | + cPrevType = cType; | |
| 543 | 556 | switch( cType ){ |
| 544 | 557 | /* |
| 545 | 558 | ** A <filename> <target> ?<source>? |
| 546 | 559 | ** |
| 547 | 560 | ** Identifies an attachment to either a wiki page or a ticket. |
| 548 | 561 |
| --- src/manifest.c | |
| +++ src/manifest.c | |
| @@ -534,14 +534,27 @@ | |
| 534 | /* Begin parsing, card by card. |
| 535 | */ |
| 536 | x.z = z; |
| 537 | x.zEnd = &z[n]; |
| 538 | x.atEol = 1; |
| 539 | while( (cType = next_card(&x))!=0 && cType>=cPrevType ){ |
| 540 | lineNo++; |
| 541 | if( cType<'A' || cType>'Z' ) SYNTAX("bad card type"); |
| 542 | seenCard |= 1 << (cType-'A'); |
| 543 | switch( cType ){ |
| 544 | /* |
| 545 | ** A <filename> <target> ?<source>? |
| 546 | ** |
| 547 | ** Identifies an attachment to either a wiki page or a ticket. |
| 548 |
| --- src/manifest.c | |
| +++ src/manifest.c | |
| @@ -534,14 +534,27 @@ | |
| 534 | /* Begin parsing, card by card. |
| 535 | */ |
| 536 | x.z = z; |
| 537 | x.zEnd = &z[n]; |
| 538 | x.atEol = 1; |
| 539 | while( (cType = next_card(&x))!=0 ){ |
| 540 | if( cType<cPrevType ){ |
| 541 | /* Cards must be in increasing order. However, out-of-order detection |
| 542 | ** was broken prior to 2021-02-10 due to a bug. Furthermore, there |
| 543 | ** was a bug in technote generation (prior to 2021-02-10) that caused |
| 544 | ** the P card to occur before the N card. Hence, for historical |
| 545 | ** compatibility, we do allow the N card of a technote to occur after |
| 546 | ** the P card. See tickets 15d04de574383d61 and 5e67a7f4041a36ad. |
| 547 | */ |
| 548 | if( cType!='N' || cPrevType!='P' || p->zEventId==0 ){ |
| 549 | SYNTAX("cards not in lexicographical order"); |
| 550 | } |
| 551 | } |
| 552 | lineNo++; |
| 553 | if( cType<'A' || cType>'Z' ) SYNTAX("bad card type"); |
| 554 | seenCard |= 1 << (cType-'A'); |
| 555 | cPrevType = cType; |
| 556 | switch( cType ){ |
| 557 | /* |
| 558 | ** A <filename> <target> ?<source>? |
| 559 | ** |
| 560 | ** Identifies an attachment to either a wiki page or a ticket. |
| 561 |
+43
-2
| --- www/fileformat.wiki | ||
| +++ www/fileformat.wiki | ||
| @@ -86,11 +86,13 @@ | ||
| 86 | 86 | the card type. All arguments are separated from each other |
| 87 | 87 | and from the card-type character by a single space |
| 88 | 88 | character. There is no surplus white space between arguments |
| 89 | 89 | and no leading or trailing whitespace except for the newline |
| 90 | 90 | character that acts as the card separator. All cards must be in strict |
| 91 | -lexicographical order. There may not be any duplicate cards. | |
| 91 | +lexicographical order (except for an | |
| 92 | +[./fileformat.wiki#outofordercards|historical bug compatibility]). | |
| 93 | +There may not be any duplicate cards. | |
| 92 | 94 | |
| 93 | 95 | In the current implementation (as of 2017-02-27) the artifacts that |
| 94 | 96 | make up a fossil repository are stored as delta- and zlib-compressed |
| 95 | 97 | blobs in an <a href="http://www.sqlite.org/">SQLite</a> database. This |
| 96 | 98 | is an implementation detail and might change in a future release. For |
| @@ -873,11 +875,50 @@ | ||
| 873 | 875 | <h2>4.0 Addenda</h2> |
| 874 | 876 | |
| 875 | 877 | This section contains additional information which may be useful when |
| 876 | 878 | implementing algorithms described above. |
| 877 | 879 | |
| 878 | -<h3>4.1 R-Card Hash Calculation</h3> | |
| 880 | +<a name="outofordercards"></a> | |
| 881 | +<h3>4.1 Relaxed Card Ordering Due To An Historical Bug</h3> | |
| 882 | + | |
| 883 | +All cards of a structural artifact should be in lexicographical order. | |
| 884 | +The Fossil implementation verifies this and rejects any structural | |
| 885 | +artifact which has out-of-order cards. Futhermore, when Fossil is | |
| 886 | +generating new structural artifacts, it runs the generated artifact | |
| 887 | +through the parser to confirm that all cards really are in the correct | |
| 888 | +order before committing the transaction. In this way, Fossil prevents | |
| 889 | +bugs in the code from accidentally inserting misformatted artifacts. | |
| 890 | +The test parse of newly created artifacts is part of the | |
| 891 | +[./selfcheck.wiki|self-check strategy] of Fossil. It takes a | |
| 892 | +few more CPU cycles to double check each artifact before inserting it. | |
| 893 | +The developers consider those CPU cycles well-spent. | |
| 894 | + | |
| 895 | +However, the card-order safety check was accidentally disabled due to | |
| 896 | +[15d04de574383d61|a bug]. | |
| 897 | +And while that bug was lurking undetected in the code, | |
| 898 | +[5e67a7f4041a36ad|another bug] caused the N cards of Technical Notes | |
| 899 | +to occur after the P card rather than before. | |
| 900 | +Thus for a span of several years, Technical Note artifacts were being | |
| 901 | +inserted into Fossil repositories that had their N and P cards in the | |
| 902 | +wrong order. | |
| 903 | + | |
| 904 | +Both bugs have now been fixed. However, to prevent historical | |
| 905 | +Technical Note artifacts that were inserted by users in good faith | |
| 906 | +from being rejected by newer Fossil builds, the card ordering | |
| 907 | +requirement is relaxed slightly. The actual implementation is this: | |
| 908 | + | |
| 909 | +<blockquote> | |
| 910 | +"All cards must be in strict lexicographic order, except that the | |
| 911 | +N and P cards of a Technical Note artifact are allowed to be | |
| 912 | +interchanged." | |
| 913 | +</blockquote> | |
| 914 | + | |
| 915 | +Future versions of Fossil might strengthen this slightly to only allow | |
| 916 | +the out of order N and P cards for Technical Notes entered before | |
| 917 | +a certain date. | |
| 918 | + | |
| 919 | +<h3>4.2 R-Card Hash Calculation</h3> | |
| 879 | 920 | |
| 880 | 921 | Given a manifest file named <tt>MF</tt>, the following Bash shell code |
| 881 | 922 | demonstrates how to compute the value of the <b>R</b> card in that manifest. |
| 882 | 923 | This example uses manifest [28987096ac]. Lines starting with <tt>#</tt> are |
| 883 | 924 | shell input and other lines are output. This demonstration assumes that the |
| 884 | 925 |
| --- www/fileformat.wiki | |
| +++ www/fileformat.wiki | |
| @@ -86,11 +86,13 @@ | |
| 86 | the card type. All arguments are separated from each other |
| 87 | and from the card-type character by a single space |
| 88 | character. There is no surplus white space between arguments |
| 89 | and no leading or trailing whitespace except for the newline |
| 90 | character that acts as the card separator. All cards must be in strict |
| 91 | lexicographical order. There may not be any duplicate cards. |
| 92 | |
| 93 | In the current implementation (as of 2017-02-27) the artifacts that |
| 94 | make up a fossil repository are stored as delta- and zlib-compressed |
| 95 | blobs in an <a href="http://www.sqlite.org/">SQLite</a> database. This |
| 96 | is an implementation detail and might change in a future release. For |
| @@ -873,11 +875,50 @@ | |
| 873 | <h2>4.0 Addenda</h2> |
| 874 | |
| 875 | This section contains additional information which may be useful when |
| 876 | implementing algorithms described above. |
| 877 | |
| 878 | <h3>4.1 R-Card Hash Calculation</h3> |
| 879 | |
| 880 | Given a manifest file named <tt>MF</tt>, the following Bash shell code |
| 881 | demonstrates how to compute the value of the <b>R</b> card in that manifest. |
| 882 | This example uses manifest [28987096ac]. Lines starting with <tt>#</tt> are |
| 883 | shell input and other lines are output. This demonstration assumes that the |
| 884 |
| --- www/fileformat.wiki | |
| +++ www/fileformat.wiki | |
| @@ -86,11 +86,13 @@ | |
| 86 | the card type. All arguments are separated from each other |
| 87 | and from the card-type character by a single space |
| 88 | character. There is no surplus white space between arguments |
| 89 | and no leading or trailing whitespace except for the newline |
| 90 | character that acts as the card separator. All cards must be in strict |
| 91 | lexicographical order (except for an |
| 92 | [./fileformat.wiki#outofordercards|historical bug compatibility]). |
| 93 | There may not be any duplicate cards. |
| 94 | |
| 95 | In the current implementation (as of 2017-02-27) the artifacts that |
| 96 | make up a fossil repository are stored as delta- and zlib-compressed |
| 97 | blobs in an <a href="http://www.sqlite.org/">SQLite</a> database. This |
| 98 | is an implementation detail and might change in a future release. For |
| @@ -873,11 +875,50 @@ | |
| 875 | <h2>4.0 Addenda</h2> |
| 876 | |
| 877 | This section contains additional information which may be useful when |
| 878 | implementing algorithms described above. |
| 879 | |
| 880 | <a name="outofordercards"></a> |
| 881 | <h3>4.1 Relaxed Card Ordering Due To An Historical Bug</h3> |
| 882 | |
| 883 | All cards of a structural artifact should be in lexicographical order. |
| 884 | The Fossil implementation verifies this and rejects any structural |
| 885 | artifact which has out-of-order cards. Futhermore, when Fossil is |
| 886 | generating new structural artifacts, it runs the generated artifact |
| 887 | through the parser to confirm that all cards really are in the correct |
| 888 | order before committing the transaction. In this way, Fossil prevents |
| 889 | bugs in the code from accidentally inserting misformatted artifacts. |
| 890 | The test parse of newly created artifacts is part of the |
| 891 | [./selfcheck.wiki|self-check strategy] of Fossil. It takes a |
| 892 | few more CPU cycles to double check each artifact before inserting it. |
| 893 | The developers consider those CPU cycles well-spent. |
| 894 | |
| 895 | However, the card-order safety check was accidentally disabled due to |
| 896 | [15d04de574383d61|a bug]. |
| 897 | And while that bug was lurking undetected in the code, |
| 898 | [5e67a7f4041a36ad|another bug] caused the N cards of Technical Notes |
| 899 | to occur after the P card rather than before. |
| 900 | Thus for a span of several years, Technical Note artifacts were being |
| 901 | inserted into Fossil repositories that had their N and P cards in the |
| 902 | wrong order. |
| 903 | |
| 904 | Both bugs have now been fixed. However, to prevent historical |
| 905 | Technical Note artifacts that were inserted by users in good faith |
| 906 | from being rejected by newer Fossil builds, the card ordering |
| 907 | requirement is relaxed slightly. The actual implementation is this: |
| 908 | |
| 909 | <blockquote> |
| 910 | "All cards must be in strict lexicographic order, except that the |
| 911 | N and P cards of a Technical Note artifact are allowed to be |
| 912 | interchanged." |
| 913 | </blockquote> |
| 914 | |
| 915 | Future versions of Fossil might strengthen this slightly to only allow |
| 916 | the out of order N and P cards for Technical Notes entered before |
| 917 | a certain date. |
| 918 | |
| 919 | <h3>4.2 R-Card Hash Calculation</h3> |
| 920 | |
| 921 | Given a manifest file named <tt>MF</tt>, the following Bash shell code |
| 922 | demonstrates how to compute the value of the <b>R</b> card in that manifest. |
| 923 | This example uses manifest [28987096ac]. Lines starting with <tt>#</tt> are |
| 924 | shell input and other lines are output. This demonstration assumes that the |
| 925 |