Fossil SCM
Additional documentation updates.
Commit
adc0b3bfb0daaa7677179c5c48b062d88536fe4d
Parent
8d8a41d195b81dc…
4 files changed
+1
-1
+11
-3
+14
-14
+38
-21
+1
-1
| --- www/delta_encoder_algorithm.wiki | ||
| +++ www/delta_encoder_algorithm.wiki | ||
| @@ -15,11 +15,11 @@ | ||
| 15 | 15 | not necessary to understand encoder operation, can be found in the |
| 16 | 16 | companion specification titled "<a href="delta_format.wiki">Fossil |
| 17 | 17 | Delta Format</a>". |
| 18 | 18 | </p> |
| 19 | 19 | |
| 20 | -<p>The entire algorithm is inspired | |
| 20 | +<p>The algorithm is inspired | |
| 21 | 21 | by <a href="http://samba.anu.edu.au/rsync/">rsync</a>.</p> |
| 22 | 22 | |
| 23 | 23 | <a name="argresparam"></a><h2>1.0 Arguments, Results, and Parameters</h2> |
| 24 | 24 | |
| 25 | 25 | <p>The encoder takes two byte-sequences as input, the "original", and |
| 26 | 26 |
| --- www/delta_encoder_algorithm.wiki | |
| +++ www/delta_encoder_algorithm.wiki | |
| @@ -15,11 +15,11 @@ | |
| 15 | not necessary to understand encoder operation, can be found in the |
| 16 | companion specification titled "<a href="delta_format.wiki">Fossil |
| 17 | Delta Format</a>". |
| 18 | </p> |
| 19 | |
| 20 | <p>The entire algorithm is inspired |
| 21 | by <a href="http://samba.anu.edu.au/rsync/">rsync</a>.</p> |
| 22 | |
| 23 | <a name="argresparam"></a><h2>1.0 Arguments, Results, and Parameters</h2> |
| 24 | |
| 25 | <p>The encoder takes two byte-sequences as input, the "original", and |
| 26 |
| --- www/delta_encoder_algorithm.wiki | |
| +++ www/delta_encoder_algorithm.wiki | |
| @@ -15,11 +15,11 @@ | |
| 15 | not necessary to understand encoder operation, can be found in the |
| 16 | companion specification titled "<a href="delta_format.wiki">Fossil |
| 17 | Delta Format</a>". |
| 18 | </p> |
| 19 | |
| 20 | <p>The algorithm is inspired |
| 21 | by <a href="http://samba.anu.edu.au/rsync/">rsync</a>.</p> |
| 22 | |
| 23 | <a name="argresparam"></a><h2>1.0 Arguments, Results, and Parameters</h2> |
| 24 | |
| 25 | <p>The encoder takes two byte-sequences as input, the "original", and |
| 26 |
+11
-3
| --- www/fileformat.wiki | ||
| +++ www/fileformat.wiki | ||
| @@ -1,16 +1,22 @@ | ||
| 1 | 1 | <h1 align="center"> |
| 2 | 2 | Fossil File Formats |
| 3 | 3 | </h1> |
| 4 | 4 | |
| 5 | +<p>The state of a fossil repository is kept simple so that it can | |
| 6 | +endure in useful form for decades or centuries. | |
| 7 | +A fossil repository is intended to be readable, | |
| 8 | +searchable, and extensible by people not yet born.</p> | |
| 9 | + | |
| 5 | 10 | <p> |
| 6 | 11 | The global state of a fossil repository is determined by an unordered |
| 7 | -set of artifacts. | |
| 12 | +set of <i>artifacts</i>. | |
| 8 | 13 | An artifact might be a source code file, the text of a wiki page, |
| 9 | 14 | part of a trouble ticket, or one of several special control artifacts |
| 10 | 15 | used to show the relationships between other artifacts within the |
| 11 | -project. Artifacts can be text or binary. | |
| 16 | +project. Each artifact is normally represented on disk as a separate | |
| 17 | +file. Artifacts can be text or binary. | |
| 12 | 18 | </p> |
| 13 | 19 | |
| 14 | 20 | <p> |
| 15 | 21 | Each artifact in the repository is named by its SHA1 hash. |
| 16 | 22 | No prefixes or meta information is added to a artifact before |
| @@ -167,11 +173,13 @@ | ||
| 167 | 173 | <h2>2.0 Clusters</h2> |
| 168 | 174 | |
| 169 | 175 | <p> |
| 170 | 176 | A cluster is a artifact that declares the existance of other artifacts. |
| 171 | 177 | Clusters are used during repository synchronization to help |
| 172 | -reduce network traffic. | |
| 178 | +reduce network traffic. As such, clusters are an optimization and | |
| 179 | +may be removed from a repository without loss or damage to the | |
| 180 | +underlying project code. | |
| 173 | 181 | </p> |
| 174 | 182 | |
| 175 | 183 | <p> |
| 176 | 184 | Clusters follow a syntax that is very similar to manifests. |
| 177 | 185 | A Cluster is a line-oriented text file. Newline characters |
| 178 | 186 |
| --- www/fileformat.wiki | |
| +++ www/fileformat.wiki | |
| @@ -1,16 +1,22 @@ | |
| 1 | <h1 align="center"> |
| 2 | Fossil File Formats |
| 3 | </h1> |
| 4 | |
| 5 | <p> |
| 6 | The global state of a fossil repository is determined by an unordered |
| 7 | set of artifacts. |
| 8 | An artifact might be a source code file, the text of a wiki page, |
| 9 | part of a trouble ticket, or one of several special control artifacts |
| 10 | used to show the relationships between other artifacts within the |
| 11 | project. Artifacts can be text or binary. |
| 12 | </p> |
| 13 | |
| 14 | <p> |
| 15 | Each artifact in the repository is named by its SHA1 hash. |
| 16 | No prefixes or meta information is added to a artifact before |
| @@ -167,11 +173,13 @@ | |
| 167 | <h2>2.0 Clusters</h2> |
| 168 | |
| 169 | <p> |
| 170 | A cluster is a artifact that declares the existance of other artifacts. |
| 171 | Clusters are used during repository synchronization to help |
| 172 | reduce network traffic. |
| 173 | </p> |
| 174 | |
| 175 | <p> |
| 176 | Clusters follow a syntax that is very similar to manifests. |
| 177 | A Cluster is a line-oriented text file. Newline characters |
| 178 |
| --- www/fileformat.wiki | |
| +++ www/fileformat.wiki | |
| @@ -1,16 +1,22 @@ | |
| 1 | <h1 align="center"> |
| 2 | Fossil File Formats |
| 3 | </h1> |
| 4 | |
| 5 | <p>The state of a fossil repository is kept simple so that it can |
| 6 | endure in useful form for decades or centuries. |
| 7 | A fossil repository is intended to be readable, |
| 8 | searchable, and extensible by people not yet born.</p> |
| 9 | |
| 10 | <p> |
| 11 | The global state of a fossil repository is determined by an unordered |
| 12 | set of <i>artifacts</i>. |
| 13 | An artifact might be a source code file, the text of a wiki page, |
| 14 | part of a trouble ticket, or one of several special control artifacts |
| 15 | used to show the relationships between other artifacts within the |
| 16 | project. Each artifact is normally represented on disk as a separate |
| 17 | file. Artifacts can be text or binary. |
| 18 | </p> |
| 19 | |
| 20 | <p> |
| 21 | Each artifact in the repository is named by its SHA1 hash. |
| 22 | No prefixes or meta information is added to a artifact before |
| @@ -167,11 +173,13 @@ | |
| 173 | <h2>2.0 Clusters</h2> |
| 174 | |
| 175 | <p> |
| 176 | A cluster is a artifact that declares the existance of other artifacts. |
| 177 | Clusters are used during repository synchronization to help |
| 178 | reduce network traffic. As such, clusters are an optimization and |
| 179 | may be removed from a repository without loss or damage to the |
| 180 | underlying project code. |
| 181 | </p> |
| 182 | |
| 183 | <p> |
| 184 | Clusters follow a syntax that is very similar to manifests. |
| 185 | A Cluster is a line-oriented text file. Newline characters |
| 186 |
+14
-14
| --- www/index.wiki | ||
| +++ www/index.wiki | ||
| @@ -4,11 +4,11 @@ | ||
| 4 | 4 | Fossil is a new |
| 5 | 5 | <a href="http://en.wikipedia.org/wiki/Revision_control"> |
| 6 | 6 | distributed software revision control system</a> that includes an integrated |
| 7 | 7 | <a href="http://en.wikipedia.org/wiki/Wiki">Wiki</a> and an integrated |
| 8 | 8 | <a href="http://en.wikipedia.org/wiki/Bugtracker"> |
| 9 | -bug-tracking system</a> all in a single easy-to-use stand-alone | |
| 9 | +bug-tracking system</a> all in a single, easy-to-use, stand-alone | |
| 10 | 10 | executable. |
| 11 | 11 | (NB: The bug-tracker component is not yet completely functional, but |
| 12 | 12 | we expect it to be available soon.) |
| 13 | 13 | Fossil is |
| 14 | 14 | <a href="http://www.fossil-scm.org/fossil/timeline">self-hosting</a> |
| @@ -35,29 +35,30 @@ | ||
| 35 | 35 | or all three at the same time</li> |
| 36 | 36 | <li>Integrated bug tracking and wiki, along the lines of |
| 37 | 37 | <a href="http://www.cvstrac.org/">CVSTrac</a> and |
| 38 | 38 | <a href="http://www.edgewall.com/trac/">Trac</a>.</li> |
| 39 | 39 | <li>Built-in web interface that supports deep archaeological digs through |
| 40 | -historical source code.</li> | |
| 40 | +the project history.</li> | |
| 41 | 41 | <li>All network communication via |
| 42 | 42 | <a href="http://en.wikipedia.org/wiki/HTTP">HTTP</a> |
| 43 | 43 | (so that everything works from behind restrictive firewalls).</li> |
| 44 | -<li>Everything included in a single self-contained executable - | |
| 45 | - trivial to install</li> | |
| 44 | +<li>Everything (client, server, and utilities) is included in a | |
| 45 | +single self-contained executable - trivial to install</li> | |
| 46 | 46 | <li>Server runs as <a href="http://www.w3.org/CGI/">CGI</a>, using |
| 47 | -<a href="http://en.wikipedia.org/wiki/inetd">inetd</a> or | |
| 48 | -<a href="http://www.xinetd.org/">xinetd</a> or using its own built-in, | |
| 47 | +<a href="http://en.wikipedia.org/wiki/inetd">inetd</a>/<a | |
| 48 | + href="http://www.xinetd.org/">xinetd</a>, or using its own built-in, | |
| 49 | 49 | standalone web server.</li> |
| 50 | 50 | <li>An entire project contained in single disk file (which also |
| 51 | 51 | happens to be an <a href="http://www.sqlite.org/">SQLite</a> database.)</li> |
| 52 | 52 | <li>Trivial to setup and administer</li> |
| 53 | 53 | <li>Files and versions are identified by their |
| 54 | 54 | <a href="http://en.wikipedia.org/wiki/SHA-1">SHA1</a> signature.</a> |
| 55 | 55 | Any unique prefix is sufficient to identify a file |
| 56 | 56 | or version - usually the first 4 or 5 characters suffice.</li> |
| 57 | -<li>The file format is trival and requires nothing more complex | |
| 58 | -than a text editor and the "sha1sum" command-line utility to decode.</li> | |
| 57 | +<li>The <a href="fileformat.wiki">file format</a> designed to be enduring. | |
| 58 | +It is deliberately kept simple, requiring nothing more complex | |
| 59 | +than a text editor and an SHA1 checksum generator to encode or decode.</li> | |
| 59 | 60 | <li>Automatic <a href="selfcheck.wiki">self-check</a> |
| 60 | 61 | on repository changes makes it exceedingly |
| 61 | 62 | unlikely that data will ever be lost because of a software bug.</li> |
| 62 | 63 | </ul> |
| 63 | 64 | |
| @@ -73,16 +74,15 @@ | ||
| 73 | 74 | (example: <a href="http://www.he.net/hosting.html">Hurricane Electric</a>) |
| 74 | 75 | that provides nothing more than web space and CGI capability. |
| 75 | 76 | Here is <a href="http://www.hwaci.com/cgi-bin/fossil/timeline">a demo</a>.</li> |
| 76 | 77 | <li>Fossil should provide in-depth historical and status information about the |
| 77 | 78 | project through a web interface</li> |
| 78 | -<li>The integration of <a href="http://wiki.org/wiki.cgi?WhatIsWiki">Wiki</a> | |
| 79 | -and the ability to safely support anonymous check-in are features sometimes | |
| 80 | -described as | |
| 81 | -<a href="http://www.oreillynet.com/pub/a/oreilly/tim/news/2005/09/30/what-is-web-20.html">Web 2.0</a>. | |
| 82 | -Fossil attempts to better capture "collective intelligence" and | |
| 83 | -"the wisdom of crowds" by opening up write access to the masses.</li> | |
| 79 | +<li>Fossil should provide an historical record of a project that endures | |
| 80 | +for decades or centuries and across multiple generations of hardward | |
| 81 | +and software.</li> | |
| 82 | +<li>Fossil should be easily adaptable to different workflows. Fossil | |
| 83 | +implements mechanism, not policy.</li> | |
| 84 | 84 | </ul> |
| 85 | 85 | |
| 86 | 86 | <p>User Links:</p> |
| 87 | 87 | |
| 88 | 88 | <ul> |
| 89 | 89 |
| --- www/index.wiki | |
| +++ www/index.wiki | |
| @@ -4,11 +4,11 @@ | |
| 4 | Fossil is a new |
| 5 | <a href="http://en.wikipedia.org/wiki/Revision_control"> |
| 6 | distributed software revision control system</a> that includes an integrated |
| 7 | <a href="http://en.wikipedia.org/wiki/Wiki">Wiki</a> and an integrated |
| 8 | <a href="http://en.wikipedia.org/wiki/Bugtracker"> |
| 9 | bug-tracking system</a> all in a single easy-to-use stand-alone |
| 10 | executable. |
| 11 | (NB: The bug-tracker component is not yet completely functional, but |
| 12 | we expect it to be available soon.) |
| 13 | Fossil is |
| 14 | <a href="http://www.fossil-scm.org/fossil/timeline">self-hosting</a> |
| @@ -35,29 +35,30 @@ | |
| 35 | or all three at the same time</li> |
| 36 | <li>Integrated bug tracking and wiki, along the lines of |
| 37 | <a href="http://www.cvstrac.org/">CVSTrac</a> and |
| 38 | <a href="http://www.edgewall.com/trac/">Trac</a>.</li> |
| 39 | <li>Built-in web interface that supports deep archaeological digs through |
| 40 | historical source code.</li> |
| 41 | <li>All network communication via |
| 42 | <a href="http://en.wikipedia.org/wiki/HTTP">HTTP</a> |
| 43 | (so that everything works from behind restrictive firewalls).</li> |
| 44 | <li>Everything included in a single self-contained executable - |
| 45 | trivial to install</li> |
| 46 | <li>Server runs as <a href="http://www.w3.org/CGI/">CGI</a>, using |
| 47 | <a href="http://en.wikipedia.org/wiki/inetd">inetd</a> or |
| 48 | <a href="http://www.xinetd.org/">xinetd</a> or using its own built-in, |
| 49 | standalone web server.</li> |
| 50 | <li>An entire project contained in single disk file (which also |
| 51 | happens to be an <a href="http://www.sqlite.org/">SQLite</a> database.)</li> |
| 52 | <li>Trivial to setup and administer</li> |
| 53 | <li>Files and versions are identified by their |
| 54 | <a href="http://en.wikipedia.org/wiki/SHA-1">SHA1</a> signature.</a> |
| 55 | Any unique prefix is sufficient to identify a file |
| 56 | or version - usually the first 4 or 5 characters suffice.</li> |
| 57 | <li>The file format is trival and requires nothing more complex |
| 58 | than a text editor and the "sha1sum" command-line utility to decode.</li> |
| 59 | <li>Automatic <a href="selfcheck.wiki">self-check</a> |
| 60 | on repository changes makes it exceedingly |
| 61 | unlikely that data will ever be lost because of a software bug.</li> |
| 62 | </ul> |
| 63 | |
| @@ -73,16 +74,15 @@ | |
| 73 | (example: <a href="http://www.he.net/hosting.html">Hurricane Electric</a>) |
| 74 | that provides nothing more than web space and CGI capability. |
| 75 | Here is <a href="http://www.hwaci.com/cgi-bin/fossil/timeline">a demo</a>.</li> |
| 76 | <li>Fossil should provide in-depth historical and status information about the |
| 77 | project through a web interface</li> |
| 78 | <li>The integration of <a href="http://wiki.org/wiki.cgi?WhatIsWiki">Wiki</a> |
| 79 | and the ability to safely support anonymous check-in are features sometimes |
| 80 | described as |
| 81 | <a href="http://www.oreillynet.com/pub/a/oreilly/tim/news/2005/09/30/what-is-web-20.html">Web 2.0</a>. |
| 82 | Fossil attempts to better capture "collective intelligence" and |
| 83 | "the wisdom of crowds" by opening up write access to the masses.</li> |
| 84 | </ul> |
| 85 | |
| 86 | <p>User Links:</p> |
| 87 | |
| 88 | <ul> |
| 89 |
| --- www/index.wiki | |
| +++ www/index.wiki | |
| @@ -4,11 +4,11 @@ | |
| 4 | Fossil is a new |
| 5 | <a href="http://en.wikipedia.org/wiki/Revision_control"> |
| 6 | distributed software revision control system</a> that includes an integrated |
| 7 | <a href="http://en.wikipedia.org/wiki/Wiki">Wiki</a> and an integrated |
| 8 | <a href="http://en.wikipedia.org/wiki/Bugtracker"> |
| 9 | bug-tracking system</a> all in a single, easy-to-use, stand-alone |
| 10 | executable. |
| 11 | (NB: The bug-tracker component is not yet completely functional, but |
| 12 | we expect it to be available soon.) |
| 13 | Fossil is |
| 14 | <a href="http://www.fossil-scm.org/fossil/timeline">self-hosting</a> |
| @@ -35,29 +35,30 @@ | |
| 35 | or all three at the same time</li> |
| 36 | <li>Integrated bug tracking and wiki, along the lines of |
| 37 | <a href="http://www.cvstrac.org/">CVSTrac</a> and |
| 38 | <a href="http://www.edgewall.com/trac/">Trac</a>.</li> |
| 39 | <li>Built-in web interface that supports deep archaeological digs through |
| 40 | the project history.</li> |
| 41 | <li>All network communication via |
| 42 | <a href="http://en.wikipedia.org/wiki/HTTP">HTTP</a> |
| 43 | (so that everything works from behind restrictive firewalls).</li> |
| 44 | <li>Everything (client, server, and utilities) is included in a |
| 45 | single self-contained executable - trivial to install</li> |
| 46 | <li>Server runs as <a href="http://www.w3.org/CGI/">CGI</a>, using |
| 47 | <a href="http://en.wikipedia.org/wiki/inetd">inetd</a>/<a |
| 48 | href="http://www.xinetd.org/">xinetd</a>, or using its own built-in, |
| 49 | standalone web server.</li> |
| 50 | <li>An entire project contained in single disk file (which also |
| 51 | happens to be an <a href="http://www.sqlite.org/">SQLite</a> database.)</li> |
| 52 | <li>Trivial to setup and administer</li> |
| 53 | <li>Files and versions are identified by their |
| 54 | <a href="http://en.wikipedia.org/wiki/SHA-1">SHA1</a> signature.</a> |
| 55 | Any unique prefix is sufficient to identify a file |
| 56 | or version - usually the first 4 or 5 characters suffice.</li> |
| 57 | <li>The <a href="fileformat.wiki">file format</a> designed to be enduring. |
| 58 | It is deliberately kept simple, requiring nothing more complex |
| 59 | than a text editor and an SHA1 checksum generator to encode or decode.</li> |
| 60 | <li>Automatic <a href="selfcheck.wiki">self-check</a> |
| 61 | on repository changes makes it exceedingly |
| 62 | unlikely that data will ever be lost because of a software bug.</li> |
| 63 | </ul> |
| 64 | |
| @@ -73,16 +74,15 @@ | |
| 74 | (example: <a href="http://www.he.net/hosting.html">Hurricane Electric</a>) |
| 75 | that provides nothing more than web space and CGI capability. |
| 76 | Here is <a href="http://www.hwaci.com/cgi-bin/fossil/timeline">a demo</a>.</li> |
| 77 | <li>Fossil should provide in-depth historical and status information about the |
| 78 | project through a web interface</li> |
| 79 | <li>Fossil should provide an historical record of a project that endures |
| 80 | for decades or centuries and across multiple generations of hardward |
| 81 | and software.</li> |
| 82 | <li>Fossil should be easily adaptable to different workflows. Fossil |
| 83 | implements mechanism, not policy.</li> |
| 84 | </ul> |
| 85 | |
| 86 | <p>User Links:</p> |
| 87 | |
| 88 | <ul> |
| 89 |
+38
-21
| --- www/sync.wiki | ||
| +++ www/sync.wiki | ||
| @@ -5,11 +5,28 @@ | ||
| 5 | 5 | command is run on the client repository. A URL for the server repository |
| 6 | 6 | is specified as part of the command. This document describes what happens |
| 7 | 7 | behind the scenes in order to synchronize the information on the two |
| 8 | 8 | repositories.</p> |
| 9 | 9 | |
| 10 | -<h2>1.0 Transport</h2> | |
| 10 | +<h2>1.0 Overview</h2> | |
| 11 | + | |
| 12 | +<p>The global state of a fossil repository consists of an unordered | |
| 13 | +collection of artifacts. Each artifact is identified by its SHA1 hash. | |
| 14 | +Synchronization is simply the process of sharing artifacts between | |
| 15 | +servers so that all servers have copies of all artifacts. Because | |
| 16 | +artifacts are unordered, the order in which artifacts are received | |
| 17 | +at a server is inconsequential. It is assumed that the SHA1 hashes | |
| 18 | +of artifacts are unique - that every artifact has a different SHA1 hash. | |
| 19 | +To first approximation, synchronization proceeds by sharing lists | |
| 20 | +SHA1 hashes of available artifacts, then sharing those artifacts that | |
| 21 | +are not found on one side or the other of the connection. In practice, | |
| 22 | +a repository might contain millions of artifacts. The list of | |
| 23 | +SHA1 hashes for this many artifacts can be large. So optimizations are | |
| 24 | +employed that usually reduce the number of SHA1 hashes that need to be | |
| 25 | +shared to a few hundred.</p> | |
| 26 | + | |
| 27 | +<h2>2.0 Transport</h2> | |
| 11 | 28 | |
| 12 | 29 | <p>All communication between client and server is via HTTP requests. |
| 13 | 30 | The server is listening for incoming HTTP requests. The client |
| 14 | 31 | issues one or more HTTP requests and receives replies for each |
| 15 | 32 | request.</p> |
| @@ -25,11 +42,11 @@ | ||
| 25 | 42 | <p>A single push, pull, or sync might involve multiple HTTP requests. |
| 26 | 43 | The client maintains state between all requests. But on the server |
| 27 | 44 | side, each request is independent. The server does not preserve |
| 28 | 45 | any information about the client from one request to the next.</p> |
| 29 | 46 | |
| 30 | -<h3>1.1 Server Identification</h3> | |
| 47 | +<h3>2.1 Server Identification</h3> | |
| 31 | 48 | |
| 32 | 49 | <p>The server is identified by a URL argument that accompanies the |
| 33 | 50 | push, pull, or sync command on the client. (As a convenience to |
| 34 | 51 | users, the URL can be omitted on the client command and the same URL |
| 35 | 52 | from the most recent push, pull, or sync will be reused. This saves |
| @@ -49,11 +66,11 @@ | ||
| 49 | 66 | |
| 50 | 67 | <blockquote> |
| 51 | 68 | http://fossil-scm.hwaci.com/fossil/xfer |
| 52 | 69 | </blockquote> |
| 53 | 70 | |
| 54 | -<h3>1.2 HTTP Request Format</h3> | |
| 71 | +<h3>2.2 HTTP Request Format</h3> | |
| 55 | 72 | |
| 56 | 73 | <p>The client always sends a POST request to the server. The |
| 57 | 74 | general format of the POST request is as follows:</p> |
| 58 | 75 | |
| 59 | 76 | <blockquote><pre> |
| @@ -87,17 +104,17 @@ | ||
| 87 | 104 | </pre></blockquote> |
| 88 | 105 | |
| 89 | 106 | <p>The content type of the reply is always the same as the content type |
| 90 | 107 | of the request.</p> |
| 91 | 108 | |
| 92 | -<h2>2.0 Fossil Synchronization Content</h2> | |
| 109 | +<h2>3.0 Fossil Synchronization Content</h2> | |
| 93 | 110 | |
| 94 | 111 | <p>A synchronization request between a client and server consists of |
| 95 | 112 | one or more HTTP requests as described in the previous section. This |
| 96 | 113 | section details the "x-fossil" content type.</p> |
| 97 | 114 | |
| 98 | -<h3>2.1 Line-oriented Format</h3> | |
| 115 | +<h3>3.1 Line-oriented Format</h3> | |
| 99 | 116 | |
| 100 | 117 | <p>The x-fossil content type consists of zero or more "cards". Cards |
| 101 | 118 | are separate by the newline character ("\n"). Leading and trailing |
| 102 | 119 | whitespace on a card is ignored. Blank cards are ignored.</p> |
| 103 | 120 | |
| @@ -105,11 +122,11 @@ | ||
| 105 | 122 | The first token on each card is the operator. Subsequent tokens |
| 106 | 123 | are arguments. The set of operators understood by servers is slightly |
| 107 | 124 | different from the operators understood by clients, though the two |
| 108 | 125 | are very similar.</p> |
| 109 | 126 | |
| 110 | -<h3>2.2 Login Cards</h3> | |
| 127 | +<h3>3.2 Login Cards</h3> | |
| 111 | 128 | |
| 112 | 129 | <p>Every message from client to server begins with one or more login |
| 113 | 130 | cards. Each login card has the following format:</p> |
| 114 | 131 | |
| 115 | 132 | <blockquote> |
| @@ -131,11 +148,11 @@ | ||
| 131 | 148 | |
| 132 | 149 | <p>Privileges are cumulative. There can be multiple successful |
| 133 | 150 | login cards. The session privileges are the bit-wise OR of the |
| 134 | 151 | privileges of each individual login.</p> |
| 135 | 152 | |
| 136 | -<h3>2.3 File Cards</h3> | |
| 153 | +<h3>3.3 File Cards</h3> | |
| 137 | 154 | |
| 138 | 155 | <p>Repository content records or files are transferred using |
| 139 | 156 | a "file" card. File cards come in two different formats depending |
| 140 | 157 | on whether the file is sent directly or as a delta from some |
| 141 | 158 | other file.</p> |
| @@ -165,11 +182,11 @@ | ||
| 165 | 182 | <p>File cards are sent in both directions: client to server and |
| 166 | 183 | server to client. A delta might be sent before the source of |
| 167 | 184 | the delta, so both client and server should remember deltas |
| 168 | 185 | and be able to apply them when their source arrives.</p> |
| 169 | 186 | |
| 170 | -<h3>2.4 Push and Pull Cards</h3> | |
| 187 | +<h3>3.4 Push and Pull Cards</h3> | |
| 171 | 188 | |
| 172 | 189 | <p>Among of the first cards in a client-to-server message are |
| 173 | 190 | the push and pull cards. The push card tell the server that |
| 174 | 191 | the client is pushing content. The pull card tell the server |
| 175 | 192 | that the client wants to pull content. In the event of a sync, |
| @@ -190,11 +207,11 @@ | ||
| 190 | 207 | |
| 191 | 208 | <p>The server will also send a push card back to the client |
| 192 | 209 | during a clone. This is how the client determines what project |
| 193 | 210 | code to put in the new repository it is constructing.</p> |
| 194 | 211 | |
| 195 | -<h3>2.5 Clone Cards</h3> | |
| 212 | +<h3>3.5 Clone Cards</h3> | |
| 196 | 213 | |
| 197 | 214 | <p>A clone card works like a pull card in that it is sent from |
| 198 | 215 | client to server in order to tell the server that the client |
| 199 | 216 | wants to pull content. But unlike the pull card, the clone |
| 200 | 217 | card has no arguments.</p> |
| @@ -205,11 +222,11 @@ | ||
| 205 | 222 | |
| 206 | 223 | <p>In response to a clone message, the server also sends the client |
| 207 | 224 | a push message so that the client can discover the projectcode for |
| 208 | 225 | this project.</p> |
| 209 | 226 | |
| 210 | -<h3>2.6 Igot Cards</h3> | |
| 227 | +<h3>3.6 Igot Cards</h3> | |
| 211 | 228 | |
| 212 | 229 | <p>An igot card can be sent from either client to server or from |
| 213 | 230 | server to client in order to indicate that the sender holds a copy |
| 214 | 231 | of a particular file. The format is:</p> |
| 215 | 232 | |
| @@ -221,11 +238,11 @@ | ||
| 221 | 238 | the sender possesses. |
| 222 | 239 | The receiver of an igot card will typically check to see if |
| 223 | 240 | it also holds the same file and if not it will request the file |
| 224 | 241 | using a gimme card in either the reply or in the next message.</p> |
| 225 | 242 | |
| 226 | -<h3>2.7 Gimme Cards</h3> | |
| 243 | +<h3>3.7 Gimme Cards</h3> | |
| 227 | 244 | |
| 228 | 245 | <p>A gimme card is sent from either client to server or from server |
| 229 | 246 | to client. The gimme card asks the receiver to send a particular |
| 230 | 247 | file back to the sender. The format of a gimme card is this:</p> |
| 231 | 248 | |
| @@ -236,11 +253,11 @@ | ||
| 236 | 253 | <p>The argument to the gimme card is the UUID of the file that |
| 237 | 254 | the sender wants. The receiver will typically respond to a |
| 238 | 255 | gimme card by sending a file card in its reply or in the next |
| 239 | 256 | message.</p> |
| 240 | 257 | |
| 241 | -<h3>2.8 Cookie Cards</h3> | |
| 258 | +<h3>3.8 Cookie Cards</h3> | |
| 242 | 259 | |
| 243 | 260 | <p>A cookie card can be used by a server to record a small amount |
| 244 | 261 | of state information on a client. The server sends a cookie to the |
| 245 | 262 | client. The client sends the same cookie back to the server on |
| 246 | 263 | its next request. The cookie card has a single argument which |
| @@ -256,11 +273,11 @@ | ||
| 256 | 273 | cookie and the server must structure the cookie payload in such |
| 257 | 274 | a way that it can tell if the cookie it sees is its own cookie or |
| 258 | 275 | a cookie from another server. (Typically the server will embed |
| 259 | 276 | its servercode as part of the cookie.)</p> |
| 260 | 277 | |
| 261 | -<h3>2.9 Error Cards</h3> | |
| 278 | +<h3>3.9 Error Cards</h3> | |
| 262 | 279 | |
| 263 | 280 | <p>If the server discovers anything wrong with a request, it generates |
| 264 | 281 | an error card in its reply. When the client sees the error card, |
| 265 | 282 | it displays an error message to the user and aborts the sync |
| 266 | 283 | operation. An error card looks like this:</p> |
| @@ -276,16 +293,16 @@ | ||
| 276 | 293 | (ASCII 0x5C) is represented as two backslashes "\\". Apart from |
| 277 | 294 | space and newline, no other whitespace characters nor any |
| 278 | 295 | unprintable characters are allowed in |
| 279 | 296 | the error message.</p> |
| 280 | 297 | |
| 281 | -<h3>2.10 Unknown Cards</h3> | |
| 298 | +<h3>3.10 Unknown Cards</h3> | |
| 282 | 299 | |
| 283 | 300 | <p>If either the client or the server sees a card that is not |
| 284 | 301 | described above, then it generates an error and aborts.</p> |
| 285 | 302 | |
| 286 | -<h2>3.0 Phantoms And Clusters</h2> | |
| 303 | +<h2>4.0 Phantoms And Clusters</h2> | |
| 287 | 304 | |
| 288 | 305 | <p>When a repository knows that a file exists and knows the UUID of |
| 289 | 306 | that file, but it does not know the file content, then it stores that |
| 290 | 307 | file as a "phantom". A repository will typically create a phantom when |
| 291 | 308 | it receives an igot card for a file that it does not hold or when it |
| @@ -316,11 +333,11 @@ | ||
| 316 | 333 | exactly is not a cluster. There must be no extra whitespace in |
| 317 | 334 | the file. There must be one or more M cards. There must be a |
| 318 | 335 | single Z card with a correct MD5 checksum. And all cards must |
| 319 | 336 | be in strict lexicographical order.</p> |
| 320 | 337 | |
| 321 | -<h3>3.1 The Unclustered Table</h3> | |
| 338 | +<h3>4.1 The Unclustered Table</h3> | |
| 322 | 339 | |
| 323 | 340 | <p>Every repository maintains a table named "<b>unclustered</b>" |
| 324 | 341 | which records the identity of every file and phantom it holds that is not |
| 325 | 342 | mentioned in a cluster. The entries in the unclustered table can |
| 326 | 343 | be thought of as leaves on a tree of files. Some of the unclustered |
| @@ -327,13 +344,13 @@ | ||
| 327 | 344 | files will be clusters. Those clusters may contain other clusters, |
| 328 | 345 | which might contain still more clusters, and so forth. Beginning |
| 329 | 346 | with the files in the unclustered table, one can follow the chain |
| 330 | 347 | of clusters to find every file in the repository.</p> |
| 331 | 348 | |
| 332 | -<h2>4.0 Synchronization Strategies</h2> | |
| 349 | +<h2>5.0 Synchronization Strategies</h2> | |
| 333 | 350 | |
| 334 | -<h3>4.1 Pull</h3> | |
| 351 | +<h3>5.1 Pull</h3> | |
| 335 | 352 | |
| 336 | 353 | <p>A typical pull operation proceeds as shown below. Details |
| 337 | 354 | of the actual implementation may very slightly but the gist of |
| 338 | 355 | a pull is captured in the following steps:</p> |
| 339 | 356 | |
| @@ -381,11 +398,11 @@ | ||
| 381 | 398 | protocol will continue to work even if there are multiple servers |
| 382 | 399 | or if servers and clients sometimes change roles. The only negative |
| 383 | 400 | effects of these unusual arrangements is that more than the minimum |
| 384 | 401 | number of clusters might be generated.</p> |
| 385 | 402 | |
| 386 | -<h3>4.2 Push</h3> | |
| 403 | +<h3>5.2 Push</h3> | |
| 387 | 404 | |
| 388 | 405 | <p>A typical push operation proceeds roughly as shown below. As |
| 389 | 406 | with a pull, the actual implementation may vary slightly.</p> |
| 390 | 407 | |
| 391 | 408 | <ol> |
| @@ -415,19 +432,19 @@ | ||
| 415 | 432 | server knows all files that exist on the client. Also, as with |
| 416 | 433 | pull, the client attempts to keep the size of the request from |
| 417 | 434 | growing too large by suppressing file cards once the |
| 418 | 435 | size of the request reaches 1MB.</p> |
| 419 | 436 | |
| 420 | -<h3>4.3 Sync</h3> | |
| 437 | +<h3>5.3 Sync</h3> | |
| 421 | 438 | |
| 422 | 439 | <p>A sync is just a pull and a push that happen at the same time. |
| 423 | 440 | The first three steps of a pull are combined with the first five steps |
| 424 | 441 | of a push. Steps (4) through (7) of a pull are combined with steps |
| 425 | 442 | (5) through (8) of a push. And steps (8) through (10) of a pull |
| 426 | 443 | are combined with step (9) of a push.</p> |
| 427 | 444 | |
| 428 | -<h2>5.0 Summary</h2> | |
| 445 | +<h2>6.0 Summary</h2> | |
| 429 | 446 | |
| 430 | 447 | <p>Here are the key points of the synchronization protocol:</p> |
| 431 | 448 | |
| 432 | 449 | <ol> |
| 433 | 450 | <li>The client sends one or more PUSH HTTP requests to the server. |
| 434 | 451 |
| --- www/sync.wiki | |
| +++ www/sync.wiki | |
| @@ -5,11 +5,28 @@ | |
| 5 | command is run on the client repository. A URL for the server repository |
| 6 | is specified as part of the command. This document describes what happens |
| 7 | behind the scenes in order to synchronize the information on the two |
| 8 | repositories.</p> |
| 9 | |
| 10 | <h2>1.0 Transport</h2> |
| 11 | |
| 12 | <p>All communication between client and server is via HTTP requests. |
| 13 | The server is listening for incoming HTTP requests. The client |
| 14 | issues one or more HTTP requests and receives replies for each |
| 15 | request.</p> |
| @@ -25,11 +42,11 @@ | |
| 25 | <p>A single push, pull, or sync might involve multiple HTTP requests. |
| 26 | The client maintains state between all requests. But on the server |
| 27 | side, each request is independent. The server does not preserve |
| 28 | any information about the client from one request to the next.</p> |
| 29 | |
| 30 | <h3>1.1 Server Identification</h3> |
| 31 | |
| 32 | <p>The server is identified by a URL argument that accompanies the |
| 33 | push, pull, or sync command on the client. (As a convenience to |
| 34 | users, the URL can be omitted on the client command and the same URL |
| 35 | from the most recent push, pull, or sync will be reused. This saves |
| @@ -49,11 +66,11 @@ | |
| 49 | |
| 50 | <blockquote> |
| 51 | http://fossil-scm.hwaci.com/fossil/xfer |
| 52 | </blockquote> |
| 53 | |
| 54 | <h3>1.2 HTTP Request Format</h3> |
| 55 | |
| 56 | <p>The client always sends a POST request to the server. The |
| 57 | general format of the POST request is as follows:</p> |
| 58 | |
| 59 | <blockquote><pre> |
| @@ -87,17 +104,17 @@ | |
| 87 | </pre></blockquote> |
| 88 | |
| 89 | <p>The content type of the reply is always the same as the content type |
| 90 | of the request.</p> |
| 91 | |
| 92 | <h2>2.0 Fossil Synchronization Content</h2> |
| 93 | |
| 94 | <p>A synchronization request between a client and server consists of |
| 95 | one or more HTTP requests as described in the previous section. This |
| 96 | section details the "x-fossil" content type.</p> |
| 97 | |
| 98 | <h3>2.1 Line-oriented Format</h3> |
| 99 | |
| 100 | <p>The x-fossil content type consists of zero or more "cards". Cards |
| 101 | are separate by the newline character ("\n"). Leading and trailing |
| 102 | whitespace on a card is ignored. Blank cards are ignored.</p> |
| 103 | |
| @@ -105,11 +122,11 @@ | |
| 105 | The first token on each card is the operator. Subsequent tokens |
| 106 | are arguments. The set of operators understood by servers is slightly |
| 107 | different from the operators understood by clients, though the two |
| 108 | are very similar.</p> |
| 109 | |
| 110 | <h3>2.2 Login Cards</h3> |
| 111 | |
| 112 | <p>Every message from client to server begins with one or more login |
| 113 | cards. Each login card has the following format:</p> |
| 114 | |
| 115 | <blockquote> |
| @@ -131,11 +148,11 @@ | |
| 131 | |
| 132 | <p>Privileges are cumulative. There can be multiple successful |
| 133 | login cards. The session privileges are the bit-wise OR of the |
| 134 | privileges of each individual login.</p> |
| 135 | |
| 136 | <h3>2.3 File Cards</h3> |
| 137 | |
| 138 | <p>Repository content records or files are transferred using |
| 139 | a "file" card. File cards come in two different formats depending |
| 140 | on whether the file is sent directly or as a delta from some |
| 141 | other file.</p> |
| @@ -165,11 +182,11 @@ | |
| 165 | <p>File cards are sent in both directions: client to server and |
| 166 | server to client. A delta might be sent before the source of |
| 167 | the delta, so both client and server should remember deltas |
| 168 | and be able to apply them when their source arrives.</p> |
| 169 | |
| 170 | <h3>2.4 Push and Pull Cards</h3> |
| 171 | |
| 172 | <p>Among of the first cards in a client-to-server message are |
| 173 | the push and pull cards. The push card tell the server that |
| 174 | the client is pushing content. The pull card tell the server |
| 175 | that the client wants to pull content. In the event of a sync, |
| @@ -190,11 +207,11 @@ | |
| 190 | |
| 191 | <p>The server will also send a push card back to the client |
| 192 | during a clone. This is how the client determines what project |
| 193 | code to put in the new repository it is constructing.</p> |
| 194 | |
| 195 | <h3>2.5 Clone Cards</h3> |
| 196 | |
| 197 | <p>A clone card works like a pull card in that it is sent from |
| 198 | client to server in order to tell the server that the client |
| 199 | wants to pull content. But unlike the pull card, the clone |
| 200 | card has no arguments.</p> |
| @@ -205,11 +222,11 @@ | |
| 205 | |
| 206 | <p>In response to a clone message, the server also sends the client |
| 207 | a push message so that the client can discover the projectcode for |
| 208 | this project.</p> |
| 209 | |
| 210 | <h3>2.6 Igot Cards</h3> |
| 211 | |
| 212 | <p>An igot card can be sent from either client to server or from |
| 213 | server to client in order to indicate that the sender holds a copy |
| 214 | of a particular file. The format is:</p> |
| 215 | |
| @@ -221,11 +238,11 @@ | |
| 221 | the sender possesses. |
| 222 | The receiver of an igot card will typically check to see if |
| 223 | it also holds the same file and if not it will request the file |
| 224 | using a gimme card in either the reply or in the next message.</p> |
| 225 | |
| 226 | <h3>2.7 Gimme Cards</h3> |
| 227 | |
| 228 | <p>A gimme card is sent from either client to server or from server |
| 229 | to client. The gimme card asks the receiver to send a particular |
| 230 | file back to the sender. The format of a gimme card is this:</p> |
| 231 | |
| @@ -236,11 +253,11 @@ | |
| 236 | <p>The argument to the gimme card is the UUID of the file that |
| 237 | the sender wants. The receiver will typically respond to a |
| 238 | gimme card by sending a file card in its reply or in the next |
| 239 | message.</p> |
| 240 | |
| 241 | <h3>2.8 Cookie Cards</h3> |
| 242 | |
| 243 | <p>A cookie card can be used by a server to record a small amount |
| 244 | of state information on a client. The server sends a cookie to the |
| 245 | client. The client sends the same cookie back to the server on |
| 246 | its next request. The cookie card has a single argument which |
| @@ -256,11 +273,11 @@ | |
| 256 | cookie and the server must structure the cookie payload in such |
| 257 | a way that it can tell if the cookie it sees is its own cookie or |
| 258 | a cookie from another server. (Typically the server will embed |
| 259 | its servercode as part of the cookie.)</p> |
| 260 | |
| 261 | <h3>2.9 Error Cards</h3> |
| 262 | |
| 263 | <p>If the server discovers anything wrong with a request, it generates |
| 264 | an error card in its reply. When the client sees the error card, |
| 265 | it displays an error message to the user and aborts the sync |
| 266 | operation. An error card looks like this:</p> |
| @@ -276,16 +293,16 @@ | |
| 276 | (ASCII 0x5C) is represented as two backslashes "\\". Apart from |
| 277 | space and newline, no other whitespace characters nor any |
| 278 | unprintable characters are allowed in |
| 279 | the error message.</p> |
| 280 | |
| 281 | <h3>2.10 Unknown Cards</h3> |
| 282 | |
| 283 | <p>If either the client or the server sees a card that is not |
| 284 | described above, then it generates an error and aborts.</p> |
| 285 | |
| 286 | <h2>3.0 Phantoms And Clusters</h2> |
| 287 | |
| 288 | <p>When a repository knows that a file exists and knows the UUID of |
| 289 | that file, but it does not know the file content, then it stores that |
| 290 | file as a "phantom". A repository will typically create a phantom when |
| 291 | it receives an igot card for a file that it does not hold or when it |
| @@ -316,11 +333,11 @@ | |
| 316 | exactly is not a cluster. There must be no extra whitespace in |
| 317 | the file. There must be one or more M cards. There must be a |
| 318 | single Z card with a correct MD5 checksum. And all cards must |
| 319 | be in strict lexicographical order.</p> |
| 320 | |
| 321 | <h3>3.1 The Unclustered Table</h3> |
| 322 | |
| 323 | <p>Every repository maintains a table named "<b>unclustered</b>" |
| 324 | which records the identity of every file and phantom it holds that is not |
| 325 | mentioned in a cluster. The entries in the unclustered table can |
| 326 | be thought of as leaves on a tree of files. Some of the unclustered |
| @@ -327,13 +344,13 @@ | |
| 327 | files will be clusters. Those clusters may contain other clusters, |
| 328 | which might contain still more clusters, and so forth. Beginning |
| 329 | with the files in the unclustered table, one can follow the chain |
| 330 | of clusters to find every file in the repository.</p> |
| 331 | |
| 332 | <h2>4.0 Synchronization Strategies</h2> |
| 333 | |
| 334 | <h3>4.1 Pull</h3> |
| 335 | |
| 336 | <p>A typical pull operation proceeds as shown below. Details |
| 337 | of the actual implementation may very slightly but the gist of |
| 338 | a pull is captured in the following steps:</p> |
| 339 | |
| @@ -381,11 +398,11 @@ | |
| 381 | protocol will continue to work even if there are multiple servers |
| 382 | or if servers and clients sometimes change roles. The only negative |
| 383 | effects of these unusual arrangements is that more than the minimum |
| 384 | number of clusters might be generated.</p> |
| 385 | |
| 386 | <h3>4.2 Push</h3> |
| 387 | |
| 388 | <p>A typical push operation proceeds roughly as shown below. As |
| 389 | with a pull, the actual implementation may vary slightly.</p> |
| 390 | |
| 391 | <ol> |
| @@ -415,19 +432,19 @@ | |
| 415 | server knows all files that exist on the client. Also, as with |
| 416 | pull, the client attempts to keep the size of the request from |
| 417 | growing too large by suppressing file cards once the |
| 418 | size of the request reaches 1MB.</p> |
| 419 | |
| 420 | <h3>4.3 Sync</h3> |
| 421 | |
| 422 | <p>A sync is just a pull and a push that happen at the same time. |
| 423 | The first three steps of a pull are combined with the first five steps |
| 424 | of a push. Steps (4) through (7) of a pull are combined with steps |
| 425 | (5) through (8) of a push. And steps (8) through (10) of a pull |
| 426 | are combined with step (9) of a push.</p> |
| 427 | |
| 428 | <h2>5.0 Summary</h2> |
| 429 | |
| 430 | <p>Here are the key points of the synchronization protocol:</p> |
| 431 | |
| 432 | <ol> |
| 433 | <li>The client sends one or more PUSH HTTP requests to the server. |
| 434 |
| --- www/sync.wiki | |
| +++ www/sync.wiki | |
| @@ -5,11 +5,28 @@ | |
| 5 | command is run on the client repository. A URL for the server repository |
| 6 | is specified as part of the command. This document describes what happens |
| 7 | behind the scenes in order to synchronize the information on the two |
| 8 | repositories.</p> |
| 9 | |
| 10 | <h2>1.0 Overview</h2> |
| 11 | |
| 12 | <p>The global state of a fossil repository consists of an unordered |
| 13 | collection of artifacts. Each artifact is identified by its SHA1 hash. |
| 14 | Synchronization is simply the process of sharing artifacts between |
| 15 | servers so that all servers have copies of all artifacts. Because |
| 16 | artifacts are unordered, the order in which artifacts are received |
| 17 | at a server is inconsequential. It is assumed that the SHA1 hashes |
| 18 | of artifacts are unique - that every artifact has a different SHA1 hash. |
| 19 | To first approximation, synchronization proceeds by sharing lists |
| 20 | SHA1 hashes of available artifacts, then sharing those artifacts that |
| 21 | are not found on one side or the other of the connection. In practice, |
| 22 | a repository might contain millions of artifacts. The list of |
| 23 | SHA1 hashes for this many artifacts can be large. So optimizations are |
| 24 | employed that usually reduce the number of SHA1 hashes that need to be |
| 25 | shared to a few hundred.</p> |
| 26 | |
| 27 | <h2>2.0 Transport</h2> |
| 28 | |
| 29 | <p>All communication between client and server is via HTTP requests. |
| 30 | The server is listening for incoming HTTP requests. The client |
| 31 | issues one or more HTTP requests and receives replies for each |
| 32 | request.</p> |
| @@ -25,11 +42,11 @@ | |
| 42 | <p>A single push, pull, or sync might involve multiple HTTP requests. |
| 43 | The client maintains state between all requests. But on the server |
| 44 | side, each request is independent. The server does not preserve |
| 45 | any information about the client from one request to the next.</p> |
| 46 | |
| 47 | <h3>2.1 Server Identification</h3> |
| 48 | |
| 49 | <p>The server is identified by a URL argument that accompanies the |
| 50 | push, pull, or sync command on the client. (As a convenience to |
| 51 | users, the URL can be omitted on the client command and the same URL |
| 52 | from the most recent push, pull, or sync will be reused. This saves |
| @@ -49,11 +66,11 @@ | |
| 66 | |
| 67 | <blockquote> |
| 68 | http://fossil-scm.hwaci.com/fossil/xfer |
| 69 | </blockquote> |
| 70 | |
| 71 | <h3>2.2 HTTP Request Format</h3> |
| 72 | |
| 73 | <p>The client always sends a POST request to the server. The |
| 74 | general format of the POST request is as follows:</p> |
| 75 | |
| 76 | <blockquote><pre> |
| @@ -87,17 +104,17 @@ | |
| 104 | </pre></blockquote> |
| 105 | |
| 106 | <p>The content type of the reply is always the same as the content type |
| 107 | of the request.</p> |
| 108 | |
| 109 | <h2>3.0 Fossil Synchronization Content</h2> |
| 110 | |
| 111 | <p>A synchronization request between a client and server consists of |
| 112 | one or more HTTP requests as described in the previous section. This |
| 113 | section details the "x-fossil" content type.</p> |
| 114 | |
| 115 | <h3>3.1 Line-oriented Format</h3> |
| 116 | |
| 117 | <p>The x-fossil content type consists of zero or more "cards". Cards |
| 118 | are separate by the newline character ("\n"). Leading and trailing |
| 119 | whitespace on a card is ignored. Blank cards are ignored.</p> |
| 120 | |
| @@ -105,11 +122,11 @@ | |
| 122 | The first token on each card is the operator. Subsequent tokens |
| 123 | are arguments. The set of operators understood by servers is slightly |
| 124 | different from the operators understood by clients, though the two |
| 125 | are very similar.</p> |
| 126 | |
| 127 | <h3>3.2 Login Cards</h3> |
| 128 | |
| 129 | <p>Every message from client to server begins with one or more login |
| 130 | cards. Each login card has the following format:</p> |
| 131 | |
| 132 | <blockquote> |
| @@ -131,11 +148,11 @@ | |
| 148 | |
| 149 | <p>Privileges are cumulative. There can be multiple successful |
| 150 | login cards. The session privileges are the bit-wise OR of the |
| 151 | privileges of each individual login.</p> |
| 152 | |
| 153 | <h3>3.3 File Cards</h3> |
| 154 | |
| 155 | <p>Repository content records or files are transferred using |
| 156 | a "file" card. File cards come in two different formats depending |
| 157 | on whether the file is sent directly or as a delta from some |
| 158 | other file.</p> |
| @@ -165,11 +182,11 @@ | |
| 182 | <p>File cards are sent in both directions: client to server and |
| 183 | server to client. A delta might be sent before the source of |
| 184 | the delta, so both client and server should remember deltas |
| 185 | and be able to apply them when their source arrives.</p> |
| 186 | |
| 187 | <h3>3.4 Push and Pull Cards</h3> |
| 188 | |
| 189 | <p>Among of the first cards in a client-to-server message are |
| 190 | the push and pull cards. The push card tell the server that |
| 191 | the client is pushing content. The pull card tell the server |
| 192 | that the client wants to pull content. In the event of a sync, |
| @@ -190,11 +207,11 @@ | |
| 207 | |
| 208 | <p>The server will also send a push card back to the client |
| 209 | during a clone. This is how the client determines what project |
| 210 | code to put in the new repository it is constructing.</p> |
| 211 | |
| 212 | <h3>3.5 Clone Cards</h3> |
| 213 | |
| 214 | <p>A clone card works like a pull card in that it is sent from |
| 215 | client to server in order to tell the server that the client |
| 216 | wants to pull content. But unlike the pull card, the clone |
| 217 | card has no arguments.</p> |
| @@ -205,11 +222,11 @@ | |
| 222 | |
| 223 | <p>In response to a clone message, the server also sends the client |
| 224 | a push message so that the client can discover the projectcode for |
| 225 | this project.</p> |
| 226 | |
| 227 | <h3>3.6 Igot Cards</h3> |
| 228 | |
| 229 | <p>An igot card can be sent from either client to server or from |
| 230 | server to client in order to indicate that the sender holds a copy |
| 231 | of a particular file. The format is:</p> |
| 232 | |
| @@ -221,11 +238,11 @@ | |
| 238 | the sender possesses. |
| 239 | The receiver of an igot card will typically check to see if |
| 240 | it also holds the same file and if not it will request the file |
| 241 | using a gimme card in either the reply or in the next message.</p> |
| 242 | |
| 243 | <h3>3.7 Gimme Cards</h3> |
| 244 | |
| 245 | <p>A gimme card is sent from either client to server or from server |
| 246 | to client. The gimme card asks the receiver to send a particular |
| 247 | file back to the sender. The format of a gimme card is this:</p> |
| 248 | |
| @@ -236,11 +253,11 @@ | |
| 253 | <p>The argument to the gimme card is the UUID of the file that |
| 254 | the sender wants. The receiver will typically respond to a |
| 255 | gimme card by sending a file card in its reply or in the next |
| 256 | message.</p> |
| 257 | |
| 258 | <h3>3.8 Cookie Cards</h3> |
| 259 | |
| 260 | <p>A cookie card can be used by a server to record a small amount |
| 261 | of state information on a client. The server sends a cookie to the |
| 262 | client. The client sends the same cookie back to the server on |
| 263 | its next request. The cookie card has a single argument which |
| @@ -256,11 +273,11 @@ | |
| 273 | cookie and the server must structure the cookie payload in such |
| 274 | a way that it can tell if the cookie it sees is its own cookie or |
| 275 | a cookie from another server. (Typically the server will embed |
| 276 | its servercode as part of the cookie.)</p> |
| 277 | |
| 278 | <h3>3.9 Error Cards</h3> |
| 279 | |
| 280 | <p>If the server discovers anything wrong with a request, it generates |
| 281 | an error card in its reply. When the client sees the error card, |
| 282 | it displays an error message to the user and aborts the sync |
| 283 | operation. An error card looks like this:</p> |
| @@ -276,16 +293,16 @@ | |
| 293 | (ASCII 0x5C) is represented as two backslashes "\\". Apart from |
| 294 | space and newline, no other whitespace characters nor any |
| 295 | unprintable characters are allowed in |
| 296 | the error message.</p> |
| 297 | |
| 298 | <h3>3.10 Unknown Cards</h3> |
| 299 | |
| 300 | <p>If either the client or the server sees a card that is not |
| 301 | described above, then it generates an error and aborts.</p> |
| 302 | |
| 303 | <h2>4.0 Phantoms And Clusters</h2> |
| 304 | |
| 305 | <p>When a repository knows that a file exists and knows the UUID of |
| 306 | that file, but it does not know the file content, then it stores that |
| 307 | file as a "phantom". A repository will typically create a phantom when |
| 308 | it receives an igot card for a file that it does not hold or when it |
| @@ -316,11 +333,11 @@ | |
| 333 | exactly is not a cluster. There must be no extra whitespace in |
| 334 | the file. There must be one or more M cards. There must be a |
| 335 | single Z card with a correct MD5 checksum. And all cards must |
| 336 | be in strict lexicographical order.</p> |
| 337 | |
| 338 | <h3>4.1 The Unclustered Table</h3> |
| 339 | |
| 340 | <p>Every repository maintains a table named "<b>unclustered</b>" |
| 341 | which records the identity of every file and phantom it holds that is not |
| 342 | mentioned in a cluster. The entries in the unclustered table can |
| 343 | be thought of as leaves on a tree of files. Some of the unclustered |
| @@ -327,13 +344,13 @@ | |
| 344 | files will be clusters. Those clusters may contain other clusters, |
| 345 | which might contain still more clusters, and so forth. Beginning |
| 346 | with the files in the unclustered table, one can follow the chain |
| 347 | of clusters to find every file in the repository.</p> |
| 348 | |
| 349 | <h2>5.0 Synchronization Strategies</h2> |
| 350 | |
| 351 | <h3>5.1 Pull</h3> |
| 352 | |
| 353 | <p>A typical pull operation proceeds as shown below. Details |
| 354 | of the actual implementation may very slightly but the gist of |
| 355 | a pull is captured in the following steps:</p> |
| 356 | |
| @@ -381,11 +398,11 @@ | |
| 398 | protocol will continue to work even if there are multiple servers |
| 399 | or if servers and clients sometimes change roles. The only negative |
| 400 | effects of these unusual arrangements is that more than the minimum |
| 401 | number of clusters might be generated.</p> |
| 402 | |
| 403 | <h3>5.2 Push</h3> |
| 404 | |
| 405 | <p>A typical push operation proceeds roughly as shown below. As |
| 406 | with a pull, the actual implementation may vary slightly.</p> |
| 407 | |
| 408 | <ol> |
| @@ -415,19 +432,19 @@ | |
| 432 | server knows all files that exist on the client. Also, as with |
| 433 | pull, the client attempts to keep the size of the request from |
| 434 | growing too large by suppressing file cards once the |
| 435 | size of the request reaches 1MB.</p> |
| 436 | |
| 437 | <h3>5.3 Sync</h3> |
| 438 | |
| 439 | <p>A sync is just a pull and a push that happen at the same time. |
| 440 | The first three steps of a pull are combined with the first five steps |
| 441 | of a push. Steps (4) through (7) of a pull are combined with steps |
| 442 | (5) through (8) of a push. And steps (8) through (10) of a pull |
| 443 | are combined with step (9) of a push.</p> |
| 444 | |
| 445 | <h2>6.0 Summary</h2> |
| 446 | |
| 447 | <p>Here are the key points of the synchronization protocol:</p> |
| 448 | |
| 449 | <ol> |
| 450 | <li>The client sends one or more PUSH HTTP requests to the server. |
| 451 |