Fossil SCM

Bring the sync protocol documentation more up-to-date. Fix typos in the command-line documentation for "push" and "sync".

drh 2016-08-04 00:42 trunk
Commit 3dcf69a7410b4066bc555bbb343a39c97b58158d
Parent a5c08fde0ab2e9f…
3 files changed +2 -2 +5 -2 +96 -27
~ src/sync.c ~ www/server.wiki ~ www/sync.wiki
M src/sync.c
+2 -2
--- src/sync.c
+++ src/sync.c
@@ -233,11 +233,11 @@
233233
 ** -B|--httpauth USER:PASS Credentials for the simple HTTP auth protocol,
234234
 ** if required by the remote website
235235
 ** --ipv4 Use only IPv4, not IPv6
236236
 ** --once Do not remember URL for subsequent syncs
237237
 ** --proxy PROXY Use the specified HTTP proxy
238
-** --private Pull private branches too
238
+** --private Push private branches too
239239
 ** -R|--repository REPO Repository to pull into
240240
 ** --ssl-identity FILE Local SSL credentials, if requested by remote
241241
 ** --ssh-command SSH Use SSH as the "ssh" command
242242
 ** -v|--verbose Additional (debugging) output
243243
 ** --verily Exchange extra information with the remote
@@ -278,11 +278,11 @@
278278
 ** -B|--httpauth USER:PASS Credentials for the simple HTTP auth protocol,
279279
 ** if required by the remote website
280280
 ** --ipv4 Use only IPv4, not IPv6
281281
 ** --once Do not remember URL for subsequent syncs
282282
 ** --proxy PROXY Use the specified HTTP proxy
283
-** --private Pull private branches too
283
+** --private Sync private branches too
284284
 ** -R|--repository REPO Repository to pull into
285285
 ** --ssl-identity FILE Local SSL credentials, if requested by remote
286286
 ** --ssh-command SSH Use SSH as the "ssh" command
287287
 ** -v|--verbose Additional (debugging) output
288288
 ** --verily Exchange extra information with the remote
289289
--- src/sync.c
+++ src/sync.c
@@ -233,11 +233,11 @@
233 ** -B|--httpauth USER:PASS Credentials for the simple HTTP auth protocol,
234 ** if required by the remote website
235 ** --ipv4 Use only IPv4, not IPv6
236 ** --once Do not remember URL for subsequent syncs
237 ** --proxy PROXY Use the specified HTTP proxy
238 ** --private Pull private branches too
239 ** -R|--repository REPO Repository to pull into
240 ** --ssl-identity FILE Local SSL credentials, if requested by remote
241 ** --ssh-command SSH Use SSH as the "ssh" command
242 ** -v|--verbose Additional (debugging) output
243 ** --verily Exchange extra information with the remote
@@ -278,11 +278,11 @@
278 ** -B|--httpauth USER:PASS Credentials for the simple HTTP auth protocol,
279 ** if required by the remote website
280 ** --ipv4 Use only IPv4, not IPv6
281 ** --once Do not remember URL for subsequent syncs
282 ** --proxy PROXY Use the specified HTTP proxy
283 ** --private Pull private branches too
284 ** -R|--repository REPO Repository to pull into
285 ** --ssl-identity FILE Local SSL credentials, if requested by remote
286 ** --ssh-command SSH Use SSH as the "ssh" command
287 ** -v|--verbose Additional (debugging) output
288 ** --verily Exchange extra information with the remote
289
--- src/sync.c
+++ src/sync.c
@@ -233,11 +233,11 @@
233 ** -B|--httpauth USER:PASS Credentials for the simple HTTP auth protocol,
234 ** if required by the remote website
235 ** --ipv4 Use only IPv4, not IPv6
236 ** --once Do not remember URL for subsequent syncs
237 ** --proxy PROXY Use the specified HTTP proxy
238 ** --private Push private branches too
239 ** -R|--repository REPO Repository to pull into
240 ** --ssl-identity FILE Local SSL credentials, if requested by remote
241 ** --ssh-command SSH Use SSH as the "ssh" command
242 ** -v|--verbose Additional (debugging) output
243 ** --verily Exchange extra information with the remote
@@ -278,11 +278,11 @@
278 ** -B|--httpauth USER:PASS Credentials for the simple HTTP auth protocol,
279 ** if required by the remote website
280 ** --ipv4 Use only IPv4, not IPv6
281 ** --once Do not remember URL for subsequent syncs
282 ** --proxy PROXY Use the specified HTTP proxy
283 ** --private Sync private branches too
284 ** -R|--repository REPO Repository to pull into
285 ** --ssl-identity FILE Local SSL credentials, if requested by remote
286 ** --ssh-command SSH Use SSH as the "ssh" command
287 ** -v|--verbose Additional (debugging) output
288 ** --verily Exchange extra information with the remote
289
M www/server.wiki
+5 -2
--- www/server.wiki
+++ www/server.wiki
@@ -6,12 +6,15 @@
66
 page you are now reading (but excepting the
77
 [https://www.fossil-scm.org/download.html|download page]),
88
 is just a Fossil server displaying the content of the
99
 self-hosting repository for Fossil.</p>
1010
 <p>This article is a guide for setting up your own Fossil server.
11
-<p>See the [./aboutcgi.wiki|How CGI Works In Fossil] document for background
12
-information on the underlying technology.</p></blockquote>
11
+<p>See "[./aboutcgi.wiki|How CGI Works In Fossil]" for background
12
+information on the underlying CGI technology.
13
+See "[./sync.wiki|The Fossil Sync Protocol]" for information on the
14
+wire protocol used for client/server communication.</p>
15
+</blockquote>
1316
 <h2>Overview</h2><blockquote>
1417
 There are basically four ways to set up a Fossil server:
1518
 <ol>
1619
 <li>A stand-alone server
1720
 <li>Using inetd or xinetd or stunnel
1821
--- www/server.wiki
+++ www/server.wiki
@@ -6,12 +6,15 @@
6 page you are now reading (but excepting the
7 [https://www.fossil-scm.org/download.html|download page]),
8 is just a Fossil server displaying the content of the
9 self-hosting repository for Fossil.</p>
10 <p>This article is a guide for setting up your own Fossil server.
11 <p>See the [./aboutcgi.wiki|How CGI Works In Fossil] document for background
12 information on the underlying technology.</p></blockquote>
 
 
 
13 <h2>Overview</h2><blockquote>
14 There are basically four ways to set up a Fossil server:
15 <ol>
16 <li>A stand-alone server
17 <li>Using inetd or xinetd or stunnel
18
--- www/server.wiki
+++ www/server.wiki
@@ -6,12 +6,15 @@
6 page you are now reading (but excepting the
7 [https://www.fossil-scm.org/download.html|download page]),
8 is just a Fossil server displaying the content of the
9 self-hosting repository for Fossil.</p>
10 <p>This article is a guide for setting up your own Fossil server.
11 <p>See "[./aboutcgi.wiki|How CGI Works In Fossil]" for background
12 information on the underlying CGI technology.
13 See "[./sync.wiki|The Fossil Sync Protocol]" for information on the
14 wire protocol used for client/server communication.</p>
15 </blockquote>
16 <h2>Overview</h2><blockquote>
17 There are basically four ways to set up a Fossil server:
18 <ol>
19 <li>A stand-alone server
20 <li>Using inetd or xinetd or stunnel
21
M www/sync.wiki
+96 -27
--- www/sync.wiki
+++ www/sync.wiki
@@ -1,20 +1,16 @@
11
 <title>The Fossil Sync Protocol</title>
22
3
-<p>Fossil supports commands <b>push</b>, <b>pull</b>, and <b>sync</b>
4
-for transferring information from one repository to another. The
5
-command is run on the client repository. A URL for the server repository
6
-is specified as part of the command. This document describes what happens
7
-behind the scenes in order to synchronize the information on the two
8
-repositories.</p>
3
+<p>This document describes the wire protocol used to synchronize
4
+content between two Fossil repositories.</p>
95
106
 <h2>1.0 Overview</h2>
117
128
 <p>The global state of a fossil repository consists of an unordered
139
 collection of artifacts. Each artifact is identified by its SHA1 hash
1410
 expressed as a 40-character lower-case hexadecimal string.
15
-Synchronization is simply the process of sharing artifacts between
11
+Synchronization is the process of sharing artifacts between
1612
 servers so that all servers have copies of all artifacts. Because
1713
 artifacts are unordered, the order in which artifacts are received
1814
 at a server is inconsequential. It is assumed that the SHA1 hashes
1915
 of artifacts are unique - that every artifact has a different SHA1 hash.
2016
 To a first approximation, synchronization proceeds by sharing lists
@@ -26,15 +22,16 @@
2622
 shared to a few hundred.</p>
2723
2824
 <p>Each repository also has local state. The local state determines
2925
 the web-page formatting preferences, authorized users, ticket formats,
3026
 and similar information that varies from one repository to another.
31
-The local state is not transferred by the <b>push</b>, <b>pull</b>,
32
-and <b>sync</b> command, though some local state is transferred during
33
-a <b>clone</b> in order to initialize the local state of the new
34
-repository. The <b>configuration push</b> and <b>configuration pull</b>
35
-commands can be used to send or receive local state.</p>
27
+The local state is not using transferred during a sync. Except,
28
+some local state is transferred during a [/help?cmd=clone|clone]
29
+in order to initialize the local state of the new repository. And
30
+the [/help?cmd=configuration|config push] and
31
+[/help?cmd=configuration|config pull]
32
+commands can be an administrator to sync local state.</p>
3633
3734
3835
 <h2>2.0 Transport</h2>
3936
4037
 <p>All communication between client and server is via HTTP requests.
@@ -43,20 +40,31 @@
4340
 request.</p>
4441
4542
 <p>The server might be running as an independent server
4643
 using the <b>server</b> command, or it might be launched from
4744
 inetd or xinetd using the <b>http</b> command. Or the server might
48
-be launched from CGI. The details of how the server is configured
49
-to "listen" for incoming HTTP requests is immaterial. The important
50
-point is that the server is listening for requests and the client
51
-is the issuer of the requests.</p>
45
+be launched from CGI.
46
+(See "[./server.wiki|How To Configure A Fossil Server]" for details.)
47
+The specifics of how the server listens
48
+for incoming HTTP requests is immaterial to this protocol.
49
+The important point is that the server is listening for requests and
50
+the client is the issuer of the requests.</p>
5251
5352
 <p>A single push, pull, or sync might involve multiple HTTP requests.
5453
 The client maintains state between all requests. But on the server
5554
 side, each request is independent. The server does not preserve
5655
 any information about the client from one request to the next.</p>
5756
57
+<h4>2.0.1 Encrypted Transport</h4>
58
+
59
+<p>In the current implementation of Fossil, the server only
60
+understands HTTP requests. The client can send either
61
+clear-text HTTP requested or or encrypted HTTPS requests. But when
62
+HTTPS requests are sent, they first must be decrypted by a webserver
63
+or proxy before being passed to the Fossil server. This limitation
64
+may be relaxed in a future release.</p>
65
+
5866
 <h3>2.1 Server Identification</h3>
5967
6068
 <p>The server is identified by a URL argument that accompanies the
6169
 push, pull, or sync command on the client. (As a convenience to
6270
 users, the URL can be omitted on the client command and the same URL
@@ -233,10 +241,26 @@
233241
 delta artifact.</p>
234242
235243
 <p>Unlike file cards, cfile cards are only sent in one direction during a
236244
 clone from server to client for clone protocol version "3" or greater.</p>
237245
246
+<h4>3.3.3 Private artifacts</h4>
247
+
248
+<p>"Private" content consist of artifacts that are not normally synced.
249
+However, private content will be synced when the
250
+the [/help?cmd=sync|fossil sync] command includes the "--private" option.
251
+</p>
252
+
253
+<p>Private content is marked by a "private" card:
254
+
255
+<blockquote>
256
+<b>private</b>
257
+</blockquote>
258
+
259
+<p>The private card has no arguments and must directly precede a
260
+file card that contains the private content.</p>
261
+
238262
 <h3>3.4 Push and Pull Cards</h3>
239263
240264
 <p>Among the first cards in a client-to-server message are
241265
 the push and pull cards. The push card tells the server that
242266
 the client is pushing content. The pull card tells the server
@@ -255,10 +279,12 @@
255279
 for the transaction to proceed.</p>
256280
257281
 <p>The server will also send a push card back to the client
258282
 during a clone. This is how the client determines what project
259283
 code to put in the new repository it is constructing.</p>
284
+
285
+<p>The <i>servercode</i> argument is currently unused.
260286
261287
 <h3>3.5 Clone Cards</h3>
262288
263289
 <p>A clone card works like a pull card in that it is sent from
264290
 client to server in order to tell the server that the client
@@ -320,19 +346,23 @@
320346
 <p>An igot card can be sent from either client to server or from
321347
 server to client in order to indicate that the sender holds a copy
322348
 of a particular artifact. The format is:</p>
323349
324350
 <blockquote>
325
-<b>igot</b> <i>artifact-id</i>
351
+<b>igot</b> <i>artifact-id</i> ?<i>flag</i>?
326352
 </blockquote>
327353
328
-<p>The argument of the igot card is the ID of the artifact that
354
+<p>The first argument of the igot card is the ID of the artifact that
329355
 the sender possesses.
330356
 The receiver of an igot card will typically check to see if
331357
 it also holds the same artifact and if not it will request the artifact
332358
 using a gimme card in either the reply or in the next message.</p>
333359
360
+<p>If the second argument exists and is "1", then the artifact
361
+identified by the first argument is private on the sender and should
362
+be ignored unless a "--private" [/help?cmd=sync|sync] is occurring.
363
+
334364
 <h3>3.7 Gimme Cards</h3>
335365
336366
 <p>A gimme card is sent from either client to server or from server
337367
 to client. The gimme card asks the receiver to send a particular
338368
 artifact back to the sender. The format of a gimme card is this:</p>
@@ -477,11 +507,50 @@
477507
 it had sent a corresponding reqconfig card in its request.
478508
479509
 <p>The content of the configuration item is used to overwrite the
480510
 corresponding configuration data in the receiver.
481511
482
-<h3>3.11 Error Cards</h3>
512
+<h3>3.11 Pragma Cards</h3>
513
+
514
+<p>The client may try to influence the behavior of the server by
515
+issuing a pragma card:
516
+
517
+<blockquote>
518
+<b>pragma</i> <i>name value...</i>
519
+</blockquote>
520
+
521
+<p>The "pragma" card has at least one argument which is the pragma name.
522
+The pragma name defines what the pragma does.
523
+A pragma might have zero or more "value" arguments
524
+depending on the pragma name.
525
+
526
+<p>New pragma names may be added to the protocol from time to time
527
+in order to enhance the capabilities of Fossil.
528
+Unknown pragmas are silently ignored, for backwards compatibility.
529
+
530
+<p>The following are the known pragma names as of 2016-08-03:
531
+
532
+<ol>
533
+<li><p><b>send-private</b>
534
+<p>The send-private pragma instructs the server to send all of its
535
+private artifacts to the client. The server will only obey this
536
+request if the user has the "x" or "Private" privilege.
537
+
538
+<li><p><b>send-catalog</b>
539
+<p>The send-catalog pragma instructs the server to transmit igot
540
+cards for every known artifact. This can help the client and server
541
+to get back in synchronization after a prior protocol error. The
542
+"--verily" option to the [/help?cmd=sync|fossil sync] command causes
543
+the send-catalog pragma to be transmitted.</p>
544
+</ol>
545
+
546
+<h3>3.12 Comment Cards</h3>
547
+
548
+<p>Any card that begins with "#" (ASCII 0x23) is a comment card and
549
+is silently ignored.</p>
550
+
551
+<h3>3.13 Error Cards</h3>
483552
484553
 <p>If the server discovers anything wrong with a request, it generates
485554
 an error card in its reply. When the client sees the error card,
486555
 it displays an error message to the user and aborts the sync
487556
 operation. An error card looks like this:</p>
@@ -497,16 +566,11 @@
497566
 (ASCII 0x5C) is represented as two backslashes "\\". Apart from
498567
 space and newline, no other whitespace characters nor any
499568
 unprintable characters are allowed in
500569
 the error message.</p>
501570
502
-<h3>3.12 Comment Cards</h3>
503
-
504
-<p>Any card that begins with "#" (ASCII 0x23) is a comment card and
505
-is silently ignored.</p>
506
-
507
-<h3>3.13 Unknown Cards</h3>
571
+<h3>3.14 Unknown Cards</h3>
508572
509573
 <p>If either the client or the server sees a card that is not
510574
 described above, then it generates an error and aborts.</p>
511575
512576
 <h2>4.0 Phantoms And Clusters</h2>
@@ -665,23 +729,28 @@
665729
 <ul>
666730
 <li> <b>login</b> <i>userid nonce signature</i>
667731
 <li> <b>push</b> <i>servercode projectcode</i>
668732
 <li> <b>pull</b> <i>servercode projectcode</i>
669733
 <li> <b>clone</b>
734
+ <li> <b>clone_seqno</b> <i>sequence-number</i>
670735
 <li> <b>file</b> <i>artifact-id size</i> <b>\n</b> <i>content</i>
671736
 <li> <b>file</b> <i>artifact-id delta-artifact-id size</i> <b>\n</b> <i>content</i>
672
- <li> <b>igot</b> <i>artifact-id</i>
737
+ <li> <b>cfile</b> <i>artifact-id size</i> <b>\n</b> <i>content</i>
738
+ <li> <b>cfile</b> <i>artifact-id delta-artifact-id size</i> <b>\n</b> <i>content</i>
739
+ <li> <b>private</b>
740
+ <li> <b>igot</b> <i>artifact-id</i> ?<i>flag</i>?
673741
 <li> <b>gimme</b> <i>artifact-id</i>
674742
 <li> <b>cookie</b> <i>cookie-text</i>
675743
 <li> <b>reqconfig</b> <i>parameter-name</i>
676744
 <li> <b>config</b> <i>parameter-name size</i> <b>\n</b> <i>content</i>
677
- <li> <b>#</b> <i>arbitrary-text...</i>
745
+ <li> <b>pragma</b> <i>name</i> <i>value...</i>
678746
 <li> <b>error</b> <i>error-message</i>
747
+ <li> <b>#</b> <i>arbitrary-text...</i>
679748
 </ul>
680749
 <li>Phantoms are artifacts that a repository knows exist but does not possess.
681750
 <li>Clusters are artifacts that contain IDs of other artifacts.
682751
 <li>Clusters are created automatically on the server during a pull.
683752
 <li>Repositories keep track of all artifacts that are not named in any
684753
 cluster and send igot messages for those artifacts.
685754
 <li>Repositories keep track of all the phantoms they hold and send
686755
 gimme messages for those artifacts.
687756
 </ol>
688757
--- www/sync.wiki
+++ www/sync.wiki
@@ -1,20 +1,16 @@
1 <title>The Fossil Sync Protocol</title>
2
3 <p>Fossil supports commands <b>push</b>, <b>pull</b>, and <b>sync</b>
4 for transferring information from one repository to another. The
5 command is run on the client repository. A URL for the server repository
6 is specified as part of the command. This document describes what happens
7 behind the scenes in order to synchronize the information on the two
8 repositories.</p>
9
10 <h2>1.0 Overview</h2>
11
12 <p>The global state of a fossil repository consists of an unordered
13 collection of artifacts. Each artifact is identified by its SHA1 hash
14 expressed as a 40-character lower-case hexadecimal string.
15 Synchronization is simply the process of sharing artifacts between
16 servers so that all servers have copies of all artifacts. Because
17 artifacts are unordered, the order in which artifacts are received
18 at a server is inconsequential. It is assumed that the SHA1 hashes
19 of artifacts are unique - that every artifact has a different SHA1 hash.
20 To a first approximation, synchronization proceeds by sharing lists
@@ -26,15 +22,16 @@
26 shared to a few hundred.</p>
27
28 <p>Each repository also has local state. The local state determines
29 the web-page formatting preferences, authorized users, ticket formats,
30 and similar information that varies from one repository to another.
31 The local state is not transferred by the <b>push</b>, <b>pull</b>,
32 and <b>sync</b> command, though some local state is transferred during
33 a <b>clone</b> in order to initialize the local state of the new
34 repository. The <b>configuration push</b> and <b>configuration pull</b>
35 commands can be used to send or receive local state.</p>
 
36
37
38 <h2>2.0 Transport</h2>
39
40 <p>All communication between client and server is via HTTP requests.
@@ -43,20 +40,31 @@
43 request.</p>
44
45 <p>The server might be running as an independent server
46 using the <b>server</b> command, or it might be launched from
47 inetd or xinetd using the <b>http</b> command. Or the server might
48 be launched from CGI. The details of how the server is configured
49 to "listen" for incoming HTTP requests is immaterial. The important
50 point is that the server is listening for requests and the client
51 is the issuer of the requests.</p>
 
 
52
53 <p>A single push, pull, or sync might involve multiple HTTP requests.
54 The client maintains state between all requests. But on the server
55 side, each request is independent. The server does not preserve
56 any information about the client from one request to the next.</p>
57
 
 
 
 
 
 
 
 
 
58 <h3>2.1 Server Identification</h3>
59
60 <p>The server is identified by a URL argument that accompanies the
61 push, pull, or sync command on the client. (As a convenience to
62 users, the URL can be omitted on the client command and the same URL
@@ -233,10 +241,26 @@
233 delta artifact.</p>
234
235 <p>Unlike file cards, cfile cards are only sent in one direction during a
236 clone from server to client for clone protocol version "3" or greater.</p>
237
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
238 <h3>3.4 Push and Pull Cards</h3>
239
240 <p>Among the first cards in a client-to-server message are
241 the push and pull cards. The push card tells the server that
242 the client is pushing content. The pull card tells the server
@@ -255,10 +279,12 @@
255 for the transaction to proceed.</p>
256
257 <p>The server will also send a push card back to the client
258 during a clone. This is how the client determines what project
259 code to put in the new repository it is constructing.</p>
 
 
260
261 <h3>3.5 Clone Cards</h3>
262
263 <p>A clone card works like a pull card in that it is sent from
264 client to server in order to tell the server that the client
@@ -320,19 +346,23 @@
320 <p>An igot card can be sent from either client to server or from
321 server to client in order to indicate that the sender holds a copy
322 of a particular artifact. The format is:</p>
323
324 <blockquote>
325 <b>igot</b> <i>artifact-id</i>
326 </blockquote>
327
328 <p>The argument of the igot card is the ID of the artifact that
329 the sender possesses.
330 The receiver of an igot card will typically check to see if
331 it also holds the same artifact and if not it will request the artifact
332 using a gimme card in either the reply or in the next message.</p>
333
 
 
 
 
334 <h3>3.7 Gimme Cards</h3>
335
336 <p>A gimme card is sent from either client to server or from server
337 to client. The gimme card asks the receiver to send a particular
338 artifact back to the sender. The format of a gimme card is this:</p>
@@ -477,11 +507,50 @@
477 it had sent a corresponding reqconfig card in its request.
478
479 <p>The content of the configuration item is used to overwrite the
480 corresponding configuration data in the receiver.
481
482 <h3>3.11 Error Cards</h3>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
483
484 <p>If the server discovers anything wrong with a request, it generates
485 an error card in its reply. When the client sees the error card,
486 it displays an error message to the user and aborts the sync
487 operation. An error card looks like this:</p>
@@ -497,16 +566,11 @@
497 (ASCII 0x5C) is represented as two backslashes "\\". Apart from
498 space and newline, no other whitespace characters nor any
499 unprintable characters are allowed in
500 the error message.</p>
501
502 <h3>3.12 Comment Cards</h3>
503
504 <p>Any card that begins with "#" (ASCII 0x23) is a comment card and
505 is silently ignored.</p>
506
507 <h3>3.13 Unknown Cards</h3>
508
509 <p>If either the client or the server sees a card that is not
510 described above, then it generates an error and aborts.</p>
511
512 <h2>4.0 Phantoms And Clusters</h2>
@@ -665,23 +729,28 @@
665 <ul>
666 <li> <b>login</b> <i>userid nonce signature</i>
667 <li> <b>push</b> <i>servercode projectcode</i>
668 <li> <b>pull</b> <i>servercode projectcode</i>
669 <li> <b>clone</b>
 
670 <li> <b>file</b> <i>artifact-id size</i> <b>\n</b> <i>content</i>
671 <li> <b>file</b> <i>artifact-id delta-artifact-id size</i> <b>\n</b> <i>content</i>
672 <li> <b>igot</b> <i>artifact-id</i>
 
 
 
673 <li> <b>gimme</b> <i>artifact-id</i>
674 <li> <b>cookie</b> <i>cookie-text</i>
675 <li> <b>reqconfig</b> <i>parameter-name</i>
676 <li> <b>config</b> <i>parameter-name size</i> <b>\n</b> <i>content</i>
677 <li> <b>#</b> <i>arbitrary-text...</i>
678 <li> <b>error</b> <i>error-message</i>
 
679 </ul>
680 <li>Phantoms are artifacts that a repository knows exist but does not possess.
681 <li>Clusters are artifacts that contain IDs of other artifacts.
682 <li>Clusters are created automatically on the server during a pull.
683 <li>Repositories keep track of all artifacts that are not named in any
684 cluster and send igot messages for those artifacts.
685 <li>Repositories keep track of all the phantoms they hold and send
686 gimme messages for those artifacts.
687 </ol>
688
--- www/sync.wiki
+++ www/sync.wiki
@@ -1,20 +1,16 @@
1 <title>The Fossil Sync Protocol</title>
2
3 <p>This document describes the wire protocol used to synchronize
4 content between two Fossil repositories.</p>
 
 
 
 
5
6 <h2>1.0 Overview</h2>
7
8 <p>The global state of a fossil repository consists of an unordered
9 collection of artifacts. Each artifact is identified by its SHA1 hash
10 expressed as a 40-character lower-case hexadecimal string.
11 Synchronization is the process of sharing artifacts between
12 servers so that all servers have copies of all artifacts. Because
13 artifacts are unordered, the order in which artifacts are received
14 at a server is inconsequential. It is assumed that the SHA1 hashes
15 of artifacts are unique - that every artifact has a different SHA1 hash.
16 To a first approximation, synchronization proceeds by sharing lists
@@ -26,15 +22,16 @@
22 shared to a few hundred.</p>
23
24 <p>Each repository also has local state. The local state determines
25 the web-page formatting preferences, authorized users, ticket formats,
26 and similar information that varies from one repository to another.
27 The local state is not using transferred during a sync. Except,
28 some local state is transferred during a [/help?cmd=clone|clone]
29 in order to initialize the local state of the new repository. And
30 the [/help?cmd=configuration|config push] and
31 [/help?cmd=configuration|config pull]
32 commands can be an administrator to sync local state.</p>
33
34
35 <h2>2.0 Transport</h2>
36
37 <p>All communication between client and server is via HTTP requests.
@@ -43,20 +40,31 @@
40 request.</p>
41
42 <p>The server might be running as an independent server
43 using the <b>server</b> command, or it might be launched from
44 inetd or xinetd using the <b>http</b> command. Or the server might
45 be launched from CGI.
46 (See "[./server.wiki|How To Configure A Fossil Server]" for details.)
47 The specifics of how the server listens
48 for incoming HTTP requests is immaterial to this protocol.
49 The important point is that the server is listening for requests and
50 the client is the issuer of the requests.</p>
51
52 <p>A single push, pull, or sync might involve multiple HTTP requests.
53 The client maintains state between all requests. But on the server
54 side, each request is independent. The server does not preserve
55 any information about the client from one request to the next.</p>
56
57 <h4>2.0.1 Encrypted Transport</h4>
58
59 <p>In the current implementation of Fossil, the server only
60 understands HTTP requests. The client can send either
61 clear-text HTTP requested or or encrypted HTTPS requests. But when
62 HTTPS requests are sent, they first must be decrypted by a webserver
63 or proxy before being passed to the Fossil server. This limitation
64 may be relaxed in a future release.</p>
65
66 <h3>2.1 Server Identification</h3>
67
68 <p>The server is identified by a URL argument that accompanies the
69 push, pull, or sync command on the client. (As a convenience to
70 users, the URL can be omitted on the client command and the same URL
@@ -233,10 +241,26 @@
241 delta artifact.</p>
242
243 <p>Unlike file cards, cfile cards are only sent in one direction during a
244 clone from server to client for clone protocol version "3" or greater.</p>
245
246 <h4>3.3.3 Private artifacts</h4>
247
248 <p>"Private" content consist of artifacts that are not normally synced.
249 However, private content will be synced when the
250 the [/help?cmd=sync|fossil sync] command includes the "--private" option.
251 </p>
252
253 <p>Private content is marked by a "private" card:
254
255 <blockquote>
256 <b>private</b>
257 </blockquote>
258
259 <p>The private card has no arguments and must directly precede a
260 file card that contains the private content.</p>
261
262 <h3>3.4 Push and Pull Cards</h3>
263
264 <p>Among the first cards in a client-to-server message are
265 the push and pull cards. The push card tells the server that
266 the client is pushing content. The pull card tells the server
@@ -255,10 +279,12 @@
279 for the transaction to proceed.</p>
280
281 <p>The server will also send a push card back to the client
282 during a clone. This is how the client determines what project
283 code to put in the new repository it is constructing.</p>
284
285 <p>The <i>servercode</i> argument is currently unused.
286
287 <h3>3.5 Clone Cards</h3>
288
289 <p>A clone card works like a pull card in that it is sent from
290 client to server in order to tell the server that the client
@@ -320,19 +346,23 @@
346 <p>An igot card can be sent from either client to server or from
347 server to client in order to indicate that the sender holds a copy
348 of a particular artifact. The format is:</p>
349
350 <blockquote>
351 <b>igot</b> <i>artifact-id</i> ?<i>flag</i>?
352 </blockquote>
353
354 <p>The first argument of the igot card is the ID of the artifact that
355 the sender possesses.
356 The receiver of an igot card will typically check to see if
357 it also holds the same artifact and if not it will request the artifact
358 using a gimme card in either the reply or in the next message.</p>
359
360 <p>If the second argument exists and is "1", then the artifact
361 identified by the first argument is private on the sender and should
362 be ignored unless a "--private" [/help?cmd=sync|sync] is occurring.
363
364 <h3>3.7 Gimme Cards</h3>
365
366 <p>A gimme card is sent from either client to server or from server
367 to client. The gimme card asks the receiver to send a particular
368 artifact back to the sender. The format of a gimme card is this:</p>
@@ -477,11 +507,50 @@
507 it had sent a corresponding reqconfig card in its request.
508
509 <p>The content of the configuration item is used to overwrite the
510 corresponding configuration data in the receiver.
511
512 <h3>3.11 Pragma Cards</h3>
513
514 <p>The client may try to influence the behavior of the server by
515 issuing a pragma card:
516
517 <blockquote>
518 <b>pragma</i> <i>name value...</i>
519 </blockquote>
520
521 <p>The "pragma" card has at least one argument which is the pragma name.
522 The pragma name defines what the pragma does.
523 A pragma might have zero or more "value" arguments
524 depending on the pragma name.
525
526 <p>New pragma names may be added to the protocol from time to time
527 in order to enhance the capabilities of Fossil.
528 Unknown pragmas are silently ignored, for backwards compatibility.
529
530 <p>The following are the known pragma names as of 2016-08-03:
531
532 <ol>
533 <li><p><b>send-private</b>
534 <p>The send-private pragma instructs the server to send all of its
535 private artifacts to the client. The server will only obey this
536 request if the user has the "x" or "Private" privilege.
537
538 <li><p><b>send-catalog</b>
539 <p>The send-catalog pragma instructs the server to transmit igot
540 cards for every known artifact. This can help the client and server
541 to get back in synchronization after a prior protocol error. The
542 "--verily" option to the [/help?cmd=sync|fossil sync] command causes
543 the send-catalog pragma to be transmitted.</p>
544 </ol>
545
546 <h3>3.12 Comment Cards</h3>
547
548 <p>Any card that begins with "#" (ASCII 0x23) is a comment card and
549 is silently ignored.</p>
550
551 <h3>3.13 Error Cards</h3>
552
553 <p>If the server discovers anything wrong with a request, it generates
554 an error card in its reply. When the client sees the error card,
555 it displays an error message to the user and aborts the sync
556 operation. An error card looks like this:</p>
@@ -497,16 +566,11 @@
566 (ASCII 0x5C) is represented as two backslashes "\\". Apart from
567 space and newline, no other whitespace characters nor any
568 unprintable characters are allowed in
569 the error message.</p>
570
571 <h3>3.14 Unknown Cards</h3>
 
 
 
 
 
572
573 <p>If either the client or the server sees a card that is not
574 described above, then it generates an error and aborts.</p>
575
576 <h2>4.0 Phantoms And Clusters</h2>
@@ -665,23 +729,28 @@
729 <ul>
730 <li> <b>login</b> <i>userid nonce signature</i>
731 <li> <b>push</b> <i>servercode projectcode</i>
732 <li> <b>pull</b> <i>servercode projectcode</i>
733 <li> <b>clone</b>
734 <li> <b>clone_seqno</b> <i>sequence-number</i>
735 <li> <b>file</b> <i>artifact-id size</i> <b>\n</b> <i>content</i>
736 <li> <b>file</b> <i>artifact-id delta-artifact-id size</i> <b>\n</b> <i>content</i>
737 <li> <b>cfile</b> <i>artifact-id size</i> <b>\n</b> <i>content</i>
738 <li> <b>cfile</b> <i>artifact-id delta-artifact-id size</i> <b>\n</b> <i>content</i>
739 <li> <b>private</b>
740 <li> <b>igot</b> <i>artifact-id</i> ?<i>flag</i>?
741 <li> <b>gimme</b> <i>artifact-id</i>
742 <li> <b>cookie</b> <i>cookie-text</i>
743 <li> <b>reqconfig</b> <i>parameter-name</i>
744 <li> <b>config</b> <i>parameter-name size</i> <b>\n</b> <i>content</i>
745 <li> <b>pragma</b> <i>name</i> <i>value...</i>
746 <li> <b>error</b> <i>error-message</i>
747 <li> <b>#</b> <i>arbitrary-text...</i>
748 </ul>
749 <li>Phantoms are artifacts that a repository knows exist but does not possess.
750 <li>Clusters are artifacts that contain IDs of other artifacts.
751 <li>Clusters are created automatically on the server during a pull.
752 <li>Repositories keep track of all artifacts that are not named in any
753 cluster and send igot messages for those artifacts.
754 <li>Repositories keep track of all the phantoms they hold and send
755 gimme messages for those artifacts.
756 </ol>
757

Keyboard Shortcuts

Open search /
Next entry (timeline) j
Previous entry (timeline) k
Open focused entry Enter
Show this help ?
Toggle theme Top nav button