Fossil SCM
Update the sync protocol documentation to describe the unversioned file sync.
Commit
d78094932026b0c3d161507d0e5b20c16b3a5d12
Parent
faa73ae0c4a82af…
1 file changed
+175
-6
+175
-6
| --- www/sync.wiki | ||
| +++ www/sync.wiki | ||
| @@ -167,18 +167,21 @@ | ||
| 167 | 167 | |
| 168 | 168 | <p>Privileges are cumulative. There can be multiple successful |
| 169 | 169 | login cards. The session privileges are the bit-wise OR of the |
| 170 | 170 | privileges of each individual login.</p> |
| 171 | 171 | |
| 172 | -<h3>3.3 File and Compressed File Cards</h3> | |
| 172 | +<h3>3.3 File Cards</h3> | |
| 173 | 173 | |
| 174 | -<p>Artifacts are transferred using either "file" cards, or "cfile" cards. | |
| 175 | -(The name "file" card comes from the fact that most artifacts correspond to | |
| 176 | -files, and "cfile" is just a compressed file.) | |
| 174 | +<p>Artifacts are transferred using either "file" cards, or "cfile" | |
| 175 | +or "uvfile" cards. | |
| 176 | +The name "file" card comes from the fact that most artifacts correspond to | |
| 177 | +files that are under version control. | |
| 178 | +The "cfile" name is an abbreviation for "compressed file". | |
| 179 | +The "uvfile" name is an abbreviation for "unversioned file". | |
| 177 | 180 | </p> |
| 178 | 181 | |
| 179 | -<h4>3.3.1 File Cards</h4> | |
| 182 | +<h4>3.3.1 Ordinary File Cards</h4> | |
| 180 | 183 | |
| 181 | 184 | <p>For sync protocols, artifacts are transferred using "file" |
| 182 | 185 | cards. File cards come in two different formats depending |
| 183 | 186 | on whether the artifact is sent directly or as a delta from some |
| 184 | 187 | other artifact.</p> |
| @@ -186,11 +189,11 @@ | ||
| 186 | 189 | <blockquote> |
| 187 | 190 | <b>file</b> <i>artifact-id size</i> <b>\n</b> <i>content</i><br> |
| 188 | 191 | <b>file</b> <i>artifact-id delta-artifact-id size</i> <b>\n</b> <i>content</i> |
| 189 | 192 | </blockquote> |
| 190 | 193 | |
| 191 | -<p>File cards are different from all other cards in that they | |
| 194 | +<p>File cards are different from most other cards in that they are | |
| 192 | 195 | followed by in-line "payload" data. The content of the artifact |
| 193 | 196 | or the artifact delta consists of the first <i>size</i> bytes of the |
| 194 | 197 | x-fossil content that immediately follow the newline that |
| 195 | 198 | terminates the file card. Only file and cfile cards have this characteristic. |
| 196 | 199 | </p> |
| @@ -257,10 +260,44 @@ | ||
| 257 | 260 | </blockquote> |
| 258 | 261 | |
| 259 | 262 | <p>The private card has no arguments and must directly precede a |
| 260 | 263 | file card that contains the private content.</p> |
| 261 | 264 | |
| 265 | +<h4>3.3.4 Unversioned File Cards</h4> | |
| 266 | + | |
| 267 | +<p>Unversioned content is sent in both directions (client to server and | |
| 268 | +server to client) using "uvfile" cards in the following format: | |
| 269 | + | |
| 270 | +<blockquote> | |
| 271 | +<b>uvfile</b> <i>name mtime hash size flags</i> <b>\n</b> <i>content</i> | |
| 272 | +</blockquote> | |
| 273 | + | |
| 274 | +<p>The <i>name</i> field is the name of the unversioned file. The | |
| 275 | +<i>mtime</i> is the last modification time of the file in seconds | |
| 276 | +since 1970. The <i>hash</i> field is the SHA1 hash of the content | |
| 277 | +for the unversioned file, or "<b>-</b>" for deleted content. | |
| 278 | +The <i>size</i> field is the (uncompressed) size of the content | |
| 279 | +in bytes. The <i>flags</i> field is an integer which is interpreted | |
| 280 | +as an array of bits. The 0x0004 bit of <i>flags</i> indicates that | |
| 281 | +the <i>content</i> is to be omitted. The content might be omitted if | |
| 282 | +it is too large to transmit, or if the send merely wants to update the | |
| 283 | +modification time of the file without changing the files content. | |
| 284 | +The <i>content</i> is the (uncompressed) content of the file. | |
| 285 | + | |
| 286 | +<p>The receiver should only accept the uvfile card if the hash and | |
| 287 | +size match the content and if the mtime is newer than any existing | |
| 288 | +instance of the same file held by the receiver. The sender will not | |
| 289 | +normally transmit a uvfile card unless all these constraints are true, | |
| 290 | +but the receiver should double-check. | |
| 291 | + | |
| 292 | +<p>A server should only accept uvfile cards if the login user has | |
| 293 | +the "y" write-unversioned permission. | |
| 294 | + | |
| 295 | +<p>Servers send uvfile cards in response to uvgimme cards received from | |
| 296 | +the client. Clients send uvfile cards when they determine that the server | |
| 297 | +needs the content based on uvigot cards previously received from the server. | |
| 298 | + | |
| 262 | 299 | <h3>3.4 Push and Pull Cards</h3> |
| 263 | 300 | |
| 264 | 301 | <p>Among the first cards in a client-to-server message are |
| 265 | 302 | the push and pull cards. The push card tells the server that |
| 266 | 303 | the client is pushing content. The pull card tells the server |
| @@ -359,10 +396,42 @@ | ||
| 359 | 396 | |
| 360 | 397 | <p>If the second argument exists and is "1", then the artifact |
| 361 | 398 | identified by the first argument is private on the sender and should |
| 362 | 399 | be ignored unless a "--private" [/help?cmd=sync|sync] is occurring. |
| 363 | 400 | |
| 401 | +<h4>3.6.1 Unversioned Igot Cards</h4> | |
| 402 | + | |
| 403 | +<p>Zero or more "uvigot" cards are sent from client to server when | |
| 404 | +synchronizing unversioned content. The format of a uvigot card is | |
| 405 | +as follows: | |
| 406 | + | |
| 407 | +<blockquote> | |
| 408 | +<b>uvigot</b> <i>name mtime hash size</i> | |
| 409 | +</blockquote> | |
| 410 | + | |
| 411 | +<p>The <i>name</i> argument is the name of an unversioned file. | |
| 412 | +The <i>mtime</i> is the last modification time of the unversioned file | |
| 413 | +in seconds since 1970. | |
| 414 | +The <i>hash</i> is the SHA1 hash of the unversioned file content, or | |
| 415 | +"<b>-</b>" if the file has been deleted. | |
| 416 | +The <i>size</i> is the uncompressed size of the file in bytes. | |
| 417 | + | |
| 418 | +<p>When the server sees a "pragma uv-hash" card for which the hash | |
| 419 | +does not match, it sends uvigot cards for every unversioned file that it | |
| 420 | +holds. The client will use this information to figure out which | |
| 421 | +unversioned files need to be synchronized. | |
| 422 | +The server might also send a uvigot card when it receives a uvgimme card | |
| 423 | +but its reply message size is already oversized and hence unable to hold | |
| 424 | +the usual uvfile reply. | |
| 425 | + | |
| 426 | +<p>When a client receives a "uvigot" card, it checks to see if the the | |
| 427 | +file needs to be transfered from client to server or from server to client. | |
| 428 | +If a client-to-server transmission is needed, the client schedules that | |
| 429 | +transfer to occur on a subsequent HTTP request. If a server-to-client | |
| 430 | +transfer is needed, then the client sends a "uvgimme" card back to the | |
| 431 | +server to request the file content. | |
| 432 | + | |
| 364 | 433 | <h3>3.7 Gimme Cards</h3> |
| 365 | 434 | |
| 366 | 435 | <p>A gimme card is sent from either client to server or from server |
| 367 | 436 | to client. The gimme card asks the receiver to send a particular |
| 368 | 437 | artifact back to the sender. The format of a gimme card is this:</p> |
| @@ -374,10 +443,26 @@ | ||
| 374 | 443 | <p>The argument to the gimme card is the ID of the artifact that |
| 375 | 444 | the sender wants. The receiver will typically respond to a |
| 376 | 445 | gimme card by sending a file card in its reply or in the next |
| 377 | 446 | message.</p> |
| 378 | 447 | |
| 448 | +<h4>3.7.1 Unversioned Gimme Cards</h4> | |
| 449 | + | |
| 450 | +<p>Sync synchronizing unversioned content, the client may send "uvgimme" | |
| 451 | +cards to the server. A uvgimme card requests that the server send | |
| 452 | +unversioned content to the client. The format of a uvgimme card is | |
| 453 | +as follows: | |
| 454 | + | |
| 455 | +<blockquote> | |
| 456 | +<b>uvgimme</b> <i>name</i> | |
| 457 | +</blockquote> | |
| 458 | + | |
| 459 | +<p>The <i>name</i> is the name of the unversioned file found on the | |
| 460 | +server that the client would like to have. When a server sees a | |
| 461 | +uvgimme card, it normally responses with a uvfile card, though it might | |
| 462 | +also send anoter uvigot card if the HTTP reply is already oversized. | |
| 463 | + | |
| 379 | 464 | <h3>3.8 Cookie Cards</h3> |
| 380 | 465 | |
| 381 | 466 | <p>A cookie card can be used by a server to record a small amount |
| 382 | 467 | of state information on a client. The server sends a cookie to the |
| 383 | 468 | client. The client sends the same cookie back to the server on |
| @@ -539,10 +624,49 @@ | ||
| 539 | 624 | <p>The send-catalog pragma instructs the server to transmit igot |
| 540 | 625 | cards for every known artifact. This can help the client and server |
| 541 | 626 | to get back in synchronization after a prior protocol error. The |
| 542 | 627 | "--verily" option to the [/help?cmd=sync|fossil sync] command causes |
| 543 | 628 | the send-catalog pragma to be transmitted.</p> |
| 629 | + | |
| 630 | +<li><p><b>uv-hash</b> <i>HASH</i> | |
| 631 | +<p>The uv-hash pragma is sent from client to server to provoke a | |
| 632 | +synchronization of unversioned content. The <i>HASH</i> is a SHA1 | |
| 633 | +hash of the names, modification times, and individual hashes of all | |
| 634 | +unversioned files on the client. If the unversioned content hash | |
| 635 | +from the client does not match the unversioned content hash on the | |
| 636 | +server, then the server will reply with either a "pragma uv-push-ok" | |
| 637 | +or "pragma uv-pull-only" card followed by one "uvigot" card for | |
| 638 | +each unversioned file currently held on the server. The collection | |
| 639 | +of "uvigot" cards sent in response to a "uv-hash" pragma is called | |
| 640 | +the "unversioned catalog". The client will used the unversioned | |
| 641 | +catalog to figure out which files (if any) need to be synchronized | |
| 642 | +between client and server and send appropriate "uvfile" or "uvgimme" | |
| 643 | +cards on the next HTTP request.</p> | |
| 644 | + | |
| 645 | +<p>If a client sends a uv-hash pragma and does not receive back | |
| 646 | +either a uv-pull-only or uv-push-ok pragma, that means that the | |
| 647 | +content on the server exactly matches the content on the client and | |
| 648 | +no further synchronization is required. | |
| 649 | + | |
| 650 | +<li><p><b>uv-pull-only</b></i> | |
| 651 | +<p>A server sends the uv-pull-only pragma to the client in response | |
| 652 | +to a uv-hash pragma with a mismatched content hash argument. This | |
| 653 | +pragma indicates that there are differences in unversioned content | |
| 654 | +between the client and server but that content can only be transfered | |
| 655 | +from server to client. The server is unwilling to accept content from | |
| 656 | +the client because the client login lacks the "write-unversioned" | |
| 657 | +permission.</p> | |
| 658 | + | |
| 659 | +<li><p><b>uv-push-ok</b></i> | |
| 660 | +<p>A server sends the uv-push-ok pragma to the client in response | |
| 661 | +to a uv-hash pragma with a mismatched content hash argument. This | |
| 662 | +pragma indicates that there are differences in unversioned content | |
| 663 | +between the client and server and that content can only be transfered | |
| 664 | +in either direction. The server is willing to accept content from | |
| 665 | +the client because the client login has the "write-unversioned" | |
| 666 | +permission.</p> | |
| 667 | + | |
| 544 | 668 | </ol> |
| 545 | 669 | |
| 546 | 670 | <h3>3.12 Comment Cards</h3> |
| 547 | 671 | |
| 548 | 672 | <p>Any card that begins with "#" (ASCII 0x23) is a comment card and |
| @@ -713,10 +837,52 @@ | ||
| 713 | 837 | The first three steps of a pull are combined with the first five steps |
| 714 | 838 | of a push. Steps (4) through (7) of a pull are combined with steps |
| 715 | 839 | (5) through (8) of a push. And steps (8) through (10) of a pull |
| 716 | 840 | are combined with step (9) of a push.</p> |
| 717 | 841 | |
| 842 | +<h3>5.4 Unversioned File Sync</h3> | |
| 843 | + | |
| 844 | +<p>"Unversioned files" are files held in the repository | |
| 845 | +where only the most recent version of the file is kept rather than | |
| 846 | +the entire change history. Unversioned files are intended to be | |
| 847 | +used to store ephemeral content, such as compiled binaries of the | |
| 848 | +most recent release. | |
| 849 | + | |
| 850 | +<p>Unversioned files are identified by name and timestamp (mtime). | |
| 851 | +Only the most recent version of each file (the version with | |
| 852 | +the largest mtime value) is retained. | |
| 853 | + | |
| 854 | +<p>Unversioned files are synchronized using the | |
| 855 | +[/help?cmd=unversioned|fossil unversioned sync] command. | |
| 856 | + | |
| 857 | +<p>A schematic of an unversioned file synchronization is as follows: | |
| 858 | + | |
| 859 | +<ol> | |
| 860 | +<li>The client sends a "pragma uv-hash" card to the server. The argument | |
| 861 | + to the uv-hash pragma is a hash of all filesnames, mtimes, and | |
| 862 | + content hashes for the unversioned files held by the client. | |
| 863 | + <hr> | |
| 864 | +<li>If the unversion content hash from the client matches the unversioned | |
| 865 | + content hash on the server, then nothing needs to be done and the | |
| 866 | + server no-ops. But if the hashes are different, then the server | |
| 867 | + replies with either a uv-pull-only or a uv-push-ok pragma followed by | |
| 868 | + uvigot cards for all unversioned files held on the server. | |
| 869 | + <hr> | |
| 870 | +<li>The client examines the uvigot cards received from the server and | |
| 871 | + determines which unversioned files need to be exchanged in order | |
| 872 | + to bring the client and server into synchronization. The client | |
| 873 | + then sends appropriate "uvgimme" or "uvfile" cards back to the | |
| 874 | + server. | |
| 875 | + <hr> | |
| 876 | +<li>The server updates its unversioned file store with received "uvfile" | |
| 877 | + cards and answers "uvgimme" cards with "uvfile" cards in its reply. | |
| 878 | +</ol> | |
| 879 | + | |
| 880 | +<p>The last two steps might be repeated multiple | |
| 881 | +times if there is more unversioned content to be transferred than will | |
| 882 | +fit comfortably in a single HTTP request. | |
| 883 | + | |
| 718 | 884 | <h2>6.0 Summary</h2> |
| 719 | 885 | |
| 720 | 886 | <p>Here are the key points of the synchronization protocol:</p> |
| 721 | 887 | |
| 722 | 888 | <ol> |
| @@ -734,13 +900,16 @@ | ||
| 734 | 900 | <li> <b>clone_seqno</b> <i>sequence-number</i> |
| 735 | 901 | <li> <b>file</b> <i>artifact-id size</i> <b>\n</b> <i>content</i> |
| 736 | 902 | <li> <b>file</b> <i>artifact-id delta-artifact-id size</i> <b>\n</b> <i>content</i> |
| 737 | 903 | <li> <b>cfile</b> <i>artifact-id size</i> <b>\n</b> <i>content</i> |
| 738 | 904 | <li> <b>cfile</b> <i>artifact-id delta-artifact-id size</i> <b>\n</b> <i>content</i> |
| 905 | + <li> <b>uvfile</b> <i>name mtime hash size flags</i> <b>\n</b> <i>content</i> | |
| 739 | 906 | <li> <b>private</b> |
| 740 | 907 | <li> <b>igot</b> <i>artifact-id</i> ?<i>flag</i>? |
| 908 | + <li> <b>uvigot</b> <i>name mtime hash size</i> | |
| 741 | 909 | <li> <b>gimme</b> <i>artifact-id</i> |
| 910 | + <li> <b>uvgimme</b> <i>name</i> | |
| 742 | 911 | <li> <b>cookie</b> <i>cookie-text</i> |
| 743 | 912 | <li> <b>reqconfig</b> <i>parameter-name</i> |
| 744 | 913 | <li> <b>config</b> <i>parameter-name size</i> <b>\n</b> <i>content</i> |
| 745 | 914 | <li> <b>pragma</b> <i>name</i> <i>value...</i> |
| 746 | 915 | <li> <b>error</b> <i>error-message</i> |
| 747 | 916 |
| --- www/sync.wiki | |
| +++ www/sync.wiki | |
| @@ -167,18 +167,21 @@ | |
| 167 | |
| 168 | <p>Privileges are cumulative. There can be multiple successful |
| 169 | login cards. The session privileges are the bit-wise OR of the |
| 170 | privileges of each individual login.</p> |
| 171 | |
| 172 | <h3>3.3 File and Compressed File Cards</h3> |
| 173 | |
| 174 | <p>Artifacts are transferred using either "file" cards, or "cfile" cards. |
| 175 | (The name "file" card comes from the fact that most artifacts correspond to |
| 176 | files, and "cfile" is just a compressed file.) |
| 177 | </p> |
| 178 | |
| 179 | <h4>3.3.1 File Cards</h4> |
| 180 | |
| 181 | <p>For sync protocols, artifacts are transferred using "file" |
| 182 | cards. File cards come in two different formats depending |
| 183 | on whether the artifact is sent directly or as a delta from some |
| 184 | other artifact.</p> |
| @@ -186,11 +189,11 @@ | |
| 186 | <blockquote> |
| 187 | <b>file</b> <i>artifact-id size</i> <b>\n</b> <i>content</i><br> |
| 188 | <b>file</b> <i>artifact-id delta-artifact-id size</i> <b>\n</b> <i>content</i> |
| 189 | </blockquote> |
| 190 | |
| 191 | <p>File cards are different from all other cards in that they |
| 192 | followed by in-line "payload" data. The content of the artifact |
| 193 | or the artifact delta consists of the first <i>size</i> bytes of the |
| 194 | x-fossil content that immediately follow the newline that |
| 195 | terminates the file card. Only file and cfile cards have this characteristic. |
| 196 | </p> |
| @@ -257,10 +260,44 @@ | |
| 257 | </blockquote> |
| 258 | |
| 259 | <p>The private card has no arguments and must directly precede a |
| 260 | file card that contains the private content.</p> |
| 261 | |
| 262 | <h3>3.4 Push and Pull Cards</h3> |
| 263 | |
| 264 | <p>Among the first cards in a client-to-server message are |
| 265 | the push and pull cards. The push card tells the server that |
| 266 | the client is pushing content. The pull card tells the server |
| @@ -359,10 +396,42 @@ | |
| 359 | |
| 360 | <p>If the second argument exists and is "1", then the artifact |
| 361 | identified by the first argument is private on the sender and should |
| 362 | be ignored unless a "--private" [/help?cmd=sync|sync] is occurring. |
| 363 | |
| 364 | <h3>3.7 Gimme Cards</h3> |
| 365 | |
| 366 | <p>A gimme card is sent from either client to server or from server |
| 367 | to client. The gimme card asks the receiver to send a particular |
| 368 | artifact back to the sender. The format of a gimme card is this:</p> |
| @@ -374,10 +443,26 @@ | |
| 374 | <p>The argument to the gimme card is the ID of the artifact that |
| 375 | the sender wants. The receiver will typically respond to a |
| 376 | gimme card by sending a file card in its reply or in the next |
| 377 | message.</p> |
| 378 | |
| 379 | <h3>3.8 Cookie Cards</h3> |
| 380 | |
| 381 | <p>A cookie card can be used by a server to record a small amount |
| 382 | of state information on a client. The server sends a cookie to the |
| 383 | client. The client sends the same cookie back to the server on |
| @@ -539,10 +624,49 @@ | |
| 539 | <p>The send-catalog pragma instructs the server to transmit igot |
| 540 | cards for every known artifact. This can help the client and server |
| 541 | to get back in synchronization after a prior protocol error. The |
| 542 | "--verily" option to the [/help?cmd=sync|fossil sync] command causes |
| 543 | the send-catalog pragma to be transmitted.</p> |
| 544 | </ol> |
| 545 | |
| 546 | <h3>3.12 Comment Cards</h3> |
| 547 | |
| 548 | <p>Any card that begins with "#" (ASCII 0x23) is a comment card and |
| @@ -713,10 +837,52 @@ | |
| 713 | The first three steps of a pull are combined with the first five steps |
| 714 | of a push. Steps (4) through (7) of a pull are combined with steps |
| 715 | (5) through (8) of a push. And steps (8) through (10) of a pull |
| 716 | are combined with step (9) of a push.</p> |
| 717 | |
| 718 | <h2>6.0 Summary</h2> |
| 719 | |
| 720 | <p>Here are the key points of the synchronization protocol:</p> |
| 721 | |
| 722 | <ol> |
| @@ -734,13 +900,16 @@ | |
| 734 | <li> <b>clone_seqno</b> <i>sequence-number</i> |
| 735 | <li> <b>file</b> <i>artifact-id size</i> <b>\n</b> <i>content</i> |
| 736 | <li> <b>file</b> <i>artifact-id delta-artifact-id size</i> <b>\n</b> <i>content</i> |
| 737 | <li> <b>cfile</b> <i>artifact-id size</i> <b>\n</b> <i>content</i> |
| 738 | <li> <b>cfile</b> <i>artifact-id delta-artifact-id size</i> <b>\n</b> <i>content</i> |
| 739 | <li> <b>private</b> |
| 740 | <li> <b>igot</b> <i>artifact-id</i> ?<i>flag</i>? |
| 741 | <li> <b>gimme</b> <i>artifact-id</i> |
| 742 | <li> <b>cookie</b> <i>cookie-text</i> |
| 743 | <li> <b>reqconfig</b> <i>parameter-name</i> |
| 744 | <li> <b>config</b> <i>parameter-name size</i> <b>\n</b> <i>content</i> |
| 745 | <li> <b>pragma</b> <i>name</i> <i>value...</i> |
| 746 | <li> <b>error</b> <i>error-message</i> |
| 747 |
| --- www/sync.wiki | |
| +++ www/sync.wiki | |
| @@ -167,18 +167,21 @@ | |
| 167 | |
| 168 | <p>Privileges are cumulative. There can be multiple successful |
| 169 | login cards. The session privileges are the bit-wise OR of the |
| 170 | privileges of each individual login.</p> |
| 171 | |
| 172 | <h3>3.3 File Cards</h3> |
| 173 | |
| 174 | <p>Artifacts are transferred using either "file" cards, or "cfile" |
| 175 | or "uvfile" cards. |
| 176 | The name "file" card comes from the fact that most artifacts correspond to |
| 177 | files that are under version control. |
| 178 | The "cfile" name is an abbreviation for "compressed file". |
| 179 | The "uvfile" name is an abbreviation for "unversioned file". |
| 180 | </p> |
| 181 | |
| 182 | <h4>3.3.1 Ordinary File Cards</h4> |
| 183 | |
| 184 | <p>For sync protocols, artifacts are transferred using "file" |
| 185 | cards. File cards come in two different formats depending |
| 186 | on whether the artifact is sent directly or as a delta from some |
| 187 | other artifact.</p> |
| @@ -186,11 +189,11 @@ | |
| 189 | <blockquote> |
| 190 | <b>file</b> <i>artifact-id size</i> <b>\n</b> <i>content</i><br> |
| 191 | <b>file</b> <i>artifact-id delta-artifact-id size</i> <b>\n</b> <i>content</i> |
| 192 | </blockquote> |
| 193 | |
| 194 | <p>File cards are different from most other cards in that they are |
| 195 | followed by in-line "payload" data. The content of the artifact |
| 196 | or the artifact delta consists of the first <i>size</i> bytes of the |
| 197 | x-fossil content that immediately follow the newline that |
| 198 | terminates the file card. Only file and cfile cards have this characteristic. |
| 199 | </p> |
| @@ -257,10 +260,44 @@ | |
| 260 | </blockquote> |
| 261 | |
| 262 | <p>The private card has no arguments and must directly precede a |
| 263 | file card that contains the private content.</p> |
| 264 | |
| 265 | <h4>3.3.4 Unversioned File Cards</h4> |
| 266 | |
| 267 | <p>Unversioned content is sent in both directions (client to server and |
| 268 | server to client) using "uvfile" cards in the following format: |
| 269 | |
| 270 | <blockquote> |
| 271 | <b>uvfile</b> <i>name mtime hash size flags</i> <b>\n</b> <i>content</i> |
| 272 | </blockquote> |
| 273 | |
| 274 | <p>The <i>name</i> field is the name of the unversioned file. The |
| 275 | <i>mtime</i> is the last modification time of the file in seconds |
| 276 | since 1970. The <i>hash</i> field is the SHA1 hash of the content |
| 277 | for the unversioned file, or "<b>-</b>" for deleted content. |
| 278 | The <i>size</i> field is the (uncompressed) size of the content |
| 279 | in bytes. The <i>flags</i> field is an integer which is interpreted |
| 280 | as an array of bits. The 0x0004 bit of <i>flags</i> indicates that |
| 281 | the <i>content</i> is to be omitted. The content might be omitted if |
| 282 | it is too large to transmit, or if the send merely wants to update the |
| 283 | modification time of the file without changing the files content. |
| 284 | The <i>content</i> is the (uncompressed) content of the file. |
| 285 | |
| 286 | <p>The receiver should only accept the uvfile card if the hash and |
| 287 | size match the content and if the mtime is newer than any existing |
| 288 | instance of the same file held by the receiver. The sender will not |
| 289 | normally transmit a uvfile card unless all these constraints are true, |
| 290 | but the receiver should double-check. |
| 291 | |
| 292 | <p>A server should only accept uvfile cards if the login user has |
| 293 | the "y" write-unversioned permission. |
| 294 | |
| 295 | <p>Servers send uvfile cards in response to uvgimme cards received from |
| 296 | the client. Clients send uvfile cards when they determine that the server |
| 297 | needs the content based on uvigot cards previously received from the server. |
| 298 | |
| 299 | <h3>3.4 Push and Pull Cards</h3> |
| 300 | |
| 301 | <p>Among the first cards in a client-to-server message are |
| 302 | the push and pull cards. The push card tells the server that |
| 303 | the client is pushing content. The pull card tells the server |
| @@ -359,10 +396,42 @@ | |
| 396 | |
| 397 | <p>If the second argument exists and is "1", then the artifact |
| 398 | identified by the first argument is private on the sender and should |
| 399 | be ignored unless a "--private" [/help?cmd=sync|sync] is occurring. |
| 400 | |
| 401 | <h4>3.6.1 Unversioned Igot Cards</h4> |
| 402 | |
| 403 | <p>Zero or more "uvigot" cards are sent from client to server when |
| 404 | synchronizing unversioned content. The format of a uvigot card is |
| 405 | as follows: |
| 406 | |
| 407 | <blockquote> |
| 408 | <b>uvigot</b> <i>name mtime hash size</i> |
| 409 | </blockquote> |
| 410 | |
| 411 | <p>The <i>name</i> argument is the name of an unversioned file. |
| 412 | The <i>mtime</i> is the last modification time of the unversioned file |
| 413 | in seconds since 1970. |
| 414 | The <i>hash</i> is the SHA1 hash of the unversioned file content, or |
| 415 | "<b>-</b>" if the file has been deleted. |
| 416 | The <i>size</i> is the uncompressed size of the file in bytes. |
| 417 | |
| 418 | <p>When the server sees a "pragma uv-hash" card for which the hash |
| 419 | does not match, it sends uvigot cards for every unversioned file that it |
| 420 | holds. The client will use this information to figure out which |
| 421 | unversioned files need to be synchronized. |
| 422 | The server might also send a uvigot card when it receives a uvgimme card |
| 423 | but its reply message size is already oversized and hence unable to hold |
| 424 | the usual uvfile reply. |
| 425 | |
| 426 | <p>When a client receives a "uvigot" card, it checks to see if the the |
| 427 | file needs to be transfered from client to server or from server to client. |
| 428 | If a client-to-server transmission is needed, the client schedules that |
| 429 | transfer to occur on a subsequent HTTP request. If a server-to-client |
| 430 | transfer is needed, then the client sends a "uvgimme" card back to the |
| 431 | server to request the file content. |
| 432 | |
| 433 | <h3>3.7 Gimme Cards</h3> |
| 434 | |
| 435 | <p>A gimme card is sent from either client to server or from server |
| 436 | to client. The gimme card asks the receiver to send a particular |
| 437 | artifact back to the sender. The format of a gimme card is this:</p> |
| @@ -374,10 +443,26 @@ | |
| 443 | <p>The argument to the gimme card is the ID of the artifact that |
| 444 | the sender wants. The receiver will typically respond to a |
| 445 | gimme card by sending a file card in its reply or in the next |
| 446 | message.</p> |
| 447 | |
| 448 | <h4>3.7.1 Unversioned Gimme Cards</h4> |
| 449 | |
| 450 | <p>Sync synchronizing unversioned content, the client may send "uvgimme" |
| 451 | cards to the server. A uvgimme card requests that the server send |
| 452 | unversioned content to the client. The format of a uvgimme card is |
| 453 | as follows: |
| 454 | |
| 455 | <blockquote> |
| 456 | <b>uvgimme</b> <i>name</i> |
| 457 | </blockquote> |
| 458 | |
| 459 | <p>The <i>name</i> is the name of the unversioned file found on the |
| 460 | server that the client would like to have. When a server sees a |
| 461 | uvgimme card, it normally responses with a uvfile card, though it might |
| 462 | also send anoter uvigot card if the HTTP reply is already oversized. |
| 463 | |
| 464 | <h3>3.8 Cookie Cards</h3> |
| 465 | |
| 466 | <p>A cookie card can be used by a server to record a small amount |
| 467 | of state information on a client. The server sends a cookie to the |
| 468 | client. The client sends the same cookie back to the server on |
| @@ -539,10 +624,49 @@ | |
| 624 | <p>The send-catalog pragma instructs the server to transmit igot |
| 625 | cards for every known artifact. This can help the client and server |
| 626 | to get back in synchronization after a prior protocol error. The |
| 627 | "--verily" option to the [/help?cmd=sync|fossil sync] command causes |
| 628 | the send-catalog pragma to be transmitted.</p> |
| 629 | |
| 630 | <li><p><b>uv-hash</b> <i>HASH</i> |
| 631 | <p>The uv-hash pragma is sent from client to server to provoke a |
| 632 | synchronization of unversioned content. The <i>HASH</i> is a SHA1 |
| 633 | hash of the names, modification times, and individual hashes of all |
| 634 | unversioned files on the client. If the unversioned content hash |
| 635 | from the client does not match the unversioned content hash on the |
| 636 | server, then the server will reply with either a "pragma uv-push-ok" |
| 637 | or "pragma uv-pull-only" card followed by one "uvigot" card for |
| 638 | each unversioned file currently held on the server. The collection |
| 639 | of "uvigot" cards sent in response to a "uv-hash" pragma is called |
| 640 | the "unversioned catalog". The client will used the unversioned |
| 641 | catalog to figure out which files (if any) need to be synchronized |
| 642 | between client and server and send appropriate "uvfile" or "uvgimme" |
| 643 | cards on the next HTTP request.</p> |
| 644 | |
| 645 | <p>If a client sends a uv-hash pragma and does not receive back |
| 646 | either a uv-pull-only or uv-push-ok pragma, that means that the |
| 647 | content on the server exactly matches the content on the client and |
| 648 | no further synchronization is required. |
| 649 | |
| 650 | <li><p><b>uv-pull-only</b></i> |
| 651 | <p>A server sends the uv-pull-only pragma to the client in response |
| 652 | to a uv-hash pragma with a mismatched content hash argument. This |
| 653 | pragma indicates that there are differences in unversioned content |
| 654 | between the client and server but that content can only be transfered |
| 655 | from server to client. The server is unwilling to accept content from |
| 656 | the client because the client login lacks the "write-unversioned" |
| 657 | permission.</p> |
| 658 | |
| 659 | <li><p><b>uv-push-ok</b></i> |
| 660 | <p>A server sends the uv-push-ok pragma to the client in response |
| 661 | to a uv-hash pragma with a mismatched content hash argument. This |
| 662 | pragma indicates that there are differences in unversioned content |
| 663 | between the client and server and that content can only be transfered |
| 664 | in either direction. The server is willing to accept content from |
| 665 | the client because the client login has the "write-unversioned" |
| 666 | permission.</p> |
| 667 | |
| 668 | </ol> |
| 669 | |
| 670 | <h3>3.12 Comment Cards</h3> |
| 671 | |
| 672 | <p>Any card that begins with "#" (ASCII 0x23) is a comment card and |
| @@ -713,10 +837,52 @@ | |
| 837 | The first three steps of a pull are combined with the first five steps |
| 838 | of a push. Steps (4) through (7) of a pull are combined with steps |
| 839 | (5) through (8) of a push. And steps (8) through (10) of a pull |
| 840 | are combined with step (9) of a push.</p> |
| 841 | |
| 842 | <h3>5.4 Unversioned File Sync</h3> |
| 843 | |
| 844 | <p>"Unversioned files" are files held in the repository |
| 845 | where only the most recent version of the file is kept rather than |
| 846 | the entire change history. Unversioned files are intended to be |
| 847 | used to store ephemeral content, such as compiled binaries of the |
| 848 | most recent release. |
| 849 | |
| 850 | <p>Unversioned files are identified by name and timestamp (mtime). |
| 851 | Only the most recent version of each file (the version with |
| 852 | the largest mtime value) is retained. |
| 853 | |
| 854 | <p>Unversioned files are synchronized using the |
| 855 | [/help?cmd=unversioned|fossil unversioned sync] command. |
| 856 | |
| 857 | <p>A schematic of an unversioned file synchronization is as follows: |
| 858 | |
| 859 | <ol> |
| 860 | <li>The client sends a "pragma uv-hash" card to the server. The argument |
| 861 | to the uv-hash pragma is a hash of all filesnames, mtimes, and |
| 862 | content hashes for the unversioned files held by the client. |
| 863 | <hr> |
| 864 | <li>If the unversion content hash from the client matches the unversioned |
| 865 | content hash on the server, then nothing needs to be done and the |
| 866 | server no-ops. But if the hashes are different, then the server |
| 867 | replies with either a uv-pull-only or a uv-push-ok pragma followed by |
| 868 | uvigot cards for all unversioned files held on the server. |
| 869 | <hr> |
| 870 | <li>The client examines the uvigot cards received from the server and |
| 871 | determines which unversioned files need to be exchanged in order |
| 872 | to bring the client and server into synchronization. The client |
| 873 | then sends appropriate "uvgimme" or "uvfile" cards back to the |
| 874 | server. |
| 875 | <hr> |
| 876 | <li>The server updates its unversioned file store with received "uvfile" |
| 877 | cards and answers "uvgimme" cards with "uvfile" cards in its reply. |
| 878 | </ol> |
| 879 | |
| 880 | <p>The last two steps might be repeated multiple |
| 881 | times if there is more unversioned content to be transferred than will |
| 882 | fit comfortably in a single HTTP request. |
| 883 | |
| 884 | <h2>6.0 Summary</h2> |
| 885 | |
| 886 | <p>Here are the key points of the synchronization protocol:</p> |
| 887 | |
| 888 | <ol> |
| @@ -734,13 +900,16 @@ | |
| 900 | <li> <b>clone_seqno</b> <i>sequence-number</i> |
| 901 | <li> <b>file</b> <i>artifact-id size</i> <b>\n</b> <i>content</i> |
| 902 | <li> <b>file</b> <i>artifact-id delta-artifact-id size</i> <b>\n</b> <i>content</i> |
| 903 | <li> <b>cfile</b> <i>artifact-id size</i> <b>\n</b> <i>content</i> |
| 904 | <li> <b>cfile</b> <i>artifact-id delta-artifact-id size</i> <b>\n</b> <i>content</i> |
| 905 | <li> <b>uvfile</b> <i>name mtime hash size flags</i> <b>\n</b> <i>content</i> |
| 906 | <li> <b>private</b> |
| 907 | <li> <b>igot</b> <i>artifact-id</i> ?<i>flag</i>? |
| 908 | <li> <b>uvigot</b> <i>name mtime hash size</i> |
| 909 | <li> <b>gimme</b> <i>artifact-id</i> |
| 910 | <li> <b>uvgimme</b> <i>name</i> |
| 911 | <li> <b>cookie</b> <i>cookie-text</i> |
| 912 | <li> <b>reqconfig</b> <i>parameter-name</i> |
| 913 | <li> <b>config</b> <i>parameter-name size</i> <b>\n</b> <i>content</i> |
| 914 | <li> <b>pragma</b> <i>name</i> <i>value...</i> |
| 915 | <li> <b>error</b> <i>error-message</i> |
| 916 |