Fossil SCM
Updates to the sync protocol documentation.
Commit
e57613dc4913fea5aa3c1a42348d0cd825c846b2cb62ddc82d08dcb90199fa29
Parent
2868a0ec4c6f854…
1 file changed
+27
-17
+27
-17
| --- www/sync.wiki | ||
| +++ www/sync.wiki | ||
| @@ -4,11 +4,12 @@ | ||
| 4 | 4 | content between two Fossil repositories. |
| 5 | 5 | |
| 6 | 6 | <h2>1.0 Overview</h2> |
| 7 | 7 | |
| 8 | 8 | The global state of a fossil repository consists of an unordered |
| 9 | -collection of artifacts. Each artifact is identified by a cryptographic | |
| 9 | +[./fileformat.wiki|collection of artifacts]. Each artifact is | |
| 10 | +identified by a cryptographic | |
| 10 | 11 | hash of its content, expressed as a lower-case hexadecimal string. |
| 11 | 12 | Synchronization is the process of sharing artifacts between |
| 12 | 13 | repositories so that all repositories have copies of all artifacts. Because |
| 13 | 14 | artifacts are unordered, the order in which artifacts are received |
| 14 | 15 | is unimportant. It is assumed that the hash names |
| @@ -17,11 +18,11 @@ | ||
| 17 | 18 | of hashes for available artifacts, then sharing the content of artifacts |
| 18 | 19 | whose names are missing from one side or the other of the connection. |
| 19 | 20 | In practice, a repository might contain millions of artifacts. The list of |
| 20 | 21 | hash names for this many artifacts can be large. So optimizations are |
| 21 | 22 | employed that usually reduce the number of hashes that need to be |
| 22 | -shared to a few hundred. | |
| 23 | +shared to a few dozen. | |
| 23 | 24 | |
| 24 | 25 | Each repository also has local state. The local state determines |
| 25 | 26 | the web-page formatting preferences, authorized users, ticket formats, |
| 26 | 27 | and similar information that varies from one repository to another. |
| 27 | 28 | The local state is not usually transferred during a sync. Except, |
| @@ -55,20 +56,24 @@ | ||
| 55 | 56 | The server is listening for incoming HTTP requests. The client |
| 56 | 57 | issues one or more HTTP requests and receives replies for each |
| 57 | 58 | request. |
| 58 | 59 | |
| 59 | 60 | The server might be running as an independent server |
| 60 | -using the <b>server</b> command, or it might be launched from | |
| 61 | -inetd or xinetd using the <b>http</b> command. Or the server might | |
| 62 | -be launched from CGI. | |
| 61 | +using the [/help?cmd=server|"fossil server" command], or it | |
| 62 | +might be launched from inetd or xinetd using the | |
| 63 | +["fossil http" command|/help?cmd=http]. Or the server might | |
| 64 | +be [./server/any/cgi.md|launched from CGI] or from | |
| 65 | +[./server/any/scgi.md|SCGI]. | |
| 63 | 66 | (See "[./server/|How To Configure A Fossil Server]" for details.) |
| 64 | 67 | The specifics of how the server listens |
| 65 | 68 | for incoming HTTP requests is immaterial to this protocol. |
| 66 | 69 | The important point is that the server is listening for requests and |
| 67 | 70 | the client is the issuer of the requests. |
| 68 | 71 | |
| 69 | -A single push, pull, or sync might involve multiple HTTP requests. | |
| 72 | +A single [/help?cmd=push|push], | |
| 73 | +[/help?cmd=pull|pull], or [/help?cmd=sync|sync] | |
| 74 | +might involve multiple HTTP requests. | |
| 70 | 75 | The client maintains state between all requests. But on the server |
| 71 | 76 | side, each request is independent. The server does not preserve |
| 72 | 77 | any information about the client from one request to the next. |
| 73 | 78 | |
| 74 | 79 | Note: Throughout this article, we use the terms "server" and "client" |
| @@ -79,16 +84,19 @@ | ||
| 79 | 84 | the repository that initiates the conversation and "server" is the repository |
| 80 | 85 | that responds. Nothing more. |
| 81 | 86 | |
| 82 | 87 | <h4>2.0.1 HTTPS Transport</h4> |
| 83 | 88 | |
| 84 | -In the current implementation of Fossil, the server only | |
| 85 | -understands HTTP requests. The client can send either | |
| 86 | -clear-text HTTP requests or encrypted HTTPS requests. But when | |
| 87 | -HTTPS requests are sent, they first must be decrypted by a web server | |
| 88 | -or proxy before being passed to the Fossil server. This limitation | |
| 89 | -may be relaxed in a future release. | |
| 89 | +HTTPS differs from HTTP only in that the HTTPS protocol is | |
| 90 | +encrypted as it travels over the wire. The underlying protocol | |
| 91 | +is the same. This document describes only the underlying, unencrypted | |
| 92 | +messages that go client to server and back to client. | |
| 93 | +Whether or not those messages are encrypted does not come into play | |
| 94 | +in this document. | |
| 95 | + | |
| 96 | +Fossil includes built-in | |
| 97 | +[./ssl-server.md|support for HTTPS encryption] in both client and server. | |
| 90 | 98 | |
| 91 | 99 | <h4>2.0.2 SSH Transport</h4> |
| 92 | 100 | |
| 93 | 101 | When doing a sync using an "ssh:..." URL, the same HTTP transport protocol |
| 94 | 102 | is used. Fossil simply uses [https://en.wikipedia.org/wiki/Secure_Shell|ssh] |
| @@ -212,12 +220,12 @@ | ||
| 212 | 220 | signature matches. If everything |
| 213 | 221 | checks out, then the client is granted all privileges of the |
| 214 | 222 | specified user. |
| 215 | 223 | |
| 216 | 224 | Privileges are cumulative. There can be multiple successful |
| 217 | -login cards. The session privileges are the bit-wise OR of the | |
| 218 | -privileges of each individual login. | |
| 225 | +login cards. The session privilege is the union of all | |
| 226 | +privileges from all login cards. | |
| 219 | 227 | |
| 220 | 228 | <h3>3.3 File Cards</h3> |
| 221 | 229 | |
| 222 | 230 | Artifacts are transferred using either "file" cards, or "cfile" |
| 223 | 231 | or "uvfile" cards. |
| @@ -229,11 +237,12 @@ | ||
| 229 | 237 | |
| 230 | 238 | <h4>3.3.1 Ordinary File Cards</h4> |
| 231 | 239 | |
| 232 | 240 | For sync protocols, artifacts are transferred using "file" |
| 233 | 241 | cards. File cards come in two different formats depending |
| 234 | -on whether the artifact is sent directly or as a delta from some | |
| 242 | +on whether the artifact is sent directly or as a | |
| 243 | +[../delta_format.wiki|delta] from some | |
| 235 | 244 | other artifact. |
| 236 | 245 | |
| 237 | 246 | <blockquote> |
| 238 | 247 | <b>file</b> <i>artifact-id size</i> <b>\n</b> <i>content</i><br> |
| 239 | 248 | <b>file</b> <i>artifact-id delta-artifact-id size</i> <b>\n</b> <i>content</i> |
| @@ -251,11 +260,12 @@ | ||
| 251 | 260 | representation of the name hash for the artifact. |
| 252 | 261 | The last argument of the file card is the number of bytes of |
| 253 | 262 | payload that immediately follow the file card. If the file |
| 254 | 263 | card has only two arguments, that means the payload is the |
| 255 | 264 | complete content of the artifact. If the file card has three |
| 256 | -arguments, then the payload is a delta and second argument is | |
| 265 | +arguments, then the payload is a | |
| 266 | +[../delta_format.wiki|delta] and second argument is | |
| 257 | 267 | the ID of another artifact that is the source of the delta. |
| 258 | 268 | |
| 259 | 269 | File cards are sent in both directions: client to server and |
| 260 | 270 | server to client. A delta might be sent before the source of |
| 261 | 271 | the delta, so both client and server should remember deltas |
| @@ -335,11 +345,11 @@ | ||
| 335 | 345 | size match the content and if the mtime is newer than any existing |
| 336 | 346 | instance of the same file held by the receiver. The sender will not |
| 337 | 347 | normally transmit a uvfile card unless all these constraints are true, |
| 338 | 348 | but the receiver should double-check. |
| 339 | 349 | |
| 340 | -A server should only accept uvfile cards if the login user has | |
| 350 | +A server will only accept uvfile cards if the login user has | |
| 341 | 351 | the "y" write-unversioned permission. |
| 342 | 352 | |
| 343 | 353 | Servers send uvfile cards in response to uvgimme cards received from |
| 344 | 354 | the client. Clients send uvfile cards when they determine that the server |
| 345 | 355 | needs the content based on uvigot cards previously received from the server. |
| 346 | 356 |
| --- www/sync.wiki | |
| +++ www/sync.wiki | |
| @@ -4,11 +4,12 @@ | |
| 4 | content between two Fossil repositories. |
| 5 | |
| 6 | <h2>1.0 Overview</h2> |
| 7 | |
| 8 | The global state of a fossil repository consists of an unordered |
| 9 | collection of artifacts. Each artifact is identified by a cryptographic |
| 10 | hash of its content, expressed as a lower-case hexadecimal string. |
| 11 | Synchronization is the process of sharing artifacts between |
| 12 | repositories so that all repositories have copies of all artifacts. Because |
| 13 | artifacts are unordered, the order in which artifacts are received |
| 14 | is unimportant. It is assumed that the hash names |
| @@ -17,11 +18,11 @@ | |
| 17 | of hashes for available artifacts, then sharing the content of artifacts |
| 18 | whose names are missing from one side or the other of the connection. |
| 19 | In practice, a repository might contain millions of artifacts. The list of |
| 20 | hash names for this many artifacts can be large. So optimizations are |
| 21 | employed that usually reduce the number of hashes that need to be |
| 22 | shared to a few hundred. |
| 23 | |
| 24 | Each repository also has local state. The local state determines |
| 25 | the web-page formatting preferences, authorized users, ticket formats, |
| 26 | and similar information that varies from one repository to another. |
| 27 | The local state is not usually transferred during a sync. Except, |
| @@ -55,20 +56,24 @@ | |
| 55 | The server is listening for incoming HTTP requests. The client |
| 56 | issues one or more HTTP requests and receives replies for each |
| 57 | request. |
| 58 | |
| 59 | The server might be running as an independent server |
| 60 | using the <b>server</b> command, or it might be launched from |
| 61 | inetd or xinetd using the <b>http</b> command. Or the server might |
| 62 | be launched from CGI. |
| 63 | (See "[./server/|How To Configure A Fossil Server]" for details.) |
| 64 | The specifics of how the server listens |
| 65 | for incoming HTTP requests is immaterial to this protocol. |
| 66 | The important point is that the server is listening for requests and |
| 67 | the client is the issuer of the requests. |
| 68 | |
| 69 | A single push, pull, or sync might involve multiple HTTP requests. |
| 70 | The client maintains state between all requests. But on the server |
| 71 | side, each request is independent. The server does not preserve |
| 72 | any information about the client from one request to the next. |
| 73 | |
| 74 | Note: Throughout this article, we use the terms "server" and "client" |
| @@ -79,16 +84,19 @@ | |
| 79 | the repository that initiates the conversation and "server" is the repository |
| 80 | that responds. Nothing more. |
| 81 | |
| 82 | <h4>2.0.1 HTTPS Transport</h4> |
| 83 | |
| 84 | In the current implementation of Fossil, the server only |
| 85 | understands HTTP requests. The client can send either |
| 86 | clear-text HTTP requests or encrypted HTTPS requests. But when |
| 87 | HTTPS requests are sent, they first must be decrypted by a web server |
| 88 | or proxy before being passed to the Fossil server. This limitation |
| 89 | may be relaxed in a future release. |
| 90 | |
| 91 | <h4>2.0.2 SSH Transport</h4> |
| 92 | |
| 93 | When doing a sync using an "ssh:..." URL, the same HTTP transport protocol |
| 94 | is used. Fossil simply uses [https://en.wikipedia.org/wiki/Secure_Shell|ssh] |
| @@ -212,12 +220,12 @@ | |
| 212 | signature matches. If everything |
| 213 | checks out, then the client is granted all privileges of the |
| 214 | specified user. |
| 215 | |
| 216 | Privileges are cumulative. There can be multiple successful |
| 217 | login cards. The session privileges are the bit-wise OR of the |
| 218 | privileges of each individual login. |
| 219 | |
| 220 | <h3>3.3 File Cards</h3> |
| 221 | |
| 222 | Artifacts are transferred using either "file" cards, or "cfile" |
| 223 | or "uvfile" cards. |
| @@ -229,11 +237,12 @@ | |
| 229 | |
| 230 | <h4>3.3.1 Ordinary File Cards</h4> |
| 231 | |
| 232 | For sync protocols, artifacts are transferred using "file" |
| 233 | cards. File cards come in two different formats depending |
| 234 | on whether the artifact is sent directly or as a delta from some |
| 235 | other artifact. |
| 236 | |
| 237 | <blockquote> |
| 238 | <b>file</b> <i>artifact-id size</i> <b>\n</b> <i>content</i><br> |
| 239 | <b>file</b> <i>artifact-id delta-artifact-id size</i> <b>\n</b> <i>content</i> |
| @@ -251,11 +260,12 @@ | |
| 251 | representation of the name hash for the artifact. |
| 252 | The last argument of the file card is the number of bytes of |
| 253 | payload that immediately follow the file card. If the file |
| 254 | card has only two arguments, that means the payload is the |
| 255 | complete content of the artifact. If the file card has three |
| 256 | arguments, then the payload is a delta and second argument is |
| 257 | the ID of another artifact that is the source of the delta. |
| 258 | |
| 259 | File cards are sent in both directions: client to server and |
| 260 | server to client. A delta might be sent before the source of |
| 261 | the delta, so both client and server should remember deltas |
| @@ -335,11 +345,11 @@ | |
| 335 | size match the content and if the mtime is newer than any existing |
| 336 | instance of the same file held by the receiver. The sender will not |
| 337 | normally transmit a uvfile card unless all these constraints are true, |
| 338 | but the receiver should double-check. |
| 339 | |
| 340 | A server should only accept uvfile cards if the login user has |
| 341 | the "y" write-unversioned permission. |
| 342 | |
| 343 | Servers send uvfile cards in response to uvgimme cards received from |
| 344 | the client. Clients send uvfile cards when they determine that the server |
| 345 | needs the content based on uvigot cards previously received from the server. |
| 346 |
| --- www/sync.wiki | |
| +++ www/sync.wiki | |
| @@ -4,11 +4,12 @@ | |
| 4 | content between two Fossil repositories. |
| 5 | |
| 6 | <h2>1.0 Overview</h2> |
| 7 | |
| 8 | The global state of a fossil repository consists of an unordered |
| 9 | [./fileformat.wiki|collection of artifacts]. Each artifact is |
| 10 | identified by a cryptographic |
| 11 | hash of its content, expressed as a lower-case hexadecimal string. |
| 12 | Synchronization is the process of sharing artifacts between |
| 13 | repositories so that all repositories have copies of all artifacts. Because |
| 14 | artifacts are unordered, the order in which artifacts are received |
| 15 | is unimportant. It is assumed that the hash names |
| @@ -17,11 +18,11 @@ | |
| 18 | of hashes for available artifacts, then sharing the content of artifacts |
| 19 | whose names are missing from one side or the other of the connection. |
| 20 | In practice, a repository might contain millions of artifacts. The list of |
| 21 | hash names for this many artifacts can be large. So optimizations are |
| 22 | employed that usually reduce the number of hashes that need to be |
| 23 | shared to a few dozen. |
| 24 | |
| 25 | Each repository also has local state. The local state determines |
| 26 | the web-page formatting preferences, authorized users, ticket formats, |
| 27 | and similar information that varies from one repository to another. |
| 28 | The local state is not usually transferred during a sync. Except, |
| @@ -55,20 +56,24 @@ | |
| 56 | The server is listening for incoming HTTP requests. The client |
| 57 | issues one or more HTTP requests and receives replies for each |
| 58 | request. |
| 59 | |
| 60 | The server might be running as an independent server |
| 61 | using the [/help?cmd=server|"fossil server" command], or it |
| 62 | might be launched from inetd or xinetd using the |
| 63 | ["fossil http" command|/help?cmd=http]. Or the server might |
| 64 | be [./server/any/cgi.md|launched from CGI] or from |
| 65 | [./server/any/scgi.md|SCGI]. |
| 66 | (See "[./server/|How To Configure A Fossil Server]" for details.) |
| 67 | The specifics of how the server listens |
| 68 | for incoming HTTP requests is immaterial to this protocol. |
| 69 | The important point is that the server is listening for requests and |
| 70 | the client is the issuer of the requests. |
| 71 | |
| 72 | A single [/help?cmd=push|push], |
| 73 | [/help?cmd=pull|pull], or [/help?cmd=sync|sync] |
| 74 | might involve multiple HTTP requests. |
| 75 | The client maintains state between all requests. But on the server |
| 76 | side, each request is independent. The server does not preserve |
| 77 | any information about the client from one request to the next. |
| 78 | |
| 79 | Note: Throughout this article, we use the terms "server" and "client" |
| @@ -79,16 +84,19 @@ | |
| 84 | the repository that initiates the conversation and "server" is the repository |
| 85 | that responds. Nothing more. |
| 86 | |
| 87 | <h4>2.0.1 HTTPS Transport</h4> |
| 88 | |
| 89 | HTTPS differs from HTTP only in that the HTTPS protocol is |
| 90 | encrypted as it travels over the wire. The underlying protocol |
| 91 | is the same. This document describes only the underlying, unencrypted |
| 92 | messages that go client to server and back to client. |
| 93 | Whether or not those messages are encrypted does not come into play |
| 94 | in this document. |
| 95 | |
| 96 | Fossil includes built-in |
| 97 | [./ssl-server.md|support for HTTPS encryption] in both client and server. |
| 98 | |
| 99 | <h4>2.0.2 SSH Transport</h4> |
| 100 | |
| 101 | When doing a sync using an "ssh:..." URL, the same HTTP transport protocol |
| 102 | is used. Fossil simply uses [https://en.wikipedia.org/wiki/Secure_Shell|ssh] |
| @@ -212,12 +220,12 @@ | |
| 220 | signature matches. If everything |
| 221 | checks out, then the client is granted all privileges of the |
| 222 | specified user. |
| 223 | |
| 224 | Privileges are cumulative. There can be multiple successful |
| 225 | login cards. The session privilege is the union of all |
| 226 | privileges from all login cards. |
| 227 | |
| 228 | <h3>3.3 File Cards</h3> |
| 229 | |
| 230 | Artifacts are transferred using either "file" cards, or "cfile" |
| 231 | or "uvfile" cards. |
| @@ -229,11 +237,12 @@ | |
| 237 | |
| 238 | <h4>3.3.1 Ordinary File Cards</h4> |
| 239 | |
| 240 | For sync protocols, artifacts are transferred using "file" |
| 241 | cards. File cards come in two different formats depending |
| 242 | on whether the artifact is sent directly or as a |
| 243 | [../delta_format.wiki|delta] from some |
| 244 | other artifact. |
| 245 | |
| 246 | <blockquote> |
| 247 | <b>file</b> <i>artifact-id size</i> <b>\n</b> <i>content</i><br> |
| 248 | <b>file</b> <i>artifact-id delta-artifact-id size</i> <b>\n</b> <i>content</i> |
| @@ -251,11 +260,12 @@ | |
| 260 | representation of the name hash for the artifact. |
| 261 | The last argument of the file card is the number of bytes of |
| 262 | payload that immediately follow the file card. If the file |
| 263 | card has only two arguments, that means the payload is the |
| 264 | complete content of the artifact. If the file card has three |
| 265 | arguments, then the payload is a |
| 266 | [../delta_format.wiki|delta] and second argument is |
| 267 | the ID of another artifact that is the source of the delta. |
| 268 | |
| 269 | File cards are sent in both directions: client to server and |
| 270 | server to client. A delta might be sent before the source of |
| 271 | the delta, so both client and server should remember deltas |
| @@ -335,11 +345,11 @@ | |
| 345 | size match the content and if the mtime is newer than any existing |
| 346 | instance of the same file held by the receiver. The sender will not |
| 347 | normally transmit a uvfile card unless all these constraints are true, |
| 348 | but the receiver should double-check. |
| 349 | |
| 350 | A server will only accept uvfile cards if the login user has |
| 351 | the "y" write-unversioned permission. |
| 352 | |
| 353 | Servers send uvfile cards in response to uvgimme cards received from |
| 354 | the client. Clients send uvfile cards when they determine that the server |
| 355 | needs the content based on uvigot cards previously received from the server. |
| 356 |