Fossil SCM

Replaced the Platform Quirks section of the www/globs.md document with a modified version of what I (Warren Young) posted to the mailing list, the differences answering Ross Berteig critiques of the original version.

tangent 2017-05-01 13:34 glob-docs
Commit 0a51f1bf6a6530edfd307f9fac117e1868f74192e091b639c0b40e2856cf5008
1 file changed +184 -62
+184 -62
--- www/globs.md
+++ www/globs.md
@@ -113,11 +113,11 @@
113113
name to be considered a match.
114114
115115
The canonical name of a file has all directory separators changed to
116116
`/`, redundant slashes are removed, all `.` path components are
117117
removed, and all `..` path components are resolved. (There are
118
-additional details we won’t go into here.)
118
+additional details we won't go into here.)
119119
120120
The goal is a name that is the simplest possible for each particular
121121
file, and will be the same on Windows, Unix, and any other platform
122122
where fossil is run.
123123
@@ -242,71 +242,193 @@
242242
[`/timeline`]: /help?cmd=/timeline
243243
[`/tarball`]: /help?cmd=/tarball
244244
[`/zip`]: /help?cmd=/zip
245245
246246
247
-## Platform quirks
248
-
249
-The versioned settings files have no platform-specific quirks. Any
250
-GLOBs that matter to your workflow belong there where they can be
251
-safely edited.
252
-
253
-Similarly, settings made through the Web UI are platform independent.
254
-
255
-GLOBs at the command prompt, however, may need to be protected from
256
-the quirks of the particular shell program you use to type the
257
-command.
258
-
259
-The GLOB language is based on common features of Unix (and Linux)
260
-shells. In some cases, this will cause confusion if the shell expands
261
-the GLOB in a way that is similar to what fossil would have done.
262
-
263
-When in doubt, the `fossil test-glob` command can be used to see what
264
-fossil saw and what it chose to do. The `fossil test-echo` command is
265
-also handy: it shows exactly what arguments fossil received.
266
-
267
-### Windows
268
-
269
-Various versions of Windows (a phrase that covers more than just
270
-Window 7 vs Windows 10 because the actual content of `MSVCRT.DLL`, other
271
-DLLs, and even the specific compiler used to build `fossil.exe` can
272
-change the behavior) have subtle differences in how quoting works.
273
-
274
-Even without subtle version changes, there are also differences
275
-between the interactive command prompt and `.BAT` or `.CMD` files.
276
-
277
-The typical problem is figuring out how to get a GLOB passed on the
278
-command line into `fossil.exe` without it being expanded by either the
279
-shell (CMD never expands globs so that part is trivial) or by the C
280
-runtime startup (which tries hard to expand globs to act like Unix). A
281
-typical example is figuring out how to set `crlf-glob` to `*`.
282
-
283
-One approach is
284
-
285
- echo * | fossil setting crlf-glob --args -
286
-
287
-which works because the built-in command `echo` does not expand its
288
-arguments, and the global option [--args][] reads from `-` which is
289
-replaced by standard input pipe from the `echo` command.
290
-
291
-[--args]: https://www.fossil-scm.org/index.html/doc/trunk/www/env-opts.md
292
-
293
-Another approach is
294
-
295
- fossil setting crlf-glob *,
296
-
297
-which, despite including the extra comma in the stored setting value,
298
-has the desired effect. The empty GLOB after the comma matches no
299
-files at all, which has no effect since the `*` matches them all.
300
-
301
-Similarly,
302
-
303
- fossil setting crlf-glob '*'
304
-
305
-also works. Here the single quotes are unneeded since no white space
306
-is mentioned in the pattern, but do no harm. The GLOB still matches
307
-all the files.
247
+## Platform Quirks
248
+
249
+Fossil glob patterns are based on the glob pattern feature of POSIX
250
+shells. Fossil glob patterns also have a quoting mechanism, discussed
251
+above. Because other parts of your operating system may interpret glob
252
+patterns and quotes separately from Fossil, it is often difficult to
253
+give glob patterns correctly to Fossil on the command line. Quotes and
254
+special characters in glob patterns are likely to interpreted when given
255
+as part of a `fossil` command, causing unexpected behavior.
256
+
257
+These problems do not affect [versioned settings
258
+files](/doc/trunk/www/settings.wiki) or Admin → Settings in Fossil
259
+UI. Consequently, it is better to set long-term `*-glob` settings via
260
+these methods than to use `fossil settings` commands.
261
+
262
+That advice doesn't help you when you are giving one-off glob patterns
263
+in `fossil` commands. The remainder of this section gives remedies and
264
+workarounds for these problems.
265
+
266
+
267
+## POSIX Systems
268
+
269
+If you are using Fossil on a system with a POSIX-compatible shell
270
+— Linux, macOS, the BSDs, Unix, Cygwin, WSL etc. — the shell
271
+may expand the glob patterns before passing the result to the `fossil`
272
+executable.
273
+
274
+Sometimes this is exactly what you want. Consider this command for
275
+example:
276
+
277
+ $ fossil add RE*
278
+
279
+If you give that command in a directory containing `README.txt` and
280
+`RELEASE-NOTES.txt`, the shell will expand the command to:
281
+
282
+ $ fossil add README.txt RELEASE-NOTES.txt
283
+
284
+…which is compatible with the `fossil add` command's argument list,
285
+which allows multiple files.
286
+
287
+Now consider what happens instead if you say:
288
+
289
+ $ fossil add --ignore RE* src/*.c
290
+
291
+This *doesn't* do what you want because the shell will expand both `RE*`
292
+and `src/*.c`, causing one of the two files matching the `RE*` glob
293
+pattern to be ignored and the other to be added to the repository. You
294
+need to say this in that case:
295
+
296
+ $ fossil add --ignore 'RE*' src/*.c
297
+
298
+The single quotes force a POSIX shell to pass the `RE*` glob pattern
299
+through to Fossil untouched, which will do its own glob pattern
300
+matching. There are other methods of quoting a glob pattern or escaping
301
+its special characters; see your shell's manual.
302
+
303
+Beware that Fossil's `--ignore` option doesn't override explicit file
304
+mentions:
305
+
306
+ $ fossil add --ignore 'REALLY SECRET STUFF.txt' RE*
307
+
308
+You might think that would add everything beginning with `RE` *except*
309
+for `REALLY SECRET STUFF.txt`, but Fossil when a file is given
310
+explicitly and found in the ignore list, Fossil asks what you want to do
311
+with it in the default case, and doesn't even ask if gave `-f` or
312
+`--force` along with `--ignore`.
313
+
314
+The spaces in the ignored file name above bring us to another point:
315
+file names must be quoted in Fossil glob patterns, but the shell
316
+interprets quotation marks itself. There are a couple of ways to fix
317
+both this and the previous problem:
318
+
319
+ $ fossil add --ignore "'REALLY SECRET STUFF.txt'" READ*
320
+
321
+The nested quotation marks cause the inner set to be passed through to
322
+Fossil, and the more specific glob pattern expanded by the shell (that
323
+is, `READ*` vs `RE*`) avoids a conflict between explicitly-listed files
324
+and `--ignore` rules in the `fossil add` command.
325
+
326
+Another solution would be to use shell escaping instead of nested
327
+quoting:
328
+
329
+ $ fossil add --ignore "\"REALLY SECRET STUFF.txt\"" READ*
330
+
331
+It bears repeating that the two glob patterns here are not interpreted
332
+the same way when running this command from a *subdirectory* of the top
333
+checkout directory as when running it at the top of the checkout tree.
334
+If these files were in a subdirectory of the checkout tree called `doc`
335
+and that was your current working directory, the command would have to
336
+be:
337
+
338
+ $ fossil add --ignore "'doc/REALLY SECRET STUFF.txt'" READ*
339
+
340
+instead. The Fossil glob pattern still needs the `doc/` prefix because
341
+Fossil always interprets glob patterns from the base of the checkout
342
+directory, not from the current working directory as POSIX shells do.
343
+
344
+When in doubt, use `fossil status` after running commands like the above
345
+to make sure the right set of files were scheduled for insertion into
346
+the repository before checking the changes in. You wouldn't want to
347
+accidentally check something like a password, an API key, or the private
348
+half of a public crypto key into Fossil repository that can be read by
349
+people who should not have such secrets.
350
+
351
+
352
+## Windows
353
+
354
+Neither standard Windows command shell — `cmd.exe` or PowerShell
355
+— expands glob patterns the way POSIX shells do. Windows command
356
+shells rely on the command itself to do the glob pattern expansion. The
357
+way this works depends on several factors:
358
+
359
+* the version of Windows you're using
360
+* which OS upgrades have been applied to it
361
+* the compiler that built your Fossil executable
362
+* whether you're running the command interactively
363
+* whether the command is built against a runtime system that does this
364
+ at all
365
+* whether the Fossil command is being run from a file named `*.BAT` vs
366
+ being named `*.CMD`
367
+
368
+These factors also affect how a program like `fossil.exe` interprets
369
+quotation marks on its command line.
370
+
371
+The fifth item above doesn't apply to `fossil.exe` when built with
372
+typical tool chains, but we'll see an example below where the exception
373
+applies in a way that affects how Fossil interprets the glob pattern.
374
+
375
+The most common problem is figuring out how to get a glob pattern passed
376
+on the command line into `fossil.exe` without it being expanded by the C
377
+runtime library that your particular Fossil executable is linked to,
378
+which tries to act like the POSIX systems described above. Windows is
379
+not strongly governed by POSIX, so it has not historically hewed closely
380
+to its strictures.
381
+
382
+(This section does not cover the [Microsoft POSIX
383
+subsystem](https://en.wikipedia.org/wiki/Microsoft_POSIX_subsystem),
384
+Windows' obsolete [Services for Unix
385
+3.*x*](https://en.wikipedia.org/wiki/Windows_Services_for_UNIX) feature,
386
+or the [Windows Subsystem for
387
+Linux](https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux). (The
388
+latter is sometimes incorrectly called "Bash on Windows" or "Ubuntu on
389
+Windows.") See the POSIX Systems section above for those cases.)
390
+
391
+For example, consider how you would set `crlf-glob` to `*`. The
392
+naïve approach will not work:
393
+
394
+ C:\...> fossil setting crlf-glob *
395
+
396
+The C runtime library will expand that to the list of all files in the
397
+current directory, which will probably cause a Fossil error because
398
+Fossil expects either nothing or option flags after the setting's new
399
+value.
400
+
401
+Let's try again:
402
+
403
+ C:\...> fossil setting crlf-glob '*'
404
+
405
+That may or may not work. Either `'*'` or `*` needs to be passed through
406
+to Fossil untouched for this to do what you expect, which may or may not
407
+happen, depending on the factors listed above.
408
+
409
+An approach that *will* work reliably is:
410
+
411
+ C:\...> echo * | fossil setting crlf-glob --args -
412
+
413
+This works because the built-in command `echo` does not expand its
414
+arguments, and the `--args -` option makes it read further command
415
+arguments from Fossil's standard input, which is connected to the output
416
+of `echo` by the pipe. (`-` is a common Unix convention meaning
417
+"standard input.")
418
+
419
+Another correct approach is:
420
+
421
+ C:\...> fossil setting crlf-glob *,
422
+
423
+This works because the trailing comma prevents the command shell from
424
+matching any files, unless you happen to have files named with a
425
+trailing comma in the current directory. If the pattern matches no
426
+files, it is passed into Fossil's `main()` function as-is by the C
427
+runtime system. Since Fossil uses commas to separate multiple glob
428
+patterns, this means "all files at the root of the Fossil checkout
429
+directory and nothing else."
308430
309431
310432
## Converting `.gitignore` to `ignore-glob`
311433
312434
Many other version control systems handle the specific case of
313435
--- www/globs.md
+++ www/globs.md
@@ -113,11 +113,11 @@
113 name to be considered a match.
114
115 The canonical name of a file has all directory separators changed to
116 `/`, redundant slashes are removed, all `.` path components are
117 removed, and all `..` path components are resolved. (There are
118 additional details we won’t go into here.)
119
120 The goal is a name that is the simplest possible for each particular
121 file, and will be the same on Windows, Unix, and any other platform
122 where fossil is run.
123
@@ -242,71 +242,193 @@
242 [`/timeline`]: /help?cmd=/timeline
243 [`/tarball`]: /help?cmd=/tarball
244 [`/zip`]: /help?cmd=/zip
245
246
247 ## Platform quirks
248
249 The versioned settings files have no platform-specific quirks. Any
250 GLOBs that matter to your workflow belong there where they can be
251 safely edited.
252
253 Similarly, settings made through the Web UI are platform independent.
254
255 GLOBs at the command prompt, however, may need to be protected from
256 the quirks of the particular shell program you use to type the
257 command.
258
259 The GLOB language is based on common features of Unix (and Linux)
260 shells. In some cases, this will cause confusion if the shell expands
261 the GLOB in a way that is similar to what fossil would have done.
262
263 When in doubt, the `fossil test-glob` command can be used to see what
264 fossil saw and what it chose to do. The `fossil test-echo` command is
265 also handy: it shows exactly what arguments fossil received.
266
267 ### Windows
268
269 Various versions of Windows (a phrase that covers more than just
270 Window 7 vs Windows 10 because the actual content of `MSVCRT.DLL`, other
271 DLLs, and even the specific compiler used to build `fossil.exe` can
272 change the behavior) have subtle differences in how quoting works.
273
274 Even without subtle version changes, there are also differences
275 between the interactive command prompt and `.BAT` or `.CMD` files.
276
277 The typical problem is figuring out how to get a GLOB passed on the
278 command line into `fossil.exe` without it being expanded by either the
279 shell (CMD never expands globs so that part is trivial) or by the C
280 runtime startup (which tries hard to expand globs to act like Unix). A
281 typical example is figuring out how to set `crlf-glob` to `*`.
282
283 One approach is
284
285 echo * | fossil setting crlf-glob --args -
286
287 which works because the built-in command `echo` does not expand its
288 arguments, and the global option [--args][] reads from `-` which is
289 replaced by standard input pipe from the `echo` command.
290
291 [--args]: https://www.fossil-scm.org/index.html/doc/trunk/www/env-opts.md
292
293 Another approach is
294
295 fossil setting crlf-glob *,
296
297 which, despite including the extra comma in the stored setting value,
298 has the desired effect. The empty GLOB after the comma matches no
299 files at all, which has no effect since the `*` matches them all.
300
301 Similarly,
302
303 fossil setting crlf-glob '*'
304
305 also works. Here the single quotes are unneeded since no white space
306 is mentioned in the pattern, but do no harm. The GLOB still matches
307 all the files.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
308
309
310 ## Converting `.gitignore` to `ignore-glob`
311
312 Many other version control systems handle the specific case of
313
--- www/globs.md
+++ www/globs.md
@@ -113,11 +113,11 @@
113 name to be considered a match.
114
115 The canonical name of a file has all directory separators changed to
116 `/`, redundant slashes are removed, all `.` path components are
117 removed, and all `..` path components are resolved. (There are
118 additional details we won't go into here.)
119
120 The goal is a name that is the simplest possible for each particular
121 file, and will be the same on Windows, Unix, and any other platform
122 where fossil is run.
123
@@ -242,71 +242,193 @@
242 [`/timeline`]: /help?cmd=/timeline
243 [`/tarball`]: /help?cmd=/tarball
244 [`/zip`]: /help?cmd=/zip
245
246
247 ## Platform Quirks
248
249 Fossil glob patterns are based on the glob pattern feature of POSIX
250 shells. Fossil glob patterns also have a quoting mechanism, discussed
251 above. Because other parts of your operating system may interpret glob
252 patterns and quotes separately from Fossil, it is often difficult to
253 give glob patterns correctly to Fossil on the command line. Quotes and
254 special characters in glob patterns are likely to interpreted when given
255 as part of a `fossil` command, causing unexpected behavior.
256
257 These problems do not affect [versioned settings
258 files](/doc/trunk/www/settings.wiki) or Admin → Settings in Fossil
259 UI. Consequently, it is better to set long-term `*-glob` settings via
260 these methods than to use `fossil settings` commands.
261
262 That advice doesn't help you when you are giving one-off glob patterns
263 in `fossil` commands. The remainder of this section gives remedies and
264 workarounds for these problems.
265
266
267 ## POSIX Systems
268
269 If you are using Fossil on a system with a POSIX-compatible shell
270 — Linux, macOS, the BSDs, Unix, Cygwin, WSL etc. — the shell
271 may expand the glob patterns before passing the result to the `fossil`
272 executable.
273
274 Sometimes this is exactly what you want. Consider this command for
275 example:
276
277 $ fossil add RE*
278
279 If you give that command in a directory containing `README.txt` and
280 `RELEASE-NOTES.txt`, the shell will expand the command to:
281
282 $ fossil add README.txt RELEASE-NOTES.txt
283
284 …which is compatible with the `fossil add` command's argument list,
285 which allows multiple files.
286
287 Now consider what happens instead if you say:
288
289 $ fossil add --ignore RE* src/*.c
290
291 This *doesn't* do what you want because the shell will expand both `RE*`
292 and `src/*.c`, causing one of the two files matching the `RE*` glob
293 pattern to be ignored and the other to be added to the repository. You
294 need to say this in that case:
295
296 $ fossil add --ignore 'RE*' src/*.c
297
298 The single quotes force a POSIX shell to pass the `RE*` glob pattern
299 through to Fossil untouched, which will do its own glob pattern
300 matching. There are other methods of quoting a glob pattern or escaping
301 its special characters; see your shell's manual.
302
303 Beware that Fossil's `--ignore` option doesn't override explicit file
304 mentions:
305
306 $ fossil add --ignore 'REALLY SECRET STUFF.txt' RE*
307
308 You might think that would add everything beginning with `RE` *except*
309 for `REALLY SECRET STUFF.txt`, but Fossil when a file is given
310 explicitly and found in the ignore list, Fossil asks what you want to do
311 with it in the default case, and doesn't even ask if gave `-f` or
312 `--force` along with `--ignore`.
313
314 The spaces in the ignored file name above bring us to another point:
315 file names must be quoted in Fossil glob patterns, but the shell
316 interprets quotation marks itself. There are a couple of ways to fix
317 both this and the previous problem:
318
319 $ fossil add --ignore "'REALLY SECRET STUFF.txt'" READ*
320
321 The nested quotation marks cause the inner set to be passed through to
322 Fossil, and the more specific glob pattern expanded by the shell (that
323 is, `READ*` vs `RE*`) avoids a conflict between explicitly-listed files
324 and `--ignore` rules in the `fossil add` command.
325
326 Another solution would be to use shell escaping instead of nested
327 quoting:
328
329 $ fossil add --ignore "\"REALLY SECRET STUFF.txt\"" READ*
330
331 It bears repeating that the two glob patterns here are not interpreted
332 the same way when running this command from a *subdirectory* of the top
333 checkout directory as when running it at the top of the checkout tree.
334 If these files were in a subdirectory of the checkout tree called `doc`
335 and that was your current working directory, the command would have to
336 be:
337
338 $ fossil add --ignore "'doc/REALLY SECRET STUFF.txt'" READ*
339
340 instead. The Fossil glob pattern still needs the `doc/` prefix because
341 Fossil always interprets glob patterns from the base of the checkout
342 directory, not from the current working directory as POSIX shells do.
343
344 When in doubt, use `fossil status` after running commands like the above
345 to make sure the right set of files were scheduled for insertion into
346 the repository before checking the changes in. You wouldn't want to
347 accidentally check something like a password, an API key, or the private
348 half of a public crypto key into Fossil repository that can be read by
349 people who should not have such secrets.
350
351
352 ## Windows
353
354 Neither standard Windows command shell — `cmd.exe` or PowerShell
355 — expands glob patterns the way POSIX shells do. Windows command
356 shells rely on the command itself to do the glob pattern expansion. The
357 way this works depends on several factors:
358
359 * the version of Windows you're using
360 * which OS upgrades have been applied to it
361 * the compiler that built your Fossil executable
362 * whether you're running the command interactively
363 * whether the command is built against a runtime system that does this
364 at all
365 * whether the Fossil command is being run from a file named `*.BAT` vs
366 being named `*.CMD`
367
368 These factors also affect how a program like `fossil.exe` interprets
369 quotation marks on its command line.
370
371 The fifth item above doesn't apply to `fossil.exe` when built with
372 typical tool chains, but we'll see an example below where the exception
373 applies in a way that affects how Fossil interprets the glob pattern.
374
375 The most common problem is figuring out how to get a glob pattern passed
376 on the command line into `fossil.exe` without it being expanded by the C
377 runtime library that your particular Fossil executable is linked to,
378 which tries to act like the POSIX systems described above. Windows is
379 not strongly governed by POSIX, so it has not historically hewed closely
380 to its strictures.
381
382 (This section does not cover the [Microsoft POSIX
383 subsystem](https://en.wikipedia.org/wiki/Microsoft_POSIX_subsystem),
384 Windows' obsolete [Services for Unix
385 3.*x*](https://en.wikipedia.org/wiki/Windows_Services_for_UNIX) feature,
386 or the [Windows Subsystem for
387 Linux](https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux). (The
388 latter is sometimes incorrectly called "Bash on Windows" or "Ubuntu on
389 Windows.") See the POSIX Systems section above for those cases.)
390
391 For example, consider how you would set `crlf-glob` to `*`. The
392 naïve approach will not work:
393
394 C:\...> fossil setting crlf-glob *
395
396 The C runtime library will expand that to the list of all files in the
397 current directory, which will probably cause a Fossil error because
398 Fossil expects either nothing or option flags after the setting's new
399 value.
400
401 Let's try again:
402
403 C:\...> fossil setting crlf-glob '*'
404
405 That may or may not work. Either `'*'` or `*` needs to be passed through
406 to Fossil untouched for this to do what you expect, which may or may not
407 happen, depending on the factors listed above.
408
409 An approach that *will* work reliably is:
410
411 C:\...> echo * | fossil setting crlf-glob --args -
412
413 This works because the built-in command `echo` does not expand its
414 arguments, and the `--args -` option makes it read further command
415 arguments from Fossil's standard input, which is connected to the output
416 of `echo` by the pipe. (`-` is a common Unix convention meaning
417 "standard input.")
418
419 Another correct approach is:
420
421 C:\...> fossil setting crlf-glob *,
422
423 This works because the trailing comma prevents the command shell from
424 matching any files, unless you happen to have files named with a
425 trailing comma in the current directory. If the pattern matches no
426 files, it is passed into Fossil's `main()` function as-is by the C
427 runtime system. Since Fossil uses commas to separate multiple glob
428 patterns, this means "all files at the root of the Fossil checkout
429 directory and nothing else."
430
431
432 ## Converting `.gitignore` to `ignore-glob`
433
434 Many other version control systems handle the specific case of
435

Keyboard Shortcuts

Open search /
Next entry (timeline) j
Previous entry (timeline) k
Open focused entry Enter
Show this help ?
Toggle theme Top nav button