Fossil SCM

fossil-scm / www / wsl_caveats.wiki
Source Blame History 176 lines
73ba763… larrybr 1 <title>Caveats and Precautions for Fossil Usage with Windows Subsystem for Linux</title>
73ba763… larrybr 2
73ba763… larrybr 3 <h2>When These Issues Matter</h2>
73ba763… larrybr 4
73ba763… larrybr 5 The discussion following is relevant to those who:
73ba763… larrybr 6
ea6b2d3… larrybr 7 * Are using the Windows Subsystem for Linux (aka "WSL");
ea6b2d3… larrybr 8 * Create a Fossil checkout in a directory accessible from WSL and Windows;
ea6b2d3… larrybr 9 * Use both Linux Fossil and Windows tools to modify files in a checkout;
ea6b2d3… larrybr 10 * Desire to or must preserve execute permissions set for repository files;
ea6b2d3… larrybr 11 * and Use Linux Fossil to commit changes made within the checkout.
73ba763… larrybr 12
73ba763… larrybr 13 Note that these criteria apply conjunctively; if any are not met,
73ba763… larrybr 14 then the consequences of the issues below are at worst annoying
73ba763… larrybr 15 and otherwise harmless or absent.
73ba763… larrybr 16
73ba763… larrybr 17 <h2>What Can Go Wrong (Why It Matters)</h2>
73ba763… larrybr 18
73ba763… larrybr 19 The most readily seen manifestation of the problem occurs when
73ba763… larrybr 20 "<tt>fossil status</tt>" or "<tt>fossil changes</tt>" is run,
73ba763… larrybr 21 using Linux Fossil from WSL after Windows tools (including
73ba763… larrybr 22 fossil.exe) have been used to modify files within a checkout.
73ba763… larrybr 23 Unless filter options block it, those subcommands will tag
73ba763… larrybr 24 some (and often many) checkout files with <b>EXECUTABLE</b>
73ba763… larrybr 25 or <b>NOEXEC</b>, indicating that the file's user execute permission
73ba763… larrybr 26 has been altered such that it differs from what is recorded
73ba763… larrybr 27 in the repository for that version of the file.
73ba763… larrybr 28
73ba763… larrybr 29 This "user execute permission" is referred to as "the X-bit"
73ba763… larrybr 30 hereafter, referring to either the recorded version state
73ba763… larrybr 31 or a checkout file attributes state.
73ba763… larrybr 32
73ba763… larrybr 33 This is merely annoying and distracting if the altered X-bit
73ba763… larrybr 34 will never be committed using Linux Fossil. It can be quite
73ba763… larrybr 35 distracting because those tags tend to mask the presence or
73ba763… larrybr 36 absence of other changes whose detection is the usual
73ba763… larrybr 37 reason for using Fossil's changes or status subcommands.
73ba763… larrybr 38
73ba763… larrybr 39 However, in the problematic usage scenario, those tags will
73ba763… larrybr 40 most often represent inadvertant toggling of the X-bit on the
73ba763… larrybr 41 affected file. The X-bit is kept in the repository for good
73ba763… larrybr 42 reason (usually), and arbitrary changes to it by means of a
73ba763… larrybr 43 commit when that change is not intended is virtually always
73ba763… larrybr 44 a bad result. (At best, the change causes useless churn; at
73ba763… larrybr 45 worst it frustrates the intended purpose of having an X-bit.)
73ba763… larrybr 46
73ba763… larrybr 47 <h2>Technical Discusion of the Problem</h2>
73ba763… larrybr 48
73ba763… larrybr 49 The genesis of altered X-bits, while not obvious at first glance,
73ba763… larrybr 50 involves obvious facts. The Windows OS does not deal with the
73ba763… larrybr 51 triple of user/group/other executable permissions the way that
73ba763… larrybr 52 Unix and similar operating systems do. Hence, tools which run
73ba763… larrybr 53 on Windows, including Fossil built for Windows, do not manage
73ba763… larrybr 54 the X-bit; it may not even exist yet for files which have not
73ba763… larrybr 55 had their permissions set by any Linux program running in WSL.
73ba763… larrybr 56 When such tools modify a file which has had its X-bit set (or
73ba763… larrybr 57 cleared) by a program in WSL, an existing X-bit value may not
73ba763… larrybr 58 be preserved depending upon how the modification is effected.
73ba763… larrybr 59
73ba763… larrybr 60 The WSL infrastructure (or virtual system) compensates for the
73ba763… larrybr 61 absence of an X-bit in Windows filesystems with two stratagems:
73ba763… larrybr 62 (1) Establishing a default for its value when no Linux program
73ba763… larrybr 63 has yet set it; and (2) stashing Linux "mode" bits in a "special"
73ba763… larrybr 64 place for each file once it has been subject to a chmod() call.
73ba763… larrybr 65 That default's default can be changed by way of /etc/wsl.conf
73ba763… larrybr 66 content. However, this default cannot be right for files which
73ba763… larrybr 67 are tracked in a Fossil repository as having the other value.
73ba763… larrybr 68 And Windows tools generally are not written to deal with "mode"
73ba763… larrybr 69 bits in that "special" place. (They are kept in a NTFS extended
73ba763… larrybr 70 file attribute named $LXMOD, not accessible through the WIN32
73ba763… larrybr 71 API; the OS layer below WIN32 must be used to get at them.)
73ba763… larrybr 72 Hence, inadvertant X-bit changes are unavoidable, or avoided
73ba763… larrybr 73 only by luck, in the general usage case noted above.
73ba763… larrybr 74
73ba763… larrybr 75 <h2>Problematic Usage Scenarios</h2>
73ba763… larrybr 76
73ba763… larrybr 77 <h3>A Simple Example</h3>
73ba763… larrybr 78
ea6b2d3… larrybr 79 * Open a checkout in Windows (using fossil.exe) from a project
73ba763… larrybr 80 whose files have a mixture of executable and non-executable files.
73ba763… larrybr 81 Use a checkout directory visible when running under WSL.
73ba763… larrybr 82
ea6b2d3… larrybr 83 * Navigate to the same directory in a Linux shell on WSL, then
73ba763… larrybr 84 run "fossil status".
73ba763… larrybr 85
ea6b2d3… larrybr 86 * Depending upon /etc/wsl.conf content (or defaults in its absence),
73ba763… larrybr 87 the status ouput will tag checkout files as EXECUTABLE or NOEXEC.
73ba763… larrybr 88
73ba763… larrybr 89 <h3>Continuation of Simple Example</h3>
73ba763… larrybr 90
ea6b2d3… larrybr 91 * In the same checkout as above "Simple Example", on WSL,
73ba763… larrybr 92 run "fossil revert" to correct all those errant X-bit values.
73ba763… larrybr 93
ea6b2d3… larrybr 94 * Run "fossil status" again in WSL to verify absence of toggled X-bits.
73ba763… larrybr 95
ea6b2d3… larrybr 96 * Run "ls -l" from WSL to find two files, one with its X-bit set
73ba763… larrybr 97 and the other with it clear.
73ba763… larrybr 98
ea6b2d3… larrybr 99 * From Windows, perform these steps on each of those files:<br>
ea6b2d3… larrybr 100 &nbsp;&nbsp;(1) read the_file content into a buffer<br>
ea6b2d3… larrybr 101 &nbsp;&nbsp;(2) rename the_file the_file.bak<br>
ea6b2d3… larrybr 102 &nbsp;&nbsp;(3) write buffer content to new file, the_file<br>
ea6b2d3… larrybr 103 &nbsp;&nbsp;(4) del the_file.bak (or leave it)<br>
ea6b2d3… larrybr 104 &nbsp;&nbsp;(Note that this sequence is similar to what many editors do when
73ba763… larrybr 105 a user modifies a file then uses undo to reverse the changes.)
73ba763… larrybr 106
ea6b2d3… larrybr 107 * Run "fossil status" again in WSL and observe that one of the
73ba763… larrybr 108 two files has had its X-bit toggled.
73ba763… larrybr 109
73ba763… larrybr 110 <h3>A Fossil-Only Example</h3>
73ba763… larrybr 111
ea6b2d3… larrybr 112 * In the another (different) checkout of the same version,
73ba763… larrybr 113 somehow cause "legitimate" X-bit toggles of two files whose
73ba763… larrybr 114 X-bits differ. (This "somehow" probably will involve WSL to
73ba763… larrybr 115 toggle file bits and fossil on WSL to commit the toggles.)
73ba763… larrybr 116
ea6b2d3… larrybr 117 * In the Simple Example checkout, use fossil.exe on Windows
73ba763… larrybr 118 to update the checkout, ostensibly bringing the X-bit toggles
73ba763… larrybr 119 into the affected checkout files.
73ba763… larrybr 120
ea6b2d3… larrybr 121 * In the Simple Example checkout, use fossil on WSL to run
73ba763… larrybr 122 "fossil status", and observe at least one X-bit discrepancy.
73ba763… larrybr 123
73ba763… larrybr 124 <h2>Recommended Workflow</h2>
73ba763… larrybr 125
73ba763… larrybr 126 There are two simple approaches for dealing with this issue when
73ba763… larrybr 127 one wishes to continue using the same checkout directory from
73ba763… larrybr 128 Windows and WSL. Either one is effective. These are:
73ba763… larrybr 129
ea6b2d3… larrybr 130 * Do not use fossil on WSL for any operations which will modify
73ba763… larrybr 131 the repository. Instead, block those operations in some manner.
73ba763… larrybr 132
ea6b2d3… larrybr 133 * Do not use any tools on Windows, (including certain subcommands
73ba763… larrybr 134 of fossil.exe,) which may modify the X-bits on files within the
73ba763… larrybr 135 shared checkout, instead restricting use of Windows tools to those
73ba763… larrybr 136 which are known to only (and actually) modify file content in place
73ba763… larrybr 137 while preserving X-bit values. (The "actually" proviso emphasizes
73ba763… larrybr 138 that tools which only simulate in-place file modification, but do
73ba763… larrybr 139 so via create combined with delete and rename, are to be avoided.
73ba763… larrybr 140 A simulation which works flawlessly on Windows may not preserve
73ba763… larrybr 141 the WSL X-bit.)
73ba763… larrybr 142
73ba763… larrybr 143 There are more complex ways to deal with this issue, involving
73ba763… larrybr 144 use of fossil on WSL to fix (or revert) toggled X-bits prior
73ba763… larrybr 145 to any commit, together with actions needed to preserve all
73ba763… larrybr 146 intended changes to the checkout as fossil revert is done.
73ba763… larrybr 147 Such methods are overly clever or fragile for elaboration here.
73ba763… larrybr 148
73ba763… larrybr 149 Another way to deal with this issue is to correct any toggled
73ba763… larrybr 150 X-bits within a checkout before using "fossil commit" on WSL
73ba763… larrybr 151 by means other than "fossil revert".
73ba763… larrybr 152
73ba763… larrybr 153 <h2>Corrective Measures or Mitigation</h2>
73ba763… larrybr 154
73ba763… larrybr 155 It is possible, by either manual or automated means, to perform
73ba763… larrybr 156 a pre-commit check and/or correction for mis-toggled X-bits.
73ba763… larrybr 157
73ba763… larrybr 158 The X-bit states are available from the repository for whatever
73ba763… larrybr 159 versions it has stored. And several Linux tools are able to read
73ba763… larrybr 160 or alter the X-bit state of files. With these components, a tool
73ba763… larrybr 161 can be readily built to aid avoidance of a commit (via fossil on
73ba763… larrybr 162 WSL) that would record mis-toggled X-bits into the repository.
73ba763… larrybr 163
73ba763… larrybr 164 Fossil itself on WSL will detect mis-toggled X-bits for files
73ba763… larrybr 165 which have not been otherise modified, but altered file content
73ba763… larrybr 166 masks such detection, and it is just such modification that is
73ba763… larrybr 167 among the problematic scenarios. So Fossil alone cannot yet
73ba763… larrybr 168 reliably do the detection or correction needed to avoid or
73ba763… larrybr 169 remedy the mis-toggled X-bit commit problem.
73ba763… larrybr 170
73ba763… larrybr 171 It is also feasible to detect or correct the mis-toggled X-bit
73ba763… larrybr 172 problem within Windows with a special-purpose tool which can
73ba763… larrybr 173 read, create or modify the X-bits stored by WSL for any file
73ba763… larrybr 174 which has been subject to the Linux chmod(...) system call.
73ba763… larrybr 175
73ba763… larrybr 176 Creation of these tools is beyond the scope of this document.

Keyboard Shortcuts

Open search /
Next entry (timeline) j
Previous entry (timeline) k
Open focused entry Enter
Show this help ?
Toggle theme Top nav button