Fossil SCM
Restore the full CGI extension documentation that was somehow truncated by an earlier commit.
Commit
4130a22ceeac77711df73e312bd68d738b0aaf918922eab7e2b82f79ef91a408
Parent
02bdcf5e9f71d7f…
1 file changed
+250
-22
+250
-22
| --- www/serverext.wiki | ||
| +++ www/serverext.wiki | ||
| @@ -1,31 +1,33 @@ | ||
| 1 | 1 | <title>CGI Server Extensions</title> |
| 2 | 2 | |
| 3 | -If you choose to have a [./server.wiki|central server] for your Fossil-based | |
| 4 | -project (which is entirely optional - Fossil can be use an a pure peer-to-peer | |
| 5 | -mode) then you can also add CGI extensions to the Fossil websiteite d. These | |
| 6 | -extensions work like any other CGI program, except that they also have access | |
| 7 | -to the Fossil login information, so that they can restrict access based on | |
| 8 | -the currently logged in user. The CGI output can be a stand-alone webpage, | |
| 9 | -or it can be incorporated into the Fossil site using the standard Fossil | |
| 10 | -header and footer. | |
| 3 | +<h2>1.0 Introduction</h2> | |
| 4 | + | |
| 5 | +If you have a [./server.wiki|Fossil server] for your project, | |
| 6 | +you can add [https://en.wikipedia.org/wiki/Common_Gateway_Interface|CGI] | |
| 7 | +extensions to that server. These extensions work like | |
| 8 | +any other CGI program, except that they also have access to the Fossil | |
| 9 | +login information and can (optionally) leverage the "skins" of Fossil | |
| 10 | +so that they appear to be more tightly integrated into the project. | |
| 11 | 11 | |
| 12 | 12 | An example of where this is useful is the |
| 13 | -[checklist application](https://sqlite.org/src/ext/checklist) on | |
| 14 | -the [SQLite](https://sqlite.org/) project. The checklist | |
| 13 | +[https://sqlite.org/src/ext/checklist|checklist application] on | |
| 14 | +the [https://sqlite.org/|SQLite] project. The checklist | |
| 15 | 15 | helps the SQLite developers track which release tests have passed, |
| 16 | -or failed, or are still to be done. The checklist used to be a | |
| 16 | +or failed, or are still to be done. The checklist program began as a | |
| 17 | 17 | stand-alone CGI which kept its own private user database and implemented |
| 18 | -its on permissions and login system. By converting checklist into | |
| 19 | -a Fossil extension, the same login that works for the | |
| 20 | -[main SQLite source repository](https://sqlite.org/src) also works | |
| 21 | -for the checklist. And permission to change elements of the checklist | |
| 22 | -is based on permission to check-in to the main source repository. | |
| 23 | - | |
| 24 | -<h2>How It Works</h2> | |
| 25 | - | |
| 26 | -Extensions are off by default. | |
| 18 | +its own permissions and login system and provided its own CSS. By | |
| 19 | +converting checklist into a Fossil extension, the same login that works | |
| 20 | +for the [https://sqlite.org/src|main SQLite source repository] also works | |
| 21 | +for the checklist. Permission to change elements of the checklist | |
| 22 | +is tied on permission to check-in to the main source repository. And | |
| 23 | +the standard Fossil header menu and footer appear on each page of | |
| 24 | +the checklist. | |
| 25 | + | |
| 26 | +<h2>2.0 How It Works</h2> | |
| 27 | + | |
| 28 | +CGI Extensions are disabled by default. | |
| 27 | 29 | An administrator activates the CGI extension mechanism by specifying |
| 28 | 30 | an "Extension Root Directory" or "extroot" as part of the server setup. |
| 29 | 31 | If the Fossil server is itself run as CGI, then add a line to the CGI |
| 30 | 32 | script file that says: |
| 31 | 33 | |
| @@ -35,12 +37,12 @@ | ||
| 35 | 37 | |
| 36 | 38 | Or, if the Fossil server is begin run as using the "fossil server" or |
| 37 | 39 | "fossil ui" or "fossil http" commands, then add an extra |
| 38 | 40 | "--extroot <i>DIRECTORY</i>" option to that command. |
| 39 | 41 | |
| 40 | -The <i>DIRECTORY</i> thus specified becomes the DOCUMENT_ROOT for the | |
| 41 | -CGI. Files in the DOCUMENT_ROOT are accessed via URLs like this: | |
| 42 | +The <i>DIRECTORY</i> is the DOCUMENT_ROOT for the CGI. | |
| 43 | +Files in the DOCUMENT_ROOT are accessed via URLs like this: | |
| 42 | 44 | |
| 43 | 45 | <blockquote> |
| 44 | 46 | https://example-project.org/ext/<i>FILENAME</i> |
| 45 | 47 | </blockquote> |
| 46 | 48 | |
| @@ -47,5 +49,231 @@ | ||
| 47 | 49 | In other words, access files in DOCUMENT_ROOT by appending the filename |
| 48 | 50 | relative to DOCUMENT_ROOT to the [/help?cmd=/ext|/ext] |
| 49 | 51 | page of the Fossil server. |
| 50 | 52 | Files that are readable but not executable are returned as static |
| 51 | 53 | content. Files that are executable are run as CGI. |
| 54 | + | |
| 55 | +<h3>2.1 Example #1</h3> | |
| 56 | + | |
| 57 | +The source code repository for SQLite is a Fossil server that is run | |
| 58 | +as CGI. The URL for the source code repository is [https://sqlite.org/src]. | |
| 59 | +The CGI script looks like this: | |
| 60 | + | |
| 61 | +<blockquote><verbatim> | |
| 62 | +#!/usr/bin/fossil | |
| 63 | +repository: /fossil/sqlite.fossil | |
| 64 | +errorlog: /logs/errors.txt | |
| 65 | +extroot: /sqlite-src-ext | |
| 66 | +</verbatim></blockquote> | |
| 67 | + | |
| 68 | +The "extroot: /sqlite-src-ext" line tells Fossil that it should look for | |
| 69 | +extension CGIs in the /sqlite-src-ext directory. (All of this is happening | |
| 70 | +inside of a chroot jail, so putting the document root in a top-level | |
| 71 | +directory is a reasonable thing to do.) | |
| 72 | + | |
| 73 | +When a URL like "https://sqlite.org/src/ext/checklist" is received by the | |
| 74 | +main webserver, it figures out that the /src part refers to the main | |
| 75 | +Fossil CGI script and so it runs that script. Fossil gets the remainder | |
| 76 | +of the URL to work with: "/ext/checklist". Fossil extracts the "/ext" | |
| 77 | +prefix and uses that to determine that this a CGI extension request. | |
| 78 | +Then it takes the leftover "/checklist" part and appends it to the | |
| 79 | +"extroot" to get the filename "/sqlite-src-ext/checklist". Fossil finds | |
| 80 | +that file to be executable, so it runs it as CGI and returns the result. | |
| 81 | + | |
| 82 | +The /sqlite-src-ext/checklist file is a | |
| 83 | +[https://wapp.tcl.tk|Wapp program]. The current source code to the | |
| 84 | +this program can be seen at | |
| 85 | +[https://www.sqlite.org/src/ext/checklist/self] and | |
| 86 | +recent historical versions are available at | |
| 87 | +[https://sqlite.org/docsrc/finfo/misc/checklist.tcl] with | |
| 88 | +older legacy at [https://sqlite.org/checklistapp/timeline?n=all] | |
| 89 | + | |
| 90 | +There is a cascade of CGIs happening here. The webserver that receives | |
| 91 | +the initial HTTP request runs Fossil as a CGI based on the | |
| 92 | +"https://sqlite.org/src" portion of the URL. The Fossil instance then | |
| 93 | +runs the checklist sub-CGI based on the "/ext/checklists" suffix. The | |
| 94 | +output of the sub-CGI is read by Fossil and then relayed on to the | |
| 95 | +main webserver which in turn relays the result back to the original client. | |
| 96 | + | |
| 97 | +<h3>2.2 Example #2</h3> | |
| 98 | + | |
| 99 | +The [https://fossil-scm.org/home|Fossil self-hosting repository] is also | |
| 100 | +a CGI that looks like this: | |
| 101 | + | |
| 102 | +<blockquote><verbatim> | |
| 103 | +#!/usr/bin/fossil | |
| 104 | +repository: /fossil/fossil.fossil | |
| 105 | +errorlog: /logs/errors.txt | |
| 106 | +extroot: /fossil-extroot | |
| 107 | +</verbatim></blockquote> | |
| 108 | + | |
| 109 | +The extroot for this Fossil server is /fossil-extroot and in that directory | |
| 110 | +is an executable file named "fileup1" - another [https://wapp.tcl.tk|Wapp] | |
| 111 | +script. (The extension mechanism does not have to use Wapp. You can use | |
| 112 | +any kind of program you like. But the creator of SQLite and Fossil is fond | |
| 113 | +of [https://www.tcl.tk|Tcl/Tk] and so he tends to gravitate toward Tcl-based | |
| 114 | +technologies like Wapp.) The fileup1 script is a demo program that lets | |
| 115 | +the user upload a file using a form, and then displays that file in the reply. | |
| 116 | +There is a link on the page that causes the fileup1 script to return a copy | |
| 117 | +of its own source-code, so you can see how it works. | |
| 118 | + | |
| 119 | +<h2>3.0 CGI Inputs</h2> | |
| 120 | + | |
| 121 | +The /ext extension mechanism is an ordinary CGI interface. Parameters | |
| 122 | +are passed to the CGI program using environment variables. The following | |
| 123 | +standard CGI environment variables are supported: | |
| 124 | + | |
| 125 | + * AUTH_TYPE | |
| 126 | + * AUTH_CONTENT | |
| 127 | + * CONTENT_LENGTH | |
| 128 | + * CONTENT_TYPE | |
| 129 | + * DOCUMENT_ROOT | |
| 130 | + * GATEWAY_INTERFACE | |
| 131 | + * HTTP_ACCEPT | |
| 132 | + * HTTP_ACCEPT_ENCODING | |
| 133 | + * HTTP_COOKIE | |
| 134 | + * HTTP_HOST | |
| 135 | + * HTTP_IF_MODIFIED_SINCE | |
| 136 | + * HTTP_IF_NONE_MATCH | |
| 137 | + * HTTP_REFERER | |
| 138 | + * HTTP_USER_AGENT | |
| 139 | + * PATH_INFO | |
| 140 | + * QUERY_STRING | |
| 141 | + * REMOTE_ADDR | |
| 142 | + * REMOTE_USER | |
| 143 | + * REQUEST_METHOD | |
| 144 | + * REQUEST_URI | |
| 145 | + * SCRIPT_DIRECTORY | |
| 146 | + * SCRIPT_FILENAME | |
| 147 | + * SCRIPT_NAME | |
| 148 | + * SERVER_NAME | |
| 149 | + * SERVER_PORT | |
| 150 | + * SERVER_PROTOCOL | |
| 151 | + | |
| 152 | +Do a web search for | |
| 153 | +"[https://duckduckgo.com/?q=cgi+environment_variables|cgi environment variables]" | |
| 154 | +to find more detail about what each of the above variables mean and how | |
| 155 | +they are used. | |
| 156 | +Live listings of the values of some or all of these environment variables | |
| 157 | +can be found at links like these: | |
| 158 | + | |
| 159 | + * [https://fossil-scm.org/home/test_env] | |
| 160 | + * [https://sqlite.org/src/ext/checklist/env] | |
| 161 | + | |
| 162 | +In addition to the standard CGI environment variables listed above, | |
| 163 | +Fossil adds the following: | |
| 164 | + | |
| 165 | + * FOSSIL_CAPABILITIES | |
| 166 | + * FOSSIL_REPOSITORY | |
| 167 | + * FOSSIL_USER | |
| 168 | + | |
| 169 | +The FOSSIL_USER string is the name of the logged-in user. This variable | |
| 170 | +is missing or is an empty string if the user is not logged in. The | |
| 171 | +FOSSIL_CAPABILITIES string is a list of | |
| 172 | +[/setup_ulist_notes|Fossil capability letters] that | |
| 173 | +indicate what permissions the user has on the Fossil repository. | |
| 174 | +The FOSSIL_REPOSITORY environment variable gives the filename of the | |
| 175 | +Fossil repository that is running. | |
| 176 | + | |
| 177 | +The [https://sqlite.org/src/ext/checklist|checklist application] uses the | |
| 178 | +FOSSIL_USER environment variable to determine the name of the user and | |
| 179 | +the FOSSIL_CAPABILITIES variable to determine if the user is allowed to | |
| 180 | +mark off changes to the checklist. Only users with check-in permission | |
| 181 | +to the Fossil repository are allowed to mark off checklist items. That | |
| 182 | +means that the FOSSIL_CAPABILITIES string must contain the letter "i". | |
| 183 | +Search for "FOSSIL_CAPABILITIES" in the | |
| 184 | +[https://sqlite.org/src/ext/checklist/self|source listing] to see how | |
| 185 | +this happens. | |
| 186 | + | |
| 187 | +If the HTTP request includes content (for example if this is a POST request) | |
| 188 | +then the CONTENT_LENGTH value will be positive and the data for the content | |
| 189 | +will be readable on standard input. | |
| 190 | + | |
| 191 | +<h2>4.0 CGI Outputs</h2> | |
| 192 | + | |
| 193 | +CGI programs construct a reply by writing to standard output. The first | |
| 194 | +few lines of output are parameters intended for the webserver that invoked | |
| 195 | +the CGI. These are followed by a blank line and then the content. | |
| 196 | + | |
| 197 | +Typical parameter output looks like this: | |
| 198 | + | |
| 199 | +<blockquote><verbatim> | |
| 200 | +Status: 200 Ok | |
| 201 | +Content-Type: text/html | |
| 202 | +</verbatim></blockquote> | |
| 203 | + | |
| 204 | +CGI programs can return any content type they want - they are not restricted | |
| 205 | +to text replies. It is OK for a CGI program to return (for example) | |
| 206 | +image/png. | |
| 207 | + | |
| 208 | +The fields of the CGI response header can be any valid HTTP header fields. | |
| 209 | +Those that Fossil does not understand are simply relayed back to up the | |
| 210 | +line to the requester. | |
| 211 | + | |
| 212 | +Fossil takes special action with some content types. If the Content-Type | |
| 213 | +is "application/x-fossil-wiki" or "application/x-markdown" then Fossil | |
| 214 | +converts the content from [/wiki_rules|Fossil-Wiki] or | |
| 215 | +[/md_rules|Markdown] into HTML, adding its | |
| 216 | +own header and footer text according to the repository skin. Content | |
| 217 | +of type "text/html" is normally passed straight through | |
| 218 | +unchanged. However, if the text/html content is of the form: | |
| 219 | + | |
| 220 | +<blockquote><verbatim> | |
| 221 | +<div class='fossil-doc' data-title='DOCUMENT TITLE'> | |
| 222 | +... HTML content there ... | |
| 223 | +</div> | |
| 224 | +</verbatim></blockquote> | |
| 225 | + | |
| 226 | +In other words, if the outer-most markup of the HTML is a <div> | |
| 227 | +element with a single class of "fossil-doc", | |
| 228 | +then Fossil will adds its own header and footer to the HTML. The | |
| 229 | +page title contained in the added header will be extracted from the | |
| 230 | +"data-title" attribute. | |
| 231 | + | |
| 232 | +Except for the three cases noted above, Fossil makes no changes or | |
| 233 | +additions to the CGI-generated content. Fossil just passes the verbatim | |
| 234 | +content back up the stack towards the requester. | |
| 235 | + | |
| 236 | +<h2>5.0 Filename Restrictions</h2> | |
| 237 | + | |
| 238 | +For security reasons, Fossil places restrictions on the names of files | |
| 239 | +in the extroot directory that can participate in the extension CGI | |
| 240 | +mechanism: | |
| 241 | + | |
| 242 | + 1. Filenames must consist of only ASCII alphanumeric characters, | |
| 243 | + ".", "_", and "-", and of course "/" as the file separator. | |
| 244 | + Files with names that includes spaces or | |
| 245 | + other punctuation or special characters are ignored. | |
| 246 | + | |
| 247 | + 2. No element of the pathname can begin with "." or "-". Files or | |
| 248 | + directories whose names begin with "." or "-" are ignored. | |
| 249 | + | |
| 250 | +If a CGI program requires separate data files, it is safe to put those | |
| 251 | +files in the same directory as the CGI program itself as long as the names | |
| 252 | +of the data files contain special characters that cause them to be ignored | |
| 253 | +by Fossil. | |
| 254 | + | |
| 255 | +<h2>6.0 Trouble-Shooting Hints</h2> | |
| 256 | + | |
| 257 | +Remember that the /ext will return any file in the extroot directory | |
| 258 | +hierarchy as static content if the file is readable but not executable. | |
| 259 | +When initially setting up the /ext mechanism, it is sometimes helpful | |
| 260 | +to verify that you are able to receive static content prior to working | |
| 261 | +on getting your CGIs working. Also remember that CGIs must be | |
| 262 | +executable files. | |
| 263 | + | |
| 264 | +Fossil likes to run inside a chroot jail, and will automatically put | |
| 265 | +itself inside a chroot jail if it can. The sub-CGI program will also | |
| 266 | +run inside this same chroot jail. Make sure all embedded pathnames | |
| 267 | +have been adjusted accordingly and that all resources needed by the | |
| 268 | +CGI program are available within the chroot jail. | |
| 269 | + | |
| 270 | +If anything goes wrong while trying to process an /ext page, Fossil | |
| 271 | +returns a 404 Not Found error with no details. However, if the requester | |
| 272 | +is logged in as a user that has Debug privilege (capability letter "D") | |
| 273 | +then additional diagnostic information may be included in the output. | |
| 274 | + | |
| 275 | +If the /ext page has a "fossil-ext-debug=1" query parameter and if | |
| 276 | +the requester is logged in as a user with Debug privilege, then the | |
| 277 | +CGI output is returned verbatim, as text/plain and with the original | |
| 278 | +header intact. This is useful for trying diagnosing problems with the | |
| 279 | +CGI script. | |
| 52 | 280 |
| --- www/serverext.wiki | |
| +++ www/serverext.wiki | |
| @@ -1,31 +1,33 @@ | |
| 1 | <title>CGI Server Extensions</title> |
| 2 | |
| 3 | If you choose to have a [./server.wiki|central server] for your Fossil-based |
| 4 | project (which is entirely optional - Fossil can be use an a pure peer-to-peer |
| 5 | mode) then you can also add CGI extensions to the Fossil websiteite d. These |
| 6 | extensions work like any other CGI program, except that they also have access |
| 7 | to the Fossil login information, so that they can restrict access based on |
| 8 | the currently logged in user. The CGI output can be a stand-alone webpage, |
| 9 | or it can be incorporated into the Fossil site using the standard Fossil |
| 10 | header and footer. |
| 11 | |
| 12 | An example of where this is useful is the |
| 13 | [checklist application](https://sqlite.org/src/ext/checklist) on |
| 14 | the [SQLite](https://sqlite.org/) project. The checklist |
| 15 | helps the SQLite developers track which release tests have passed, |
| 16 | or failed, or are still to be done. The checklist used to be a |
| 17 | stand-alone CGI which kept its own private user database and implemented |
| 18 | its on permissions and login system. By converting checklist into |
| 19 | a Fossil extension, the same login that works for the |
| 20 | [main SQLite source repository](https://sqlite.org/src) also works |
| 21 | for the checklist. And permission to change elements of the checklist |
| 22 | is based on permission to check-in to the main source repository. |
| 23 | |
| 24 | <h2>How It Works</h2> |
| 25 | |
| 26 | Extensions are off by default. |
| 27 | An administrator activates the CGI extension mechanism by specifying |
| 28 | an "Extension Root Directory" or "extroot" as part of the server setup. |
| 29 | If the Fossil server is itself run as CGI, then add a line to the CGI |
| 30 | script file that says: |
| 31 | |
| @@ -35,12 +37,12 @@ | |
| 35 | |
| 36 | Or, if the Fossil server is begin run as using the "fossil server" or |
| 37 | "fossil ui" or "fossil http" commands, then add an extra |
| 38 | "--extroot <i>DIRECTORY</i>" option to that command. |
| 39 | |
| 40 | The <i>DIRECTORY</i> thus specified becomes the DOCUMENT_ROOT for the |
| 41 | CGI. Files in the DOCUMENT_ROOT are accessed via URLs like this: |
| 42 | |
| 43 | <blockquote> |
| 44 | https://example-project.org/ext/<i>FILENAME</i> |
| 45 | </blockquote> |
| 46 | |
| @@ -47,5 +49,231 @@ | |
| 47 | In other words, access files in DOCUMENT_ROOT by appending the filename |
| 48 | relative to DOCUMENT_ROOT to the [/help?cmd=/ext|/ext] |
| 49 | page of the Fossil server. |
| 50 | Files that are readable but not executable are returned as static |
| 51 | content. Files that are executable are run as CGI. |
| 52 |
| --- www/serverext.wiki | |
| +++ www/serverext.wiki | |
| @@ -1,31 +1,33 @@ | |
| 1 | <title>CGI Server Extensions</title> |
| 2 | |
| 3 | <h2>1.0 Introduction</h2> |
| 4 | |
| 5 | If you have a [./server.wiki|Fossil server] for your project, |
| 6 | you can add [https://en.wikipedia.org/wiki/Common_Gateway_Interface|CGI] |
| 7 | extensions to that server. These extensions work like |
| 8 | any other CGI program, except that they also have access to the Fossil |
| 9 | login information and can (optionally) leverage the "skins" of Fossil |
| 10 | so that they appear to be more tightly integrated into the project. |
| 11 | |
| 12 | An example of where this is useful is the |
| 13 | [https://sqlite.org/src/ext/checklist|checklist application] on |
| 14 | the [https://sqlite.org/|SQLite] project. The checklist |
| 15 | helps the SQLite developers track which release tests have passed, |
| 16 | or failed, or are still to be done. The checklist program began as a |
| 17 | stand-alone CGI which kept its own private user database and implemented |
| 18 | its own permissions and login system and provided its own CSS. By |
| 19 | converting checklist into a Fossil extension, the same login that works |
| 20 | for the [https://sqlite.org/src|main SQLite source repository] also works |
| 21 | for the checklist. Permission to change elements of the checklist |
| 22 | is tied on permission to check-in to the main source repository. And |
| 23 | the standard Fossil header menu and footer appear on each page of |
| 24 | the checklist. |
| 25 | |
| 26 | <h2>2.0 How It Works</h2> |
| 27 | |
| 28 | CGI Extensions are disabled by default. |
| 29 | An administrator activates the CGI extension mechanism by specifying |
| 30 | an "Extension Root Directory" or "extroot" as part of the server setup. |
| 31 | If the Fossil server is itself run as CGI, then add a line to the CGI |
| 32 | script file that says: |
| 33 | |
| @@ -35,12 +37,12 @@ | |
| 37 | |
| 38 | Or, if the Fossil server is begin run as using the "fossil server" or |
| 39 | "fossil ui" or "fossil http" commands, then add an extra |
| 40 | "--extroot <i>DIRECTORY</i>" option to that command. |
| 41 | |
| 42 | The <i>DIRECTORY</i> is the DOCUMENT_ROOT for the CGI. |
| 43 | Files in the DOCUMENT_ROOT are accessed via URLs like this: |
| 44 | |
| 45 | <blockquote> |
| 46 | https://example-project.org/ext/<i>FILENAME</i> |
| 47 | </blockquote> |
| 48 | |
| @@ -47,5 +49,231 @@ | |
| 49 | In other words, access files in DOCUMENT_ROOT by appending the filename |
| 50 | relative to DOCUMENT_ROOT to the [/help?cmd=/ext|/ext] |
| 51 | page of the Fossil server. |
| 52 | Files that are readable but not executable are returned as static |
| 53 | content. Files that are executable are run as CGI. |
| 54 | |
| 55 | <h3>2.1 Example #1</h3> |
| 56 | |
| 57 | The source code repository for SQLite is a Fossil server that is run |
| 58 | as CGI. The URL for the source code repository is [https://sqlite.org/src]. |
| 59 | The CGI script looks like this: |
| 60 | |
| 61 | <blockquote><verbatim> |
| 62 | #!/usr/bin/fossil |
| 63 | repository: /fossil/sqlite.fossil |
| 64 | errorlog: /logs/errors.txt |
| 65 | extroot: /sqlite-src-ext |
| 66 | </verbatim></blockquote> |
| 67 | |
| 68 | The "extroot: /sqlite-src-ext" line tells Fossil that it should look for |
| 69 | extension CGIs in the /sqlite-src-ext directory. (All of this is happening |
| 70 | inside of a chroot jail, so putting the document root in a top-level |
| 71 | directory is a reasonable thing to do.) |
| 72 | |
| 73 | When a URL like "https://sqlite.org/src/ext/checklist" is received by the |
| 74 | main webserver, it figures out that the /src part refers to the main |
| 75 | Fossil CGI script and so it runs that script. Fossil gets the remainder |
| 76 | of the URL to work with: "/ext/checklist". Fossil extracts the "/ext" |
| 77 | prefix and uses that to determine that this a CGI extension request. |
| 78 | Then it takes the leftover "/checklist" part and appends it to the |
| 79 | "extroot" to get the filename "/sqlite-src-ext/checklist". Fossil finds |
| 80 | that file to be executable, so it runs it as CGI and returns the result. |
| 81 | |
| 82 | The /sqlite-src-ext/checklist file is a |
| 83 | [https://wapp.tcl.tk|Wapp program]. The current source code to the |
| 84 | this program can be seen at |
| 85 | [https://www.sqlite.org/src/ext/checklist/self] and |
| 86 | recent historical versions are available at |
| 87 | [https://sqlite.org/docsrc/finfo/misc/checklist.tcl] with |
| 88 | older legacy at [https://sqlite.org/checklistapp/timeline?n=all] |
| 89 | |
| 90 | There is a cascade of CGIs happening here. The webserver that receives |
| 91 | the initial HTTP request runs Fossil as a CGI based on the |
| 92 | "https://sqlite.org/src" portion of the URL. The Fossil instance then |
| 93 | runs the checklist sub-CGI based on the "/ext/checklists" suffix. The |
| 94 | output of the sub-CGI is read by Fossil and then relayed on to the |
| 95 | main webserver which in turn relays the result back to the original client. |
| 96 | |
| 97 | <h3>2.2 Example #2</h3> |
| 98 | |
| 99 | The [https://fossil-scm.org/home|Fossil self-hosting repository] is also |
| 100 | a CGI that looks like this: |
| 101 | |
| 102 | <blockquote><verbatim> |
| 103 | #!/usr/bin/fossil |
| 104 | repository: /fossil/fossil.fossil |
| 105 | errorlog: /logs/errors.txt |
| 106 | extroot: /fossil-extroot |
| 107 | </verbatim></blockquote> |
| 108 | |
| 109 | The extroot for this Fossil server is /fossil-extroot and in that directory |
| 110 | is an executable file named "fileup1" - another [https://wapp.tcl.tk|Wapp] |
| 111 | script. (The extension mechanism does not have to use Wapp. You can use |
| 112 | any kind of program you like. But the creator of SQLite and Fossil is fond |
| 113 | of [https://www.tcl.tk|Tcl/Tk] and so he tends to gravitate toward Tcl-based |
| 114 | technologies like Wapp.) The fileup1 script is a demo program that lets |
| 115 | the user upload a file using a form, and then displays that file in the reply. |
| 116 | There is a link on the page that causes the fileup1 script to return a copy |
| 117 | of its own source-code, so you can see how it works. |
| 118 | |
| 119 | <h2>3.0 CGI Inputs</h2> |
| 120 | |
| 121 | The /ext extension mechanism is an ordinary CGI interface. Parameters |
| 122 | are passed to the CGI program using environment variables. The following |
| 123 | standard CGI environment variables are supported: |
| 124 | |
| 125 | * AUTH_TYPE |
| 126 | * AUTH_CONTENT |
| 127 | * CONTENT_LENGTH |
| 128 | * CONTENT_TYPE |
| 129 | * DOCUMENT_ROOT |
| 130 | * GATEWAY_INTERFACE |
| 131 | * HTTP_ACCEPT |
| 132 | * HTTP_ACCEPT_ENCODING |
| 133 | * HTTP_COOKIE |
| 134 | * HTTP_HOST |
| 135 | * HTTP_IF_MODIFIED_SINCE |
| 136 | * HTTP_IF_NONE_MATCH |
| 137 | * HTTP_REFERER |
| 138 | * HTTP_USER_AGENT |
| 139 | * PATH_INFO |
| 140 | * QUERY_STRING |
| 141 | * REMOTE_ADDR |
| 142 | * REMOTE_USER |
| 143 | * REQUEST_METHOD |
| 144 | * REQUEST_URI |
| 145 | * SCRIPT_DIRECTORY |
| 146 | * SCRIPT_FILENAME |
| 147 | * SCRIPT_NAME |
| 148 | * SERVER_NAME |
| 149 | * SERVER_PORT |
| 150 | * SERVER_PROTOCOL |
| 151 | |
| 152 | Do a web search for |
| 153 | "[https://duckduckgo.com/?q=cgi+environment_variables|cgi environment variables]" |
| 154 | to find more detail about what each of the above variables mean and how |
| 155 | they are used. |
| 156 | Live listings of the values of some or all of these environment variables |
| 157 | can be found at links like these: |
| 158 | |
| 159 | * [https://fossil-scm.org/home/test_env] |
| 160 | * [https://sqlite.org/src/ext/checklist/env] |
| 161 | |
| 162 | In addition to the standard CGI environment variables listed above, |
| 163 | Fossil adds the following: |
| 164 | |
| 165 | * FOSSIL_CAPABILITIES |
| 166 | * FOSSIL_REPOSITORY |
| 167 | * FOSSIL_USER |
| 168 | |
| 169 | The FOSSIL_USER string is the name of the logged-in user. This variable |
| 170 | is missing or is an empty string if the user is not logged in. The |
| 171 | FOSSIL_CAPABILITIES string is a list of |
| 172 | [/setup_ulist_notes|Fossil capability letters] that |
| 173 | indicate what permissions the user has on the Fossil repository. |
| 174 | The FOSSIL_REPOSITORY environment variable gives the filename of the |
| 175 | Fossil repository that is running. |
| 176 | |
| 177 | The [https://sqlite.org/src/ext/checklist|checklist application] uses the |
| 178 | FOSSIL_USER environment variable to determine the name of the user and |
| 179 | the FOSSIL_CAPABILITIES variable to determine if the user is allowed to |
| 180 | mark off changes to the checklist. Only users with check-in permission |
| 181 | to the Fossil repository are allowed to mark off checklist items. That |
| 182 | means that the FOSSIL_CAPABILITIES string must contain the letter "i". |
| 183 | Search for "FOSSIL_CAPABILITIES" in the |
| 184 | [https://sqlite.org/src/ext/checklist/self|source listing] to see how |
| 185 | this happens. |
| 186 | |
| 187 | If the HTTP request includes content (for example if this is a POST request) |
| 188 | then the CONTENT_LENGTH value will be positive and the data for the content |
| 189 | will be readable on standard input. |
| 190 | |
| 191 | <h2>4.0 CGI Outputs</h2> |
| 192 | |
| 193 | CGI programs construct a reply by writing to standard output. The first |
| 194 | few lines of output are parameters intended for the webserver that invoked |
| 195 | the CGI. These are followed by a blank line and then the content. |
| 196 | |
| 197 | Typical parameter output looks like this: |
| 198 | |
| 199 | <blockquote><verbatim> |
| 200 | Status: 200 Ok |
| 201 | Content-Type: text/html |
| 202 | </verbatim></blockquote> |
| 203 | |
| 204 | CGI programs can return any content type they want - they are not restricted |
| 205 | to text replies. It is OK for a CGI program to return (for example) |
| 206 | image/png. |
| 207 | |
| 208 | The fields of the CGI response header can be any valid HTTP header fields. |
| 209 | Those that Fossil does not understand are simply relayed back to up the |
| 210 | line to the requester. |
| 211 | |
| 212 | Fossil takes special action with some content types. If the Content-Type |
| 213 | is "application/x-fossil-wiki" or "application/x-markdown" then Fossil |
| 214 | converts the content from [/wiki_rules|Fossil-Wiki] or |
| 215 | [/md_rules|Markdown] into HTML, adding its |
| 216 | own header and footer text according to the repository skin. Content |
| 217 | of type "text/html" is normally passed straight through |
| 218 | unchanged. However, if the text/html content is of the form: |
| 219 | |
| 220 | <blockquote><verbatim> |
| 221 | <div class='fossil-doc' data-title='DOCUMENT TITLE'> |
| 222 | ... HTML content there ... |
| 223 | </div> |
| 224 | </verbatim></blockquote> |
| 225 | |
| 226 | In other words, if the outer-most markup of the HTML is a <div> |
| 227 | element with a single class of "fossil-doc", |
| 228 | then Fossil will adds its own header and footer to the HTML. The |
| 229 | page title contained in the added header will be extracted from the |
| 230 | "data-title" attribute. |
| 231 | |
| 232 | Except for the three cases noted above, Fossil makes no changes or |
| 233 | additions to the CGI-generated content. Fossil just passes the verbatim |
| 234 | content back up the stack towards the requester. |
| 235 | |
| 236 | <h2>5.0 Filename Restrictions</h2> |
| 237 | |
| 238 | For security reasons, Fossil places restrictions on the names of files |
| 239 | in the extroot directory that can participate in the extension CGI |
| 240 | mechanism: |
| 241 | |
| 242 | 1. Filenames must consist of only ASCII alphanumeric characters, |
| 243 | ".", "_", and "-", and of course "/" as the file separator. |
| 244 | Files with names that includes spaces or |
| 245 | other punctuation or special characters are ignored. |
| 246 | |
| 247 | 2. No element of the pathname can begin with "." or "-". Files or |
| 248 | directories whose names begin with "." or "-" are ignored. |
| 249 | |
| 250 | If a CGI program requires separate data files, it is safe to put those |
| 251 | files in the same directory as the CGI program itself as long as the names |
| 252 | of the data files contain special characters that cause them to be ignored |
| 253 | by Fossil. |
| 254 | |
| 255 | <h2>6.0 Trouble-Shooting Hints</h2> |
| 256 | |
| 257 | Remember that the /ext will return any file in the extroot directory |
| 258 | hierarchy as static content if the file is readable but not executable. |
| 259 | When initially setting up the /ext mechanism, it is sometimes helpful |
| 260 | to verify that you are able to receive static content prior to working |
| 261 | on getting your CGIs working. Also remember that CGIs must be |
| 262 | executable files. |
| 263 | |
| 264 | Fossil likes to run inside a chroot jail, and will automatically put |
| 265 | itself inside a chroot jail if it can. The sub-CGI program will also |
| 266 | run inside this same chroot jail. Make sure all embedded pathnames |
| 267 | have been adjusted accordingly and that all resources needed by the |
| 268 | CGI program are available within the chroot jail. |
| 269 | |
| 270 | If anything goes wrong while trying to process an /ext page, Fossil |
| 271 | returns a 404 Not Found error with no details. However, if the requester |
| 272 | is logged in as a user that has Debug privilege (capability letter "D") |
| 273 | then additional diagnostic information may be included in the output. |
| 274 | |
| 275 | If the /ext page has a "fossil-ext-debug=1" query parameter and if |
| 276 | the requester is logged in as a user with Debug privilege, then the |
| 277 | CGI output is returned verbatim, as text/plain and with the original |
| 278 | header intact. This is useful for trying diagnosing problems with the |
| 279 | CGI script. |
| 280 |