FossilRepo

fossilrepo / docs / architecture / overview.md
Source Blame History 141 lines
45192ef… ragelink 1 # Architecture
45192ef… ragelink 2
45192ef… ragelink 3 ## Overview
45192ef… ragelink 4
45192ef… ragelink 5 FossilRepo is a Django web application that wraps Fossil SCM repositories with a modern UI. It reads `.fossil` files directly as SQLite databases for speed, and uses the `fossil` CLI binary for write operations to maintain artifact integrity.
45192ef… ragelink 6
45192ef… ragelink 7 ```
45192ef… ragelink 8 Browser (HTMX + Alpine.js + Tailwind CSS)
45192ef… ragelink 9 |
45192ef… ragelink 10 v
45192ef… ragelink 11 Django 5 (views, ORM, permissions)
45192ef… ragelink 12 |
45192ef… ragelink 13 |-- FossilReader (direct SQLite reads, ?mode=ro)
45192ef… ragelink 14 |-- FossilCLI (subprocess for writes: commit, ticket, wiki, push/pull)
45192ef… ragelink 15 |-- fossil http (CGI proxy for clone/push/pull)
45192ef… ragelink 16 |
45192ef… ragelink 17 |-- PostgreSQL 16 (app data: users, orgs, teams, projects, settings)
45192ef… ragelink 18 |-- Redis 7 (Celery broker, cache)
45192ef… ragelink 19 |-- Celery (background: metadata sync, git mirror, webhooks, digest)
45192ef… ragelink 20 |
45192ef… ragelink 21 v
45192ef… ragelink 22 .fossil files (SQLite: code + wiki + tickets + forum + technotes)
45192ef… ragelink 23 |
45192ef… ragelink 24 v
45192ef… ragelink 25 Litestream --> S3 (continuous SQLite replication)
45192ef… ragelink 26 ```
45192ef… ragelink 27
45192ef… ragelink 28 ## Core Components
45192ef… ragelink 29
45192ef… ragelink 30 ### FossilReader
45192ef… ragelink 31
45192ef… ragelink 32 Opens `.fossil` repository files directly as read-only SQLite databases. No network calls to a running Fossil server. Python's `sqlite3` module with `?mode=ro` URI.
45192ef… ragelink 33
45192ef… ragelink 34 Handles:
45192ef… ragelink 35 - Blob decompression (zlib with 4-byte size prefix)
45192ef… ragelink 36 - Delta chain resolution (Fossil's delta-encoded artifacts)
45192ef… ragelink 37 - Julian day timestamp conversion
45192ef… ragelink 38 - Timeline queries, file tree at any checkin, ticket/wiki/forum reads
45192ef… ragelink 39 - Commit activity aggregation, contributor stats, search
45192ef… ragelink 40
45192ef… ragelink 41 ### FossilCLI
45192ef… ragelink 42
45192ef… ragelink 43 Thin subprocess wrapper around the `fossil` binary. Used for all write operations:
45192ef… ragelink 44 - Repository init, clone, push, pull, sync
45192ef… ragelink 45 - Ticket create/change, wiki create/commit
45192ef… ragelink 46 - Technote creation, blame, Pikchr rendering
45192ef… ragelink 47 - Git export for mirror sync
45192ef… ragelink 48 - Tarball and zip archive generation
45192ef… ragelink 49 - Unversioned file management
45192ef… ragelink 50 - Artifact shunning
45192ef… ragelink 51
45192ef… ragelink 52 All calls set `USER=fossilrepo` in the environment and call `ensure_default_user()` to prevent "cannot figure out who you are" errors.
45192ef… ragelink 53
45192ef… ragelink 54 ### HTTP Sync Proxy
45192ef… ragelink 55
45192ef… ragelink 56 The `fossil_xfer` view proxies Fossil's wire protocol through Django. Clients clone/push/pull via:
45192ef… ragelink 57
45192ef… ragelink 58 ```
45192ef… ragelink 59 fossil clone http://your-server/projects/<slug>/fossil/xfer repo.fossil
45192ef… ragelink 60 ```
45192ef… ragelink 61
45192ef… ragelink 62 Django handles authentication and access control. Public repos allow anonymous pull (no `--localauth`). Authenticated users with write access get full push via `--localauth`. Branch protection rules are enforced at this layer.
45192ef… ragelink 63
45192ef… ragelink 64 ### SSH Sync
45192ef… ragelink 65
45192ef… ragelink 66 An `sshd` instance runs on port 2222 with a restricted `fossil-shell` forced command. Users upload SSH public keys via their profile. The `authorized_keys` file is regenerated from the database on key add/remove.
45192ef… ragelink 67
45192ef… ragelink 68 ```
45192ef… ragelink 69 fossil clone ssh://fossil@host:2222/<slug> repo.fossil
2eca4eb… ragelink 70 ```
2eca4eb… ragelink 71
45192ef… ragelink 72 ## Data Architecture
45192ef… ragelink 73
45192ef… ragelink 74 ### Two Databases
45192ef… ragelink 75
45192ef… ragelink 76 1. **PostgreSQL** — application state: users, organizations, teams, projects, releases, webhooks, API tokens, workspace claims, code reviews, notification preferences
45192ef… ragelink 77 2. **Fossil .fossil files** — repository data: code history, tickets, wiki, forum, technotes, unversioned files
45192ef… ragelink 78
45192ef… ragelink 79 ### Model Base Classes
45192ef… ragelink 80
45192ef… ragelink 81 - `Tracking` (abstract) — `version`, `created_at/by`, `updated_at/by`, `deleted_at/by`, `history` (django-simple-history)
45192ef… ragelink 82 - `BaseCoreModel(Tracking)` — adds `guid` (UUID4), `name`, `slug` (auto-generated), `description`
45192ef… ragelink 83 - Soft deletes only: `obj.soft_delete(user=request.user)`, never `.delete()`
45192ef… ragelink 84 - `ActiveManager` on `objects` excludes soft-deleted; `all_objects` includes them
45192ef… ragelink 85
45192ef… ragelink 86 ### Permission Model
45192ef… ragelink 87
45192ef… ragelink 88 Two layers:
45192ef… ragelink 89 1. **Org-level roles** (Admin/Manager/Developer/Viewer) — Django Groups with permission bundles, assignable per user
45192ef… ragelink 90 2. **Project-level RBAC** (read/write/admin) — per team, via ProjectTeam model
45192ef… ragelink 91
45192ef… ragelink 92 Project visibility:
45192ef… ragelink 93 - **Public** — anyone can read (including anonymous)
45192ef… ragelink 94 - **Internal** — authenticated users can read
45192ef… ragelink 95 - **Private** — team members only
45192ef… ragelink 96
45192ef… ragelink 97 ### Encryption
45192ef… ragelink 98
45192ef… ragelink 99 SSH keys and OAuth tokens encrypted at rest using Fernet (AES-128-CBC + HMAC-SHA256), keyed from Django's `SECRET_KEY`. Implemented as `EncryptedTextField` in `core/fields.py`.
45192ef… ragelink 100
45192ef… ragelink 101 ## Infrastructure
45192ef… ragelink 102
45192ef… ragelink 103 ### Docker (Omnibus)
45192ef… ragelink 104
45192ef… ragelink 105 Single multi-stage Dockerfile:
45192ef… ragelink 106 1. Stage 1: compile Fossil 2.24 from source (Debian bookworm)
45192ef… ragelink 107 2. Stage 2: Python 3.12 runtime with Fossil binary, sshd, gosu
45192ef… ragelink 108
45192ef… ragelink 109 Entrypoint starts sshd as root, drops to unprivileged `app` user for gunicorn.
2eca4eb… ragelink 110
45192ef… ragelink 111 ### Celery Tasks
2eca4eb… ragelink 112
45192ef… ragelink 113 | Task | Schedule | Purpose |
45192ef… ragelink 114 |------|----------|---------|
45192ef… ragelink 115 | sync_metadata | Every 5 min | Update repo stats (size, checkin count) |
45192ef… ragelink 116 | check_upstream | Every 15 min | Check for new upstream artifacts |
45192ef… ragelink 117 | dispatch_notifications | Every 5 min | Send pending email notifications |
45192ef… ragelink 118 | send_digest (daily) | Every 24h | Daily notification digest |
45192ef… ragelink 119 | send_digest (weekly) | Every 7d | Weekly notification digest |
45192ef… ragelink 120 | dispatch_webhook | On event | Deliver webhook with retry |
45192ef… ragelink 121 | run_git_sync | On schedule | Git mirror export |
2eca4eb… ragelink 122
2eca4eb… ragelink 123 ### Caddy
2eca4eb… ragelink 124
45192ef… ragelink 125 SSL termination and subdomain routing. Each repo can get its own subdomain.
2eca4eb… ragelink 126
2eca4eb… ragelink 127 ### Litestream
2eca4eb… ragelink 128
45192ef… ragelink 129 Continuous SQLite-to-S3 replication for `.fossil` files. Point-in-time recovery.
45192ef… ragelink 130
45192ef… ragelink 131 ## Django Apps
45192ef… ragelink 132
45192ef… ragelink 133 | App | Purpose |
45192ef… ragelink 134 |-----|---------|
45192ef… ragelink 135 | `core` | Base models, permissions, pagination, sanitization, encryption |
45192ef… ragelink 136 | `accounts` | Login/logout, SSH keys, user profile, personal access tokens |
45192ef… ragelink 137 | `organization` | Org settings, teams, members, roles |
45192ef… ragelink 138 | `projects` | Projects, project groups, project stars, team assignment |
45192ef… ragelink 139 | `pages` | FossilRepo KB (knowledge base articles) |
45192ef… ragelink 140 | `fossil` | Everything Fossil: reader, CLI, views, sync, webhooks, releases, CI, workspaces, reviews |
45192ef… ragelink 141 | `mcp_server` | MCP server for AI tool integration |

Keyboard Shortcuts

Open search /
Next entry (timeline) j
Previous entry (timeline) k
Open focused entry Enter
Show this help ?
Toggle theme Top nav button