FossilRepo

fossilrepo / docs / architecture / overview.md
1
# Architecture
2
3
## Overview
4
5
FossilRepo is a Django web application that wraps Fossil SCM repositories with a modern UI. It reads `.fossil` files directly as SQLite databases for speed, and uses the `fossil` CLI binary for write operations to maintain artifact integrity.
6
7
```
8
Browser (HTMX + Alpine.js + Tailwind CSS)
9
|
10
v
11
Django 5 (views, ORM, permissions)
12
|
13
|-- FossilReader (direct SQLite reads, ?mode=ro)
14
|-- FossilCLI (subprocess for writes: commit, ticket, wiki, push/pull)
15
|-- fossil http (CGI proxy for clone/push/pull)
16
|
17
|-- PostgreSQL 16 (app data: users, orgs, teams, projects, settings)
18
|-- Redis 7 (Celery broker, cache)
19
|-- Celery (background: metadata sync, git mirror, webhooks, digest)
20
|
21
v
22
.fossil files (SQLite: code + wiki + tickets + forum + technotes)
23
|
24
v
25
Litestream --> S3 (continuous SQLite replication)
26
```
27
28
## Core Components
29
30
### FossilReader
31
32
Opens `.fossil` repository files directly as read-only SQLite databases. No network calls to a running Fossil server. Python's `sqlite3` module with `?mode=ro` URI.
33
34
Handles:
35
- Blob decompression (zlib with 4-byte size prefix)
36
- Delta chain resolution (Fossil's delta-encoded artifacts)
37
- Julian day timestamp conversion
38
- Timeline queries, file tree at any checkin, ticket/wiki/forum reads
39
- Commit activity aggregation, contributor stats, search
40
41
### FossilCLI
42
43
Thin subprocess wrapper around the `fossil` binary. Used for all write operations:
44
- Repository init, clone, push, pull, sync
45
- Ticket create/change, wiki create/commit
46
- Technote creation, blame, Pikchr rendering
47
- Git export for mirror sync
48
- Tarball and zip archive generation
49
- Unversioned file management
50
- Artifact shunning
51
52
All calls set `USER=fossilrepo` in the environment and call `ensure_default_user()` to prevent "cannot figure out who you are" errors.
53
54
### HTTP Sync Proxy
55
56
The `fossil_xfer` view proxies Fossil's wire protocol through Django. Clients clone/push/pull via:
57
58
```
59
fossil clone http://your-server/projects/<slug>/fossil/xfer repo.fossil
60
```
61
62
Django handles authentication and access control. Public repos allow anonymous pull (no `--localauth`). Authenticated users with write access get full push via `--localauth`. Branch protection rules are enforced at this layer.
63
64
### SSH Sync
65
66
An `sshd` instance runs on port 2222 with a restricted `fossil-shell` forced command. Users upload SSH public keys via their profile. The `authorized_keys` file is regenerated from the database on key add/remove.
67
68
```
69
fossil clone ssh://fossil@host:2222/<slug> repo.fossil
70
```
71
72
## Data Architecture
73
74
### Two Databases
75
76
1. **PostgreSQL** — application state: users, organizations, teams, projects, releases, webhooks, API tokens, workspace claims, code reviews, notification preferences
77
2. **Fossil .fossil files** — repository data: code history, tickets, wiki, forum, technotes, unversioned files
78
79
### Model Base Classes
80
81
- `Tracking` (abstract) — `version`, `created_at/by`, `updated_at/by`, `deleted_at/by`, `history` (django-simple-history)
82
- `BaseCoreModel(Tracking)` — adds `guid` (UUID4), `name`, `slug` (auto-generated), `description`
83
- Soft deletes only: `obj.soft_delete(user=request.user)`, never `.delete()`
84
- `ActiveManager` on `objects` excludes soft-deleted; `all_objects` includes them
85
86
### Permission Model
87
88
Two layers:
89
1. **Org-level roles** (Admin/Manager/Developer/Viewer) — Django Groups with permission bundles, assignable per user
90
2. **Project-level RBAC** (read/write/admin) — per team, via ProjectTeam model
91
92
Project visibility:
93
- **Public** — anyone can read (including anonymous)
94
- **Internal** — authenticated users can read
95
- **Private** — team members only
96
97
### Encryption
98
99
SSH keys and OAuth tokens encrypted at rest using Fernet (AES-128-CBC + HMAC-SHA256), keyed from Django's `SECRET_KEY`. Implemented as `EncryptedTextField` in `core/fields.py`.
100
101
## Infrastructure
102
103
### Docker (Omnibus)
104
105
Single multi-stage Dockerfile:
106
1. Stage 1: compile Fossil 2.24 from source (Debian bookworm)
107
2. Stage 2: Python 3.12 runtime with Fossil binary, sshd, gosu
108
109
Entrypoint starts sshd as root, drops to unprivileged `app` user for gunicorn.
110
111
### Celery Tasks
112
113
| Task | Schedule | Purpose |
114
|------|----------|---------|
115
| sync_metadata | Every 5 min | Update repo stats (size, checkin count) |
116
| check_upstream | Every 15 min | Check for new upstream artifacts |
117
| dispatch_notifications | Every 5 min | Send pending email notifications |
118
| send_digest (daily) | Every 24h | Daily notification digest |
119
| send_digest (weekly) | Every 7d | Weekly notification digest |
120
| dispatch_webhook | On event | Deliver webhook with retry |
121
| run_git_sync | On schedule | Git mirror export |
122
123
### Caddy
124
125
SSL termination and subdomain routing. Each repo can get its own subdomain.
126
127
### Litestream
128
129
Continuous SQLite-to-S3 replication for `.fossil` files. Point-in-time recovery.
130
131
## Django Apps
132
133
| App | Purpose |
134
|-----|---------|
135
| `core` | Base models, permissions, pagination, sanitization, encryption |
136
| `accounts` | Login/logout, SSH keys, user profile, personal access tokens |
137
| `organization` | Org settings, teams, members, roles |
138
| `projects` | Projects, project groups, project stars, team assignment |
139
| `pages` | FossilRepo KB (knowledge base articles) |
140
| `fossil` | Everything Fossil: reader, CLI, views, sync, webhooks, releases, CI, workspaces, reviews |
141
| `mcp_server` | MCP server for AI tool integration |
142

Keyboard Shortcuts

Open search /
Next entry (timeline) j
Previous entry (timeline) k
Open focused entry Enter
Show this help ?
Toggle theme Top nav button