Fossil Forum

Fossil's support for binary assets is roughly equal to Git's. Nothing about the storage model prevents it, though fossil commit will give a warning about binary blobs by default. The biggest limitation is that Fossil is restricted to blobs of around 1 GiB in size, a result of them being stored in an SQLite database. (I don't think it's a limitation of Fossil per se, an alternate storage backend could probably allow unlimited size blobs in Fossil at the cost of compatibility.)

Epic Games recently released their own home grown version control system specifically for games: https://github.com/EpicGames/lore It seems cool, though if I had to switch away from git, I'd rather use Fossil to be honest....

There's hardly a need to try to use one tool for everything. Fossil is still excellent at code projects. Lore is most likely a better fit for your game asset needs. Use the best tool for the job. :-)

That makes a lot of sense, thank you!

I completely understand that Fossil is probably not the best tool here compared to like Perforce or Lore I guess. But, if I WERE to try to hack together some sort of Git LFS solution for Fossil, what would it look like?

I started hacking together a git annex solution for fossil and had an almost-functional prototype. Basically, you'd use a pre-commit hook to move large binary files out of the repository and replace them with symlinks to an externally-managed directory structure (or whatever blob store you can imagine). At that point, fossil is only tracking a symlink to the hash of the file. And your tool will have to connect the symlink to an actual file.

git-annex does this fairly well. I use it to manage all kinds of large binaries between different machines and backup drives. I might've preferred using my home-grown fossil solution, but the reality is that I don't have the time or energy to aim for feature parity with git-annex, which offers a ton of useful options.

On the basis of a quick read of each, I am not sure fossil LFS gains much. LFS seems to have similar file limits to Fossil ('up to 2GB').

'fossil Lore' or even 'Lore fossil' might be just as easy (or equally hard) and give greater benefits. Rock solid and familiar Fossil, with good (though bleeding edge) large file handling.

The interface between the two is essentially large file metadata including a hash/signature held as text in Fossil, with the actual file elsewhere. On-demand loading of the file is a function of your IDE/development tools, not the VCS. There are details to handle like making sure that when a large file is changed the metadata file is also updated so that Fossil will know this is a changed file when it is committed, and deciding when the file content is uploaded - as soon as you change it, or only when committing?

I noticed the Lore concepts 'Link' and 'Layer' - ways of combining more than one repository into the same workspace. I'm guessing the other repo is also a Lore one, but I didn't read far enough to check that.

Warning: I am well known for my optimism around assessing how complicated something might be, and wishing I had kept my mouth shut...

Trevor

That sounds fairly similar to what I was thinking of, in fact I'm pretty sure "hash/link to actual binary file stored somewhere else" is exactly how Git LFS works.

Git also has started work on a built-in feature that's similar to Git LFS called large object promisors: https://git-scm.com/docs/large-object-promisors

The main idea being that the binary files are store on an external "object promisor" that you can host somewhere else

There's a fairly good blog post about it here: https://tylercipriani.com/blog/2025/08/15/git-lfs/

fossil is only tracking a symlink to the hash of the file

When you commit a symlink to a Fossil repo, it is only hashing the text of the target. Proof:

$ cd /somewhere/temporary
$ dd if=/dev/random of=target count=1k bs=1k
$ ln -s target source
$ fossil init foo.fossil
$ fossil open -f foo.fossil
$ fossil set allow-symlinks 1    # not enabled by default!
$ fossil add source
ADDED  source
$ fossil ci -m 'added source'
New_Version: c99df466e17707fc1064a6e708fa93327bc43fe06e2af737422dcb1ea96b3c18
$ fossil artifact c99df466e17707fc1064a6e708fa93327bc43fe06e2af737422dcb1ea96b3c18
C added\ssource
D 2026-06-18T01:07:53.372
F source 943056f259ad83a42bdb0a94a9d61758e6361f31c51a7fe03cb63c9858ad239b l
⋮
$ fossil artifact 943056f259ad83a42bdb0a94a9d61758e6361f31c51a7fe03cb63c9858ad239b
target%
$ echo -n 'target' | fossil sha3sum -
943056f259ad83a42bdb0a94a9d61758e6361f31c51a7fe03cb63c9858ad239b  -

Notice that the content referred to by the F card matches the SHA3 hash of the link target, without any terminator. (Thus echo -n.)

The other hashes will vary due to local effects, but this one will be the same no matter where or when you run it.

The implication for your plan should be clear, but just in case, I'll spell it out: nothing in this scheme helps Fossil find updated version of your blob when it changes. It only works if you use a name-based content store. It seems plain to me that you were expecting a content-addressable store instead, such that when the file content changes, the hash changes, giving Fossil a different hash to store to indicate that the content has changed. Z

large object promisors

I'm not sure I'd trust any repo that shipped me a script that ran under my user account on every checkout. 😳

What would be more tolerable is if that script was written in TH1 with effectively this API:

.fossil-settings/fetch-content 943056f259ad83a42bdb0a94a9d61758e6361f31c51a7fe03cb63c9858ad239b > ./relative/path/to/output/location.jpg

The fossil binary would call that script for each remote file in turn, with its output connected to a file on disk under the checkout, by its own enforcement.

This is enough to write a script to fetch an OCI artifact using TH1's http call, with suitable POST parameters for authentication.

(Why OCI? Because it is the vendor-neutral cloud-native alternative to proprietary and/or idiosyncratic content-addressable storage systems. I dare say most orgs have one already, if only via one of the big code forges or cloud providers.) Z

large object promisors

I'm not sure I'd trust any repo that shipped me an arbitrary shebang style script that ran under my user account on every checkout. 😳

What would be more tolerable is if that script was written in TH1 with effectively this API:

.fossil-settings/fetch-content \
  943056f259ad83a42bdb0a94a9d61758e6361f31c51a7fe03cb63c9858ad239b \
  > ./relative/path/to/output/location.jpg

The fossil binary would call only scripts provided as "settings," once for each remote file mentioned in a manifest and not locally present, with its output connected to a file on disk under the checkout, by its own enforcement. That I might trust.

This API should suffice to write scripts for fetching OCI artifacts using TH1's http[s] command, given suitable POST auth parameters.

(Why OCI? Because it is the vendor-neutral cloud-native alternative to proprietary and/or idiosyncratic content-addressable storage systems. I dare say most orgs have one already, if only via one of the big code forges or cloud providers.) Z

The main idea being that the binary files are store on an external "object promisor" that you can host somewhere else

The thing is, once you enter the "data warehouse" phase of asset management, there's a ton of functionality overlap between that and a VCS (be it Git or Fossil or other) or making backups. Presumably that's what Epic is on about.

But the other thing is that such advanced asset usage is multidimensional and usually core to the project itself. That is to say, whatever mapping you do to identify assets (let's say content hashing, because that's what it usually is these days), you probably need the app to be aware of it so that it can load (and often download) the correct resource. It's not just "v3 needs v3 assets" it's also "user has selected high resolution graphics and a French localization". That's all more within the realm of the build system rather than a version control system.

I'm sure what Epic has in Lore works great for their workflow. Whether or not anyone else can really make use of it comes down to whatever similar "promisor" dependencies they make. Most are probably going to be fine sticking with their current VCS and just storing whatever maps to maps of objects makes the most sense for them.

nothing in this scheme helps Fossil find updated version of your blob when it changes.

That's right, and that's also how git-annex works. When you add a file to git-annex, it is hashed and moved into a directory structure under .git/annex/objects, and set as chmod 444 to prevent writing. If you want to modify the file, you have to run git annex unlock first.

All of this locking and unlocking and symlinking makes it a little cumbersome to use as a general purpose VCS for large files (and it's also difficult to reimplement for fossil), but there are plenty of use cases that git-annex is great for.

(I wrote the original comment about git-annex in this thread. Didn't realize I wasn't logged in)

I have used a concocted arrangement of Fossil and Restic (restic.net) to achieve a very workable, albeit slightly cumbersome solution to binary "check-ins". By storing binary files - source graphics, modified graphics, audio and video files, etc., in a defined directory structure within projects, it is possible to create Restic repository check-ins named with the fossil check-in id. Its a manual process, but it works. I have had grand thoughts of using a hook on check-in to launch a script that would execute the restic binary and pass the input values. It comes to mind frequently amidst the busyness of the day.

Restic works at the block level and I also for checking in compressed 6 GB+ database dumps. I have done this over a long period of time while periodically restoring one at random for integrity testing. I have full confidence in Restic's ability to handle extremely large binary files, and when a project is done, I can efficiently archive both repositories.

I haven't given up on the idea of scripting a post-check-in hook, but time is the enemy there right now. My original set up with the two tools actually included the fossil repo in the restic directory check-in, but that can got pretty sketchy when it came to managing the check-outs. Maybe some value in these thoughts.