Fossil Forum

Now i fear that all my files are also stored within the .fossil database, and soft-deleted.

Fossil doesn’t “move” your files into the repo DB. You would have to go out of your way to delete the checkout copy, and even then, having checked them in previously, a fossil update will bring them back. (Until you fix the initial problem, that is.)

this repository is now ruined

Hasty generalization.

As long as you are in control of all clones, this guide will work.

It is only when others clone from that repo and do not wish to cooperate that you may have a problem here.

loaded into memory by sqlite every time.

Fossil does not load the entire repo into memory. It only loads the necessary pieces for each operation. The usual worst-case is diffing a file, where it can take ~3x the storage space of that file: the two copies it is diffing plus the diffs themselves, which can match the size of the original file when the difference is total.

Anything I could do or should have done?

You sound like you are already aware of the compressed content issue and its consequences. That doc is intended to push you toward plaintext file formats (e.g. LaTeX) over binary ones (PDF, DOCX) or at least to uncompressed binaries (TIFF, TGA) in preference to compressed ones (PNG, JPEG). The fact that you committed binary files without first adding globs for them to .fossil-settings/binary-glob is the actual issue here. It means you would have been warned about the binary files, then told Fossil to commit them anyway. Fossil did what you told it to.

GDPR Article 17 – Right to Erasure

That’s a legal question, not a technical one, and it wouldn’t be the first time a legislature has attempted to dictate realities beyond the bounds of reality.

I am not in the EU, so you should discount my opinion appropriately, but since you ask, I’ll tell you that I think the closest application between that law and the Fossil reality is that it gives the complainant the right to go to every clone of that Fossil repo and individually demand removal, and that each demand is enforceable only when the clone is also in the EU. Otherwise, I don’t see how you can enforce this, which then means subsequent syncs bring those artifacts back unless they have been separately shunned. Z faff72b4dc1c3

Now i fear that all my files are also stored within the .fossil database, and soft-deleted.

Fossil doesn’t “move” your files into the repo DB. You would have to go out of your way to delete the checkout copy, and even then, having checked them in previously, a fossil update will bring them back. (Until you fix the initial problem, that is.)

this repository is now ruined

Hasty generalization.

As long as you are in control of all clones, this guide will work.

It is only when others clone from that repo and do not wish to cooperate that you may have a problem here.

loaded into memory by sqlite every time.

Fossil does not load the entire repo into memory. It only loads the necessary pieces for each operation. The usual worst-case is diffing a file, where it can take ~3x the storage space of that file: the two copies it is diffing plus the diffs themselves, which can match the size of the original file when the difference is total.

Anything I could do or should have done?

You sound like you are already aware of the compressed content issue and its consequences. That doc is intended to push you toward plaintext file formats (e.g. LaTeX) over binary ones (PDF, DOCX) or at least to uncompressed binaries (TIFF, TGA) in preference to compressed ones (PNG, JPEG). The fact that you committed binary files without first adding globs for them to .fossil-settings/binary-glob is the actual issue here. It means you would have been warned about the binary files, then told Fossil to commit them anyway. Fossil did what you told it to.

GDPR Article 17 – Right to Erasure

That’s a legal question, not a technical one, and it wouldn’t be the first time a legislature has attempted to dictate behavior beyond the bounds of reality.

I am not in the EU, so you should discount my opinion appropriately, but since you ask, I’ll tell you that I think the closest application between that law and the Fossil reality is that it gives the complainant the right to go to every clone of that Fossil repo and individually demand removal, and that each demand is enforceable only when the clone is also in the EU. Otherwise, I don’t see how you can enforce this, which then means subsequent syncs bring those artifacts back unless they have been separately shunned. Z 18d3eb53a3199

Fossil tracks the history of your files, doing a fossil rm removes the files from the current tree, and fossil commit makes the change permanent. Even still, the old version of the tree remains, with all the old files. They are part of the fossil record, one might say.

it sounds like this was merely a test, and in that case, I just can say: you learned the lesson. Don't do that in a production repository if you don't want it. :-)

You really shouldn't try to delete content from Fossil, the system tries to fight you from doing it. Two allowances exist: shunning and purging. Shunning is somewhat nicer in a collaborative setting, since it only prohibits artifacts being transferred to or from a remote (typically your central server); purging can really remove artifacts from the historical record, and shouldn't be used when you have multiple clones. Feel free to try them in your tests, but in production, they ought to be extremely rarely deployed. (GDPR issues are a candidate for using shuns, for what it's worth.)

All in all, the best advise for what you "should have done" is don't do it in the first place. Be thankful it was a test.

TL;DR: Partially solved.

Yes. It is a Test Repo. I am noob with fossil. My goal is Not-yet a single text-only devel-project. I'm aware that diffs/delta on binaries is not recommended.

My wrong assumption was unversioned binary files or versionated by just renaming.

Original Events:

• First deleted the folder
• then with autosync enabled:
  • fossil update
  • fossil addremove
  • fossil commit -m "autosync" --no-warnings
• something went wrong, I believe a second update downloaded the files again.
• executed fossil rm FOLDER...

and it was probably there when I have created the bunch of deltas in the database?

Propossed Shunning method:

"A shunned artifact will not be pushed nor accepted in a pull and the artifact content will be purged from the repository the next time the repository is rebuilt."

Since I Do want to sync, the shunning/exceptuating binary files wont help.

Anyway, it could help to 'repair' the repo.fossil and remove useless artifacts from there after a fossil rebuild...

I tried going to /shun and it requires hashes. Not file names. Neither provides a list of all the hashes stored to be able to select from there... Then, what is the command?

I tried opening the repo.fossil with dbbrowser and the artifact view contains a large list of hashes without filenames for which i grabed one, and pasted it into /shun, shunned it, fossil rebuild it.. refreshed the database aaaanddd nothing!

Propossed Purging method:

Executed fossil purge files DELETED_FOLDER_NAME

And now the repo.fossil is 974 Mb

I believe this recovered the database.

Yet, I am still able to see the folder the web under 'Files→All'

.../repository/FossilTest/dir?name=DELETED_FOLDER_NAME

Now on my objective workflow

How can I tell fossil to avoid deltas or blob storage of binary files but just record the upload and keep the file in-sync?

And what if

Since I wouldn't use it as a project-store, but let's say as a DMS storage, I may not need super long term Deltas at all. How can I reset/clear the stored diffs after a 'release' is made. Z

fossil addremove

This is your mistake. You would be better off if you explicitly add the files you want fossil to track.

Think of 'fossil add' as 'add this file (name) to the list of things I want Fossil to track'.

'addremove' is a shortcut meaning 'if you see any files in the checkout which I haven't already told you about, add them'.

Neither add nor addremove do anything with the repo, the fun starts when you say 'commit'.

Whenever you use something new, use the -n|--dry-run option to check what is about to happen.

As to the repo - start a new one. Messing with things like 'shun' is likely to make things worse rather than better.

Trevor

Since I Do want to sync, the shunning/exceptuating binary files wont help.

And from your original post:

I have been using github to sync some personal but work-related documents and notes that I do in my work computer and my personal one.

Is this the same 'sync'? If so I wonder if you have a clear understanding of what 'sync' means to Fossil.

It has nothing to do with copying the contents of a checkout to/from a local repository - your repo.fossil file.

'sync' only comes into it when you have another copy (clone) of the repository somewhere else, and you have the choice of either keeping it in step automatically or only pushing changes when you say so. It is synchronising repositories, not checkouts. To do the equivalent of what (I think) you are doing with github you would need to host a fossil repo somewhere and then clone it to each of your work and personal computers. But I suspect that still has the issue that you have to remember to commit before you switch from one computer to the other.

Have you considered applications specifically designed for this, e.g. Syncthing? I use that for replicating folders to several different machines, including checkout folders. Committing etc is always done from the same machine though.

The /info page for a commit links to the /file pages involved, and at the top of each is a Shun button that pre-populates the /shun form.

Yes, several clicks, but shunning isn’t meant to be easy. Z

I will consider your reference to syncthing if it has a free cloud.

Yes. I have used GitHub so far as a solution for a while as syncing tool and it works and makes life easier with a free cloud service keeping my important docs in private repo and a GitHub desktop quickly highlighting the changed docs. What (you think) is correct. Just syncing the files. No real developer usage.

I found fossil looking for something smaller weight and portable than git without installation and liked it's ground design concepts so far.

Yet, if the db goes crazy when dropping binaries to the synced folder. Then is different than my current solution.

I would guess an unversioned setting for binaries must be there around.

I would guess an unversioned setting for binaries must be there around.

See and .