Fossil SCM

Allow deconstruct to ignore already existing files

Open

72172cc8184d0b3… · opened 11 years, 3 months ago

Type
Feature_Request
Priority
Severity
Cosmetic
Resolution
Open
Subsystem
Created
Dec. 31, 2014 2:28 a.m.

Request: Add a command line switch to deconstruct to tell fossil to ignore/skip pre-existing files already present. For the purposes of explanation below I'll call the switch "--keep" to match other existing options already available.

The problem:

When using "fossil deconstruct" currently fossil will create new files of the entire repository and will overwrite any old files. This works fine but creates two problems.

1) For large projects it takes a long time to deconstruct. 2) Backup program see all the files as new and needlessly backup everything again.

What would be nicer is to have an option to tell fossil to skip files that are already present (and not alter the date modified). Then the above two problems can be eased because 1) only new entries in the repository are written to disk so the process is faster, and 2) backup programs only need to makes copies of the new files, not the entire set.

Since all the file names are the SHA1 hash of the contents, and are immutable, we are guaranteed that the contents have not changed and thus don't need rewriting. Naturally if the user decides to edit such a file that is their responsibility, and their problem, if they alter things. And they can always do a full rebuild again later by deconstructing without the --keep switch.

Comments (2)

nobody 11 years, 3 months ago

Request: Add a command line switch to deconstruct to tell fossil to ignore/skip pre-existing files already present. For the purposes of explanation below I'll call the switch "--keep" to match other existing options already available.

The problem:

When using "fossil deconstruct" currently fossil will create new files of the entire repository and will overwrite any old files. This works fine but creates two problems.

1) For large projects it takes a long time to deconstruct. 2) Backup program see all the files as new and needlessly backup everything again.

What would be nicer is to have an option to tell fossil to skip files that are already present (and not alter the date modified). Then the above two problems can be eased because 1) only new entries in the repository are written to disk so the process is faster, and 2) backup programs only need to makes copies of the new files, not the entire set.

Since all the file names are the SHA1 hash of the contents, and are immutable, we are guaranteed that the contents have not changed and thus don't need rewriting. Naturally if the user decides to edit such a file that is their responsibility, and their problem, if they alter things. And they can always do a full rebuild again later by deconstructing without the --keep switch.

nobody 11 years, 2 months ago

I found that the following small change to rebuild.c will successfully skip existing files during deconstruction.

But presently the command line option "--keep" isn't read so it is currently always active:

File src/rebuild.c

   216    216   */
   217    217   static void rebuild_step(int rid, int size, Blob *pBase){
   218    218     static Stmt q1;
   219    219     Bag children;

   220    220     Blob copy;
   221    221     Blob *pUse;
   222    222     int nChild, i, cid;
          223  +  FILE *out;

   223    224  
   224    225     while( rid>0 ){
   225    226  
   226    227       /* Fix up the "blob.size" field if needed. */
   227    228       if( size!=blob_size(pBase) ){

   228    229         db_multi_exec(
   229    230            "UPDATE blob SET size=%d WHERE rid=%d", blob_size(pBase), rid
................................................................................
   254    255         /* We are doing "fossil rebuild" */
   255    256         manifest_crosslink(rid, pUse, MC_NONE);

   256    257       }else{
   257    258         /* We are doing "fossil deconstruct" */
   258    259         char *zUuid = db_text(0, "SELECT uuid FROM blob WHERE rid=%d", rid);
   259    260         char *zFile = mprintf(zFNameFormat /*works-like:"%s:%s"*/,
   260    261                               zUuid, zUuid+prefixLength);

   261         -      blob_write_to_file(pUse,zFile);
          262  +      out = fopen(zFile, "r");
          263  +      fclose(out);

          264  +      if( out==0 ){
          265  +        blob_write_to_file(pUse,zFile);
          266  +      }

   262    267         free(zFile);
   263    268         free(zUuid);
   264    269         blob_reset(pUse);
   265    270       }
   266    271       assert( blob_is_reset(pUse) );

   267    272       rebuild_step_done(rid);
   268    273  

Keyboard Shortcuts

Open search /
Next entry (timeline) j
Previous entry (timeline) k
Open focused entry Enter
Show this help ?
Toggle theme Top nav button