Fossil SCM
Non-ASCII characters in file and folder names are not handled correctly
e399bc1edfe45b2…
· opened 13 years, 4 months ago
- Type
- Code_Defect
- Priority
- —
- Severity
- Critical
- Resolution
- Fixed
- Subsystem
- —
- Created
- Nov. 28, 2012 9:54 p.m.
Using the latest version 1.24 the issue occurs after the following steps:
A. For the folder and filenames:
# Create a folder named "MañósosCarácteres" # Create a file named "cönáñón" inside the new folder # fossil addremove # fossil commit -m "Añadí una carpeta y un archivo con tildes"
B. then on a different machine # fossil update The new folder named "MañósosCarácteres" is now available # Add a new file called "éstedesdemác.rst" to the folder "MañósosCarácteres" # fossil commit -m "Añadí nuevo archivo con tildes desde mi mac"
C. Back to the first machine # fossil update A second folder with the name "MañósosCarácteres" has been added. How it can add a folder with exactly the same name and have the file system accept this I do not understand. It happens on both Ubuntu 12.04LTS and OSX10.7.
One folder contains both files, and the other contains only the first file ("cönáñón").
In batch of files I was working on for a customer, that contained many non-ASCII characters, sometimes the file names would suddenly duplicate with inverse accents (going from á to à). If you need more examples let me know and I will do a similar run with all non-ascii characters that we use in Spanish.
Comments (3)
Konstantin Khomoutov writes on the fossil-users mailing list:
I'm just handwaving, but Git's code base recently received some modifications to specifically deal with issues a native Mac OS X filesystem have with regard to UTF-8. AFAIK the deal was about that filesystem pefrorming one of standard UTF-8 normalizations either when writing or when reading (or both) so that when you create a directory entry and then read it back, you might get an octet string different from that you wrote.
See the extensive commit message in [1] and [2] in general.
- [https://github.com/git/git/commit/76759c7dff53e8c84e975b88cb8245587c14c7ba]
- [http://en.wikipedia.org/wiki/HFS_Plus]
I checked this a bit further to include a Windows 7 environment. Win 7 and OSX 10.7 respond differently
A. On the Ubuntu 12.04LTS machine # add the folder and "WiThCaPiTals" and "WiThCaPiTalsó" # In both folders "withcapitals" and "WiThCaPiTalsó" add the file "FilEWithCapItals.rst" # In the folder "withcapitals" add the file "filewithcapitals.rst"
B. Update the repo on OSX 10.7 # fossil: changes 3 files modified # fossil: WARNING: 1 unmanaged files were overwritten # the result is one folder: "WiThCaPiTalsó" containing the file "FilEWithCapItals.rst" # Two folders and two files dissapeared.
B. Update the repo on Windows 7 # fossil: changes 2 files modified # fossil: WARNING: 1 unmanaged files were overwritten # the result is two folders: "WiThCaPiTals" and "WiThCaPiTalsó" both containing the file "FilEWithCapItals.rst" # One folder and one file dissapeared
It looks indeed as if this is caused by how the underlying OS deals with capitalization and non-ascii characters. I used fossil for a workshop on version control and some participants had issues like the change of direction of the accent. But I have not been able to replicate those. Neither have I been able to replicate the issues in the Wiki we saw, that is why I have not reported it separately.
See also: [http://en.wikipedia.org/wiki/Unicode_equivalence#Errors_due_to_normalization_differences]