Fossil SCM
Fossil on Windows has trouble with Unicode charatcres in filenames
edab7bb5c973f41…
· opened 12 years, 2 months ago
- Type
- Build_Problem
- Priority
- —
- Severity
- Important
- Resolution
- Open
- Subsystem
- —
- Created
- Feb. 10, 2014 9:51 p.m.
Thank you for two great pieces of software. I've been using SQLite in projects of all sizes for the past eight years. When I discovered Fossil, I knew it would be the same; simple, powerful, correct!
After starting a project in Fossil, in part due to the support for Unicode filenames, I discovered that the Windows version only supports some characters. I started on v1.27 and later upgraded to v1.28, which is equally affected. When trying to add files with an unacceptable character in the name, fossil says "not found: [filename]" although the files show up when I run fossil extra. After running out of other ideas, I booted into Linux to try the same files there. With my NTFS volume mounted in Linux, I was able to add to the repo the files with troublesome characters in their names with absolutely no hassle. Booted back into Windows, I was then able to retrieve those files from the repo. So, fossil on Windows can see the files (they appear in extras), it can create the files on disk if they exist in the repo, but it can't add the files to the repo if they only exist on disk.
I am unsure what the exact problem is. Most files have only simple Latin characters without accents, but some have diacritic marks used in Slavic languages. Here is a summary of the accented characters encountered thus far in my project. Perhaps there is a pattern that is not obvious to me. Characters which are accepted include: é (e acute), í (i acute), š (s caron) Characters which are not accepted include: č (c caron), ě (e caron)
I considered the possibility that the real problem is cmd.exe since it shows only the base letters without accent. I have been using ? or * in place of accented characters and that seemed to work fine for characters in the first list but not those in the second. I tried running Fossil under PowerShell only to see it's exactly the same (why am I unsurprised Microsoft wrote a new shell without proper Unicode support). I tried the Fuel GUI to try to take the CLI out of the picture but it's the same situation. Fuel shows the files as untracked but attempting to add any of them results in the not found message.
I hope this report does not fall on deaf ears as threatened by the large red text. I attempted to discuss this on the mailing list before making the report, but was unable to do so. I sent a message to the list last night but as of this evening it still does not appear in the list archives. It is unclear if the list is moderated or completely closed to non-subscribers.
Comments (6)
Thank you for two great pieces of software. I've been using SQLite in projects of all sizes for the past eight years. When I discovered Fossil, I knew it would be the same; simple, powerful, correct!
After starting a project in Fossil, in part due to the support for Unicode filenames, I discovered that the Windows version only supports some characters. I started on v1.27 and later upgraded to v1.28, which is equally affected. When trying to add files with an unacceptable character in the name, fossil says "not found: [filename]" although the files show up when I run fossil extra. After running out of other ideas, I booted into Linux to try the same files there. With my NTFS volume mounted in Linux, I was able to add to the repo the files with troublesome characters in their names with absolutely no hassle. Booted back into Windows, I was then able to retrieve those files from the repo. So, fossil on Windows can see the files (they appear in extras), it can create the files on disk if they exist in the repo, but it can't add the files to the repo if they only exist on disk.
I am unsure what the exact problem is. Most files have only simple Latin characters without accents, but some have diacritic marks used in Slavic languages. Here is a summary of the accented characters encountered thus far in my project. Perhaps there is a pattern that is not obvious to me. Characters which are accepted include: é (e acute), í (i acute), š (s caron) Characters which are not accepted include: č (c caron), ě (e caron)
I considered the possibility that the real problem is cmd.exe since it shows only the base letters without accent. I have been using ? or * in place of accented characters and that seemed to work fine for characters in the first list but not those in the second. I tried running Fossil under PowerShell only to see it's exactly the same (why am I unsurprised Microsoft wrote a new shell without proper Unicode support). I tried the Fuel GUI to try to take the CLI out of the picture but it's the same situation. Fuel shows the files as untracked but attempting to add any of them results in the not found message.
I hope this report does not fall on deaf ears as threatened by the large red text. I attempted to discuss this on the mailing list before making the report, but was unable to do so. I sent a message to the list last night but as of this evening it still does not appear in the list archives. It is unclear if the list is moderated or completely closed to non-subscribers.
No, actually it's a problem in MinGW. See: [https://sourceforge.net/p/mingw/bugs/771/] If you compile with MinGW-w64 (32-bit or 64-bit) everything works fine.
Thank you for the pointer to the bug in MinGW. On Linux I compiled from source, but on Windows I have been using the binaries provided. If the cause of the Unicode handling problem is the way those binaries are compiled, then it is not a defect in the source code, but still a defect in the distributed binary code. Is there some particular reason to not compile with the corrected MinGW?
(Changing type from code_defect to build_problem in light of the information.)
Build instructions for Cross-compiling on Ubuntu (or any other OS which has a MinGW-w64 cross-compiler):
sudo apt-get install mingw-w64 cd /compat tar xfz /openssl-1.0.1f.tar.gz cd .. make -f win/Makefile.mingw.mistachkin PREFIX=i686-w64-mingw32- openssl fossil.exe i686-w64-mingw32-strip fossil.exe
That's it. (be patient, the "make" line takes about 5-10 minutes depending on the speed of your machine/hard-disk)
Fossil (1.29) binaries in which this problem is fixed can be downloaded from: [http://sourceforge.net/projects/cyqlite/files/fossil/] both for win32 and win64.
Fossil binaries in which this problem is fixed can be downloaded from: [https://sourceforge.net/projects/cyqlite/files/fossil/] both for win32 and win64