Re: git archive --format zip utf-8 issues
From: René Scharfe <hidden>
Date: 2016-06-15 22:54:51
Hi, I found a way to make unzip respect the UTF-8 flag in ZIP files: Apparently (from looking at the source) an extended field needs to be present in order for it to even look at general purpose flag 11. I sent a patch to add an extended timestamp field that fits the bill. Here are new numbers on ZIP international filename compatibility: 7-Zip PeaZip builtin unzip unzip unzip 7z Windows Windows Windows Linux mingw Windows Linux git Linux 1 1 1 7 1 1 1 git 1 Linux 37 37 1 7 1 1 37 git 2 Linux 37 37 1 37 1 1 37 git 3 Linux 37 37 1 37 15 15 37 git mingw 1 1 1 7 1 1 1 git 1 mingw 37 37 1 7 1 1 37 git 2 mingw 37 37 1 37 1 1 37 git 3 mingw 37 37 1 37 15 15 37 7-Zip Windows 37 37 14 24 15 15 24 PeaZip Windows 37 37 14 24 15 15 24 zip Linux 37 37 1 37 15 15 37 zip Windows 14 14 0 37 15 15 1 builtin Windows 14 14 14 1 14 14 1 The test corpus still consists of 37 files based on the pangrams on the following web page: http://www.columbia.edu/~fdc/utf8/index.html#quickbrownfox The files can be created using the attached script. It also provides a check command to count the files with correct names, and the results of that for different ZIP extractors are give in the table. The built-in ZIP functionality on Windows was only able to pack 14 of the 37 files, which explains the low score across the board for this packer. "git 1" is the patch "archive-zip: support UTF-8 paths" added, which let's archive-zip make use of the UTF-8 flag. "git 2" is "git 1" plus the patch "archive-zip: declare creator to be Unix for UTF-8 paths". Both have been posted before. "git 3" is "git 1" plus the new patch "archive-zip: write extended timestamp". Let's drop patch 2 (Unix as creator) and keep patches 1 (UTF-8 flag) and 3 (mtime field) to make archive-zip record non-ASCII filenames in a portable way. It's not perfect, but I don't know how to do any better given that Windows' built-in ZIP functionality expects filenames in the local code page and with an international audience for projects distributing ZIP files. René
Attachments
- pangrams.sh [text/plain] 2367 bytes · preview