[PATCH] Limited git-gui to displaying 5000 new files

When there is a large number of new or modified files,
"display_all_files" takes a long time, and git-gui appears to
hang. Limit the display to 5000 files, by default. This number
is configurable as gui.maxfilesdisplayed.

Show a warning if the list of files is truncated.

Signed-off-by: Dan Zwell <redacted>
---
 git-gui.sh |   17 ++++++++++++++++-
 1 files changed, 16 insertions(+), 1 deletions(-)

diff --git a/git-gui.sh b/git-gui.sh
index 14b92ba..5a20923 100755
--- a/git-gui.sh
+++ b/git-gui.sh

@@ -745,6 +745,8 @@ set default_config(gui.newbranchtemplate) {}
 set default_config(gui.spellingdictionary) {}
 set default_config(gui.fontui) [font configure font_ui]
 set default_config(gui.fontdiff) [font configure font_diff]
+# TODO: this option should be added to the git-config documentation
+set default_config(gui.maxfilesdisplayed) 5000
 set font_descs {
 	{fontui   font_ui   {mc "Main Font"}}
 	{fontdiff font_diff {mc "Diff/Console Font"}}

@@ -1702,6 +1704,8 @@ proc display_all_files {} {
 	global ui_index ui_workdir
 	global file_states file_lists
 	global last_clicked
+	global files_warning
+	global default_config
 
 	$ui_index conf -state normal
 	$ui_workdir conf -state normal

@@ -1713,7 +1717,18 @@ proc display_all_files {} {
 	set file_lists($ui_index) [list]
 	set file_lists($ui_workdir) [list]
 
-	foreach path [lsort [array names file_states]] {
+	set to_display [lsort [array names file_states]]
+	set display_limit $default_config(gui.maxfilesdisplayed)
+	if {[llength $to_display] > $display_limit} {
+		if {![info exists files_warning] || !$files_warning} {
+			set warning "Displaying only $display_limit of "
+			append warning "[llength $to_display] files."
+			info_popup [mc $warning]
+			set files_warning 1
+		}
+		set to_display [lrange $to_display 0 [expr {$display_limit-1}]]
+	}
+	foreach path $to_display {
 		set s $file_states($path)
 		set m [lindex $s 0]
 		set icon_name [lindex $s 1]

-- 
1.6.3.3

Dan Zwell [off-list ref] wrote:

When there is a large number of new or modified files,
"display_all_files" takes a long time, and git-gui appears to
hang. Limit the display to 5000 files, by default. This number
is configurable as gui.maxfilesdisplayed.

Show a warning if the list of files is truncated.

This should use [get_config gui.maxfilesdisplayed] so that the
user can actually set this property in a configuration file and
have git-gui honor it.  Reading from $default_config means you are
only looking at the hardcoded value you set in git-gui.sh.

+	if {[llength $to_display] > $display_limit} {
+		if {![info exists files_warning] || !$files_warning} {

Wouldn't it be easier to just set files_warning to 0 at the start
of the script, so that you don't need to do this info exists test?

+			set warning "Displaying only $display_limit of "
+			append warning "[llength $to_display] files."
+			info_popup [mc $warning]

This needs to be in the translated strings.

-- 
Shawn.

On Mon, Aug 10, 2009 at 17:38, Shawn O. Pearce[off-list ref] wrote:

quoted

+                     set warning "Displaying only $display_limit of "
+                     append warning "[llength $to_display] files."
+                     info_popup [mc $warning]

This needs to be in the translated strings.

Will do, as soon as this hits git://repo.or.cz/git-gui.git.
Or should I have looked at the internationalization repo
git://repo.or.cz/git-gui/git-gui-i18n.git?

Alex Riesen [off-list ref] wrote:

On Mon, Aug 10, 2009 at 17:38, Shawn O. Pearce[off-list ref] wrote:

quoted

quoted

+ ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? set warning "Displaying only $display_limit of "
+ ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? append warning "[llength $to_display] files."
+ ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? info_popup [mc $warning]

This needs to be in the translated strings.

Will do, as soon as this hits git://repo.or.cz/git-gui.git.
Or should I have looked at the internationalization repo
git://repo.or.cz/git-gui/git-gui-i18n.git?

No, I meant that the patch needs to use [mc] or whatever it is
to enable the string to be localized.  After its marked with the
necessary code change, I can apply the patch, regenerate the .pot,
and let translators update their .po when they have time.

-- 
Shawn.

Shawn O. Pearce wrote:

Alex Riesen [off-list ref] wrote:

quoted

On Mon, Aug 10, 2009 at 17:38, Shawn O. Pearce[off-list ref] wrote:

quoted

quoted

+ ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? set warning "Displaying only $display_limit of "
+ ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? append warning "[llength $to_display] files."
+ ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? info_popup [mc $warning]

This needs to be in the translated strings.

Will do, as soon as this hits git://repo.or.cz/git-gui.git.
Or should I have looked at the internationalization repo
git://repo.or.cz/git-gui/git-gui-i18n.git?

No, I meant that the patch needs to use [mc] or whatever it is
to enable the string to be localized.  After its marked with the
necessary code change, I can apply the patch, regenerate the .pot,
and let translators update their .po when they have time.

I will change it to use [mc] (and make the other changes that Shawn 
mentioned) and resubmit the patch.

-Dan

When there is a large number of new or modified files,
"display_all_files" takes a long time, and git-gui appears to
hang. This change limits the number of files that are displayed.
This limit can be set as gui.maxfilesdisplayed, and is
5000 by default.

A warning is shown when the list of files is truncated.

Signed-off-by: Dan Zwell <redacted>
---
By the way, is the right way to deal with strings to be
translated? See the end of the patch.

 git-gui.sh     |   18 +++++++++++++++++-
 po/git-gui.pot |    5 +++++
 2 files changed, 22 insertions(+), 1 deletions(-)

diff --git a/git-gui.sh b/git-gui.sh
index 3c0ce26..a4dde9e 100755
--- a/git-gui.sh
+++ b/git-gui.sh

@@ -745,6 +745,8 @@ set default_config(gui.newbranchtemplate) {}
 set default_config(gui.spellingdictionary) {}
 set default_config(gui.fontui) [font configure font_ui]
 set default_config(gui.fontdiff) [font configure font_diff]
+# TODO: this option should be added to the git-config documentation
+set default_config(gui.maxfilesdisplayed) 5000
 set font_descs {
 	{fontui   font_ui   {mc "Main Font"}}
 	{fontdiff font_diff {mc "Diff/Console Font"}}

@@ -1698,10 +1700,12 @@ proc display_all_files_helper {w path icon_name m} {
 	$w insert end "[escape_path $path]\n"
 }
 
+set files_warning 0
 proc display_all_files {} {
 	global ui_index ui_workdir
 	global file_states file_lists
 	global last_clicked
+	global files_warning
 
 	$ui_index conf -state normal
 	$ui_workdir conf -state normal

@@ -1713,7 +1717,19 @@ proc display_all_files {} {
 	set file_lists($ui_index) [list]
 	set file_lists($ui_workdir) [list]
 
-	foreach path [lsort [array names file_states]] {
+	set to_display [lsort [array names file_states]]
+	set display_limit [get_config gui.maxfilesdisplayed]
+	if {[llength $to_display] > $display_limit} {
+		if {!$files_warning} {
+			# do not repeatedly warn:
+			set files_warning 1
+			set warning "Displaying only $display_limit of "
+			append warning "[llength $to_display] files."
+			info_popup [mc $warning]
+		}
+		set to_display [lrange $to_display 0 [expr {$display_limit-1}]]
+	}
+	foreach path $to_display {
 		set s $file_states($path)
 		set m [lindex $s 0]
 		set icon_name [lindex $s 1]

diff --git a/po/git-gui.pot b/po/git-gui.pot
index 53b7d36..fb60472 100644
--- a/po/git-gui.pot
+++ b/po/git-gui.pot

@@ -90,6 +90,11 @@ msgstr ""
 msgid "Ready."
 msgstr ""
 
+#: git-gui.sh:1725
+#, tcl-format
+msgid "Displaying only %s of %s files."
+msgstr ""
+
 #: git-gui.sh:1819
 msgid "Unmodified"
 msgstr ""

-- 
1.6.4


Shawn O. Pearce wrote:

> Dan Zwell <dzwell@gmail.com> wrote:
>   
>> When there is a large number of new or modified files,
>> "display_all_files" takes a long time, and git-gui appears to
>> hang. Limit the display to 5000 files, by default. This number
>> is configurable as gui.maxfilesdisplayed.
>>
>> Show a warning if the list of files is truncated.
>>     
>
>   
>> @@ -1713,7 +1717,18 @@ proc display_all_files {} {
>> 	set file_lists($ui_index) [list]
>> 	set file_lists($ui_workdir) [list]
>>
>> -	foreach path [lsort [array names file_states]] {
>> +	set to_display [lsort [array names file_states]]
>> +	set display_limit $default_config(gui.maxfilesdisplayed)
>>     
>
> This should use [get_config gui.maxfilesdisplayed] so that the
> user can actually set this property in a configuration file and
> have git-gui honor it.  Reading from $default_config means you are
> only looking at the hardcoded value you set in git-gui.sh.
>
>   
>> +	if {[llength $to_display] > $display_limit} {
>> +		if {![info exists files_warning] || !$files_warning} {
>>     
>
> Wouldn't it be easier to just set files_warning to 0 at the start
> of the script, so that you don't need to do this info exists test?
>
>   
>> +			set warning "Displaying only $display_limit of "
>> +			append warning "[llength $to_display] files."
>> +			info_popup [mc $warning]
>>     
>
> This needs to be in the translated strings.
>
>   

Dan Zwell [off-list ref] wrote:

When there is a large number of new or modified files,
"display_all_files" takes a long time, and git-gui appears to
hang. This change limits the number of files that are displayed.
This limit can be set as gui.maxfilesdisplayed, and is
5000 by default.

A warning is shown when the list of files is truncated.

Signed-off-by: Dan Zwell <redacted>
---
By the way, is the right way to deal with strings to be
translated? See the end of the patch.

No.

+			set warning "Displaying only $display_limit of "
+			append warning "[llength $to_display] files."
+			info_popup [mc $warning]

This should be:

info_popup [mc "Displaying only %s of %s files." $display_limit [llength $to_display]]

+msgid "Displaying only %s of %s files."
+msgstr ""

So that then the placeholders are available here...

-- 
Shawn.

When there is a large number of new or modified files,
"display_all_files" takes a long time, and git-gui appears to
hang. This change limits the number of files that are displayed.
This limit can be set as gui.maxfilesdisplayed, and is
5000 by default.

A warning is shown when the list of files is truncated.

Signed-off-by: Dan Zwell <redacted>
---
 git-gui.sh     |   17 ++++++++++++++++-
 po/git-gui.pot |    5 +++++
 2 files changed, 21 insertions(+), 1 deletions(-)

diff --git a/git-gui.sh b/git-gui.sh
index 3c0ce26..eae1f81 100755
--- a/git-gui.sh
+++ b/git-gui.sh

@@ -745,6 +745,8 @@ set default_config(gui.newbranchtemplate) {}
 set default_config(gui.spellingdictionary) {}
 set default_config(gui.fontui) [font configure font_ui]
 set default_config(gui.fontdiff) [font configure font_diff]
+# TODO: this option should be added to the git-config documentation
+set default_config(gui.maxfilesdisplayed) 5000
 set font_descs {
 	{fontui   font_ui   {mc "Main Font"}}
 	{fontdiff font_diff {mc "Diff/Console Font"}}

@@ -1698,10 +1700,12 @@ proc display_all_files_helper {w path icon_name m} {
 	$w insert end "[escape_path $path]\n"
 }
 
+set files_warning 0
 proc display_all_files {} {
 	global ui_index ui_workdir
 	global file_states file_lists
 	global last_clicked
+	global files_warning
 
 	$ui_index conf -state normal
 	$ui_workdir conf -state normal

@@ -1713,7 +1717,18 @@ proc display_all_files {} {
 	set file_lists($ui_index) [list]
 	set file_lists($ui_workdir) [list]
 
-	foreach path [lsort [array names file_states]] {
+	set to_display [lsort [array names file_states]]
+	set display_limit [get_config gui.maxfilesdisplayed]
+	if {[llength $to_display] > $display_limit} {
+		if {!$files_warning} {
+			# do not repeatedly warn:
+			set files_warning 1
+			info_popup [mc "Displaying only %s of %s files." \
+				$display_limit [llength $to_display]]
+		}
+		set to_display [lrange $to_display 0 [expr {$display_limit-1}]]
+	}
+	foreach path $to_display {
 		set s $file_states($path)
 		set m [lindex $s 0]
 		set icon_name [lindex $s 1]

diff --git a/po/git-gui.pot b/po/git-gui.pot
index 53b7d36..074582d 100644
--- a/po/git-gui.pot
+++ b/po/git-gui.pot

@@ -90,6 +90,11 @@ msgstr ""
 msgid "Ready."
 msgstr ""
 
+#: git-gui.sh:1726
+#, tcl-format
+msgid "Displaying only %s of %s files."
+msgstr ""
+
 #: git-gui.sh:1819
 msgid "Unmodified"
 msgstr ""

-- 
1.6.4

Dan Zwell [off-list ref] wrote:

When there is a large number of new or modified files,
"display_all_files" takes a long time, and git-gui appears to
hang. This change limits the number of files that are displayed.
This limit can be set as gui.maxfilesdisplayed, and is
5000 by default.

A warning is shown when the list of files is truncated.

Thanks, applied.

-- 
Shawn.

Signed-off-by: Alex Riesen <redacted>
---

Shawn O. Pearce, Wed, Aug 12, 2009 16:43:45 +0200:

Dan Zwell [off-list ref] wrote:

quoted

When there is a large number of new or modified files,
"display_all_files" takes a long time, and git-gui appears to
hang. This change limits the number of files that are displayed.
This limit can be set as gui.maxfilesdisplayed, and is
5000 by default.

A warning is shown when the list of files is truncated.

Thanks, applied.

And the updated translation.

But now, when I really think about the change, it looks useless.
What has the _number_ of files has to do with the files you actually
have to handle? As the sorting of the file list cannot be changed (and
it wouldn't be a big help anyway), you have no chance to get to your
file if it happens to be past the limit!

Wouldn't a pathname/glob filter in the command-line (or file/path
selection dialog) to limit the scope be more appropriate and useful?
And have the file list reading to happen in background, as gitk does?

 po/ru.po |   28 ++++++++++++++++------------
 1 files changed, 16 insertions(+), 12 deletions(-)

diff --git a/po/ru.po b/po/ru.po
index 0ffc4a4..364c074 100644
--- a/po/ru.po
+++ b/po/ru.po

@@ -90,12 +90,18 @@ msgstr "Вызов программы поддержки репозитория
 
 #: git-gui.sh:1384
 msgid "Commit declined by prepare-commit-msg hook."
-msgstr "Сохранение прервано программой поддержки репозитория prepare-commit-msg"
+msgstr ""
+"Сохранение прервано программой поддержки репозитория prepare-commit-msg"
 
 #: git-gui.sh:1542 lib/browser.tcl:246
 msgid "Ready."
 msgstr "Готово."
 
+#: git-gui.sh:1726
+#, tcl-format
+msgid "Displaying only %s of %s files."
+msgstr "Показано %s из %s файлов."
+
 #: git-gui.sh:1819
 msgid "Unmodified"
 msgstr "Не изменено"

@@ -1297,8 +1303,8 @@ msgid ""
 msgstr ""
 "Невозможно исправить состояние во время операции слияния.\n"
 "\n"
-"Текущее слияние не завершено. Невозможно исправить предыдущее "
-"сохраненное состояние, не прерывая эту операцию.\n"
+"Текущее слияние не завершено. Невозможно исправить предыдущее сохраненное "
+"состояние, не прерывая эту операцию.\n"
 
 #: lib/commit.tcl:48
 msgid "Error loading commit data for amend:"

@@ -1723,8 +1729,7 @@ msgid ""
 msgstr ""
 "Невозможно выполнить слияние во время исправления.\n"
 "\n"
-"Завершите исправление данного состояния перед выполнением операции "
-"слияния.\n"
+"Завершите исправление данного состояния перед выполнением операции слияния.\n"
 
 #: lib/merge.tcl:27
 msgid ""

@@ -1888,8 +1893,8 @@ msgstr ""
 #, tcl-format
 msgid "File %s seems to have unresolved conflicts, still stage?"
 msgstr ""
-"Файл %s кажется содержит необработаные конфликты. "
-"Продолжить подготовку к сохранению?"
+"Файл %s кажется содержит необработаные конфликты. Продолжить подготовку к "
+"сохранению?"
 
 #: lib/mergetool.tcl:60
 #, tcl-format

@@ -2213,8 +2218,8 @@ msgid ""
 "One or more of the merge tests failed because you have not fetched the "
 "necessary commits.  Try fetching from %s first."
 msgstr ""
-"Некоторые тесты на слияние не прошли, потому что Вы не "
-"получили необходимые состояния. Попытайтесь получить их из %s."
+"Некоторые тесты на слияние не прошли, потому что Вы не получили необходимые "
+"состояния. Попытайтесь получить их из %s."
 
 #: lib/remote_branch_delete.tcl:207
 msgid "Please select one or more branches to delete."

@@ -2381,8 +2386,8 @@ msgstr "Выполнение: %s"
 
 #: lib/tools.tcl:149
 #, tcl-format
-msgid "Tool completed succesfully: %s"
-msgstr "Программа %s успешно завершилась."
+msgid "Tool completed successfully: %s"
+msgstr "Программа %s завершилась успешно."
 
 #: lib/tools.tcl:151
 #, tcl-format

@@ -2538,4 +2543,3 @@ msgstr "Использовать thin pack (для медленных сетев
 #: lib/transport.tcl:179
 msgid "Include tags"
 msgstr "Передать метки"
-

-- 
1.6.4.140.gc6dfd

Alex Riesen [off-list ref] wrote:

Shawn O. Pearce, Wed, Aug 12, 2009 16:43:45 +0200:

quoted

Dan Zwell [off-list ref] wrote:

quoted

When there is a large number of new or modified files,
"display_all_files" takes a long time, and git-gui appears to
hang. This change limits the number of files that are displayed.
This limit can be set as gui.maxfilesdisplayed, and is
5000 by default.

A warning is shown when the list of files is truncated.

Thanks, applied.

But now, when I really think about the change, it looks useless.
What has the _number_ of files has to do with the files you actually
have to handle? As the sorting of the file list cannot be changed (and
it wouldn't be a big help anyway), you have no chance to get to your
file if it happens to be past the limit!

Wouldn't a pathname/glob filter in the command-line (or file/path
selection dialog) to limit the scope be more appropriate and useful?
And have the file list reading to happen in background, as gitk does?

Good point.  I suspect the problem wasn't so much with Tcl doing the
list processing as it was with Tk actually creating the underlying
icons and stuff for each file name.  But with the list clipped,
you are right, you are basically SOL.  You can't do much beyond
dropping back to the CLI and using the CLI tools.

IMHO, if we aren't going to handle 20k file names, we should at least
punt and tell the user we aren't going to handle 20k file names,
rather than just play Outlook wannabe and lockup the entire UI until
the user gets bored and kill -9's us.  So this patch is better than
nothing, it at least lets the user know we have given up on them.

-- 
Shawn.

Alex Riesen [off-list ref] wrote:

quoted

But now, when I really think about the change, it looks useless.
What has the _number_ of files has to do with the files you actually
have to handle? As the sorting of the file list cannot be changed (and
it wouldn't be a big help anyway), you have no chance to get to your
file if it happens to be past the limit!

Good point.  I suspect the problem wasn't so much with Tcl doing the
list processing as it was with Tk actually creating the underlying
icons and stuff for each file name.  But with the list clipped,
you are right, you are basically SOL.  You can't do much beyond
dropping back to the CLI and using the CLI tools.

IMHO, if we aren't going to handle 20k file names, we should at least
punt and tell the user we aren't going to handle 20k file names,
rather than just play Outlook wannabe and lockup the entire UI until
the user gets bored and kill -9's us.  So this patch is better than
nothing, it at least lets the user know we have given up on them.

A warning would be good, but this gives users more information. In my 
case, I only wanted to see a few files, but the rest were from a 
directory that should have been in .gitignore. If I had seen the 
filenames, I would have known which directory was the culprit. This way, 
you don't need to drop to the command line to fix the problem.

-Dan