[StGit RFC] Make "stg branch -l" faster by getting all git config information in one call

This isn't a real patch yet, but it is good enough for my usage.

I have a fair amount of branches, and I noticed that "stg branch -l"
takes ridiculously long to finish. The problem is that it creates
stack objects for all branches, and indiviudally extract information
about them one after one.  With 20 branches, it took almost 2 seconds
to run.  Compare that with the 0.01 seconds it takes to run "git
branch".

I made a patch that uses "git config --get-regexp" to get the
description and stgit.stackformatversion options for all branches at
once, and ignore the "protected" flag that I don't use. With this
change, I'm almost down to half a second, which almost makes it
usable.

There are still a bunch of redundant invokations of git, but python
startup times are hard to get around.

Maybe someone can help me find a quicker replacement for the
get_protected call?

---

 stgit/commands/branch.py |   39 ++++++++++++++++++++++++++++++---------
 1 files changed, 30 insertions(+), 9 deletions(-)

diff --git a/stgit/commands/branch.py b/stgit/commands/branch.py
index 50684bb..f4b0c33 100644
--- a/stgit/commands/branch.py
+++ b/stgit/commands/branch.py

@@ -72,21 +72,21 @@ options = [make_option('-c', '--create',
 def __is_current_branch(branch_name):
     return crt_series.get_name() == branch_name
 
-def __print_branch(branch_name, length):
+def __print_branch(branch_name, description, version, length):
     initialized = ' '
     current = ' '
     protected = ' '
 
-    branch = stack.Series(branch_name)
+    #branch = stack.Series(branch_name)
 
-    if branch.is_initialised():
+    if version != None:
         initialized = 's'
     if __is_current_branch(branch_name):
         current = '>'
-    if branch.get_protected():
-        protected = 'p'
+    #if branch.get_protected():
+    #    protected = 'p'
     out.stdout(current + ' ' + initialized + protected + '\t'
-               + branch_name.ljust(length) + '  | ' + branch.get_description())
+               + branch_name.ljust(length) + '  | ' + (description or ''))
 
 def __delete_branch(doomed_name, force = False):
     doomed = stack.Series(doomed_name)

@@ -100,6 +100,23 @@ def __delete_branch(doomed_name, force = False):
     doomed.delete(force)
     out.done()
 
+class FormatException(StgException):
+    pass
+
+def __get_all_branch_config(key):
+    key = re.escape(key)
+    lines = git.GRun('config', '--get-regexp',
+                     r'branch\..*\.'+key).returns([0,1]).output_lines()
+    val_re = re.compile(r'branch\.(.*)\.%s (.*)' % key)
+    result = {}
+    for line in lines:
+        m = val_re.match(line)
+        if not m:
+            raise FormatException("unknown output from git config")
+        branch, data = m.groups()
+        result[branch] = data
+    return result
+
 def func(parser, options, args):
 
     if options.create:

@@ -198,11 +215,15 @@ def func(parser, options, args):
         branches = git.get_heads()
         branches.sort()
 
+        descriptions = __get_all_branch_config('description')
+        versions = __get_all_branch_config('stgit.stackformatversion')
+
         if branches:
             out.info('Available branches:')
-            max_len = max([len(i) for i in branches])
-            for i in branches:
-                __print_branch(i, max_len)
+            max_len = max(len(i) for i in branches)
+            for branch in branches:
+                __print_branch(branch, descriptions.get(branch),
+                               versions.get(branch), max_len)
         else:
             out.info('No branches')
         return

David Kågedal [off-list ref] writes:

I made a patch that uses "git config --get-regexp" to get the
description and stgit.stackformatversion options for all branches at
once, and ignore the "protected" flag that I don't use. With this
change, I'm almost down to half a second, which almost makes it
usable.

One thing that annoyed me what that "git config --get-regexp" will
return zero, one, or more matches, which are all valid reponses. But
it treats the zero-match special and return an exit status of 1.

Is that a conscious choice, or just an effect of how "git config
--get" works?

Since zero matches isn't really an error, I would like the exit status
to be 0. At least for this use case :-)

-- 
David Kågedal

On 13/12/2007, David Kågedal [off-list ref] wrote:

I have a fair amount of branches, and I noticed that "stg branch -l"
takes ridiculously long to finish.

I have the same problem.

Maybe someone can help me find a quicker replacement for the
get_protected call?

We can have the standard --list command which ignores the protected
flag or even the stgit.formatversion. Just a simple listing of the
branches (that's what I need most of the time). To get the
description, the 's' and 'p' flags, we could use --list-full or
something similar and wait a bit longer. This would also improve the
bash completion of commands taking branch names as arguments.

We had a similar issue in the past with 'series' as it was checking
whether the patch is empty. We ended up adding a '--empty' option for
this case.

-- 
Catalin

On 2007-12-13 14:04:26 +0000, Catalin Marinas wrote:

On 13/12/2007, David Kågedal [off-list ref] wrote:

quoted

Maybe someone can help me find a quicker replacement for the
get_protected call?

We can have the standard --list command which ignores the protected
flag

Exactly what is the p flag useful for anyway?

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

On 13/12/2007, Karl Hasselström [off-list ref] wrote:

On 2007-12-13 14:04:26 +0000, Catalin Marinas wrote:

quoted

On 13/12/2007, David Kågedal [off-list ref] wrote:

quoted

Maybe someone can help me find a quicker replacement for the
get_protected call?

We can have the standard --list command which ignores the protected
flag

Exactly what is the p flag useful for anyway?

It was added so that you don't rebase the stack by mistake. Yann
suggested to have a specific policy for this and make the protected
flag freeze the stack completely.

-- 
Catalin