From: Jeff King <hidden> Date: 2016-06-15 22:53:12
The Google Summer of Code 2012 application process is underway. We have
participated for the past 5 years, so I assume we want to do so again.
If you don't know what GSoC is, read this:
https://code.google.com/soc/
I've volunteered to be the organization admin again, but I would be
happy to step aside if somebody else wants to do it. I've started things
off by posting a draft of our application and starting a wiki page for
project ideas:
https://github.com/peff/git/wiki/SoC-2012-Applicationhttps://github.com/peff/git/wiki/SoC-2012-Ideas
The application is due March 9th (next Friday).
At this point, we need:
1. Somebody to volunteer as a backup admin.
2. Project ideas to be added to the ideas page.
3. Volunteers to mentor students. Our criteria for mentors in the
past has been to accept only those who have a history of
contributing to git. But if you have an idea for a project and
don't feel that you would be capable of mentoring, please feel free
to start a discussion on the list. You may interest other people
who can volunteer to mentor for it.
4. People to read over the application and suggest improvements. It's
adapted from last year's application. The general meaning should be
the same, though I did bring it up to date as well clean up a few
awkward bits.
In the past, we've used the kernel.org wiki for developing the SoC
materials. However, it's still not editable, and I don't think we have a
plan for what the "new" git wiki will be. I've put things up on this
GitHub wiki in the meantime; they can be migrated to an official wiki if
and when that happens.
I think you'll need a GitHub login to edit the wiki pages via the web
interface. You can also clone it here:
git://github.com/peff/git.wiki.git
and I will be happy to take patches on the list.
-Peff
@@ -140,3 +140,44 @@ The programming work will be in C, as it replaces a core part of git. Proposed by: Thomas Rast Possible mentor(s): Thomas Rast++Improving the `git add -p` interface+------------------------------------++The interface behind `git {add|commit|stash|reset} {-p|-i}` is shared+and called `git-add--interactive.perl`. This project would mostly+focus on the `--patch` side, as that seems to be much more widely+used; however, improvements to `--interactive` would probably also be+welcome.++The `--patch` interface suffers from some design flaws caused largely+by how the script grew:++*Application is not atomic: hitting Ctrl-C midway through patching+ may still touch files.++*The terminal/line-based interface becomes a problem if diff hunks+ are too long to fit in your terminal.++*Cannot go back and forth between files.++*Cannot reverse the direction of the patch.++*Cannot look at the diff in word-diff mode (and apply it normally).++Due to the current design it is also pretty hard to add these features+without adding to the mess. Thus the project consists of:++*Come up with more ideas for features/improvements and discuss them+ with users.++*Cleanly redesigning the main interface loop to allow for the above+ features.++*Implement the new features.++As the existing code is written in Perl, that is what you will use for+this project.++Proposed by: Thomas Rast+Possible mentor(s): Thomas Rast
@@ -68,3 +68,34 @@ work to be done: and only accessed on demand. Proposed mentor: Jeff King++Improving parallelism in various commands+-----------------------------------------++Git is mostly written single-threaded, with a few commands having+bolted-on extensions to support parallel operation (notably git-grep,+git-pack-objects and the core.preloadIndex feature).++We have recently looked into some of these areas and made a few+optimizations, but a big roadblock is that pack access is entirely+single-threaded. The project would consist of the following steps:++*In preparation (the half-step): identify commands that could+ benefit from parallelism. `git grep --cached` and `git grep+ COMMIT` come to mind, but most likely also `git diff` and `git log+ -p`. You can probably find more.++*Rework the pack access mechanisms to allow the maximum possible+ parallel access.++*Rework the commands found in the first step to use parallel pack+ access if possible. Along the way, document the improvements with+ performance tests.++The actual programming must be done in C using pthreads for obvious+reasons. At the very least you should not be scared of low-level+programming. Prior experience and access to one or more multi-core+computers is a plus.++Proposed by: Thomas Rast+Possible mentor(s): Thomas Rast
@@ -99,3 +99,44 @@ computers is a plus. Proposed by: Thomas Rast Possible mentor(s): Thomas Rast++Designing a faster index format+-------------------------------++Git is pretty slow when managing huge repositories in terms of files+in any given tree, as it needs to rewrite the index (in full) on+pretty much every operation. For example, even though _logically_+`git add already_tracked_file` only changes a single blob SHA-1 in the+index, Git will verify index correctness during loading and recompute+the new hash during writing _over the whole index_. It thus ends up+spending a large amount of time simply on hashing the index.++A carefully designed index format could help in several ways. (For the+complexity estimates below, let n be the number of index entries or+the size of the index, which is roughly the same.)++*The work needed for something as simple as entering a new blob into+ the index, which is possibly the most common operation in git+ (think `git add -p` etc.) should be at most log(n).++*The work needed for a more complex operation that changes the+ number of index entries will have to be larger unless we get into+ database land. However the amount of data that we SHA-1 over+ should still be log(n).++*It may be possible to store the cache-tree data directly as part of+ the index, always keeping it valid, and using that to validate+ index consistency throughout. If so, this would be a big boost to+ other git operations that currently suffer from frequent cache-tree+ invalidation.++Note that there are other criteria than speed: the format should also+be as easy to parse as possible, so as to simplify work for the other+.git-reading programs (such as jgit and libgit2). For the same+reason, you will also have to show a significant speed boost as+otherwise the break in compatibility is not worth the fallout.++The programming work will be in C, as it replaces a core part of git.++Proposed by: Thomas Rast+Possible mentor(s): Thomas Rast
On Fri, Mar 2, 2012 at 6:05 PM, Thomas Rast [off-list ref] wrote:
+ * In preparation (the half-step): identify commands that could
+ benefit from parallelism. `git grep --cached` and `git grep
+ COMMIT` come to mind, but most likely also `git diff` and `git log
+ -p`. You can probably find more.
I just had a thought this afternoon whether "git add" may benefit from
parallelism. It's most likely I/O-bound, although I think if we add a
bunch of large files, it might become CPU-bound. To generalize,
anything that calls hash_sha1_file() might benefit from parallelism.
Another candidate may be git-apply. Actually I just want to speed up
git-rebase and think git-apply may be the culprit. Or it could be
unpack-trees code..
--
Duy
On Fri, Mar 2, 2012 at 4:11 PM, Jeff King [off-list ref] wrote:
At this point, we need:
2. Project ideas to be added to the ideas page.
The master index to speed up pack access by avoid going through all
pack indices one by one when there are a lot of packs. If building the
master index is cheap, it could take "repack -ad"'s position in "gc
--auto" and delay repacking longer. I think this should be easy to do,
just not sure big enough for gsoc.
--
Duy
From: James Pickens <hidden> Date: 2016-06-15 22:53:12
[Resend since the first try had HTML and the list rejected it]
On Fri, Mar 2, 2012, Thomas Rast [off-list ref] wrote:
+ * In preparation (the half-step): identify commands that could
+ benefit from parallelism. `git grep --cached` and `git grep
+ COMMIT` come to mind, but most likely also `git diff` and `git log
+ -p`. You can probably find more.
For those of us who must work on NFS for various reasons, it would help
tremendously to write out work tree files in parallel, during 'git clone',
'git reset --hard', and any other command that writes lots of files to the
work tree. You can get a huge speedup (benchmarked at ~3.5x) without even
unpacking those files in parallel; unpacking them serially and writing them
to disk in parallel is sufficient.
I submitted a patch [1] ~2 years ago that added that capability. It was
not accepted, but it did demonstrate the huge potential speedup on NFS, and
the pitfall of degrading performance on a local drive. The patch itself is
probably not useful any more, but it includes some benchmarks, and the
discussion may be helpful.
James
[1] http://thread.gmane.org/gmane.comp.version-control.git/103489
@@ -181,3 +181,44 @@ this project. Proposed by: Thomas Rast Possible mentor(s): Thomas Rast++Modernizing and expanding Git.pm+--------------------------------++Git.pm was created in 2006 to make it easy to call git commands from+Perl scripts safely, in a portable way (including workarounds required+for ActiveState Perl on MS Windows). Its error handling via exceptions+also comes from that year.++Git.pm module uses Error (and Error::Simple) for its exception handling.+Unfortunately, while it might looked like a good choice in 2006, Error+module is deprecated:++> WARNING+> -------+>+> Using the "Error" module is **no longer recommended** due to the+> black-magical nature of its syntactic sugar, which often tends to+> break. Its maintainers have stopped actively writing code that uses+> it, and discourage people from doing so. See the "SEE ALSO" section+> below for better recommendations.++Nowadays the recommended solution to exception handling in Perl are+Try::Tiny (or TryCatch, but I don't think Git.pm would need this more+heavyweight module) for capturing and handling exceptions, and+Exception::Class (or Throwable, but that requires heavyweight Moose+object system) for throwing OO exceptions.++The major goal would be to update Git.pm to modern Perl conventions,+amon others moving from Error / Error::Simple to Try::Tiny and+Exception::Class, preserving backwards compatibility, but perhaps also+adding a better interface and using it in git commands implemented in+Perl.++Other optional goals would be to extend Git.pm, for example adding+Git::Config module which would read git configuration once like gitweb+does, or Git::Commit module for parsing commit objects, etc.++Programming language: Perl +Proposed by: Jakub Narębski +Possible mentor(s): Jakub Narębski (?)
From: Jeff King <hidden> Date: 2016-06-15 22:53:12
On Sat, Mar 03, 2012 at 10:14:41PM +0100, Jakub Narebski wrote:
---
I'm not sure if my Markdown is entirely correct.
SoC-2012-Ideas.md | 41 +++++++++++++++++++++++++++++++++++++++++
1 files changed, 41 insertions(+), 0 deletions(-)
Thanks, applied. Your markdown looks fine, though I dropped the quoted
heading line here:
+> WARNING
+> -------
As it was difficult to tell that that large text was part of a quoted
bit (probably there is some quoting mode that does not expand such
things, but I didn't think it was worth fiddling with too much).
-Peff
From: Jeff King <hidden> Date: 2016-06-15 22:53:14
On Fri, Mar 02, 2012 at 04:11:14AM -0500, Jeff King wrote:
The Google Summer of Code 2012 application process is underway. We have
participated for the past 5 years, so I assume we want to do so again.
[...]
At this point, we need:
People have been very good about contributing to the ideas page, and I
think we have a number of possible mentors (though more ideas and more
mentor volunteers are always good). However, we still need somebody to
volunteer as backup admin. If I don't hear from somebody, I will just
put down Shawn's name, whether he accepts it or not. :)
-Peff
On Wed, Mar 7, 2012 at 06:36, Jeff King [off-list ref] wrote:
On Fri, Mar 02, 2012 at 04:11:14AM -0500, Jeff King wrote:
quoted
The Google Summer of Code 2012 application process is underway. We have
participated for the past 5 years, so I assume we want to do so again.
[...]
At this point, we need:
People have been very good about contributing to the ideas page, and I
think we have a number of possible mentors (though more ideas and more
mentor volunteers are always good). However, we still need somebody to
volunteer as backup admin. If I don't hear from somebody, I will just
put down Shawn's name, whether he accepts it or not. :)
@@ -264,6 +264,60 @@ Programming language: Perl Proposed by: Jakub Narębski Possible mentor(s): Jakub Narębski (?)+Use JavaScript library / framework in gitweb+--------------------------------------------++Gitweb (git web interface) includes some *optional* client-side+scripting using JavaScript. This includes checking if JavaScript is+available and remembering this information so gitweb can choose+JavaScript-only version of a view (javascript-detection), selecting+common timezone to use when showing dates (adjust-timezone), and+AJAX-y incremental blame view (blame_incremental).++Currently all this is done using hand-written JavaScript. This means+that gitweb scripting includes handling cookies, formatting output,+processing dates, and smoothing out incompatibilities between browsers+(like e.g. XmlHttpRequest creation).++This is redoing work which JavaScript libraries, such as jQuery,+MooTools or YUI already did. Moreover, if we want to add new features+(e.g. table sorted using JavaScript), or improve existing ones, we+would have to re-implement existing JavaScript code. Also our+hand-crafted code is not as well tested as widely used JavaScript+libraries.++The goal of this project is to move gitweb client side scripting to+use some JavaScript library / JavaScript framework.++The project would consist of the following steps:++*Add support for configuring and loading external JavaScript library+ to `gitweb/gitweb.perl` and `gitweb/Makefile`. It would be nice+ (though not necessary) to be able to use local version of library,+ and have such feature well documented.++*Remove gitweb's JavaScript mini-library in `gitweb/static/js/lib`+ and replace it part by part by appropriate JavaScript library+ functions (methods).++*Replace DOM selectors by library version, if applicable.++*Optional: emulate 'onprogress' in XmlHttpRequest using native+ JavaScript library mechanism (creating a class, or whatever).++*Optional: better deferring of repainting in incremental blame.++Note that we require that client-side scripting in gitweb follow+[progressive enhancement] strategy; gitweb should work correctly,+perhaps with reduced functionality, even if JavaScript is turned off,+or external JavaScript library cannot be loaded.++[progressive enhancement]: http://en.wikipedia.org/wiki/Progressive_enhancement++Programming language: JavaScript +Proposed by: Jakub Narębski +Possible mentor(s): Jakub Narębski+ Finishing network support for libgit2 -------------------------------------
From: Jakub Narebski <hidden> Date: 2016-06-15 22:53:16
---
I don't know if "git-serve" is something that would be realy useful,
or just checkmark in the comparison of Git and Mercurial.
SoC-2012-Ideas.md | 54 +++++++++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 54 insertions(+), 0 deletions(-)
@@ -318,6 +318,60 @@ Programming language: JavaScript Proposed by: Jakub Narębski Possible mentor(s): Jakub Narębski+`git instaweb --serve`+----------------------++[git-instaweb] is a tool for browsing a repository (in gitweb) with+a web browser. To use it, simply run `git instaweb` inside+repository. It would set up gitweb and a web server, and by default+also run a web browser. Web interface would be available at+`http://localhost:1234`.++For informal, ad-hoc sharing it would be nice if there was an option+to `git instaweb` that would make it also allow remote machines+to **pull** from you (via HTTP), similar to [hg serve] command in+Mercurial version control system.++git-instaweb supports many web servers. Currently apache2, lighttpd,+mongoose, plackup and webrick are supported. It is not necessary to+implement support for `--serve` in all of them.++The project would consist of the following steps:++*Configure web server to run `git-http-backend` CGI program to serve+ git repositories over HTTP. It should probably allow by default+ only read-only use.++*Configure gitweb to show fetch URL in 'summary' page.++*If possible for given web server (and with available modules),+ configure it so that "smart" HTTP server and gitweb share common+ URL (are available under the same URL).++*If possible use mechanism native for a web server used, for example + [Plack::App::GitSmartHttp] for 'plackup' web server, or [grack]+ with WEBrick.++*Optionally: add support for pushing (disabled by default).++*Optionally: add documentation about using `git instaweb` to+ "[Git User's Manual]" (or one of tutorials), similar to appropriate+ chapter in [hgbook] ("Mercurial: The Definitive Guide").++The minimum would be to implement pull support under different URL+than web interface, and only for one web server.++Programming language: shell script +Proposed by: Jakub Narębski +Possible mentor(s): Jakub Narębski, Eric Wong, ...++[git-instaweb]: http://schacon.github.com/git/git-instaweb.html+[hg serve]: http://mercurial.selenic.com/wiki/hgserve+[Plack::App::GitSmartHttp]: http://search.cpan.org/perldoc?Plack::App::GitSmartHttp+[grack]: https://github.com/schacon/grack+[Git User's Manual]: http://schacon.github.com/git/user-manual.html+[hgbook]: http://hgbook.red-bean.com/read/collaborating-with-other-people.html#sec:collab:serve+ Finishing network support for libgit2 -------------------------------------
From: Jakub Narebski <hidden> Date: 2016-06-15 22:53:16
---
I don't have much hope that we can find a student that would be
interested in this feature and knows Tcl/Tk enough so that hs or she
can atually implement it...
But I can hope.
SoC-2012-Ideas.md | 62 +++++++++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 62 insertions(+), 0 deletions(-)
@@ -421,6 +421,68 @@ Write a tool that can be used for the task, and optionally wrap an interactive U Programming language: Any Possible mentors: ??? +Graphical diff in git-gui+-------------------------++[git-gui] is a portable Tcl/Tk based graphical interface to Git,+focused on commit generation and single file annotation. It is part+of Git, even though it is developed in a [separate repository].++git-gui can show differences as syntax-highlighted unified diff. The+goal of this project would be to add graphical side-by-side diff.+One possibility is to make use of [TkDiff] code (a graphical diff and+merge tool), which is also GPL licensed.++The project would consist of the following steps (not all must be+implemented during Google Summer of Code):++*Add "git gui diff" subcommand, which would show single file+ side-by-side graphical compare or/and graphical diff.++ Graphical part and code for side-by-side compare (showing full+ contents of both files) can be taken from TkDiff, while graphical+ side-by-side diff (showing changes plus context) can be translated+ from gitweb's side-by-side diff code.++*Integrate graphical diff with main git-gui application (including+ switching between side-by-side and unified diff).++*Add highlighting of changes in diff both to side-by-side+ (new code), and to unified diff (current code).++ It can be done using code for TkDiff (perhaps with Git performing+ word diff instead of doing it in Tcl), or/and using algorithm from+ gitweb and diff-highlight in contrib. The difference is that one+ uses word diff or character diff to highlight changes, the other+ just skips common prefix and suffix.++*Add graphical merge / graphical 3-way diff support. Both are to be+ used in case a file has textual conflicts; graphical merge also+ includes resolving a merge by taking 'our' side, 'their' side (or+ optionally also 'ancestor' side), or by editing merge result.++*Graphical side-by-side tree level diff, or side-by-side directory+ listing with differences highlighted.++ UI can be taken from two-panel filemanagers (like MC, or Total+ Commander), or from synchronization tools (like e.g. Unison).++ The difficulty can be in showing rename and copy detection results,+ and in showing type (filemode) changes.++The minimum would be to implement side-by-side diff or side-by-side+compare of two files, without highlighting changes (diff refinement+highlighting) in the form of separate "`git gui diff <file>`"+command.++[git-gui]: http://schacon.github.com/git/git-gui.html+[separate repository]: http://repo.or.cz/w/git-gui.git+[TkDiff]: http://freecode.com/projects/tkdiff++Programming language: Tcl/Tk +Proposed by: Jakub Narębski +Possible mentor(s): Pat Thoyts, Paul Mackerras (?)+ Other sources of inspiration ----------------------------