[PATCH 0/5] Fix the Visual Studio 2008 .sln generator

Potential Windows developers are likely to be using Visual Studio as
their IDE. The tool stack required for Windows can be tortuous as it
crosses the boundaries between platforms and philosophies. This patch
series seeks to maintain the tools that could assist such developers.
In particular, those tools that generate an initial Visual Studio
project (.sln ) file.

The .sln generator in contrib began to break when internationalisation
introduced an extra -o option. This recently worsened with the addition
of invalidcontinue.obj for 'improved POSIX compatibility'.

I hacked a bit when I first attempted to use the VS IDE and noticed the
i18n issue. I didn't completely solve all my issues because of further
issues with VS2010 Express, so no patches were submitted at the time.

Now, with a fresh copy of VS20008 Express, I saw the additional problem
of the addition of the invalidcontinue.obj reference causing the .sln
generation to fail.

The series has been tested against both recent git/git and msysgit/git.
Both produce good VS2008 .sln project files though only msysgit/git
compiles without error, both using the msysgit msvc-build script.
Those compile errors are not related to this patch series.

An initial RFC version was discussed on the msysgit list
https://groups.google.com/forum/#!topic/msysgit/3MQ_NMNM390 or
http://thread.gmane.org/gmane.comp.version-control.msysgit/21132.

The final patch adds a poorman's --verbose by a few carefully selected
commented out debug print statements to assist others in resolving
future breakages.

I've cc'd those who have contributed or patched the engine.pl, or
appear to be interested via a $gmane search, who can hopefully comment.


Philip Oakley (5):
  engine.pl: Fix i18n -o option in msvc buildsystem generator
  engine.pl: Properly accept quoted spaces in filenames
  engine.pl: ignore invalidcontinue.obj which is known to MSVC
  vcbuild/readme: Improve layout and reference msvc-build script
  engine.pl: provide more debug print statements

 compat/vcbuild/README          | 27 +++++++++++++++++++--------
 contrib/buildsystems/engine.pl | 27 +++++++++++++++++++++------
 2 files changed, 40 insertions(+), 14 deletions(-)

-- 
2.1.0

The i18n 5e9637c (i18n: add infrastructure for translating
Git with gettext, 2011-11-18) introduced an extra '-o' option
into the make file, which broke engine.pl code for extracting
the git.sln for msvc gui-IDE.

Add tests to remove these non linker options, in same vein as
74cf9bd (engine.pl: Fix a recent breakage of the buildsystem
generator, 2010-01-22).

Signed-off-by: Philip Oakley <redacted>
---
 contrib/buildsystems/engine.pl | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/contrib/buildsystems/engine.pl b/contrib/buildsystems/engine.pl
index 23da787..f48c28a 100755
--- a/contrib/buildsystems/engine.pl
+++ b/contrib/buildsystems/engine.pl

@@ -140,6 +140,12 @@ sub parseMakeOutput
             next;
         }
 
+        if ($text =~ /^(mkdir|msgfmt) /) {
+            # options to the Portable Object translations
+            # the line "mkdir ... && msgfmt ..." contains no linker options
+            next;
+        }
+
         if($text =~ / -c /) {
             # compilation
             handleCompileLine($text, $line);

-- 
2.1.0

Commit 4b623d8 (MSVC: link in invalidcontinue.obj for better
POSIX compatibility, 2014-03-29) is not processed correctly
by the buildsystem. Ignore it.

Also split the .o and .obj processing; 'make' does not produce .obj
files. Only substitute filenames ending with .o when generating the
source .c filename.

Signed-off-by: Philip Oakley <redacted>
---
 contrib/buildsystems/engine.pl | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/contrib/buildsystems/engine.pl b/contrib/buildsystems/engine.pl
index 7f45b80..e913280 100755
--- a/contrib/buildsystems/engine.pl
+++ b/contrib/buildsystems/engine.pl

@@ -289,7 +289,7 @@ sub handleLibLine
 #    exit(1);
     foreach (@objfiles) {
         my $sourcefile = $_;
-        $sourcefile =~ s/\.o/.c/;
+        $sourcefile =~ s/\.o$/.c/;
         push(@sources, $sourcefile);
         push(@cflags, @{$compile_options{"${sourcefile}_CFLAGS"}});
         push(@defines, @{$compile_options{"${sourcefile}_DEFINES"}});

@@ -333,8 +333,12 @@ sub handleLinkLine
         } elsif ($part =~ /\.(a|lib)$/) {
             $part =~ s/\.a$/.lib/;
             push(@libs, $part);
-        } elsif ($part =~ /\.(o|obj)$/) {
+        } elsif ($part eq 'invalidcontinue.obj') {
+            # ignore - known to MSVC
+        } elsif ($part =~ /\.o$/) {
             push(@objfiles, $part);
+        } elsif ($part =~ /\.obj$/) {
+            # do nothing, 'make' should not be producing .obj, only .o files
         } else {
             die "Unhandled lib option @ line $lineno: $part";
         }

@@ -343,7 +347,7 @@ sub handleLinkLine
 #    exit(1);
     foreach (@objfiles) {
         my $sourcefile = $_;
-        $sourcefile =~ s/\.o/.c/;
+        $sourcefile =~ s/\.o$/.c/;
         push(@sources, $sourcefile);
         push(@cflags, @{$compile_options{"${sourcefile}_CFLAGS"}});
         push(@defines, @{$compile_options{"${sourcefile}_DEFINES"}});

-- 
2.1.0

The engine.pl script barfs on the properly quoted spaces in
filename options prevalent on Windows. Use shellwords() rather
than split() to separate such options.

Signed-off-by: Philip Oakley <redacted>
---
 contrib/buildsystems/engine.pl | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/contrib/buildsystems/engine.pl b/contrib/buildsystems/engine.pl
index f48c28a..7f45b80 100755
--- a/contrib/buildsystems/engine.pl
+++ b/contrib/buildsystems/engine.pl

@@ -12,6 +12,7 @@ use File::Basename;
 use File::Spec;
 use Cwd;
 use Generators;
+use Text::ParseWords;
 
 my (%build_structure, %compile_options, @makedry);
 my $out_dir = getcwd();

@@ -237,7 +238,7 @@ sub removeDuplicates
 sub handleCompileLine
 {
     my ($line, $lineno) = @_;
-    my @parts = split(' ', $line);
+    my @parts = shellwords($line);
     my $sourcefile;
     shift(@parts); # ignore cmd
     while (my $part = shift @parts) {

@@ -271,7 +272,7 @@ sub handleLibLine
     my (@objfiles, @lflags, $libout, $part);
     # kill cmd and rm 'prefix'
     $line =~ s/^rm -f .* && .* rcs //;
-    my @parts = split(' ', $line);
+    my @parts = shellwords($line);
     while ($part = shift @parts) {
         if ($part =~ /^-/) {
             push(@lflags, $part);

@@ -312,7 +313,7 @@ sub handleLinkLine
 {
     my ($line, $lineno) = @_;
     my (@objfiles, @lflags, @libs, $appout, $part);
-    my @parts = split(' ', $line);
+    my @parts = shellwords($line);
     shift(@parts); # ignore cmd
     while ($part = shift @parts) {
         if ($part =~ /^-IGNORE/) {

-- 
2.1.0

Assist developers transitioning between the two cultures
by including appropriate, but commented out, debug statements.

This acts as a poor man's --verbose option. The test suite doesn't
cover the contrib/buildsystems (or Msysgit's msvc-build) contributions
so fails to notice breakages there-in.

Signed-off-by: Philip Oakley <redacted>
---
 contrib/buildsystems/engine.pl | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/contrib/buildsystems/engine.pl b/contrib/buildsystems/engine.pl
index e913280..24a4c61 100755
--- a/contrib/buildsystems/engine.pl
+++ b/contrib/buildsystems/engine.pl

@@ -41,6 +41,7 @@ EOM
 # Parse command-line options
 while (@ARGV) {
     my $arg = shift @ARGV;
+	#print "Arg: $arg \n";
     if ("$arg" eq "-h" || "$arg" eq "--help" || "$arg" eq "-?") {
 	showUsage();
 	exit(0);

@@ -123,6 +124,7 @@ sub parseMakeOutput
     print "Parsing GNU Make output to figure out build structure...\n";
     my $line = 0;
     while (my $text = shift @makedry) {
+		#print "Make: $text\n"; # show the makedry line
         my $ate_next;
         do {
             $ate_next = 0;

@@ -257,6 +259,7 @@ sub handleCompileLine
         } elsif ($part =~ /\.(c|cc|cpp)$/) {
             $sourcefile = $part;
         } else {
+            print "full line: $line\n";
             die "Unhandled compiler option @ line $lineno: $part";
         }
     }

@@ -282,6 +285,7 @@ sub handleLibLine
             $libout = $part;
             $libout =~ s/\.a$//;
         } else {
+            print "full line: $line\n";
             die "Unhandled lib option @ line $lineno: $part";
         }
     }

-- 
2.1.0

Layout the 'either/or' with more white space to clarify
which alternatives are matched up.

Reference the Msysgit build script which automates one sequence of options.

Signed-off-by: Philip Oakley <redacted>
---
 compat/vcbuild/README | 27 +++++++++++++++++++--------
 1 file changed, 19 insertions(+), 8 deletions(-)

diff --git a/compat/vcbuild/README b/compat/vcbuild/README
index df8a657..7548dc4 100644
--- a/compat/vcbuild/README
+++ b/compat/vcbuild/README

@@ -3,20 +3,24 @@ The Steps of Build Git with VS2008
 1. You need the build environment, which contains the Git dependencies
    to be able to compile, link and run Git with MSVC.
 
-   You can either use the binary repository:
+   You can either:
+      use the binary repository:
 
        WWW: http://repo.or.cz/w/msvcgit.git
        Git: git clone git://repo.or.cz/msvcgit.git
        Zip: http://repo.or.cz/w/msvcgit.git?a=snapshot;h=master;sf=zip
 
-   and call the setup_32bit_env.cmd batch script before compiling Git,
-   (see repo/package README for details), or the source repository:
+      and call the setup_32bit_env.cmd batch script before compiling Git,
+     (see repo/package README for details),
+
+   or:
+      use the source repository:
 
        WWW: http://repo.or.cz/w/gitbuild.git
        Git: git clone git://repo.or.cz/gitbuild.git
        Zip: (None, as it's a project with submodules)
 
-   and build the support libs as instructed in that repo/package.
+     and build the support libs as instructed in that repo/package.
 
 2. Ensure you have the msysgit environment in your path, so you have
    GNU Make, bash and perl available.

@@ -33,18 +37,25 @@ The Steps of Build Git with VS2008
        make common-cmds.h
    to generate the common-cmds.h file needed to compile git.
 
-4. Then either build Git with the GNU Make Makefile in the Git projects
-   root
+4. Then either
+
+     build Git with the GNU Make Makefile in the Git projects root
        make MSVC=1
-   or generate Visual Studio solution/projects (.sln/.vcproj) with the
+   or
+
+   generate Visual Studio solution/projects (.sln/.vcproj) with the
    command
        perl contrib/buildsystems/generate -g Vcproj
    and open and build the solution with the IDE
        devenv git.sln /useenv
-   or build with the IDE build engine directly from the command line
+     or
+
+   build with the IDE build engine directly from the command line
        devenv git.sln /useenv /build "Release|Win32"
    The /useenv option is required, so Visual Studio picks up the
    environment variables for the support libraries required to build
    Git, which you set up in step 1.
 
 Done!
+
+Or, use the Msysgit msvc-build script; available from that project.

-- 
2.1.0

From: "Philip Oakley" <redacted>

Potential Windows developers are likely to be using Visual Studio as
their IDE. The tool stack required for Windows can be tortuous as it
crosses the boundaries between platforms and philosophies. This patch
series seeks to maintain the tools that could assist such developers.
In particular, those tools that generate an initial Visual Studio
project (.sln ) file.

The .sln generator in contrib began to break when internationalisation
introduced an extra -o option. This recently worsened with the 
addition
of invalidcontinue.obj for 'improved POSIX compatibility'.

I hacked a bit when I first attempted to use the VS IDE and noticed 
the
i18n issue. I didn't completely solve all my issues because of further
issues with VS2010 Express, so no patches were submitted at the time.

Now, with a fresh copy of VS20008 Express, I saw the additional 
problem
of the addition of the invalidcontinue.obj reference causing the .sln
generation to fail.

The series has been tested against both recent git/git and 
msysgit/git.
Both produce good VS2008 .sln project files though only msysgit/git
compiles without error, both using the msysgit msvc-build script.
Those compile errors are not related to this patch series.

Now, having tried using the VS project output for some follow on work, 
I've found that the resultant .sln file is incomplete, though does 
compile fine as noted.

Investigation has shown that the 'failure' to provide a complete Git 
project would apper to have begun somewhere between v1.7.9 and v1.8.0. 
V1.7.9 has 'git', while v1.8.0 doesn't (along with other missing 
artefacts).

Looks like firther fixes are needed once the cause is determined.

An initial RFC version was discussed on the msysgit list
https://groups.google.com/forum/#!topic/msysgit/3MQ_NMNM390 or
http://thread.gmane.org/gmane.comp.version-control.msysgit/21132.

The final patch adds a poorman's --verbose by a few carefully selected
commented out debug print statements to assist others in resolving
future breakages.

I've cc'd those who have contributed or patched the engine.pl, or
appear to be interested via a $gmane search, who can hopefully 
comment.


Philip Oakley (5):
 engine.pl: Fix i18n -o option in msvc buildsystem generator
 engine.pl: Properly accept quoted spaces in filenames
 engine.pl: ignore invalidcontinue.obj which is known to MSVC
 vcbuild/readme: Improve layout and reference msvc-build script
 engine.pl: provide more debug print statements

compat/vcbuild/README          | 27 +++++++++++++++++++--------
contrib/buildsystems/engine.pl | 27 +++++++++++++++++++++------
2 files changed, 40 insertions(+), 14 deletions(-)

-- 

Philip 

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "Git for Windows" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

From: "Philip Oakley" <redacted>

The i18n 5e9637c (i18n: add infrastructure for translating
Git with gettext, 2011-11-18) introduced an extra '-o' option
into the make file, which broke engine.pl code for extracting
the git.sln for msvc gui-IDE.

Add tests to remove these non linker options, in same vein as
74cf9bd (engine.pl: Fix a recent breakage of the buildsystem
generator, 2010-01-22).

It now looks like this resolution is in the wrong direction. The
correct right solution is to include
    NO_GETTEXT = YesPlease
into the _Windows_ section of the Makefile.

Mysgit uses the _MinGW_ section, and fixed the NO_GETTEXT requirement, 
so the breakage in the Windows section build wasn't noticed.

However, the test is still useful for adding a detection step and 
providing a warning to users
    NO_GETTEXT = YesPlease

There are still more issues to be resolved e.g.
  test-svn-fe.obj : error LNK2001: unresolved external symbol _strtoull
 etc.
Philip 

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "Git for Windows" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.