RE: Porting git to HP NonStop

From: git-owner@vger.kernel.org [mailto:git-owner@vger.kernel.org] On
Behalf Of Joachim Schmitz
Sent: Friday, August 10, 2012 5:01 PM
To: git@vger.kernel.org
Cc: rsbecker@nexbridge.com
Subject: RE: [PATCH v2] add tests for 'git rebase --keep-empty'

Hi folks

I'm a brand new subscriper of this mailing list, so please forgive if I

violate

some protocol or talk about things that had been discussed to death

earlier.

Ahrgl, 1st mistake: wrong subject, sorry

I'm currently in the process of porting git (1.7.11.4 for now) to the HP

NonStop

platform and found several issues:

- HP NonStop is lacking poll(), git is making quite some use of it.
My Solution: I 'stole' the implementation from GNUlib, which implements
poll() using select().
Git should either provide its own poll(), not use it at all or resort to

using

GNUlib, what do you think?.

- HP NonStop is lacking getrlimit(), fsync(), setitimer() and memory

mapped IO.

For now I've commented out the part that used getrlimit() and use a home
brewed implementation for fsync(), setitimer() and mmap().

- git makes use of some C99 features or at least feature that are not

availabe in

C89, like 'inline'
C89 is the default compiler on HP NonStop, but we also habe a c99

compiler, so

telling configure to search for c99  should help here.

- libintl and libiconv sem to get linked in the wrong order, resulting in
unresolved symbols.
I've just moved the "ifndef NO_GETTEXT" section of Makefile to above the
"ifdef NEEDS_LIBICONF" section.

- HP NonStop doesn't have stat.st_blocks, this is used in

builtin/count-objects.c

around line 45, not sure yet how to fix that.

- HP NonStop doesn't have stat.st_?time.nsec, there are several places

what an

"#ifdef USE_NSEC" is missing, I can provide a diff if needed (offending
files: builtin/fetch-pack.c and read-cache.c).

- HP NonStop doesn't know SA_RESTART
I fixed that with a "#define SA_RESTART 0" in the 3 files affected

(builtin/log.c,

fast-import.c and progress.c)

- using C99 but not using #include <strings.h> results in compiler errors

due to

a missing prototype for strcasecmp() I fixed it by adding that to

git-compat-

util.h

- HP NonStop doesn't have intptr_t and uintpr_t (in its stdint.h) I added

them to

git-compat-util.h

- HP NonStop doesn't need the " #define _XOPEN_SOURCE 600", just like
__APPLE__, __FreeBSD__ etc, so I added a "&& !defined(__TANDEM) in git-
compat-util.h

- there seems to be an issue with compat/fnmatch/fnmatch.c not including
string.h, seems that HAVE_STRING_H is not #define'd anywhere.


- Once compiled and installed, a simple jojo@\hpitug:/home/jojo/GitHub $

git

clone git://github.com/git/git.git fails with:
/home/jojo/GitHub/git/.git/branches/: No such file or directory After

creating

those manually it fails because the directory isn't empty,
catch-22
After some trial'n'error I found that the culprit seems to be the

subdirectories

branches, hook and info in /usr/local/share/git-core/templates/, if I
remove/rename those, the above command works fine.
I have no idea why that is nor how to properly fix it, anyone out there?

Bye, Jojo

On Fri, Aug 10, 2012 at 8:04 AM, Joachim Schmitz
[off-list ref] wrote:

quoted

- HP NonStop is lacking poll(), git is making quite some use of it.
My Solution: I 'stole' the implementation from GNUlib, which implements
poll() using select().
Git should either provide its own poll(), not use it at all or resort to

using

quoted

GNUlib, what do you think?.

poll() is usually better than select() because you don't need to worry
about FD_SETSIZE. That said, the compat/ directory contains
implementations of some functions. You could contribute a fake poll
that uses select if it was under the GPLv2.

quoted

- HP NonStop is lacking getrlimit(), fsync(), setitimer() and memory

mapped IO.

quoted

For now I've commented out the part that used getrlimit() and use a home
brewed implementation for fsync(), setitimer() and mmap().

There is no need to define your own mmap(). Define NO_MMAP=1 in the
Makefile. Git already has its own fake mmap and knows how to write it
back to disk when making changes.

quoted

- git makes use of some C99 features or at least feature that are not

availabe in

quoted

C89, like 'inline'
C89 is the default compiler on HP NonStop, but we also habe a c99

compiler, so

quoted

telling configure to search for c99  should help here.

You could also disable inline by #define inline /**/, but this will
probably result in a slower binary.

quoted

- HP NonStop doesn't have stat.st_blocks, this is used in

builtin/count-objects.c

quoted

around line 45, not sure yet how to fix that.

IIRC the block count is only used to give the user some notion of how
much disk was wasted by the repository. You could hack a macro that
redefines this as st_size.

quoted

- HP NonStop doesn't have stat.st_?time.nsec, there are several places

what an

quoted

"#ifdef USE_NSEC" is missing, I can provide a diff if needed (offending
files: builtin/fetch-pack.c and read-cache.c).

I think this would be appreciated by anyone else that has a similar
problem where the platform lacks nsec.

quoted

- Once compiled and installed, a simple jojo@\hpitug:/home/jojo/GitHub $

git

quoted

clone git://github.com/git/git.git fails with:
/home/jojo/GitHub/git/.git/branches/: No such file or directory After

creating

quoted

those manually it fails because the directory isn't empty,
catch-22
After some trial'n'error I found that the culprit seems to be the

subdirectories

quoted

branches, hook and info in /usr/local/share/git-core/templates/, if I
remove/rename those, the above command works fine.
I have no idea why that is nor how to properly fix it, anyone out there?

This sounds like the templates directory was not created correctly
during installation, or is being copied incorrectly during the git
init process. I would start by comparing the structure and permissions
of the templates directory on your HP NonStop system to one on a Linux
system and see if there was a mistake made during the installation
process. If the directory matches, I would then use `git init --bare`
in a new directory to copy in the templates, and see if its the
template copying code that is making an incorrect copy.

From: Shawn Pearce [mailto:spearce@spearce.org]
Sent: Friday, August 10, 2012 6:28 PM
To: Joachim Schmitz
Cc: git@vger.kernel.org; rsbecker@nexbridge.com
Subject: Re: Porting git to HP NonStop

On Fri, Aug 10, 2012 at 8:04 AM, Joachim Schmitz [off-list ref]
wrote:

quoted

quoted

- HP NonStop is lacking poll(), git is making quite some use of it.
My Solution: I 'stole' the implementation from GNUlib, which
implements
poll() using select().
Git should either provide its own poll(), not use it at all or resort
to

using

quoted

GNUlib, what do you think?.

poll() is usually better than select() because you don't need to worry

about

FD_SETSIZE. That said, the compat/ directory contains implementations of
some functions. You could contribute a fake poll that uses select if it

was under

the GPLv2.

This is what I did. Just to see now that compat/win32/poll.c has exacly the
same stuff...
 

quoted

quoted

- HP NonStop is lacking getrlimit(), fsync(), setitimer() and memory

mapped IO.

quoted

For now I've commented out the part that used getrlimit() and use a
home brewed implementation for fsync(), setitimer() and mmap().

There is no need to define your own mmap(). Define NO_MMAP=1 in the
Makefile. Git already has its own fake mmap and knows how to write it back

to

disk when making changes.

Ah, excellent. Esp. as our home brewed implementation is pretty primitive.

quoted

quoted

- git makes use of some C99 features or at least feature that are not

availabe in

quoted

C89, like 'inline'
C89 is the default compiler on HP NonStop, but we also habe a c99

compiler, so

quoted

telling configure to search for c99  should help here.

You could also disable inline by #define inline /**/, but this will

probably result

in a slower binary.

Even our C99 compiler doesn't inline, it merly recognizes the keyword and
then warns about (unable to inline...)
But there are other C99 features used too.

quoted

quoted

- HP NonStop doesn't have stat.st_blocks, this is used in

builtin/count-objects.c

quoted

around line 45, not sure yet how to fix that.

IIRC the block count is only used to give the user some notion of how much

disk

was wasted by the repository. You could hack a macro that redefines this

as

st_size.

OK, thanks, will try that.

quoted

quoted

- HP NonStop doesn't have stat.st_?time.nsec, there are several
places

what an

quoted

"#ifdef USE_NSEC" is missing, I can provide a diff if needed
(offending
files: builtin/fetch-pack.c and read-cache.c).

I think this would be appreciated by anyone else that has a similar

problem

where the platform lacks nsec.

Will do.
 

quoted

quoted

- Once compiled and installed, a simple
jojo@\hpitug:/home/jojo/GitHub $

git

quoted

clone git://github.com/git/git.git fails with:
/home/jojo/GitHub/git/.git/branches/: No such file or directory After

creating

quoted

those manually it fails because the directory isn't empty,
catch-22
After some trial'n'error I found that the culprit seems to be the

subdirectories

quoted

branches, hook and info in /usr/local/share/git-core/templates/, if I
remove/rename those, the above command works fine.
I have no idea why that is nor how to properly fix it, anyone out

there?

This sounds like the templates directory was not created correctly during
installation, or is being copied incorrectly during the git init process.

I would

start by comparing the structure and permissions of the templates

directory on

your HP NonStop system to one on a Linux system and see if there was a
mistake made during the installation process. If the directory matches, I

would

jojo@\hpitug:/usr/local/share/git-core/templates $ ls -laR
total 41
drwxr-xr-x    1 SUPER.SUPER        SUPER       4096 Aug 10 12:10 .
drwxr-xr-x    1 SUPER.SUPER        SUPER       4096 Aug 10 08:19 ..
drwxr-xr-x    1 SUPER.SUPER        SUPER       4096 Aug 10 07:26 branches
drwxr-xr-x    1 SUPER.SUPER        SUPER       4096 Aug 10 07:26 hooks
drwxr-xr-x    1 SUPER.SUPER        SUPER       4096 Aug 10 07:26 info
-rw-r--r--    1 SUPER.SUPER        SUPER         73 Aug 10 07:26 description

./branches:
total 16
drwxr-xr-x    1 SUPER.SUPER        SUPER       4096 Aug 10 07:26 .
drwxr-xr-x    1 SUPER.SUPER        SUPER       4096 Aug 10 12:10 ..

./hooks:
total 43
drwxr-xr-x    1 SUPER.SUPER        SUPER       4096 Aug 10 07:26 .
drwxr-xr-x    1 SUPER.SUPER        SUPER       4096 Aug 10 12:10 ..
-rwxr-xr-x    1 SUPER.SUPER        SUPER        452 Aug 10 07:26
applypatch-msg.sample
-rwxr-xr-x    1 SUPER.SUPER        SUPER        896 Aug 10 07:26
commit-msg.sample
-rwxr-xr-x    1 SUPER.SUPER        SUPER        189 Aug 10 07:26
post-update.sample
-rwxr-xr-x    1 SUPER.SUPER        SUPER        398 Aug 10 07:26
pre-applypatch.sample
-rwxr-xr-x    1 SUPER.SUPER        SUPER       1704 Aug 10 07:26
pre-commit.sample
-rwxr-xr-x    1 SUPER.SUPER        SUPER       4957 Aug 10 07:26
pre-rebase.sample
-rwxr-xr-x    1 SUPER.SUPER        SUPER       1251 Aug 10 07:26
prepare-commit-msg.sample
-rwxr-xr-x    1 SUPER.SUPER        SUPER       3611 Aug 10 07:26
update.sample

./info:
total 17
drwxr-xr-x    1 SUPER.SUPER        SUPER       4096 Aug 10 07:26 .
drwxr-xr-x    1 SUPER.SUPER        SUPER       4096 Aug 10 12:10 ..
-rw-r--r--    1 SUPER.SUPER        SUPER        240 Aug 10 07:26 exclude
jojo@\hpitug:/usr/local/share/git-core/templates $

SUPER.SUPER on NonStop is equivalent to root in UNIX. Everything is readable
to everybody. Looks OK to me?

then use `git init --bare` in a new directory to copy in the templates,

and see if

its the template copying code that is making an incorrect copy.

"git init --bare" gives the same error. It isn't copying any of the
subdirectories, only the file 'description'

Bye, Jojo

On Fri, Aug 10, 2012 at 10:32 AM, Joachim Schmitz
[off-list ref] wrote:

quoted

then use `git init --bare` in a new directory to copy in the templates,

and see if

quoted

its the template copying code that is making an incorrect copy.

"git init --bare" gives the same error. It isn't copying any of the
subdirectories, only the file 'description'

Time to start debugging copy_templates_1 in builtin/init-db.c. :-(

From: Joachim Schmitz [mailto:jojo@schmitz-digital.de]
Sent: Friday, August 10, 2012 7:33 PM
To: 'Shawn Pearce'
Cc: 'git@vger.kernel.org'; 'rsbecker@nexbridge.com'
Subject: RE: Porting git to HP NonStop

quoted

From: Shawn Pearce [mailto:spearce@spearce.org]
Sent: Friday, August 10, 2012 6:28 PM
To: Joachim Schmitz
Cc: git@vger.kernel.org; rsbecker@nexbridge.com
Subject: Re: Porting git to HP NonStop

On Fri, Aug 10, 2012 at 8:04 AM, Joachim Schmitz
[off-list ref]
wrote:

<snip>

quoted

quoted

quoted

- HP NonStop doesn't have stat.st_blocks, this is used in

builtin/count-objects.c

quoted

around line 45, not sure yet how to fix that.

IIRC the block count is only used to give the user some notion of how
much disk was wasted by the repository. You could hack a macro that
redefines this as st_size.

OK, thanks, will try that.

Setting "NO_ST_BLOCKS_IN_STRUCT_STAT = YesPlease" in Makefile helps, no need
for further hacking ;-).

quoted

quoted

quoted

- HP NonStop doesn't have stat.st_?time.nsec, there are several
places

what an

quoted

"#ifdef USE_NSEC" is missing, I can provide a diff if needed
(offending
files: builtin/fetch-pack.c and read-cache.c).

I think this would be appreciated by anyone else that has a similar
problem where the platform lacks nsec.

Will do.

OK, here we go:

/usr/local/bin/diff -EBbu ./builtin/fetch-pack.c.orig ./builtin/fetch-pack.c

--- ./builtin/fetch-pack.c.orig 2012-07-30 15:50:38 -0500
+++ ./builtin/fetch-pack.c      2012-08-10 01:50:28 -0500

@@ -1096,7 +1096,9 @@
                int fd;

                mtime.sec = st.st_mtime;
+#ifdef USE_NSEC
                mtime.nsec = ST_MTIME_NSEC(st);
+#endif
                if (stat(shallow, &st)) {
                        if (mtime.sec)
                                die("shallow file was removed during

fetch");
/usr/local/bin/diff -EBbu ./read-cache.c.orig ./read-cache.c

--- ./read-cache.c.orig 2012-07-30 15:50:38 -0500
+++ ./read-cache.c      2012-08-09 10:57:57 -0500

@@ -72,8 +72,10 @@
 {
        ce->ce_ctime.sec = (unsigned int)st->st_ctime;
        ce->ce_mtime.sec = (unsigned int)st->st_mtime;
+#ifdef USE_NSEC
        ce->ce_ctime.nsec = ST_CTIME_NSEC(*st);
        ce->ce_mtime.nsec = ST_MTIME_NSEC(*st);
+#endif
        ce->ce_dev = st->st_dev;
        ce->ce_ino = st->st_ino;
        ce->ce_uid = st->st_uid;

@@ -1465,7 +1467,9 @@
        }
        strbuf_release(&previous_name_buf);
        istate->timestamp.sec = st.st_mtime;
+#ifdef USE_NSEC
        istate->timestamp.nsec = ST_MTIME_NSEC(st);
+#endif

        while (src_offset <= mmap_size - 20 - 8) {
                /* After an array of active_nr index entries,

@@ -1821,7 +1825,9 @@
        if (ce_flush(&c, newfd) || fstat(newfd, &st))
                return -1;
        istate->timestamp.sec = (unsigned int)st.st_mtime;
+#ifdef USE_NSEC
        istate->timestamp.nsec = ST_MTIME_NSEC(st);
+#endif
        return 0;
 }

Hope this helps?

Could you also consider adding the following:

/usr/local/bin/diff -EBbu ./git-compat-util.h.orig ./git-compat-util.h

--- ./git-compat-util.h.orig    2012-07-30 15:50:38 -0500
+++ ./git-compat-util.h 2012-08-10 09:59:56 -0500

@@ -74,7 +74,8 @@
 # define _XOPEN_SOURCE 500
 # endif
 #elif !defined(__APPLE__) && !defined(__FreeBSD__) && !defined(__USLC__) &&

\
-      !defined(_M_UNIX) && !defined(__sgi) && !defined(__DragonFly__)
+      !defined(_M_UNIX) && !defined(__sgi) && !defined(__DragonFly__) && \
+      !defined(__TANDEM)
 #define _XOPEN_SOURCE 600 /* glibc2 and AIX 5.3L need 500, OpenBSD needs
600 for S_ISLNK() */
 #define _XOPEN_SOURCE_EXTENDED 1 /* AIX 5.3L needs this */
 #endif

@@ -98,6 +99,11 @@
 #include <stdlib.h>
 #include <stdarg.h>
 #include <string.h>
+#ifdef __TANDEM
+# include <strings.h> /* for strcasecmp() */
+  typedef long int intptr_t;
+  typedef unsigned long int uintptr_t;
+#endif
 #include <errno.h>
 #include <limits.h>
 #include <sys/param.h>

Bye, Jojo

Am 10.08.2012 18:27, schrieb Shawn Pearce:

There is no need to define your own mmap(). Define NO_MMAP=1 in the
Makefile. Git already has its own fake mmap and knows how to write it
back to disk when making changes.

Or better to say: the fake mmap has functionality that is sufficient for
git. In particular, it does *not* write back changes to disk (it
supports only MAP_PRIVATE), and the mapped area does not change if the
file is changed by a third party.

-- Hannes

From: Joachim Schmitz [mailto:jojo@schmitz-digital.de]
Sent: Friday, August 10, 2012 10:09 PM
To: 'Shawn Pearce'
Cc: 'git@vger.kernel.org'; 'rsbecker@nexbridge.com'
Subject: RE: Porting git to HP NonStop

quoted

From: Joachim Schmitz [mailto:jojo@schmitz-digital.de]
Sent: Friday, August 10, 2012 7:33 PM
To: 'Shawn Pearce'
Cc: 'git@vger.kernel.org'; 'rsbecker@nexbridge.com'
Subject: RE: Porting git to HP NonStop

quoted

From: Shawn Pearce [mailto:spearce@spearce.org]
Sent: Friday, August 10, 2012 6:28 PM
To: Joachim Schmitz
Cc: git@vger.kernel.org; rsbecker@nexbridge.com
Subject: Re: Porting git to HP NonStop

On Fri, Aug 10, 2012 at 8:04 AM, Joachim Schmitz
[off-list ref]
wrote:

<snip>

quoted

quoted

quoted

quoted

- HP NonStop doesn't have stat.st_?time.nsec, there are several
places

what an

quoted

"#ifdef USE_NSEC" is missing, I can provide a diff if needed
(offending
files: builtin/fetch-pack.c and read-cache.c).

I think this would be appreciated by anyone else that has a similar
problem where the platform lacks nsec.

Will do.

OK, here we go:

/usr/local/bin/diff -EBbu ./builtin/fetch-pack.c.orig

./builtin/fetch-pack.c
<snip>

Sorry, this is not needed if I just set NO_NSEC, so just forget about it
(and thanks to Junio for telling be)

\

+      !defined(__TANDEM)
 #define _XOPEN_SOURCE 600 /* glibc2 and AIX 5.3L need 500, OpenBSD
needs 600 for S_ISLNK() */  #define _XOPEN_SOURCE_EXTENDED 1 /* AIX 5.3L
needs this */  #endif @@ -98,6 +99,11 @@  #include <stdlib.h>  #include
<stdarg.h>  #include <string.h>
+#ifdef __TANDEM
+# include <strings.h> /* for strcasecmp() */
+  typedef long int intptr_t;
+  typedef unsigned long int uintptr_t;
+#endif
 #include <errno.h>
 #include <limits.h>
 #include <sys/param.h>

This one still stands though, unless someone can come up with a better idea?

Bye, Jojo

From: Shawn Pearce [mailto:spearce@spearce.org]
Sent: Friday, August 10, 2012 7:38 PM
To: Joachim Schmitz
Cc: git@vger.kernel.org; rsbecker@nexbridge.com
Subject: Re: Porting git to HP NonStop

On Fri, Aug 10, 2012 at 10:32 AM, Joachim Schmitz

[off-list ref]

wrote:

quoted

quoted

then use `git init --bare` in a new directory to copy in the
templates,

and see if

quoted

its the template copying code that is making an incorrect copy.

"git init --bare" gives the same error. It isn't copying any of the
subdirectories, only the file 'description'

Time to start debugging copy_templates_1 in builtin/init-db.c. :-(

Found the problem: our mkdir(dir,flags) fails with ENOENT when dir ends with
a '/'.
Not sure whether this us a bug on out platform or just allowed by POSIX and
as such a wrong assumption in git though?

[shortly after]
A bit of googleing revealed that there is a GNUlib solution for this, which
claims that at least NetBSD 1.5.2 has the same problem.
(http://www.opensource.apple.com/source/gpatch/gpatch-2/patch/mkdir.c)

And apparently this has been discussed on the git mailing list too, 2 years
ago:
http://lists-archives.com/git/728359-git-s-use-of-mkdir-2.html, there's a
patch too.

For now I've fixed it like this:
/usr/local/bin/diff -EBbu ./builtin/init-db.c.orig ./builtin/init-db.c

--- ./builtin/init-db.c.orig    2012-08-19 03:55:50 -0500
+++ ./builtin/init-db.c 2012-08-19 03:39:57 -0500

@@ -25,7 +25,16 @@

 static void safe_create_dir(const char *dir, int share)
 {
+#ifdef __TANDEM /* our mkdir() can't cope with a trailing '/' */
+       char mydir[PATH_MAX];
+
+       strcpy(mydir,dir);
+       if (dir[strlen(dir)-1] == '/')
+               mydir[strlen(dir)-1] = '\0';
+       if (mkdir(mydir, 0777) < 0) {
+#else
        if (mkdir(dir, 0777) < 0) {
+#endif
                if (errno != EEXIST) {
                        perror(dir);
                        exit(1);

Bye, Jojo