Git Modifying DLL

I'm having an issue with git modifying a DLL file and corrupting it.
When I download the original working file, it has a hash starting with
8FE400... I then commit the DLL and push it to our repo. When I
download the file from the repo, the DLL can't be loaded, and it has a
hash starting with E004FB...

Opening the DLL in a hex editor and using the compare feature, there's
a single change to the file. In the original, the byte code starting
at 0x0074 is 2E 0D 0D 0A 24.
In the git file, the byte code starting at 0x0074 is 2E 0D 0A 24

A single carriage return character (0x0D) has been removed, and the
file size has changed from 260,608 bytes to 260,607 bytes.

I ruled out the possibility that the repo server was doing anything to
the file because I deleted the file in my local repository then ran
"git reset --hard HEAD" to restore the file, and the hash had once
again changed to E004FB...

OS: Windows 10.0.19043 pro
git: 2.32.0.windows.1

I have no settings configured for git behavior handling line endings.

The original DLL can be found here:
https://www.powershellgallery.com/packages/PSWindowsUpdate/2.1.1.2

Navigate to "Manual Download", download the nuget package and unzip
it. The file is PSWindowsUpdate.dll

Thank you,
Jonathon

On August 19, 2021 1:59 PM, Jonathon Anderson wrote:

I'm having an issue with git modifying a DLL file and corrupting it.
When I download the original working file, it has a hash starting with 8FE400... I then commit the DLL and push it to our repo. When I
download the file from the repo, the DLL can't be loaded, and it has a hash starting with E004FB...

Opening the DLL in a hex editor and using the compare feature, there's a single change to the file. In the original, the byte code starting
at 0x0074 is 2E 0D 0D 0A 24.
In the git file, the byte code starting at 0x0074 is 2E 0D 0A 24

A single carriage return character (0x0D) has been removed, and the file size has changed from 260,608 bytes to 260,607 bytes.

I ruled out the possibility that the repo server was doing anything to the file because I deleted the file in my local repository then ran "git
reset --hard HEAD" to restore the file, and the hash had once again changed to E004FB...

OS: Windows 10.0.19043 pro
git: 2.32.0.windows.1

I have no settings configured for git behavior handling line endings.

The original DLL can be found here:
https://www.powershellgallery.com/packages/PSWindowsUpdate/2.1.1.2

Navigate to "Manual Download", download the nuget package and unzip it. The file is PSWindowsUpdate.dll

Have you set up an entry for *.dll as binary in your .gitattributes file?

-Randall

I had not. I tested that and it worked. I assumed that git would
automatically treat dll files as binary. Thanks for the help!

On Thu, Aug 19, 2021 at 1:13 PM Randall S. Becker
[off-list ref] wrote:

On August 19, 2021 1:59 PM, Jonathon Anderson wrote:

quoted

I'm having an issue with git modifying a DLL file and corrupting it.
When I download the original working file, it has a hash starting with 8FE400... I then commit the DLL and push it to our repo. When I
download the file from the repo, the DLL can't be loaded, and it has a hash starting with E004FB...

Opening the DLL in a hex editor and using the compare feature, there's a single change to the file. In the original, the byte code starting
at 0x0074 is 2E 0D 0D 0A 24.
In the git file, the byte code starting at 0x0074 is 2E 0D 0A 24

A single carriage return character (0x0D) has been removed, and the file size has changed from 260,608 bytes to 260,607 bytes.

I ruled out the possibility that the repo server was doing anything to the file because I deleted the file in my local repository then ran "git
reset --hard HEAD" to restore the file, and the hash had once again changed to E004FB...

OS: Windows 10.0.19043 pro
git: 2.32.0.windows.1

I have no settings configured for git behavior handling line endings.

The original DLL can be found here:
https://www.powershellgallery.com/packages/PSWindowsUpdate/2.1.1.2

Navigate to "Manual Download", download the nuget package and unzip it. The file is PSWindowsUpdate.dll

Have you set up an entry for *.dll as binary in your .gitattributes file?

-Randall

First of all, thanks for the report.
(And please, no top-posting in this list, see my comments at the end)

On Thu, Aug 19, 2021 at 01:21:03PM -0500, Jonathon Anderson wrote:

I had not. I tested that and it worked. I assumed that git would
automatically treat dll files as binary. Thanks for the help!

On Thu, Aug 19, 2021 at 1:13 PM Randall S. Becker
[off-list ref] wrote:

quoted

On August 19, 2021 1:59 PM, Jonathon Anderson wrote:

quoted

I'm having an issue with git modifying a DLL file and corrupting it.
When I download the original working file, it has a hash starting with 8FE400... I then commit the DLL and push it to our repo. When I
download the file from the repo, the DLL can't be loaded, and it has a hash starting with E004FB...

Opening the DLL in a hex editor and using the compare feature, there's a single change to the file. In the original, the byte code starting
at 0x0074 is 2E 0D 0D 0A 24.
In the git file, the byte code starting at 0x0074 is 2E 0D 0A 24

A single carriage return character (0x0D) has been removed, and the file size has changed from 260,608 bytes to 260,607 bytes.

I ruled out the possibility that the repo server was doing anything to the file because I deleted the file in my local repository then ran "git
reset --hard HEAD" to restore the file, and the hash had once again changed to E004FB...

OS: Windows 10.0.19043 pro
git: 2.32.0.windows.1

I have no settings configured for git behavior handling line endings.

The original DLL can be found here:
https://www.powershellgallery.com/packages/PSWindowsUpdate/2.1.1.2

Navigate to "Manual Download", download the nuget package and unzip it. The file is PSWindowsUpdate.dll

Have you set up an entry for *.dll as binary in your .gitattributes file?

-Randall

I downloaded the dll.

I can not reproduce the issue here - so I wonder, what is going on.

Could you run the following experiment, please ?
(in git bash or so)

mkdir  dlltest
cd dlltest
git init
cp whereveritis/PSWindowsUpdate.dll .

git ls-files -o --eol

# I get
git ls-files --eol -o PSWindowsUpdate.dll
i/      w/-text attr/			PSWindowsUpdate.dll

This means, that the file is "classified" as "-text" in the working tree.
"-text" means the same as binary.
And git should not change CRLF at all.

Do you have any more .gitattributes in you repo?
Of course, you can run the command inside your repo.
Omit the "-o" for other and simply run
git ls-files --eol

On Thu, Aug 19, 2021 at 01:21:03PM -0500, Jonathon Anderson wrote:

I had not. I tested that and it worked. I assumed that git would
automatically treat dll files as binary. Thanks for the help!

Git doesn't know about any file extensions by default. Its default "is
it binary" test looks for NUL bytes in the first 8k or so of the file.
I'd expect your DLL would probably have such a NUL byte.

Is it possible you have other .gitattributes set which are confusing
things?

You might try:

  git check-attr --all <path>

or:

  git ls-files --stdin | git check-attr --stdin --all

-Peff

On Fri, Aug 20, 2021 at 1:00 PM Jeff King [off-list ref] wrote:

On Thu, Aug 19, 2021 at 01:21:03PM -0500, Jonathon Anderson wrote:

quoted

I had not. I tested that and it worked. I assumed that git would
automatically treat dll files as binary. Thanks for the help!

Git doesn't know about any file extensions by default. Its default "is
it binary" test looks for NUL bytes in the first 8k or so of the file.
I'd expect your DLL would probably have such a NUL byte.

Is it possible you have other .gitattributes set which are confusing
things?

You might try:

  git check-attr --all <path>

or:

  git ls-files --stdin | git check-attr --stdin --all

-Peff

When I remove '*.dll binary" from .gitattributes, I get this:

$ git check-attr --all ./PSWindowsUpdate.dll
./PSWindowsUpdate.dll: text: set
./PSWindowsUpdate.dll: eol: lf



When I add "*.dll binary" to .gitattributes, I get this:

$ git check-attr --all ./PSWindowsUpdate.dll
./PSWindowsUpdate.dll: binary: set
./PSWindowsUpdate.dll: diff: unset
./PSWindowsUpdate.dll: merge: unset
./PSWindowsUpdate.dll: text: unset
./PSWindowsUpdate.dll: eol: lf


I can also confirm that there are null bytes in the first 8k bytes.
These are the first 16 bytes

4D 5A 90 00 03 00 00 00 04 00 00 00 FF FF 00 00

Thanks,
Jonathon

On 2021-08-20 at 18:47:02, Jonathon Anderson wrote:

On Fri, Aug 20, 2021 at 1:00 PM Jeff King [off-list ref] wrote:

quoted

On Thu, Aug 19, 2021 at 01:21:03PM -0500, Jonathon Anderson wrote:

quoted

I had not. I tested that and it worked. I assumed that git would
automatically treat dll files as binary. Thanks for the help!

Git doesn't know about any file extensions by default. Its default "is
it binary" test looks for NUL bytes in the first 8k or so of the file.
I'd expect your DLL would probably have such a NUL byte.

Is it possible you have other .gitattributes set which are confusing
things?

You might try:

  git check-attr --all <path>

or:

  git ls-files --stdin | git check-attr --stdin --all

-Peff

When I remove '*.dll binary" from .gitattributes, I get this:

$ git check-attr --all ./PSWindowsUpdate.dll
./PSWindowsUpdate.dll: text: set
./PSWindowsUpdate.dll: eol: lf

Yes, this is definitely not correct.  The flag "text" being set tells
Git to do line-ending conversion and "eol=lf" says to convert line
endings into LF.

You should look for things in your .gitattributes file that say
something like "* text", which you probably don't want.  You could use
"* text=auto", which should be fine for most cases, though.  It's also
possible those aren't in a .gitattributes file in your repository but
one elsewhere on your system.  You can check gitattributes(5) for the
locations of other files that can affect it.

As a note, it is best practice not to check binary dependencies or build
artifacts into the repo.  Those are best stored elsewhere, such as an
artifact server.
-- 
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA

On August 20, 2021 3:15 PM, brian m. carlson wrote:

On 2021-08-20 at 18:47:02, Jonathon Anderson wrote:

quoted

On Fri, Aug 20, 2021 at 1:00 PM Jeff King [off-list ref] wrote:

quoted

On Thu, Aug 19, 2021 at 01:21:03PM -0500, Jonathon Anderson wrote:

quoted

I had not. I tested that and it worked. I assumed that git would
automatically treat dll files as binary. Thanks for the help!

Git doesn't know about any file extensions by default. Its default
"is it binary" test looks for NUL bytes in the first 8k or so of the file.
I'd expect your DLL would probably have such a NUL byte.

Is it possible you have other .gitattributes set which are confusing
things?

You might try:

  git check-attr --all <path>

or:

  git ls-files --stdin | git check-attr --stdin --all

-Peff

When I remove '*.dll binary" from .gitattributes, I get this:

$ git check-attr --all ./PSWindowsUpdate.dll
./PSWindowsUpdate.dll: text: set
./PSWindowsUpdate.dll: eol: lf

Yes, this is definitely not correct.  The flag "text" being set tells Git to do line-ending conversion and "eol=lf" says to convert line endings
into LF.

You should look for things in your .gitattributes file that say something like "* text", which you probably don't want.  You could use
"* text=auto", which should be fine for most cases, though.  It's also possible those aren't in a .gitattributes file in your repository but one
elsewhere on your system.  You can check gitattributes(5) for the locations of other files that can affect it.

As a note, it is best practice not to check binary dependencies or build artifacts into the repo.  Those are best stored elsewhere, such as an
artifact server.

OT: Unless you are using git as an artifact repository to retain the dependency relationship between objects and the source commits that built them for PCI and SWIFT compliance. git is exceptional when it comes to evidence-of-origin forensics of specific installation artifacts when release structures are more complex than "deploy everything" use cases. But that may be a discussion we can have elsewhere.

Just my $20.00.
-Randall

So, I somehow didn't realize what this was when I was looking at it,
but this is the first line in our .gitattributes

* text eol=lf

It all makes sense now.

And, yea, we're using git to retain dependency relationships and bring
those files under version control so we can audit changes and go
through the full peer review process, etc

On Fri, Aug 20, 2021 at 2:51 PM Randall S. Becker
[off-list ref] wrote:

On August 20, 2021 3:15 PM, brian m. carlson wrote:

quoted

On 2021-08-20 at 18:47:02, Jonathon Anderson wrote:

quoted

On Fri, Aug 20, 2021 at 1:00 PM Jeff King [off-list ref] wrote:

quoted

On Thu, Aug 19, 2021 at 01:21:03PM -0500, Jonathon Anderson wrote:

quoted

I had not. I tested that and it worked. I assumed that git would
automatically treat dll files as binary. Thanks for the help!

Git doesn't know about any file extensions by default. Its default
"is it binary" test looks for NUL bytes in the first 8k or so of the file.
I'd expect your DLL would probably have such a NUL byte.

Is it possible you have other .gitattributes set which are confusing
things?

You might try:

  git check-attr --all <path>

or:

  git ls-files --stdin | git check-attr --stdin --all

-Peff

When I remove '*.dll binary" from .gitattributes, I get this:

$ git check-attr --all ./PSWindowsUpdate.dll
./PSWindowsUpdate.dll: text: set
./PSWindowsUpdate.dll: eol: lf

Yes, this is definitely not correct.  The flag "text" being set tells Git to do line-ending conversion and "eol=lf" says to convert line endings
into LF.

You should look for things in your .gitattributes file that say something like "* text", which you probably don't want.  You could use
"* text=auto", which should be fine for most cases, though.  It's also possible those aren't in a .gitattributes file in your repository but one
elsewhere on your system.  You can check gitattributes(5) for the locations of other files that can affect it.

As a note, it is best practice not to check binary dependencies or build artifacts into the repo.  Those are best stored elsewhere, such as an
artifact server.

OT: Unless you are using git as an artifact repository to retain the dependency relationship between objects and the source commits that built them for PCI and SWIFT compliance. git is exceptional when it comes to evidence-of-origin forensics of specific installation artifacts when release structures are more complex than "deploy everything" use cases. But that may be a discussion we can have elsewhere.

Just my $20.00.
-Randall