Bug: dir.c traversing the filesystem: unindexed directories do not get recursed into when there is a (non-excluding) pathspec

What did you do before the bug happened? (Steps to reproduce your issue)
I created a new untracked directory `foo/`.
In .gitattributes i added `bar_yes* text`
and created files `foo/bar_yes` and `foo/bar_no` with content 'chicken'

I tried the following commands:
git grep --no-index chicken -- ':(attr:text)foo/*'
git ls-files --others --exclude-standard ':(attr:text)foo/*'
git status ':(attr:text)foo/*'


What did you expect to happen? (Expected behavior)
For all of the above, I expected `foo/bar_yes` to be returned.

What happened instead? (Actual behavior)
For all three commands, nothing was returned.

Anything else you want to add:
If I change the pathspec to ':(exclude,attr:text)foo/*' it works as expected.
`foo/bar_no` is returned.

(except for git status where the output is just `foo/`, but I think
this is expected.)

This behavior seems due to this section in dir.c:
https://github.com/git/git/blob/master/dir.c#L1893-L1908
for a pathspec like ':(attr:text)foo/*', this section of code will prevent
`foo/` from getting recursed.
It only checks for a pathspec match against the directory name, so any
files under the dir aren't checked.
Commenting that part of the code out, seems to fix this issue.
(for `git status`, `foo/` is returned both with `exclude` in the
pathspec and without)
But I am not sure if simply removing that code is the right fix.

[Enabled Hooks]
commit-msg
pre-commit
prepare-commit-msg