On Tue, Nov 5, 2019 at 8:43 AM Mike Salvatore
[off-list ref] wrote:

quoted

quoted

but such approach is not mainstream.
I personally like the idea of testing the lowest level bits in isolation even if
they are not a part of any interface. I think that specifying the
interface using
unit tests and ensuring implementation correctness are complementary but
I haven't had much luck arguing this with our esteemed colleagues.

In general, testing public interfaces is preferable, however, I think it's
important to avoid becoming dogmatic. IMHO, it's more important to have tests
that are clear in what they test than to not write tests (or write confusing
tests) in order to adhere to a generalized principle.

That's a really good point.

quoted

So I think this is a very subtle point which is very widely
misunderstood. Most people write code and then write their tests,
following this practice along with only testing public interfaces
often causes people to just not test all of their code, which is
wrong.

The very nature of this situation is that the code was written before the tests.

quoted

The idea of only testing public interfaces is supposed to make people
think more carefully about what the composite layers of the program
is. If you are having difficulty getting decent coverage by only
testing your public interfaces, then it likely tells you that you have
one of two problems:

1) You have code that you don't need, and you should remove it.

2) One of the layers in your program is too think, and you should
introduce a new layer with a new public interface that you can test
through.

I think the second point here is problematic with how C is written in
the kernel. We don't really have any concept of public vs. private
inside the kernel outside of static vs. not static, which is much more
restricted.

I don't think we can expect developers to refactor large portions of complex
kernel code in order to improve its testability. I imagine this will happen
naturally over time, but I think we need to allow for developers to test
"private" code in the meanwhile.

My opinion is that it's more important to have tests than not. As evidence, I
submit the following commit:
https://github.com/torvalds/linux/commit/156e42996bd84eccb6acf319f19ce0cb140d00e3.

While not a major bug, this bug was discovered as a direct result of writing
these unit tests. So, in summary, I see value in "testing the lowest level bits
in isolation", even if it doesn't necessarily represent the Gold Standard in
Unit Testing.

You're right.

I think, in summary, it seems that pretty much everyone agrees that we
need to provide a mechanism for testing low level bits of code in
isolation in such a way that the tests are easy to write, and don't
require the code under test to be massively refactored. Beyond that it
seems that we are mostly in between either including tests in the
source for the code under test or using the __visible_for_testing
mechanism. Between the two, it seems the preference is for including
the test in the source.

So I think I will still send out a patch to add in the
__visible_for_testing mechanism in case someone wants to use it in the
future.

Nevertheless, I will reformat this patch to include the tests in the
files that are under test. One question, do we have a preference
between putting all the tests in the same file as the code under test?
Or do we want to put them in a separate file that gets #included in
the file under test? I have a preference for the latter, but will
defer to what everyone else wants.

Thanks everyone!