[squid-dev] cppunit -> googletest / gmock?

[Alex Rousskov]

On 5/30/20 2:36 PM, Amos Jeffries wrote:
> On 31/05/20 5:27 am, Francesco Chemolli wrote:
>> starting from a PR in a conversation with Alex about our current
>> approach to unit testing being painful, I've checked what alternatives
>> would we have and how practical would they be.

FWIW, I would start from the other end -- trying to understand what the
primary problems are. What unit testing problems do we want to solve?


>> - googlemock promises to be vastly superior to our current approach

> Where are you seeing evidence of this?

I too would appreciate a brief/summary answer to this question -- what
are the primary advantages? If this is already listed on the web
somewhere, a reference is enough, of course.


> The main issue we have with cppunit itself is that when a test fails it
> is not clear from the output which assertion occurred, nor why.

FWIW, the above problem is only a minor issue for me.

The main issue for me is the amount of time spent on keeping "make
check" operational compared to the actual benefits of those unit tests.
I speculate that Squid would be in an overall better place today if all
those tests were simply deleted ten years ago.


> IIRC Alex an I have different ideas about the ideal focus of testing;
>  * I prefer the micro-test approach to demonstrate a high quality proof
> of code reliability.
>  * Alex has stated a preference for high level black-box testing of
> behaviour vs design requirements.

The above summary does not accurately reflect my position. And the
juxtaposition of "reliability" and "satisfying design requirements" is
misleading at best.

Any test, including any micro- and macro-test, can be valuable or a
waste of time. The "level" of the test is not what determines its value.
The focus should be on tests that, over the long term, maximize return
on investment. In the ideal world, that approach would result in a
_combination_ of micro/unit/macro/white/black/etc. tests, each applied
to problems where its effectiveness is maximized.

In our current situation, we can discuss, among other things, whether
the continued investment in unit tests is the right thing to do, and
whether that investment (if any) should be conditional on significantly
reducing maintenance overheads. It is a difficult discussion, especially
when there is a lack of agreed upon testing principles and goals.


HTH,

Alex.