Comments on: Test Anti-Pattern: Proving The Code is Written Like the Code Is Written http://barelyenough.org/blog/2005/10/test-anti-pattern-proving-the-code-is-written-like-the-code-is-written/ Thu, 04 Dec 2008 02:19:15 +0000 http://wordpress.org/?v=2.5 By: Peter Williams » Blog Archive » Mocking http://barelyenough.org/blog/2005/10/test-anti-pattern-proving-the-code-is-written-like-the-code-is-written/#comment-19108 Peter Williams » Blog Archive » Mocking Thu, 14 Jun 2007 04:07:02 +0000 http://pezra.barelyenough.org/blog/?p=175#comment-19108 <p>[...] Mr Whitney recently posted an article in which he described mock objects as “bug aggregators”. I once held a similar point of view. Back then my belief was that test doubles (mock, stub, etc) should only be used when real objects would not work, either because they were too hard to setup or because they were too slow. Recently, however, my thinking towards mocks has become a bit more nuanced1. [...]</p> […] Mr Whitney recently posted an article in which he described mock objects as “bug aggregators”. I once held a similar point of view. Back then my belief was that test doubles (mock, stub, etc) should only be used when real objects would not work, either because they were too hard to setup or because they were too slow. Recently, however, my thinking towards mocks has become a bit more nuanced1. […]

]]> By: Peter Williams http://barelyenough.org/blog/2005/10/test-anti-pattern-proving-the-code-is-written-like-the-code-is-written/#comment-486 Peter Williams Sat, 12 Nov 2005 20:27:22 +0000 http://pezra.barelyenough.org/blog/?p=175#comment-486 <p>Basically the unit test creates a mock version of the object to be tested. Then it subclass the class to be tested and overrides the protected methods that the method being tested will call with calls to the mock object.</p> <p>This results in a test which verifies that the method being tested really does call the protected methods with the correct arguments.</p> <p>I do not think there are any major syntax errors in that code, but I have not compiled it so there might be a couple of small ones.</p> Basically the unit test creates a mock version of the object to be tested. Then it subclass the class to be tested and overrides the protected methods that the method being tested will call with calls to the mock object.

This results in a test which verifies that the method being tested really does call the protected methods with the correct arguments.

I do not think there are any major syntax errors in that code, but I have not compiled it so there might be a couple of small ones.

]]> By: Robert Konigsberg http://barelyenough.org/blog/2005/10/test-anti-pattern-proving-the-code-is-written-like-the-code-is-written/#comment-485 Robert Konigsberg Sat, 12 Nov 2005 16:41:32 +0000 http://pezra.barelyenough.org/blog/?p=175#comment-485 <p>I'm not sure I understand the flow of your unit tests -- your syntax looks wrong. Perhaps you're assuming I see something where precise syntax is not necessary, but I don't see it.</p> I’m not sure I understand the flow of your unit tests — your syntax looks wrong. Perhaps you’re assuming I see something where precise syntax is not necessary, but I don’t see it.

]]> By: Peter Williams http://barelyenough.org/blog/2005/10/test-anti-pattern-proving-the-code-is-written-like-the-code-is-written/#comment-484 Peter Williams Thu, 10 Nov 2005 17:20:38 +0000 http://pezra.barelyenough.org/blog/?p=175#comment-484 <p>Adrian, I do not believe that proving correctness is an reasonable goal for non-trival applications and testing for portability sets off my "your not going to need it" filter. :) There are certainly situations where testing private methods the most cost effective approach but I think that testing a private method should be presumed wrong. When there is sufficient cause to test private methods an <a href="http://en.wikipedia.org/wiki/Affirmative_defense" rel="nofollow">affirmative defense</a> can be used to show that it was the best workable approach.</p> Adrian, I do not believe that proving correctness is an reasonable goal for non-trival applications and testing for portability sets off my “your not going to need it” filter. :) There are certainly situations where testing private methods the most cost effective approach but I think that testing a private method should be presumed wrong. When there is sufficient cause to test private methods an affirmative defense can be used to show that it was the best workable approach.

]]> By: Adrian Mouat http://barelyenough.org/blog/2005/10/test-anti-pattern-proving-the-code-is-written-like-the-code-is-written/#comment-483 Adrian Mouat Fri, 04 Nov 2005 08:43:39 +0000 http://pezra.barelyenough.org/blog/?p=175#comment-483 <p>>I think that verifying that all of the required behavior is provided is >sufficient. After you have proved that what else is there?</p> <p>Proving that the behaviour is correct? Making the code easier to port/maintain by avoiding future bugs?</p> <p>A bug in class may produce sub-optimal or almost correct behaviour that is difficult to reproduce easily or directly from public interfaces.</p> <p>Also, when a bug is found, it will probably seem logical and easier to make the regression test as close to the method as possible.</p> <p>It may, in general, be better to avoid testing private methods but I'd be wary of giving others any rules about this.</p> >I think that verifying that all of the required behavior is provided is >sufficient. After you have proved that what else is there?

Proving that the behaviour is correct? Making the code easier to port/maintain by avoiding future bugs?

A bug in class may produce sub-optimal or almost correct behaviour that is difficult to reproduce easily or directly from public interfaces.

Also, when a bug is found, it will probably seem logical and easier to make the regression test as close to the method as possible.

It may, in general, be better to avoid testing private methods but I’d be wary of giving others any rules about this.

]]> By: Peter Williams http://barelyenough.org/blog/2005/10/test-anti-pattern-proving-the-code-is-written-like-the-code-is-written/#comment-481 Peter Williams Thu, 03 Nov 2005 22:22:52 +0000 http://pezra.barelyenough.org/blog/?p=175#comment-481 <p>Adrian,</p> <p>I do not think that "unit" is a very useful way to think about what to test. Is a unit a single control structure, a method, a class, all the classes/mixins in package/namespace (or logical equivelent), etc? A unit is all of those things as one time or another. Which one just depends on your point of view. I think that what we should be testing is not "units" but behavior. I think that verifying that all of the required behavior is provided is sufficient. After you have proved that what else is there?</p> <p>I am not really arguing that testing private methods is intrinsically evil. Just that, in the general case, it is far more expensive than it is worth. There are certainly exceptions to this rule, I even gave a few of them in my post, but I think that testing non-globally public methods is a design smell.</p> Adrian,

I do not think that “unit” is a very useful way to think about what to test. Is a unit a single control structure, a method, a class, all the classes/mixins in package/namespace (or logical equivelent), etc? A unit is all of those things as one time or another. Which one just depends on your point of view. I think that what we should be testing is not “units” but behavior. I think that verifying that all of the required behavior is provided is sufficient. After you have proved that what else is there?

I am not really arguing that testing private methods is intrinsically evil. Just that, in the general case, it is far more expensive than it is worth. There are certainly exceptions to this rule, I even gave a few of them in my post, but I think that testing non-globally public methods is a design smell.

]]> By: Peter Williams http://barelyenough.org/blog/2005/10/test-anti-pattern-proving-the-code-is-written-like-the-code-is-written/#comment-480 Peter Williams Thu, 03 Nov 2005 21:33:50 +0000 http://pezra.barelyenough.org/blog/?p=175#comment-480 <p>Andrew,</p> <p>I did not mean to imply that tests should only assert things about the results of a method. Rather that if a "bug" in the code is not detectable from a public interface of the target class, or some closely related class, it probably does not matter. When you have methods that do not return a result that is sufficient for verification you can usually call one of more other methods in the public interface to verify that the behavior of the object in question changed in an appropriate way.</p> <p>Interactions with external systems, like databases, are generally difficult to deal with in unit testing, I think. However, for that class of problems I usually have the object to be tested operate on a mock database connection rather mocking and subclassing the class to be tested.</p> Andrew,

I did not mean to imply that tests should only assert things about the results of a method. Rather that if a “bug” in the code is not detectable from a public interface of the target class, or some closely related class, it probably does not matter. When you have methods that do not return a result that is sufficient for verification you can usually call one of more other methods in the public interface to verify that the behavior of the object in question changed in an appropriate way.

Interactions with external systems, like databases, are generally difficult to deal with in unit testing, I think. However, for that class of problems I usually have the object to be tested operate on a mock database connection rather mocking and subclassing the class to be tested.

]]> By: Adrian Mouat http://barelyenough.org/blog/2005/10/test-anti-pattern-proving-the-code-is-written-like-the-code-is-written/#comment-479 Adrian Mouat Thu, 03 Nov 2005 15:56:13 +0000 http://pezra.barelyenough.org/blog/?p=175#comment-479 <p>I was thinking it's possible to test via reflection, which is another way to use methods, but not a very nice one.</p> <p>For another viewpoint on all this, see:</p> <p>http://www.artima.com/suiterunner/private.html</p> I was thinking it’s possible to test via reflection, which is another way to use methods, but not a very nice one.

For another viewpoint on all this, see:

http://www.artima.com/suiterunner/private.html

]]> By: Peter Seymour http://barelyenough.org/blog/2005/10/test-anti-pattern-proving-the-code-is-written-like-the-code-is-written/#comment-478 Peter Seymour Thu, 03 Nov 2005 13:54:41 +0000 http://pezra.barelyenough.org/blog/?p=175#comment-478 <p>If you can write a test for a method and the method is called from your user code then that means there are at least two distinct ways to use the method. Perhaps this means it is logically complete and maybe even generic enough to be a unit of its own.</p> If you can write a test for a method and the method is called from your user code then that means there are at least two distinct ways to use the method. Perhaps this means it is logically complete and maybe even generic enough to be a unit of its own.

]]> By: Adrian Mouat http://barelyenough.org/blog/2005/10/test-anti-pattern-proving-the-code-is-written-like-the-code-is-written/#comment-477 Adrian Mouat Thu, 03 Nov 2005 13:07:43 +0000 http://pezra.barelyenough.org/blog/?p=175#comment-477 <p>Not sure I entirely agree with this.</p> <p>A unit test should be exactly that - it should test a "unit" of code.</p> <p>Whether this code is public or private should be irrelevant. I certainly can think of cases in my own code where I would <em>like</em> to test a private function but it is difficult via junit.</p> <p>Why shouln't I test that my algorithm (say a sort or lcs algorithm) returns the correct results directly? I don't believe that it is private due to bad design.</p> <p>I agree that tests which show the intended use of the software are better, but I don't believe they are necessarily sufficient.</p> Not sure I entirely agree with this.

A unit test should be exactly that - it should test a “unit” of code.

Whether this code is public or private should be irrelevant. I certainly can think of cases in my own code where I would like to test a private function but it is difficult via junit.

Why shouln’t I test that my algorithm (say a sort or lcs algorithm) returns the correct results directly? I don’t believe that it is private due to bad design.

I agree that tests which show the intended use of the software are better, but I don’t believe they are necessarily sufficient.

]]>