Docs And Blog Posts

What if unit tests are not tests?

Frustratingly I’ve lost the original conversation because it happened on Google hangouts on an account I no longer have access to (my TW account).

But Dan North’s view is that writing unit tests is not really “testing”.

There are some hints at the detail of this on this Twitter conversation. That conversation brings up the idea of talking about “examples” instead of tests. You can read more about that here.

Dan North also did a lightning talk on this topic at NDC London 2019, and here are my notes from that:

  • Kent Beck and Ward Cunningham wanted Incremental 
  • Soul of testing stolen by TDD
  • Not really test driven 
  • Is really automated checking
  • Increasing confidence for stakeholders through evidence 
  • Need to be ingenious 
  • Need to be pragmatic 
  • Make some serious trade offs
  • How confident is confident enough 
  • Customers don’t even know the systems they’re running 
  • Want everything delivered all at once 
  • TDD allows us to move at a measured pace, quickly and easily do stuff 
  • TDD had the wrong name 
  • Testing is another set of skills 
  • Naming things is really hard 

Mocks and Mocking

Mocking Terminology (Doubles, Mocks, Stubs, Fakes, Dummies and Spies)

My notes here were originally taken from this article.

Test Doubles

  • Overall term for different kinds of mocking objects, eg
    • Dummy
    • Stub
    • Spy
    • Mock
    • Fake

Dummy / Dummies

  • You pass this into something when you don’t care how it’s used
  • Like when you must pass an argument, but you know the argument will never be used.
  • The difference between a dummy and a stub is that the stub will actually get used by the code you pass it to, whereas the dummy won’t.

Stubs

  • A replacement for functionality you don’t want to call and you don’t want to test
  • Eg an authoriser that just returns true

Spy / Spies

  • An object that records when it is called and what arguments are passed to it
  • ! Be careful. It can make your tests fragile / brittle.
    • The more you spy, the tighter you couple your tests to the implementation of your system.
    • This is because your test is expecting certain functions to be called with certain arguments, so your tests will be broken by a refactor

Mocks

  • Mocks are always spies
  • Mocks know what they are testing
  • You move the assertion from the test, into the verify method of the mock
  • What the mock is testing is behaviour
  • The mock is not so interested in the return values of functions. It’s more interested in what function were called, with what arguments, when, and how often.
  • Moving the assertion in this way does create more coupling, but it makes it a lot easier to write a mocking tool
  • Famous paper by Martin Fowler that explains it well

Fakes, aka simulators

  • Like a stub, but contains behaviour
  • For instance, a user called Bob who is always authorised
    • He doesn’t exist really
    • But he will get us past code we are not interested in
  • “A Fake has business behavior. You can drive a fake to behave in different ways by giving it different data
  • A fake is the only kind of test double that has real business behaviour
  • But it can get very complicated – you can end up writing tests to test the fakes
  • They’re best avoided if possible”

Auto-mocking

  • Available from RhinoMocks, StructureMap, Moq, and many others.
  • Will build instances for you, injecting mock dependencies, so when your class has a lot of dependencies you don’t have to manually construct a mock for each dependency passed in
  • Nice description here

Mocks and TDD

  • Andy Longshaw: Reflections on brittle tests and mocking
  • “When we came up with mocks, they weren’t a TDD technique. They were a way to use TDD when designing the protocols between state machines. … We designed mock frameworks to be a brutal early warning sign that our design was going off the rails.” @natpryce
  • @DeeJayGraham: “mocks should only be used in early development of code to help sketch out an interface quickly. Then code should be testable not to need them - contrary to previous popular advice to use mocks where things are “a bit hard” to test”
    • @theMrTortoise: “When things are hard to test … Refactor.”
  • @tooky “All this talk of TDD, how it makes you face your design, how test doubles make things brittle, when to use mocks, when not to use mocks. Reminds me to share @sandimetz’ Magic Tricks of Testing: youtube / slide deck
  • Unit Testing Principles, Practices, and Patterns by @vkhorikov
  • @AndreasM_DK: “It might be tempting to mock away flakiness but… Simulate only things you completely understand - from @gojkoadzic - old but good”
  • Mocks should be used to highlight how your object affects and is affected by other objects and to see how the state of other components is being affected.
  • Mocks should not be exposing the innards of other objects.
  • You should be putting most of your thought into the protocols between objects rather than the objects themselves.
  • My original question on Twitter that led to some of the answers above

Ideas / Approaches

  • Steve Freeman: “Early XP trainings used to include a week-long “pure” project to really get the flow.”
  • Detroit TDD vs London TDD: https://tinnedfruit.com/list/20181004#:~:text=London-school%20TDD%20works%20’top,spies%2C%20stubs%2C%20etc
    • More on that, re double-loop and outside-in from Emily Bache, who says London School TDD has two features that distinguish it from Classic TDD. They are: Outside-In development with Double Loop TDD, and “Tell, Don’t Ask” Object Oriented Design.
  • “The problem is that the gap between katas and production code is still to big. Lately I’ve found mob programming in production code to be really effective. My current formula is katas + mob programming with someone who knows what they are doing.” @emilybache
  • “Moving from katas to real world problems, its important to know that you don’t have to write the tests in programming languages. It could be in the domain language - or @emilybache’s TDD with diagrams.” @thebddadvocate
    • “Approval testing is the term to google to learn more about how you could use sketches to do TDD” @emilybache
  • “You can try ignoring the refactor step for a while and measure the difference. Using code complexity metrics etc.” @thejonanshow
  • On the idea that it’s hard to convince stakeholders of the benefits of TDD because they think it will slow us down: “So, we are asking permission from management to think carefully? It makes my heart sink.” @keithb_b Go slow to go fast.

Misc

  • TDD tag in private Evernote
  • Cyber-dojo is a fantastic tool for getting straight into a kata in any language without setting up an IDE or dev environment. Really useful for workshopping in groups. Note that if you are not using it in a not-for-profit context, you should buy a (very cheap) licence. Details here.

Tips and Terms and Tools

Sliming

  • Sliming is the technique where you make your code do something trivial and hard-coded just to make your test pass - it’s unlikely this will end up being production code. Nice description here from Denise Yu (scroll down a bit)
  • “http://cyber-dojo.org is a great place to practice TDD.” - @sebrose
  • “Code retreat started by @coreyhaines is another fun way to learn.” @thejonanshow

NCrunch

  • If all tests are running after every change:
    • Change the NCrunch config so InstrumentationMode is set to Optimised
    • This is possible via Extensions NCrunch Configuration within Viksual Studio, but it didn’t seem to take effect until I manually edited [solution name].v3.ncrunchsolution to include the line <InstrumentationMode>Optimised</InstrumentationMode> in the settings section.
    • I confess I’m quite confused about this though - it definitely seemed to have an impact when I did this on the live-demo-2020-11-27 branch of the Reconciliate code base (see this commit), but the most up to date version of the master branch does not have this line in the config file… and it definitely doesn’t run all the tests on every change. [shrug]
  • File not found:
    • If a test fails because a file was not found, it may be that you need to include a test file in your NCrunch settings:
    • In Visual Studio:
      • Extensions NCrunch Configuration
      • In the top pane, double-click the shared settings for your project
      • In the bottom pane, set “Additional files to include” in the General section.
    • If you’re using approval tests (as used by Gilded Rose), the first time you run the test it will create a “received” file (eg ApprovalTest.ThirtyDays.received.txt). Create a copy of this with the suffix .approved.txt instead of .received.txt, and the file not found error should go away.

GitHub Repos

Sadly by necessity some of my repos are private. Those that are private are clearly marked. For those that are, please don’t ask me to share the code, because I can’t. They’re listed here purely for my reference.