BDD: Testing absence of behaviour in Mockito via a custom Mock Controller

One thing I really miss with Mockito is the use of a Mock Controller (such as with EasyMock) to collect and control mocks particularly in testing for the absence of behaviour.

To illustrate lets look at the following example:

public class ServiceImpl implements Service {

    @Autowired
    private ServiceDao serviceDao;

    public void doSomething(String value){
        serviceDao.doSomething(value);
    }
}

 

public class ServiceImplTest {

    @InjectMocks
    private ServiceImpl serviceImpl;

    @Mock
    private ServiceDao serviceDaoMock;

    @BeforeMethod
    public void setUp(){
        serviceImpl = new ServiceImpl();
        MockitoAnnotations.init(this);
    }

    @Test
    public void shouldServiceDoSomething(){
        serviceImpl.doSomething("value");
        verify(serviceDaoMock).doesSomething("value");
        MockitoAnnotations.verifyNoMoreInteractions(serviceDaoMock);
    }
}

Which looks okay on first inspection.

You can from the 'shouldServiceDoSomething' test that the mock does something and that there is no more behaviour on it. But what happens when say another developer adds another service which effects 'doSomething' but misses the test?

 To illustrate:

public class ServiceImpl implements Service {

    @Autowired
    private ServiceDao serviceDao;

    @Autowired
    private AnotherServiceDao anotherServiceDao;

    public void doSomething(String value){
        serviceDao.doSomething(value);
        anotherServiceDao.doSomething(value);
    }

    public void doSomethingElse(String value){
        anotherServiceDao.doSomething(value);
    }
}

 

public class ServiceImplTest {

    @InjectMocks
    private ServiceImpl serviceImpl;

    @Mock
    private ServiceDao serviceDaoMock;

    @Mock
    private AnotherServiceDao anotherServiceDaoMock;

    @BeforeMethod
    public void setUp(){
        serviceImpl = new ServiceImpl();
        MockitoAnnotations.init(this);
    }

    @Test
    public void shouldServiceDoSomething(){
        serviceImpl.doSomething("value");
        verify(serviceDaoMock).doesSomething("value");
        // Behaviour changed but will not break ;-(
        MockitoAnnotations.verifyNoMoreInteractions(serviceDaoMock);
    }

    @Test
    public void shouldServiceDoSomethingElse(){
        serviceImpl.doSomethingElse("value");
        verify(anotherServiceDaoMock).doesSomething("value");
        MockitoAnnotations.verifyNoMoreInteractions(anotherServiceDaoMock);
    }
}

Do you see the problem? 

 Both tests pass, yet the true behaviour of the method 'doSomething' is no longer tested by 'shouldServiceDoSomething' And the developers are none the wiser to this expectation as nothing breaks.

The developers are forced to update the tests for 'verifyNoMoreInteractions' passing in all mocks for the test to fail, which is great in theory but not practical. 

This is where a MockitoAnnotationsExtension does it's work. 

To illustrate:

public class ServiceImplTest {

    @InjectMocks
    private ServiceImpl serviceImpl;

    @Mock
    private ServiceDao serviceDaoMock;

    @Mock
    private AnotherServiceDao anotherServiceDaoMock;

    @BeforeMethod
    public void setUp(){
        serviceImpl = new ServiceImpl();
        MockitoAnnotationsExtension.init(this);
    }

    @Test
    public void shouldServiceDoSomething(){
        serviceImpl.doSomething("value");
        verify(serviceDaoMock).doesSomething("value");
        // Will break now :-)
        MockitoAnnotationsExtension.verifyNoMoreInteractions();
    }

    @Test public void shouldServiceDoSomethingElse(){
        serviceImpl.doSomethingElse("value");
        verify(anotherServiceDaoMock).doesSomething("value");
        MockitoAnnotationsExtension.verifyNoMoreInteractions();
    }
}

In which case we have complete control over all the mocks and the test 'shouldServiceDoSomething' will break to indicate to us that there has been a change in the expected behaviour of the method. 

 Which is exactly what we want! 

 To illustrate a MockitoAnnotationsExtension:

package com.foo.test;

import org.mockito.InOrder;
import org.mockito.Mockito;
import org.mockito.MockitoAnnotations;
import org.mockito.MockitoDebugger;
import org.mockito.internal.debugging.MockitoDebuggerImpl;

import java.lang.reflect.Field;
import java.util.HashSet;
import java.util.Set;

import static org.apache.commons.lang.Validate.notNull;

/**
* An extension which allows for assertions over all Mockito mocks defined in a test class
* In this way behaviour against ALL test class defined mocks (@Mock) can be asserted.
*
* @see org.mockito.MockitoAnnotations
* @see org.mockito.InOrder
* @see org.mockito.Mockito
*/
public final class MockitoAnnotationsExtension {

    private MockitoAnnotationsExtension(){} // NOSONAR

    private static final MockitoDebugger mockitoDebugger = new MockitoDebuggerImpl();

    private static Object[] allAnnotatedMocksCapture = null;

    public static void initMocksExtension(final Object testClass){
        MockitoAnnotations.initMocks(testClass);
        allAnnotatedMocksCapture = getAllAnnotatedMocks(testClass);
    }

    public static void tearDown(){
        allAnnotatedMocksCapture = null;
    }

    /**
    * Wrapper for Mockito verifyNoMoreInteractions
    * @see org.mockito.Mockito#verifyNoMoreInteractions(Object...)
    */
    public static void verifyNoMoreInteractionsExtension() {
        validateMethodUse();
        Mockito.verifyNoMoreInteractions(allAnnotatedMocksCapture);
    }

    /**
    * Wrapper for Mockito verifyZeroInteractions
    * @see org.mockito.Mockito#verifyZeroInteractions(Object...)
    */
    public static void verifyZeroInteractionsExtension() {
        validateMethodUse();
        Mockito.verifyZeroInteractions(allAnnotatedMocksCapture);
    }

    /**
    * Wrapper for Mockito reset
    * @see org.mockito.Mockito#reset(Object[])
    */
    public static void resetExtension() {
        validateMethodUse();
        Mockito.reset(allAnnotatedMocksCapture);
    }

    /**
    * Wrapper for Mockito inOrder
    * @see org.mockito.Mockito#inOrder(Object...)
    */
    public static InOrder inOrderExtension() {
        validateMethodUse();
        return Mockito.inOrder(allAnnotatedMocksCapture);
    }

    /**
    * Wrapper for MockitoDebuggerImpl printInvocations
    * @see org.mockito.MockitoDebugger#printInvocations(Object...)
    */
    public static void debugPrintInvocationsExtension(){
        validateMethodUse();
        mockitoDebugger.printInvocations(allAnnotatedMocksCapture);
    }

    private static void validateMethodUse(){
        notNull(allAnnotatedMocksCapture, "MockitoAnnotationsExtension.initMocksExtension must be called prior to     this method call");
    }

    private static boolean isInjectedMock(Field field){
        return field.getAnnotation(org.mockito.Mock.class) != null;
    }

    static Object[] getAllAnnotatedMocks(final Object testClass) {
        try{
            Set <Object> mocks = new HashSet<Object>();
            for (Field field : testClass.getClass().getDeclaredFields()) {
                if (isInjectedMock(field)){
                    field.setAccessible(true);
                    mocks.add(field.get(testClass));
                }
            }
            return mocks.toArray();
        }catch (Exception unexpectedWrapped){
            throw new RuntimeException(unexpectedWrapped);
        }
    }
}

I was in discussions with Mockito over the use of the control API and it might be a feature in future. But until then this is a work around!

BDD: Running Twist Features Inside Unit Tests

The following code is a work around to support running Twist features within Unit Test through any IDE.

It's quite useful when debugging scenarios and you've just had enough of the Eclipse Plugin or would just prefer to use something else! :-)

I hope it proves useful to other Twist users out there.

package com.test.twist.runner;

import com.thoughtworks.twist.core.execution.ant.ScenarioExecutorAntMain;
import org.springframework.test.util.ReflectionTestUtils;

import java.io.File;
import java.util.HashMap;
import java.util.Map;

import static com.thoughtworks.twist.core.execution.ant.ScenarioExecutorAntMain.*;

public abstract class AbstractTwistScenarioRunner {

    private static final String PROJECT_FOLDER = "integrationtest";

    enum TwistScenarioResult{

        PASS(PASS_EXIT_CODE),
        FAILURE(FAILURE_EXIT_CODE),
        ERROR(ERROR_EXIT_CODE),
        FAILURE_AND_ERROR(FAILURE_AND_ERROR_EXIT_CODE),
        UNKNOWN(-1);

        private int exitCode;

        TwistScenarioResult(int exitCode){
            this.exitCode = exitCode;
        }

        public static TwistScenarioResult fromExitCode(int exitCode){
            for(TwistScenarioResult result : TwistScenarioResult.values()){
                if (result.exitCode == exitCode){
                    return result;
                }
            }
            return UNKNOWN;
        }
    }

    protected ScenarioExecutorAntMain initExecutor(String reportsDirectory, String configurationDirectory, String         scenariosDirectory){
        Map parameters = new HashMap(6);
        parameters.put("reportsDir", new File(PROJECT_FOLDER + reportsDirectory));
        parameters.put("confDir", new File(PROJECT_FOLDER + configurationDirectory));
        parameters.put("scenarioDir", new File(PROJECT_FOLDER + scenariosDirectory));
        parameters.put("tags", "ignored");
        parameters.put("numberOfThreads", 1);
        parameters.put("scenariosListFile", null);
        return new ScenarioExecutorAntMain(parameters);
    }

    protected TwistScenarioResult executeScenarios(ScenarioExecutorAntMain executor, String tags){
        ReflectionTestUtils.setField(executor, "tags", tags);
        ScenarioExecutorAntMain.ExitCodeListener exitCodeListener = executor.start();
        int twistExitCode = executor.exitCode(exitCodeListener);
            return TwistScenarioResult.fromExitCode(twistExitCode);
        }
}

And the test implementation:

package com.test.twist.runner;

import com.thoughtworks.twist.core.execution.ant.ScenarioExecutorAntMain;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;

import java.util.regex.Matcher;
import java.util.regex.Pattern;

import static org.fest.assertions.Assertions.assertThat;

/**
* Allows Twist Tests to be run inside Intellij and makes use of Intellij debugging!
*/
public class TwistScenarioRunner extends AbstractTwistScenarioRunner {

  private ScenarioExecutorAntMain scenarioExecutor;

  @BeforeMethod
  public void setUp(){
      String reportsDirectory = "/target/surefire-reports";
      String configurationDirectory = "/twist-conf";
      String scenariosDirectory = "/scenarios";
      scenarioExecutor = initExecutor(reportsDirectory, configurationDirectory, scenariosDirectory);
  }

  @Test
  public void shouldExecuteSpecificScenarios() throws Exception {
      assertThat(executeScenarios(scenarioExecutor, "my-scenario,!in-progress,!ignore")).as("My Scenarios").isEqualTo(TwistScenarioResult.PASS);
  }

}

Lessons Learned: Retrospectives: A union of team honesty and responsibilty

Having been in a number of retrospectives I love the dialogue where team members are not afraid to say what they really think.

• If something was a lemon, then lets chat about it! 
• If something was great, let's praise and continue it! 
• If something was only lukewarm let's at least acknowledge it and see how we can improve it!

And equally so where the team takes responsibility for the evolving action points to put things right!

• What is it that I can do that can make a difference?
• If the test team cannot easily write tests how can we adjust the process that it is so?
• A developer might say I don't think I can solve this, but let's time box it for 2 days to see what can be achieved.

Without the honesty a team does not have the true chance to improve the development cycle for all team members! And without individual responsibility on action points the same faults will keep arising and we loose faith in the retrospective process. So perhaps with no surprise, a combination of both lack of honesty and responsibility may mean cancelling the retrospective process altogether! Horror!!

So how exactly do we encourage the team to be honest?

• The faults of the team should not be attributed the individual. 
• Supporting the team to be honest about what they really think.

Then encouraging individual responsibility on action points? This is not as clear cut. But there at least needs to be:

• Room to manoeuvre in the backlog to implement the action point tasks. (That not on top of normal allocated tasks.)
• The support of the team where a team member could not resolve an action point in the time allocated.

There is of course much more to retrospectives than team honesty and responsibility of action tasks, but it's a great start!

Lessons Learned: BDD: More than just behaviour driven development

BDD: More than just behaviour driven development

Quite often when looking at BDD scenarios and objectives we can miss the key point about why we do it and the real benefit it can give to development teams. That is if the behaviour can be expressed to a wide enough audience then it becomes more valuable both in terms of long term robustness and the effective nature as Real Time Documentation.

Real Time Documentation has the advantage that it never goes out of date like that of Wiki type documentation, and typically cannot lie if written correctly! But not all BDD scenarios represent Real Time Documentation!

I always suggest creating scenarios which have the most amount of power in terms of audience, suitability for Real Time Documentation and least amount of immediate change needed.

An example of BDD scenarios and compatibility to Real Time Documentation

1) The Poor BDD Scenario

Load Page 'xyz.html'
Enter 'foo' in '\\div[3]\p[5]\table\tr[1]\td[1]'
Assert 'specific error message'

Please notice:

• It is readable only to a technical audience
• It is extremely brittle and should fail on the slightest number of changes
• It cannot be easily read grammatically and what it is meant to achieve in terms of behaviour is not clear
• It would be hard to a tester or developer to maintain this test
• This scenario is not suitable as Real Time Documentation.

2) The average BDD Scenario

Given we go to the login page
When we enter 'foo' into field 'j_password'
Then we should get 'x_errors' value 'com.custom.LoginException foo is not recognised as a known'

Please notice:

• The scenario is readable to a larger audience but the terms involved make it appear very techie.
• The test is much less brittle but it still relys on development criteria 
• It can be read grammatically but again is far too techie
• It can be maintained more easily than the first scenario. But the references to the fields and Exception messages make it far more brittle than it needs to be
• The scenario has average suitability for Real Time Documentation

3) The ideal BDD Scenario

Given we go to the login page
When an unknown username is entered
Then an error message should be displayed

Please note:

• The scenario is grammatically readable to technical and business audiences
• The scenario is not brittle (although care should be taken in the underlying code to prevent passing this further down the line)
• The scenario is unlikely to have to change that often
• The scenario is ideal for Real Time Documentation.

Looking to learn more about NoSQL?

A found a free online course/certification on MongoDB here

You can catchup on the requirements if you start now!

Lessons Learned: Avoiding Distributed Transaction Management

In one particular project we were using an Oracle 11g Database and XA Datasources to persist transactions to multiple database environments. Whilst in theory this is fine, it wasn't really needed and by reverting to a more modular separate transaction management (as illustrated bellow) huge gains in performance were obtained.

Therefore I would strongly recommend alternatives to global transactions on XA Datasources to multiple databases. Unless it is really needed

Example Seperate Datasource Configurations

Multiple Transaction Manager IOC Configurations

<beans xmlns:tx="http://www.springframework.org/schema/tx"
       xsi:schemaLocation="http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.0.xsd">

    <bean id="firstTransactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
        <property name="entityManagerFactory" ref="firstEntityManagerFactory"/>
        <qualifier value="first"/>
    </bean>

    <bean id="secondTransactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
        <property name="entityManagerFactory" ref="secondEntityManagerFactory"/>
        <qualifier value="second"/>
    </bean>

    <tx:annotation-driven />
</beans>

Custom transactional annotations for the transaction managers

@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
@Transactional("first")
public @interface FirstTransactional {
}

@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
@Transactional("second")
public @interface SecondTransactional {
}

Custom transactional annotations embedded within a service

@Service("exampleService")
public class ExampleServiceImpl implements ExampleService{

    @FirstTransactional
    public boolean saveToFirstDatabase(ReceivedEvent toSave){
         ....
    }

    @SecondTransactional
    public boolean saveToSecondDatabase(ReceivedEvent toSave){
         ....
    }
}

In the Beginning!

The Very First Entry!

The Silverback is excited to be here!

I'm looking forward to sharing both technical and agile lessons learned in development to help out individuals and groups. Hopefully this will encourage further discussion and thought in the technical/agile development.