API Mocking: Necessary and Unnecessary

I like contradictions where both states of truth can operate at the same time. The case for this tweet about a joke from the other week:


If you use mocks, you are not testing.

My answer was:

Mocks help to emphasize limited fixed functionality and they do it faster.

What are unit tests about? Mocks are an essential part of a comprehensive quality strategy. Otherwise you will spend your time brainstorming integration tests.

No, they are not enough. But they test.

It sounds like we’re both saying the exact opposite, but it’s not really necessary. I understand what Maxi is saying here and he is very true. Mocks are problematic.

problems with mocks

Mocks hide external dependencies on the actual API and as a result they do not test them. This limits the scope of testing to a very narrow, hard-coded set. It depends on internal implementation details, such as dependencies to implement the test. This means that the test will probably fail if the contract does not change, even if the implementation changes, for example, let’s look at an example:

public int countUserCities() {
    return db.executeInt(“select count(“city”) from users”);
}

We can mock the db function, because the result returned will be bad. But it will break if we replace the original API call with something like:

public int countUserCities() {
    return db.count(“city”,”users”);
}

Nothing is included in this. There’s a better way to add mock data to a temporary database, which is much easier to do thanks to projects. Test Container. We can spin up the containers dynamically and “properly” check the method with a database that we have in production. It does proper checks, it will fail for bugs like typos in the query and doesn’t rely on the internal implementation.

Unfortunately, this approach is problematic. It takes time to load a new database. Even with containers. As the scope of testing increases, doing it for each suite can become a problem. That’s why we separate unit and integration tests. Performance matters.

performance problem

You know what the worst kind of test is?

The ones you don’t run and delete.

It is important to test frequently, continuous testing allows us to fail early during development. A quick failover means that our developer mode is still fresh on the change that triggered the failure. You don’t need git bisect , you can fix things almost instantly. For this to work properly we need to run test cycles all the time. If it takes a while to go through the test workflow and requires some configuration (eg, Docker, etc.)

We mock external dependencies for unit testing so that performance is improved. But we can’t skip the full call to the actual API because mocking has problems. Although these can run in CI process, we don’t need to run them all the time. Does it cause duplication? Yes. It sucks and I don’t like it.

Unfortunately, I don’t have any other way that I’m aware of at this time. I tried to think of a way around this with code that would act as a unit test in one execution and as an integration test invoking the actual equivalent when running in CI. But I couldn’t do anything that wouldn’t make things worse.

Because of this I think it’s important to only check the coverage of integration tests. Unit test coverage is interesting but not as important as integration coverage. I am not in favor of 100% coverage. But it is an important statistic to monitor.

What should we ridicule?

I gave database example but I am not in favor of mocking database. For Java, I usually use a lite in-memory database which works well in most cases. The databases accept spurious csv formats to fill and can even come up with their own spurious data. It’s better than mocking and lets us get closer to integration test quality with unit test performance.

However, we cannot consistently connect to web service dependencies without mocking them. In that sense, I think it’s a good idea to make fun of everything that’s out of our control. At that point, we are faced with the choice of where to ridicule. We can use fake servers which contain coded requests and responses. This makes sense when working with integration tests. Not that much for unit test, but if the server is stateless and local then we can do that. Although I would prefer to mock the calls to the API endpoints in this case. It will be faster than the actual API but it can still cause problems.

Over-mocking is the process of applying mocks too liberally to too many API calls. A developer can engage in that to increase the reputed coverage metric. This further strengthens my claim that coverage should not apply to unit tests as it can lead to such a situation. The behavior should not be mocked for most local resources used by the framework.

eventually

I like to have fun. This made possible the development of certain features. Without it I could not test plugins, APIs, servers etc properly. However, like all good sweets, too much “good stuff” can corrupt our code. It’s also a small part of a balanced diet (spreading metaphors, but it works). We can just create functional test and call it day, we can’t rely only on mocking. Conversely, they are not the “real” badges of quality we seek.

Integration testing is in place. When we have coverage, we have important, valuable coverage. Mocking is wonderful for minimizing problems and avoiding regression. Mock testing is the perfect tool when I need to check that an improvement is still in progress. The development of such components is problematic and fragile. But that’s a good thing, we want that code to be a bit linked to the implementation.

Some operations would be difficult to cover without proper mocking. When testing the whole system that might be reasonable to expect but not for functional testing. In these cases, we need fast response and the actual API may not be enough.

Leave a Comment