Mocking elasticsearch-py calls

Question

I'm writing a CLI to interact with elasticsearch using the elasticsearch-py library. I'm trying to mock elasticsearch-py functions in order to test my functions without calling my real cluster.

I read this question and this one but I still don't understand.

main.py

Escli inherits from cliff's App class

class Escli(App):
    _es = elasticsearch5.Elasticsearch()

settings.py

from escli.main import Escli

class Settings:
    def get(self, sections):
        raise NotImplementedError()

class ClusterSettings(Settings):

    def get(self, setting, persistency='transient'):
        settings = Escli._es.cluster\
                    .get_settings(include_defaults=True, flat_settings=True)\
                    .get(persistency)\
                    .get(setting)

        return settings

settings_test.py

import escli.settings


class TestClusterSettings(TestCase):
    def setUp(self):
        self.patcher = patch('elasticsearch5.Elasticsearch')
        self.MockClass = self.patcher.start()

    def test_get(self):
        # Note this is an empty dict to show my point
        # it will contain childs dict to allow my .get(persistency).get(setting)
        self.MockClass.return_value.cluster.get_settings.return_value = {}

        cluster_settings = escli.settings.ClusterSettings()
        ret = cluster_settings.get('cluster.routing.allocation.node_concurrent_recoveries', persistency='transient')
        # ret should contain a subset of my dict defined above

I want to have Escli._es.cluster.get_settings() to return what I want (a dict object) in order to not make the real HTTP call, but it keeps doing it.

What I know:

In order to mock an instance method I have to do something like MagicMockObject.return_value.InstanceMethodName.return_value = ...
I cannot patch Escli._es.cluster.get_settings because Python tries to import Escli as module, which cannot work. So I'm patching the whole lib.

I desperately tried to put some return_value everywhere but I cannot understand why I can't mock that thing properly.

You have to mock with respect to where you are testing. Since you are testing the settings.py module, then you need to reference your mocking accordingly. So, in your setUp, your patch path should be something like: patch('escli.settings.Escli'). So now when you call the settings code explicitly like you are in your tests, you are now properly mocking in the right place. — idjaw
– idjaw, Commented May 25, 2018 at 0:07
Furthermore, you might want spec your Mock with the Elasticsearch class in order to validate you are using the valid attributes available in the class. So, you might want to do the following: patch('escli.settings.Escli', Mock(Elasticsearch)) — idjaw
– idjaw, Commented May 25, 2018 at 0:20
Just to put it out there, pytest does provide a plugin you can use for elasticsearch if you are interested in other options as well: pypi.org/project/pytest-elasticsearch — idjaw
– idjaw, Commented May 25, 2018 at 0:21
You made my day ! I will dig to understand that Mock() thing. Would you consider making an answer as you solved the case ? — Jérôme Pin
– Jérôme Pin, Commented May 25, 2018 at 7:58

idjaw · Accepted Answer · 2018-05-25 13:25:26Z

You should be mocking with respect to where you are testing. Based on the example provided, this means that the Escli class you are using in the settings.py module needs to be mocked with respect to settings.py. So, more practically, your patch call would look like this inside setUp instead:

self.patcher = patch('escli.settings.Escli')

With this, you are now mocking what you want in the right place based on how your tests are running.

Furthermore, to add more robustness to your testing, you might want to consider speccing for the Elasticsearch instance you are creating in order to validate that you are in fact calling valid methods that correlate to Elasticsearch. With that in mind, you can do something like this, instead:

self.patcher = patch('escli.settings.Escli', Mock(Elasticsearch))

To read a bit more about what exactly is meant by spec, check the patch section in the documentation.

As a final note, if you are interested in exploring the great world of pytest, there is a pytest-elasticsearch plugin created to assist with this.

Collectives™ on Stack Overflow

Mocking elasticsearch-py calls

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related