0

Say, for example, you have a program that sends a message to a server and that server validates it.

Note: The server validation process can have additional test cases due to it being implemented in various countries (you would have to comply with their rules).

If you wanted to automate the testing on this, you could write various test cases in some programming language (e.g. Java). However, as more test cases arise, you would have to go back to the program and keep adding more code. You wouldn't be able to reuse it for different countries as some require certain tests to be ran and others don't. You would always be modifying the code.

What would a better strategy be to go about making the automation work in this scenario? (Sorry if this is too broad, not really sure where to ask this).

Scenario: (The server is doing all the testing)

A program sends data to a server and the server must validate if the data given is indeed correct or if the data follows some rule set on the server. There can be many rules from country to country. For example, lets say you send attachments, there must be a certain naming scheme or they must correspond with the data somehow. In another country, they might not care about the naming scheme.

Improve this question
5
  • 1
    What are you testing? Your code that sends messages, or the server that validates them? Commented May 20, 2016 at 19:26
  • The server that validates them. I'll add a scenario to the op. Commented May 20, 2016 at 19:26
  • 1
    Are all these rules known?. If you don't want to code every single test/validation probably a good approach would be to set tests in descriptors (lets say an XML). Then just implement a program that consumes these descriptors. Code becomes super abstract because it's only a interpreter. Basically you turned rules into metadata instead of code. Program should not have any reference to countries. Code can not be segmented by any reason Commented May 20, 2016 at 20:17
  • The rules from the different countries are known. But there are still generic test cases to handle. When you say descriptors (XML in this case), do you mean every descriptor file will be a separate template for a test case? EDIT: Just saw your edit. I guess I understood you correctly. Commented May 20, 2016 at 20:22
  • Well, you have countless way to do this with XML. Basically what I tried to say is to do it as functional programming lenguaje, where rules are typed with a readable syntaxis and interpreted by language. So your xml would contain tags for hosting data, but also for defining operations (e.g <when field="country" equals="en"> <then/> ... <else/> </when>. In this way you are doing a metalanguaje and interpreter no need to be mosified at every new possible rule. An example of this strategy would be Robot Framework. Commented May 20, 2016 at 21:34

3 Answers 3

Reset to default
1

One important thing is that we know these rules. Now we should know which ones applies to each country

Lets suppose we know all this info.

What I have been explaining in the comments is that, instead of to code test cases, would be more flexible to turn rules into metadata. Any metadata also needs a interpreter.

I have said XML as candidate to modelate these descriptors, but it could be any other format.

Instead of XML, It could be also a functional programming language. This is the case of Robotframework. Which it's customizable and allow to you to implement libs, keywords, syntaxis,...

Back to the approach with XMLs.

In your case, you need to validate the server responses, so the point is to modelate the inputs. Our interpreter perform 3 actions:

  1. Load/Parse XML
  2. Transform each case into a server request
  3. Compare response with the expected response

Example:

<SuiteCase country="en">
      <Cases>
          <Case id="contactDataValidation" result="OK">
            <Field name="email" value="[email protected]"/>
            <Field name="phone" value=""/>
            <Field name="movile" value="000000000"/>
         </Case>
          <Case id="contactDataValidation" result="KO">
            <Field name="email" value="xxx@mail."/>
            <Field name="phone" value=""/>
            <Field name="movile" value=""/>
         </Case>
     </Cases>
</SuiteCase>

In case of rules change we only need to change the XML (instead of code). And as new tests arais, we only have to add them into the XML.

All this is quite easy to do on Java with XMLBeans.

Well, it's a poor example, but the posibilities and the complexity have no limits. The more tags/details/levels has our descriptor the more possible scenarios can be meta-modeled and more acurated.

I have used this approach in few projects. Scenarios were designed as XML (data, rules, flows,...). In consecuence, to simulate scenarios was really simple. Our interpreter was built on Java + XmlBeans + SWT (gui).

We had also tags that allowed us to simulate random values, incrementals overtime, random nulls, etc.

Improve this answer
1

To give a different perspective on your system, you have a server that can perform a large number of validations, and next to those validations you have a (country-specific) configuration that specifies which subset of validations must be active and which subset must not be active (and that leaves a subset where you don't really care if it is active).

The testing of this system can also be split into two parts:

  1. Tests that verify that each of the individual validations works as expected. This can easily be automated if you have a few testing configurations that are independent of a country, but that are fully controlled by your team. I expect you will need multiple configurations for these test as I expect there will be validations that conflict with each other or that affect each other.
    You could even make a separate testing configuration for each validation rule, where only that validation rule is active.

  2. Tests that verify that the configuration for each country is correct. These tests would verify that all required validations for country A are enabled and that the validations that must not be enabled are really deactivated.

Both of these classes of tests can be automated independently of each other, where especially in the first group it is unlikely that you will ever need to change existing tests.
The tests in the second group will change if the rules for a particular country change. If a country changes its rules frequently, then it might be better to keep the tests for that country manual.

Improve this answer
0

Ultimately there is also a question of business value. The purpose is of the test is ultimately to be able to comply with the rules as code, software environment, staff and scale change. So you surely have tests up to date for major markets such as US, Germany, UK, China. Maybe also for Luxembourg, because they have I important decision makers. Probably not for Tunisia (unless of course that's your prime target market)

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.