Runners
Until now, I have shown you run your test cases directly on feature files. But, you will need runners to run your test cases on the CI/CD pipelines. Here, you can implement the JUnit runner classes and use them to execute your test cases.
Karate will execute all the feature files with the same level and the levels below within the runner class. For example, if you have a runner under com.mycompany, it will also run com.mycompany.demo1 and com.mycompany.demo2 and so on. So it is beneficial to use a flat directory folder structure to have a finer control over test execution.
JUnit 4
In JUnit 4, you should create one empty runner class and add the related annotation to run your test scripts. RunWith annotation is required to run your test scripts, and you can use @KarateOptions to specify some options like tag name or feature path, etc. Here is one example of the JUnit 4 runner class:
import com.intuit.karate.KarateOptions;
import com.intuit.karate.junit4.Karate;
import org.junit.runner.RunWith;
@RunWith(Karate.class)
@KarateOptions(
tags = "@wip"
)
public class UsersRunnerTest {
}
To run your tests on IDE, you should click the run button near the class name.
JUnit 5
If you want to work with JUnit 5, you should create one runner class with the related test methods. Besides JUnit 4, you should not use class-level annotations here. Instead, you should use @Karate.Test annotation in the class and define your test there. Here is one example of usage JUnit 5 runner class:
import com.intuit.karate.junit5.Karate;
class SampleTest {
@Karate.Test
Karate testSample() {
return Karate.run("sample").relativeTo(getClass());
}
@Karate.Test
Karate testTags() {
return Karate.run("tags").tags("@second").relativeTo(getClass());
}
}
As you see in the example above, you can define more than one method in your test class. You can run each method individually while developing your scripts in the IDE by clicking the run button near the method name.
Parallel runner Cucumber Report
Karate provides the parallel execution, and you don’t need to do any other setup other than creating one parallel runner. Usually, it is not easy to run your scripts parallel in different languages or frameworks such as Java, REST-Assured. But, parallel execution is built-in for Karate. The only thing to do is create a test method inside the JUnit runner and call the parallel execution function.
import com.intuit.karate.Results;
import com.intuit.karate.Runner;
import static org.junit.Assert.*;
import org.junit.Test;
public class TestParallel {
@Test
public void testParallel() {
Results results = Runner.path("classpath:some/package").tags("~@ignore").parallel(5);
assertTrue(results.getErrorMessages(), results.getFailCount() == 0);
}
}
Parallel runner class
======================================================
elapsed: 2.35 | threads: 5 | thread time: 4.98
features: 54 | ignored: 25 | efficiency: 0.42
scenarios: 145 | passed: 145 | failed: 0
======================================================
Parallel stats on the console output
Above code will execute all of the features under some/package without @ignore tag.
Karate will run all the features in parallel default. Also, every scenario will run as parallel, including any row of the outline table.
Karate will produce HTML reports by default but you can implement Cucumber Reports as well. Cucumber HTML report is a fancy report and it provides a ton of detail.
Cucumber Report Example
If you have a parallel runner class, you can implement a Cucumber Report in your framework and you can get an HTML report as a cucumber report. First, you need to add Cucumber Report dependency in your pom.xml and sync it:
<dependency>
<groupId>net.masterthought</groupId>
<artifactId>cucumber-reporting</artifactId>
<version>5.3.1</version>
<scope>test</scope>
</dependency>
Cucumber Report dependency
Then you need to adjust the parallel runner class to produce a cucumber report.
public class TestParallel {
@Test
public void testParallel() {
Results results = Runner.path("classpath:pets/features").tags("@done").parallel(10);
generateReport(results.getReportDir());
assertTrue(results.getErrorMessages(), results.getFailCount() == 0);
}
public static void generateReport(String karateOutputPath) {
Collection jsonFiles = FileUtils.listFiles(new File(karateOutputPath), new String[] {"json"}, true);
List jsonPaths = new ArrayList(jsonFiles.size());
jsonFiles.forEach(file -> jsonPaths.add(file.getAbsolutePath()));
Configuration config = new Configuration(new File("target"), "KarateBaseFramework");
ReportBuilder reportBuilder = new ReportBuilder(jsonPaths, config);
reportBuilder.generateReports();
}
}
Parallel Runner-Cucumber Report implementation
Cucumber HTML report
Most of the CI tools, such as Jenkins, are suitable with the cucumber HTML report, so you can quickly implement a parallel runner and get a Cucumber HTML report on your pipeline.
Command Line Parameters
Until now, I have shown you running test scripts on the IDE. But, in your pipeline environment, you need to run your test cases via the command line. Now let’s take a look at the command line arguments for Karate.
mvn test
Since you are using the maven project structure, you can call your test with the mvn command. Your test scope is a test, and if you run the mvn test command in your project path, it will run all test classes that include the test keyword inside the class name.
mvn clean test
Note: clean command will clear all of the previously created test outputs. So we will only have the current output.
Tags
Sometimes, you may want to call specific features or scenarios in your framework instead of running the entire suite. To do this, you can assign tags to your scenarios or features and call them at running time to execute tagged scripts.
For example; if you assigned @regression tags for regression’s test cases, you should call @regression tag to run your @regression suite.
Calling tags
If you want to run specific tags, you should call karate.options after the mvn command. Let’s say you want to run regressions tags, so you should call the command below;
mvn clean test "karate.options=--tags @regression"
Besides tags, you can specify the feature folder through the command line:
mvn clean test "karate.options=--tags @regression classpath:some/package/someFeature.feature"
Calling specific runner
Sometimes, you can have more than one runner class and may want to call different callers each time. So you need to call specific callers via command line just like that:
mvn clean test -Dtest=SomeRunner
In addition to the runner class, you can call tags in the same line of command.
mvn clean test "-Dkarate.options=--tags @wip" -Dtest=SomeRunner
Conclusion
I have explained the runners and command line parameters in this article. Until now, I have covered most of the topics that you need to have to run automated API test cases. The following article will be about Karate-Gatling integration and how to use API test cases as performance testing.
Stay on the line and keep learning...
Resources: https://github.com/intuit/karate