Capybara vs Karate UI Comparison

Introduction

Over a decade, the throne of UI Testing has been occupied by Selenium WebDriver. Many solutions strengthened Selenium’s hold on the throne by making it more accessible, and easier to use but some dared to challenge Selenium, to dethrone it and to take over its stead. In this blog post, we will tell the story of these courageous contenders.

The first challenger is Capybara. With its powerful DSL and useful utilities, usually used with Cucumber to strengthen readability and usability, Capybara is a solid candidate for the throne. 

The second challenger is Karate. It defies the Page Object Model and wants to rewrite the formulaic ways of Selenium and create a brand new framework containing everything required for a complete testing environment without depending on any other framework.

Now, before we get into the arena and see the clash of these two champions, let us take a quick glance at the fundamentals.

What is UI Testing?

To sum up in simple terms, UI Testing is about ensuring two things:

• Whether the user actions performed by mouse, keyboard, and other input devices are handled properly.
• Whether the visual elements (buttons, images, links, etc.) function and are displayed as intended.

And here, UI testing gets divided into two parts: Manual Testing and Automated Testing. Manual Testing is basically the foundation of Automation Testing, since the latter follows the scenarios created by the former. But not everything can be automated and not everything might be feasible to automate. Thus, Manual Testing is still kicking and it is an indispensable part of the testing process.

It is not possible to talk about UI testing without mentioning Selenium. Selenium is an open-source automated testing framework, which is used to validate web applications across different platforms and browsers. It is compatible with different programming languages such as JAVA, C#, PHP, Ruby, Perl, and Python. The framework has an undeniable authority in the UI testing - it is so fundamental that you get to know it even before you choose your starter pokémon. 

With the fact that it’s supported across many programming languages and different web browsers, it wouldn’t be wrong to say that Selenium is the most popular web automation tool at the moment. 

Sure, UI Testing is much more than these, and we could write hundreds of pages about it, but we don’t want to agitate the spectators more than we already did, do we?

What is Capybara?

Capybara is a Ruby framework that helps to create automation scenarios for user stories in behavior-driven development.

In the projects that follow Behavior Driven Development (BDD), we use Capybara and Cucumber. By combining Cucumber and Capybara, it’s possible to create infinite scenarios for your test suites with the help of predefined steps. Some highlights about Capybara:

• Tons of cool helpers such as click_button, click_link, fill_in, enabling you to locate elements and make actions without writing css or xpath.
• It has been around for a fair amount of time and if you use it with Selenium Webdriver, then pretty much every question has been asked already.
• It supports different kinds of web drivers. 
• Capybara automatically waits for the content to appear on the page, eliminating the need for manual waits.
• Follows page object model.

Speaking of which… Have you seen our BDD Helper gem which enables you to develop test suits swiftly yet?

What is Karate UI?

Karate is an open-source test automation framework that provides both API and UI testing tools. In kloia, we use Karate for mainly API testing purposes, since it covers pretty much everything we need for API testing. It uses the Gherkin syntax, making writing code as easy as writing plain text.

However, since its UI side takes slow but firm steps forward, it provides a solid alternative for other frameworks such as Selenium, and Capybara.

This blogpost written by Peter Thomas, the creator of the Karate framework may give you better insight about what it actually tries to accomplish. Some highlights about Karate:

• It’s a pretty new framework, thus there are not enough resources to look upon.
• It uses the Gherkin syntax, making it easy to write and understand the code.
• Provides different kinds of waiting options to suit your needs.
• It uses Chrome DevTools Protocol for automation.
• It makes it possible to mix API and UI tests within the same script.
• Debug option is available only for Visual Studio Code users at the moment.
• No need to create step definitions.
• It does not follow the page object model.

Direct Comparison

 It might give a better insight to see how the implementation of a same scenario is handled by different frameworks.

Example scenario: Clicking a random product on a website and verifying its category name.

​​Karate (.feature File):

@kloiaCase
  
Feature: Browser automation

 Background:
   * configure driver = { type: 'chrome', showBrowserLog: false, showProcessLog: false, showDriverLog: false}
   * def mainPage = read('classpath:tests/data/mainPage.json')
   * def searchPage = read('classpath:tests/data/searchPage.json')
   * def productPage = read('classpath:tests/data/productPage.json')
   Given driver 'https://n11.com'

And optional(mainPage.surveyDeny).click()
And input('#searchData', 'deodorant')
When submit().click(".searchBtn")
* def first = searchPage.searched_items_by_position_css
* replace first.number = "1"
Then click(first)
#waitFor is enough normally. it may be used as assertion since it fails when can not find any element
* waitFor(productPage.add_to_cart_button)
* if (!exists(productPage.add_to_cart_button)) karate.fail("Add to cart button does not exist.")
  

Capybara (.feature file):

Feature: Browser automation

 Background:
   Given visit homepage
   And decline survey

 Scenario: Click random product
   And hover random category on categories menu
   When select random sub category on sub categories
   And click random product on category detail page
   Then verify category name 
 

Note that it takes less time to create scenarios on Karate at first, since implementing step definitions is not needed. However, after initial struggle and reaching certain milestones of step definitions, Capybara might seem more swift and easy to read. But hey, you know what they say, it’s different strokes for different folks.

Capybara (WebElements)

Since we follow the Page Object Model, the web elements and function definitions for each page are stored in separate files.

Here is an example of how it looks:

class MainPage
 def initialize
   @survey_decline_css=".dn-slide-buttons.horizontal > .dn-slide-deny-btn"
   @search_box_id="searchData"
   @search_button_css=".searchBtn"
   @cat_menu_items_css=".catMenuItem>[title='%s']"
   @cat_sub_menu_items_css=".subCatMenuItem>[title='%s']"
   @cat_menu_items_list_css="li.catMenuItem"
   @cat_sub_menu_items_list_css="li.catMenuItem.active li.subCatMenuItem a[title]"
 end

 def decline_survey
   find(@survey_decline_css).click if page.has_css?(@survey_decline_css)
   click_link
 end

 def search_item(arg)
   fill_in(@search_box_id, with: arg)
 end

 def click_search_button
   find(@search_button_class).click
 end

 def hover_on_category_by_name arg
   find(@cat_menu_items_css % arg).hover
 end
end
 

Karate (WebElements)

In Karate, it is possible to define WebElements as key-value pairs in a Json format and declare them as variables by calling them in the feature file with read function. See the feature file example above.

{ "surveyDeny_css": ".dn-slide-buttons.horizontal > .dn-slide-deny-btn",
 "submit_id_css": "#submitButton",
 "search_box_id_css": "#searchData",
 "search_button_css": ".searchBtn",
 "cat_menu_items_css": ".catMenuItem>[title='']",
 "cat_sub_menu_items_css": ".subCatMenuItem>[title='']",
 "cat_menu_items_list_css": "li.catMenuItem",
 "cat_sub_menu_items_list_css": "li.catMenuItem.active li.subCatMenuItem a[title]"
 }
 

Changing generic WebElement locators to use specific names or the item order seems simpler in Capybara, only using % operator to send the text to be replaced. Since using new lines for different actions is encouraged in Karate, using the replace() method makes the scenario body longer. 

When it comes to other features such as wait functions, Karate introduces a variety of options. Although most of them are useful, it’s worth mentioning that some of them do not work as intended. For example submit() function might let you down at an unexpected time since it is unreliable most of the time. However with the expanding community and increasing usage of the Framework, these kind of functions can be corrected or removed entirely to ensure more consistent function in the future updates.

The main disadvantage of Karate against Capybara is the limited amount of helpers which allows you to find elements by using identifiers other than CSS and XPath such as name, id, value, title, alt text of image, etc. 

Summary

Here is our quick comparison table:

In the end, these two champions have their own strong suits and some of these strengths may not apply to your case. Ultimately it falls to your preferences and specific needs. To ensure efficiency, specific tools must be chosen for every task. A baseball bat can drive a nail as well as a hammer but you may hurt the hammer’s feelings along the way. So pick wisely, hammer watches...