Creating End-to-End Web Test Automation Project from Scratch — Part 4

In the previous blog post, you have configured and executed your tests in parallel with Selenium Grid. Now it is time to dockerize your web project! Let’s start with installing docker on your machine.

1. Let’s Create and Configure Our Web Test Automation Project!
2. Let’s Write Our Test Scenarios!
3. Bonus: Recording Failed Scenario Runs in Ruby
4. Let’s Configure Our Web Test Automation Project for Remote Browsers and Parallel Execution
5. Let’s Dockerize Our Web Test Automation Project
6. Bonus: Recording Scenario Runs on Docker with Selenium Video!
7. Let’s Integrate Our Dockerized Web Test Automation Project with CI/CD Pipeline!
8. Auto-Scaling and Kubernetes Integration with KEDA

Installing Docker

Docker’s official page gives elaborate instructions about installing Docker on various machine types, so I won’t get into details. You can download and install the corresponding Docker package to your device:

• Docker for MacOS
• Docker for Windows
• Docker for Linux

Let’s start Docker Desktop and open the terminal. If everything is installed successfully, you should see a similar response when you write `docker` in the terminal:

Then let’s create your project image!

Create an Image of Your Project

Docker images are configured and created via a `dockerfile`. This is the blueprint of your image.

Let’s create a file named `dockerfile` without a file extension in your project folder and start populating the file with your configurations:

Base Image: Base images are self-explanatory. They form the basis of your image. The base image comes with OS distribution, some programs, and dependencies. You can search and find images in the dockerhub.

 FROM ruby:3.0

Using the FROM keyword, you chose your ruby:3.0 as your base image with all the dependencies you need to execute your web project. So all the things you install and configure will be upon this base image. I decided on this ruby image since it has all you need (and probably more) to run your project. But if you want to create more lightweight images, you may want to use ruby-slim or alpine images as your base image. But beware that you need to install more packages manually.

RUN apt update && apt install git
#RUN apk update && apk add git

With RUN keyword, you can execute commands. With apt update && apt install git, you install git to your image. Since the ruby:3.0 is a debian distribution, you use apt command. If you use an alpine based distro, you will need apk command.

Now let’s specify your working directory.

WORKDIR /usr/src/app

The WORKDIR instruction sets the working directory for any RUN, CMD, ENTRYPOINT, COPY and ADD instructions that follow it in the Dockerfile.

COPY Gemfile /usr/src/app

Then let's copy your Gemfile to your working directory.

COPY Gemfile /usr/src/app

And then install the gems that your projects require.

RUN gem install bundler && bundle install --jobs=3 --retry=3

Now let’s copy the rest of the files of your project.

COPY . /usr/src/app

Note: You might ask why you separately added your gemfile to the directory, instead of copying all of your project files first and then installing gemfile contents. I will touch on this subject in the next section. Now you state with which command your image will be started:

CMD parallel_cucumber -n 2

And finally, you state which ports you will expose:

EXPOSE 5000:5000

Then putting everything together:

FROM ruby:3.0

RUN apt update && apt install git
#RUN apk update && apk add git

WORKDIR /usr/src/app

COPY Gemfile /usr/src/app
RUN gem install bundler && bundle install --jobs=3 --retry=3 

COPY . /usr/src/app

CMD parallel_cucumber -n 2 -o '-p parallel'

EXPOSE 5000:5000

Building Docker Image

Now you have your dockerfile ready, let’s build your image!

First, navigate to your project directory in the terminal.

docker build -t muhammettopcu/dockerize-ruby-web:1.0 .

Let’s look at the above code where:

• `docker build` is your main command
• `-t` is an option flag that enables you to tag your image
• `muhammettopcu` is my user name in dockerhub. You change it with yours.
• `dockerize-ruby-web` is your image’s name
• `1.0` is the version of your image
• `.` refers to the current directory, in which your dockerfile resides.

And your docker image is successfully created! Let’s check it with the following command:

docker images

 

Building Multi-Arch Images

If you want your image to work on host machines with different CPU architectures, you need to build images compatible with different architectures.

docker buildx build -t muhammettopcu/dockerize-ruby-web:1.0 --push --platform linux/amd64,linux/arm64 . 

As you can see, you added `buildx` command and your platform types with the  `--platform` option. `--push` lets you push your images to dockerhub. Now you can see your image has different arch types!

Note that you can use different processor types as well, such as linux/arm64/v8 and linux/arm/v7.

Now let’s talk about why the order of the command is important in a dockerfile and how you benefit from them.

Docker Layers 

Docker builds the images layers upon layers. Every layer has a unique hash ID. This layered structure makes it possible to re-build, download or upload images faster by only writing the changed layers and getting the other layers from the cache.

To make it more clear, let me write it in a list:

1. Docker uses cached files.
2. In a docker file, every line creates a layer.
3. When re-building, it uses cached files from top to bottom until it finds a changed layer.
4. Then docker builds the rest of the layers from scratch.

When creating the dockerfile, you should write the lines from the least likely to change to the most likely to change. Let’s demonstrate this to make it more clear:

First, I am going to make a small change to your project file and rebuild your image:

As you can see, I fully utilized my cache and didn’t need to download or install anything else other than updating your source code.

Now I am going to change the order of your dockerfile like below. Note that with this configuration, first, I copy all your project files to your image and then install the gems.

FROM ruby:3.0

RUN apt update && apt install git

WORKDIR /usr/src/app

COPY . /usr/src/app
RUN gem install bundler && bundle install

#RUN apk update && apk add --no-cache build-base && apk add git

CMD parallel_cucumber -n 2 -o '-p parallel'

EXPOSE 5000:5000

Now let’s say I made a change in your code and want to rebuild your image. (I added a space to one of your files for this purpose.)

As you can see, even though I only changed the source code, I needed to re-download and install the gem dependencies. That is because when building an image, docker cancels cache usage completely after the first changed layer.

So it is best to locate your code source near the end of the file since it is the most likely to change.

Now let’s run your project and see if your code is executed or not:

First, with the `docker images` command, list the images:

And copy the id of the latest version of your image from here, which is “25f5ff8c731a” in my case. Then type `docker run 25f5ff8c731a`.

You see that your scenarios do not run since it can not find drivers, but do not worry. You will Dockerize Selenium Grid as well! :)

Dockerize Selenium Grid

Let’s download Selenium Grid images to your machine.

If you use a Macbook with Apple Silicon (M1/M2) download the below images:

1. seleniarm/hub
2. seleniarm/node-chromium
3. seleniarm/node-firefox

Otherwise, download these:

1. selenium/hub
2. selenium/node-chrome
3. selenium/node-firefox

To download these images, you need to use `docker image pull <image_name>` command. So since I use MacBook with M1 chip, I will use: `docker image pull seleniarm/hub`

Now if you downloaded three of them, let’s configure them with Docker Compose.

Docker Compose Configuration

Docker Compose is a yml file to configure more than one image for them to be run in coordination. It allows you to write simple command lines without specifying everything with long strings. Let's create your file bit by bit.

version: "3"
services:
 selenium-hub:
     image: seleniarm/hub
     container_name: selenium-hub
     ports:
       - "4442:4442"
       - "4443:4443"
       - "4444:4444"
     networks:
       - dockerize-network

• `version` is the version of your docker compose
• `services` where you list your service name and the image it will use.
• `selenium-hub` is the service name
• `image` is your image
• `container_name` is optional. If you do not state, docker would generate randomly.
• `ports` is to map your host’s ports with the container’s. It is in the HOST:CONTAINER format. 

• `networks` defines the network this container will be connected to.

Now your nodes:

chrome:
    image: seleniarm/node-chromium
    container_name: selenium-chrome
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_NODE_MAX_INSTANCES=4
      - SE_NODE_MAX_SESSIONS=4
      - SE_NODE_SESSION_TIMEOUT=180
    networks:
      - dockerize-network

• `shm_size` is the shared memory size. You need a larger one since you run multiple browser instances on these nodes.
• `depends_on` allows you to prioritize the execution of services. If a service depends on another service, it will not start until the service it depends on has started.
• `environment` lets you define environment variables. Most of the variables are self-explanatory:
  
  • `SE_NODE_MAX_INSTANCES` defines how many instances of the same version of the browser can run.
  • `SE_NODE_MAX_SESSIONS` defines the maximum number of concurrent sessions that will be allowed.

The firefox node:

firefox:
   image: seleniarm/node-firefox
   container_name: selenium-firefox
   shm_size: 2gb
   depends_on:
     - selenium-hub
   environment:
     - SE_EVENT_BUS_HOST=selenium-hub
     - SE_EVENT_BUS_PUBLISH_PORT=4442
     - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
     - SE_NODE_MAX_INSTANCES=4
     - SE_NODE_MAX_SESSIONS=4
     - SE_NODE_SESSION_TIMEOUT=180
   networks:
     - dockerize-network

Now let’s save your file and spin up your grid with the command below:

And the network configuration:

networks:
   dockerize-network:
   name: dockerize-network
   driver: bridge

• `network` is for the network configuration.
• `dockerize-network` is the network declaration.
• `name` is the name of your network.
• `driver` is the type of your network.

# To execute this docker-compose yml file use `docker compose -f docker-compose-seleniarm.yml up`
# Add the `-d` flag at the end for detached execution
# To stop the execution, hit Ctrl+C, and then `docker compose -f docker-compose-seleniarm.yml down`
version: "3"
services:
  selenium-hub:
    image: seleniarm/hub
    container_name: selenium-hub
    ports:
      - "4442:4442"
      - "4443:4443"
      - "4444:4444"
    networks:
      - dockerize-network
  chrome:
    image: seleniarm/node-chromium
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_NODE_MAX_INSTANCES=4
      - SE_NODE_MAX_SESSIONS=4
      - SE_NODE_SESSION_TIMEOUT=180
    networks:
      - dockerize-network
  firefox:
    image: seleniarm/node-firefox
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_NODE_MAX_INSTANCES=4
      - SE_NODE_MAX_SESSIONS=4
      - SE_NODE_SESSION_TIMEOUT=180
    networks:
      - dockerize-network
networks:
  dockerize-network:
    name: dockerize-network
    driver: bridge

Now let’s save your file and spin up your grid with the command below:

docker compose -f docker-compose-seleniarm.yml up

Note that here -f option means file and docker-compose-seleniarm.yml is the name of your compose file.

Now your grid is up and running. You can check it with http://localhost:4444/.

Let’s see which networks you have with `docker network ls` command:

Your network is here. Let’s inspect it with docker network inspect <network-id> to see which containers are connected to it:

Okay, then let’s run your project image using this network!

docker run --network dockerize-network muhammettopcu/dockerize-ruby-web:1.0

It looks like 2 of 8 scenarios failed. Since these node images include debugging packages, you can watch the browsers in your container!

First, click on the Sessions Tab, then the camera icon beside the session you want to see.

The password for the VNC is “secret” by default.

Now you can see what is going on in your container!

As a final note, you can scale the node number of your browsers by using `--scale` option.

For example, let’s say that you want to spin up the grid with 4 Firefox nodes and 2 Chrome nodes. Then you can use:

docker compose -f docker-compose-seleniarm.yml up --scale chrome=2 --scale firefox=4

With this, we completed Dockerizing Selenium Grid and our project. In the next chapter, we will look at how to integrate a CI/CD pipeline into our project with Jenkins. But before that, we will have a bonus chapter as well! Stay tuned :)