Solution
Architecture
Initially, there were 4 test automation projects that were planned to be created and developed for İstegelsin. These projects were Web, Android, iOS, and API test projects. A roadmap covering a period of 6 months was planned to be executed on these projects.
A test automation project can be prepared with different programming languages and various frameworks. There were two main factors for us to decide which toolset we were going to use:
- Toolset was to cover everything İstegelsin needed and to offer a tailor-made infrastructure for them.
- The internal test team of Yıldız Tech, including people who had never written automation tests before, could adapt and contribute to the project swiftly.
Considering these main factors, we prepared POCs with different language and framework combinations. In the end, we decided on the following software language and framework combinations for each project:
Web: Ruby - CapybaraAndroid: Ruby - AppiumiOS: Ruby - AppiumAPI: Java - Karate
Methodology
It was a fresh start, and we needed a fresh approach to test automation project management. We decided to practice Scrum methodology with 2 weeks long sprints for the projects mentioned above. Created a Jira board to monitor and log sprint tasks. We set regular meetings including daily meetings, groomings, sprint reviews, and retrospectives.
We have created a list of smoke test cases to be automated for Web UI, Mobile, and API automation projects by closely working with the internal team of Yıldız Tech. There were two main reasons for starting with the smoke test cases:
- Smoke test cases allow us to cover core business functionality which is essential.
- Since the smoke tests are related to various points of the application, after the development of the smoke tests we get:
- A strengthened project structure
- A set of reusable functions which makes developing further test cases easier
- Integration points for data and reporting
- Coding conventions adopted by the team
We determined the reviewers of test automation projects, formed project development teams, and assigned projects to manual testing and test automation teams.
Development
We created test automation project architectures for Web, Android, iOS, and API projects. We defined project standards. These standards included coding conventions such as test case writing with Cucumber Framework, defining variables, and Cucumber tag naming.
We supported the team in setting up the project installations on different operating systems. To share our know-how, we scheduled onboarding sessions. These sessions included topics such as "Web element inspection", "Git usage", "Ruby language", "Capybara Framework", "Karate Framework", and "Appium". These sessions were especially important for our teammates with limited or no hands-on coding experience to adapt and contribute to the project swiftly.
All these conventions, installations, and usage information related to these projects were documented on Confluence.
We followed the Behaviour Driven Development (BDD) approach which enables us to reduce test steps to everyday English on these projects. Thus, making the test cases understandable even by the people outside of the developing circle.
We also built our test architectures for them to work as a distributed system, running tests in parallel. So we can run different tests in different instances of various browsers or devices.
With further analysis of the newly-developing projects, a new requirement emerged. To reduce repetitive code and to provide data for the client projects WEB, Android, and iOS, we created another project, which was a private Ruby gem (a library) covering more than 150 test data needs, that can be called from client projects. In the scope of this project, every data of the client projects required such as user info, customer cart, or payment info, were generated in the run time dynamically.
CI/CD Integration
When the test automation coverage exceeded the 30% threshold, we established Jenkins pipeline architecture for WEB and API projects. We designed the pipelines as able to run Smoke, Regression, Production test cases, or specific features.
Projects that are built in test environments are automatically triggered and test results are automatically transmitted to the Slack channel created for report tracking.
Smoke cases are run automatically every morning. After deployments to the production environment, the production pipeline is triggered and the cases in the production environment are run regularly on a daily basis, just like Smoke cases.
After the pipeline integration, we informed the whole organization about what we have achieved during that 3-month period. We added the entire project team to the test run groups, so we dynamically shared the test run reports with all stakeholders. Since every team in the organization was able to see these results, it became much easier to track down broken features and amend them.
Mobile Device Farm and Pipelines
We carried out two different operations for mobile projects’ pipeline integrations simultaneously.
- Managing Apps Dynamically: We integrated the application build tool Yıldız Tech used with Bitrise to manage apps dynamically. With Bitrise APIs, the app can be downloaded and installed automatically on a device with a specific app version or with the last successful build version.
- Purchasing a Device Farm Service: We prepared numerous POCs with different Device Farm services. Ultimately, our customer decided to use the digital.ai Continuous Testing platform in light of the POCs we presented. With the device farm, we integrated mobile test automation projects into the CI/CD pipeline.
Load Tests
In the beginning, there were only 5 automation projects. But in the meantime, we also included internal projects serving Cepte Şok product in the Yıldız Holding.
After making progress in UI and API projects, we developed load test scripts using Karate - Gatling and integrated these scripts into the pipeline. We dynamically generated the data we needed in load tests and built an architecture where the whole project team could run load tests.
