Automation-friendly framework for Continuous Testing
Automation-friendly framework for Continuous Testing
How to Easily Run Taurus with BambooBy: Yuri Bushnev, May 2017 Why Use Taurus with CI toolsContinuous Integration (CI) is a way to automate our tests and make the test process itself easier and more productive. With the help of CI it takes much less time to release, build, deploy and run tests automatically on a daily basis. At the same time, most performance testing tools are not well-designed for a smooth CI pipeline integration. By using Taurus, an open source automation testing framework, combined with some continuous integration tools like Bamboo, we can achieve outstanding results in automation of our performance gates. An Introduction to TaurusTaurus is an open source automation framework that provides the ability to create test performance scripts from scratch or use existing scripts, and to configure run options. These options include many features that help execute and analyze script results. The main idea behind the Taurus framework is that the script creation/configuration language is more user-friendly than options provided by other testing tools. Taurus, with its YAML-based syntax or JSON languages, helps to generate scripts in Apache JMeter™, Gatling and additional tools. There are also many additional advantages:
The installation of Taurus is very straightforward and should take just a few minutes, by using this guide. An Introduction to BambooBamboo is a continuous integration server developed by Atlassian. It provides automated deployment, building and testing capabilities and built-in integration with other Atlassian products like Jira or Confluence. Because Jira is the most popular bug tracking tools, Bamboo is one of the most popular CI systems that is used in many companies over the world. Bamboo installation is very well covered in its official documentation. Running Taurus with BambooOur workspace:
First, let’s create a simple YAML script named “script1.yml” execution: - concurrency: 20 hold-for: 1m ramp-up: 10s scenario: Thread Group scenarios: Thread Group: requests: - label: HTTP Request BlazeDemo method: GET url: http://www.blazedemo.com/ As you can see, Taurus script files are human-readable. In the provided example we simulate simple get requests to blazedemo.com from 20 different users for 1 minute. Ramp-up means the onboarding time for all users, and the hold-for parameter means that the target load is held for 1 minute. Taurus uses JMeter as a default test executor, but you can choose different executors: Selenium, Gatling, Locust, etc. To run that script using Taurus execute the following command: bzt script1.yml During test execution, you can see console real-time dashboard with lots of useful metrics:
If there are any errors, we can see them in the right upper corner. In the middle of dashboard we can see such test parameters as latency, the requests’ response codes, errors, etc., which are necessary to know during and after the test run. Pretty simple, isn’t it? Now, what about Bamboo - Bamboo should be installed on your computer and available on a URL (http://localhost:8085/ in our case). If you open Bamboo for the first time, you will see the Start window.
If you’re running Bamboo for the first time, click “Create your first build plan” button. Otherwise, click on the Bamboo main menu panel “Create -> Create a new plan” to create a new test plan and follow steps below:
Here we should specify our project parameters.
After configuring these parameters, click the “Configure plan” button. Next you’ll see a “Create a new plan” window:
Here you should check the “Yes, please” CheckBox to make your test plan enabled, so you can configure and run it. The next step - press “Add task” button to add new task to your test plan. You’ll see a window like this:
Here the user is asked of what type of task should be used. In “Task types” window choose “Script” type.
Mention the Task description and choose a command interpreter depending on our OS architecture (“Shell” for Linux, “Windows PowerShell” for Windows, “bin/sh or cmd.exe” as universal option). In “Script body” field type: bzt /{path_to_yml_script}/script1.yml
After this click “Add task” button and click on “Create” button to finish your test plan creation.
In the newly opened window click “Run” -> “Run plan” to execute our test plan. That’s it! Our performance test is up and running.
ReportingAs soon as your test is finished you’ll see result performance metrics at the end of the log file in your finished build run.
If you take a look at the mentioned log’s lines, you can find there are parameters as test duration, latency, samples count and response time. But, of course, reading metrics from logs is not the best option. That’s why Taurus provides better test analysis. Through Taurus, you can easily view, analyze and collect all performance metrics, through the Blazemeter app. To do so and to see your tests results, change the command in the “Script body” field to: bzt /{path_to_yml_script}/script1.yml -report
The “-report” command runs a BlazeMeter reporter. It uploads your test to the BlazeMeter app, which contains everything you need to get as much as possible from your performance metrics. It’s a sufficient reporting functionality that feeds results to the server, so you don’t need to provide any additional configuration. The report link is received in the console text. This link is automatically opened in the browser, and you will see the web application with your test execution:
BlazeMeter reporting provides the ability to see various parameters of test running. You can find all necessary information about time (duration time, response time, latency, etc.), loading and request stats, information about errors, response codes and many other helpful parameters that allow us to make our test analyze deeper and fuller. That’s it! Now you are able to run Taurus YAML test scripts via the Bamboo CI tool. |
On this page:
|
