Skip to content

Software Development News: .NET, Java, PHP, Ruby, Agile, Databases, SOA, JavaScript, Open Source

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Vendor

Fully integrate your Ranorex Report with CI tools like Jenkins using xUnit

Ranorex.com: Latest News - Tue, 04/15/2014 - 15:53

A Ranorex test automation project creates an executable file which returns a two-state test result indicating a success or a failure. Continuous Integration (CI) tools like Jenkins, Hudson, TeamCity, CruiseControl, Bamboo etc. are able to call automated tests and do understand these two-state test results. This is similar with ALM tools like Polarion or Jira. In short, in the world of CI and ALM, test automation results are reflected by a single flag, success or failure.

This blog describes how a Ranorex Report can be automatically transformed into xUnit format by using an XSL Transformation (XSLT). This transformation allows the CI tool to fully understand the result of each test case, not only whether the entire test run passed or failed.



The following example shows how UI test results can be interpreted following the same standards as xUnit tests do. Furthermore an example implementation of an XSL transformation file can be downloaded at the end of this blog. In case there is not CI-Plugin available assisting you with the transformation you can simply implement a standalone transformation with Ranorex.

Background

The blog post “Integrating Ranorex Automation in Jenkins CI Process” is introducing the entire CI topic and gives a step by step introduction on how to integrate Ranorex into Jenkins CI. On the top of that, optimizations can be done supporting the machined understanding of the Ranorex Report.

Nowadays most of the modern unit testing frameworks such as JUnit, NUnit etc. do implement the xUnit notation (list of implementations). This comes with the advantage that tests and their results are defined once following the xUnit standard, and can then be interpreted and understood independent from the respective target language (such as Java, C#, etc.). In addition, continuous integration servers like Jenkins do use xUnit to interpret the results of a test in a unified way. Although Ranorex is a UI testing and not a unit testing framework, there still is a way to also interpret the Ranorex test results (Ranorex Reports) in the same way as you might be used to from your (automated) unit tests.

Install xUnit Plugin for Jenkins

First, the xUnit-plugin needs to be installed on Jenkins. Therefore please navigate to “Manage Jenkins” in the main menu and click the “Manage Plugins” menu item.

 
The tab sheet “Available” now provides the possibility to install the “xUnit plugin”.

Please proceed by installing the plugin.

Activate Custom Transformation (XSLT)

Next, the transformation of the Ranorex Report to a xUnit-based report format has to be activated. Therefore navigate to your Ranorex project and bring up the project configuration.

Start by adding a “post build action” called “Publish xUnit test result report”.

Two new panels appear allowing to define the thresholds for failed and skipped tests (both, total and new). More importantly click the “Custom Tool” menu item in the drop down:

The file name of the Ranorex Report needs to be defined in the setting “Custom Tool Pattern”:

Note: The xUnit plugin awaits a relative path expression in the “Custom Tool Pattern”. The starting path is the Workspace.

Please make sure that path and file name of your report file pattern match the definition of your build step that calls the Ranorex executable. Here is the one used by this example:

TestCIProject.exe /rf:Report-Build-%BUILD_NUMBER%

Note: The Ranorex command line argument “rf” (reportfile) allows defining the report file name including the extension. If no file-extension is provided, the default extension “rxlog“ will be used for the report frame file. Additionally, the XML file containing the relevant data will be created with a file-extension “rxlog.data” then. For details on all command line arguments please refer to the user guide, chapter Running Tests via Command Line.

Note also: The above used placeholders %BUILD_NUMBER% and ${BUILD_NUMBER} are provided by Jenkins. At the beginning of a test run they will be replaced automatically by the current build number. Please take care to use these notations correctly.

Another relevant setting is the path to the transformation file (XSLT). A sample file is shared on GitHub and can be downloaded here:

The entire github-repository can be found here: https://github.com/ranorex/Ranorex-To-XUnit

Given these settings, right after every Ranorex test execution triggered by Jenkins the Ranorex Report is automatically transformed into a xUnit-based XML file. This allows a deeper integration of test results into CI which will be described in the next section.

Exploring Test Results

Having applied the mentioned settings to your Ranorex project on Jenkins, a section “Test Result” will appear for every new test run.

Following the “Test Result” link, an overview site is presented providing useful information like “All Failed Tests” including their duration and age (since how many builds does test fail). Additionally, a short summary of all passed, skipped and failed tests is presented.

By clicking the plus icon of a failed test Jenkins presents error details and the stack trace additionally:

Also, Jenkins can now distinguish on the test case level whether a test case passed or failed. Before the conversion to xUnit format, Jenkins was only able to interpret the entire test result.

Standalone Transformation (if your CI Tool does not support xUnit Transformations)

In case your CI tool does not support XSL transformations and there are no plugins around assisting you, you can simply call the transformation at the end of your test run automatically.

Please follow the step by step instructions below to transform your Ranorex Report into xUnit format directly out of your Ranorex Test Suite. The resulting file can be used by any (CI) tool understanding this widely spread format.

  1. Download the XSL-file (Right-Click: Save as) and save it to your Ranorex project folder. After doing so, add the file to your project (see screenshot below). 
  2. Select the file (Ranorex-to-xunit.xsl) in the Projects View and hit the F4 key to open the Properties Pad. For the property “Copy to output directory”, choose “PreserveNewest” in the dropdown.
  3. Download the Code Module “TransformReport” (Right-Click: Save as) and add it to your project (“Add existing item”, see step 1 above). This module is open source and is hosted on GitHub.
  4. Assign the Module to your Test Suite and place it at the very end. Ideally you embed it in the global teardown container (additional information on teardown section).

The xUnit report file will have the same name as your report file, but “_xunit.xml” will be added at the end of the file name. After the test run you can find the file in the same directory where your report file is located.

Conclusion

Calling your Ranorex test automation project from a Continuous Integration (CI) environment provides a lot of benefits and allows you to represent the results of the entire build process over time. However, even more information of every single test run can be gathered out when the Ranorex Report is fully understood by the CI tool. This understanding can be achieved by transforming the test report into xUnit format. As visible above, there are only a few steps needed to see the results of every single test case instead of only one result of the entire test run. In addition these test case detail results can be used easier for further processing.

Although the sample above is using Jenkins as a CI tool this approach is the same for all CI tools understanding xUnit. Even if your CI tool does not support the transformation, the Standalone Transformation described above enables you to still make use of this powerful feature.

Further Reading

Share

Categories: Vendor

Embarcadero Launches RAD Studio XE6

Software Development Tools Directory - Tue, 04/15/2014 - 15:14
Embarcadero Technologies, a provider of software solutions for application and database development, today launched Embarcadero® RAD Studio XE6, the complete multi-device, natively compiled app development platform for Windows, Android, iOS a ...
Categories: Vendor

StrongLoop Releases StrongOps 2.0 DevOps Service for Node.js

Software Development Tools Directory - Tue, 04/15/2014 - 15:10
StrongLoop has announced the general availability of StrongOps 2.0, the first Node.js specific application lifecycle management (ALM) solution to include both monitoring and DevOps capabilities for Linux, Windows and Mac. Node.js continues to gai ...
Categories: Vendor

Chef and Rackspace Collaborate to Expand Reach of Enterprise Chef

Software Development Tools Directory - Tue, 04/15/2014 - 14:59
Chef, the leader in IT automation, today announced that Rackspace, the open cloud company, is using Enterprise Chef for its DevOps Service. Rackspace DevOps Automation Service helps developers and IT operations teams automate the scaling of hybri ...
Categories: Vendor

WANdisco Releases SmartSVN 8.5

Software Development Tools Directory - Tue, 04/15/2014 - 14:51
WANdisco, a provider of high-availability software for global enterprises to meet the challenges of Big Data and distributed software development, today announced SmartSVN 8.5, the latest version of its platform independent graphical client for A ...
Categories: Vendor

Neotys Announces Neotys Team Server 2.0 for Collaboration and Concurrent Testing

Software Development Tools Directory - Tue, 04/15/2014 - 14:37
Neotys has announced Neotys Team Server 2.0, a central web-based server used to collaborate on test projects and test multiple applications concurrently.
Categories: Vendor

Ranorex 5.0.2 Released

Ranorex.com: Latest News - Tue, 04/15/2014 - 09:00
We are proud to announce that Ranorex 5.0.2 is now available for download. General changes/Features
  • Added support for Firefox 29
  • Added "durationms" field to report activity XML nodes and a "timestampiso" (ISO8601) timestamp field to the report root activity XML node
Please check out the release notes for more details about the changes in this release.

Download Ranorex version here.
(You can find a direct download link for the latest Ranorex version on the Ranorex Studio start page.)

Categories: Vendor

Resource Article on Covering IIS and Web Apps

Covering IIS and Web AppsCollecting code coverage on your web apps is crucial as part of both the development and QA process for any .NET team.  NCover’s approach for collecting code coverage allows you to perform both automated and manual tests and aggregate that collection regardless of how you like to test your web apps.  The key to this approach is setting up a project and using pre-coverage filters.

Specifically, NCover watches the w3wp.exe, the IIS worker process, in order to collect coverage. This approach allows you to automatically capture coverage once you have properly configured your project.

If this is your first time covering IIS with NCover, you may find the “auto-configure” method the easiest.  If you are experienced, you may prefer the manual approach.

The three-step process for covering IIS with NCover can be found at http://www.ncover.com/resources/articles/covering-iis-and-your-net-web-apps.

If you have any questions with getting setup, please feel free to contact technical support.

The post Resource Article on Covering IIS and Web Apps appeared first on NCover.

Categories: Vendor

Welcome to Morris Medical Monday

Polarion Software - Mon, 04/14/2014 - 11:00
First of a new series "Morris Medical Monday" by Polarion consultant Daniel Morris. Each Monday medical device other medical related companies will get useful information about using Polarion and Polarion extensions to streamline development, make compliance easier, and accelerate time to market.
Categories: Vendor

Featured Extension: Polarion Work Item Integration with Tortoise SVN

Polarion Software - Fri, 04/11/2014 - 17:41
Featured Extension Polarion4Tortoise boosts developer productivity by integration Polarion with the Tortoise SVN client.
Categories: Vendor

Increase Test Efficiency with Parametrization

Intland Blog - Fri, 04/11/2014 - 10:50
In advanced Test Management solutions, parametrized testing supports the re-use of test cases and the importing or defining of test parameters for test runs with many different parameters. Parameters are attributes with different values that describe the live environment. Without
Categories: Vendor

QSM Updates SLIM-WebServices Software Project Estimation Tool

Software Development Tools Directory - Thu, 04/10/2014 - 14:43
Quantitative Software Management, Inc. (QSM) has launched SLIM-WebServices 1.2, the newest version of its software project estimation tool. Part of QSM's Software Lifecycle Management (SLIM) Suite, SLIM-WebServices 1.2 allows all project team ...
Categories: Vendor

Azul Systems Extends Zulu Runtime for Java to Support Java 8

Software Development Tools Directory - Thu, 04/10/2014 - 14:39
Azul Systems has announced the general availability of Zulu 8.0, a commercialized build of OpenJDK that is compatible and compliant with the Java SE 8 standard.
Categories: Vendor

MongoDB 2.6 NosQL Database Released

Software Development Tools Directory - Thu, 04/10/2014 - 14:37
MongoDB has announced the general availability of MongoDB 2.6, the newest release of the database that defines the standard in modern application development. The release builds on five years of innovation and hundreds of thousands of deployment ...
Categories: Vendor

New Ranorex Training Partner in Switzerland

Ranorex.com: Latest News - Thu, 04/10/2014 - 09:00
We are proud to announce that we have partnered with the Swiss training provider "Zaar Teach-IT" to expand our German-speaking training and to meet the demand in this area.

Get firsthand training with Ranorex professionals and learn how to get the most out of Ranorex Studio and the Ranorex Test Automation Tools at this two day workshop.

This workshop covers introductory issues such as the benefits of automated testing and the related return on investment. The main focus will be the practical utilization of Ranorex.

For further details, please have a look at our workshop info sheet (PDF).

May 19-20

08:30AM - 04:30PM CET

Basisschulung – Test-Automatisierung mit Ranorex

Zurich, Switzerland


Check the schedules for additional workshops in the next few months.


We look forward to seeing you there!
Categories: Vendor

NCover Update April 2014

april-2014-releaseWe have a minor NCover update available which is primarily focused on Bolt and Desktop users.  The current version is 4.5.2595.1126.  A full list of what is included can be found on our Release Notes page.

Expanded Bolt Capabilities

In addition to several performance and reliability fixes, NCover Bolt now supports the ExpectedException properties on the TestCase attribute, including ExpectedException, ExpectedExceptionName, ExpectedMessage, and MatchType.

  • ExpectedException specifies a the Type of an exception that should be thrown by this invocation.
  • ExpectedExceptionName specifies a the FullName of an exception that should be thrown by this invocation.
  • ExpectedMessage specifies the message text of the expected exception.
  • MatchType is a MessageMatch enum value indicating how to test the expected message.

If you have any questions about any of these updates, please let us know and we will be happy to help out.

The post NCover Update April 2014 appeared first on NCover.

Categories: Vendor

Agile-Waterfall Hybrid Requirements

Intland Blog - Wed, 04/09/2014 - 14:37
Requirements Management solutions enable companies to define and elaborate requirements and manage them throughout the development lifecycle and testing phases. Requirements are usually „owned” by clients and are typically determined by the marketplace, an external force constantly changing along with
Categories: Vendor

TestCast 6.8.0 supports TTCN-3:2014

Elvior - Wed, 04/09/2014 - 09:14


Elvior has released TestCast T3 version 6.8.0.
  
Version 6.8.0 includes full support for TTCN-3:2014. This includes support for lazy and fuzzy variables and templates as well as all the other useful and convenient features of this new TTCN-3 standard.
This release also includes bug fixes and improvements for XSD encoding and decoding; for details see release notes.
Customers can download this new version of TestCast T3 from Elvior online support. Versions for Windows and Linux are available.
Categories: Vendor

Web App Template

Software Development Tools Directory - Tue, 04/08/2014 - 17:01
Web App Template is a tool that uses your Web App as the core for a full featured Store App for Windows 8.1 and Windows Phone 8. The Web App Template (WAT) is implemented as a Visual Studio template that you install through our one click installe ...
Categories: Vendor

Intern

Software Development Tools Directory - Tue, 04/08/2014 - 16:59
Intern takes the stress out of testing your Web app. Run tests using the browser or node.js and seamlessly integrate with SauceLabs or Selenium. Write tests using object, TDD, or BDD styles and get full code coverage reporting with Istanbul. Cont ...
Categories: Vendor