Saturday, 16 June 2012

Testing GWT Apps with Selenium or WebDriver

Good functional testing is one of the most difficult tasks for web application developers and their teams. It is a challenge to develop tests that are cheap to maintain and yet provide good test coverage, which helps reduce QA costs and increase quality.

Both Selenium and WebDriver (which is essentially now the successor to Selenium) provide a good way to functionally test web applications in multiple target environments without manual work. In the past, web UIs were built using the page navigation to allow users to submit forms, etc. These days, more and more web applications use Ajax and therefore act and look a lot more like desktop applications. However, this poses problems for testing – Selenium and WebDriver are designed to work with user interations resulting in page navigation and don’t play well with AJAX apps out of the box.

GWT-based applications in particular have this problem, but there are some ways I’ve found to develop useful and effective tests. GWT also poses other issues in regards to simulating user input and locating DOM elements, and I discuss those below. Note that my code examples use Groovy to make them concise, but they can be pretty easily converted to Java code.

Problem 1: Handling Asynchronous Changes

One issue that developers face pretty quickly when testing applications based on GWT is detecting and waiting for a response to user interaction. For example, a user may click a button which results in an AJAX call which would either succeed and close a window or, alternatively, show an error message. What we need is a way to block until we see the expected changes, with a timeout so we can fail if we don’t see the expected changes.

Solution: Use WebDriverWait

The easiest way to do this is by taking advantage of the WebDriverWait (or Selenium’s Wait). This allows you to wait on a condition and proceed when it evaluates to true. Below I use Groovy code for the conciseness of using closures, but the same can be done in Java, though with a bit more code due to the need for anonymous classes.
def waitForCondition(Closure closure) {
int timeout = 20
WebDriverWait w = new WebDriverWait(driver, timeout)
w.until({
closure() // wait until this closure evaluates to true
} as ExpectedCondition)
}

def waitForElement(By finder) {
waitForCondition {
driver.findElements(finder).size() > 0;
}
}

def waitForElementRemoval(By finder) {
waitForCondition {
driver.findElements(finder).size() == 0;
}
}

// now some sample test code

submitButton.click() // submit a form

// wait for the expected error summary to show up
waitForElement(By.xpath("//div[@class='error-summary']"))
// maybe some more verification here to check the expected errors

// ... correct error and resubmit

submitButton.click()
waitForElementRemoval(By.xpath("//div[@class='error-summary']"))
waitForElementRemoval(By.id("windowId"))
As you can see from the example, your code can focus on the actual test logic while handling the asynchronous nature of GWT applications seamlessly.

Problem 2: Locating Elements when you have little control over DOM

In web applications that use templating (JSPs, Velocity, JSF, etc.), you have good control and easy visibility into the DOM structure that your pages will have. With GWT, this isn’t always the case. Often, you’re dealing with nested elements that you can’t control at a fine level.

With WebDriver and Selenium, you can target elements using a few methods, but the most useful are by DOM element ID and XPath. How can we leverage these to get maintainable tests that don’t break with minor layout changes?

Solution: Use XPath combined with IDs to limit scope

In my experience, to develop functional GWT tests in WebDriver, you should use somewhat loose XPath as your primary means of locating elements, and supplement it by scoping these calls by DOM ID, where applicable.

In particular, use IDs at top level elements like windows or tabs that are unique in your application and won’t exist more than once in a page. These can help scope your XPath expressions, which can look for window or form titles, field labels, etc.

Here are some examples to get you going. Note that we use // and * in our XPath to keep our expressions flexible so that layout changes do not break our tests unless they are major.
By byUserName = By.xpath("//*[@id='userTab']//*[text()='User Name']/..//input")
WebElement userNameField = webDriver.findElement(byUserName)
userNameField.sendKeys("my new user")

// maybe a user click and then wait for the window to disappear
By submitLocator = By.xpath("//*[@id='userTab']//input[@type='submit']")
WebElement submit = webDriver.findElement(submitLocator)
submit.click()

// use our helper method from Problem 1
waitForElementRemoval By.id("userTab")
Problem 3: Normal element interaction methods don’t work!

GWT and derivatives (Vaadin, GXT, etc.) often are doing some magic behind the scenes as far as managing the state of the DOM goes. To the developer, this means you’re not always dealing with plain or

Environment installation



We created this instruction for Windows platform as we use it on our workstations. We are not trying to build a powerful web service here, we just trying to figure out how to auto-test web applications. We love Unix too, and if your plan is to configure this on Unix, we expect that you will be able to assemble a similar setup using this post as a guidance.
Table of content


1.1 Java Development Kit (JDK)

1.2 Ant
1.3 Selenium

IE)


1.4 Браузер (Firefox,


2.1 TestNG

2.2 log4j



3.1 PHP


3.2 PEAR


3.3 PHPUnit


3.4 Testing_Selenium

3.5 log4php



4.1 TortoiseSVN


5. Additional information




1. Let’s start with required components
1.1 JDK

Please load it from here and make sure you have selected JDK (not JRE). All you need to do after download is to start the installation.
If you are a regular Java developer, you will most likely see many different versions of Java installed on you machine.  We installed downloaded package into С:\program files\Java\jdk1.6.0_12
Fig. 1 JDK install
Once installation is complete, we need to check that we can execute and compile Java code without specifying its home dir. Please check that the PATH variable set to C:\program files\Java\jdk1.6.0_12\bin. Confirm that the value is exactly as listed here. Close enough is not good enough. Also we have to create (check value) JAVA_HOME = C:\Program Files\Java\jdk1.6.0_12.
Fig. 2 All is looking good (so far)
1.2 Ant
To install Ant, we used the Ant 1.7.1 binaries and this installation manual. We will refer to installation directory as ANT_HOME. It was installed into C:\ant, and for us ANT_HOME = C:\ant. It is important to confirm that the PATH contains ANT_HOME\bin (in our case C:\ant\bin).
There is an assumption that our readers are already familiar with intuitively-obvious OS Windows interface for manipulating PATH.
To test the Ant install, we can execute following command ant to check that everything is ok.
Fig. 3 Validate Ant, install and config
This output indicates a successful install of Ant. We are satisfied with output shown on Fig.3.
1.3 Selenium Grid

We load it from here and unpack into C:\. The Selenium directory as in the archive, is best renamed to ‘selenium’. After the directory rename, we should see C:\selenium and we will refer to it as SELENIUM_HOME.
To validate you can run command – ant -f C:\selenium\build.xml sanity-check.
Fig. 4 Validating Selenium installation
1.4 Firefox*
Firefox can be downloaded here. We would recommend installing it into the default location – С:\Program Files\Mozilla Firefox, or obediently agree with all of the default options/selections during installation.
2. Preparing for Java tests launch
2.1 TestNG
We have chosen to use TestNG version 5.8-jdk15, which can be downloaded from here. It is enough to unpack content of downloaded archive into C:\testng. I will call this folder TESTNG_HOME. As you realised my TESTNG_HOME = C:\testng.
2.2 Log4j
We will load log4j for debugging and analysis of the ‘uncut’ logs of Selenium from Java. Load the unpacked archive into C:\log4j. LOG4J_HOME = C:\log4j.
3. Assembling to run PHP tests
There is a long road to install PHP driver for Selenium. I hope this walkthrough will save you some time. We will install PHP and PEAR. With help of PEAR we will install PHPUnit and finally we will install Testing_Selenium. Let’s begin!
3.1 PHP install
We use version 5.2.8 of PHP for Windows, using the msi-installer. You can load it from here. We prefer to use the msi-installer, because it allows us to configure all extensions, PEAR and even documentation in one quick step. Unlike the zip-package install. We definitely need PEAR and we recommend installing all extensions it offers – 16M. By the way, installer will set all PATH variables required for you too. I installed PHP into C:\php (PHP_HOME=C:\php).
Two points to mention during install:
Fig. 5 We do not need web server to run tests
Рис. 6 Do not forget to install PEAR and extensions
For those of you who does not like installer and likes to control installation process – please unpack archive into C:\php, set PATH and create environment variable PHPRC (PHP Runtime Configuration), which should point to a directory containing PHP configuration file – php.ini (in my case it is also C:\php).
Apr 16th, 2009 by Alena

Note that the curl extension is obligatory for installation. If it’s not installed you’ll see following error at execution of php-test:
P/1.1 400 Bad Request
Connection: close
Server: Jetty(6.0.x)
The presence of php_curl.dll file in C:\php\ext and ‘extension=php_curl.dll’ line in php.ini file confirm that curl extension is installed. Curl library functions are used to solve the problem of PHP Selenium driver and Selenium Grid interaction.
To check our progress at this point, we can run the ‘php –v’ command at command prompt. You should see something similar to Fig. 7.
Fig. 7 Validating PHP install
Comment: later, while running tests, we had to increase value of default_socket_timeout in file php.ini. I have it set to 300 seconds as sometimes SalesForce site is running tooo slooooow.
3.2 PEAR

At this point, we assume that PHP was installed using the msi method and we have included the PEAR support.
If so, we will find a file go-pear.bat. If you execute this batch file, and agree with all prompts, it will install PEAR. At the end of installation you will see a new file created in the PHP_HOME directory: PEAR_ENV.reg. We need to double click on this registry file in order to import necessary environment settings into the Windows registry.
Content of this file PEAR_ENV.reg:

REGEDIT4
[HKEY_CURRENT_USEREnvironment]
"PHP_PEAR_SYSCONF_DIR"="C:\\php"
"PHP_PEAR_INSTALL_DIR"="C:\\php\\pear"
"PHP_PEAR_DOC_DIR"="C:\\php\\docs"
"PHP_PEAR_BIN_DIR"="C:\\php"
"PHP_PEAR_DATA_DIR"="C:\\php\\data"
"PHP_PEAR_PHP_BIN"="C:\\php\\.\\php.exe"
"PHP_PEAR_TEST_DIR"="C:\\php\\tests"
By running command pear version we can validate PEAR install.

C:\>pear version
PEAR Version: 1.7.2
PHP Version: 5.2.8
Zend Engine Version: 2.2.0
Running on: Windows NT PHILKA 5.1 build 2600
3.3 Installing PHPUnit

PHPUnit represents an environment for Unit-tests creation. We will install PHPUnit with PEAR.
Initialising PEAR-channel for install:

C:\>pear channel-discover pear.phpunit.de
Adding Channel "pear.phpunit.de" succeeded
Discovery of channel "pear.phpunit.de" succeeded
Deutschland channel is open – we can ’sail’.

C:\>pear install channel://pear.phpunit.de/PHPUnit
Did not download optional dependencies: pear/Image_GraphViz, pear/Log, use --alldeps to download automatically
phpunit/PHPUnit can optionally use package "pear/Image_GraphViz" (version >= 1.2.1)
phpunit/PHPUnit can optionally use package "pear/Log"
phpunit/PHPUnit can optionally use PHP extension "pdo"
phpunit/PHPUnit can optionally use PHP extension "pdo_mysql"
phpunit/PHPUnit can optionally use PHP extension "pdo_sqlite"
phpunit/PHPUnit can optionally use PHP extension "xdebug" (version >= 2.0.0)
downloading PHPUnit-3.3.15.tgz ...
Starting to download PHPUnit-3.3.15.tgz (272,118 bytes)
...........................................done: 272,118 bytes
install ok: channel://pear.phpunit.de/PHPUnit-3.3.15
3.4 Testing_Selenium

Testing_Selenium is a PHP driver for Selenium RC. A more detailed package description can be obtained from here.
Let’s ‘deodorise’ channel pear.php.net …

C:>\pear channel-update pear.php.net
Updating channel "pear.php.net"
Update of Channel "pear.php.net" succeeded
… and now we start the install of Testing_Selenium component!

C:\>pear install channel://pear.php.net/Testing_Selenium-0.4.3
downloading Testing_Selenium-0.4.3.tgz ...
Starting to download Testing_Selenium-0.4.3.tgz (2,417,750 bytes)
...............................................done: 2,41
7,750 bytes
install ok: channel://pear.php.net/Testing_Selenium-0.4.3
March 02th, 2011 by Dima
This PEAR driver ver. 0.4.3 has big problems with driver-server interaction. So we reccommend you to replace downloaded Selenium.php with this.
3.5 log4php

During the selection of the components to use, we thought twice and decided that log4php logging will be more convenient for this exercise. We know that log4php is a younger twin of log4j. You can download log4php here or checkout log4php from this place.
If you do not know what an SVN ‘checkout’ operation is, you might like to install one of the SVN clients such as TortoiseSVN (see 4 below), and read its documentation to learn more.
Log4php file structure looks like this:
Fig. 8 log4php – file structure
After install, we must rename directory log4php\trunk\src\main\php into log4php and copy it to PHP_HOME. In this case the directory is C:\php. Once log4php is copied into c:\PHP. We are done!
4. TortoiseSVN**
Tortoise SVN is a convenient SVN client which is available from here. It is integrated into Windows Explorer, and this is the reason we must reboot after its installation. You will need it a bit later to checkout the Recruiting application tests from our SVN repository.
__________________________________________
* – You can use IE. As we expect this is already installed on this Windows platform. For those of you who need an instruction how to use IE instead of Firefox – we will explain before we start running tests.
** – Installation of TortoiseSVN is also not required especially if you have one of standalone or integrated SVN clients . There are plenty of tools which allow to checkout from SVN. Inquisitive minds can point their Browser at http://deepshiftlabs.com/svn/tests/trunk/salesforce/ and get files from out website. Although we don’t expect that you will.

Reference



As a morning exercise, we propose to run first workable revision of Java tests (Revision 54). We assume that you have installed environment (Selenium and Java). Our short guide on how to do this can be found on page ‘Environment installation’.
It is important to have Recruiting application created on SalesForce platform (in fact, you only need to create Recruiting Web Site object in it for now).
While creating Recruiting application it is good to follow book instructions precisely. Otherwise tests will bring many errors and it will be hard to figure out what was the cause, in this early version of Selenium tests on Java. Current version of the same tests will explain all errors in a user friendly way but we will discuss it later in this blog).
If you do not have Recruiting application, you will be able to look at code and resulting log file only (link to log file will be provided in next post). But if you are like us – determined and hardworking – and have application, you only need to configure tests and Selenium launchers. All information below is for you to do it quickly and painlessly.
Firstly, we extract revision 54 from SVN repository…
… Secondly, extract scripts which allow conveniently launch and stop Selenium Hub and RC. If you have own scripts or want to do it manually in a way described here – no problem.
We can launch one RC or four with these scripts.
All main settings are in start_hub_rc.bat and detailed instructions are in README file. We will use this bat-file to start Selenium.
If you want to start four browsers – please read last paragraph in a README file. README also explains what to do to use IE instead of Firefox.
Fig. 3
We will now change file settings.java and put our SalesForce account username and password. We will create a folder to store screenshots and write its location into settings.java too. You will need to change BROWSER here too if you want to run on IE.
Fig. 4
All we have to do now:
- rebuild.bat – compile (we do it every time we change one of Java files)
- start_hub_rc.bat – stat Selenium Hub and RC
- run.bat – start tests execution
and enjoy the action.

Fig. 5
Additional information about software installation and configuration can be obtained from the following trusted and reliable sources:
Environment installation – how to assemble components to be able to run tests we will create.
First launch – simple test to check setup by logging into your SalesForce account.

Reference

selenium + salesforce

Microsoft Dynamics CRM + Selenium

Coming Soon................

Monday, 28 May 2012

Automate Web Application Testing with Selenium for ABAP


Update: You find the current version of the SeleniumABAP Project at SAP Code Exchange.
Unfortunately I've caught a bad cold and I have to stay in bed exactly to the time I want to travel the Cluetrain to Vienna. Also today, which is the night of the SAP TechEd 2009 Demo Jam, I was not well enough to travel to Vienna. So I had to call Craig Cmehil this morning and canceled my attendance. But as already planned I release the source code of my project to the public. 

Demo Videos

To get an insight of what Selenium and Selenium for ABAP can do for you, please watch this Videos:

Source

The first thing what you need to record tests is Selenium IDE which comes as a Firefox plugin. If you're not running Firefox get it here.
To convert the recorded test in Selenium IDE to a ABAP Unit test class you need abap-rc.js which is a JavaScript file that can be imported into Selenium IDE by navigating to Options - Options ... - Formats in the Selenium IDE Menu. There you can click on the Add button, provide a name for the new format where I would suggest "ABAP Selenium RC" and then paste the code into the box below.
To run the generated ABAP Unit test you have to install the SAPLink Nugget ZSELENIUMABAP.nugg.zip. It provides the Selenium RC ABAP Client Driver class and also the XSL Transformations to create this class from the iedoc.xml which is the Selenium API definition.

I need your support

This is the current status of this project. What I would like to add is a possibility to generate eCATT tests in Selenium IDE and import them as an eCATT test script. If you know a way how to do that please contact me.