How to use the Advanced User Interactions API + Keyboard+ Mouse

• Introduction
• Getting started (short how-to)
• Keyboard interactions
• Mouse interactions
• Current status
• Outline
• A single action
• Generating Action chains
• Guidelines for extending the Action interface
• Implementation details
• A word of warning
• Keyboard
• Mouse
• Native events versus synthetic events
• Examples

Introduction

The Advanced User Interactions API is a new, more comprehensive API for describing actions a user can perform on a web page. This includes actions such as drag and drop or clicking multiple elements while holding down the Control key.

Getting started (short how-to)

In order to generate a sequence of actions, use the Actions generator to build it. First, configure it:

   Actions builder = new Actions(driver);

   builder.keyDown(Keys.CONTROL)
       .click(someElement)
       .click(someOtherElement)
       .keyUp(Keys.CONTROL);

Then get the action:

   Action selectMultiple = builder.build();

And execute it:

   selectMultiple.perform();

The sequence of actions should be short - it's better to perform a short sequence of actions and verify that the page is in the right state before the rest of the sequence takes place. The next section lists all available actions and how can they be extended.

Keyboard interactions

Until now, keyboard interaction took place through a specific element and WebDriver made sure the element is in the proper state for this interaction. This mainly consisted of scrolling the element into the viewport and focusing on the element.

Since the new Interactions API takes a user-oriented approach, it is more logical to explicitly interact with the element before sending text to it, like a user would. This means clicking on an element or sending a Keys.TAB when focused on an adjacent element.

The new interactions API will (first) support keyboard actions without a provided element. The additional work to focus on an element before sending it keyboard events will be added later on.

Mouse interactions

Mouse actions have a context - the current location of the mouse. So when setting a context for several mouse actions (using onElement), the first action will be relative to the location of the element used as context, the next action will be relative to the location of the mouse at the end of the last action, etc.

Current status

The API is (mostly) finalized for the actions and actions generator. It is fully implemented for HtmlUnit and Firefox and in the process of being implemented for Opera and IE.

Outline

A single action

All actions implement the Action interface. This action only has one method: perform(). The idea being that each action gets the required information passed in the Constructor. When invoked, the action then figures out how it should interact with the page (for example, finding out the active element to send the key to or calculating the screen coordinates of an element for a click) and calls the underlying implementation to actually carry out the interaction.

There are currently several actions:

• ButtonReleaseAction - Releasing a held mouse button.
• ClickAction - Equivalent to WebElement.click()
• ClickAndHoldAction - Holding down the left mouse button.
• ContextClickAction - Clicking the mouse button that (usually) brings up the contextual menu.
• DoubleClickAction - double-clicking an element.
• KeyDownAction - Holding down a modifier key.
• KeyUpAction - Releasing a modifier key.
• MoveMouseAction - Moving the mouse from its current location to another element.
• MoveToOffsetAction - Moving the mouse to an offset from an element (The offset could be negative and the element could be the same element that the mouse has just moved to).
• SendKeysAction - Equivalent to WebElement.sendKey(...)

The CompositeAction contains other actions and when its perform method is invoked, it will invoke the perform method of each of the actions it contains. Usually, the actions should not created directly - the ActionChainsGenerator should take care of that.

Generating Action chains

The Actions chain generator implements the Builder pattern to create a CompositeAction containing a group of other actions. This should ease building actions by configuring an Actions chains generator instance and invoking it's build() method to get the complex action:

   Actions builder = new Actions(driver);

   Action dragAndDrop = builder.clickAndHold(someElement)
       .moveToElement(otherElement)
       .release(otherElement)
       .build();

   dragAndDrop.perform();

A planned extension to the Actions class is adding a method that will append any Action to the current list of actions it holds. This will allow adding extended actions without manually creating the CompositeAction. On extending actions, see below.

Guidelines for extending the Action interface

Thie Action interface only has one action - perform(). In addition to the actual interaction itself, any evaluation of conditions should be performed in this method. It's possible that the page state has changed between creation of the action and when it was actually performed - so things like element's visibility and coordinates shouldn't be found out in the Action constructor.

Implementation details

To achieve separation between the operations each action is performing and the actual implementation of the operations, all actions rely on two interfaces: Mouse and Keyboard. These interfaces are implemented by every driver that supports the advanced user interactions. Note that these interfaces are designated to be used by the actions - not by end users - the information in this section is only useful for developers planning to extend WebDriver.

A word of warning

The keyboard and mouse interface are designed to be used by the various action classes. For this reason, their API is less stable than that of theActions chain generator. Directly using these interfaces may not yield the expected results, as the actions themselves do additional work to make sure the right conditions are met before events are actually generated. Such preliminary work includes focusing on the right element or making sure the element is visible before any mouse interaction.

Keyboard

The Keyboard interface has three methods:

• void sendKeys(CharSequence... keysToSend) - Similar to the existing sendKeys(...) method.
• void pressKey(Keys keyToPress) - Sends a key press only, without releasing it. Should only be implemented for modifier keys (Control, Alt and Shift).
• void releaseKey(Keys keyToRelease) - Releases a modifier key.

It is the implementation's responsibility to store the state of modifier keys between calls. The element which will receive those events is the active element.

Mouse

The Mouse interface includes the following methods (This interface will change soon):

• void click(WebElement onElement) - Similar to the existing click() method.
• void doubleClick(WebElement onElement) - Double-clicks an element.
• void mouseDown(WebElement onElement) - Holds down the left mouse button on an element.

Action selectMultiple = builder.build();

• void mouseUp(WebElement onElement) - Releases the mouse button on an element.
• void mouseMove(WebElement toElement) - Move (from the current location) to another element.
• void mouseMove(WebElement toElement, long xOffset, long yOffset) - Move (from the current location) to new coordinates: (X coordinates of toElement + xOffset, Y coordinates of toElement + yOffset).
• void contextClick(WebElement onElement) - Performs a context-click (right click) on an element.

Native events versus synthetic events

In WebDriver advanced user interactions are provided by either simulating the Javascript events directly (i.e. synthetic events) or by letting the browser generate the Javascript events (i.e. native events). Native events simulate the user interactions better whereas synthetic events are platform independent, which can be important in Linux when alternative window managers are used, see native events on Linux. Native events should be used whenever it is possible.

The following table shows which browsers support which kind of events:

Browser	Operating system	Native events	Synthetic events
Firefox	Linux	supported	supported (default)
Firefox	Windows	supported (default)	supported
Internet Explorer	Windows	supported (default)	not supported
Chrome	Linux/Windows	supported*	not supported
Opera	Linux/Windows	supported (default)	not supported
HtmlUnit	Linux/Windows	supported (default)	not supported

*) ChromeDriver provides two modes of supporting native events called WebKit events and raw events. In the WebKit events the ChromeDrivercalls the WebKit functions which trigger Javascript events, in the raw events mode operating systems events are used.

In the FirefoxDriver, native events can be turned on and off in the FirefoxProfile.

FirefoxProfile profile = new FirefoxProfile();
profile.setEnableNativeEvents(true);
FirefoxDriver driver = new FirefoxDriver(profile);

Examples

These are some examples where native events behave different to synthetic events:

• With synthetic events it is possible to click on elements which are hidden behind other elements. With native events the browser sends the click event to the top most element at the given location, as it would happen when the user clicks on the specific location.
• When a user presses the 'tab' key the focus jumps from the current element to the next element. This is done by the browser. With synthetic events the browser does not know that the 'tab' key is pressed and therefore won't change the focus. With native events the browser will behave as expected.

reference

http://code.google.com/p/selenium/wiki/AdvancedUserInteractions

Selenium testing your GWT application using maven

Selenium is a great tool to do browser based testing of your web user interface. While it can be a bit of a pain to set up properly, it is fun to see your application being used very fast in the test and comforting to know that your user interface logic is also verified by your continuous integration system.

There are two important steps in the process. You need to get your maven configuration running and you have to write the actual test.

For this example, we are showing a selenium test which verifies the security aspects in Geomajas (a GIS application framework) using the staticsecurity plug-in. This example verifies a small GWT application which uses the SmartGWT widget library. The running test looks like this:

Maven configuration

To be able to use Selenium for the test, you need to include some dependencies.

  org.seleniumhq.selenium
  selenium-java
  2.5.0
  test


  org.seleniumhq.selenium
  selenium-server
  2.5.0
  test

To run the selenium tests, the web application has to be runnable. I did this by configuring jetty. This is done as a build plug-in and allows “mvn jetty:run” to work. As this is a GWT application, you need to add a location to assure the GWT compiled stuff is included in the application.

  org.mortbay.jetty
  maven-jetty-plugin
  6.1.20
  
    
      /
      
        
        
          ${basedir}/src/main/webapp,${project.build.directory}/${project.build.finalName}
        
      
    
    manual
  

I normally put the selenium tests in a profile to allow your build to run faster by switching off the selenium tests.
The profile is defined below. You can disable the selenium tests by including “-PskipSelenium” on the maven command line. The actual steps are included as build plug-ins in the profile. Please beware that this type of configuration will cause profiles which are indicates as activeByDefault to be disabled.

  selenium-tests
  
    
      !skipSelenium
    
  
  
    
      
    
  

The selenium tests are run in the integration test phase.
We need to assure the application is started. We will start the application using the jetty servlet engine.
The most important part are the executions. There is one execution to start jetty in the pre-integration-test phase and another to stop jetty in the post-integration-test phase. The jetty runs on port 9080 instead of 8080. This is not required but is done to prevent clashes when working on several projects at the same time.

  org.mortbay.jetty
  maven-jetty-plugin
  
    
      /
    
    manual
    9966
    stop-jetty
  
  
    
      start-jetty
      pre-integration-test
      
        run
      
      
        true
        5
        
          
            9080
            60000
          
        
      
    
    
      stop-jetty
      post-integration-test
      
        stop
      
    
  

The tests are actually run by the surefire plug-in. Integration tests are marked as such in the class name. This is done by making the class name start with “IntTest” or end in “TestInt” (you can change this pattern, but is is recommended not to make it start or end in “Test” to prevent the need for special configuration to exclude it from the test phase).

  org.apache.maven.plugins
  maven-surefire-plugin
  
    
      integration-test
      
        test
      
      
        
          **/*TestInt.java
          **/IntTest*.java
        
      
    
  

Note that no server side component needs to run to execute the test itself, just the web application. So while developing, you could just as well keep the application running (for example by using “mvn jetty:run”) and just run the test itself from your IDE.

You can have a look at the full pom here.

Test code

For the test, I will use the selenium driver to connect with the web browser. This is initialised using the code below. You can use different implementation of WebDriver to switch between different browsers. There is also a version which does not use a browser at all. In this case, I am using the FirefoxDriver.

private WebDriver driver;

@Autowired
private CommandCountAssert commandCountAssert;

@Before
public void setUp() {
  driver = new FirefoxDriver();
}

@After
public void tearDown() {
  driver.quit();
}

Some constants have been defined for the string which are used in the test.

private static final String LAYER_VECTOR = "-clientLayerCountries";
private static final int LAYER_VECTOR_LENGTH = LAYER_VECTOR.length() - 1;
private static final String LAYER_VECTOR_XPATH =
  "//*[substring(@id, string-length(@id)-" + LAYER_VECTOR_LENGTH + ")= '" + LAYER_VECTOR + "']";
private static final String LAYER_RASTER = "-clientLayerOsm";
private static final int LAYER_RASTER_LENGTH = LAYER_RASTER.length() - 1;
private static final String LAYER_RASTER_XPATH =
  "//*[substring(@id, string-length(@id)-" + LAYER_RASTER_LENGTH + ")= '" + LAYER_RASTER + "']";

As you could see from the clip at the beginning, the test verifies many things. Let’s look at this is small steps.

To begin, we need to initialise some objects which do the actual testing later on. The wait service allows you to wait for something to appear in the DOM tree. It is configured to wait at most 20 seconds. There is also a commandCountAssert service which is initialised. This is a spring service which counts the command interactions.

String source;
List elements;
WebDriverWait wait = new WebDriverWait(driver, 20);
wait.pollingEvery(500, TimeUnit.MILLISECONDS);
commandCountAssert.init();

Now connect to the actual application. The port here matches the port from the jetty configuration in the pom.

driver.get("http://localhost:9080/");

We are testing the security aspects and the program should immediately display a login window. Verify this by waiting for the window to appear. We do this by trying to select the object using the class name.

// the login window should appear
wait.until(new ExpectedCondition() {
  public Boolean apply(WebDriver d) {
    return null != d.findElement(By.className(TokenRequestWindow.STYLE_NAME_WINDOW));
  }
});

We also want to check that only only one command was sent to the server and that the other commands are queued for when the authentication is done.

commandCountAssert.assertEquals(1);

We need to find some elements in the DOM tree for this. The “aria-label” XPath expressions is required because of how SmartGWT handles buttons. This actually finds the button with the given label. We start by just finding the useful bits from the login window.

WebElement userName = driver.findElement(By.name("userName"));
WebElement password = driver.findElement(By.name("password"));
WebElement login = driver.findElement(By.xpath("//*[@aria-label='Log in']"));
WebElement reset = driver.findElement(By.xpath("//*[@aria-label='Reset']"));

Let’s fill the login window with some invalid credentials. We fill in the user name and password fields and click to login. We then wait for the “Login attempt has failed” message.

userName.sendKeys("blabla");
password.sendKeys("blabla");
login.click();
wait.until(new ExpectedCondition() {
  public Boolean apply(WebDriver d) {
    return d.findElement(By.className(TokenRequestWindow.STYLE_NAME_ERROR)).getText().
        contains("Login attempt has failed");
  }
});

Verify that only the login attempt command was sent.

commandCountAssert.assertEquals(1);

Now clear the form and check that the error message is also cleared.

WebElement error = driver.findElement(By.className(TokenRequestWindow.STYLE_NAME_ERROR));
reset.click();
Assert.assertEquals("", error.getText());

Make sure that the correct error is displayed when no user name is specified.

reset.click();
userName.clear();
password.sendKeys("luc");
login.click();
wait.until(new ExpectedCondition() {
  public Boolean apply(WebDriver d) {
    return null != d.findElement(By.xpath("//*[contains(.,'Please fill in a user name.')]"));
  }
});

And this should be entirely client-side.

commandCountAssert.assertEquals(0);

Similarly, trying to login without password should also fail.

reset.click();
userName.sendKeys("luc");
password.clear();
login.click();
wait.until(new ExpectedCondition() {
  public Boolean apply(WebDriver d) {
    return null != d.findElement(By.xpath("//*[contains(.,'Please fill in a password.')]"));
  }
});
commandCountAssert.assertEquals(0);

Now pass some valid credentials.

reset.click();
userName.sendKeys("luc");
password.sendKeys("luc");
login.click();

The application title should be displayed and the map has to appear.

wait.until(new ExpectedCondition() {
  public Boolean apply(WebDriver d) {
    return null != d.findElement(By.className(Application.APPLICATION_TITLE_STYLE));
  }
});
wait.until(new ExpectedCondition() {
  public Boolean apply(WebDriver d) {
    List elements = driver.findElements(By.xpath(LAYER_VECTOR_XPATH));
    return !elements.isEmpty();
  }
});

Logging in should display the user name on screen. It should also display a raster layer and a “blabla” button. For the button it checks that it is actually visible (when the button is removed, SmartGWT will hide it, not remove it from the DOM tree).

WebElement user = driver.findElement(By.className(Application.APPLICATION_USER_STYLE));
Assert.assertEquals("user: Luc Van Lierde", user.getText());
elements = driver.findElements(By.xpath(LAYER_RASTER_XPATH));
Assert.assertFalse(elements.isEmpty()); // there should be a raster layer
WebElement blabla = driver.findElement(By.xpath("//*[@aria-label='blabla']")); 
Assert.assertNotNull(blabla); // should exist
Assert.assertFalse(blabla.getAttribute("style").contains("visibility: hidden"));

Now check that the login window disappeared. We search for the elements with the class name of the login window and verify that nothing exists.

Assert.assertEquals(0, driver.findElements(By.className(TokenRequestWindow.STYLE_NAME_WINDOW)).size());

With all this work, somewhere between 20 and 40 commands should have been executed (the exact number depends on screen size).

commandCountAssert.assertBetween(20, 40);

Now logout again. This should make the login window appear again. The layers in the map should have disappeared and two commands should have been sent in the process.

WebElement logout = driver.findElement(By.xpath("//*[@aria-label='Log out']"));
logout.click();
wait.until(new ExpectedCondition() {
  public Boolean apply(WebDriver d) {
    return null != d.findElement(By.className(TokenRequestWindow.STYLE_NAME_WINDOW));
  }
});
source = driver.getPageSource();
Assert.assertFalse(source.contains(LAYER_VECTOR));
Assert.assertFalse(source.contains(LAYER_RASTER));
commandCountAssert.assertEquals(2);

We will now login as a different user.

// login as other user
userName = driver.findElement(By.name("userName"));
password = driver.findElement(By.name("password"));
login = driver.findElement(By.xpath("//*[@aria-label='Log in']"));
userName.sendKeys("marino");
password.sendKeys("marino");
login.click();

The raster layer should appear again. This time there should be no vector layer and the “blabla” button should not be usable.

wait.until(new ExpectedCondition() {
  public Boolean apply(WebDriver d) {
    List elements = driver.findElements(By.xpath(LAYER_RASTER_XPATH));
    return !elements.isEmpty();
  }
});
source = driver.getPageSource();
Assert.assertFalse(source.contains(LAYER_VECTOR));
blabla = driver.findElement(By.xpath("//*[@aria-label='blabla']"));
Assert.assertTrue(blabla.getAttribute("style").contains("visibility: hidden"));

reference
http://blog.progs.be/199/selenium-testing-your-gwt-application-using-maven

This covers a basic GWT Selenium WebDriver setup

Download Source Basic Selenium Setup 0. install the Chrome web driver     - Chrome WebDriver can be found here: http://code.google.com/p/chromium/downloads/list 1. Install maven m2e eclipse plugin 2. Create plain old java project 3. Convert project to maven project.      > right click on project > goto menu configure > Convert to maven project 4. open the pom.xml 5. goto lower right tab in pom.xml to the tab pom.xml, which will show xml source 6. add this to pom.xml: xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">   4.0.0   SLP_Testing   SLP_Testing   0.0.1-SNAPSHOT                                               org.seleniumhq.selenium                 selenium-java                 2.14.0                             7. wait while the dependencies download.  8. create a source folder and package and add this class: package com.gonevertical.client; import java.io.IOException; import java.util.Arrays; import junit.framework.TestCase; import org.junit.After; import org.junit.AfterClass; import org.junit.Before; import org.junit.BeforeClass; import org.junit.Test; import org.junit.runner.RunWith; import org.junit.runners.BlockJUnit4ClassRunner; import org.openqa.selenium.By; import org.openqa.selenium.WebDriver; import org.openqa.selenium.WebElement; import org.openqa.selenium.chrome.ChromeDriver; import org.openqa.selenium.remote.DesiredCapabilities; /**  * {@link http://c.gwt-examples.com}  */ @RunWith(BlockJUnit4ClassRunner.class) public class TestGwtAppLoading extends TestCase {   private static WebDriver driver;   @BeforeClass   public static void initWebDriver() throws IOException {         // set the path to the chrome driver     // http://code.google.com/p/chromium/downloads/list     System.setProperty("webdriver.chrome.driver", "/Users/branflake2267/bin/chromedriver");         // chrome driver will load with no extras, so lets tell it to load with gwtdev plugin     // get plugin here in chrome url type > "chrome://plugins/" > hit top right details + > find gwt dev mode pluging location:     DesiredCapabilities capabilities = DesiredCapabilities.chrome();     String gwtDevPluginPath = "--load-plugin=/Users/branflake2267/Library/Application Support/Google/Chrome/Default/Extensions/jpjpnpmbddbjkfaccnmhnkdgjideieim/1.0.9738_0/Darwin-gcc3/gwtDev.plugin";     capabilities.setCapability("chrome.switches", Arrays.asList(gwtDevPluginPath));         // init driver with GWT Dev Plugin     driver = new ChromeDriver(capabilities);   }   @AfterClass   public static void theEnd() {     driver.quit();   }   @Before   public void before() {   }   @After   public void after() {   }   @Test   public void testElement() {             // navigate to gwt app - change this to your app url     String pathToGwtApp = "http://127.0.0.1:8888/index.html?gwt.codesvr=127.0.0.1:9997";     driver.get(pathToGwtApp);     // Don't forget to add the inherits Debug && set textAreaWidget.ensureDebugId("myTextAreaId"); in your gwt app code     WebElement elementRta = driver.findElement(By.id("gwt-debug-myTextAreaId"));     // do something with the element             // set a break point on this, b/c there isn't any pausing     System.out.println("finished");       }   } Finding Elements in Selenium with ensureDebugId Selenium will need to reference the widgets by there id using widget.ensureDebugId("seleniumElementReferencetId"); in your code.  1. add   to your project.gwt.xml file 2. If in your code your textAreaWidget.ensureDebugId("myTextAreaId"); is like then 3. Reference the element in selenium driver.findElement(By.id("gwt-debug-myTextAreaId")); 4. If your unsure it is working use the developer tools and search for the elementId. ClickAndWait for a page to load This snippet is loading my application and then clicking on the Google login link then selecting isAdmin and then clicking on login and back to the application as a logged in (dev) user. Also showing below is the method of waiting for completion of page load before moving on.  @Test   public void testSignIn() {         // navigate to gwt app     String pathToGwtApp = "http://127.0.0.1:8888/index.html?gwt.codesvr=127.0.0.1:9997";     driver.get(pathToGwtApp);     // wait till the app is ready for use     (new WebDriverWait(driver, 10)).until(new ExpectedCondition<Boolean>() {       public Boolean apply(WebDriver d) {         WebElement element = driver.findElement(By.id("gwt-debug-Login_SignIn"));         return element.isEnabled();       }     });         WebElement element = driver.findElement(By.id("gwt-debug-Login_SignIn"));     element.click();         // wait till where on the dev login page     (new WebDriverWait(driver, 2)).until(new ExpectedCondition<Boolean>() {       public Boolean apply(WebDriver d) {         WebElement isAdmin = driver.findElement(By.id("isAdmin"));         return isAdmin.isEnabled();       }     });         // click on isAdmin     WebElement isAdmin = driver.findElement(By.id("isAdmin"));     isAdmin.click();         // click on login     WebElement form = driver.findElement(By.xpath("/html/body/form/div/p[3]/input[1]"));     form.click();         // wait till we get back to the user logged in with there tabs     (new WebDriverWait(driver, 20)).until(new ExpectedCondition<Boolean>() {       public Boolean apply(WebDriver d) {         WebElement element = driver.findElement(By.id("gwt-debug-tabsSchoolUser"));         return element.isEnabled();       }     });         // set a break point on this, b/c there isn't any pausing     System.out.println("finished");   } reference http://c.gwt-examples.com/home/testing/selenium-testing

How-to click a GWT 2.0 button with Selenium

I am testing a web application with selenium.
Some buttons of the web app are rendered with GWT 2.0, they cannot be clicked by "click" command.

There are other pages that describe and address this problem, such as:
Selenium Testing of GWT 2.0 and 
Simulating clicks on GWT push buttons with Selenium RC.

Nevertheless, when I was also pondering this problem, an old GUI technique came to my mind. You can "simulate" mouse click by pressing tab until the target widget get focus, then press Enter to activate that widget. Similarly, you can press Enter on the web element by

keyPress(elementLocator, \13)

Where 13 is ASCII for return




reference

http://dingyichen.livejournal.com/23628.html