Selenium server Arguments.

Following are the arguments we can use while starting Selenium Server. If we use these arguments properly we can reduce the amount of time spent for finding a solution for complex scenarios.

-port : the port number the selenium server should use
(default 4444)

-timeout : an integer number of seconds before we should give up

-interactive: puts you into interactive mode.  See the tutorial for more details

-singleWindow: puts you into a mode where the test web site executes in a frame. This mode should only be selected if the application under test does not use frames.

-profilesLocation: Specifies the directory that holds the profiles that java clients can use to start up selenium.  Currently  supported for Firefox only.

-forcedBrowserMode : sets the browser mode to a single  argument (e.g. “*iexplore”) for all sessions, no matter what is passed to getNewBrowserSession

-forcedBrowserModeRestOfLine : sets the browser mode to all the remaining tokens on the line (e.g. “*custom /some/random/place/iexplore.exe”) for all sessions, no matter what is passed to getNewBrowserSession

-userExtensions : indicates a JavaScript file that will be loaded into selenium

-browserSessionReuse: stops re-initialization and spawning of the browser between tests

-avoidProxy: By default, we proxy every browser request; set this flag to make the browser use our proxy only for URLs containing ‘/selenium-server’

-firefoxProfileTemplate

: normally, we generate a fresh empty Firefox profile every time we launch.  You can specify a director to make us copy your profile directory instead.

-debug: puts you into debug mode, with more trace information anddiagnostics on the console

-browserSideLog: enables logging on the browser side; logging messages will be transmitted to the server.  This can affect performance.

-ensureCleanSession: If the browser does not have user profiles, make sure every new session has no artifacts from previous  sessions.  For example, enabling this option will cause all user cookies to be archived before launching IE, and restored after IE is closed.

-trustAllSSLCertificates: Forces the Selenium proxy to trust all  SSL certificates.  This doesn’t work in browsers that don’t use the  Selenium proxy.

-log : writes lots of debug information out to a log  file

-htmlSuite : Run a single HTML Selenese (Selenium Core) suite and then exit immediately, using the specified browser (e.g. “*firefox”) on the specified URL (e.g. “http://www.google.com”).  You need to specify the absolute path to the HTML test suite as well as the path to the HTML results file we’ll generate.

-proxyInjectionMode: puts you into proxy injection mode, a mode  where the selenium server acts as a proxy server for all content going to the test application.  Under this mode, multiple domains can be visited, and the following additional flags are supported:

-dontInjectRegex : an optional regular expression that proxy injection mode can use to know when to bypss injection

-userJsInjection : specifies a JavaScript file which will then be injected into all pages

-userContentTransformation : a regular  expression which is matched against all test HTML content; the  second is a string which will replace matches.  These flags can be used any number of times.  A simple example of how this could be useful: if you add “-userContentTransformation https http” then all “https” strings in the HTML of the test application will be changed to be “http”.

How to stop selenium Grid from Ant build?

How to stop selenium Grid from Ant build?

This is also a very simple process as similar to starting the Selenium Grrid from a bat/sh file[mentioned here].

First of all we need to create a bat/file with either of following commands

curl -d action=shutdown http://localhost:4444/lifecycle-manager
or 
wget --post-data action=shutdown action=shutdown http://localhost:4444/lifecycle-manager

Save the bat file in the Selenium Grid base directory. Double check bat file whether its working fine by double clicking on that[pre condition : Selenium Hub and RCs must started before].

I am using this way because there is no default ant target for shutting down the Selenium Grid.

Coming to Ant Build, we need to create a custom ant target in build.xml to execute the above bat file. It look like

Save the build.xml and execute target as ‘ant stopGrid‘.

How to start Selenium Grid hub and RCs with one click.

How to start Selenium Grid hub and RCs with one click.

Its a so simple process. We can done it by creating a bat/sh file.  Here is the steps

Open a notepad and enter following

start /B ant launch-hub 
start /B ant -Denvironment=”*firefox” -Dport=5555  -DhubURL=http://localhost:4444 launch-remote-control
start /B ant -Denvironment=”*iehta” -Dport=5557  -DhubURL=http://localhost:4444 launch-remote-control

Save the notepad as bat file in Selenium Grid base directory.  Double click on the bat file and it will start Hub and RCs.

How to configure selenium scripts through Hudson.

Through this post, I am trying to explain  ‘How to configure Selenium tests in Hudson’. We all know about Selenium and Hudson, these are two well know open source tools. And Hudson is very much helpful for building the selenium scripts in a scheduled manner hence Selenium will take more time to execute tests if suite contains more tests.

Before moving to configuration part we need to create a build.xml file for building the selenium scripts using Ant build. Because I am going to use Ant process to Invoke build.

Preconditions :

Selenium test cases should be able to run through Ant build.

Hudson Setup:

First of all, we need to install Hudson in our system. Its a straight forward and easy step. You can find that in http://wiki.hudson-ci.org/display/HUDSON/Installing+Hudson. After the installation hudson should be up in http://localhost:8080.

Configure the Hudson Jobs for building Selenium scripts:

Now we need to configure selenium Junit test cases in Hudson, for that follow below steps.

1. Click on ‘New Job’ link in the side panel.

2. Enter Job name[say DemoProject]

3. Select ‘Build a free-style software project’ and click on OK button. 

4. You can configure your build in different ways.

• Build the code from your machine
• From any of the Source Code Management’ system like CVS, SVN, GIT etc

5. Steps to build test cases from your local machine

5.1 Click on Advanced button under ‘Advanced Project Options’

5.2 Check ‘Use custom workspace’ check box.

5.3 Enter you path where test cases located.

6. Steps to build test cases from any of the ‘Source Code Management’ system like CVS, SVN, GIT [In this step, we don't want use 'custom workspace' as explained above]

6.1 Select any of the system under Source Code Management’ system [say Subversion]

6.2 Enter ‘Repository URL’

6.3 Keep ‘Local module directory’ as .

7. We can configure build time also in Hudson using ‘Build Triggers‘. Currently following build triggers are available

• Build after other projects are built
• Poll SCM
• Build Periodically[generally used one and using syntax of cron]

8. Now, we need to tell Hudson about how do we need to Build the test cases. For that we can use any methods in ‘Add Build Steps’ under ‘Build’ section. We can use either of

• Execute Windows batch command
• Invoke Ant
• Execute Shell
• Invoke top-level Maven targets.

9. Here I am going to use ‘Invoke Ant’  option and select it from the list.

10. Specify the target name in the ‘Targets’ field.

11. So you finished all the steps for build test cases. Now we need to show the results for that you need to check ‘Publish JUnit test result report’ check box.

12. Enter the path where your reports generates.

13. Click on ‘Save’ button.

Ohh now you are ready for your first Build.

14. Click on ‘Build Now’ button and wait for the build to finish.

You can see the terminal output by clicking ‘Console Output’ link [in corresponding Build page].

Hope it will help you to configure selenium test cases through Hudson.

Continuous Integration with Hudson

by Amit Easow

To give you a little history

Before I discovered Selenium grid and Hudson, I put in place a system for continuous testing using Selenium scripts written in HTML, Cruise Control for continuous integration and Subversion for version control of software code.

Prerequisites:

• Subversion (http://subversion.tigris.org/getting.html)
• Selenium Grid (http://selenium-grid.seleniumhq.org/)
• Hudson (https://hudson.dev.java.net/)

Download and install Selenium Grid. Run the sample demo to get a feel of what you are working with. I decided to teach myself ruby and what better way to do that than to play with an open source system, which recommends it.

Running the test cases locally??

After downloading the selenium grid project, I renamed it to seleniumSystemTests. Then I created my testSuite directory using the code in the examples/ruby directory, as a reference. Copy the Rake file and the equivalent of the spec_helper file from the examples/ruby directory to the testSuite directory. Modify the URL for application under test, in the spec_helper file equivalent.

application_host = ENV['SELENIUM_APPLICATION_HOST'] || "www.comcast.net"

I added my test cases in the testSuite directory and was able to run the test suite locally by following the steps below

In the first terminal (Mac OS X)

• cd seleniumSystemTests
• rake hub:start
• Open http://localhost:4444/console to see the supported browsers

In the second terminal

• cd seleniumSystemTests
• rake rc:start_all
• In the console above you will see 15 remote controls available

In the third terminal

• cd seleniumSystemTests/testSuite
• rake tests:run_in_parallel
• See the tests cases run in parallel on Firefox browsers

Add the seleniumSystemTests project into subversion

svn import seleniumSystemTests https:///path/to/repo/seleniumSystemTests -m "Checking in the selenium project"??

Running the test cases remotely

Now that the test cases are running successfully with the hub and remote controls running on the same local machine, the next logical step is to run the hub and remote controls on separate machines.

Machine A:??

• Download the seleniumSystemTests project
• cd seleniumSystemTests
• rake hub:start
• Open http://localhost:4444/console to see the browsers supported

Machine B:

• Check out the seleniumSystemTests project
• cd seleniumSystemTests
• rake HOST= HUB_URL=http://:4444 rc:start_all
• In the console above you will see 15 remote controls available

Go back to Machine A (continuing from where we left off)

• Open http://localhost:4444/console to see the 15 browsers available
• Open another terminal to the testSuite directory and run the test cases

rake tests:run_in_parallel

FAIL!!!!!

This is because the remote control server is set to localhost. Open the testSuite directory and find the spec_helper file equivalent. Modify it to say

remote_control_server = ENV['SELENIUM_RC_HOST'] || "" ?? ??

Running the test cases remotely and continuously

Now that the test cases are running remotely, the next logical step is to integrate it with Hudson to run it continuously. Sign into Hudson and configure it as below.

First create a node called Selenium Grid

• On the Hudson main page, click Manage Hudson link
• Click on Manage Nodes link

• Click New Node link

• Give the node a name, like Selenium Grid
• Choose Dumb Slave and click Ok button
• For Name, enter Selenium Grid
• For Description, enter Selenium Grid??
• For # of executors, enter 1
• For Remote FS root, enter your root directory path (/Users/)
• Leave Labels blank
• For Usage, choose Leave this machine for tied jobs only
• For Launch method, choose Launch slave methods via JNLP
• For Availability, choose Keep this slave on-line as much as possible
• Click Save button

Secondly create a Hudson project to run the test cases regularly

• From the Hudson home page, click New Job link.
• Give the new job a name, like seleniumSystemTests??
• Choose Build a free-style software project option and click Ok button
• Tie this project to the node named ?Selenium Grid?
• Choose subversion option and enter the SVN repo URL
• Set the build to run 5 min past every hour
• Enter the commands to execute the tests

cd testSuite

rake tests:run_in_parallel

• Click ?E-mail notification? option and enter your email address
• By default, the system is set to send you an email only when the build breaks
• Click ?Save? button

Finally, click the Build Now button and sit back.??

• Under Build History find the first build run.??
• Click the currently running build link
• Click Console Output link??
• See the test cases run and the Success message appears at the end

Hope this helps. Now you will receive an email only when the build breaks. In that case, once you open the URL in the email click the Console Output link. Now you can debug the problem.

Running the test cases remotely and continuously

Now that the tests are running continuously against the web application on an hourly basis, the next logic step I am going to take is to run it against more environments. Right now the test cases were running against the Mac/FF environment. Mac/Safari, Windows/IE and Windows/FF are on my list. From my initial investigation it looks like this should work by adding more parameters, indicating the browser and operating system being used, while starting up the remote control and while running the test cases.

Further logical steps include

• When new code is checked into subversion, Hudson should automatically deploy the application build to a web server
• Once this completes, a trigger goes off in the Hudson seleniumSystemTests project that imports any new test cases added to subversion. Then it runs the test cases against the new version of the web application.

Look out for further posts when I put the final pieces in place.

Integrating Selenium tests with Hudson using ant [Selenium + Hudson + Ant]

For most folks the challenge is that their CI system lives in some Unix environment and Selenium tests need to run on a Windows machine.To get this working I did the following

My goal was to

a)Have the test drive and report selenium tests from a single interface

b)Ability to run it on different machines with various browsers

I picked Hudson for a) for the simple reason that it’s widely used in our organization as well as it had a Selenium Grid plugin which will probably be used for accomplishing b).It’s easily available and can display JUnit reports in graphical format with drilldown capabilities.In short the works and then some:)

Tools used are :Hudson,Selenium RC 1.0.1,FF 3.5,Ant 1.7,Selenium Grid plugin for hudson,Java,TestNG,SVN,Windows XP,Hudson JUnit plugin

Note:Selenium Grid is only introduced to run the tests on multiple machines and different browsers.If this is what you don’t need,you can avoid one more layer of complexity

Step 1

This step is crucial and can differ from org to org,but if your hudson master and slave are in same subnet this should generally work.Make sure of this by pinging each host to other.If not you might need to get help from your IT such that the master hudson can ping the slave hudson and viceversa.

Step 2 (skip if not running on multiple machines)

Install Selenium Grid plugin from Manage Hudson->Plugins->Install

Hudson restart will be required

Step 3

Create Hudson slave using following method

a)Go to Manage Hudson->Manage Nodes->Add Node(Dumb Slave)

b) Enter in Name field anything you wish.However hudson recommends giving same name as the host name of slave.So give the computer name of your windows machine where the tests will run

c)Num of executors is directly corresponding to number of RC’s you will run.For now keep it 1.If you want to run two browsers(e.g FF 3.5 and IE 7) this number needs to be bumped upto 2

d)Remote FS Root:c:\hudson (this is where hudson Workspace artifacts will be stored if needed).If you are going to use Selenium Grid plugin to run,you won’t need this

e)Labels:Give a unique label e.g(windowsxp_1).You will see it’s use later with Selenium Grid

f)Usage:Leave this machine for tied jobs only

g)Launch slave agents via JNLP

h)Availability keep this machines as much online as possible

Rest of the options keep blank.

Repeat the process for each Slave machine,but to keep things simple let’s just configure 1 machine at a time

Step 4

Go to the Slave machine and

a)Open your browser and point to Hudson Master

b)Go to the Manage Nodes page and you should see a launch JNLP jar icon.Click on it and it should start the process of downloading “remoting.jar” from Hudson Master(I had trouble with this and found out that somehow hudson master didn’t have the jar file at a specific location.With the help of my Hudson admin,I was able to copy this jar correctly).So if you repeatedly fail to launch the slave agent,first thing to check would be the location of remoting.jar.Also look at the java exception and it will give you clues in what could have gone wrong.

c)If all goes well you should see a small window with Hudson icon showing the message “Connected”.

The above step is very important and needs to happen in order to Hudson run selenium tests.If this doesn’t work look into hudson logs,slave logs found on hudson server and there should be some clue on what’s going on.

Step 5

Create a new job on hudson and the configure the option to “run it on the slave node” that you just created.

Publish the Junit report by pointing to the output directory xml file.This will be something like c:\hudson\workspace\xx\xx.xml

Depending on what your hudson job is doing the slave machine should have appropriate software.E.g if you are going to run ANT files,the slave machine needs to have ANT properly configured.Same thing goes with SVN.Essentially everything that your job would need to run on master should be available on slave as well.

Step 6

Start a selenium server on the slave machine as you would if it was your local machine.For this testing make sure that versions of browser that you used for your environment to run selenium test locally is same.This will eliminate unnecessary issues that are not due to this setup but browser configuration issues.

Step 7

Run the hudson job from your master and you should see the browser window popping up on the slave and running the tests.

The above should accomplish the task of running your Selenium tests on a single Windows machine.However many times you would need to run your tests in parallel as well as to optimize hardware use,make sure that Hudson can trigger test on same machine with different browser and report results back.In order to accomplish this we will use the Selenium Grid plugin which I will cover in next part if folks are interested.

Some resources that were helpful to me in this exercise were

Amit Easow’s writeup on Hudson integration