Friday 20 July 2012

Saturday 7 July 2012

QAJobs.net - Software Testing & Quality Assurance QA Jobs Network

QAJobs.net - Software Testing & Quality Assurance QA Jobs Network


QAJobs.net - Software Testing & Quality Assurance QA Jobs Network

www.qajobs.net/
Software Testing and Quality Assurance Jobs and Resumes, QA Training Consulting, Automated Testing, QA Test Automation, Software Testing Tools, software ...

Software Testing & Quality ...

QAJobs.net - Software Testing & Quality Assurance Jobs ...

New Candidate?

If you've already registered with SQAjobs.com, please do not re ...

Advanced Search

Advanced Search. Search Jobs. Category. Managerial ...

QAJobs.net - Software Testing ...

QAJobs.net - Software Testing & Quality Assurance Jobs ...

Employer Registration

Registration is free. You will not be charged until you post job ...

QA Web Analyst

Requirements. Berean Group is seeking a QA Web Analyst Job ...

Wednesday 4 July 2012

Choosing a test automation framework


Basing an automated testing effort on using only a capture tool such as IBM Rational® Robot to record and play back test cases has its drawbacks. Running complex and powerful tests is time consuming and expensive when using only a capture tool. Because these tests are created ad hoc, their functionality can be difficult to track and reproduce, and they can be costly to maintain.
A better choice for an automated testing team that's just getting started might be to use a test automation framework, defined as a set of assumptions, concepts, and practices that constitute a work platform or support for automated testing. In this article I'll attempt to shed a little light on a handful of the test automation frameworks I'm familiar with -- specifically, test script modularity, test library architecture, keyword-driven/table-driven testing, data-driven testing, and hybrid test automation. I won't evaluate which framework is better or worse but will just offer a description and a demonstration of each and, where appropriate, some tips on how to implement it in the IBM Rational toolset.
The test script modularity framework requires the creation of small, independent scripts that represent modules, sections, and functions of the application-under-test. These small scripts are then used in a hierarchical fashion to construct larger tests, realizing a particular test case.
Of all the frameworks I'll review, this one should be the simplest to grasp and master. It's a well-known programming strategy to build an abstraction layer in front of a component to hide the component from the rest of the application. This insulates the application from modifications in the component and provides modularity in the application design. The test script modularity framework applies this principle of abstraction or encapsulation in order to improve the maintainability and scalability of automated test suites.
To demonstrate the use of this framework, I'll automate a simple test case for the Windows Calculator program (see Figure 1) to test the basic functions (add, subtract, divide, and multiply).

Figure 1. The Windows Calculator
calculator
At the bottom level of the script hierarchy are the individual scripts testing addition, subtraction, multiplication, and division. As examples, the first script that follows tests addition and the second script tests subtraction.
Sub Main

Window Set Context, "Caption=Calculator", ""

'5
PushButton Click, "ObjectIndex=10"

'+
PushButton Click, "ObjectIndex=20"

'6
PushButton Click, "ObjectIndex=14"

'=
PushButton Click, "ObjectIndex=21"

'11
Result = LabelUP (CompareProperties, "Text=11.", "UP=Object Properties")

End Sub

Sub Main

Window Set Context, "Caption=Calculator", ""

'20
PushButton Click, "ObjectIndex=11"
PushButton Click, "ObjectIndex=8"

'-
PushButton Click, "ObjectIndex=19"

'10
PushButton Click, "ObjectIndex=7"
PushButton Click, "ObjectIndex=8"

'=
PushButton Click, "ObjectIndex=21"

'10
Result = LabelUP (CompareProperties, "Text=10.", "UP=Object Properties")

End Sub

The two scripts at the next level of the hierarchy would then be used to represent the Standard view and the Scientific view available from the View menu. As the following script for the Standard view illustrates, these scripts would contain calls to the scripts we built previously.
'Test Script Modularity Framework
'Script for Standard View

Sub Main

'Test Add Functionality
CallScript "Test Script Mod Framework - Add"

'Test Subtract Functionality
CallScript "Test Script Mod Framework - Substract"

'Test Divide Functionality
CallScript "Test Script Mod Framework - Divide"

'Test Multiply Functionality
CallScript "Test Script Mod Framework - Multiply"

End Sub

And finally, the topmost script in the hierarchy would be the test case to test the different views of the application.
'Test Script Modularity Framework
'Top level script - represents test case

Sub Main

'Test the Standard View
CallScript "Test Script Mod Framework - Standard"

'Test the Scientific View
CallScript "Test Script Mod Framework - Scientific"

End Sub

From this very simple example you can see how this framework yields a high degree of modularization and adds to the overall maintainability of the test suite. If a control gets moved on the Calculator, all you need to change is the bottom-level script that calls that control, not all the test cases that test that control.
The test library architecture framework is very similar to the test script modularity framework and offers the same advantages, but it divides the application-under-test into procedures and functions instead of scripts. This framework requires the creation of library files (SQABasic libraries, APIs, DLLs, and such) that represent modules, sections, and functions of the application-under-test. These library files are then called directly from the test case script.
To demonstrate the use of this framework, I'll automate the same test case as above but use an SQABasic library. The library contains a function to perform the operations. Following are the header file (.sbh) and the library source file (.sbl).
'Header File
'Test Library Architecture Framework
"Functions Library

Declare Sub StandardViewFunction BasicLib "Functions Library" (OperandOne As Integer, _
OperandTwo As Interger, _
Operation As String)

'Library Source File
'Test Library Architecture Framework
'Functions Library

Sub StandardViewFunction (OperandOne As Integer, _
OperandTwo As Interger, _
Operation As String)

'Click on first operand
Select Case OperandOne
Case 0
PushButton Click, "ObjectIndex=8"
Case 1
PushButton Click, "ObjectIndex=7"
Case 2
PushButton Click, "ObjectIndex=11"
Case 3
PushButton Click, "ObjectIndex=15"
Case 4
PushButton Click, "ObjectIndex=6"
Case 5
PushButton Click, "ObjectIndex=10"
Case 6
PushButton Click, "ObjectIndex=14"
Case 7
PushButton Click, "ObjectIndex=5"
Case 8
PushButton Click, "ObjectIndex=9"
Case 9
PushButton Click, "ObjectIndex=13"
End Select

'Click on first operand
Select Case OperandOne
Case "+"
PushButton Click, "ObjectIndex=8"
Case "-"
PushButton Click, "ObjectIndex=7"
Case "*"
PushButton Click, "ObjectIndex=11"
Case "/"
PushButton Click, "ObjectIndex=15"
End Select

'Click on first operand
Select Case OperandOne
Case 0
PushButton Click, "ObjectIndex=8"
Case 0
PushButton Click, "ObjectIndex=7"
Case 0
PushButton Click, "ObjectIndex=11"
Case 0
PushButton Click, "ObjectIndex=15"
Case 0
PushButton Click, "ObjectIndex=6"
Case 0
PushButton Click, "ObjectIndex=10"
Case 0
PushButton Click, "ObjectIndex=14"
Case 0
PushButton Click, "ObjectIndex=5"
Case 0
PushButton Click, "ObjectIndex=9"
Case 0
PushButton Click, "ObjectIndex=13"
End Select

'=
PushButton Click, "ObjectIndex=21"

End Sub

Using this library, the following test case script can be made.
'Test Library Architecture Framework
'Test Case script

'$Include "Functions Library.sbh"

Sub Main

'Test the Standard View
Window Set Context, "Caption=Calculator", ""

'Test Add Functionalty
StandardViewFunction 3,4,"+"
Result = LabelVP (CompareProperties, "Text=7.", "VP=Add")

'Test Subtract Functionalty
StandardViewFunction 3,2,"-"
Result = LabelVP (CompareProperties, "Text=1.", "VP=Sub")

'Test Multiply Functionalty
StandardViewFunction 4,2,"*"
Result = LabelVP (CompareProperties, "Text=8.", "VP=Mult")

'Test Divide Functionalty
StandardViewFunction 10,5,"/"
Result = LabelVP (CompareProperties, "Text=2.", "VP=Div")

End Sub

From this example, you can see that this framework also yields a high degree of modularization and adds to the overall maintainability of the test suite. Just as in test script modularity, if a control gets moved on the Calculator, all you need to change is the library file, and all test cases that call that control are updated.
Keyword-driven testing and table-driven testing are interchangeable terms that refer to an application-independent automation framework. This framework requires the development of data tables and keywords, independent of the test automation tool used to execute them and the test script code that "drives" the application-under-test and the data. Keyword-driven tests look very similar to manual test cases. In a keyword-driven test, the functionality of the application-under-test is documented in a table as well as in step-by-step instructions for each test.
If we were to map out the actions we perform with the mouse when we test our Windows Calculator functions by hand, we could create the following table. The "Window" column contains the name of the application window where we're performing the mouse action (in this case, they all happen to be in the Calculator window). The "Control" column names the type of control the mouse is clicking. The "Action" column lists the action taken with the mouse (or by the tester). And the "Arguments" column names a specific control (1, 2, 3, 5, +, -, and so on).
WindowControlActionArguments
CalculatorMenuView, Standard
CalculatorPushbuttonClick1
CalculatorPushbuttonClick+
CalculatorPushbuttonClick3
CalculatorPushbuttonClick=
CalculatorVerify Result4
CalculatorClear
CalculatorPushbuttonClick6
CalculatorPushbuttonClick-
CalculatorPushbuttonClick3
CalculatorPushbuttonClick=
CalculatorVerify Result3
This table represents one complete test; more can be made as needed in order to represent a series of tests. Once you've created your data table(s), you simply write a program or a set of scripts that reads in each step, executes the step based on the keyword contained the Action field, performs error checking, and logs any relevant information. This program or set of scripts would look similar to the pseudocode below:
Main Script / Program 
Connect to data tables.
Read in row and parse out values.
Pass values to appropriate functions.
Close connection to data tables.
Menu Module
Set focus to window.
Select the menu pad option.
Return.
Pushbutton Module
Set focus to window.
Push the button based on argument.
Return.
Verify Result Module
Set focus to window.
Get contents from label.
Compare contents with argument value.
Log results.
Return.

From this example you can see that this framework requires very little code to generate many test cases. The data tables are used to generate the individual test cases while the same code is reused. The IBM Rational toolset can be extended using interactive file reads, queries, or datapools, or you can use other tools (freeware, other development tools, and such) along with IBM Rational tools in order to build this type of framework.
Data-driven testing is a framework where test input and output values are read from data files (datapools, ODBC sources, cvs files, Excel files, DAO objects, ADO objects, and such) and are loaded into variables in captured or manually coded scripts. In this framework, variables are used for both input values and output verification values. Navigation through the program, reading of the data files, and logging of test status and information are all coded in the test script.
This is similar to table-driven testing in that the test case is contained in the data file and not in the script; the script is just a "driver," or delivery mechanism, for the data. Unlike in table-driven testing, though, the navigation data isn't contained in the table structure. In data-driven testing, only test data is contained in the data files.
The IBM Rational toolset has native data-driven functionality when using the SQABasic language and the IBM Rational datapool structures. To demonstrate the use of this framework, we'll test the order form from the test sample application Classics A (see Figure 2).

Figure 2. Order form from the sample application Classics A
sample order form
If we record data entry into this window, we get the following:
'Data Driven Framework
'Test Case Script

Sub Main

'Make An Order
Window Set Context, "Name=frmOrder", ""

'Card Number
EditBox Click, "Name=txtCreditCard", "Coords=16,9"
InputKeys "3333444455556666"

'Expiration Date
EditBox Click, "Name=txtExpirationDate", "Coords=6,7"
InputKeys "3333444455556666"

'Place Order
PushButton Click, "Name=cmdOrder"

'Confirmation Screen
Window SetContext, "Name=frmConfirm", ""
PushButton Click, "Name=cmdOK"

End Sub

We can use datapools to set up test cases that test valid and invalid credit card numbers and expiration dates. The datapool shown in Figure 3, for example, would be for a test case that would test the date field.

Figure 3. Sample datapool for a test case that would test the date field 
datapool
If we modify the script to accept this data, we get the following:
'Data Driven Framework
'Test Case Script

'$Include "SQAUTIL.SBH"

Sub Main
Dim Result As Integer
Dim DatapoolHandle As Long
Dim DatapoolReturnValue As Variant

'Open the datapool
DatapoolHandle = SQADatapoolOpen("OrderFormDP")
'...Add error checking....

'Loop through the datapool
While SQADatapoolFetch(DatapoolHandle) = dqaDpSuccess
'Open Order Form
Window SetContext, "Name=frmMain", ""
PushButton Click, "Name=cmdOrder"
Window SetContext, "Name=frmOrder", ""

'Card Number
Result = SQADatapoolValue(DatapoolHanle, "Credit Card Number", DatapoolReturnValue)
"...Add error checking....
EditBox Click, "Name=txtCreditCard", "Coords=16,9"
'...Clear Value....
InputKeys DatapoolReturnValue

'Expiration Date
Result = SQADatapoolValue(DatapoolHandle, "Expiration Date", DatapoolReturnValue)
'...Add error checking....
'...Clear Value...
EditBox Click, "Name=txtExpirationDate", "Coords=6,7"
InputKeys DatapoolReturnValue

'Place Order
Result = SQADatapoolValue(DatapoolHandle, "Order", DatapoolReturnValue)
If UCASE(DatapoolReturnValue) = "YES" Then
PushButton Click, "Name=cmdOrder"

'Confirmation Screen
Window SetContext, "Name=frmConfirm", ""
Pushbutton Click, "Name=cmdOK"
Else
PushButton Click, "Name=cmdCancel"
End If
Wend 'Go fetch next row

'Close datapool
Result = SQADatapoolClose(DatapoolHandle)
'...Add error checking....
End Sub

I had to add SQABasic commands to manipulate the datapools. I also added a While loop to allow for the processing of each row in the datapool. I should also mention the use of the SQABasic command UCase within the If Then statement. UCase is used to make the argument (in this case, the datapool return value) all uppercase. This way the comparisons aren't case sensitive, making the code more robust.
This framework tends to reduce the overall number of scripts you need in order to implement all of your test cases, and it offers the greatest flexibility when it comes to developing workarounds for bugs and performing maintenance. Much like table-driven testing, data-driven testing requires very little code to generate many test cases. This framework is very easy to implement using the IBM Rational toolset, and there's a lot of detailed documentation available with how-tos and examples.
The most commonly implemented framework is a combination of all of the above techniques, pulling from their strengths and trying to mitigate their weaknesses. This hybrid test automation framework is what most frameworks evolve into over time and multiple projects. Figure 4 gives you an idea of how you could combine the approaches of the different frameworks within the IBM Rational toolset.


hybrid framework flow

Monday 2 July 2012

Selenium tutorial/overview


What does selenium do ? 
You can use opensource(ie free!) - Selenium tool ( selenium IDE is a plugin to firefox) to record and playback tests (like WinRunner, QTP). You can then export the recorded test in most language e.g. html, Java , .net , perl , ruby etc. The exported test can be run in any browser and any platform using "selenium remote control". 


Selenium Flash Demo(Highly recommended for beginners) : 
http://wiki.openqa.org/download/attachments/400/Selenium+IDE.swf?version=1
What are the components of selenium ? 

Selenium IDE - Plugin to Firefox to record and play test in firefox
and also export tests in different languages. The most appealing
format is the html test case which seems to based on fit html .
Selenium RC- Allows playing of exported test in different platform/OS
Selenium Grid - Allows to control lots of selenium machines(you typically dont need this only for load test -and hence may ignore it)..


Other Project Management/wrapper tools around selenium:

Bromine(wrapper for selenium tests): 
http://bromine.openqa.org/ 
NOTE: The above tool seemed hard to install - besides looking primitive when I last checked - and did not support "HTML test case"
How does Selenium remote control internally works ? 
Selenium RC launches the browser with itself as the proxy server (hence you may get certificate warning in some modes) and then it injects javascript - to play the test. This also means it can easily work in ALL browsers/platform - and it can be easily used to test AJAX(unlike professional tools). 
What are the different modes that Selenium RC uses ? 
With latest release of Selenium RC(1.0 beta 2) there
are two modes for IE *iexplore(same as *iehta) and *iexploreproxy.
Similarly for Firefox use *firefox(same as *chrome) and *firefoxproxy.
You should prefere *iexplore(ie *iehta) and *firefox (ie *chrome)
respectively as they work with cross domain.

Who should use it ? 


Developers can use it - for “browser” regression testing ( and replace htmlunit/httpunit in some cases) .
Per the one of the forces behind selenium(Neal ford) - it should really be used by Business Analyst first .
QA should enhance/use it do regression test/cross browsers testing on all platforms .

Selenium IDE
In below screen shot - you can see that you need to do recording in main firefox window.
You can use selenium IDE to type any additional command that you may have missed(e.g. timeout)
In selenium IDE - you can open Selenium IDE Options - to change default timeout or
to add you own user extension to extend selenium.
selenium/
Pre Requisite :
Install Firefox 
Install Java jdk: Download jdk (not JRE) without Netbeans IDE or JEE. I am using jdk1.5(but it should work with jdk1.6)
Install (optional) Junit for java rc client(ONLY if you are using java client, you also need to add selenium java client driver in classpath)
Main TOOLS Installation:
Install Selenium IDE 
After installation - you should see selenium ide option in firefox.
Install Selenium RC 
After installation - go to directory containing file 'selenium-server.jar'-
you will need to add appropriate bat/sh script as mentioned in
below steps.

Step a) 
Record your test case using Selenium IDE as shown in earlier flash demo.
For intial recording- avoid using "cross domain" testing (eg www.abc.com, www.def.com) and
preferably use http site only.(till you figure out the workarounds)
eg You can use my test case that opens this tutorial(it searches for selenium tutorial
on google and opens this tutorial):








New Test

































































New Test
open /
type q selenium tutorial
clickAndWait btnG
clickAndWait link=Selenium Tutorial for Beginner/Tips for Experts









Step b) Play your test case using selenium IDE itself to make sure
that your test case works..
 Modify the recorded code if necessary(eg replace click by clickAndWait 
or change timeout) as explained in Quirks section.
Step c) Export your recording test case - as html file 
and/or optionally in programming languages like java, perl etc

Do I Export in HTML or do I export in programming language like java client ?
If you need dynamic capability (e.g. reading data from csv file or database) then
you have to export it in programming language like java.
You would also need to export in java - if your page has lots of flash
components. e.g.
http://www.adobe.com/devnet/flash/articles/flash_selenium_04.html
There is also attempt to make silverlight work with java and .net out here.
The client code using java also is more stable (less permission denied errors) then
the html case. 
But the html is easy to modify/maintain and it generates nice results - whereas in
java client code it tends to be cryptic(but again html test case - does not seem to be flexible).
I personally prefer the html option for the long term(if i don't need dynamic capability)
Step d) Create a test suite
After you saved your html test case (eg save as a.html) - you need to create
a "Test Suite" (eg suite.html) - as you can NOT play test case directly.
Test suite is just a collection of test case which are defined/ordered using html table tags .
If you are adding another test case in the same test suite, then - initially - try to keep it to the 
same - base URL(ie starting point)
- eg below
Suite.html



Overall Tests
TEST CASE A

Note: 
1) The first row (e.g Overall Tests) in your suite is always considered as Header 
2) Selnium RC does not support a TestSuite where you go across domains
(ie the only domain that you would be allowed is for e.g. www.abc.com) 
You may be able to work around it with *chrome for firefox and *iehta for IE.
Continue Next step?
If you are happy in running you test suite manually in Firefox only with Selenium
IDE - then you can skip below steps.(Its also the most stable - with no cross domain/security
warnings)
But If you wish to run you test suite in IE or if you want to schedule your 
test suite bat file automatically via inbuilt Microsoft Scheduler on XP
or cron on Unix - then follow below steps.
Step e) Run your exported html test case in IE(or firefox) using selenium rc
run it in command line/bat file with below command :
java -jar selenium-server.jar -multiwindow -htmlSuite "*iexplore" "https://www.abc.com" "C:\suite.html" "C:\results.html"
REM Use *chrome for firefox and try *iexploreproxy
You can then see the nice results it generated and you can write a simple script to
email the results to yourself in case of failure(The result html file will contained "failed"
keyword). 
You may also schedule your test suite with default/inbuilt Microsoft Scheduler on XP.
NOTE: 
1) You may use -port option to change ports in above.
2) You may have problems in using *iehta (necessary for removing security warning
and going across domains) with above options..
3)Warning:Its better to use *chrome mode - which is more stable. Else- You are likely
to get weird permission denied error or some script error with this option - or 
maybe it will ask you to download your -.do/.faces page. 
(But note - I don't seem to get similar problem with java client code ie
using the code that was exported to java)
In short firefox is more likely to be stable as compared with IE for selenium.
In rare cases - You may get some lock errors with firefox though.(in
which case you may try to create seperate profile eg http://seleniumhq.org/docs/05_selenium_rc.html#specifying-the-firefox-profile 
or in worst case - reboot machine/reinstall it )
Continue Next step ?
If you want dynamic capability (eg reading from CSV file or reading from database) - then
you may want to export your test case in programming language like java etc.
else skip next section.
Step f) OPTIONAL - Run your exported test case in programming like java .
Once selenium rc is extracted/installed then go to the directory containing selenium-server.jar.
If you want to use selenium-rc using programming language like java/.net/php etc(most cases)
You may want to add a bat/sh script to keep selenium rc always running..
REM set PATH variable to contain location of firefox/IE binary
REM set PATH=%PATH%;C:\firefox/bin
java -jar selenium-server.jar -multiwindow 
REM -interactive is more user friendly - but does not work with websites using non frameable pages
REM - optionally use -port , if you want to run more then 1 instance
Now compile your selenium java client code with junit jar files , use selenium
java client driver, point to above selenium RC server and run it.
Assuming you exported the test case with name "NewTest.java" - you 
can try to use below bat file(after getting the junit*jar and selelenium-java-client.jar - and creating clases directory in same folder)


--------- eg. put below in compileJava.bat ---------------------------- 


REM Get junit*.jar (got from junit.org)

REM Get selenium-java-client-driver.jar

REM (got from selenum rc installation/selenium-java-client-driver* folder)

REM NOTE: classes directory should already exist in current directory



set CLIENT_JAR=.

set CLASSPATH=%CLASSPATH%;.;.\classes;;%CLIENT_JAR%\junit-4.4.jar;%CLIENT_JAR%\selenium-java-client-driver.jar


REM COMPILE(note default package at first line is com.example.tests)

javac -d classes NewTest.java


REM ensure that selenim rc server is already running on current machine

REM Now running the actual test.


java junit.textui.TestRunner com.example.tests.NewTest


REM - remove pause if you are going to schedule it.

pause;




-------------------------------------------------------------------
This implies that java(or programming) code is dependent on 
selenium rc server to be running on same(or other) machine. But the "html test case" - launches the "selenium rc server" every time.(hence you need different port - if you run html test cases in parallel)




TIP: 
By default selenium java client does not give nice reports(seen with html option)
You may use loggingselenium" project to get the nice reports(along with timing).
Also if you use this, it will also embed the screenshot that you called- in the html file. 
Best way is to run/edit- after right click/saving this attached code as the usage on their website is incomplete/confusing. 
It depends on loggingselenium and apache commons jar.
Step g)(OPTIONAL) - Migrating your test case to Selenium 2 
Since Selenium IDE does not support Selenium 2(also note that Selenium 2 does not support HTML test case). 
You may use WebDriverBackedSelenium steps to migrate to Selenium 2 .


package com.example.tests;

import com.thoughtworks.selenium.*;

import org.junit.After;

import org.junit.Before;

import org.junit.Test;

import java.util.regex.Pattern;

//*NEW _ ADDED THIS IMPORT

import org.openqa.selenium.*;

import org.openqa.selenium.ie.*;

import org.openqa.selenium.firefox.*;

import org.openqa.selenium.chrome.*;

public class Selenium2Test extends SeleneseTestCase {

@Before

public void setUp() throws Exception {

//*NEW - Commented below line and added the remaining 2 lines.

//selenium = new DefaultSelenium

//("localhost", 4444, "*chrome", "http://www.bing.com/");

// WebDriver driver = new InternetExplorerDriver();



WebDriver driver = new FirefoxDriver();

selenium = new WebDriverBackedSelenium

(driver, "http://www.bing.com");

//comenting below implies taht the default selenium rc server at port 4444 is running

//else if you uncommment it - then it will automatically launch selenium rc server.

// selenium.start();

}


@Test

public void testUntitled2() throws Exception {

selenium.open("/");

selenium.type("sb_form_q", "selenium tutorial");

selenium.click("sb_form_go");

selenium.waitForPageToLoad("80000");

selenium.click("link=www.jroller.com");

selenium.waitForPageToLoad("80000");

}

@After

public void tearDown() throws Exception {

selenium.stop();

}

}



Warning: all cases may not be supported and even though selenium 2 is faster due to native support in browser - certain methods in WebDriverBackedSelenium 
will take more time. 
Hence it may be better to write clean Selenium 2 tests as shown at :http://seleniumhq.org/docs/03_webdriver.html 
)

You can choose to export the test in JUnit(WebDriver) in Selenium IDE 
- and then modify it to use PageFactory pattern.
You can get overview of internal Selenium (Webdriver) architecture at :
http://www.aosabook.org/en/selenium.html 
You may download all the above code/scripts here.
  • Do check the reference manual here to get concepts of selenium locators 
    and the various commands that are available. You could get condensed version of the manual here. 

    Without reading the manual - You may easily miss useful commands like pause(to test session timeout), storeHtmlSource (you can potentially apply selenium regular expression to verify the values of hidden variables insides that html) , typeKeys (to do things like page up etc)
  • Assert vs verify: If you use seleniums verify command - then the test suite/case - will still run - and fail at the end. Hence in most cases you should use assert command which will cause the test to stop.
  • Sometimes selenium IDE may use "click" instead of "clickAndWait" for some
    "submit" button cases resulting in your next page's assert/verify to fail. Hence
    you should change it manually or add waitForPageToLoad with some timeout.
  • If the recording is not working - then go to firefox and click on "Tools->Add on -> Selenium IDE " .First disable it and then enable it and restart the browser. Sometimes clicking on Selenium IDEs
    Options->HTML Format helps.
  • If web site does not allow itself to be frameable then use -multiwidow option(instead of -interactive) with selenium rc.
  • To eliminate certificate warning(and work cross domains ) use chrome/iehta modes -
    which is now default(as compared with rc beta1 version)
    http://selenium-rc.openqa.org/experimental.html

  • If you check selenium rc options eg "java -jar selenium-server.jar -h"
    then you will find that it has a "timeout" option. But that timeout is for the overall
    run(and not for individual "clickAndWait" ) . Hence you may be advised to use "setTimeOut" via your 
    java program or in case of HTML use -Selenium IDE open test case-> Add Insert new command
    -> setTimeout with value(not Target) of say 240000 (ms)

  • Selneium has this cool - captureEntirePageScreenShot command which works
    well with IDE/HTML . Consider using it at critical points(to debug
    issues, if you schedule your test cases).
    But with programming language like java - it may not work(try
    using the experimental - captureScreenshot .
  • Consider writing a wrapper for your TestCase/TestSuite in .net/jsp
    so that you can maintain your testcase/test suite in HTML.
  • Sometimes the browsers or the java process may not get killed after the test is run.. So you may need to manually
    kill it(you can try to use some automated tools like Pskill on Windows ).
  • When your client java test (using RC) finishes with error - then it does not give clear error - 
    you just the line number of the failed assert in stack trace - Hence you may need to trap all exceptions and - print the last body test
  • If you are testing with page where the id dynamically changes - then You may need to use lots of xpath in your test . In that case install firebug and use "Inspect element" by right clicking on the element -> Copy XPath by right click on bottom-> Add "/" in front of value you got in your clipboard-> paste this as a "target" in Selenium IDE for your "clickAndWait" command. You can check the validility by - click on find button in Selenium IDE to ensure that your XPath is correct(assuming that page is open) . You may find more tip on xpath using firebug/xpather here
  • If you are using Selenium HTML test case - then one way to remove hard coded values
    in your selenium test is to have simple JSP(or asp/php) that converts CSV file to HTML file.
    Then you just need to invoke that JSP page (for particular CSV) - and call various "store" values 
    and replace hard coded values in your test with "get" values . 
    That way you can just change your test/data by just replacing the CSV file.
  • Do use (minor) javascript in HTML test case. You would need javascript to access stored values anyway.
    eg store 1 abc (store on one page)
    , type id javascript{storedVars['abc'].toUpperCase()} (extract or evaluate it on another page)
    However if ther are too much javascripts - consider use programming languate.
  • The latest selenium rc(1.0 beta 2) does support Firefox 3.. 
    With firefox 3 you can easily add your local https server with invalid certificate in safe list(to remove warning message). 
    Its difficult to do that with Firefox 2.
  • If you are running selenium rc test with IE a site which already has invalid certificate -
    then IE will give a warning for "invalid certificate". You can have selenium
    automatically click on "Continue" link in it(which has ID of overridelink) by adding 
    below command in your IE specific test case.

    clickAndWait
    //a[@id='overridelink']

    With firefox 3 - you can permanently store the exception. However selenium -rc starts
    new firefox profile every time(probably to get around lock issues) - hence you may 
    need to start firefox with same/particular profile.


    You may also consider changing IE options to be strict (eg Tool->Internet Options->Advanced->Display
    Notification on every script error) to have selenium catch any javascript issues/bugs in your website.

  • Consider use of UI-Element - which is now supported in RC here
  • There are other good tips here.
  • if you read above FAQ - you will realize selenium has limitation in IE with certain dialog box(eg upload file, basic auth). You can get around some of the limitation with autoit script
    from this site.
    Its advisable not to directly download the exe from above site - but rather install autoit, compile autoit script - from one of it program menus options - to Exe and than use the compiled exe.
    For whatever reasons (firefox does not expose its model)- the above scripts(or autoit) does not help in Firefox dialog boxes(but you dont need it anyway as in chrome mode - selenium automatically gets rids of dialog boxes)
    However if you are trying to get rid of basic authentication dialog - you can directly pass user/pass in URL.(however for IE you would need to add some registery settings)
    PS: Autoit and autohotkey seem to have same codebase - and both have primitive recorder.
    (else we probably wont have needed selenium)
    But in autoit vs autohotkey - I will give slight edge to autoit.
    NOTE: On windows - you should use 'start' in front of exe/.au3 file to make it run in background -
    and then run your selenium rc test in the same batch file(eg start autotItscript.au3) 

There is nice article which links to - google download code - to extend selenium to
support loop(while) and goto..
http://51elliot.blogspot.com/2008/02/selenium-ide-goto.html 
You should though export your test case in programming language - if you need
this kind of facility.(or see if you can wrap the html in JSP/asp .net)
You can get a list of all GOOD extension(eg UI-Element) at:
http://wiki.openqa.org/display/SIDE/Contributed+Extensions+and+Formats
Selenium is NOT designed for load test. However - some companies
can try to use Selenium Grid for low load testing or even give 
it to some external companies - who will test your selenium test - with lots
of browsers.
There is also interesting attempt by "PushtoTest" to convert selenium
test to low level - "htmlunit" test - and hence make it suitable
for load testing. I don't think its stable yet - but please let me know if i am wrong.
eg
http://www.pushtotest.com/products/selenium-how-to/
You may have noted that selenium requires lots of workarounds - as it uses javascript (which are restricted by browsers) to drive the tests.
However in future selenium(2.0) - will merge with webdriver - allowing selenium tests to
easily run even without browser(and hence suitable for load test). Selenium 2 uses
native plugin to browser to run its tests - and hence does not faces the javascript limitations.
Disadvantage of Selenium 2:
It may not be easy for selenium 2 to support newer browser (eg androids) or newer IE versions- as they are not going
to provide native browser plugin for all browser(or even make it consistent).
As of now (may 2010)- I dont see any support for IDE(i expect this to happen eventually) nor is 
there any talk of supporting HTML Test case(not sure if they would ever support this). (in fact I was hoping that
would support JSP natively - which should be easy for selenium rc)
If you are using "exported" java client code with selenium RC,
then the API for two main class are:
(There is facility for screen shot - but it appears to be protected )
b) 

Selenium vs Low/Network level tools like Webtest/HttpUnit/HtmlUnit(java)

If you are using a "low/network" level tool like htmlunit then it
has the advantage of being fast - but its javascript will 
not work work in most complex websites(besides not being a browser test)
Selenium(java, .net) code also has the disadvantage of being dependent on RC server(not big deal).
You should continue low level tools like httpunit etc to use 
it for monitoring - but don't use those tools for browser/regression testing.
Selenium vs Watir 
If you like your test cases in Ruby (or maybe .net) and are willing to
dig through "hard to understand" documentation - then Watir is a tool 
for you.(if you are perl/php/java shop - don't even bother considering Watir
for now)
Watir does have advantage that its recorder uses IE to record.
With Selenium - if your site does not work with Firefox - you will
have to manually write the code(not big deal)..
But the biggest advantage of Selenium is its support for FIT HTML
and the vast languages/browsers/platform that it supports.
(with good documentation)
But maybe things may change in future.
Selenium vs Imacros 
Imacros appeared better in recording. But overall its not as powerful for automation(or scheduling).
Selenium vs Dejaclick 
Dejaclick is probably the best (free) macro tool for websites record and play. However Selenium beats DejaClick in overall features(multiple language suport vs only XML support, free automation vs paid automation).
Dejaclick Advantage:
It nicely plays back the recorded stuff - by highlighting appropriate items in green.
It can easily record flash/silverlight - by recording position.(however it may mean that it may not
nicely work in all computers)
It has facility to encrypt sensitive data in your file.
Dejaclick easily allow you to bookmark your script. So that you can play it back in one click.
You can easily toggle Dejaclick to True Screen mode(ie recording by mouse positions) by pressing Alt + \
(some stuff are hard to record by name/ids)
Disadvantage:
Dejaclick recording tool is free - But for automation you have to rely on paid site alertsite (AlertSite however provides great facility for running your dejaclick script and even soap xml from most of cities/region in US/world. This will allow you to monitor your CDN performance/reliability or what content your website is serving across different regions)
About me/Other blogs(unrelated to selenium):
If you want to use other cool firefox plugins(eg yslow), 
or use olap(pentaho) in java(eg seam) then you may check my main bloghttp://www.jroller.com/zahid .
I have worked in past for clients like Hitachi(PCS), Bank One/Chase, Bill Me Later(Kaloke)
and Verizon. I have designed/coded in various systems using languages like MacApp
(Mac C++), VC++ and Java. 
All of my blogs are my personal views - and they dont represent the views of 
any companies that I have worked in past/present. 
I am currently trying to develop blog on choosing Android phone/tips/top apps/app development