Skip to main content

Selenium-ANT-TestNG (SAT) Framework

Posted by aditya_dada on May 20, 2008 at 10:39 AM PDT


Architecture

This framework is created using ANT, TestNG and Selenium.
Through this framework, a user is able to create an automated test case which can be run later by executing a single command. The uses of different frameworks are as follows:
Selenium: This framework is needed for recording a testcase, exporting it into Java, and executing it on a browser
TestNG: This framework is used to organize different tests, and to report results.
ANT: provides the glue to hold everything together, and execute basic commands.

Pre-requisites:


Basic knowledge of Selenium IDE, Selenium RC, TestNG and ANT.

Setup:


1. Install Selenium IDE on your Firefox browser.
2. Make sure you have ANT installed.
3. Grab the testws.zip file from here. Unzip it into a directory of your choice.

About the sample
There are 2 bundled samples:
1. AdminLoginTest.java: This test logs into the glassfish admin console using the username/password: admin/administrator. It then clicks on the 'Enterprise Applications' link, and tests to see if there are no applications deployed. Else, test will fail.
2. SimpleTest.java: This test accesses http://localhost:8080/ page, and tests whether the term "Glassfish Project" appears or not.

The config directory contains testng.xml file, and is a place holder for configuration files.
The lib directory contains required jar files, and is a place holder for other library jars/zips.
The test directory contains test java files and is a place holder for other test files.
Once the tests are run, build and test-output directories will be created.
The build directory contains compiled classes, and is a volatile directory that is deleted by the target clean in build.xml file.
The test-output directory contains result files, and is generated by TestNG.
Notes:
To run the bundled sample, follow the steps for "Execute your test through ANT" (see below).
Currently, I face some issues running this on IE. I have successfully been able to run tests using Firefox.

How To:


Write a test case
The basic structure that one has to follow to create a test case is:
1. Record a testcase using Selenium IDE (Firefox browser only), Export the test case into Java
2. Edit the test case – java file, and add TestNG annotations
3. Execute the test case using ANT

1. Record test using selenium IDE and export it into Java:
i. Open the website you’d like to test in Firefox browser
ii. Start the selenium IDE from your Firefox browser
iii. Ensure that the Base URL in Firefox is the same as the website you want to test
iv. Start clicking on the website. Selenium will record your actions.
v. When finished, do: File -> Export Test As... -> Java – Selenium RC and save the java file.
seleniumIDE-export.JPG
After exporting, your java test case should look like this:
selenium-java-test.JPG

2. Add TestNG to your test case:
i. Open the java test file you just created in an editor
ii. Add the following:
(a) to the import statements:
import org.testng.annotations.*;
(b) To variable declarations:

private DefaultSelenium selenium;
private static String TIMEOUT_PERIOD ="10000";

(c) To The class:

@BeforeSuite(alwaysRun = true)
public void setUp(String browser) {
selenium = new DefaultSelenium("localhost", 4444, "*firefox", "http://localhost:8080");
selenium.start();
selenium.open("http://localhost:8080");
}

@AfterSuite(alwaysRun = true)
private void stopTest() {
selenium.stop();
}

(d) After each statement in the test that will take time for a page to load:

selenium.waitForCondition("selenium.browserbot.isNewPageLoaded()", TIMEOUT_PERIOD);

(e) On top of the test method:

@Test(groups = { "<your test group name>" })
e.g. @Test(groups = { "roller" })

(f) Edit page to be opened. Make sure arg to ‘open’ method of selenium points to the right context root e.g. If http://localhost:8080/<context_root> is the correct URL to test, then since we already have tried to open localhost:8080 in setUp, we only need to open the context root page.
e.g.
selenium.open("<context_root>");

iii. Save the test file
(a). (Optional) Update testng.xml file. TestNG uses this file to read which groups to execute and also to pass parameters to the test. This file is only required if build.xml file uses testng.xml file to run the test.
Note: in sample test included, testng.xml is used, and will need to be updated.

iv. Ensure that the test file you created is mentioned in the build.xml compile target.

3. Execute your test through ANT
i. Start domain required by your webpage. e.g. asadmin start-domain
ii. Start database if needed (Not needed for bundled sample test)
iii. Start selenium server e.g. cd testws/lib, java -jar selenium-server.jar
iv. Execute the command: “ant run” from ‘testws’. You should see a new firefox browser window open up, and run through your test.
v. Check reports generated by TestNG under ‘testws/test-output’ directory. Open the index.html to see different output from the test.

SAT-html-output.JPG
Console output:

SAT-cmd-output.JPG

Related Topics >>

Comments

&nbsp;Hi Aditya, Can you please explain the procedure for ...

Hi Aditya,

Can you please explain the procedure for the same using eclipseIDE.

Thanks much.

Selenium-ANT-TestNG (SAT)

How can we run test cases in parallel or sequence with ant run command ??
And how to create test suites like when run 1 method it should execute a group of test cases like follows :
public void testSuite1() throws Exception {
testLogin();
testTest1();
testTest2();
testTest3();
}
thanks in advance .

Selenium-ANT-TestNG (SAT)

Hi aditya,
I tried installing your demo test setup on Ubuntu . I went half way successfully but failing while doing ant run .
It is showing Build Successfull but Configuration Failures: 2, Skips: 0
when open index.html results shows :

java.lang.IllegalArgumentException: wrong number of arguments
... Removed 15 stack frames
java.lang.IllegalArgumentException: wrong number of arguments
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.testng.internal.MethodHelper.invokeMethod(MethodHelper.java:645)
at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:385)
at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:141)
at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:79)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:239)
at org.testng.SuiteRunner.run(SuiteRunner.java:168)
at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:987)
at org.testng.TestNG.runSuitesLocally(TestNG.java:951)
at org.testng.TestNG.run(TestNG.java:719)
at org.testng.TestNG.privateMain(TestNG.java:1019)
at org.testng.TestNG.main(TestNG.java:997)

Could you please tell me where i'm going wrong.
Thanks in advance.

Code not working.

Hi Aditya, I have followed all the steps you have told in the blog, but the "testng.xml" file is not able to recognize the testSuite and gives the error: "No test suite found. Nothing to run java.io.FileNotFoundException: test-output/testng.css (No such file or directory)". Can you please tell me where is the mistake. Following is the testng.xml code.

Aditya, I'd just like to say this rocks, the set up was easy and all my tests ran the first time. Kudos

Hi, I tried executing my project after I did the setup, but i have got some. Issue: The script was trying to open the local host using 4848 port ans it was not available. Can you please help in this?

When I added the sequential attribute after each of the @Test method lines, the test runs in Random order. Only way to get the test to say that it is running sequentially is by adding the attribute before the class level, like... @Test(sequential = true) public class Main extends SeleneseTestCase { ... (@BeforeSuite and @AfterSuite ... @Test(groups = {"Test_1" }) public void MainTest_1() throws Exception { Selenium Tests; } @Test(groups = {"Test_1" }) public void MainTest_2() throws Exception { Selenium Tests; } @Test(groups = {"Test_1" }) public void MainTest_3() throws Exception { Selenium Tests; } } When I run this, it reports [testng] [TestRunner] WILL BE RUN IN RANDOM ORDER: [testng] [TestRunner] WILL BE RUN SEQUENTIALLY: [testng] [TestRunner] com.example.tests.main.MainTest_1() [testng] [TestRunner] com.example.tests.main.MainTest_3() [testng] [TestRunner] com.example.tests.main.MainTest_2() [testng] [TestRunner] ==== Obviously, those are not in the sequential order.

Correct the typo @Test(groups = {'Test_1"} , sequential = true) public void MainTest_1() throws Exception { Selenium Tests; @Test(groups = {'MainTest_2"} , sequential = true) for each test class And, testng.xml file <suite .......> <test name="MainTest_1"> <groups> <run> <include name="Test_1"/> </run> </group> <classes> <class name="com.......Test_1/> </classes> </test name> <test name=....> </test name> </suite>

Try this @Test(groups = {'Test_1" , sequential = true) public void MainTest_1() throws Exception { Selenium Tests; @Test(groups = {'MainTest_2" , sequential = true) ........ for each test class And, testng.xml file .......................... ........................

In the TestNG documentation, I found that the sequential attribute can only be used in the class level and will be ignored if used at the method level. Once I moved it to the class level, it's working as desired. Thanks for the suggestion.

Would the sequential attribute be defined in all of the @Test instances as below? @AfterSuite(alwaysRun = true) private void stopTest() { selenium.stop(); } @Test(sequential = true) public void MainTest_1() throws Exception { Selenium Tests; } @Test(sequential = true) public void MainTest_2() throws Exception { Selenium Tests; } @Test(sequential = true) public void MainTest_3() throws Exception { Selenium Tests; } It compiles and runs but still, not sequentially. Thought it may be the testNG version, so I tried using 5.8 with the same results.

Yes, you can use the sequential attribute in the annotation. @Test (sequential=true)

Nice blog. I'm trying to add multiple tests to the main testcase. They all ran successfully, however, they ran in random order. Would anyone know how to get the tests to run sequentially? [testng] [TestRunner] WILL BE RUN IN RANDOM ORDER: [testng] [TestRunner] com.example.tests.main.MainTest_5() [testng] [TestRunner] com.example.tests.main.MainTest_3() [testng] [TestRunner] com.example.tests.main.MainTest_2() [testng] [TestRunner] com.example.tests.main.MainTest_1() [testng] [TestRunner] com.example.tests.main.MainTest_4() [testng] [TestRunner] WILL BE RUN SEQUENTIALLY: [testng] [TestRunner] === [testng] [TestRunner] Found 5 applicable methods [testng] [Invoker 8568863] Invoking com.example.tests.main.MainTest_5 [testng] [Invoker 8568863] Invoking com.example.tests.main.MainTest_3 [testng] [Invoker 8568863] Invoking com.example.tests.main.MainTest_2 [testng] [Invoker 8568863] Invoking com.example.tests.main.MainTest_1 [testng] [Invoker 8568863] Invoking com.example.tests.main.MainTest_4 Thanks

Hi i have tried to create a frame work following Aditya's steps. e) On top of the test method: @Test(groups = { "" }) e.g. @Test(groups = { "roller" }) My script is below and it gives error compile: [echo] making directory... [mkdir] Created dir: C:\Framework\testws\build [echo] classpath: C:\Framework\testws\lib\junit.jar:C:\Framework\testws\lib \selenium-java-client-driver-tests.jar:C:\Framework\testws\lib\selenium-java-cli ent-driver.jar:C:\Framework\testws\lib\selenium-server-0.9.1-20070223.200626-116 -standalone.jar:C:\Framework\testws\lib\selenium-server.jar:C:\Framework\testws\ lib\testng-5.5-jdk15.jar:C:\Framework\testws\lib\testng.jar [echo] compiling... [javac] Compiling 1 source file to C:\Framework\testws\build [javac] C:\Framework\testws\test\UsersFilter.java:17: expected [javac] usersPage(); [javac] ^ [javac] 1 error BUILD FAILED C:\Framework\testws\build.xml:68: Compile failed; see the compiler error output for details. import com.thoughtworks.selenium.*; import org.testng.annotations.*; import junit.framework.*; import java.util.*; import java.lang.*; @Test(groups = { "UserFilter" }) public class UsersFilter extends SeleneseTestCase { public static Selenium browser; public static void main(String args[]) { browser = new DefaultSelenium("localhost", 4444, "*chrome", "http://192.168.2.63/aveksa/main"); browser.start(); @Test(groups = { "UserFilter" }) usersPage(); } public static void usersPage() { //Logging in to the application browser.open("http://192.168.2.63/aveksa/main"); browser.type("Name1", "AveksaAdmin"); browser.type("Pass1", "Aveksa123"); browser.click("//span[@id='login']/span"); System.out.println("Login successful"); browser.waitForPageToLoad("30000"); assertTrue(browser.isElementPresent("UsersTab")); browser.click("UsersTab"); browser.waitForPageToLoad("30000"); System.out.println("Opened Users Tab"); //Checking the no. of records under Users tab assertTrue(browser.isTextPresent("Rows 1 - 10 of 75")); System.out.println("Rows 1 - 10 of 75 displayed"); assertTrue(browser.isElementPresent("//img[@title='Filter Data']")); /*assertTrue(browser.isElementPresent("//img[@title='Table Columns']")); browser.click("//img[@title='Table Columns']"); browser.waitForPopUp("ColumnsDialogData", "30000"); browser.selectWindow("name=DialogIframe"); assertTrue(browser.isElementPresent("//fieldset[@id='ColsFieldset']/div")); browser.open("/aveksa/main?ReqType=Table&PageID=ColumnsDialogData&Action=Content&TableID=UsersSummaryTable"); browser.click("col_department"); browser.click("col_supervisorName"); browser.click("col_title"); browser.click("col_jobStatus"); browser.click("col_CUS_ATTR_USER_CAS_2"); browser.click("col_CUS_ATTR_USER_CAS_3"); browser.click("col_CUS_ATTR_USER_CAS_4"); browser.click("col_UNIQUE_ID"); browser.selectFrame("relative=up"); browser.click("//table[@id='tbl']/tbody/tr[6]/td/span[1]/span/span"); browser.waitForPageToLoad("30000"); browser.open("/aveksa/main?ReqType=GetPage&PageID=UserSummaryPage");*/ // Filtering for "last name = AveksaAdmin" browser.click("//img[@title='Filter Data']"); browser.waitForPopUp("FilterDialogData", "50000"); assertTrue(browser.isElementPresent("//form[@id='MainForm']/table/tbody/tr/td")); browser.selectWindow("FilterDialogData"); System.out.println("Filter options displayed"); assertTrue(browser.isElementPresent("lastName_OPERATOR")); browser.select("lastName_OPERATOR", "value=EQUAL"); browser.type("lastName_VALUE", "AveksaAdmin"); browser.selectFrame("relative=up"); System.out.println("Filtering for Last Name value=AveksaAdmin"); browser.click("//table[@id='tbl']/tbody/tr[6]/td/span[1]/span/span"); // Refreshing Users tab to view the filtered data browser.selectWindow("Aveksa - Unified Users"); browser.windowFocus(); assertTrue(browser.isElementPresent("UsersTab")); browser.click("UsersTab"); browser.waitForPageToLoad("50000"); assertTrue(browser.isTextPresent("Rows 1 - 1 of 1")); System.out.println("Filter successful"+ "\n" +"Rows 1 - 1 of 1 displayed"); // Removing the previous filter assertTrue(browser.isElementPresent("//img[@title='Filter Data']")); browser.click("//img[@title='Filter Data']"); browser.waitForPopUp("FilterDialogData", "50000"); assertTrue(browser.isElementPresent("//form[@id='MainForm']/table/tbody/tr/td")); browser.selectWindow("FilterDialogData"); System.out.println("Filter options displayed again"); for (int second = 0;; second++) { if (second >= 60) fail("timeout"); try { if (browser.isElementPresent("lastName_OPERATOR")) break; } catch (Exception e) {} //Thread.sleep(1000); } assertTrue(browser.isElementPresent("lastName_OPERATOR")); browser.select("lastName_OPERATOR", "value=nullOp"); browser.selectFrame("relative=up"); System.out.println("Removed filter"); browser.click("//table[@id='tbl']/tbody/tr[6]/td/span[1]/span/span"); // Refreshing users tab again to view unfiltered data browser.selectWindow("Aveksa - Unified Users"); browser.windowFocus(); for (int second = 0;; second++) { if (second >= 60) fail("timeout"); try { if (browser.isTextPresent("Rows 1 - 10 of 75")) break; } catch (Exception e) {} // Thread.sleep(1000); } //assertTrue(browser.isElementPresent("UsersTab")); //browser.click("UsersTab"); //browser.waitForPageToLoad("50000"); //browser.refresh(); assertTrue(browser.isTextPresent("Rows 1 - 10 of 75")); System.out.println("Filter removed"+ "\n" +"Rows 1 - 10 of 75 displayed"); // Filtering for "last name contains tony" assertTrue(browser.isElementPresent("//img[@title='Filter Data']")); browser.click("//img[@title='Filter Data']"); browser.waitForPopUp("FilterDialogData", "50000"); assertTrue(browser.isElementPresent("//form[@id='MainForm']/table/tbody/tr/td")); browser.selectWindow("FilterDialogData"); System.out.println("Filter options displayed again"); assertTrue(browser.isElementPresent("lastName_OPERATOR")); browser.select("lastName_OPERATOR", "label=contains"); browser.type("lastName_VALUE", "tony"); browser.selectFrame("relative=up"); System.out.println("Filtering for Last Name contains 'tony'"); for (int second = 0;; second++) { if (second >= 60) fail("timeout"); try { if (browser.isElementPresent("//table[@id='tbl']/tbody/tr[6]/td/span[1]/span/span")) break; } catch (Exception e) {} //Thread.sleep(1000); } browser.click("//table[@id='tbl']/tbody/tr[6]/td/span[1]/span/span"); // Refreshing Users tab to view the filtered data browser.selectWindow("Aveksa - Unified Users"); browser.windowFocus(); assertTrue(browser.isElementPresent("UsersTab")); browser.click("UsersTab"); browser.refresh(); browser.waitForPageToLoad("50000"); assertTrue(browser.isTextPresent("Rows 1 - 2 of 2")); System.out.println("Filter successful"+ "\n" +"Rows 1 -2 of 2 displayed"); // Removing the previous filter assertTrue(browser.isElementPresent("//img[@title='Filter Data']")); browser.click("//img[@title='Filter Data']"); browser.waitForPopUp("FilterDialogData", "50000"); assertTrue(browser.isElementPresent("//form[@id='MainForm']/table/tbody/tr/td")); browser.selectWindow("FilterDialogData"); System.out.println("Filter options displayed again"); assertTrue(browser.isElementPresent("lastName_OPERATOR")); browser.select("lastName_OPERATOR", "value=nullOp"); browser.selectFrame("relative=up"); System.out.println("Removed filter"); browser.click("//table[@id='tbl']/tbody/tr[6]/td/span[1]/span/span"); // Refreshing users tab again to view unfiltered data browser.selectWindow("Aveksa - Unified Users"); browser.windowFocus(); for (int second = 0;; second++) { if (second >= 60) fail("timeout"); try { if (browser.isTextPresent("Rows 1 - 10 of 75")) break; } catch (Exception e) {} // Thread.sleep(1000); } //assertTrue(browser.isElementPresent("UsersTab")); //browser.click("UsersTab"); //browser.waitForPageToLoad("50000"); //browser.refresh(); assertTrue(browser.isTextPresent("Rows 1 - 10 of 75")); System.out.println("Filter removed"+ "\n" +"Rows 1 - 10 of 75 displayed"); assertTrue(browser.isElementPresent("//img[@title='Logout']")); browser.click("//img[@title='Logout']"); System.out.println("Logout"); browser.waitForFrameToLoad("Aveksa - Logout", "30000"); System.out.println("Popup displayed"); browser.selectFrame("PopupIframe"); System.out.println("Select popup"); //assertTrue(browser.isElementPresent("//table[@id='tbl']/tbody/tr[6]/td/span[1]/span/span")); browser.click("//table[@id='tbl']/tbody/tr[6]/td/span[1]/span/span"); System.out.println("Confirm Logout"); browser.waitForPageToLoad("30000"); } public void tearDown() { browser.stop(); } } Appreciate you help. Thanks Niju

Yes, the selenium server can be started through ANT script. Here's the snippet that will start the server: ... <target name="start-selenium-server"> < java jar="${ws.home}/lib/selenium-server.jar" fork="true"/> </target> ... The current issue is that the command will hang after it starts server. so you 'll have to do the rest of the steps in another window. or use 'start' command on windows to start selenium in the background. e.g. <target name="start-selenium-server"> <exec executable="start" failonerror="false" dir="testws/lib"> <arg line="java -jar selenium-server.jar"/> </exec> </target>

The blog was informative.Can you paste your build.xml used by ant script? Can we start the selenium server through ant script itself?

Hi, I have problem : 1. When i try to run selenium test and the host target is http://localhost:8080/, my browser (opera 9.5) hang, and do nothing after initial new window. Command on console to start selenium server: java -jar selenium-server.jar -multiwindow. Everything is ok until process try to create new DefaultSelenium(......) 2. I try to change browser to Firefox 3, and everything is stop for a long time, when selenium try to prepare profile for firefox. Thanks. Regards, Sai