SELENIUM clickandWait or Timeout or else

This support forum board is for support questions relating to Nagios XI, our flagship commercial network monitoring solution.
dlukinski
Posts: 1130
Joined: Tue Oct 06, 2015 9:42 am

SELENIUM clickandWait or Timeout or else

Post by dlukinski »

Hello Nagios support forum

I am very new to SELENIUM IDE and ran into question many other did too: waits or timeouts while pages are loading or redirecting.

Is there a wait command (perl) to use while waiting for redirects?

use strict;
use warnings;
use Time::HiRes qw(sleep);
use Test::WWW::Selenium;
use Test::More "no_plan";
use Test::Exception;

my $sel = Test::WWW::Selenium->new( host => "localhost",
port => 4444,
browser => "*chrome",
browser_url => "https://login.microsoftonline.com/" );

$sel->open_ok("/login.srf?wa=wsignin1.0&rpsnv=4&ct=1466534348&rver=6.7.6640.0&wp=MCMBI&wreply=https%3a%2f%2fportal.office.com%2flanding.as
px%3ftarget%3d%252fdefault.aspx&lc=1033&id=501392&msafed=0&client-request-id=64b6eb2d-3b6f-4bcb-8138-08f686ad8ebd");
$sel->type_ok("id=cred_userid_inputtext", "nagximon\@mydomain.com");
!!! ----- at this point redirect happens, opening another page to enter password ----- !!! / SELENIUM proceeds to the next step right away and fails
$sel->type_ok("id=passwordInput", "mypassword");
$sel->click_ok("id=submitButton");
$sel->wait_for_page_to_load_ok("30000");
$sel->click_ok("xpath=(//button[\@type='button'])[10]");
$sel->click_ok("//a[\@id='O365_SubLink_ShellSignout']/div/span[2]");
$sel->wait_for_page_to_load_ok("30000");
$sel->click_ok("id=submitButton");
$sel->wait_for_page_to_load_ok("30000");
User avatar
mcapra
Posts: 3739
Joined: Thu May 05, 2016 3:54 pm

Re: SELENIUM clickandWait or Timeout or else

Post by mcapra »

dlukinski wrote: Is there a wait command (perl) to use while waiting for redirects?
sleep(int);
http://perldoc.perl.org/functions/sleep.html

So, for your current test case, you could try the following:

Code: Select all

....
$sel->type_ok("id=cred_userid_inputtext", "nagximon\@mydomain.com");
sleep(10);
$sel->type_ok("id=passwordInput", "mypassword");
....
sleep(10); basically says "wait 10 seconds" in this context.
Former Nagios employee
https://www.mcapra.com/
spurrellian
Posts: 43
Joined: Tue Jan 06, 2015 6:26 am
Location: Bath, UK

Re: SELENIUM clickandWait or Timeout or else

Post by spurrellian »

Have you recorded this using the firefox plugin, then export to perl?

Has the script been run through the clean selenium script process as it doesn't look 100% right to me


Below is a script I use for monitoring our company billing site. Selenium will browse to it, log in, check you can get to the bills section and give you back performance data. There is also a redirect on the first page

Code: Select all

use strict;
use warnings;
use Time::HiRes qw(sleep);
use Test::WWW::Selenium;
use Test::More "no_plan";
use Test::Exception;
use TimePerf2;

my $time= TimePerf2->new(60,90,120);


my $sel = Test::WWW::Selenium->new( host => "wxapwselep01",
                                    port => 4444,
                                    browser => "*iexplore",
                                    browser_url => "http://www.awebaddress.co.uk/" );

$time->startTime("ALL");$sel->set_timeout_ok("120000");
if ( !  $sel->open_ok("") ) {print "Location: " . $sel->get_location() . "\n";}
elsif ( ! $sel->set_speed("1500") ) { }

elsif ( !  $sel->type_ok("id=emailText", "LOGINNAME") ) {print "Location: " . $sel->get_location() . "\n";}

elsif ( !  $sel->type_ok("id=passwordText", "PASSWORd") ) {print "Location: " . $sel->get_location() . "\n";}

elsif ( !  $sel->click_ok("id=signinButton") ) {print "Location: " . $sel->get_location() . "\n";}

elsif ( ! $sel->wait_for_page_to_load_ok("120000") ) {}

elsif ( ! $sel->pause("2000") ) {}

elsif ( !  $sel->click_ok("id=Bills") ) {print "Location: " . $sel->get_location() . "\n";}

elsif ( ! $sel->pause("2000") ) {}

elsif ( ! $sel->wait_for_page_to_load_ok("120000") ) {}

elsif ( !  $sel->location_is("https://www.awebaddress.co.uk/bills.aspx") ) {print "Location: " . $sel->get_location() . "\n";}

elsif ( ! $sel->pause("2000") ) {}

elsif ( !  $sel->click_ok("id=navBar_signOutLink") ) {print "Location: " . $sel->get_location() . "\n";}

elsif ( ! $sel->wait_for_page_to_load_ok("120000") ) {}

elsif ( !$time->endTime("ALL") ) { }

else { $time->getTimes(); }
$sel->stop();

I found the script would fail though if I didn't have a pause during the script

For example the following lines will make selenium wait 2000ms

Code: Select all

$sel->pause("2000")
There are a couple of ways to do it
Paul S - Using Nagios XI, Network Analyzer, Log Server
User avatar
mcapra
Posts: 3739
Joined: Thu May 05, 2016 3:54 pm

Re: SELENIUM clickandWait or Timeout or else

Post by mcapra »

spurrellian wrote: For example the following lines will make selenium wait 2000ms

Code: Select all

$sel->pause("2000");
This is actually the best way to do it, since clean_selenium_script won't always handle sleep() appropriately.
Former Nagios employee
https://www.mcapra.com/
dlukinski
Posts: 1130
Joined: Tue Oct 06, 2015 9:42 am

Re: SELENIUM clickandWait or Timeout or else

Post by dlukinski »

mcapra wrote:
spurrellian wrote: For example the following lines will make selenium wait 2000ms

Code: Select all

$sel->pause("2000");
This is actually the best way to do it, since clean_selenium_script won't always handle sleep() appropriately.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Came to this one too in the past hour. However perl refuses to run it siting compilation errors (around first pause):
- UNKNOWN: syntax error at /usr/local/nagios/libexec/msportal-1 line 25, near "elsif"
Execution of /usr/local/nagios/libexec/msportal-1 aborted due to compilation errors.
1..0

Code: Select all

use strict;
use warnings;
use Time::HiRes qw(sleep);
use Test::WWW::Selenium;
use Test::More "no_plan";
use Test::Exception;
use TimePerf2;

my $time= TimePerf2->new(60,90,120);


my $sel = Test::WWW::Selenium->new( host => "10.96.36.114",.
                                    port => 4444,.
                                    browser => "*firefox",.
                                    browser_url => "https://login.microsoftonline.com/" );

$time->startTime("ALL");
$sel->set_timeout_ok("120000");
if ( !  $sel->open_ok("/login.srf?wa=wsignin1.0&rpsnv=4&ct=1466534348&rver=6.7.6640.0&wp=MCMBI&wreply=https%3a%2f%2fportal.office.com%2flanding.aspx%3ftarget%3d%252fdefault.aspx
elsif ( ! $sel->set_speed("1500") ) { }

elsif ( !  $sel->type_ok("id=cred_userid_inputtext", "nagximon\@mydomain.com") ) {print "Location: " . $sel->get_location() . "\n";}

$sel->pause(10000);
elsif ( !  $sel->type_ok("id=passwordInput", "mypassword") ) {print "Location: " . $sel->get_location() . "\n";}

elsif ( !  $sel->click_ok("id=submitButton") ) {print "Location: " . $sel->get_location() . "\n";}

elsif ( ! $sel->wait_for_page_to_load_ok("120000") ) {}.
$sel->pause(10000);
elsif ( !  $sel->click_ok("xpath=(//button[\@type='button'])[10]") ) {print "Location: " . $sel->get_location() . "\n";}

elsif ( !  $sel->click_ok("//a[\@id='O365_SubLink_ShellSignout']/div/span[2]") ) {print "Location: " . $sel->get_location() . "\n";}

elsif ( ! $sel->wait_for_page_to_load_ok("120000") ) {}.
$sel->pause(10000);
elsif ( !  $sel->click_ok("id=submitButton") ) {print "Location: " . $sel->get_location() . "\n";}

elsif ( ! $sel->wait_for_page_to_load_ok("120000") ) {}.
elsif ( !$time->endTime("ALL") ) { }

else { $time->getTimes(); }
$sel->stop();
User avatar
mcapra
Posts: 3739
Joined: Thu May 05, 2016 3:54 pm

Re: SELENIUM clickandWait or Timeout or else

Post by mcapra »

Yeah, clean_selenium_script is unfortunately not perfect. I'm going to see what can be done internally about revising it.

The issue is that, by creating this big if/elsif chain, you need to make sure there aren't any actions inbetween the elsif statements. The first place your script will run into problems is here:

Code: Select all

....
elsif ( !  $sel->type_ok("id=cred_userid_inputtext", "nagximon\@mydomain.com") ) {print "Location: " . $sel->get_location() . "\n";}

$sel->pause(10000);
elsif ( !  $sel->type_ok("id=passwordInput", "mypassword") ) {print "Location: " . $sel->get_location() . "\n";}
....
Since $sel->pause(10000); is breaking the elsif chain. You could try replacing all of your $sel->pause(10000); lines like so:

Code: Select all

elsif ( !  $sel->pause("10000") ) {}
Former Nagios employee
https://www.mcapra.com/
dlukinski
Posts: 1130
Joined: Tue Oct 06, 2015 9:42 am

Re: SELENIUM clickandWait or Timeout or else

Post by dlukinski »

mcapra wrote:Yeah, clean_selenium_script is unfortunately not perfect. I'm going to see what can be done internally about revising it.

The issue is that, by creating this big if/elsif chain, you need to make sure there aren't any actions inbetween the elsif statements. The first place your script will run into problems is here:

Code: Select all

....
elsif ( !  $sel->type_ok("id=cred_userid_inputtext", "nagximon\@mydomain.com") ) {print "Location: " . $sel->get_location() . "\n";}

$sel->pause(10000);
elsif ( !  $sel->type_ok("id=passwordInput", "mypassword") ) {print "Location: " . $sel->get_location() . "\n";}
....
Since $sel->pause(10000); is breaking the elsif chain. You could try replacing all of your $sel->pause(10000); lines like so:

Code: Select all

elsif ( !  $sel->pause("10000") ) {}

Still fails for other reasons >> new post
dlukinski
Posts: 1130
Joined: Tue Oct 06, 2015 9:42 am

Re: SELENIUM clickandWait or Timeout or else

Post by dlukinski »

Here is the original perl
----------------------------

Code: Select all

use strict;
use warnings;
use Time::HiRes qw(sleep);
use Test::WWW::Selenium;
use Test::More "no_plan";
use Test::Exception;

my $sel = Test::WWW::Selenium->new( host => "10.96.36.114",.
                                    port => 4444,.
                                    browser => "*firefox",.
                                    browser_url => "https://login.microsoftonline.com/" );

$sel->open_ok("/login.srf?wa=wsignin1.0&rpsnv=4&ct=1466534348&rver=6.7.6640.0&wp=MCMBI&wreply=https%3a%2f%2fportal.office.com%2flanding.aspx%3ftarget%3d%252fdefault.asp
$sel->type_ok("id=cred_userid_inputtext", "nagximon\@mydomain.com");
sleep(10);
$sel->type_ok("id=passwordInput", "mypassword");
$sel->click_ok("id=submitButton");
$sel->wait_for_page_to_load_ok("30000");
sleep(10);
$sel->click_ok("xpath=(//button[\@type='button'])[10]");
$sel->click_ok("//a[\@id='O365_SubLink_ShellSignout']/div/span[2]");
$sel->wait_for_page_to_load_ok("30000");
sleep(10);
$sel->click_ok("id=submitButton");
$sel->wait_for_page_to_load_ok("30000");
--------------------------------------------------------------
Next one is Fixed Perl (error at line 19 on execution)
--------------------------------------------------------------

Code: Select all

use Test::WWW::Selenium;
use Test::More "no_plan";
use Test::Exception;
use TimePerf2;

my $time= TimePerf2->new(60,90,120);


my $sel = Test::WWW::Selenium->new( host => "10.96.36.114",.
                                    port => 4444,.
                                    browser => "*firefox",.
                                    browser_url => "https://login.microsoftonline.com/" );

$time->startTime("ALL");
$sel->set_timeout_ok("120000");
if ( !  $sel->open_ok("/login.srf?wa=wsignin1.0&rpsnv=4&ct=1466534348&rver=6.7.6640.0&wp=MCMBI&wreply=https%3a%2f%2fportal.office.com%2flanding.aspx%3ftarget%3d%252fdef
elsif ( ! $sel->set_speed("1500") ) { }

elsif ( !  $sel->type_ok("id=cred_userid_inputtext", "nagximon\@mydomain.com") ) {print "Location: " . $sel->get_location() . "\n";}

elsif ( !  $sel->pause("10000") ) { }
elsif ( !  $sel->type_ok("id=passwordInput", "mypassword") ) {print "Location: " . $sel->get_location() . "\n";}

elsif ( !  $sel->click_ok("id=submitButton") ) {print "Location: " . $sel->get_location() . "\n";}

elsif ( !  $sel->wait_for_page_to_load_ok("120000") ) {}.
elsif ( !  $sel->pause("10000") ) { }
elsif ( !  $sel->click_ok("xpath=(//button[\@type='button'])[10]") ) {print "Location: " . $sel->get_location() . "\n";}

elsif ( !  $sel->click_ok("//a[\@id='O365_SubLink_ShellSignout']/div/span[2]") ) {print "Location: " . $sel->get_location() . "\n";}

elsif ( !  $sel->wait_for_page_to_load_ok("120000") ) {}.
elsif ( !  $sel->pause("10000") ) { }
elsif ( !  $sel->click_ok("id=submitButton") ) {print "Location: " . $sel->get_location() . "\n";}

elsif ( ! $sel->wait_for_page_to_load_ok("120000") ) {}.
elsif ( !$time->endTime("ALL") ) { }

else { $time->getTimes(); }
$sel->stop();
--------------------------------------------
Output when running from CLI
--------------------------------------------
ok 1 - set_timeout, 120000
ok 2 - open, /login.srf?wa=wsignin1.0&rpsnv=4&ct=1466534348&rver=6.7.6640.0&wp=MCMBI&wreply=https%3a%2f%2fportal.office.com%2flanding.aspx%3ftarget%3d%252fdefault.aspx&lc=1033&id=501392&msafed=0&client-request-id=64b6eb2d-3b6f-4bcb-8138-08f686ad8ebd
ok 3 - type, id=cred_userid_inputtext, [email protected]
ok 4 - type, id=passwordInput, mypassword
ok 5 - click, id=submitButton
ok 6 - wait_for_page_to_load, 120000
ok 7 - click, xpath=(//button[@type='button'])[10]
ok 8 - click, //a[@id='O365_SubLink_ShellSignout']/div/span[2]
ok 9 - wait_for_page_to_load, 120000
# Error requesting http://10.96.36.114:4444/selenium-server/driver/:
# ERROR: Element id=submitButton not found
not ok 10 - click, id=submitButton
# Failed test 'click, id=submitButton'
# at msportal-1 line 19.
Location: https://login.microsoftonline.com/login ... 5d1e9ddde5
1..10
# Looks like you failed 1 test of 10.
------------------------------------------
(it works in SELENIUM IDE when recording steps even with these pauses)
What is the best practice in this case?
spurrellian
Posts: 43
Joined: Tue Jan 06, 2015 6:26 am
Location: Bath, UK

Re: SELENIUM clickandWait or Timeout or else

Post by spurrellian »

I think you are getting a similar issue Ive had in the past that when you export in from Firefox into Perl it breaks some of the dynamic links as it inserts special characters.
.

Are you recording a test case of logging into a office 365 portal?

When I'm back in the office tomorrow I'll take a look and try it on our tenancy
Paul S - Using Nagios XI, Network Analyzer, Log Server
User avatar
mcapra
Posts: 3739
Joined: Thu May 05, 2016 3:54 pm

Re: SELENIUM clickandWait or Timeout or else

Post by mcapra »

If the button exists within an iframe, selecting the iframe before interacting with the element is something you could try. eg, if the button is nested in an iframe like so:

Code: Select all

<iframe id="myCoolFrame" name="myCoolFrame">
           <a href="http://www.google.com">Lets Go To Google</a>
</iframe>
In order to click the "Lets Go To Google" link, you (in some cases) would need to first select the iframe "myCoolFrame". In the Selenium IDE, it would look something like this:

Code: Select all

.....
selectFrame            myCoolFrame
click                  link=Lets Go To Google
.....
You might also try selecting the current window:

Code: Select all

.....
selectWindow           title=My Window's Title
click                  link=Lets Go To Google
.....
Just a few considerations. Those might not be applicable to your current situation. As I do not know what the page immediately following "https://login.microsoftonline.com/" contains (I assume its a protected portal), I cannot offer much insight into why your test case is failing.

All I can say is that # ERROR: Element id=submitButton not found means that Selenium is unable to find an element with the id "submitButton" for some reason. There's lots of reasons why that check might fail, but I can't say for sure without knowing the structure of the page. If its a protected portal, I hardly expect that knowledge to be given to me ;)
Former Nagios employee
https://www.mcapra.com/
Locked