New way of conducting web performance testing

It has been long time that performance testing meaning commercial tools like LoadRunner from Mercury, later added Avalanche from Spirent and IxLoad from Ixia.

However, none of the traditional tools can handle:

• Complicated web transactions
• Different OS/Browser behaviors

Now, with some recent (cloud-computing based) developments; there are few alternatives that one can leverage other people's infrastructure and conduct vast load of testing (while still pricy):

• Gomez.com
• BrowserMob
• utest.com

These tools/sites are also providing other tests as monitoring/alert services that could also be add-on values for live traffic. However, I like the easy way to add thousands of real web-traffic.

Also, as cloud-computing is getting more popular and one can hope the price will drop down more soon!

My Python Based Test Framework (PAT) (1)

After used and reviewed so many test-frameworks; now I have been working on developing my own, again. More precisely, it will be just another re-factor work and who knows how much will be left out not touched.

Well, before really dig into the details of the new framework, reviewing the ones I have used before.

1. STAF. It's really a nice and cool setup; except setting it up maybe a pain; especially that my typical test environment has different types of machines need be managed and get all machines running STAF is really a challenging work. Last time I have seen it all working successfully was actually on a small-scale setup with relative coherent machines.
2. Shell scripts. In one of my previous projects (jobs), we got a full set of shell scripts working very nicely that monitor different set of machines (DUTs ,controlling machines, and tester-machines). They are mostly Linux-ish machines; and Tcl/Expect was used to manage the login process. There are also lot of text-processing the extract the logs (test-data, logs frm DUT etc), MIBs etc to correlate the data and finding isues. The correlation part is so wonderful that it had given huge visibility since all power-point presentation were backed up by serious of data; which were extracted out from pages and pages of original logs etc.
3. Perl/Python scripts. In my last job, there was some Perl scripts that drive serious of data packets and then extract information out from the logs (yes, logs again) and captured packets to determine efficiency and effectiveness of (virus!) detection. It's an interesting setup of scripts and it was kind of tedious and I converted portion of code into Python to make the lot faster. Gosh, the Perl scirpts were so slow to send out packets but it's quite good in parsing results
4. Now, here, in my current place, we had a serious of shell-scripts (before me) done by group of genious software engineers (but without much sense of testing). Fr months, I tried to throw the wholething awaya; but I had to live with it since it does take times to develop a new onw.

And so, the journey started... my new test-framework based on Python; I called it PAT (Python Automation Test).

Open Source Automated Test Tools Written in Java

Well, I'm not a Java fun; though I have seen it growing from its inception.
However, I found this blog quite interesting for the testing tools based on Java.

Make extra cash from software testing...

I recently cross this site and thought it is interesting: http://www.utest.com/. I dont know how effective they are in qualifying customers and testers; and also I don't know how the compensation is determined, but it sounds a possible way of leveraging the skills in software testing for some extra cash. It may not be great $$, but I guess anything may be helpful if one doesn't have any income (which is the case for lot of people in this economic environment).

In addition, I feel strike is that I actually thought about the similar business model, inspired by the Rent-A-Coder. Well, I never had time/effort/resources to get it implemented; though I could do it better?

Selenium vs. Watir

One of the main testing activities I have been involved (or studied) in the last couple years is more or less related to testing web-based applications (either pure e-commerce, web-app, web-security or even appliance). For all of these related technologies, testing web-page is a norm and there are few methods to get the job done:

• Manual testing. The dumb way but still sometime the most productive way for the initial exploratary testing. I don't see any way to totally avoid it!
• Simple scripting…like wget. Sometimes, if one just needs check if a site is up and one or couple pages are there, using wget combined with some simple shell scriptsis probably one of the best choices.
• Regression Testing (or even functional testing); here we need some heavy-duty scripting tools. I did try the most expensive tools; like SilkTest, WinRunner, and also the so-called QTP (now part of HP); but I have never liked it: way too expensive and dumb (prietary scripting).

Well, the alternatives are available; and I believe they are quite useful and, even (IMO) better than the commercial QTP or like.

Our of all of the alternatives, two of them stand out: Selenium and Watir. There are lot of discussions and comparison between these two and here are my reasons of choosing Selenium over WAITR:

• Linux support: Selenium is much better than Watir; Watir was rooted for MS-IE-Ruby platform
• Python support: Selenium had Python support at beginning and Watir was a pure Ruby toy. Oh, why Python over Ruby? Speed!
• Doesn't Watir have anything better? Yes, Watir is better in supporting DOM objects, purely IMO. In addition, if the AUT is written in Ruby, there should be some build-in benefits,…, but dont quote me on it; I didn't go that far on this approach
• At last, if one has exposed to any unit-testing framewrok, like Xunit, Junit, and Selenium could integrate with the Python unit test: Pyunit.