excel-testng: driving TestNG tests through MS Excel

Over last few days, I have been working on a small project in my spare time. It's called excel-testng and it provides a way to drive TestNG tests through MS Excel. The code repository is here: http://code.google.com/p/excel-testng/ and the jars can be downloaded from here. I also put together a small automation project using Selenium WebDriver to demonstrate its use here: https://github.com/randomsync/excel-testng-demo.

Introduction

During functional testing projects, after we create tests for a certain application, we need to review them with the rest of the team (developers, analysts, project managers etc). We may use a test management tool to document the test cases and then export them into an easily distributable format like MS Excel or PDF. Sometimes, we may also create the tests in Excel directly specifying the test name, description, parameters and other data in the spreadsheets. And then finally, we may automate a part (or all) of the tests.
So after the tests are automated and when executing the tests, we need to specify which tests to run and the test data (parameters). In Quality Center, it involves creating a test set (kind of like a test suite) and then adding tests to it. If they are automated in QTP which integrates with QC, they can be executed by running the test set. But we have started using Selenium WebDriver for its cross browser capabilities and that means we need to specify the tests in a format that can drive the Selenium tests. We use TestNG as the framework for test execution, assertions and reporting.

Extending TestNG

TestNG is a great framework for test execution and its input can be in form of an XML file that specifies which tests to run, where to find the test classes/methods, test parameters and a whole lot of other features that gives you a fine grained control over each test execution. However for our UI functional tests, we wanted to be able to specify the tests and test executions in an easily distributable format (like MS Excel) as mentioned above. The good thing is that TestNG provides the capability to extend it so that its input can be created and executed programmatically. So this way, the test specification in Excel files can be parsed and driven through TestNG. And this is exactly what excel-testng does. It externalizes all Excel parsing and TestNG XmlSuite creation so that the focus can be in creating the test classes and methods. Once that is done and Excel specifications created, all that needs to be done is to provide a main method that does this:

   1: ExcelTestNGRunner runner = new ExcelTestNGRunner("input.xls"); // this can be a single file

2: // or a directory, in which case all spreadsheets in that directory are parsed

3: runner.run(); // run the tests

With ExcelTestNGRunner class, you just specify the input location of the Excel file(s) that constitute the test specifications and then call the run() method, which parses the excel file using default parser (an instance of ExcelSuiteParser) into XmlSuites, creates a TestNG object if not already created and then runs them. After that TestNG takes care of test execution and reporting.

Excel Test Specification

ExcelTestNGRunner parses each worksheet in the Excel file(s) into a separate suite using the included parser (ExcelSuiteParser). Each suite can have suite level parameters specified in the worksheet and the test specifications that specify which tests will be run, their name, description, parameters and the test classes. Here's what a test suite in Excel looks like (a demo spreadsheet can also be downloaded from here and can be used as a starting point):

Specifying TestNG tests in Excel File

The top few rows in the worksheet provide the suite information. ExcelSuiteParser looks for the string "Suite Name" and retrieves the name of the suite from next cell in the same row. Similarly, it looks for string "Suite Parameters" and retrieves suite parameters from the next cell. "Suite Configuration" is not currently used. You can customize the location of these values by providing your own map to the parser. See "Customizing Input" for more details.

To retrieve the tests that will be executed, it looks for the 1st row containing "Id" in the 1st column. This will be the header row, below which each row is a separate test. The header row and tests must have columns specifying Id, Test Name, Test Parameters and Test Configuration (which specifies the classes containing the Test methods). If "Id" is left blank, the test will not be added to the suite. Finally, it parses each row under the header row into a TestNG XmlTest and adds it to the suite. The test specifications are provided as:

• Test Name: generated by concatenating Id & Test Name
• Test Parameters: retrieved from "Test Parameters" column and need to be provided in valid properties (<key>=<value> etc.) format. You can also specify functions as parameter values and then add the logic to parse and evaluate the functions in your test classes (maybe a Base Test class)
• Test Classes: specified under "Test Configuration" column as classes property. Currently, you can only specify a single test class of which, all @Test annotated methods will be executed as a part of this test execution. I'm going to add the ability to select the test methods in later releases.

Customizing Input

If your test cases are specified in Excel but in different format, there are 2 levels of customizations you can do with ExcelTestNGRunner on how to parse the input spreadsheet(s):

1. Custom Parser Map (currently not implemented): You can use the in-built parser but specify your own parser map, which tells the parser where it can find the suite and test data
2. Custom Parser: You can create your own parser by implementing IExcelFileParser interface. You need to parse the spreadsheet file and return a list of TestNG XmlSuites.

ExcelTestNGRunner also provides helper methods to customize the TestNG object it uses to execute tests. For example, you can specify any custom listeners using addTestNGListener() method. If you need to have more control, you can create your own TestNG object and then pass it to ExcelTestNGRunner. Please see javadocs for more details.

Putting it together

You can see the project at https://github.com/randomsync/excel-testng-demo for a complete working demo of excel-testng to parse the input test specifications in Excel. It uses Selenium WebDriver to automate the testing of basic Google search functionality.

Update (3/12/2012): I'm now using google code to host this project because of the provision to host the downloadable jars, javadocs and wiki easily. I'll keep it synced with github but you can find the project documentation and downloads there.

Friday Lunch: Content-Type, Content-Disposition HTTP Headers and MIME type handling in IE

We have a scripting tool that is used to create web-based scripts that simulate application scenarios. These scripts are run through a monitoring tool on a continuous basis to assess the health of these applications. The scripts & scripting tool are also used to validate the scenarios if the monitoring tool issues an alert. The scripting tool uses IE to run the HTTP request/response scenarios. When recording the script for a recent new application, it was noticed that after submitting the request which was an HTTP POST with XML data, the tool was bringing up a dialog box to save the response file, something you’d get when clicking on a HTML link in the browser to a file (maybe a word .doc etc) and if you don’t have an application to open that kind of file. We checked the response and it contained a standard XML that should usually be displayed within the browser (like if you click on Amazon’s AWS service). The fact that it was bringing up a download prompt and not displaying the XML in the browser was a problem because the non-technical team that uses it to validate the application will not be able to determine whether the script ran successfully or not.

The application is a servlet that takes the initial POST request and returns an XML response. After further investigation with the tool and looking at the response headers, it was found that the application was returning “application/x-www-form-urlencoded” in “Content-Type” header erroneously. I haven’t found something specific that would say that content type is not correct for HTTP response, but it is more common to use this content type when submitting form-submission data that is URL encoded (see here).

Investigating with WebScarab

My first conjecture was that the file download prompt was being displayed because of this Content-type header’s value. If we could intercept the response and replace its value with something like “text/xml”, the tool or browser will be able to display the content within the application instead of bringing up the file download prompt. And to validate that, I used WebScarab to intercept the request/responses. But for some reason (maybe because of invalid Content-Type header), it didn’t intercept the response. It would intercept the request and after accepting, it would just complete the request and we’ll end up seeing the file download prompt in the tool. Clearing the “Only MIME-Types matching:” didn’t help.

So the workaround was to write a BeanShell script within WebScarab to replace the “Content-Type” header with a proper value instead of manually intercepting the response. That turned out to be quite straightforward. All we needed to do was put:

response.setHeader("Content-Type","text/xml");

in WebScarab’s Bean Shell interface under the fetchResponse method. As is obvious, this replaces the value of Content-Type header in the response to “text/xml”.

Once we tried it out, it worked and the browser displayed the XML response without bringing up the file download prompt. This proved that the Content-Type header was the problem. But what exactly was the problem? What makes the browser display a prompt to the user to download a file instead of just displaying the content? Further searching revealed this article: http://www.jtricks.com/bits/content_disposition.html

So basically the Content-Disposition header along with Content-Type specifies the MIME type of the file returned and whether the browser should display it or prompt the user to download it. The RFC has all the details but the gist is that the Content-Disposition header can be set to “attachment” for the browser to display the file download prompt. But the response we were getting didn’t have a Content-Disposition header. And IE was still displaying the file download prompt. Further reading warranted.

How does IE handle MIME types in a response, anyways?

I kept on searching and reading on how and when a browser brings up the file dialog prompt. The Content-Type header still bothered me. Then I came across this MSDN article: MIME Type Detection in IE. It provides details on what steps IE takes to detect the content-type of the content before deciding what to do with the file:

The purpose of MIME type detection, or data sniffing, is to determine the MIME type (also known as content type or media type) of downloaded content using information from the following four sources:

• The server-supplied MIME type, if available
• An examination of the actual contents associated with a downloaded URL
• The file name associated with the downloaded content (assumed to be derived from the associated URL)
• Registry settings (file name extension/MIME type associations or registered applications) in effect during the download

If you look at the known MIME types for FindMimeFromData, “application/x-www-form-urlencoded” is not one of them. It is also not an ambiguous MIME type ("text/plain," "application/octet-stream," an empty string, or null) either. So with the MIME type as unknown,

If the "suggested" (server-provided) MIME type is unknown (not known and not ambiguous), FindMimeFromData immediately returns this MIME type as the final determination.

And what happens after that is in this MSDN article: Handling MIME Types in IE. It doesn’t mention exactly what happens if the MIME type is unknown, there’s no application associated with it in the registry and there’s no Content-Disposition header. But I’m guessing that the only thing for it left to do is to let the user decide by showing the file download prompt as was happening in our case. And we were able to prove that. I looked up the registry key for “text/xml” under HKEY_CLASSES_ROOT\MIME\Database\Content Type and added it to a new key “application/x-www-form-urlencoded”. The new registry key looked like this:

[HKEY_CLASSES_ROOT\MIME\Database\Content Type\application/x-www-form-urlencoded]
"CLSID"="{48123BC4-99D9-11D1-A6B3-00C04FD91555}"
"Extension"=".xml"
"Encoding"=hex:08,00,00,00

This tells IE that for responses received with Content-Type equal to “application/x-www-form-urlencoded”, use extension “.xml” and open it however XML files are opened (which is to display them inline). Once added, we went back to the tool and ran the script. And this time, it displayed the response xml without bringing up the download prompt.

So through this exercise, I learned about how IE displays different MIME types and about Content-Type and Content-Disposition headers.

A Generic Function to enhance BPT components

About 2 years ago, we created a set of QTP function libraries to drive the automation effort using Quality Center's Business Process Testing (BPT) functionality. What those libraries provide are a set of generic functions from clicking on a UI object to verifying an object's property, which then can be used to create both generic BPT components that can be used on any application and specific components that target a particular application or a screen within an application. We have used those libraries and components to successfully automate our application testing without much modification and maintenance.

Recently, we were using the same set of components to create/update the test cases for a new application and I took this opportunity to enhance the libraries by adding a few functions. One of them is a generic Eval function that provides some more flexibility in creating components. As we know, VBScript has its own Eval function that evaluates the provided expression and returns the result. So to be able to use that functionality from within the components, I created a new function that returns the result of that expression or the string itself if it is not an expression. Here's the function listing:

' Function EvalFunction
' ------------------
' Evals the function call specified in the parameter and returns the result
' Parameter: expr - Any VBScript expression
'@Description Evals the function call specified in the parameter and returns the result
'@Documentation Evals the function call specified in the parameter and returns the result
Public Function EvalFunction(ByVal expr)
  Dim res
  On Error Resume Next
  If (expr <> "") Then
    res = Eval(expr)
    If (Err <> 0)Then
      res = -1
      Err.Clear
    Elseif res = "" Then
      res = expr
    End If
  End If
  EvalFunction = res
  On Error Goto 0
End Function

The function itself is quite simple but it provides a lot of added functionality. I can use this function in my BPT components to evaluate any expression during run-time. And in addition, I also create a generic component that calls this function and returns the result. When I need to create a test case that needs a run-time value, I can use this component within my test cases to have that functionality.

Using the EvalFunction function in components:

For a part of our application's functionality, we had a component ("AddIP") that adds an IP to a user. Since there were 4 text fields for each of the octets, we had 4 component input parameters for each octet. In the picture below of AddIP component, each octet is a different text field (“IP1”…”IP4”) requiring a different input parameter (BrowserWebEditSet is another generic function to enter value in a WebEdit object)

But while revisiting the testing, I felt that its much easier to provide the IP parameter as a whole when you're creating a large test case where multiple IPs need to be added. So instead of having to create a new function, I just used the EvalFunction to create another component ("AddIPv2") that takes the full IP address and splits it into each octet and then enters the values as required in different text boxes. I use the VBScript “split” function to split the input parameter (the IP address) into each octet like this: "Split(""" & Parameter("IP") & """,""."",-1,vbTextCompare)(0)" which returns the 1st octet. As in the picture of the component below, I call the function 4 times for each octet which returns the result in a local parameter which I can then use to set for each text field.

So using this enhanced component, the user has to provide only the whole IP address “xxx.xxx.xxx.xxx” instead of each octet in a separate parameter.

Using the EvalFunction component in BPT tests:

The EvalFunction component is a one operation component that calls the EvalFunction and returns the result in a component output parameter. This component can be used in any test to evaluate any expression and use the result in a subsequent component.

For example, a lot of our test cases require creating a new user. Instead of specifying a fixed value for the user ID or having the tester enter a value in a run-time parameter before every run of the test, the user ID can be generated using a timestamp which guarantees its uniqueness. This is done by using the EvalFunction component. In the BPT test case below, the EvalFunction component is used with input parameter a string prefix concatenated with the current timestamp (DateFormatter is another function that returns the current date/time in appropriate format). It returns the result in a component parameter “result” which is then later used in AddUser component to create a user with that user ID. 

This way, this test can be run any number of times without requiring any modification.

Epilogue:

The function that I showed in this post is a part of the automation libraries that we created. Over time, as we use them for testing multiple applications, we have enhanced the libraries even more. These libraries, along with the BPT functionality provided by Quality Center, have helped us create reusable components that require very less maintenance. And these components are being used by testers to create and run automated test cases for different applications and different functionalities.

Intercepting SSL traffic using WebScarab

The last time I wrote about intercepting web requests using WebScarab, I was successful in intercepting SSL traffic generated through a custom Java client. Even though the process to do that was quite tedious – involving exporting the WebScarab server certificates into .cer format, importing the certificate into a Java keystore and then running WebScarab as a reverse proxy – I was able to intercept and view the SSL traffic that was being generated. But there was an inherent issue with that process that I overlooked.

When a proxy is setup to intercept SSL traffic, the security issue is that the SSL certificate that is presented by the proxy is not signed by a trusted authority. Web browsers detect this and give the user an option to accept or not accept this risk. So there is no problem in using the proxy to intercept web traffic to secure sites and we can just point the browser to the proxy and accept when warned about certificate error. But in case of Java clients using JSSE, there is no assumption of an interactive user session and so by default it throws an exception if there are any certificate related issues – be it an unknown certificate in which case it throws the exception:

javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

or a hostname mismatch:

javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No name matching … found

The latter is thrown because when a new HTTPS connection is created using HttpsURLConnection class, it implements a default HostnameVerifier interface which checks if the host we’re trying to connect to matches the name in the certificate in its certificate store (specifically the cn within the certificate). If it doesn’t, it throws the above exception. The client I was using earlier overrode the default HostnameVerifier with a custom one, which ignored the hostname mismatch. But this time with a new client for a different application, it didn’t and I had to go one extra step to intercept the requests, which is detailed below. So first:

1. Start WebScarab and run it as a reverse proxy on port 443. This is so that WebScarab behaves as a secure server to the client, even if with a self-signed certificate instead of one signed by a trusted authority. (If running WebScarab from the same machine that is generating the requests, we should also select “Intercept requests” check box. This is important because in that case, the proxy is an infinite loop to its own interface and so we want to be able to break the flow and Abort after the first intercept)
   
2. Modify the hosts file to point WebScarab hostname to the IP of the machine where it is running. In case of local, it should be:
   127.0.0.1    WebScarab
   This is specifically so we can get around the issue of hostname mismatch because we’ll try to connect to host “WebScarab” instead of actual target server. If the client overrides the default HostnameVerifier to ignore those errors, it can be setup so that the client points to the actual host:
   <ip where WebScarab is running>    <target hostname>
   
3. Use the java program available here to create a keystore with the WebScarab certificate
   >>java InstallCert WebScarab
   Since WebScarab hostname is pointing to the WebScarab proxy, this program will connect to it and retrieve its certificate. It will create a keystore file called jssecacerts with WebScarab’s certificate (keystore password is blank by default).
   
4. Configure the client to use WebScarab as the host within the URL. So instead of https://<hostname>/<path>, it should be https://WebScarab/<path>.
   
5. Run the java client with the truststore and password properties: -Djavax.net.ssl.trustStore=<location to jssecacerts file> -Djavax.net.ssl.trustStorePassword=<password, default blank>

At this point, WebScarab proxy should intercept the request. I can review it, and abort it so it doesn’t repeat. Obviously, the request can’t be sent to the actual destination server. As I’ve noted above and as far as I know, there’s no way to get around the hostname mismatch error unless the default HostnameVerifier is overridden. But in my case, I was fine with just intercepting the request and creating my LoadRunner scripts using the raw HTTP request.

QuickTest Pro: Handling XMLs using XMLDOM object

This is how diverse my writings are going to be. Testing tools, that’s it. I could write about my adventures in photography but that wouldn’t be very interesting until I put some more time and money into it. Even though I’ve been using my 50mm f/1.8 lens with some interesting results, I still feel that there’s lots more I need to learn before I can write about it …maybe someday soon. But this one is about a different, though as interesting topic - handling XML objects in QTP.

Background:

I’m currently involved in test automation for IDM application. If you are familiar with Sun’s IDM product, you know that all objects are stored in database as XML and these objects can be viewed/checked out/edited through the debug interface. Now the way we have customized it is that managers (users with Manager role) or end users are allowed to submit requests for creating users or modifying users’ attributes. The requests are submitted through the user interface and the workflows take care of taking appropriate actions in the background. There is very little information displayed on the web page once the request is submitted that can be used to validate if the action actually succeeded or not. The only way to validate that the action was successful is to either login through admin interface and check the task results or through debug pages, pull up the user’s XML object and verify the changes have gone through.

As I was creating the BPT components and test cases to handle different test scenarios, it was increasingly obvious to me that the most foolproof way to validate the results of the test would be to create a component that validates the user’s XML. I wasn’t much familiar with handling XML objects in VBScript so I left it for later. And later is now (or was yesterday). So I spent yesterday going through the XML Document Object Model (DOM) and Microsoft’s implementation of it and creating a VBScript function to validate an XML’s attribute value. It worked out fine with a little initial struggle.

Please keep in mind that what I’m providing below is in no way a complete solution of using XMLDOM to do whatever you want to do. All I needed to do was to retrieve an attribute value from an XML using the XPath that is provided as input parameter and to compare it with the expected value. What you want do with XMLDOM may be different and based on your application’s and automation needs. The reason I’m posting this is that if you’re using XMLDOM with QTP/VBScript for the first time, you can avoid the initial struggle that I went through and get the job done faster.

For Microsoft DOM reference (this is where I got most of the information I needed), visit this: http://msdn.microsoft.com/en-us/library/ms764730(VS.85).aspx. The XPath language is described at: http://www.w3.org/TR/xpath

Solution:

1. The first step is to create a parser object and load the XML. The XML can be from a file or from a string. Here I’m using the loadXML() method to load it from the innertext property of a WebElement object that I retrieved earlier. To use a file, use the load() method instead.

Set xmlDoc = CreateObject("Microsoft.XMLDOM")
xmlDoc.async = false
xmlDoc.loadXML(objs(0).GetROProperty("innertext"))

You set the async property to false so that it doesn’t move on before the document is completely loaded. The reason I’m using CreateObject(“Microsoft.XMLDOM”) instead of XMLUtil.CreateXML() which is provided in QTP OM Reference is that when I used that, it gave me an error specifying that it couldn’t find the DTD file referenced in the XML. It seems that it needs the DTD if it’s mentioned in XML to be able to load it. I had the DTD and when I uploaded it to my local component folder, it worked fine. But since I could be running the test on a remote host, I didn’t want to have to upload that DTD to all my host machines. And I didn’t look into any other way. But if you decided to use the XMLData object provided by QTP, the objects and method names are different even though the overall steps will be the same. For example, to load the document, you’ll use:

Set xmlDoc = XMLUtil.CreateXML()
xmlDoc.load(objs(0).GetROProperty("innertext"))

2. Once you have the XML object, you need to get to the desired element. I kept it simple and asked for the XPath to the desired element as an input parameter. Once I have that, I used the selectNodes() method to get all the nodes in the desired XPath.

Set objNodes = xmlDoc.selectNodes(nodeXPath)

This gives me a collection of all the nodes (specifically, the IXMLDOMNodeList object) that match the specified parameter ‘nodeXPath’. With QTP XMLData, you can use:

Set objNodes = xmlDoc.ChildElementsByPath(nodeXPath)

If nodeXPath is invalid or doesn’t match any elements, the length of the collection will be 0.

3. So now, I need to check the length of the collection and get to the actual element that I want. Since I didn’t need to care if multiple elements match the XPath, I just took the first element in the collection. The item property returns a single node (IXMLDOMNode) from the collection specified by the index, in this case 0.

If objNodes.length > 0 Then
Set objNode = objNodes.item(0)

4. Now that I have the actual element, I need to get the value of specified attribute. The getAttribute(name) method returns the value of the ‘name’ attribute. The attribute name and its expected value are passed as input parameters:

If StrComp(attrValue, objNode.getAttribute(attrName), vbTextCompare) = 0 Then
'expected value matches actual value

attrValue is the input parameter containing the expected value and attrName is the attribute name. If it matches, I write a Pass in the results and if not, it’s a fail. Once done, I clean up the objects.

Set xmlDoc = Nothing
Set objNodes = Nothing
Set objNode = Nothing

That’s pretty much it. Once we start using the component actively, I may come up with some more ideas to improve it. I’ll write about it if they provide me enough of a challenge.

----------------------------------------------------------------------------------

Sample XML:

<User name="abcdefg" creator="Configurator" email="a.a@x.com" disabled="false" locked="false">
<Roles>
<ObjectRef name="Employee" isWeak="false"/>
</Roles>
<Attribute name="employmenttype" type="string" value="Employee" syntax="string"/>
<Attribute name="firstname" type="string" value="a" syntax="string"/>
<Attribute name="fullname" type="string" value="a, a" syntax="string"/>
<Attribute name="ismanager" type="string" value="true" syntax="string"/>
<Attribute name="lastname" type="string" value="a" syntax="string"/>
<Attribute name="middlename" type="string" value="M" syntax="string"/>
<Attribute name="phone" type="string" value="9999999999" syntax="string"/>
<AdminRoles>
<ObjectRef type="AdminRole" name="Manager" isWeak="false"/>
</AdminRoles>
</User>

For example, the XPath to get to <Attribute> with name=employmenttype is: User/Attribute[@name='employmenttype’]

Business Case for Testing Tools & Automation

This post had been lying in my drafts for a long time. For some reason, I decided not to publish it. But it's still relevant...except that I have submitted the business case since then and it's lying at somebody at Finance's desk or email Inbox for review/approvals.

----------------------------------------------------------------------------------

I'm currently working on a business case to invest in testing tools and functional test automation. The case will be sent to upper management and finance for approval and once they do, we're going to start working on building an automation framework. I foresee a lot of hurdles and am not keeping my hopes high based on the tightening of budget and how the business case is coming out to be.

One of the things that I have to add in the business case is financial analysis which includes an estimate on return of investment. And it has been a very "enlightening" process. Based on the current estimates on the capital and labor costs, it'll take us 2.9 years to get a return on investment. And of course, it's based on a lot of assumptions and I'm kind of skeptic when claiming that number for ROI.

You see, one of things that are never in short supply at my workplace is the list outstanding tasks. And with that comes extreme work pressure and very little time to invest in exploring new technologies and coming up with new ideas. I'm quite sure that designing the automation framework will be a time consuming and learning process and I will have to be devoted at least half-time (if not more) in this effort. And I'm quite sure that when we actually start working on that effort, the estimates that I put in for labor in the business case will start looking like best case scenario.

The brighter side is that I have now under my belt the experience of creating a business case. And once/if approved, we'll be undertaking the effort of functionally automating the test cases for our applications. We're also planning to expand our QA service by offering the testing tools and/or automation to other teams.

My colleague and I had been talking about the design of the framework since we first conceptualized the idea of automation. We had this whole idea in our minds on how to make it reusable, to maximize customizability with least rework. He has experience in building these kind of frameworks and I have had a brief encounter at my last job. I recently looked up online and found that it is now one of the established practices, one of the buzzwords surrounding test automation process - Keyword Driven Test Automation Frameworks. It must have been a long time since both of us were in automation business because what I read matched what we had been planning to do. Seems like a lot of people have written their experiences with building automation frameworks with this idea. Seems like we'll be able to use some of the knowledge to our benefit.

Coming up...

A few weeks ago, I spent some time going over JMeter and building some advanced web test plans. But then I got distracted and buried under some intense projects, including a 1-week training on a product that I didn't know much about and that I have to be an advanced user of within a few weeks. (BTW...if you want to know why attending a 1-week advanced training on a product without having a solid background is a bad idea, let me know).

Now I have some time again but it's right before my vacation and I just don't feel like getting immersed into it again. But when I come back, I plan on posting some of my notes on how to build an advanced web test plan in JMeter. I plan on using a more advanced scenario than the one with basic Get requests, probably something similar to my previous post about using an MD5 library to create a LoadRunner script. If I can't find something else, I'll use that very example. I also plan on exploring some webservices and pure XML over HTTP kind of scripts. Stay tuned...

Oh... and I should also mention my reasons for looking into JMeter. It has been increasingly frustrating to borrow time on LoadRunner Controller from an external team, partially because of having to justify to the project team the additional charge back incurred to the project. I can't believe how many emails I have to write to justify a few 100 dollars. But anyway, turning misery into a learning experience, I wanted to explore if JMeter provides enough capabilities and is robust enough to be used for production performance testing.

LoadRunner Scripting Challenge - LiveJournal

My last few posts have unabashedly been about LoadRunner even though I vowed to diversify my writings into more distinct areas. But this one again is about LoadRunner and is too interesting to pass...

Background:

I had an old journal in LiveJournal back from the times when 'blogging' wasn't a very familiar word or activity and most of the popular blogging sites like Blogger and WordPress (these are the only free ones that I've liked so far) didn't exist. I came across LiveJournal and was immediately hooked. I wrote prolifically at that time and I should also mention that my creative writing abilities then were much superior to my current showcase. But over time, I've outgrown LiveJournal because their free version doesn't provide the most needed features and the one that does is ad-supported. To get rid of the ads, you have to pay a nominal amount. I think that wasn't the only reason and I was a paid member for sometime supporting their efforts. But I thought I needed a more traditional 'blog' rather than a journal. I also haven't been writing much lately and have no communities that I would like to keep track of in LiveJournal.

So the result was some of my most cherished writings were stored in LiveJournal's history of posts and I wanted to import all of them to a WordPress blog. WordPress allows importing an XML file of LiveJournal formatted entries. The problem however is that LiveJournal allows exporting the posts only by month. So if I wanted to export my posts from back in 2002, I would have to export it month by month for over 6 years worth of posts. That was certainly not very exciting. But soon it became pretty exciting when I decided to use LoadRunner to export all my posts month by month in XML format and I learned something very interesting in the process - which is of course the whole idea of this post.

Challenge:

When I recorded the script by going to the export page and logging in, I noticed in the script that my password wasn't hard coded anywhere. I expected to find it with the request when I submitted the form containing my username and password. Instead, there were the "chal" field and the "response" field and the password field was blank.

web_submit_data("login.bml",
"Action=http://www.livejournal.com/login.bml?ret=1",
"Method=POST",
"RecContentType=text/html",
"Referer=http://www.livejournal.com/?returnto=/export.bml",
"Snapshot=t5.inf",
"Mode=HTML",
ITEMDATA,
"Name=returnto", "Value=/export.bml", ENDITEM,
"Name=mode", "Value=login", ENDITEM,
"Name=chal", "Value=c0:1193421600:2223:300:DsqGiIDCk0A4hs9sNsmH:c7f09b46d0ba2d82bb68945ea84532fc", ENDITEM,
"Name=response", "Value=b84434e3c2d193c4a1c345d7874a89a3", ENDITEM,
"Name=user", "Value=myusername", ENDITEM,
"Name=password", "Value=", ENDITEM,

Honestly, that was surprising to me. At that time, I couldn't come up with any explanation of how my user ID can be authenticated and successfully logged in without sending the password in form submission. I could smell the hint of another challenge. If you want to try it before reading on to get a better handle, try recording a script on LiveJournal (http://www.livejournal.com) by logging in and making it work.

Solution:

On thinking a little more and looking at the script again, I realized that they could be using JavaScript to generate the response which was some kind of hashed form of the password. On viewing the source of the initial page, I found a js file with this code:

var pass = pass_field.value;
var chal = chal_field.value;
var res = MD5(chal + MD5(pass));
resp_field.value = res;
pass_field.value = ""; // dont send clear-text password!

So on further investigation, I figured what was happening:

1. On initial navigation to any page with the login form, a challenge string is sent to the client. I guess it has to have some kind of expiration time because if you use it sometime later with the correct response, it'll return that the challenge has expired.

2. An event handler is registered with the submit button of the login form.

3. When the submit button is pressed, the event handler takes the field values including the hidden challenge string. It takes care of creating an MD5 digest of the challenge string concatenated with the MD5 digest of the password and clearing the actual password field value. This is the value that you see in the "response" form parameter in the code above. The values are then submitted and the server takes care of validating the response value.

This page explains this in high-level: http://blog.paranoidferret.com/index.php/2007/07/22/secure-authentication-without-ssl-using-javascript/

So to make the LR script work, I had to do 2 things:

1. Get the value of the challenge by correlating appropriately

2. Once I have the challenge string, create the 'response' value by getting the MD5 digest of the challenge string concatenated with digest of the password string.

The MD5 digest calculation is based on the MD5 Message Digest Algorithm by Ron Rivest of MIT. There was no way I was going to write an MD5 function myself. A quick search revealed some implementations of this algorithm. The one I used was from http://www.fourmilab.ch/md5/ because this was in C and saved me from including a lot of legalese. All I had to do was include the header and C file and use the MD5 functions. So this is the outline of final script. I have left out the details which should serve as an interesting exercise.

1. Firstly, get the MD5 digest of the password. This will later be used to concatenate to the challenge string.

MD5Init(&md5c);
MD5Update(&md5c,(unsigned char *)mesg,strlen(mesg));
MD5Final(signature,&md5c);

where mesg is the password string.

2. Correlate the script to save the challenge string at the appropriate place. This is the string that is sent by the server as a hidden form parameter of the login form.

3. Once you have the challenge string, concatenate the MD5 digest of the password (calculated in Step 1) to it. Remember that this digest has to be concatenated as a lower-case hexadecimal number. Get the MD5 digest of this concatenated string.

4. Pass this MD5 digest calculated in Step 3 as the value to 'response' parameter in the login step:

"Name=chal", "Value={chal}", ENDITEM,
"Name=response", "Value={response}", ENDITEM,

That should do it. Now I can parameterize the year and month values in export step and export all my journal entries by month in XML files. It was a simple task to concatenate all the files into 1 big XML file and importing it through WordPress blogging features. I'm a happy WordPress blog user now...we'll just have to see how long that lasts before I start looking for other options.

I guess I should also add that the exercise gave me a great opportunity to learn how web site authentication can be handled without using SSL and how I can script that in LoadRunner.

Note 1: Since creating this script, I explored LiveJournal's server protocol documentation and found that they have documented the authentication mechanism pretty well. Great work in encouraging other users to build custom clients.

Note 2: I felt I should mention that LoadRunner licensing agreement specifically prohibits using the software to run tests on public domain sites. It basically means that you cannot load up your LR Controller with the script you created above and run a load test with any number of virtual users.

Random Virtual User Pacing in LoadRunner

I guess there's always a first time. I had never used LoadRunner's random virtual user (VU) pacing earlier, but am working on a project that will need to use this feature for the first time. And as I thought about it a little more, I may start using it more frequently now. Here's how it happened:

This is one of the rare projects that provided me with excellent documentation - not only system documentation like the system specs, API Guides etc but also performance requirements like actual production volume reports and capacity models that estimated the projected volumes.

The capacity models estimated the maximum transaction load by hour as well as by minute (max TPM). What I needed to do was take maximum hourly load, divide it by 60 to get a per minute transactional load and use this as the average TPM. The idea was to vary the VU pacing so that over the whole duration of test, average load stays at this Average TPM but it also reaches the Max TPM randomly.

For example, if the maximum hourly transaction rate is 720 requests and maximum TPM is 20, the average TPM will be 720/60 = 12 and I will need to vary the pacing so that the load varies between 4TPM and 20TPM and averages to around 12TPM.

The Calculation:

To vary the transactional load, I knew I had to vary the VU Pacing randomly. Taking above example, I had to achieve 12TPM and I knew the transactions were taking around 1-2 seconds to complete. So I could have the pacing of around 120 seconds if I needed to generate a fixed load of 12TPM with a 5 second Ramp-up and 24 users.

Script	TPM	Number of VUs	Pacing (sec)	Ramp Up
Script 1	12	24	120	1 VU/5sec

So now to vary the TPM to x with the same 24 virtual users, I will need to have a pacing of 24*60/x. I got this from an old-fashioned logic which goes in my head this way:

24 users with a pacing of 60 seconds generate a TPM of 24
24 users with a pacing of 120 seconds generate a TPM of 24 * 60/120
24 users with a pacing of x seconds generate a TPM of 24 * 60/x

So using above formula, to vary the load from 20 to 4TPM I will need to vary the VU pacing from 72 to 360. So now we have:

Script	TPM	Number of VUs	Pacing (sec)	Ramp Up
Script 1	4 to 20	24	Random (72 to 360)	1 VU/5sec

Of course, there's a caveat. The range of 72 to 360 seconds has an arithmetic mean of 216. 120 is actually the harmonic mean of the 2 numbers. So the actual variation in TPM will depend on the distribution of random numbers that LoadRunner generates within the given range. If it generates the numbers with a uniform distribution around the arithmetic mean of the range, then we have a problem.

I ran a quick test to find this out. I created an LR script and used the rand() function to generate 1000 numbers between the range with the assumption that LR uses a similar function to generate the random pacing values.

int i;
srand(time(NULL));
for (i=0;i<1000;i++){
lr_output_message("%d\n", rand() % 289 + 72);
}

And of course, the average came out close to the arithmetic mean of 72 and 360, which is 216.

So with the assumption that the function used by LoadRunner for generating random pacing values generates numbers that are uniformly distributed around the arithmetic mean of the range, we'll need to modify the range of pacing values so that the arithmetic mean of the range gives us the arithmetic mean of the TPM that we want...phew. What it means is that the above pacing values need to be modified from 72 to 360 (arithmetic mean = 216) to 72 to 168 (arithmetic mean = 120). However, this gives us the TPM range of 20 to 8.6 TPM with a harmonic mean of 12TPM.

But I'll live with it. I would rather have the average load stay around 12TPM. So here are the new values. Note the asterisk on TPM. I need to mention in the test plan that the actual TPM will vary from 8.6 to 20TPM with an average of 12TPM.

Script	TPM*	Number of VUs	Pacing (sec)	Ramp Up
Script 1	4 to 20	24	Random (72 to 168)	1 VU/5sec

Why I'll continue to recommend and use LoadRunner instead of eLoad

I've raised this concern in meetings with Empirix and haven't heard this as being a priority. However, this is a major reason why I'll continue to recommend and use LoadRunner instead of eLoad:

Lack of ability to generate specified transactional load in eLoad and why it's very important

Background:
Web based (3 and n-tier) applications are different from client-server types of applications in terms of how they deal with user load (specifically how the system performs when users use the system) as explained below:

Load in web-based applications (3 or n-tier) is generally represented in terms of number of transactions expected per unit time (seconds/minutes etc) and not in terms of users per unit time. This is because in web/application systems, a user using the system can generate varying amount of load on the system depending on how active the user is. For example, a single user who submits 10 transactions every second will use much more system resources than 100 users who submit 1 transaction every minute (given the 'transaction' as defined is equal in both cases, and let's rule out caching as well). Even though those 100 users in latter case are using some fixed amount of resources on the system (for maintaining session information, other objects etc) that is 100 times more than that being use by the single user, significant resources are only consumed when the users are actively interacting with the system or waiting for response from the system.

So if while planning performance tests for an application, I get business requirements stating that X users are expected to use the system in a day, I have to work with them to further refine these requirements to specify what is the user activity profile (or scenario profile) and what is the frequency of their actions. For example, how many users will logon every hour/minute, how many will navigate to certain web pages or consume a service every hour/minute and how many will logoff every hour/minute. For simple scenarios, a transaction can be 1 to 3 steps - logon, stepA and logoff. Based on answers from previous questions, I will then refine the requirements to state how many transactions are expected to be executed every minute (or TPM). For complex scenarios, multiple transaction types may have to be defined - usertype1(stepA, stepB...stepX), usertype2(stepB...stepY) etc. In this case as well, I will have to use same questions to refine the requirements for each transaction type, i.e., for each transaction type, what is the expected transactional load in TPM or TPS (transactions per second). Then I'll go ahead and create the load test scenarios based on these transactional loads.

This kind of transaction based load specification is even more important in XML over HTTP/web services applications because there is no concept of user. Rather the system deals with requests (or transactions) that can come from a third-party web application, a custom client etc. that are generally referred to as service consumers. There is rarely a need to maintain session information and each transaction is idempotent (at least from the web/application server perspective). Note that the whole transaction may be non-idempotent but still be made up of a series of idempotent transactions. For example, a credit check service may be non-idempotent because it writes the number of inquiries to the user's credit profile and gives it a weight in calculating the user's credit score. But from the web and application server perspective and especially in test environments where limited test data is available, each transaction can be considered idempotent since we don't care about test user's credit score and need to generate a production load using limited data.

How the 2 tools handle (or do not handle) this:
Coming back to the point of why I prefer to use LoadRunner over eLoad in these scenarios (there are other reasons too but let's just focus on this one for now)...
Both eLoad and LoadRunner let me specify the iteration delay (or VU pacing in LoadRunner terminology, don't confuse it with VU pacing in eLoad which actually means think time!) that controls how long a Virtual User (VU) waits before starting the next iteration. However there is only one way to specify this in eLoad - from the time previous iteration ends (see Figure 1). This creates a problem because the time when next iteration will be started depends on how long the previous transaction took. For example, if you specify this delay to be 30 seconds and the previous transaction took 30 seconds, next iteration will be started after the end of 30 + 30 = 60th second. However, if the previous transaction took 5 seconds to complete, next transaction will start after 5 + 30 = 35th second. Now suppose you want to generate a transactional load of 10 TPM. You use 10 virtual users and specify the delay to be 55 seconds expecting each transaction to take around 5 seconds. This way you can have each of the 10 VUs submitting 1 transaction every minute, thus generating a load of 10 TPM. But when you run the load, server (or the application under test) gets busy and takes 30 seconds or more to return a response. eLoad's virtual users are still going to wait 30 + 55 seconds before starting subsequent iterations, thus reducing the overall transactional load by almost 30% (or 10 * (1/85) * 60 = 7 TPM as compared to required 10 * (1/60) * 60 = 10 TPM). But you really need to find out how the system behaves at production load of 10 TPM even at busy periods...do the requests keep queuing up and ultimately cause the system to become unresponsive or the system returns to stable state soon after! Well...hard luck, because eLoad is going to decrease the load if the system starts taking longer to return the responses. You can possibly add more virtual users to the scenario to increase the load when this happens but I don't want to have to sit and watch the load test for this to happen when I'm running a 12 hour test and am only allowed to run the tests in non-business hours. And I'm not even sure if I'll be able to calculate that fast how many users to increase/decrease everytime this happens.

Figure 1: eLoad VU Settings

LoadRunner gives 3 options in setting the iteration delay/pacing (see Figure 2):
a) As soon as the previous iteration ends
b) After the previous iteration ends: with a fixed/random delay of XXX seconds (In case of random delay, it lets you specify a range)
c) At fixed/random intervals, every XXX seconds.

Figure 2: LoadRunner VU Settings

So the above situation is handled very easily by selecting the 3rd option and choosing a delay of 60 seconds. In the above example, if the previous iteration took 5 seconds, it'll wait for 55 seconds and if it took 30 second, it'll wait another 30 seconds before starting next iteration. No matter how long the previous iterations take (as long as they are less than 60 seconds), it will always start subsequent transactions at specified intervals of 60 seconds from the start of previous transactions. thus keeping the load stable at 10 TPM. If I expect the transactions to take longer than 60 seconds, I can start with more VUs and increase the delay. For example, I can use 20 VUs and set the delay to 120 seconds. This will still generate a load of 10 TPM if I specify the ramp-up time correctly.

Conclusion:
So my conclusion is that I will use LoadRunner as much as I'm able to. In case you're wondering why my company has 2 load testing tools when buying 1 is costly enough, my team is under a different business unit that owns eLoad licenses because LoadRunner was considered too expensive. However, there is another business unit that has LoadRunner licenses and even though I had to go through a lengthy procedure, I got them to agree on letting us use LoadRunner and charge us for the usage.

---------------------------------------

Note 1: You can argue that load in real-life production scenario is never stable. But when it comes to defining the system's performance, I prefer to use multiple scenarios with increasingly different loads. For example, if the business expects about 10,000 transactions per day, considering a 10 hour business day this comes to 16.67 TPM. I will run load tests at 16 TPM (1x), around 40 TPM (2.5x) and around 80 TPM (5x) to give them numbers on how the system can be expected to perform if the transactional load varies from 16 to 80 TPM. I will probably also run some stress tests by running a background load of stable 16 TPM and then submitting a batch of multiple requests (100/200 etc.) to see how the system recovers in this case. Again, this will depend on business requirements and expectations. Also, If I really need to vary the load, I would rather use the random option in LoadRunner and vary the load but still keep the overall load stable.

Note 2: I am not implying that defining load in terms of number of concurrent users is not important. For some applications (e.g., citrix or remote desktop applications) it is the most important load defining criteria. Even for web based applications, you may want to find out how many users you can concurrently support before the server runs out of memory. This will help you determine when you'll need to buy extra hardware. But any commercial load testing tool has to support the ability to generate transactional load as well now that XML and webservices are becoming more and more common.

Note 3: Current eLoad version that I'm using is 8.10 and LoadRunner is 8.1.4

WebScarab

There have been times during LoadRunner scripting that I needed to see the low-level HTTP request that is being sent from my client (which I am using to record, e.g., a browser, or a custom client) to the server. Earlier, I used Ethereal (http://www.ethereal.com/) successfully but the problem is that it doesn't support SSL directly. So if the communication is over SSL, all I would see is encrypted data and there was no way to see the headers/data being transferred. This made me look for alternatives. I recently came across WebScarab and I wouldn't say it's free of bugs but I'm sticking with this tool for as far as I can see in future. Here's how I was able to solve some of the problems in LoadRunner scripting using this tool.

Solution 1: (SSL Intercept)
First things first, WebScarab proxy

is able to observe both HTTP and encrypted HTTPS traffic, by negotiating an SSL connection between WebScarab and the browser instead of simply connecting the browser to the server and allowing an encrypted stream to pass through it.

This is a major advantage. So I no longer have to hope that one of the test environments will not have SSL implemented and will let me observe the non-SSL HTTP traffic. Watching browser traffic was just as easy as starting the WebScrarab proxy and pointing the browser to the local proxy. It gives the options to intercept request and/or responses and lets you modify the requests in any way before passing them on the server. For a custom client, I can capture the exact headers being sent and add them to my web_custom_request:

    web_add_header("Cache-Control", "no-cache");
    web_add_header("SOAPAction", "\"\"");
    web_add_header("Accept-Encoding", "gzip, deflate");

And I can also copy the body if it's an HTTP XML post for example, to get the exact XML data being sent and put that in the body of the request:

    web_custom_request("SampleService",
        "URL={URL}",
        "Method=POST",
        "EncType=text/xml; charset=utf-8",
        "TargetFrame=",
        "Resource=0",
        "RecContentType=text/xml",
        "Mode=HTTP",
        "Body="…

and so on. See screenshot.


Solution 2: (Reverse Proxy/Act as a web server)
As it happened, the client I was using (for more details on this, see my previous post) had pre-configured options of selecting the URL. So I didn't have any way to point the client to the local WebScarab proxy. I looked through the help contents and found this:

WebScarab allows you to specify a "base address" for a Listener. The base address instructs the Listener to operate as a reverse proxy, and should be formatted as a HTTP or HTTPS URL. In this mode, it will act as a web server, rather than as a proxy server, and will construct the URL by concatenating the base URL and the path that appears in the request line. If the base URL is an HTTPS URL, it will immediately negotiate an SSL tunnel prior to trying to read the request from the browser. This is useful for the situation where you are using a custom HTTP-based client that does not support configuring an upstream proxy. Simply change the hosts file on the computer on which the custom client is running to point the site in question to the computer on which WebScarab is running on, and WebScarab will receive requests for the targeted website.

This meant that I could use the hosts file to point to the proxy and specify the base address in the proxy listener to intercept the requests. In a few tries, I was able to intercept the SSL requests over a non-local base address. Again, I could get the headers and body and use it in the web_custom_request. See screenshot.





Solution 3: (SSL Server Certificate)
Another client that I was recording my script against had a similar issue. The client didn't support pointing to an upstream proxy so I configured the hosts file to point to the listener proxy. However when running the client, it threw this exception:

{http://xml.apache.org/axis/}stackTrace:javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target.

I tried adding the WebScarab certificate to the keystore through Java Control Panel but no luck. After googling a little more, I came across this forum thread: http://forum.java.sun.com/thread.jspa?threadID=220329&tstart=165
Apparently, Java uses its default keystore and to use another keystore, it has to be created and provided in one of the arguments. So after importing the WebScarab certificates into IE trusted certificates and then exporting it into .cer, creating a keystore with the certificates and modifying the batch run file to add
-Djavax.net.ssl.trustStore= -Djavax.net.ssl.trustStorePassword=

I had my fingers crossed when running the client again. But fortunately, this time it worked as expected and intercepted the HTTPS requests without any errors. Once again, I used the custom headers and the XML body to create LR web_custom_header request.

LoadRunner function: web_convert_param

After spending about 6 hours trying to figure out a solution to the problem of parameterizing URL encoded strings in my LoadRunner script, I finally found the solution: a built-in function provided by LR.

The problem:
I recently recorded a script against an Java application that ultimately sends a HTTP POST request to the server being tested with an XML in the Body of the request. The script came out fine, except that the body (XML content) was URL encoded. It would have been fine if I didn't need to parameterize the data in the body but since I had to, I realized in a few seconds that I would have to do some extra work to convert the data from parameter file to URL encoded format. The data in parameter file is of the form:

Fname,Mname,Lname,Gen,SSN,A1,A2,A3,A4,A5,City,State,Zip
John,,Smith,,123456789,123,Main,St,#,88,TestCity,TestState,987654321

However, in the script, address is one field that is a concatenated string of A1 - A5 parameters with a space between each of them.

Further Research:
A1 - A5 are the components that make up the street address that is supposed to be concatenated to one field in the script. No problem.
sprintf(as,"%s %s %s %s %s", lr_eval_string("{a1}"), lr_eval_string("{a2}"), lr_eval_string("{a3}"), lr_eval_string("{a4}"), lr_eval_string("{a5}"));
lr_save_string(as, "addressStreet");

…but the problem is there are spaces in between this components, which we all know get converted to a '+' character in URL encoding. Ok, so I could've just used sprintf(as,"%s+%s+%s+%s+%s",…
But some of the addresses also have '#' signs that get converted into '%23'. Now I had 2 options:
1. remove all the addresses that have a '#' sign
2. write a URLEncode(char *) function that does the obvious.

1st option wasn't very tempting because however unlikely it may be, the addresses can have other characters that need to get URL encoded as well. And if a sizeable chunk of the data given to me has these characters, I could lose a lot of records. Also I didn't want to have to modify the data every time I get a new 10,000 record file.
2nd option seemed the way to go. But it was late Friday and I had to start the tests in a short time so I was lacking the much needed patience. Anyways, I ended up running my tests by removing the records with '#' character. Luckily, there were few.

Solution:
On Monday after running all the tests, I got back to writing the URLEncode function. After researching online on what the characters should get converted to and trying to find an already published function in C that I could tweak to my purpose, and stumbling on PHP, Java, JavaScript functions, I started getting a sense that LR may have some kind of built-in function that does that or something similar. I don't know why I hadn't thought of using Mercury Knowledgebase before that. So a simple search of "url encoding" brought up 3 articles - none of which seemed helpful. But reading through the 2nd one (KB Problem ID: 18880), I couldn't believe what I saw:
Pass the 'XMLSource' parameter as an input to the web_convert_param function, and store the result as 'TargetXML'
web_convert_param("TargetXML", "SourceString={XMLSource}", "SourceEncoding=HTML", "TargetEncoding=URL", LAST);

So this web_convert_param function seemed to do exactly what I needed. I tried it out with a couple of different strings and it did as promised. So my new script had:
sprintf(as,"%s %s %s %s %s", lr_eval_string("{a1}"), lr_eval_string("{a2}"), lr_eval_string("{a3}"), lr_eval_string("{a4}"), lr_eval_string("{a5}"));
lr_save_string(as, "addressStreet");

web_convert_param("encAddressStreet", "SourceString={addressStreet}", "SourceEncoding=PLAIN", "TargetEncoding=URL", LAST);
lr_output_message("%s", lr_eval_string("{encAddressStreet}"));

It took me only a few minutes to make that change. And I didn't need to worry about the '#' characters or any other unsafe characters in the address string. Phew…

Morale:
Use the Mercury Knowledgebase more often. If you think there should be any other, let me know.