Software Testing Social Network

Customizing test scripts
Because load tests involve multiple virtual users, customization of recorded test scripts is required in nearly all load test scenarios. However, script customization is also necessary when the script recorder cuts off client logic that is required to fully emulate a client application.
Additionally, to increase test coverage you may want to add content verifications to recorded test scripts.
The more automated the script customization process is, the more productive a user can be.
TrueLog Script Customization
TrueLog Script Customization supports visual script customization using point and click; the tester can work directly on the visually rendered script execution log (TrueLogM), thus making it easy to understand what needs to be customized.
TrueLog Script Customization mainly involves the following activities:
• Visual analysis of output-input correlations
• Visual user data customization
• Visual content verification

Visual analysis of output-input correlations
When the script recorder cuts off an output-input correlation that is required to fully emulate a client application, you can use SilkPerformer’s TrueLog Explorer to add the correlation to the script via a simple point and click process.
A typical example of such a scenario is a transaction that inserts a new dataset into a database table. To do this, a unique primary key is required, which the client application usually gets by reading the last used key from a table and increasing the value by one. Then, the client application uses the key to insert the new data. Of course, if you simply record and replay such
a database transaction you will receive a session error at replay because the key is no longer valid, as it was already used during recording.
Using SilkPerformer’s TrueLog compare mode, you can automatically detect and customize such output-input correlations that are statically included in test scripts. This means that cumbersome manual scripting to resolve such situations is no longer necessary.

Visual user data customization
TrueLog Explorer’s customization wizards allow you to easily customize the data that virtual users provide to simulate the varied, random nature of real-world user data input. Using point and click operations on rendered visual transaction logs—which visually show the actual results
of test transactions rather than simple visual script representations—random values can be used in place of initially recorded static values. The data for these record fields are bound to table elements of selected comma separated value (.csv) files .

Visual content verification
TrueLog Explorer’s customization wizards allow you to create content verification checks via point and click operations. All customization is done on rendered visual transaction logs, which allow you to define your verifications based on actual results from test runs. All verification checks are then performed during load tests—with no significant performance impact.

Optical Character Recognition for Citrix
To support the processing of session-dependent, text-based data in Citrix testing scenarios, SilkPerformer provides Optical Character Recognition (OCR) capabilities for text displayed in the Citrix client GUI.
With OCR, you can easily add reliable synchronization points with text-based updates on the screen, parse session information for reuse in the script (for example, user logins) and add textbased functional verifications, such as checking to ensure the user is correctly logged in. These synchronizations are all independent of any screen settings on the client. For example, when
the screen display changes slightly but the text to be verified stays the same, the script will still run and will remain synchronized with what happens on the Citrix ICA client screen.
Manual script customization
SilkPerformer uses BDL—its own powerful, yet easy to learn and use, scripting language. BDL is particularly well-suited for load and stress testing purposes and provides complete control over user activity. A set of powerful functions and full flow control mechanisms allow for easy customization of script parameters.
Moreover, SilkPerformer’s code completion-enabled script editor simplifies and accelerates manual scripting significantly as it brings BDL API knowledge directly to the scripter's fingertips.
Data such as function arguments and short descriptions of functions are displayed while editing scripts with SilkPerformer.

Project attributes
Project attributes provide a powerful means of enhancing test scripts and making them more flexible. They allow parameters to easily be passed into scripts, enabling users to adapt their scripts to new test environments without the need for changes to the BDL code. This means that load testing scripts can be re-used in a much broader context again making for a more
productive and efficient use of test resources/personnel.
Customizing error handling
SilkPerformer offers the possibility of redefining (overwriting) the default severity of errors. This is useful in scenarios in which SilkPerformer reports an error that is known, but is not considered to be important (for example, during development). Such an irrelevant error could have its default severity overwritten with a lower severity, for example “informational.”
Consequently error counts are not increased when such errors occur. The converse is also possible, where the severity of an error needs to be increased (e.g., raising an error that results in test termination from a severity level of “error” to “severe error”).
SilkPerformer’s scripting language also provides a powerful exception handling technology that allows for the handling of various events such as errors, warnings and transaction ends to be managed on a global, rather than individual, basis.

Defining a load test workload
Baselining
SilkPerformer enables you to automatically set meaningful threshold values for your page and transaction timers. Thresholds are derived from the baseline performance of the application under test, which reflects the ideal performance of the application. Therefore, interpreting load
test results is much easier—simply determine what percentage of virtual users experience acceptable response times (measured against the established thresholds).
Thresholds can later be re-used directly in SilkCentral Performance Manager to verify that performance benchmarks are still met in the production environment.
Guided workload set-up (Workload Wizard)
SilkPerformer’s Workload Wizard helps facilitate a workload parameter set-up that meets your needs. Based on average session times measured by baseline tests and throughput information from real-world applications, the Workload Wizard automatically calculates the number of virtual users required to simulate the expected load the application will support in production.
Choosing the right workload model Because the behavior of server applications varies between load scenarios, SilkPerformer
provides different workload models to test applications under varying load and stress conditions.
Using the SilkPerformer Workload Wizard, you can easily define multiple workloads and save them for later use.

Workloads with guaranteed transaction rates
SilkPerformer also provides transaction rate goals for the following workload models:
• Increasing
• Steady State
• All-Day
• Dynamic
Using dynamic think time adaptation, you can specify workloads based on transaction rates that SilkPerformer will meet even if server response times slow down as load increases. For example, if server response time slows, SilkPerformer can automatically reduce think times to avoid an increase in session times, which would directly result in a decrease of the throughput
rate.

Test Execution
Once real-time monitors have been configured, you can begin load tests at specified times. SilkPerformer then automatically executes tests and collects results.
Real-time monitoring
SilkPerformer’s real-time server monitors can provide a live view of the entire system under test, enabling you to quickly find and eliminate performance bottlenecks by correlating response times with back-end server statistics. Setting up these monitors is easy. The Server Analysis Module™ (SAM™) automatically detects what types of data sources are available on the
system to be monitored. To simplify access to remote machines for monitoring, login credentials can be entered via SilkPerformer’s user interface.
SilkPerformer also provides a live view of virtual user response times. This enables you to react quickly to response time issues during load tests, saving you time and simplifying tuning operations. 

Code-level performance monitoring
SilkPerformer integrates with best of breed code-level diagnostics solutions through an open interface to facilitate analysis of performance bottlenecks that exist deep within J2EE or .NET application tiers.
Data from these tools are displayed in real-time within SilkPerformer Performance Explorer through standard JMX or SNMP interfaces.
Accurate Client Emulation To support high-scalability load tests, load testing solutions such as SilkPerformer emulate clients, rather than driving them (thin-client vs. fat client load testing). This is especially true for
load testing Web applications. With Web applications, the load testing solution must accurately emulate the Web browser’s connection behavior, cookie management and caching strategies on a per-user basis. SilkPerformer opens as many concurrent HTTP connections per virtual user as a user’s real browser would and doesn’t reuse connections across virtual users. To
guarantee realistic simulation for returning Web users, SilkPerformer provides a persistent cookie database. Plus, its TrueCache technology also provides a separate cache for each virtual user, accurately emulating conditional requests. Additionally, SilkPerformer’s TrueModemTM technology complements its unrivalled accuracy in simulating modem speeds (for example, ISDN, Cable, DSL and others) by continuously throttling network throughput—not by simply inserting wait periods.

TrueBehavior
Real users will not wait indefinitely for Web pages to load and SilkPerformer’s emulated Web users will not wait, either. Based on adjustable user tolerance levels, TrueBehavior allows you to specify time-out values and retry attempts for the loading of documents, embedded objects (such as images) or entire pages. Thus real-world cancellations of long running Web page
requests and retries can be simulated correctly.

TrueScale Replay engine
SilkPerformer uses its replay engine to execute scripts and run virtual users. The more scalable and powerful a replay engine is, the more virtual users it can run concurrently while requiring fewer agent computers to execute tests. Scalability depends on footprint per virtual user, which includes RAM and CPU usage.
Particularly in enterprise load testing scenarios that must simulate high numbers of concurrent virtual users, a highly scalable replay engine is required to keep load testing infrastructure costs down. SilkPerformer’s powerful TrueScale load testing engine actually makes it the most
scalable load testing product on the market; it works without performance or scalability loss even when utilizing IP spoofing, NTLM authentication and SOCKS proxy replay.

Dynamic agent load balancing
The number of concurrent virtual users each individual agent computer can handle depends on an agent computer’s processing power and memory. To simplify load test configuration, SilkPerformer automatically balances generated load across all agents, while taking the power
of each agent into consideration. Even in cases where the number of concurrent virtual users changes (for example, during increased or dynamic load) the load is still distributed out evenly to all agents involved in the test.

Load test health control
Incorrect test results lead to ill-informed decisions, which can in turn lead to business interruptions when applications are deployed with unrealistic performance expectations. If left unrecognized, invalid load test results caused by overloaded agents can be a serious problem.
That’s why SilkPerformer monitors the health of all agents involved in load tests, making it easy to have confidence in the validity of load test results.
Reliable error detection

SilkPerformer provides the most reliable error detection method available as it not only detects system-level errors, but also application-level errors through its advanced content verification under load technology. Examples of such errors include incorrect text on Web pages, incorrect values computed and displayed, and application-related error messages such as “Servlet Error” and “Server Too Busy.”

Efficient result generation
To support large, extended, load tests SilkPerformer uses an efficient approach to results generation. Rather than saving each timed event, SilkPerformer uses adjustable intervals for the writing of results files (time series data (TSD) files). This reduces disk operations considerably.
SilkPerformer’s TSD format packs results data into small data packages. Additionally, all data results sent from agents are merged before they are sent to the controller. These techniques lead to relatively small results files and a fast results-data merge process.

continues in next page..