Description

This is being written to describe both the Prolink Speed Test application as well as the overall process that allows the results to be distilled into percentiles for comparison. The main goal of the application is to address customer complaints of “slowness” within Prolink’s software products. Since “slow” is a subjective term, we need to quantify exactly how fast things are running on a customer’s network server and/or SQL Server in order to find out if the issue is with the customer or with our software. Prolink Speed Test Application

Prolink Speed Test is a simple application that does a few standard tests so we can compare the tests vs. other customers. As of QC-CALC 4.0.63, Speed Test is already installed on the PC along with our software and is accessible via the Help menu. The idea is to run the tests and compare them vs. other customers. There are three tabs on the screen. The first tab is the testing screen. Speed Test Screen

The Speed Test screen is where the testing is performed. When launched, the application attempts to download the latest benchmark ranges from the Prolink website. If successful, a green label will be displayed saying “Latest SpeedTest ranges were downloaded. Ready for comparison”. If the download from the website is not successful, the application will still look into the folder to see if an existing range file exists. If it does, a yellow message will appear stating, “Download failed, but SpeedTest ranges were found. Using available ranges.” If no file is found, a red message will appear stating, “No SpeedTest ranges found. Comparison disabled.” In all cases, the speed test can still be performed. However, comparison colors in the results list as well as Population Comparison lines will only appear if there is a range file present.

To run the test, the customer should provide a folder path on the network (or mapped drive) where s/he has read/write access. If running QC-CALC, the admin path would be a good example to try.

After entering a path, enter a database connection. The application automatically detects other Prolink software installed on the same PC and adds the database connection paths to a drop down list. Choosing one from the list will automatically use the same connection as the application chosen. Alternatively, you can specify a connection manually to either a QCQ path or SQL Server database.

At the bottom, you can then optionally choose to run the test once or multiple times with an interval (minutes) between runs. For instance, it is possible to run it 10 times with a 6 minute interval which means it will run for an hour.

When ready, click Start Test to start the run. The tests will be performed and timed and the results will begin to appear in the window. If there is a ranges file found, the results will be colorized according to where they fall in comparison to other customers. Results in the top 40th percentile of other users are colored green, results that are between the top 60th and top 40th percentile are colored yellow, and values in the bottom 40th percentile are colored red.

When a test is run, the results are saved to the main repository path (c:\ProgramData\Prolink\Prolink Speed Test 4.0\) as speedtest[yyyyMMddhhmmss].log files. Each test gets its own time stamped log file. These log files have all the metrics of the database as well as the workstation name, time of the test, and test results.

The following tests are run and timed:

1. File Write - A 5Mb file is copied to the network location.
2. File Read - The same 5Mb file is copied from the network to the local path.
3. Database Metrics (SQL Server only) - The metrics of the database are gathered (file count, record count, etc).
4. Database Connection - The connection to the database itself is tested. For SQL Server this happens immediately after file copy. For QCQ, this happens after creation of QCQ.
5. Part File Conversion - Sample100.qcq is converted into the database path as a [GUID].speedtest. If QCQ, then it will have a .qcq file extension. This file is a slightly modified Sample100 in that it has 5 trace fields (2 numeric and 3 text). Otherwise, it is the normal Sample100 containing 100 characteristics and 100 records.
6. Lastx Query - The last 50 records are queried from the database to mimic a typical QC-CALC RT screen’s worth of data.
7. Part File Deletion - The part file is deleted from the database in case of SQL. QCQ file is deleted in case of QCQ.

Comparisons against percentiles are based on database type (QCQ vs. SQL) and database size based on arbitrary size ranges. As of this writing the following sizes are assigned based on the number of records in the part table.

Time Comparison Screen

The Time Comparison screen allows you to compare multiple speed tests on the same PC. This is especially handy if you run the interval test with multiple runs. The screen automatically points to its local repository path (c:\ProgramData\Prolink\Prolink Speed Test 4.0) but can be redirected to a different folder if desired. When a new folder is chosen, a unique list of workstations is found and loaded into the drop down list. Choose the workstation from the drop down list and select Compare Runs.

The runs will be loaded by date/time and displayed in the grid. To colorize, the list of results for each test are averaged and a standard deviation is calculated. Each result is then compared against the standard deviation to place it somewhere in the bell curve. Values within 1 standard deviation of the average are gold. As the numbers get higher, they become closer to red in the spectrum since the results are relatively slower. Likewise, lower numbers are relatively faster and therefore trend toward green.

It is important to understand that the colors are not a judgement of good or bad despite the green and red coloring since they are simply relative to the other runnings on the same PC. However, it can identify times of the day where the workstation runs slower due to network and/or database traffic, etc. The benchmark average is also added to the first row as a comparison. If the benchmark average is red, it means the current average is above (faster than) the benchmark of other customers. Likewise, if the benchmark is green, it means the current average is below (slower than) than other customers.

The results can be saved to Excel (assuming present on PC) for later comparison. Log Comparison Screen

The Log Comparison screen allows you to compare logs from different PCs. This can be handy if one workstation is on an older network link and is experiencing slower performance than others. The same math applies as the Time Comparison with the average and standard deviation of each test and the colorizing of the grid cells based on their distance from the average. Again, this is not necessarily measuring good or bad. The goal is to compare relative scoring of various workstations on the network. For instance, if the average 5Mb upload speed is .03 seconds and one of the PCs is 1 second, it will be colored red but it still a very fast upload speed overall.

If multiple tests from the same workstation are found in the folder, they are averaged together before being compared to the other workstations.

This screen will most often be used by an Implementation Specialist during a customer implementation. The test would be run on each PC and the results copied to a memory stick or central network folder. The folder path on this screen can then be pointed to the memory stick and the results would be compared to look for outlier workstations.

The results can be saved to Excel (assuming present on PC) for later comparison.