Simply Testable Updates #110: Broken Link Checking Fixes; Performance++ Nearly Ready
October 8, 2014
|This is the online archive for the Simply Testable weekly behind-the-scenes newsletter.
Subscribe to get weekly updates on the latest changes and the newest planned features.
This is the 110th of weekly progress updates on the development of Simply Testable, your professional automated web frontend testing service providing one-click testing for your entire site.
This week I continued work on system performance with regards to making best use of the available server resources and fixed a bug in the broken link checker.
Broken link checking fixes
Our broken link checker has a preprocessor that runs immediately before a broken link test is assigned out to a worker.
The preprocessor retrieves the web page to be tested and compares the links found within the page to the results of previous pages within the same full-site test. Links that have already been checked are ignored. This significantly speeds up most full-site broken link tests.
Yesterday I discovered a broken link test that failed to be assigned out to a worker. Upon investigation, I saw that the URL of the page to be tested was not for a web page but for a MP3 music file.
It turns out the broken link checker preprocessor didn't handle that too well. Instead of finding links to be compared against past results, the preprocessor ran around in circles with a confused look on its face and failed rather gracelessly.
I've now updated the broken link checker preprocessor to only run if the URL of the page to be tested is actually for a web page. These changes are currently being tested locally and will go live once seen to be correct.
Performance++ nearly ready
The change two weeks ago to a different background task processing system resulted in some performance improvements. Last week I continued along this line of improvements with changes to how test tasks are farmed out to workers and this week I continued that same work.
Quick recap: our core application is the brains of the operation. It figures out what needs testing when a new test is started and farms out each specific test to our pool of workers. After receiving a task, a worker will carry out the specified test and will report the results back to the core application.
The core application periodically selects the next set of test tasks to be carried out and farms them out to workers. This process works but is not particularly efficient. It worked fine when originally created, continues to work but can be better.
I'm finishing up testing changes that allow the workers to request tests to work on. These changes have been nearly ready for a couple of days but don't quite yet work correctly. I'm currently testing, fixing bugs and re-testing.
I'll be completing the above-mentioned changes to speed things up a bit.
As always, if you'd like to see web testing you find boring handled automatically for you, add a suggestion or vote up those that interest you. This really helps.
Feedback, thoughts or ideas: email email@example.com, follow @simplytestable or keep an eye on the Simply Testable blog.