XH 1.6: Linkchecker overhaul

[cmb]

Hi Holger,

thanks for testing.

But I've noticed that redirected pages are now reported to check by the user.
For example say you have a sub-navigation with a list of internal links in a newsbox.
The links are redirected to pages like "Impressum"...

Indeed, I have not thought about the redirection of internal links. However, there should be a general improvement regarding redirects. It seems to me that it's not a bad idea to hint that there's a redirection (for instance, see the change from cmsimple-xh.com to cmsimple-xh.org), but the redirects should be followed and checked anyway.

Before elaborating on this variant further, it would be good to know, if the performance is good for many internal links (I suppose that external links are checked faster, but internal links take longer). So it would be nice to have some more real-world tests; I encourage everyone to download the plugin and compare it with the built-in link check, and to report the results.

Christoph

[cmb]

I have added the following of redirections to the link checker draft. It is configurable whether any redirects should be reported as hints or simply suppressed. Furthermore I have added appropriate background colors for the errors resp. hints.

Please download and test the plugin and give feedback: http://3-magi.net/downloads/Linkcheck_XH.zip. The plugin works with CMSimple_XH 1.5.7 (and probably older versions and with CMSimple 4), so you don't need to download and set up a snapshot of CMSimple_XH 1.6.

Thanks in advance for helping improving CMSimple_XH.

[cmb]

Any test results?

[bca]

Could not get it to work in 1.5.7.
Froze the whole site and had to delete the plugin to unfreeze it!!
B

[cmb]

Thanks for testing.

What did you mean with "Froze the whole site"? Did you always get a blank browser window, or did the request actually hang (i.e. never return)? Have you enabled debug mode?

[bca]

I installed the plugin.
Logged in and selected Linkchecker in dropdown menu.
Screen opened which just said Linkchecker with three menu options.
Waited but nothing happened.
Clicked on menu option Main Settings(?) the left hand one.
The site then stopped and couldn't access any pages. Page eventually came up which said that there was an error in the script and the site froze. I deleted linkchecker using ftp and a few minutes later the site came up again.

in the interests of progress i just enabled debug, reinstalled linkchecker and tried again. Same result and debug no error messages referring to Linkchecker

[cmb]

Clicked on menu option Main Settings(?) the left hand one.

Yes, that's the way it's meant to work (for testing purposes).

The site then stopped and couldn't access any pages. Page eventually came up which said that there was an error in the script and the site froze. I deleted linkchecker using ftp and a few minutes later the site came up again.

in the interests of progress i just enabled debug, reinstalled linkchecker and tried again. Same result and debug no error messages referring to Linkchecker

I guess an Apache worker process/thread did still hang in the background, so the server refused further requests. I'll review the code later today, and try to improve the timeout handling.

[cmb]

I investigated some more about the issue, and now I'm not sure, if the behavior was caused by a timeout, or rather a congestion of the server and possibly a preventive mitigation thereof. As it's now the LinkChecker opens a HTTP connection for every unique link simultenously, what might easily be regarded as a (D)DOS attack by the server.

RFC2616 states that a client should not maintain more than 2 connections with any server or proxy (modern browsers seem to use some more, though). Heeding this advice would increase the complexity of the LinkCheck quite considerably, or otherwise substantially slow it down (what is likely to bring back timeout issues or at least unresponsiveness of the check).

However, to make at least some progress, I have modified the latest version of Linkcheck_XH to use the curl_multi_* functions (these are less error prone than the manual way and restrict the simultenaous connections to at most 10). The new version requires PHP 5 and the "curl" extension to be installed (a small system check is included). Please download, test it and provide feedback. (even if it's just that you can't test it, because one or both of the requirements fail)

A completely different strategy might also be considered. Basically this is employed by the Broken Link Checker Wordpress Plugin. The idea is to run the checks in the background one by one while one's logged in as admin; all broken links are gathered and can be viewed later, when all checks are finished.

[oldnema]

Download link does not exist

[cmb]

Sorry, Josef, I've forgotten the file extension (.zip). It's fixed now.