CMSimple needs a better plugin system

[bjorn]

The CMSimple plugin system is extremly wasteful. Whenever CMSimple is invoked all the plugins are included. Even when no plugin is need, because the user view a page that doesn't call any plugin, the pluginloader still includes all the plugins in the plugins directory.

It was perhaps not so important earlier but now plugins are getting bigger and bigger.

I think it would be better to develop some sort of Just-In-Time plugin loading. In other words CMSimple should only include a plugin when it is needed,

Bjorn

[Tata]

Not bad idea. CMSimple is an excellent CMS. Simple-small-smart. But plugins are really what extends its value extremly.
However, to check the content and decide which part of the content needs a plugin to be loaded is IMHO not that simple. Maybe the pagedata.php could be the source if hi_pdscripting would be standardly used.
Holger, what do you think?

[cmb]

Hello Bjorn, hello Tata,

ahh, an interesting topic. Okay, so what does the plugin loader do, when a CMSimple (I'll ignore additional steps required by _XH) page is requested by an UA? I'm not talking about the admin part, only the front-end:

/plugins/index.php is included
/plugins/*/index.php are included

So if there are 10 (!) plugins installed, 11 files must be read and parsed by PHP (I'm ignoring the PHP cache, which might not help to much on a hosted solution, but I'm not sure about that). I guess for typical plugins the total size will be several hundret KB total.

The CMSimple plugin system is extremly wasteful

IMO it is wasteful, but not extremely. Consider content.htm. I've read several times that content.htm > 2 MB should be avoided for performance reasons. This content.htm is also read everytime a CMSimple page is requested. And it's parsed not by the optimized Zend engine, but by simple PHP regexps!

But these are theoretical speculations. So I've made a test with CMSimple_XH (which has even more to do than the plugin loader for CMSimple or CMSimple LE; I've currently have only 2 plugins installed in CMSimple 3.3). I have 7 front-end plugins installed in my plugin dir. I've extended the template with

Code: Select all

<?php echo microtime(TRUE)-sv('REQUEST_TIME'); ?>

I've requested several different pages with results between 0.3 and 1 seconds. Then I've switched to a non-existent plugin directory, and tried again. I had results between 0.04 and 0.9 seconds. I guess the plugin inclusion makes for an avarage overhead of about 20-30% for a content.htm of 7KB!

Then I tried again with a content.htm of 2.3 MB (343 pages). Without the plugins I've had typical response times from 0.7-1.2 sec. With the plugins it was 0.7-1.5 secs.

My conclusion: the overhead of many plugins is noticeable, as is the overhead of a very large content.htm. At least on my local machine. It is an Intel Atom 1.6. I'm quite sure that a typical webserver will only need a small fraction of the times I've measured (say 5-10%, so we're talking about <100ms). IMHO this could be ignored in comparison to delayed responses from the server and network transmission times.

Another more practical problem is, that all plugins are written to be treated by the plugin loader that way. So if JIT-plugin-loader solution would be developed, it will only help, if the plugins where updated to make use of it. But the current solution of the plugin loader is a very simple and clever one. It reads all necessary PHP files, so that all functions used by templates or content are available immediatly, without requiring special hooks and registrations, so that the loader knows, when to include the files (which would not work for templates, at least the existing ones, which need e.g. newsrotator).

So is there a way to avoid too much overhead by newer plugins growing bigger and bigger without avoiding compatibility problems with older plugins or older plugin loaders? Yes, IMO it is. The only functions that must be defined always, are those that may be used from templates or content. And those functions can be simple stubs, which include the main code only when called, e.g.

Code: Select all

function genizforum($name) {
    global $pth
    include_once($pth['folder']['plugins].'genizforum/genizforum.php);
    return real_genizforum($name);
}
 

It is possible to write conforming plugins using such stubs right now. The author of the plugin is responsible for this. This way the only overhead of including the plugin when it is not needed, will be the inclusion of the index.php which has a size of say 1KB. The rest of the plugin's code will be loaded on demand (=JIT).

The situation is somewhat different for the plugin's administration, but that's a) not too important, because it wont be requested as often and b) it could be solved in a similar way.

Christoph

[Gert]

Hallo Bjorn, you are right, but:

if we would change something in Plugins System or Pluginloader, many plugins would not work anymore correctly. Many of plugin developers would be very busy, and there are many plugins not under development at the moment.

But I remember, you are loving the old plugins, sleeping in the Wiki ...

[cmb]

Hello Gert, hello Bjorn,

my suggestion will be even a solution for those old plugins. To use it:
1. check that you have permission to do (license!)
2. rename index.php to e.g. real.php
3. create a new index.php with stubs for all "public" functions into it
4. enjoy

For simple plugins with one or two public functions point 2 and 3 will take about 5 min. Point 4 might take much longer

Christoph

[bjorn]

Hi all

Thanks for the comments.

@Tata
I agree, the plugins are extremly important.

@Christoph
The pluginloader does not only include the index.php files but also all the config, css, and language files. Actually I'm surprised to see that you test only shows very little that very little extra time is spend. However it must slow down even a fast server that simultaneously respond to many requests and it must also swallow a lot of its memory.

Well, perhaps I'm a little to conserned about these tings. But you see, back in the good old days I was programming in assembler. So I'm in the habit of counting the bytes

@Gert
You and Christoph have a very important point. A new pluginloader should of course be able to handle all the existing plugins. Or at worst only demand very few and very small changes to the old plugins.

But I think it is possible solve this.

In the front end previously loaded/included plugins are called by CMSimple scripting like this: #CMSimple $output .= main-function-name();# I think most plugins use the same name for the plugin, the directory in "/plugins", and the main function. If this is true, it is not difficult to include the necessary files from the plugin directory JUST-IN-TIME before the main function is called. But perhaps it might be a little more difficult do something similar from the back end.

Well, just my two cents worth

Björn
http://www.cmsimple-le.eu

[cmb]

Hello Björn,

I agree, the plugins are extremly important.

+ 1

The pluginloader does not only include the index.php files but also all the config, css, and language files.

Good point! I've overlooked that. But at least the css will not be read by the plugin loader. It will just be send to the browser, which will use a cached version on further requests. The config.php and LANG.php are not too big in typical cases.

However it must slow down even a fast server that simultaneously respond to many requests and it must also swallow a lot of its memory.

Well, my test on a local machine isn't actually representative. I'll try that on a hosted service ASAP. But I encourage you (and other developers) to make those benchmarks for yourself. Every webserver might have it's very special results.

But you see, back in the good old days I was programming in assembler. So I'm in the habit of counting the bytes

I wouldn't call those days good days, at least not for a progammer But I understand your point. I was very surprised when I found that CMSimple reads and "parses" the whole content.htm at least once for every request, and that it didn't take too long.

In the front end previously loaded/included plugins are called by CMSimple scripting like this: #CMSimple $output .= main-function-name();# I think most plugins use the same name for the plugin, the directory in "/plugins", and the main function. If this is true, it is not difficult to include the necessary files from the plugin directory JUST-IN-TIME before the main function is called. But perhaps it might be a little more difficult do something similar from the back end.

This case could be handled IMO, if the genuine CMSimple scripting isn't used. But consider it's complexity if it is used to change a page heading:

Code: Select all

#CMSimple $output=preg_replace('/(<h[1-3]>).*?(<\/h[1-3]>)/i', '\\1New heading\\2', $output);#
 

Is there a plugin call involved?

I've seen several plugins using different public calls for different purposes. So at least those plugins must register these calls for the plugin loader in some way.

And what about the template.htm? This is currently include()'d. That's simple and fast. But if the plugin loader will include plugins JIT, it has to be parsed by the plugin loader. AFAIK this is not really possible without writing a own PHP parser, or require a PECL extension.

If performance is really a concern, another possibility might be to use cached output of a page, i.e. what is send to the browser, will be written as pure .html to a separate cache folder. That's what some other CMS's do. If the cache is up to date, it could be immediatly send when requested; otherwise it has to be rebuild by the CMS. That might be simpler than including plugins just in time, and probably even faster. But--is that really necessary?

CMSimple has worked fast enough many years ago. Even when new plugins make greater demands, this should be catered by Moore's law. And at least the suggested stub solution should reduce the amount of work a server has to do, even if extensive plugins are involved.

Christoph

[Gert]

Hallo,

the "stub solution" we have tested yesterday (Martin and me) with FotoGalerie_XH - nice idea

The loading of css file the plugin developer can avoid by renaming the css folder and load the css file by hjs. Then the css file is added to the head section only if plugin call is active. But then it is not editable online in plugins backend.

The plugin developer can do many things without changin the plugin system - most important maybe the "stub solution"

Al in all, the most is peanuts. If the user includes a image 2000 x 1500 px and shows it in 200 x 150 px - All our efforts are for nothing ...

[cmb]

Hello Gert,

the "stub solution" we have tested yesterday (Martin and me) with FotoGalerie_XH - nice idea

The loading of css file the plugin developer can avoid by renaming the css folder and load the css file by hjs.

And the CSS has nothing to do with server side performance. If read once by the browser, it will be reused from it's cache (except when caching is turned off clompletely, which is IMHO not the best solution).

If the user includes a image 2000 x 1500 px and shows it in 200 x 150 px - All our efforts are for nothing ...

+ 1

Christoph

[snafu]

Al in all, the most is peanuts. If the user includes a image 2000 x 1500 px and shows it in 200 x 150 px - All our efforts are for nothing ...

i think, this is the real problem!
i found more than one CMSimple Installation (or wordpress, joomla, dokuwiki, etc,..), with 2,3,4 pictures, rescaled with html width/height statements, a header/logo and Backgroundpicture and loading speed like a Snail in the hibernating. uncompressed jpgs and big pngs ... 3-4 megabyte unnecessary dataspam for a startpage with 1-2 kilobyte text, and some small pictures.... thats the reality

anyway, speed optimation is always a good thing