Compression and minification of text resources

[cmb]

Hello Community,

I'd like to discuss the issue of compression and minification of text resources (e.g. HTML, CSS, JS).

With regard to compression I had a look at available possibilities, and it's simplest to let the webserver do that. For instance, for Apache 2 the following in a .htaccess in the web root might suffice:

Code: Select all

<IfModule deflate_module>
    <IfModule filter_module>
        AddOutputFilterByType DEFLATE text/html text/plain text/xml text/css
        AddOutputFilterByType DEFLATE text/javascript application/javascript
    </IfModule>
</IfModule>

This reduces the total size of the start page of an unmodified CMSimple_XH 1.6.4 from 41.2 KB to 12.9 KB in my environment. Pretty nice! The only requirements are that mod_filter and mod_deflate are available, what might be a problem.

A more portable alternative would be to do the compression with PHP, but I'm not sure how to handle that best. zlib.output_compression seems to be recommended, but that wouldn't play well with output_handler which might be used in some configurations. Anyhow, this would not work for JS and CSS files, for instance, because these are not delivered by PHP.

With regard to minification I'm even less sure what to do. For JavaScript it seems sensible to already deploy minified scripts. However, that's probably no option for stylesheets, as these should be editable from the back-end. And the generated (X)HTML can't be minified in advance, anyway. Searching the web I found loads of short PHP snippets which were supposed to minify HTML and CSS (for plugins.css), but they seem to be broken by design resp. relying on certain conventions. Tidy is probably no option, because it removes conditional HTML comments (IE). There are several libraries which would allow minification, but I can't comment on them, and I'm not convinced that there would be huge savings by minifying before compressing.

Any thoughts?

[meltemi]

Moin,

mod_deflate is not available on many servers. Forget it.

Use PHP compression for HTML.

JS and CSS: Upload two versions (compressed with 7zip / not compressed). Then code in .htaccess:

Code: Select all

#--- Vorab-Komprimierung GZIP für css|js: --------
<IfModule mod_headers.c>
<FilesMatch "\.css\.gz$">
  ForceType text/css
  Header set Content-Encoding: gzip
</FilesMatch>
<FilesMatch "\.js\.gz$">
  ForceType text/javascript
  Header set Content-Encoding: gzip
</FilesMatch>
</IfModule>
RewriteCond %{HTTP:Accept-Encoding} gzip
RewriteCond %{REQUEST_URI} \.(css|js)$
RewriteCond %{REQUEST_FILENAME}.gz -f
RewriteRule ^(.*)$ /$1.gz [L]

This saves time on server And changing CSS files in web interface is no more possible

CSS files, for instance, because these are not delivered by PHP.

Would be possible. Look here in Jörgs Forum (in German, wrong language for you, I know ).
But it's not my proposal. I don't use it, and I don't like it.

[cmb]

mod_deflate is not available on many servers.

Are you sure? AIUI mod_deflate would be necessary to uncompress incoming requests where the body is compressed. Anyhow, on my server mod_deflate is available, but mod_filter is not, but of course that isn't representative.

JS and CSS: Upload two versions (compressed with 7zip / not compressed). Then code in .htaccess:
[ code ]
This saves time on server And changing CSS files in web interface is no more possible

If I'm not mistaken, changing the files would still be possible, but these files wouldn't normally be delivered to the client. And anyway, removing the feature to edit the stylesheets from the back-end wouldn't please many users, I suppose.

Of course, there would be another option. Store the files compressed on the server, uncompress them before they're going to be edited, and compress them on saving. Nonetheless we would need a second set of uncompressed files for clients which don't accept compressed responses.

However, I don't like to have two sets of files on the server, unless the compressed files would be treated as a server side cache. Otherwise editing the files offline would be a PITA.

CSS files, for instance, because these are not delivered by PHP.

Would be possible. Look here in Jörgs Forum (in German, wrong language for you, I know ).
But it's not my proposal. I don't use it, and I don't like it.

ACK. That seems to be a doubtful workaround.

[meltemi]

mod_deflate is not available on many servers.

Are you sure?

I don't have any statistics. But I know 3 providers without mod_deflate in shared hosting: The first one begins with 1, the second one begins with s, and the third one is my provider.

removing the feature to edit the stylesheets from the back-end wouldn't please many users, I suppose

I prefer to test locally at first

CSS files, for instance, because these are not delivered by PHP.

Would be possible. Look here in Jörgs Forum [...].
But it's not my proposal. I don't use it, and I don't like it.

ACK. That seems to be a doubtful workaround.

That seems to be used often. But I don't like it because I prefer

JS and CSS: Upload two versions (compressed with 7zip / not compressed).

Changes in CSS and JS files are seldom, and I go this way. Delivery by server runs faster.

My last post in English, sorry.

[cmb]

I don't have any statistics. But I know 3 providers without mod_deflate in shared hosting: The first one begins with 1, the second one begins with s, and the third one is my provider.

Well, that's bad. Do others have experience regarding the availability of mod_deflate? Anyhow, we must not forget that CMSimple_XH is supposed to run on webservers other than Apache (it's known to be run on Nginx and IIS), so any .htaccess based solution would not cater to all users, anyway.

Changes in CSS and JS files are seldom, and I go this way. Delivery by server runs faster.

Yes, indeed.

My last post in English, sorry.

I think, it's okay to occussionally post in German in OD -- alternatively, you could open a (follow-up) thread in the German forum. Keeping the OD discussions mostly in English seems appropriate, however, because not all users and developers speak German, and English is the lingua franca (no pun intended) of web development.

[Bob]

Hi
"mod_deflate" is not available on mutualized servers by one of the biggest hosters in France, which turns out to be German
We have the same problem with "mod_gzip". I recently changed the hoster of cmsimple-xh.fr (and some other sites) for that reason, where only mod_gzip is available (but very effective with static contents).

[cmb]

"mod_deflate" is not available on mutualized servers by one of the biggest hosters in France, which turns out to be German
We have the same problem with "mod_gzip". I recently changed the hoster of cmsimple-xh.fr (and some other sites) for that reason, where only mod_gzip is available (but very effective with static contents).

So it seems reasonable to compress the generated HTML with PHP, and maybe to offer only some documentation about options to compress the assets.

[Holger]

It seems Bob & meltemi are right and .htaccess is not the option with the best compatibility .

So it seems reasonable to compress the generated HTML with PHP, and maybe to offer only some documentation about options to compress the assets.

What do you think about compressing JS & CSS with PHP too?
For example the combined plugins.css could get compressed with a few lines:
In XH 1.6.5, functions php, function XH_pluginStylesheet():
change line 1957 to:

Code: Select all

$ofn = $pth['folder']['corestyle'] . 'plugins.css.php';

and insert in line 2004 to the begin of $o:

Code: Select all

        $o = '<?php' . PHP_EOL
            . 'ob_start ("ob_gzhandler");' . PHP_EOL
            . 'header ("content-type: text/css; charset: UTF-8");' . PHP_EOL
            . '?>' . PHP_EOL

That will compress the contents from around 30kb to 2,6kb.
Of course that's not the finished code -- we have to make a check if extension gzip is available and add some code to cache the file on the client.
And it might be a good idea to turn of that compression by config, if other solutions are available.

Compressing the html could be made the same way in cms.php for example.

But some things are still missig: what about core.css and the stylesheet of the active template?
Maybe we find a way to merge those files to the combined plugins.css too? Or we give them their own request -- but compressed the same way?

Another question is how to handle external js scripts of plugins and the core.
Compressing will work the same way, but we should think about combining these files to a single request too.
At another topic we've started to discuss changes to the Plugin-API. I think we should extend the new API to get rid from writing script-tags to $hjs but adding the contents of the files to a combined js-file.

It might not be the "cleanest" solution to compress CSS and JS with PHP. But IMO the few extra turns for the CPU are acceptable if you see how much traffic could be saved to deliver the page to the client.

[cmb]

For example the combined plugins.css could get compressed with a few lines:

An interesting idea. Another option might be to save both plugins.css (as before) and plugin.css.gz (gzipped version of plugins.css)[1], and inject the appropriate URL into the <head>.

Compressing the html could be made the same way in cms.php for example.

I'm not sure if we could use ob_start('gzhandler'). It seems to me that would that would collide with our ob_start('XH_finalCleanUp'). Can these reasonably be nested?

But some things are still missig: what about core.css and the stylesheet of the active template?
Maybe we find a way to merge those files to the combined plugins.css too? Or we give them their own request -- but compressed the same way?

When plugins.css has been introduced I thought about combining all stylesheets, but to cater to different templates (mostly page specific ones), it seems it is better to keep the template's stylesheet separate. Unfortunately, this stylesheet is the second one, so combining core.css and plugins.css would not be possible without changes in the order of the rules.

Another question is how to handle external js scripts of plugins and the core.
Compressing will work the same way, but we should think about combining these files to a single request too.
At another topic we've started to discuss changes to the Plugin-API. I think we should extend the new API to get rid from writing script-tags to $hjs but adding the contents of the files to a combined js-file.

I would welcome a new API to get rid of writing to $hjs, $bjs, or the contents area, because IMO it's cumbersome to always add the `<script>// <![CDATA[` boilerplate, and because that would make it easier for the core to do optimizations[2]. However, besides that combining JS files could cause problems (consider "use strict" on the file level[3]), not all requests (think pages) need the same JS, so combining the files would need to be dynamic (what doesn't exclude caching, but it makes it more complex).

Anyhow, if we're going to deliver JS/CSS via PHP, we should consider whether dynamic PHP shall be allowed, for instance:

Code: Select all

.some_class {color: <?php echo $plugin_cf['someplugin']['color']?>}

I'm not a big fan of that, but others may be.

And we should think about the Content Security Policy. As this would forbid any inline JS, some form of combining scripts would be appropriate.

Another (quite contrary to combining) technique we should consider are dynamic JS loaders (such as AMD).

[1] Maybe even further compressions, such as deflate.
[2] Maybe it's best to make this new API a separate point on the roadmap.
[3] I'm not aware that anybody uses this, though.