JS performance and global eval() (attn: p01 and cb/adinpsz)
category: code [glöplog]
Quote:
153 bytes and time to go to bed.<canvas id=c><img onload=with(c.getContext('2d'))for(p=e='';t=getImageData(drawImage(this,p--,0),0,1,1).data[0];)e+=String.fromCharCode(t);eval(e) src=#>
NB: I'd prefer a bootstrap that sets a global variable to the 2D context of the Canvas so that both the Canvas and its context can be reused directly in the prod.
I did some benchmarks for comparing PNG compression with various parameters and there is something I do not understand. It's like the browser is rounding RGB values when PNG has an alpha layer. Let me explain the problem :
I created a 1x1 pixel PNG with following color : R=40,G=102,B=117.
Then I created a copy of this pixel but with an alpha value : A=110.
When I read the first PNG through a canvas data, I got the real color : R=40, G=102, B=117 (and A=255 of course).
But when I read the second PNG, I have slightly different values : R=39, G=102, V=115 (and A=110).
Here is an online test page :
http://boccato.carru.free.fr/rgbatest/rgbatest.htm
Does somebody have any idea about this ?
I created a 1x1 pixel PNG with following color : R=40,G=102,B=117.
Then I created a copy of this pixel but with an alpha value : A=110.
When I read the first PNG through a canvas data, I got the real color : R=40, G=102, B=117 (and A=255 of course).
But when I read the second PNG, I have slightly different values : R=39, G=102, V=115 (and A=110).
Here is an online test page :
http://boccato.carru.free.fr/rgbatest/rgbatest.htm
Does somebody have any idea about this ?
@cb: latest crome-dev here, same results. my js agnostic guess is that the rounding comes from an utterly bad alpha blending algorithm somewhere in the browser
Quote:
That's a side effect of browsers doing premultiplied alpha for you. ;) It may sound stupid but pretty much all browsers do that to optimize drawImage() calls which is supposed to be the most common use case.the browser is rounding RGB values when PNG has an alpha layer.
...
Does somebody have any idea about this ?
sweet
Oh and I just realized that the last bootstrap I posted are completely size agnostic. You can just append them as is at the end of your PNG and voila!
p01: So that's what it was when you talked about premultiplication :)
I looked into Webkit sources and their current premultiplication/unpremultiplication implementation follows the intuitive approach which can be reduced in JS to :
Next we find a simple index mapping function : it would map our 95 ASCII characters to the 95 values of the found range (which allows the function to be bijective).
And the resulting function has to take less than a few bytes, otherwise this solution would be useless :D
I looked into Webkit sources and their current premultiplication/unpremultiplication implementation follows the intuitive approach which can be reduced in JS to :
Code:
Now let me explain That gave me a crazy idea : because we only use the 95 ASCII graphic characters in a JS source file, maybe we can find a restriction of the previous function which would be bijective.// c = R/G/B [0..255]
// a = alpha [0..255]
Math.floor(Math.floor(c * a / 255) * 255 / a)
Next we find a simple index mapping function : it would map our 95 ASCII characters to the 95 values of the found range (which allows the function to be bijective).
And the resulting function has to take less than a few bytes, otherwise this solution would be useless :D
Well, it seems that PNG compression ratio is more or less the same with RGB or RGBA formats, so I guess my idea is definitely crazy :)
Yes.
Your idea is sound, but I wonder if the overhead of the mapping function would not outweigh the gain we get by using PNG32 over a grayscale PNG8. Consider me doubtful.
Quote:
Do we ? Non-printable characters are VERY useful to store data ( e.g.: a table of formant filters for a speech synth, coordinates for a mesh, ... ) or pack code.we only use the 95 ASCII graphic characters in a JS source file
Your idea is sound, but I wonder if the overhead of the mapping function would not outweigh the gain we get by using PNG32 over a grayscale PNG8. Consider me doubtful.
You're right but in case of deflate compression, repetition is the main problem. So I'm not sure that compressing binary data is a lot more efficient that compressing the same data in ASCII format, particularly if we don't have only binary data to compress (because in all cases there will be some javascript code with).
Daeken: BTW do you have an updated version of your Fl0wer demo built with the latest version of your framework ? It could be used as a reference for my tests :)
cb: Just posted what I believe is the current most highly packed version. It's only 4b smaller, at 894b, since I've been updating Fl0wer fairly often.
Woops, that version was broken. New version is working properly and it's 891b, so 7b saved.
fl0wer is 891 bytes packed, using a 163 bytes bootstrap, and clocks at 1119 bytes unpacked, with 445 bytes of shader and 674 bytes of setup+loop. mmmh.... that's way bigger than GL1K Cotton Candy which clocks at 842 bytes unpacked with 401 bytes of setup+loop. I know my setup code is dirty but it should be possible to come up with a clean setup code that's somewhere around 450-500 bytes thus saving ~200 bytes on fl0wer.
OK, looks like everyone else is waaay ahead of me on the tiny-intro-bootstrapping front, but at least I can finish what I started :-)
I've updated my pnginator script to optionally pipe the image through PNGOUT (which saves 77 bytes over zlib on Fabrik), and handle both multi-line and single-line PNGs via slightly-tweaked versions of p01's bootstrap snippets. So, it now works as a general-purpose JS-to-self-extracting-PNG packer, with particular relevance for 4K-64K intros.
Here's a pnginated build of Fabrik, at 3928 bytes...
p01: I couldn't get your snippets to work in their original form on any current browsers: the drawImage call returns undef, which isn't accepted as a parameter of getImageData (it throws an 'operation not supported' exception). I didn't try very hard to recover the bytes I lost from shuffling those calls around (and changing the eval to an indirect one), so there might still be some savings to be made there.
I've updated my pnginator script to optionally pipe the image through PNGOUT (which saves 77 bytes over zlib on Fabrik), and handle both multi-line and single-line PNGs via slightly-tweaked versions of p01's bootstrap snippets. So, it now works as a general-purpose JS-to-self-extracting-PNG packer, with particular relevance for 4K-64K intros.
Here's a pnginated build of Fabrik, at 3928 bytes...
p01: I couldn't get your snippets to work in their original form on any current browsers: the drawImage call returns undef, which isn't accepted as a parameter of getImageData (it throws an 'operation not supported' exception). I didn't try very hard to recover the bytes I lost from shuffling those calls around (and changing the eval to an indirect one), so there might still be some savings to be made there.
Oh! :\ Well, at worst, a simple 0| in front of the drawImage would do, but there might be a way to not loose any byte and get this to work.
Quote:
Since Fabrik's PNG is multi-line, I think this is only due to the fact that it's a single-line PNG. The disadvantages of multi-line were something else which I understood just after the party :)I've updated my pnginator script to optionally pipe the image through PNGOUT (which saves 77 bytes over zlib on Fabrik)
In fact I'm almost finished with a new version of the tool I used during the party for PNG compression, it can reduce Fabrik's size by a few %, I hope I may release it tonight.
gasman: your version of Fabrik does not work in Opera 12 :\
Sorry about the broken bootstraps. Work, life and my daughter keep me plenty busy these days. I'll try to squeeze some geeky time in the coming weeks.
Sorry about the broken bootstraps. Work, life and my daughter keep me plenty busy these days. I'll try to squeeze some geeky time in the coming weeks.
cb: Nope, the 77 byte saving was from a like-for-like comparison on my 4096x3 image. PNGOUT really is that good :-) Looking forward to seeing your PNG compression tool...
p01: Doh, it was failing on another bit of typecasting abuse (using e='' for a 0 in drawImage). Fixed now, at the cost of another two bytes...
p01: Doh, it was failing on another bit of typecasting abuse (using e='' for a 0 in drawImage). Fixed now, at the cost of another two bytes...
I finally had time to do a pre-release of my command line tool. I put it there :
http://boccato.carru.free.fr/jsexe_100.zip
Called "JsExe", it's clearly demo-oriented. It aims to take a standalone JavaScript source file and produce the smallest self-extracting HTML file, using the best combination of compression tricks. In the default mode, here what it does more specifically :
- It takes an JS file as input parameter.
- If it helps, the JS file is optimized using an improved version of Google Closure Compiler (with no line break, better float formatting, etc).
- File byte order is reversed or not (depending on whether it improves final compression rate or not).
- Similarly, it chooses the best PNG format, RGB or gray.
- Then the most efficient PNG optimizer tool (PNGOUT or another one).
- Then output PNG is stripped of CRC and IEND block.
- Finally the loader is appended to the output file using Daeken's trick. In the loader, V is the name of canvas element and C is the name of its 2D context, so that the JS code could use this variables for the display.
Using JsExe, Fabrik is reduced to 3794 bytes. I don't think JsExe does a better job than Daeken's tool, but it could be useful to have the entire compression process in an unique command line tool. Closure Compiler does a pretty good work and avoid many time-consuming micro-optimizations.
Some improvements still need to be made, for example multirow PNG support or customizable variable names. But feel free to test JsExe and give me feedbacks and advices :) Then I'll do a first release of it.
http://boccato.carru.free.fr/jsexe_100.zip
Called "JsExe", it's clearly demo-oriented. It aims to take a standalone JavaScript source file and produce the smallest self-extracting HTML file, using the best combination of compression tricks. In the default mode, here what it does more specifically :
- It takes an JS file as input parameter.
- If it helps, the JS file is optimized using an improved version of Google Closure Compiler (with no line break, better float formatting, etc).
- File byte order is reversed or not (depending on whether it improves final compression rate or not).
- Similarly, it chooses the best PNG format, RGB or gray.
- Then the most efficient PNG optimizer tool (PNGOUT or another one).
- Then output PNG is stripped of CRC and IEND block.
- Finally the loader is appended to the output file using Daeken's trick. In the loader, V is the name of canvas element and C is the name of its 2D context, so that the JS code could use this variables for the display.
Using JsExe, Fabrik is reduced to 3794 bytes. I don't think JsExe does a better job than Daeken's tool, but it could be useful to have the entire compression process in an unique command line tool. Closure Compiler does a pretty good work and avoid many time-consuming micro-optimizations.
Some improvements still need to be made, for example multirow PNG support or customizable variable names. But feel free to test JsExe and give me feedbacks and advices :) Then I'll do a first release of it.
cool, have to test it when i have time.
except i don't have windows. doh!
except i don't have windows. doh!
well done cb!
First public release of JsExe :
http://pouet.net/prod.php?which=59298
I tested it with some prods, including the recent "Cyboman 5", and its size is reduced from 4087 bytes to 3694 bytes. It's interesting to see that RGB format produces better results that Gray format, even if the loader is larger :
http://pouet.net/prod.php?which=59298
I tested it with some prods, including the recent "Cyboman 5", and its size is reduced from 4087 bytes to 3694 bytes. It's interesting to see that RGB format produces better results that Gray format, even if the loader is larger :
Code:
JsExe 1.0.1 - JavaScript demo packer
by Charles Boccato (cb / adinpsz), 2012
Process cyboman5.js...
* CC based / No reverse / Gray / No optimizer : 4681 bytes (53%) <-
* CC based / No reverse / Gray / OptiPNG : 3741 bytes (43%) <-
* CC based / No reverse / Gray / Pngcrush : 3734 bytes (43%) <-
* CC based / No reverse / Gray / PNGOUT : 3935 bytes (45%)
* CC based / Reverse / Gray / No optimizer : 4655 bytes (53%)
* CC based / Reverse / Gray / OptiPNG : 3726 bytes (42%) <-
* CC based / Reverse / Gray / Pngcrush : 3722 bytes (42%) <-
* CC based / Reverse / Gray / PNGOUT : 3923 bytes (45%)
* CC based / No reverse / RGB / No optimizer : 5568 bytes (64%)
* CC based / No reverse / RGB / OptiPNG : 3762 bytes (43%)
* CC based / No reverse / RGB / Pngcrush : 3755 bytes (43%)
* CC based / No reverse / RGB / PNGOUT : 3704 bytes (42%) <-
* CC based / Reverse / RGB / No optimizer : 5538 bytes (63%)
* CC based / Reverse / RGB / OptiPNG : 3747 bytes (43%)
* CC based / Reverse / RGB / Pngcrush : 3743 bytes (43%)
* CC based / Reverse / RGB / PNGOUT : 3694 bytes (42%) <-
Output file : cyboman5_mine.htm, 3694 bytes (42%)
interesting indeed.
were you crunching it with closure as described in your info about jsexe?
whenever i tried minifying my .js with closure i got problems. which was why i was using uglifyjs to pack it.
also couldnt use crush on js, would get infinite loop, didnt bother debugging it.
would be nice to have those different cruncher/packer options available on the JsExe toolchain though :)
and portability to linux/mac :p
were you crunching it with closure as described in your info about jsexe?
whenever i tried minifying my .js with closure i got problems. which was why i was using uglifyjs to pack it.
also couldnt use crush on js, would get infinite loop, didnt bother debugging it.
would be nice to have those different cruncher/packer options available on the JsExe toolchain though :)
and portability to linux/mac :p
Yes it used a patched version of Closure Compiler. Do you remember what kind of error did you have with CC ? Maybe it's due to some specific eval() or with() statements ?
AFAIK UglifyJS does not do heavy code refactoring like function inlining or dead code elimination, and I did not found any demo where CC does not give better results.
For now, if CC is bugued somewhere you can bypass it in JsExe by using the "-cn" parameter.
AFAIK UglifyJS does not do heavy code refactoring like function inlining or dead code elimination, and I did not found any demo where CC does not give better results.
For now, if CC is bugued somewhere you can bypass it in JsExe by using the "-cn" parameter.