pouët.net

Quite some hack you have pulled off there, hitchhikr, with the absolute offsets into d3dx9_30.dll. I suppose it does work across different systems, since that DLL is always the same, but it is a bit bold anyway. :)

I took a look at your PE header (thanks for sharing the code - very interesting) and noticed that the DLL Characteristics field contains a jmp instruction. This field has some new bits added for Vista (in particular the lowest 4 bits) which are looked at by the Vista loader. The change that made Crinkler Vista compatible was to make sure DLL Characteristics was 0.

I also seem to remember that the Reserved field must be zero, but this might be a side effect of the way we overlay the import table on top of the header.

As far as I can see, your code should be DEP safe, as the entry point is placed inside the loaded section, rather than inside the header itself (which is not executable when DEP is enabled).

rbz: i just tested himalaya on my laptop (vista 32bit) and it crashes, so i have updated the himalaya archive with an executable compressed with the latest version of my 1k compressor. Let me know if it solves your problems as well.
Beware that the browser might cache the file, so make sure you get the new 1003byte one.

hitch: The zlib hack is certainly interesting, but as you are per definition limited to zlib compression, it doesn't seem like the most promising approach.

btw, which compressor are you using to create the zlib data? As zlib compression leaves a lot of decisions to the compressor, it might be worth trying out a lot of alternatives, or better yet, roll your own zlib compatible compressor :)

Do you have some numbers comparing this technique with plain crinkler and cab dropping?


I'd like to think that we are about a bit more than just compessors. For one, we use the same compressor for 4ks as pretty much everybody else, it's called crinkler 1.1a.

So really, you got it all wrong, it's everybody else who have an advantage over us, as they effortlessly get the same compression and compatibility for 4ks as us, without having to worry about the gritty details. In the case of 1ks, we are on a true level playing field, as everybody, at least for now, will have to roll their own tools, just as we did :)

I have actually talked to people who think less of our 1ks because we use our own tools, as they seem to be much less of a feat, when you apply magical compression. This is actually one of the best arguments for eventually releasing the 1k compressor, as then we could maybe at least get credit for the intros themselves.

Big cudos to hitch for doing stuff instead of just whining.

Blueberry, hitch: just a thought: what about 64-bit windows versions? do they use identical d3dx9_xx.dll binaries or 64-bit optimized versions? In the latter case, it might not be such a good idea to jump to absolute addresses :)

mentor: Yeah, I wondered about that as well, so now I went and checked...

The d3dx DLLs in the Windows\SysWOW64 folder on my 64-bit Vista have the same sizes as the ones in the Windows\system32 folder on my XP machine. But the ones in Windows\System32 on Vista are different.

I checked which one was used by a 32-bit application - it was the one in Windows\SysWOW64. So this indicates that the DLL is the same across all Windows versions.

TBC are cheating because they have this secret version of Crinkler developed by NASA that compresses anything down to 256 bytes! They are padding their 4ks with random data to make it all believable!

that must be THE NORDIC CONSPIRACY! TBC are the NORDIC CONSPIRACY!

this is nothing but rumors and heresay - pay no attention to these preposterous accusations!

Mentor: no implication that your 1ks or 4ks for that matter arent great. Heck I personally gave you a cdc for one ff them - chill. I was merely stating facts for tigrou so he can be realistic about what to expect with crinkler out of the box compression at 1k. No disrespect to your 1k shaders/4k coding.

Peace.

As for advantage, I stick with what I said. You have gone to the trouble of developing a fabulous 1k compressor which gives you about 200 bytes advantage over using an out of the box crinkler. You yourself said to me once that as far as you see it, sizecoding (at least above 512 bytes) is as much about the compressor as the product - and I've come to agree with that point of view.

v0.2 with fixed DLL characteristic field maybe compatible with vista, maybe not:

ftp://ftp.untergrund.net/users/hitchhikr/1kpack_v0.2.zip



Good idea, i'm using the standard zlib 1.2.3, i'll check the output.

The only test i made is here, 1024 bytes with crinkler, 941 bytes with 1kpack but for anything bigger than 1k the public version of crinkler is outmatched.

ftp://ftp.untergrund.net/users/hitchhikr/test_v0.2.zip

auld: sorry, if you got cought by the blast of my almighty rant cannon. I assure you i was not pointing it in your direction, although you were the one to inadvertently ignite it. I know for a fact that you are not 'people', but a pretty reasonable guy :)

hitch: works flawlessly now on my vista 32-bit laptop, good job :)

1024 with the public release of Crinkler (1.1a), 941 with 1kpack. Could you Mentor tell us the size with the 1k-optimized non-public release of Crinkler? (just to get an idea of where you guys are)

For compressing data, would use of Ken Silverman's KZIP be any help? At least his PNGOUT utility crunches pngs nicely tighter than any other utility that I know of, and KZIP uses the same code.

ok, here goes:

with the assistance of my young apprentice Decipher, we have assembled and compressed the colorstate76 intro with crinkler 1.1a and the 1k compressor.

to make the comparison more reasonable, we changed the dll calls from call _d3dxwhatever to call dword ptr [_imp__d3dxwhater], so crinkler won't be forced to generate call stubs. The rest of the code is untouched.

crinkler 1.1a (HASHSIZE:200 HASHTRIES:1000 COMPMDDE:SLOW UNSAFEIMPORT):
995 bytes

1k packer:
841 bytes

You can fetch the binaries here: http://daimi.au.dk/~stubbe/colorstate76_test.zip

Let me know, if you have any problems with the 1k version, as it is still pretty experimental :)

@hitchhikr: version 0.2 works like a charm, both framework and test file, nice job.
Tested OK on Vista 32 / WinXP (svp 3) / Win2000


@Mentor: tested the new 1003 bytes Himalaya on this Vista machine and it gave me a "blue screen of death" and complete reset my PC :(

For the record, your older safe himalaya (crinkled) works fine.

Specs of this machine:
Vista 32 + Core2duo + 2GB Ram + GeForce 8800

Also your 1kb ColorState76 version doesn't work, it just crash, Crinkler version works fine.

ftp://ftp.untergrund.net/users/hitchhikr/1kpack_v0.3.zip
ftp://ftp.untergrund.net/users/hitchhikr/test_v0.3.zip

933 bytes with the v0.3 (tweaked a bit with 7zip).

As i said, it outmatch the public version of crinkler for 1k effects only (which was my goal anyway).

rbz: blue screen of death, really?

I shouldn't be able to provoke that with a normal usermode application, so something more privileged must be crashing.

my initial guess is that it's antivirus or a video driver (yes, i have seen instances where they try to parse the header of the running executable). It could also be the PE loader disliking the header so much that it decides to commit suicide, but that seems far less likely.

@hitchhikr: v.0.3 still working fine on Vista

@mentor: nvidia drivers is up to date, latest version.
I've disabled my anti-virus and it still crashing (colorstate76 version).

Btw, I don't have any service pack installed on this vista

rbz: crashing the application or bsod?
also, are you talking across all of your machines, or just the vista one?

Also, i'd really like to hear from more people, if they have similar problems.

@mentor: colorstate76 version just crash, for himalaya I'm afraid to try it again :)


This happens only on Vista32, works fine on WinXP svp 3, and for Win2000 it loads and exit without any error message.

I'm not sure but maybe it still need that dummy "lz32.dll" import to make it compatible?

rbz:
win2k definitely needs the fake import, but I don't care about win2k, as everybody else gave up on it eons ago. Crinkler still supports it though, but for 1ks, i think it is reasonable to at least require xp or up.

The vista issue, however, concerns me, as this is a platform that I want to support. And is actually what i am using myself, so it is surprising that it doesn't work. And if it is consistently giving you a BSOD something weird is definitely going on.

btw, are you on msn or skype? If you don't feel like posting your account here, you can e-mail it to mynick @ crinkler.net

Works fine here on Vista x64 with 6GB ram and 8800 GTX

I got 3 monitors connected to my machine and the primary is rotated.
If I keep it like that it will crash, but if I switch my primary monitor to a non-rotated one it works as expected.

ftp://ftp.untergrund.net/users/hitchhikr/1kpack_v0.4.zip
ftp://ftp.untergrund.net/users/hitchhikr/test_v0.4.zip

A bit more tweaking of the packer, the (same) test file is now 926 bytes.

hitchhikr, i tester 0.3 and 0.4 on two XP machines (both with SP2). It works like a charm, 0.4 after compression is two bytes more than 0.3