Compressed File Library (CFL)
aka packfile, wad, virtual filesystem, etc
Downloads
cfl3r3.zip
135kB
Compressed file library engine 3 source, release 3.
Completely rewritten c++ version of CFL. Additional functionality includes pluggable class factories, easy CFL file creation runtime, and possibility of having multiple separate CFL objects in memory. Also included are cfl testing tool source to ease porting and making of additional plug-in compressors, preprocessors and ciphers, and also a completely redesigned makecfl utility, which makes creating ini files a snap. A win32 executable binary of makecfl is also included.
Most of the new features were implemented with savegames in mind.
You may wish to get zlib from http://www.gzip.org/zlib/, although cfl3 works without it as well.
Release 3 changes:
- Added Jetro Lauha's uncfl
- Lots of source clean-ups
- getFilePtr now returns correct pointer (was off by 4 bytes)
- ZLIB stuff finally actually tested with 1.1.3 - couple typecasts were still missing
Regardless of the fact that CFL has a slight bug in the ZLIB handler (last byte or so is not stored; this wasn't a problem with older ZLIB versions), it's been successfully used in dozens of games in lots of platforms, as well as other software.
I've been planning a CFL successor for ages; we'll see if it ever materializes.
cfl3r2.zip
72kB
Release 2 changes:
- delta preprocessors added
- zlib unit now works with zlib 1.1.3
cfl3r1.zip
68kB
ramfile.zip
2kB
cfl2_0.zip
9kB
Compressed file library engine 2.0 source. This is the file library used (in slightly modified form) in several Trauma demos, screensavers, DEE musicdisks etc. Doom .WAD like filelibrary, only much better. Can be used to make one-file demos, for example. Completely dynamic, uses hashes for speed. You may wish to get zlib too.
This version works with visual C.
Mostly for historical curiosity - not recommended for new software.
Background
CFL began its life in DOS, when as a budding software developer I wanted to release my programs in as few distributable files as possible. The basic use was to compile all data files into a CFL file, and then use "copy /b" to append the CFL at the end of the executable.
Back then there wasn't any DLLs to clutter the filesystem, so this move made it possible to have everything in just one executable file. With a suitable exe packer, you didn't even need to pack the thing into a zip necessarily before distribution!
Anyway, times have moved on, and such things are not quite as necessary anymore, but CFL can still un-clutter distributions and shrink disk space requirements. Additionally, it can radically reduce read times from spinning media like CD, DVD, blu-ray etc.
CFL3 has been successfully used in a bunch of commercial games on various platforms, desktop and mobile.
Limitations
CFL3 is not perfect. It was designed from the viewpoint that when you want a file, you want to load it completely anyway, so no file pointer stuff exists - you just ask for a resource and get a pointer to it. As a workaround for when you do want file pointers, I wrote a 'ramfile' hack, which, after loading the whole file in memory, fakes file i/o routines.
This was further developed in Fathammer's X-Forge engine into a C++ class.
Reading the whole file at once has its good and bad sides. Memory requirements are the bad ones, while good sides include faster file i/o, no need to worry about media failure in the middle of file, etc.
Known Bugs
The ZLIB handler in CFL3 has a known bug - it doesn't store the last byte (or so), which wasn't an issue with the ZLIB version that was used at the time. As a workaround, you can change the handler to ignore end-of-stream errors. It'll unpack fine. Better solution is to fix the bug, but you'll lose compatibility with existing CFL3 files (assuming you care).
What About CFL4?
I've been pondering on a new version of CFL for ages; I have plenty of ideas that would help people optimize their file i/o, reducing level load times and debugging possible issues etc.
CFL4 would also support streaming, "solid" compression, and so on.
Whether I'll ever find the time to actually implement it remains to be seen. After all, I have no idea if anyone even cares about such things anymore.