1 Program Description
Program XBW is intended for compression of XML files but can be used for all files. The biggest advantage of this program is that the XML files do not have to be valid XML files. It has very good compression ratio for any large text files (better than bzip2, gzip or rar). It has good compression ratio for even not textual large files with parser turned off.
The supported platforms are x86 running Windows XP and Linux. The source code of XBW is written in C language and no platform specific features have been used. Hence the code can be compiled on any platform, but may require modifications of compiler parameters.
2 System Requirements
The program supports files in Unicode. This feature is dependent on iconv library which supports dozens of encodings. Hence you are required to have libiconv installed on your system. If you want to compile the source you need development version of the library. Iconv library is part of glibc on newer installations of Linux systems.
XBW including installation of iconv will not require more than 20MB on disk. Memory requirements depend on the file being compressed. We suggest that your RAM is at least 10 times bigger than the file for compression. This is highly dependent on method of compression used.
For compiling under Unix like systems Makefile is provided, which requires make tools. As compiler gcc has been tested.
The program uses both signed and unsigned integers and expects them to have 4 bytes. If your system uses smaller size for these types please modify definitions of types Int32 and UInt32 in file config.h. This should not happen on any PC newer than 10 years. For systems using 8 bytes, if you do not redefine the types, twice the memory needed will be used.
2.1 Program Limitations
The program supports only files smaller than 2GB. This limitation is due to use of 4 byte integers.
3 Installation
3.1 Linux
Before installing XBW you need to have libiconv installed. In new versions of glibc iconv is part of it. If not, you need to install iconv including headers yourself. Only source code of XBW is provided at this time. So you have to compile it yourself using Makefile included in the source package and optionally copy the xbw executable to any of the directories listed in your $PATH, e.g. /usr/local/bin.
3.2 Windows
Although MS Windows is one of the supported platforms, we do not provide neither windows installer nor windows executable at this time. If you'd like to try XBW on Windows, you have to compile the source yourself. However, it is probable that future versions of XBW will provide some simple installer.
4 Program Usage
4.1 Simple Examples
To compress a file called file into file file.xbw use:
- xbw file
- xbw -x file.xbw
- xbw -d file.xbw
- xbw [-x]
- xbw -h
4.2 Specifying input and output
You can either specify the input and output using options -i= and -o= or as last parameters to the program. The first parameter not starting with - is considered to be the input. If the string is equal to stdin or - standard input is used. For any other value the program assumes file of that name. If another parameter follows the input it is assumed to be specification of output. Strings stdout and - are understood as standard output otherwise file of that name will be created.
If no input is specified standard input is used as default.
If no output is specified the output is determined based on input. If standard input is used than we use standard output. If file filename is specified that program creates a file called filename.xbw.
4.3 Synopsis
xbw [ parameters ] [ input_file [ output_file ] ]
4.4 Parameters
- [-h] Display help with lists of supported variants for parameters.
- [-x -d] Decompress.
- [-i= < filename | - | stdin >] Another way to specify input. Use stdin if you want input to go from standard input.
- [-o= < filename | - | stdout >] Another way to specify output. Use stdout if you want output to go to standard output.
- [-file_coding= < encoding >] Specify encoding of the file to be compressed. UTF-8 is default. BINARY is safe for all files. Help shows list of currently supported encodings.
- [-Parser= < on / off >] Use on to use parser. Use off to turn the parser off. Using parser makes sense only for text files.
- [-Syl_partition= < variant >] Specify which partiting of words to use. The best method depends on the language and size of the file. Use help for the list.
- [-BWT= < variant / off >] Specify algorithm to be used for Burrows-Wheeler Transform. All produce the same results, but their speed is different. Their relative speed depends on the file size and its repetitiveness. Sadakane is used as default, because of its robustness and good performance. Use help for the list.
- [-MTF= < on / off >] Specify whether to use Move-To-Front algorithm. Defaultly turned on.
- [-RLE= < variant / off >] Run-Length Encoding. Default variant is 3. Only one of the RLE/LZ/PPM can be used.Use help for the list.
- [-LZ= < variant / off >] Lempel-Ziv algorithms. By default it is turned off. Only one of the RLE/LZ/PPM can be used. Use help for the list.
- [-PPM= < on / off >] Prediction by Partial Match algorithm. Defaultly turned off. Only one of the RLE/LZ/PPM can be used. Beware of memory requirements.
- [-PPM_exclusions= < on / off >] Specify whether to use exclusions. Defaultly turned off.
- [-PPM_order= < number >] Specify the length of contexts that the PPM will work with. Beware that PPM uses lots of memory with high values of PPM_order.