KungFQ: a Simple and Fast Approach to Compress Fastq Files

Motivation: nowadays storing data derived from deep sequencing experiments has become pivotal and standard compression algorithms do not exploit in a satisfying manner their structure. A number of reference based compression algorithms have been developed but they are less adequate when approaching new species without fully sequenced genomes or non genomic data (RNA sequencing data should be aligned against the entire "theoretically possible" transcriptome which comprises exon-intron boundaries which have to be recomputed for each different read length). Methods: we developed a tool that takes advantages of fastq characteristics and encodes them in a binary format optimized in order to be further compressed with standard tools (such as gzip or lzma). The algorithm is straightforward and does not need any external reference file, it scans the fastq only once and has a constant memory requirement. Moreover we added the possibility to perform lossy compression, losing some of the original informations (IDs and/or qualities) but resulting in smaller files. When qualities are not stored in the compressed file it is possible to define a cutoff under which corresponding base calls are converted to N. Results: we achieve 2.84 to 7.84 compression ratios on various fastq files without losing infos and 5.38 to 8.93 losing IDs, which are often not used in common analysis pipelines. In this article we compare the algorithm performance with known tools, usually obtaining higher compression levels. This software should be useful for whose that are interested in using next generation sequencing data for further bioinformatics analyses.