qUncompress and gzipped files

[roxton]

Hello!
I need some help again
I'm trying to decompress gzipped files using the qUncompress. As far I know, the difference between gzip-created file and the qCompressed one consists in:
1. Different headers. On my testing example (the same file that was compressed with gzip and qCompress) the header of qCompressed file = 10 bytes. But according to the source and docs, it must be only 4 bytes with the data length value. I'm confused.
2. Different tails. I know what gzip writes the 8-bytes trail (CRC32 and ISIZE). But what is 4 bytes of qCompressed trail?
Here is my testing code, that produces "Z_DATA_ERROR: Input data is corrupted" error:

//.gz file header
typedef struct {
        quint8 id1;
        quint8 id2;
        quint8 cm;
        quint8 flg;
        float mtime;
        quint8 xfl;
        quint8 os;
       } t_gzip_header;
 
//bit flags
#define gzip_hdr_FTEXT 0
#define gzip_hdr_FHCRC 1
#define gzip_hdr_FEXTRA 2
#define gzip_hdr_FNAME 3
#define gzip_hdr_FCOMMENT 4
 
 
void rvln::test()
{
  QFile file ("/home/test/test.cpp.gz");
 
  if (! file.open (QFile::ReadOnly))
     return;
 
  t_gzip_header gzip_header;
 
  QDataStream s_in (&file);
 
//read the header to the struct instance:
  s_in >> gzip_header.id1;
  s_in >> gzip_header.id2;
  s_in >> gzip_header.cm;
  s_in >> gzip_header.flg;
  s_in >> gzip_header.mtime;
  s_in >> gzip_header.xfl;
  s_in >> gzip_header.os;
 
//read some additional chunks if flags are set
 
  if (gzip_header.flg & (1 << gzip_hdr_FEXTRA))
     {
      quint16 size;
      s_in >> size;
      char *data = new char [size];
      s_in.readRawData (data,size);
      delete data;
     }
 
  if (gzip_header.flg & (1 << gzip_hdr_FNAME))
     {
      qint8 c;
      QString original_fname;
      do
        {
         s_in >> c;
         original_fname += static_cast<char> (c);
        }
       while (c != 0);
 
      qDebug() << original_fname;
    }
 
 
    if (gzip_header.flg & (1 << gzip_hdr_FCOMMENT))
      {
       qint8 c;
       QString comment;
       do
         {
          s_in >> c;
          comment += static_cast<char> (c);
         }
       while (c != 0);
 
       qDebug() << comment;
      }
 
    if (gzip_header.flg & (1 << gzip_hdr_FHCRC))
       {
        quint16 crc16;
       s_in >> crc16;
       }
 
//here is the stream reaches to the compressed data
//so I read this data
//and gzip's 8-bytes trail
//to the buffer compressed_data
 
  int len = file.size() - sizeof (t_gzip_header);
  char *compressed_data = new char [len];
  s_in.readRawData (compressed_data, len);
 
//the temporary array for compressed data
//plus the 4-bytes header with the 
//uncompressed data length,
//but how I can know the uncompressed data length
//before uncompression?
 
  QByteArray ta;
 
//trying to set the 4-bytes header
//with the approx. uncompressed data length
 
  len *= 10;
 
  ta.resize(4);
 
  ta[0] = (len & 0xff000000) >> 24;
  ta[1] = (len & 0x00ff0000) >> 16;
  ta[2] = (len & 0x0000ff00) >> 8;
  ta[3] = (len & 0x000000ff);
 
//add the compressed data to temp array
 
  ta.append (compressed_data);
 
//remove the 8-byte gzip trail
 
  ta.chop(8);
 
//trying to uncompress
 
  QByteArray aucomp = qUncompress (ta);
 
//oops! Got "Z_DATA_ERROR: Input data is corrupted"
//here
 
//and close the file in a deep sorrow
  file.close();
}

//.gz file header
typedef struct {
        quint8 id1;
        quint8 id2;
        quint8 cm;
        quint8 flg;
        float mtime;
        quint8 xfl;
        quint8 os;
       } t_gzip_header;

//bit flags
#define gzip_hdr_FTEXT 0
#define gzip_hdr_FHCRC 1
#define gzip_hdr_FEXTRA 2
#define gzip_hdr_FNAME 3
#define gzip_hdr_FCOMMENT 4


void rvln::test()
{
  QFile file ("/home/test/test.cpp.gz");

  if (! file.open (QFile::ReadOnly))
     return;

  t_gzip_header gzip_header;

  QDataStream s_in (&file);

//read the header to the struct instance:
  s_in >> gzip_header.id1;
  s_in >> gzip_header.id2;
  s_in >> gzip_header.cm;
  s_in >> gzip_header.flg;
  s_in >> gzip_header.mtime;
  s_in >> gzip_header.xfl;
  s_in >> gzip_header.os;

//read some additional chunks if flags are set

  if (gzip_header.flg & (1 << gzip_hdr_FEXTRA))
     {
      quint16 size;
      s_in >> size;
      char *data = new char [size];
      s_in.readRawData (data,size);
      delete data;
     }

  if (gzip_header.flg & (1 << gzip_hdr_FNAME))
     {
      qint8 c;
      QString original_fname;
      do
        {
         s_in >> c;
         original_fname += static_cast<char> (c);
        }
       while (c != 0);

      qDebug() << original_fname;
    }


    if (gzip_header.flg & (1 << gzip_hdr_FCOMMENT))
      {
       qint8 c;
       QString comment;
       do
         {
          s_in >> c;
          comment += static_cast<char> (c);
         }
       while (c != 0);

       qDebug() << comment;
      }

    if (gzip_header.flg & (1 << gzip_hdr_FHCRC))
       {
        quint16 crc16;
       s_in >> crc16;
       }

//here is the stream reaches to the compressed data
//so I read this data
//and gzip's 8-bytes trail
//to the buffer compressed_data

  int len = file.size() - sizeof (t_gzip_header);
  char *compressed_data = new char [len];
  s_in.readRawData (compressed_data, len);

//the temporary array for compressed data
//plus the 4-bytes header with the 
//uncompressed data length,
//but how I can know the uncompressed data length
//before uncompression?

  QByteArray ta;

//trying to set the 4-bytes header
//with the approx. uncompressed data length

  len *= 10;

  ta.resize(4);

  ta[0] = (len & 0xff000000) >> 24;
  ta[1] = (len & 0x00ff0000) >> 16;
  ta[2] = (len & 0x0000ff00) >> 8;
  ta[3] = (len & 0x000000ff);

//add the compressed data to temp array

  ta.append (compressed_data);

//remove the 8-byte gzip trail

  ta.chop(8);

//trying to uncompress

  QByteArray aucomp = qUncompress (ta);

//oops! Got "Z_DATA_ERROR: Input data is corrupted"
//here

//and close the file in a deep sorrow
  file.close();
}

To copy to clipboard, switch view to plain text mode 

P.S. all gzip-format related data is taken from http://www.gzip.org/zlib/rfc-gzip.html

[rbp]

As far I know, the difference between gzip-created file and the qCompressed one consists in:

I've compressed and uncompressed a lot of data and experienced no problems with these functions. The compress format might be different that you expect. Why don't you try qCompress on the original unzipped data and then compare it to the gzipped version to confirm what is the difference.

[Doru]

Hi,
Did you succeed in decompressing the file?

I have the same problem with the trail.

I have a ByteArray of 9 bytes ->uncompressed.
If I use qCompress on it and after that qUncompress it works ok.
The problem is that after using qCompress I get an Array of 21 bytes .
I tryied to read from the original file 17 bytes, prepend with 4 (decompressed length) and after that to qUncompress.
I get the same error.

From those 17 bytes from the original compressed file the last 6 are different than the last six generated with qCompress.

Do you have a siggestion?

Thanks

[falconium]

I'm also suffering because of reading LZW compressed files. I think GZIP is using deflate method, so qUncompress won't work.
Please, let me know if I'm wrong.