unzip and untar into new directory

[jshafferman]

I have a file that is file.tar.gz and I would like to implement an open function from GUI menu that allows the user to select the tar.gz file and the application unzip, untar, and place those files in a specified directory. I don't think there is a Qt function that can do all of the above (but maybe some) but I was curious on any suggestions on how to implement this functionality.

Thanks for any help!

[amleto]

use QProcess to start an unzip utility e.g. 7z or equivalent.

[jshafferman]

Is there anyway to do it without using QProcess? I am trying to avoid needing to install any third party software with my application.

[jshafferman]

Ok so I have decided to use peazip for the QProcess but I do not understand how to use QProcess very well. I downloaded the peazip portable and placed the executable in the same directory as my application. I tried to do the following:



MainWindow::open()
{
QProcess *unarchive = new QProcess(this);
QStringList arguments;
arguments << "-ext2here" << "file.tar.gz";
unarchive->start("peazip", arguments);

// other things happen
}


Unfortunately nothing seems to be happening so I am curious what I am missing. Thanks for any help!

[ChrisW67]

What sort of 'nothing' is happening? Is the QProcess object emitting an error() signal? Where is the peazip executable in relation to the current working directory of your process?

[amleto]

how are you testing? by debugging through an IDE or just running your program 'manually'? If the former, you probably just have a problem where your app is not run from the same location as the unzip utility

[jshafferman]

Duh! I was running it in debug mode and didn't put the peazip executable in the correct directory... It seems to be working fine now!!

I do have another question that might be answered here, can you do an OS check with Qt? I don't see a Linux version of the peazip portable so I am thinking I will need a #if statement to check for Windows OS and use peazip but if it is Linux use standard unix unarchiving/archiving commands. How would you call the unix terminal to unzip and untar a file?

Thanks again for any help!

[jshafferman]

So I am able to have the QProcess execute the unzip and untar but when I try and run something after the untar nothing appears to be happening.



QMainWindow::open()
{
if(okToContinue())
{
QString fileName = QFileDialog::getOpenFileName(this, tr("Open file.tar.gz file"), ".", tr("gzip tarball"
"(*.tar.gz)"));

if(!fileName.isEmpty())
{
QProcess *unarchive = new QProcess(this);

QStringList arguments;
arguments << "-ext2here" << fileName;
unarchive->start("peazip", arguments);

if(unarchive->waitForFinished())
{
fileName.remove(".gz");
arguments.clear();
arguments << "-ext2here" << fileName;
int exitCode = unarchive->execute("peazip", arguments);

if(unarchive->waitForFinish())
{
fileName.remove("file.tar");
// this is the path that is extracted from tar and the .csv is what I am after
fileName.append(("tmp/file/A.csv"));

loadFile(fileName);
}
}
}
}
}


It never seems to make it inside the second waitForFinish if statement. What am I doing wrong?

Thanks for help!

[amleto]

why are you mixing execute/start? Why are you allocating on the heap when you only want a local instance?

I believe the problem is that you are using execute, which is a static method, therefore the finished state required for 'waitForFinished' is not modified.

[jshafferman]

I was just experimenting with how QProcess works that is why the execute/start mixture is being used. The second question was again just me mimicking an exmaple I saw in the Qt Assitant manual. I went ahead and switched QProcess to allocate on the stack instead of the heap (which is definitely a better idea). Anyway I can't seem to figure out how to make the unzip and untar work and then have the newly created directory pick up the new file created in there.

I keep getting the following error:
Cannot read File:
C:/.../tmp/file/A.csv
The system cannot find path specified..

I am a little confused as to why it can't find it when I go to the file directory and everything is as it should be.

Here is the new code I am using:



void MainWindow::open()
{
if(okToContinue())
{
QString fileName = QFileDialog::getOpenFileName(this, tr("Open file.tar.gz file"), ".", tr("gzip tarball"
"(*.tar.gz)"));

if(!fileName.isEmpty())
{
QProcess unarchive(this);

QStringList arguments;
arguments << "-ext2here" << fileName;
unarchive.start("peazip", arguments);

if(unarchive.waitForFinished())
{
unarchive.close();

fileName.remove(".gz");
arguments.clear();
arguments << "-ext2here" << fileName;
unarchive.start("peazip", arguments);

if(unarchive.waitForFinished())
{
unarchive.close();

fileName.remove("file.tar");
cout << fileName.toStdString() << endl;
fileName.append(("tmp/file/A.csv"));
cout << fileName.toStdString() << endl;
}
}

loadFile(fileName);
}
}
}


My problem is when loadFile is being called the file is not being found and I am not sure why that is. FYI, I wasn't unziping and untaring automatically at first so I know if the fileName being passed to loadFile is correct it works as it should.

Thanks for any help!

Added after 20 minutes:

It appears to be a permission problem maybe... I am still unsure but this code should help a little bit I think.



if(!fileName.contains("global_urn_r.csv"))
{
QMessageBox::warning(this, tr("Loading Error"), tr("Must open a global_urn_r.csv file in order to load properly"));

return false;
}

QFile inFile(fileName);

if(!inFile.open(QIODevice::ReadOnly))
{
QMessageBox::warning(this, tr("Unable to Load File"), tr("Cannot read file %1:\n%2.").arg(inFile.fileName()).arg(inFile.errorString()) );


statusBar()->showMessage(tr("Loading canceled"), 2000);

return false;
}

[jshafferman]

Figured out that the loadFile function is running when the file is being untared... Not sure why it doesn't wait until untar is done?

[amleto]

probably down to you re-using the same QProcess instance. Create a new instance for the second run.

[jshafferman]

Tried that theory out as well and unfortunately doesn't seem to matter. But I was able to just make a while loop to wait until I find the directory I am looking for. This appears to be working but then I get completely lost as to why I can't traverse the directory. The code seems to find the directory then the very next statement says that it didn't find it, very confusing so I will just show it to you.



QProcess untar(this);
untar.start("peazip", arguments);

if(untar.waitForFinished())
{
untar.close();

fileName.remove("file.tar");

QDir dir(fileName);

while(!dir.cd("tmp"))
cout << "Waiting for tmp to be created\n";
}

QDir dir(fileName);

if(dir.cd("tmp"))
cout << "tmp has been created and found\n";

if(!dir.cd("tmp"))
qWarning("Cannot find the \"/tmp\" directory");
else
{
if(!dir.cd("file"))
qWarning("Cannot find the \"/file\" directory");
else
{
QFile file(dir.filePath("A.csv"));

if(!file.exists())
qWarning("A.csv");
else
loadFile(file.fileName());
}
}


I don't understand why the output is the following:
Waiting for tmp to be created
...
tmp has been created and found
Cannot find "/tmp" directory

I am really confused as to why one line finds it the other doesn't...

[amleto]

do you have directory tmp/tmp ? If not, what are you thinking?


And also, what on earth are you doing with infinite loops waiting for directories? It just such a hack! If you really need to wait for file system things, then use qfilesystemwatcher

[jshafferman]

The directory path is correct it is C:\...\tmp\file\A.csv. Anyway on to your second question, I am still very new to Qt (just been using it about a year or so) and I haven't worked with traversing directories on my own. The infinite loops, in theory aren't infinite loops but if the files weren't extracted correctly then it could potentially have some serious issues. I know it is a hack but for now that is the only way I know how to keep checking directories. I have read some of the stuff on QFileSystemWatcher but I don't know how to use it properly although I will try. Could you maybe show me a sample of how that class would be used for what I am trying to do?

[amleto]

if the directory is c:\...\tmp\a.csv, you cannot say you are also correct to look for c:\...\tmp\tmp, which is what you are doing.

[jshafferman]

Oh I see what I did there, that was just me not understanding that cd took it to that directory when it found it. So when I successfully cd to temp, then the next call to cd tmp doesn't exst. I understand I made a mistake there. However I am still not sure how to use QFileSystemWatcher, I tried using a connect statement like the following:



connect(&watcher, SIGNAL(directoryChanged(QString), this, SLOT(directoryUpdated(QString));


Where 'this' is the QMainWindow and the slot is a newly created slot that should be picking when the directory changes. In the slot function I currently have a simple cout statement so that I know it was making the connection however it doesn't appear that it is ever making it to the slot function. Basically I took out the first while loop while(!dir.cd("tmp")) inside of the if statement and got ride of all the other code and just have the following:



QFileSystemWatcher watcher;
watcher.addPath(fileName); // path where the tarball is being extracted

// connection statement here


Any Ideas what I am doing wrong?

[amleto]

Where 'this' is the QMainWindow and the slot is a newly created slot that should be picking when the directory changes. In the slot function I currently have a simple cout statement so that I know it was making the connection however it doesn't appear that it is ever making it to the slot function. Basically I took out the first while loop while(!dir.cd("tmp")) inside of the if statement and got ride of all the other code and just have the following:



QFileSystemWatcher watcher;
watcher.addPath(fileName); // path where the tarball is being extracted

// connection statement here


Any Ideas what I am doing wrong?

why don't you just show some complete code. Instead everybody has to code and compile in their head. It's not very endearing...

[jshafferman]

No problem here is the full chunk of code that I am trying to execute...



void MainWindow::open()
{
if(okToContinue())
{
QString fileName = QFileDialog::getOpenFileName(this, tr("Open file.tar.gz file or A.csv"), ".", tr("Data Files (*.tar.gz *.tar *.csv)"));

if(!fileName.isEmpty() && fileName.contains("tar.gz"))
{
QProcess unarchive(this);

QStringList arguments;
arguments << "-ext2here" << fileName;
unarchive.start("peazip", arguments);

if(unarchive.waitForFinished())
{
unarchive.close();

fileName.remove(".gz");
arguments.clear();
arguments << "-ext2here" << fileName;
}

QProcess untar(this);
untar.start("peazip", arguments);

if(untar.waitForFinished())
{
untar.close();

fileName.remove("file.tar");

QDir dir(fileName);

while(!dir.cd("tmp"))
cout << "Waiting for tmp to be created\n";
}

QDir dir(fileName);

if(dir.cd("tmp"))
{
cout << "tmp has been created and found\n";

if(dir.cd("file"))
{
cout << "file has been created and found\n";
cout << "Absolute path: " << dir.absolutePath().toStdString() << endl;

// says 11 when it should be 40...
while(dir.count() < 40)
{
cout << "Number of files: " << dir.count() << endl;
dir.refresh();
}

QFile file(dir.filePath("A.csv"));

if(file.exists())
{
cout << "A.csv has been created and found\n";
cout << "Filename is: " << file.fileName().toStdString() << endl;
loadFile(file.fileName());
}
else
{
cout << "Cannot find A.csv\n";
}
}
}
}
else if(!fileName.isEmpty() && fileName.contains("A.csv"))
{
loadFile(fileName);
}
}
}


That is my 'hack' way of doing it right now. This is me trying to figure out how QFileSystemWatcher works in order to make it work without the nasty hacks.



void MainWindow::open()
{
if(okToContinue())
{
QString fileName = QFileDialog::getOpenFileName(this, tr("Open file.tar.gz file or A.csv"), ".", tr("Data Files (*.tar.gz *.tar *.csv)"));

if(!fileName.isEmpty() && fileName.contains("tar.gz"))
{
QProcess unarchive(this);

QStringList arguments;
arguments << "-ext2here" << fileName;
unarchive.start("peazip", arguments);

if(unarchive.waitForFinished())
{
unarchive.close();

fileName.remove(".gz");
arguments.clear();
arguments << "-ext2here" << fileName;
}

QProcess untar(this);
untar.start("peazip", arguments);

if(untar.waitForFinished())
{
untar.close();

fileName.remove("file.tar");
}

QFileSystem watcher;
watcher.addPath(fileName);

connect(&watcher, SIGNAL(directoryChanged(String), this, SLOT(directoryModified(QString));
}
}

void MainWindow::directoryModified(const QString& path)
{
QMessageBox::information(this,"Directory Modified", "Your Directory is modified");
// not sure what I am going to do yet...
}


When I run the 'correct' way of doing things I never see the information box pop up... So I am assuming I am somehow not using QFileSystemWatcher class correctly. Any help is greatly appreciated.

[amleto]

you put the watcher on the stack...

[jshafferman]

Ah ok, well I saw it used the way I was trying to use it in an example but apparently that example was incorrect. Just so I am clear the reason it doesn't work is because the connection is made but then shortly after the connection is made the object falls out of scope and thus is destroyed before anything can ever happen? I think I am right on this but just want to make sure. So I was able to get the extraction working properly but when I try and use another process to re-archive it using peazip it works but after about 3 minutes it just kills the process while it is still running. Any idea what would cause that? I will post the relevant code if necessary but thought I would get a general idea of what might be causing the process to stop while running. Thanks again for any help and all the help that has come thus far.

[amleto]

"after the connection is made the object falls out of scope and thus is destroyed before anything can ever happen?"
Yes.

"So I was able to get the extraction working properly but when I try and use another process to re-archive it using peazip it works but after about 3 minutes it just kills the process while it is still running. Any idea what would cause that?"
If you are suggesting that peazip kills your app, then I think that is unlikely. If your app dies after three minutes then you need to start debugging your app. If you mean that the peazip process is killed, then maybe it is just from a timeout in QProcess? Is 3 minutes a reasonable time for the process to take? Probably not (sounds too long) so maybe something has gone wrong with your archive command.

[wysota]

I don't think QProcess has any timeouts. That would not be very logical. There is a timeout in QProcess::waitForFinished() but it wouldn't kill the child process, just would timeout the suspension of the parent process.

[jshafferman]

What happens is the process starts and brings up the peazip GUI for archiving the file. I think it is roughly about 2-3 minutes later when the peazip GUI closes out and I get the following message in the debugger: QProcess: Destroyed while process is still running.

Here is the code segment


bool MainWindow::saveFile(const QString &fileName)
{
// file manipulation...

QDir dir(directory);

QProcess tar(this);
QStringList arguments;
arguments << "-add2archive" << dir.path();
tar.start("peazip", arguments);

if(tar.waitForFinished())
tar.close();

// reset some toggle values and return true
}


I am simply curious what would cause the peazip GUI to crash/close. Thanks for any help!

[wysota]

Quoting the docs for QProcess::close():


Closes all communication with the process and kills it.

[jshafferman]

If I take out the if(tar.waitForFinished()) statement then the application immediately gives me the same debug error. If I take out the tar.close() statement inside of the if statement then it allows the application to open and then crashes/closes after 2-3 minutes still. Is there a way to make the process wait until I have finished using the application? I thought it was waitForFinsihed() but that doesn't appear to be the case.

[wysota]

Did you carefully read the docs for waitForFinished() before using that method?

[jshafferman]

I thought I had understood it. I did read it again a few times just to see if I misunderstood what I had initially read. It appears that the waitForFinish() is not well suited for GUI application (according to the documentation) however I don't see any other function that would allow me to wait for a boolean statement to signal that the process has finished correctly. The documentaiton mentions something about an event loop but I haven't been able to see any examples or functions I could use to run an event loop waiting for the process (peazip archiving) to be finished. How would I go about doing this? My guess is to use a while loop and wait for the state() function to return NotRunning... Am I on the right track with that?

[amleto]

yeah, we have both (wysota and I) mentioned timeout - why haven't you tried waitForfinished(-1) or waitForfinished(300000), say?
Have you tried running the zip util from the command line with the same arguments? does it work ok?

How would I go about doing this? My guess is to use a while loop and wait for the state() function to return NotRunning... Am I on the right track with that?
No.
rtfm already!

[jshafferman]

Grrr... I did know about the msec parameter that you can optionally put in the waitForFinished() but I didn't want it to timeout until the user was done. After seeing your post I read through it again and I released I was glazing over one of the most important parts of the waitForFinished() parameter -1. I assumed that if I put -1 it wouldn't ever close the peazip process because the documentation said it would never timeout but I realize now that just meant the QProcess itself not the application it is using. So basically if I put -1 in for the waitForFinish() it waits until I click the ok button on the peazip, archives the files, and then the application closes out. Since the QProcess is built on the stack, I am assuming that as soon as the program exits the scope of that function the QProcess is destroyed. This will work perfectly for me and I want to say thanks to both of you for helping me figure it out and especially since you let me get there partially on my own :).

Thanks I think I have it working properly now!

[amleto]

glad that you seemed to have blasted all the gremlins, and also that you appreciate that (in the long run) the more you figure out for yourself, the easier things will be for you.