Slow FTP Program using QFtp

[neveffects]

I am trying to upload 2 files to one machine and another 3 files to another machine using my FTP program. In this situation we have two threads running at a time for the two machine. This threads will check a folder named outbox in a particular time interval using the QTimer for any files to be uploaded. User will continuously copy files to be uploaded into the Outbox folder at irregular time intervals. For two or three times the file uploading will complete successfully. After that the datatransfer will slow down gradually and finally it will work endless without any progress in datatransfer. For monitoring the transfer progress I have connected the dataTransferProgress Signal. I am using many pointers and Objects and also I am deleting the pointer objects after its usage. Is there any other debugging methods for memory leak in Qt4.7 Please help me. I am trying it for 2 Weeks

Thanks in advance

[tbscope]

I am trying to upload 2 files to one machine and another 3 files to another machine using my FTP program. In this situation we have two threads running at a time for the two machine. This threads will check a folder named outbox in a particular time interval using the QTimer for any files to be uploaded. User will continuously copy files to be uploaded into the Outbox folder at irregular time intervals. For two or three times the file uploading will complete successfully. After that the datatransfer will slow down gradually and finally it will work endless without any progress in datatransfer. For monitoring the transfer progress I have connected the dataTransferProgress Signal.

Without seeing any implementation details or any code it is very difficult to answer.

I am using many pointers and Objects and also I am deleting the pointer objects after its usage. Is there any other debugging methods for memory leak in Qt4.7 Please help me. I am trying it for 2 Weeks

If you want other tools, you first need to define which ones you already tried.
As far as for memory leak detection in Qt4.7, I think there isn't any.

You can use the standard tools like a debugger or tracer. Valgrind is popular.

[neveffects]

Below I am attaching my full working code. Here in main.cpp i am creating maximum 10 threads for ten machines . Now I ll start the threads soon after its creation. then inside the FileMonitoring.cpp which inherits the QTimer base class so after a particular time interval this FileMonitoring Class is called, inside which the checking of files inside an Outbox folder is happening. This FileMonitoring class is called from main.cpp inside which the Timer will emit a timeout signal every 6000 mSec. Now inside TransferThread.cpp file another timer will emit a timeout signal after every 500 mSec. in this timeOut slot connecting FTP and sending file is happening. While transfering data dataTransferProgress signal is working and after sending a particular size of the file or group of bits it will stop emitting this signal and an endless while loop will occur inside the TransferThread::run() method's else loop.

============TransferThread.cpp================
TransferThread::TransferThread(QString ThreadName)  : QThread(0) {
    stopFlag = false;
    _name = ThreadName;
    _userName = "administrator";
    _password = "admin";
    port = 9012;
    connectingDB();
}
 
void TransferThread::initFTP() {
    ftp = new QFtp(this);
    QTimer *loginTimer=new QTimer();
    loginTimer->start(500);
    connect(loginTimer,SIGNAL(timeout()),this,SLOT(loginFTP()));
    connect(ftp, SIGNAL(commandFinished(int,bool)),this,SLOT(commandFinished(int,bool)));
    connect(ftp, SIGNAL(dataTransferProgress(qint64,qint64)),this,SLOT(dataTransferProgress(qint64,qint64)));
    connect(ftp, SIGNAL(stateChanged(int)),this,SLOT(stateChanged(int)));
}
void TransferThread::loginFTP() {
    if((ftp->state()==QFtp::Unconnected || ftp->state()==QFtp::Closing) )   {
        this->terminate();
        ftp->abort();
        if(_checkingHostName==_hostName) {
            _checkingHostName = _alternateHostName;
        } else {
            _checkingHostName = _hostName;
        }
        ftp -> connectToHost(_checkingHostName,port);
        ftp -> login(_userName, _password);
    }
    if(ftp -> state() == QFtp::LoggedIn) {
        this->start();
    }
}
void TransferThread::run() {
    complete=0;
    f_done=0;
    f_total=0;
    while(!stopFlag) {
        if(complete==0) {
            if(fileSelection(branchName)) {
                fileSendProcess();
            }
        } else {
                qWarning() << "ENDLESS LOOP NOT STOPPING"
        }
        sleep(2);
    }
}
void TransferThread::fileSendProcess() {
    if(sendFileProcess() ) {
        QMutex *mut=new QMutex();
        mut->lock();
        sendDataFTP(_filename);
        mut->unlock();
    }
}
void TransferThread::sendDataFTP(QString fileName) {
    Process *proc = new Process();
    QString file="../"+_destination+"/outbox/"+fileName;
    QFile *filetrans= new QFile(file);
    if(filetrans->exists()) {
        if ( !filetrans->open( IO_ReadOnly ) ) {
            unsendFileProcess(fileName, "Cannot Open File");
            proc->process("mv "+fileName+" ../files_unsent", "FILE MOVING TO UNSEND FOLDER");
            filetrans = NULL;
            proc->close();
            filetrans->close();
            proc = NULL;
            delete proc;
            delete filetrans;
            return;
        }
    } else {
        filetrans = NULL;
        proc->close();
        proc = NULL;
        delete proc;
        delete filetrans;
        return ;
    }
    QSqlQuery qry;
    qry.exec("insert into filessending values('"+_filename+"','"+_destination+"');");
    QByteArray buffer;
    QMutex *mut_buffer=new QMutex();
    mut_buffer->lock();
    buffer = filetrans->readAll();
    mut_buffer->unlock();
    ftp->put(buffer,fileName,QFtp::Binary);
    complete=1;
    proc->close();
    proc = NULL;
    delete filetrans;
    delete proc;
}
void TransferThread::dataTransferProgress(qint64 done, qint64 total)
{
    Process *proc = new Process();
    QSqlQuery qryUpdate;
    QSqlQuery query;
    QString pbid;
    QString brid;
    QString fileName=_filename;
    pbid=publisherId(_filename.mid(0,2));
    brid=branchId(_destination);
    str_pbId=pbid;
    str_brId=brid;
    QString strUpdate = "update transferstatus set "
                        " done = "+QString::number(done)+","
                        " total="+QString::number(total)+","
                        " pb_id="+pbid+
                        " where filename='"+_filename+"' and br_id="+_destinationId+"";
    qryUpdate.exec(strUpdate);
    sendFileSize = done;
    int count=0;
    while(sendFileSize>1024) {
        count++;
        sendFileSize=sendFileSize/1024;
    }
    f_total=total;
    f_done=done;
    if(done==total) {
        qryUpdate.exec("delete from filessending where filename='"+_filename+"' and destination='"+_destination+"';");
        qryUpdate.exec("delete from file_transfer_queue where filename='"+_filename+"' and br_id="+brid+" and pb_id="+pbid+";");
        QTime now;
        QString csq;
        QTime strtTime;
        QString duration=getDuration(strtTime,now);
        QString timecurrent=QTime::currentTime ().toString(Qt::TextDate);
        strtTime=strtTime.fromString(_timestart, Qt::TextDate);
        now=now.fromString( timecurrent,Qt::TextDate );
        csq="insert into transreport(file, size, dot, start,"
            " finish, br_id, pb_id, duration,rid)values('"+_alias+
            "','"+_fileSize+"','"+QDate::currentDate ().toString()+
            "','"+_starttime+"','"+QTime::currentTime ().toString("hh:mm:ss ap")+
            "',"+brid+","+pbid+",'"+duration+"',"+_releaseid+");";
        query.exec(csq);
        int count=1;
        while(checkingSendFile(fileName)) {
            fileName.append(QString::number(count));
            count++;
        }
        proc->process("mv ../"+_destination+"/outbox/"+fileName + " ../sentfiles/", "MOVING FILE TO SEND FOLDER");
        complete=0;
    } else {
        QSqlQuery queryselect;
        QString selectqry="select status from transferstatus"
                          " where filename='"+_alias+
                          "' and br_id="+brid+" and pb_id="+pbid+";";
        queryselect.exec(selectqry);
        if(queryselect.next()) {
            QString statusfile=queryselect.value(0).toString();
            if(statusfile.compare("stop")==0) {
                ftp->abort();
                unsendFileProcess(_filename, "Transfer Cancelled");
                proc->process("mv ../"+_destination+"/outbox/"+_filename+" ../files_unsent","MOVING FILES FROM BRANCH _ OUTBOX TO FILES UNSENT");
                QSqlQuery qry2;
                qry2.exec("delete from filessending where filename='"+_filename+"' and destination='"+_destination+"';");
            }
        }
    }
    proc->close();
    proc = NULL;
    delete proc;
}
=======================FileMonitoring.cpp===================
FileMonitoring::FileMonitoring() : QTimer(0)
{
    connect(this,SIGNAL(timeout()),this,SLOT(timeout()));
}
void FileMonitoring::timeout()
{
    QDir *qd=new QDir();
    QStringList stringList;
    bool flag = true;
   qd->refresh();
    qd->setCurrent("../outbox");
    stringList=qd->entryList(QDir::Files,QDir::Time);
    stop();
    for (int index = 0; index < stringList.size(); ++index) {
        QFileInfo fileInfo=QFileInfo(stringList.at(index));
        QDateTime modtime=fileInfo.lastRead();
        flag = patternCheck(fileInfo.fileName()); //PATTERN CHECKING
        if(flag) {
            int filestay=modtime.secsTo(QDateTime::currentDateTime());
            if(filestay>=20) {
                flag = checkingFile(fileInfo);
                if(flag) {
                    flag = branchExtract(fileInfo);
                }
           }
        } else {
            Process *pro=new Process();
            unsendFileProcess(fileInfo.fileName(), "Pattern Mismatch!");
            pro->process("mv "+fileInfo.fileName()+" ../files_unsent", "FILE MOVING TO UNSEND FOLDER");
            pro->close();
            pro = NULL;
            delete pro;
        }
    }
    delete qd;
    start(interval());
}
==================main.cpp========================
int main(int argc, char *argv[]) {
    QApplication a(argc,argv);
    connectingDB();
    fileTransfer();
    startProcessThread();   
    return a.exec();
}
void fileTransfer() {
    FileMonitoring *objMonitor=new FileMonitoring();
    objMonitor->start(6000);
}
void startProcessThread() {
    QSqlQuery query;
    query.exec(" select br_id, "
               " br_code, "
               " br_name, "
               " ipaddress, "
               " alt_ipaddress "
               " from branch"
               //" where br_id in(42)"
               " order by br_name;");
    int index = 0;
    while(query.next() && index<10)   {
        QString threadName = "THREAD_" + query.value(0).toString()
                             + "_" + query.value(1).toString()
                             + "_" + query.value(2).toString();
        QString ipaddress=query.value(3).toString();
        QString alternateIpaddress=query.value(4).toString();
        QString branchName=query.value(2).toString();
        thread[index]=new TransferThread(threadName);
        thread[index]->setBranch(branchName);
        thread[index]->setAlternateHostName(alternateIpaddress);
        thread[index]->setHostName(ipaddress);
        thread[index]->initFTP();
        index++;
    }
    totalThread=index+1;
}
}
=============Process.cpp====================
Process::Process() : QProcess(0) {  
bool Process::process(QString process, QString message) {
    _processCommand=process;
    _processMessage=message;
    start(process);
    if(!waitForStarted()) {
        return false;
    }
    if(!waitForFinished()) {
        return false;
    }
    return true;
}

============TransferThread.cpp================
TransferThread::TransferThread(QString ThreadName)  : QThread(0) {
    stopFlag = false;
    _name = ThreadName;
    _userName = "administrator";
    _password = "admin";
    port = 9012;
    connectingDB();
}

void TransferThread::initFTP() {
    ftp = new QFtp(this);
    QTimer *loginTimer=new QTimer();
    loginTimer->start(500);
    connect(loginTimer,SIGNAL(timeout()),this,SLOT(loginFTP()));
    connect(ftp, SIGNAL(commandFinished(int,bool)),this,SLOT(commandFinished(int,bool)));
    connect(ftp, SIGNAL(dataTransferProgress(qint64,qint64)),this,SLOT(dataTransferProgress(qint64,qint64)));
    connect(ftp, SIGNAL(stateChanged(int)),this,SLOT(stateChanged(int)));
}
void TransferThread::loginFTP() {
    if((ftp->state()==QFtp::Unconnected || ftp->state()==QFtp::Closing) )   {
        this->terminate();
        ftp->abort();
        if(_checkingHostName==_hostName) {
            _checkingHostName = _alternateHostName;
        } else {
            _checkingHostName = _hostName;
        }
        ftp -> connectToHost(_checkingHostName,port);
        ftp -> login(_userName, _password);
    }
    if(ftp -> state() == QFtp::LoggedIn) {
        this->start();
    }
}
void TransferThread::run() {
    complete=0;
    f_done=0;
    f_total=0;
    while(!stopFlag) {
        if(complete==0) {
            if(fileSelection(branchName)) {
                fileSendProcess();
            }
        } else {
                qWarning() << "ENDLESS LOOP NOT STOPPING"
        }
        sleep(2);
    }
}
void TransferThread::fileSendProcess() {
    if(sendFileProcess() ) {
        QMutex *mut=new QMutex();
        mut->lock();
        sendDataFTP(_filename);
        mut->unlock();
    }
}
void TransferThread::sendDataFTP(QString fileName) {
    Process *proc = new Process();
    QString file="../"+_destination+"/outbox/"+fileName;
    QFile *filetrans= new QFile(file);
    if(filetrans->exists()) {
        if ( !filetrans->open( IO_ReadOnly ) ) {
            unsendFileProcess(fileName, "Cannot Open File");
            proc->process("mv "+fileName+" ../files_unsent", "FILE MOVING TO UNSEND FOLDER");
            filetrans = NULL;
            proc->close();
            filetrans->close();
            proc = NULL;
            delete proc;
            delete filetrans;
            return;
        }
    } else {
        filetrans = NULL;
        proc->close();
        proc = NULL;
        delete proc;
        delete filetrans;
        return ;
    }
    QSqlQuery qry;
    qry.exec("insert into filessending values('"+_filename+"','"+_destination+"');");
    QByteArray buffer;
    QMutex *mut_buffer=new QMutex();
    mut_buffer->lock();
    buffer = filetrans->readAll();
    mut_buffer->unlock();
    ftp->put(buffer,fileName,QFtp::Binary);
    complete=1;
    proc->close();
    proc = NULL;
    delete filetrans;
    delete proc;
}
void TransferThread::dataTransferProgress(qint64 done, qint64 total)
{
    Process *proc = new Process();
    QSqlQuery qryUpdate;
    QSqlQuery query;
    QString pbid;
    QString brid;
    QString fileName=_filename;
    pbid=publisherId(_filename.mid(0,2));
    brid=branchId(_destination);
    str_pbId=pbid;
    str_brId=brid;
    QString strUpdate = "update transferstatus set "
                        " done = "+QString::number(done)+","
                        " total="+QString::number(total)+","
                        " pb_id="+pbid+
                        " where filename='"+_filename+"' and br_id="+_destinationId+"";
    qryUpdate.exec(strUpdate);
    sendFileSize = done;
    int count=0;
    while(sendFileSize>1024) {
        count++;
        sendFileSize=sendFileSize/1024;
    }
    f_total=total;
    f_done=done;
    if(done==total) {
        qryUpdate.exec("delete from filessending where filename='"+_filename+"' and destination='"+_destination+"';");
        qryUpdate.exec("delete from file_transfer_queue where filename='"+_filename+"' and br_id="+brid+" and pb_id="+pbid+";");
        QTime now;
        QString csq;
        QTime strtTime;
        QString duration=getDuration(strtTime,now);
        QString timecurrent=QTime::currentTime ().toString(Qt::TextDate);
        strtTime=strtTime.fromString(_timestart, Qt::TextDate);
        now=now.fromString( timecurrent,Qt::TextDate );
        csq="insert into transreport(file, size, dot, start,"
            " finish, br_id, pb_id, duration,rid)values('"+_alias+
            "','"+_fileSize+"','"+QDate::currentDate ().toString()+
            "','"+_starttime+"','"+QTime::currentTime ().toString("hh:mm:ss ap")+
            "',"+brid+","+pbid+",'"+duration+"',"+_releaseid+");";
        query.exec(csq);
        int count=1;
        while(checkingSendFile(fileName)) {
            fileName.append(QString::number(count));
            count++;
        }
        proc->process("mv ../"+_destination+"/outbox/"+fileName + " ../sentfiles/", "MOVING FILE TO SEND FOLDER");
        complete=0;
    } else {
        QSqlQuery queryselect;
        QString selectqry="select status from transferstatus"
                          " where filename='"+_alias+
                          "' and br_id="+brid+" and pb_id="+pbid+";";
        queryselect.exec(selectqry);
        if(queryselect.next()) {
            QString statusfile=queryselect.value(0).toString();
            if(statusfile.compare("stop")==0) {
                ftp->abort();
                unsendFileProcess(_filename, "Transfer Cancelled");
                proc->process("mv ../"+_destination+"/outbox/"+_filename+" ../files_unsent","MOVING FILES FROM BRANCH _ OUTBOX TO FILES UNSENT");
                QSqlQuery qry2;
                qry2.exec("delete from filessending where filename='"+_filename+"' and destination='"+_destination+"';");
            }
        }
    }
    proc->close();
    proc = NULL;
    delete proc;
}
=======================FileMonitoring.cpp===================
FileMonitoring::FileMonitoring() : QTimer(0)
{
    connect(this,SIGNAL(timeout()),this,SLOT(timeout()));
}
void FileMonitoring::timeout()
{
    QDir *qd=new QDir();
    QStringList stringList;
    bool flag = true;
   qd->refresh();
    qd->setCurrent("../outbox");
    stringList=qd->entryList(QDir::Files,QDir::Time);
    stop();
    for (int index = 0; index < stringList.size(); ++index) {
        QFileInfo fileInfo=QFileInfo(stringList.at(index));
        QDateTime modtime=fileInfo.lastRead();
        flag = patternCheck(fileInfo.fileName()); //PATTERN CHECKING
        if(flag) {
            int filestay=modtime.secsTo(QDateTime::currentDateTime());
            if(filestay>=20) {
                flag = checkingFile(fileInfo);
                if(flag) {
                    flag = branchExtract(fileInfo);
                }
           }
        } else {
            Process *pro=new Process();
            unsendFileProcess(fileInfo.fileName(), "Pattern Mismatch!");
            pro->process("mv "+fileInfo.fileName()+" ../files_unsent", "FILE MOVING TO UNSEND FOLDER");
            pro->close();
            pro = NULL;
            delete pro;
        }
    }
    delete qd;
    start(interval());
}
==================main.cpp========================
int main(int argc, char *argv[]) {
    QApplication a(argc,argv);
    connectingDB();
    fileTransfer();
    startProcessThread();   
    return a.exec();
}
void fileTransfer() {
    FileMonitoring *objMonitor=new FileMonitoring();
    objMonitor->start(6000);
}
void startProcessThread() {
    QSqlQuery query;
    query.exec(" select br_id, "
               " br_code, "
               " br_name, "
               " ipaddress, "
               " alt_ipaddress "
               " from branch"
               //" where br_id in(42)"
               " order by br_name;");
    int index = 0;
    while(query.next() && index<10)   {
        QString threadName = "THREAD_" + query.value(0).toString()
                             + "_" + query.value(1).toString()
                             + "_" + query.value(2).toString();
        QString ipaddress=query.value(3).toString();
        QString alternateIpaddress=query.value(4).toString();
        QString branchName=query.value(2).toString();
        thread[index]=new TransferThread(threadName);
        thread[index]->setBranch(branchName);
        thread[index]->setAlternateHostName(alternateIpaddress);
        thread[index]->setHostName(ipaddress);
        thread[index]->initFTP();
        index++;
    }
    totalThread=index+1;
}
}
=============Process.cpp====================
Process::Process() : QProcess(0) {  
bool Process::process(QString process, QString message) {
    _processCommand=process;
    _processMessage=message;
    start(process);
    if(!waitForStarted()) {
        return false;
    }
    if(!waitForFinished()) {
        return false;
    }
    return true;
}

To copy to clipboard, switch view to plain text mode 

[neveffects]

Finally I have got the Solution for my problem. I have rechecked all the code and finally I found out that , while we are accessing any database inside the thread then use the connection created inside the thread itself. Donot use the default connection which may opened inside the main function. So that was the problem for the slow process of QFtp.

Thank you