It seems that Qt does not release unused memory

[jpn]

Posting the code is very difficult, it is a big project!

Then perhaps you could prepare a small example which reproduces the problem?

[iw2nhl]

Then perhaps you could prepare a small example which reproduces the problem?

It's not an easy task, because I don't know what part of the program causes it. Anyway I'll try...

I checked also with Valgrind and no memory leaks have been shown.

Then I did this test:
imagine that I need to load FILE1 and FILE2
FILE1 loaded takes 30 MB of RAM, FILE2 loaded takes 10 MB of RAM
now, if I load FILE1 and FILE2, the program occupies 40 MB of RAM
but if I load FILE1 and unload it, then load FILE2 and unload it, the program occupies 30 MB of RAM

This validates my first description: the memory previously used by FILE1 is marked as deleted and reused by FILE1, but never released to the OS.
If there was a memory leak, I would find 40 MB of RAM occupied in both the cases described above, but it's not the case.

[jacek]

How do you measure the memory used by your application?

[iw2nhl]

How do you measure the memory used by your application?

I use KDE System Guard 1.2.0, part of KDE 3.5.7: in the process table I read the columns "VmSize" and "VmRss".
While on Windows 2000 I use "Task Manager", with "VM Size" and "Mem Usage" columns.

I noticed a difference between Linux and Windows: Linux never releases the memory, while Windows releases most of it, but not all.
Eg.:
If the program, just started takes 10 MB of RAM and I load data for 30 MB (total 40 MB), when I unload the data, the program occupies 12 MB. From this point on, when I load and unload a file, it returns always to 12 MB (and never to 10 MB).
On Linux instead it remains always at the peak memory usage reached (in this example 40 MB), irrespective of what I load/unload.

[aMan]

How do you save your data?
In a QList? Do you delete the container or just empty it?

Maybe Qt doesn't frees the memory of the container but keeps it as a buffer.

[wysota]

On Linux instead it remains always at the peak memory usage reached (in this example 40 MB), irrespective of what I load/unload.

As far as I know this is a normal behaviour. Your kernel prefers to have the block of memory assigned to a given process in case it wants more memory at some later point in time. If you ran out of memory, it'd be freed and assigned to the process being in need of that memory. Try opening a large file, processing it and then repeating with a smaller file. You'll probably notice the memory footprint remains the same, which means the memory for the smaller file was allocated from within the previously deallocated chunk.

[iw2nhl]

Try opening a large file, processing it and then repeating with a smaller file. You'll probably notice the memory footprint remains the same, which means the memory for the smaller file was allocated from within the previously deallocated chunk.

This is exactly what is happening.

I could try to use all available memory with another program and see if the memory footprint is reduced.
I'll try it asap!
Thanks

[iw2nhl]

Ok, I loaded files up to use 304 MB of VmSize (about 270-280 MB of VmRss), then I closed all of them and I started opening every sort of program.
The result?
Memory usage of the program was identical for VmSize, while VmRss diminished about of 20-30 MB.
More than 130 MB of Swap memory were used (it was never happened before).

I have 512 MB of RAM in my system and a Swap partition of 258 MB.

I don't know how to interpret the result. Is this a behaviour of the kernel? Is this a Qt bug? Is there something wrong in my code?
I really don't know

[wysota]

In your case VmSize can go up to over 700MB. I think VmRss is the amount of memory really occupied by the application.

http://www.mozilla.org/projects/foot...int-guide.html

BTW. Increase your swap space. It should be about twice as large as the amount of RAM you have in your machine.

[niko]

you can find memory leaks with this very handy tool:
valgrind (http://valgrind.org/)

its fairly easy to use....


niko

[iw2nhl]

you can find memory leaks with this very handy tool:
valgrind (http://valgrind.org/)

Thanks for the suggestion, but I already used it!
See my post #5.
No memory leaks are shown.
Just a few KB caused by Qt and other linked libraries.

[iw2nhl]

I decided to make a very simple test.
It allocates a QVector, then it shows a message so that you can test memory usage, then the vector is deleted and you can test again memory usage.
In my system, both VmRss and VmSize do not change (actually just some KB).
Please, tell me if I'm doing something wrong!

#include <QtGui>
 
int main(int argc, char *argv[]) {
    QApplication app(argc, argv);
 
    // Memory allocation
    QVector<double> *vector = new QVector<double>;
    vector->resize(10000);
    QMessageBox::information(0, "Memory Allocation Test", "Memory allocated");
 
    // Memory deletion
    vector->clear();
    vector->squeeze();
    delete vector;
    QMessageBox::information(0, "Memory Allocation Test", "Memory deleted");
 
    return 0;
}

#include <QtGui>

int main(int argc, char *argv[]) {
    QApplication app(argc, argv);

    // Memory allocation
    QVector<double> *vector = new QVector<double>;
    vector->resize(10000);
    QMessageBox::information(0, "Memory Allocation Test", "Memory allocated");

    // Memory deletion
    vector->clear();
    vector->squeeze();
    delete vector;
    QMessageBox::information(0, "Memory Allocation Test", "Memory deleted");

    return 0;
}

To copy to clipboard, switch view to plain text mode 

[iw2nhl]

I tested also with a standard vector and it has the same behaviour, so probably it is a kernel bug/feature.

double *vector = new double[10000];
delete[] vector;

double *vector = new double[10000];
delete[] vector;

To copy to clipboard, switch view to plain text mode 

[wysota]

10000*sizeof(double) = ~800kB

BTW. Nobody said resize() actually causes any allocation... Try assigning a value to one of the last elements.

[Brandybuck]

As far as I know this is a normal behaviour. Your kernel prefers to have the block of memory assigned to a given process in case it wants more memory at some later point in time. If you ran out of memory, it'd be freed and assigned to the process being in need of that memory.

This is true for Linux, but other Unix systems have differing memory allocation behavior. Try your application under FreeBSD or Solaris for a comparison.

[wysota]

Yes, of course. That's natural as every system uses a different allocator implementation. The author said he was running Linux. Furthermore one Linux based system may behave differently than another, it depends on the kernel it is running (both version and options).

[fluxar]

As far as I know this is a normal behaviour. Your kernel prefers to have the block of memory assigned to a given process in case it wants more memory at some later point in time. If you ran out of memory, it'd be freed and assigned to the process being in need of that memory. Try opening a large file, processing it and then repeating with a smaller file. You'll probably notice the memory footprint remains the same, which means the memory for the smaller file was allocated from within the previously deallocated chunk.

I also thought I had a memleak, but after I tested this (loaded a bunch of other applications to fill the memory) my application freed the memory that it no longer used. Thanks wysota!