QStandardItemModel crash in its contructor

[cafu]

Hi everybbody,

i facing a small trouble using the QStandardItemModel class. Here is the example code:

numRow = 3883439;
numCol = 23;
QStandardItemModel * model = 0;			
model = new QStandardItemModel(numRow, numCol);

numRow = 3883439;
numCol = 23;
QStandardItemModel * model = 0;			
model = new QStandardItemModel(numRow, numCol);

To copy to clipboard, switch view to plain text mode 

when i call new the program crash. If numRow is < 2900000 is working fine otherwise no;a least with 3000000 is no working, with middle values I am not sure, but i think this show my point.
I suppose is a memory allocation problem. I have implemented if_condition to check the values of numRow and numCol before the calling of new to avoid the crash, but i would like to know whay is thi happening an if there is a way to catch error like this type in QT, and if exist wich it is? or how to know the limit for the memory allocation of each class?,
cause in the documentation of the class I did not see any specification for the limits for row and cols.

Thank for the Help

[wysota]

You're probably running out of memory. 3.8M rows and 23 columns, assuming each cell would occupy around 50 bytes of memory gives a total of >4GB of memory required to hold that data. Unless you have a 64b computer with more than 4GB of memory (and more than 4GB addressable for a single process), you won't be able to create such a model. With standard 32b installation of Windows a single process can only allocate around 2,7G of memory.

[cafu]

HI Wysota, thanks for the answer,

So the only way is to prevent to call the constructor with correct values, within the limit of the memory or any other limit.
But there is no way to catch if miss in somewhere to check the values, i know is no best but, as now with this problem, I do not know all this limit and perhaps i miss then, and i do not want the software to crash, i would like to correct the error or at least know the reason of this.

Something else, where can I find some of this Information, some documents or link to study and know more about this?

Can I have the same problem also when i load a big file and i use the function read all?

Thanks in advance

[wysota]

So the only way is to prevent to call the constructor with correct values, within the limit of the memory or any other limit.

There is a question why would you want to construct a model to hold so many items upfront. You can always add new rows when they are needed. And you can use other (custom) models that would not require so much memory.

Something else, where can I find some of this Information, some documents or link to study and know more about this?

Some of what information? In general you have to read how computer memory works.

Can I have the same problem also when i load a big file and i use the function read all?

Yes, of course. Provided your filesystem allows so large files of course. Just consider the fact that when using readAll() in practice you use memory at least twice the size of the file:

QString content = file.readAll();

QString content = file.readAll();

To copy to clipboard, switch view to plain text mode 

content is a QString and readAll() returns a byte array so both variables have to co-exist in memory for some time and QString is unicode-based so it may take up much more memory than the byte array.

[cafu]

There is a question why would you want to construct a model to hold so many items upfront. You can always add new rows when they are needed. And you can use other (custom) models that would not require so much memory.

The answer will be: cause I am trying to load big log files and i thoug if i ask for need it rows it will be faster. About the custom model i am not so familiar with this but i suppose i have to create a class that inherit from QAbstractItemModel.

Some of what information? In general you have to read how computer memory works.

Ok, thanks.

Yes, of course. Provided your filesystem allows so large files of course. Just consider the fact that when using readAll() in practice you use memory at least twice the size of the file:

QString content = file.readAll();

QString content = file.readAll();

To copy to clipboard, switch view to plain text mode 

content is a QString and readAll() returns a byte array so both variables have to co-exist in memory for some time and QString is unicode-based so it may take up much more memory than the byte array.

actually i am using QByteArray. but i got the point. thanks

[wysota]

The answer will be: cause I am trying to load big log files

I doubt you'll have as many entries. And even if you do, there is no point in creating all the items upfront. Create them as you need them or better yet load only the part of the log you really need.

actually i am using QByteArray. but i got the point. thanks

If you're displaying the data in the view, you'll eventually be holding everything in QString.

[cafu]

Hey Thanks for the fast answer,

and believe me, there are so many rows, it is a old log implemantation and some one forgot to put limit to the log files, they can grow as much as (im)posibble. But anyway this are rare cases.

OK, Thanks so much, one last question about the custom model, is that what you mind or am I mistaken.

[wysota]

and believe me, there are so many rows, it is a old log implemantation and some one forgot to put limit to the log files, they can grow as much as (im)posibble.

They can't because the filesystem imposes a limit on the file size. Anyway displaying the whole log wouldn't make much sense, you wouldn't be able to do anything with it as any manipulation of such model would max out your cpu usage.

OK, Thanks so much, one last question about the custom model, is that what you mind or am I mistaken.

Yes, more or less.

[cafu]

Hi,

Well the file size is around 1 GB,and I think with such as file there is no problem with the OS

Thanks for the help