SelectAll causes freeze

[Khal Drogo]

Greetings.

I've described the main functionality of my intended application several times, but let me reiterate.

I receive data really fast from a worker thread, which i add to my model, and i update the view on every tenth packet received, with beginInsertRows/endInsertRows. I use a treeview (mainly because of the usefulness of uniformRowHeights, since at this moment, there are no child items, only top level items).
I added a popupmenu to be shown when i right click inside the view, and i have a select all command, for which i call selectAll() on the view.

At this point, the application freezes. This doesnt happen, when all the data is already received, i.e. when no more rows are inserted. I even tried disabling updates, blocking signals from the model, and such unorthodox practices, but it does not work.

Any ideas?

[marcel]

Probably this behavior appears when you call selectAll while the model data is being changed.
Try queuing the data when the menu becomes visible. You will not update the view every ten packets, instead you put them in a list and refresh the view when the popup closes.

But I don't know if it works.

[Khal Drogo]

Yes, i'm aware that that is the probable cause. Which i tried to eliminate by blocking signals from the model before i call the selectAll() method and unblock them right afterwards.
Even if this isn't the correct way to do it, the way i understand it, it should work, or at least it should eliminate the probability that the model inserts are causing this.
Am i wrong?

[marcel]

How do you exactly receive the data from the worker thread?

[Khal Drogo]

As objects which contain the data for an entire row in the view, through the signals and slots mechanism, the fastest way i can right now, 1000 every second.

[marcel]

It might be because the view does not refresh as fast as the model.
Since selectAll has an impact on the view too, and it queries the model directly for the items, you could obtain unpredictable behavior.

For example if you have 1000 items in the model and the view updates so it too contains 1000 items, and in the very next moment you add another 1000 items to the model, and call selectAll(before the view gets updated) then selectAll will try to update 2000 items in the view. Therefore the behavior.

But this is just an assumption.
What you can do is to queue model updating until selectAll end.

[Khal Drogo]

Let me see if i understood you correctly. So you are saying that i should take the following course, when selectAll is pressed:
1) set a flag, which will have the effect of the newly arrived data to be directed into a queue, instead of being added directly to the model data
2) call selectAll() on the view
3) clear the flag
4) add all items saved up in the queue to the model data.

And this all happens, because between the call of selectAll and the actual updating of the view, the model most certainly changes, since it gets data really fast?

I have to tell you, i didnt think it will be an issue, since somewhere down the path, selectAll uses a list a model indices which have to be set to selected and which are available at that exact moment. Now, it may happen that by the time the view updating occurs, there are many more actual items in the model, and thus, more indices, but there are never less, and i dont see how the originally selected indices could become invalid.
What is my misconception about all of this?

[marcel]

I have to tell you, i didnt think it will be an issue, since somewhere down the path, selectAll uses a list a model indices which have to be set to selected and which are available at that exact moment. Now, it may happen that by the time the view updating occurs, there are many more actual items in the model, and thus, more indices, but there are never less, and i dont see how the originally selected indices could become invalid.
What is my misconception about all of this?

First of all, all I said is just a guess. It is pretty hard to state some facts.
Second of all, you misunderstood me.
I was trying to say that it is possible to have a discrepancy between the number of indexes in the model and the number of items actually painted in the view.

When you call selectAll, it will try to select N items in the view, while the view contains only M items, where N >= M.

So, this could even be a stupid idea, but it would be great if you could do some debug to see if it is true, or even implement it, it's not a big deal.

[Khal Drogo]

Sorry for the misunderstanding.

When i said "I have to tell you", i didn't mean to sound condescending or preachy, or anything, it just means, that it really didn't occur to me. This expression is an unfortunate american mannerism that i picked up.

I understand you now, i hope and i believe you are right.
I modified the code so that for every new item, a beginInsertRows/endInsertRows method is called, not just for every tenth.
The select all now works, and i think its what you said, because whenever i update the view for only every tenth item, it may happen that the model contains up to 9 items more than the view actually shows.

Nevertheless, i need this item "buffering" for speed reasons. What can i do to keep this kind of buffering and also to be able to use selectAll? The two issues seem irreconciliable.

[marcel]

Nevertheless, i need this item "buffering" for speed reasons. What can i do to keep this kind of buffering and also to be able to use selectAll? The two issues seem irreconciliable.

What about buffering the row data in the GUI thread? From the point selectAll starts, everything that comes from the worker thread goes to a temporary buffer. After select all ends, emit a signal and in the slot just update the model with the items in the temp buffer.

[wysota]

Could we see the code you use for adding data to the model?

[Khal Drogo]

This is all i use right now to add/modify my model data.

if (m_rowNumsForItems.contains(item.ObjectId) == false)
	{	
        /// if the item isn't already in the list, it is added, and the resulting rownum recorded.
		if (rowCount() - m_lastUpdatedRow == 10)
		{
			beginInsertRows(QModelIndex(), m_lastUpdatedRow, rowCount());
		}
		m_items.push_back(item);
		m_rowNumsForItems.insert(item.ObjectId, rowCount() - 1);
		if (rowCount() - m_lastUpdatedRow == 11)
		{
			endInsertRows();
			m_lastUpdatedRow = rowCount();
		}
	}
	else
	{
        /// if the item is in the list, it is replaced and dataChanged is emitted.
		rowNum = m_rowNumsForItems.value(item.ObjectId);
		m_items.replace(rowNum, item);
		emit dataChanged(index(rowNum, 0), index(rowNum, columnCount() - 1));
	}

if (m_rowNumsForItems.contains(item.ObjectId) == false)
	{	
        /// if the item isn't already in the list, it is added, and the resulting rownum recorded.
		if (rowCount() - m_lastUpdatedRow == 10)
		{
			beginInsertRows(QModelIndex(), m_lastUpdatedRow, rowCount());
		}
		m_items.push_back(item);
		m_rowNumsForItems.insert(item.ObjectId, rowCount() - 1);
		if (rowCount() - m_lastUpdatedRow == 11)
		{
			endInsertRows();
			m_lastUpdatedRow = rowCount();
		}
	}
	else
	{
        /// if the item is in the list, it is replaced and dataChanged is emitted.
		rowNum = m_rowNumsForItems.value(item.ObjectId);
		m_items.replace(rowNum, item);
		emit dataChanged(index(rowNum, 0), index(rowNum, columnCount() - 1));
	}

To copy to clipboard, switch view to plain text mode 

(The 10 is purely arbitrary, and of course, i need to add some kind of timer, which i dont currently have, so that it doesnt happen that the last <10 items are never shown )

[wysota]

And how do you call it?

[Khal Drogo]

The worker thread emits a signal, with an item as parameter. In the slot connected to it in the main thread i call this addItem function, with the received item as parameter.

[wysota]

For each single row separately? Why not insert all pending data at once?

[Khal Drogo]

I don't really understand what you mean by pending. Slot is called with 1 item as parameter, addItem is called in the model with that one item. That is what happens. I tried to lessen the burden on the ui by doing that beginInsertRows/endInsertRows only for every 10 of these items, in bulk, but maybe that was a mistake, because it has the effect that the view has always less items then the model behind it.
Maybe i should queue the items *always*, in the slot, and add them to the model in bulk?
Actually taking this buffering outside the model?

[wysota]

Now I think you are doing something weird... Isn't it true that you do:
insert row 1
insert row 2
...
insert row 9
beginInsertRows
insert row 10
endInsertRows

Instead you should be doing:
add row 1 to other list
add row 2 to other list
...
add row 10 to other list
beginInsertRows
add all 10 rows from the other list to the model list
endInsertRows

[Khal Drogo]

Exactly what im doing now, and exactly the solution which i asked i should do.

Yes, now that i think of it, it is weird.

I'll try to do just that + a timer (because it can happen that after a while, the initial surge of data dies down, and maybe i receive 10 new items in 5 minutes, instead of 10ms, so i periodically have to transfer the list to the model list).

I will try this, hope it works.

Thanks a lot, Wysota, and you too Marcel, i'm very glad you both are willing to devote time to sort out through my rookie mistakes.

[wysota]

You can probably use the canFetchMore() and fetchMore() feature of the model to do it without timers.

[Khal Drogo]

I use the treeview currently as a tableview, i.e. only top level items, no children.
I've never used canFetchMore() and fetchMore(), could you please explain a bit how it can help?