Custom proxy model issue

[Khal Drogo]

Greetings.

I have a custom AbstractTableModel, which receives data really fast. I buffer them and emit a beginInsertRows/endInsertRows for a few items at a time.
In my custom view, in rowsInserted, i continually update a label which shows the rowCount.
Until now, i used a basic QSortFilterProxyModel, no problems.
Recently though, i had to switch to a custom one, derived from QSortFilterProxyModel. I basically followed the Custom Sort/Filter Model example.
I have the following problem:
I set the dynamic sort/filter flag on and set a custom filter. The items keep arriving, and not only the number representing the number of items does not refresh correctly, in the view the number of rows seems to grow, then shrink, grow again, etc.
Note: those did not happen when i used a simple QSortFilterProxyModel.
If all the data is fully loaded, no more row inserts happen, the filtering works fast and correctly.

What could be the problem?

Thank you.

[wysota]

What do you mean by a custom filter? Did you reimplement filterAccepts*() methods?

[Khal Drogo]

Hello. The filterAcceptsRow method, i did.
The options provided by filterAcceptsColumn i don't need it yet, but i'll reimplement it, when i do. (basically i was thinking that i should use that instead of setting the column visibility in the view, to regulate which columns are visible, but i haven't decided yet).

[wysota]

So what is in your filterAcceptsRow()?

[Khal Drogo]

The following:
I have an "AdvancedFilter" class, which contains a list of "and" filters and a list of "or" filters. Each one of these filters contain the column ID, the action and the value.
Basically, one AdvancedFilter looks like this:
"name has 'xy' and date < '11-02-2008' or age > 18"
andList:
{ nameId, FilterAction_Has, "xy"} { dateId, FilterAction_LessThan, '11-02-2008' }
orList:
{ ageId, FilterAction_MoreThan, "18"}

for (it = andList.begin(); it != andList.end(); it++)
	{
		index = sourceModel()->index(sourceRow, (*it).m_column, sourceParent);
		switch ((*it).m_action)
		{
		case FilterAction_Has:
			if (sourceModel()->data(index).toString().contains((*it).m_value) == false)
				accept = false;
			break;

for (it = andList.begin(); it != andList.end(); it++)
	{
		index = sourceModel()->index(sourceRow, (*it).m_column, sourceParent);
		switch ((*it).m_action)
		{
		case FilterAction_Has:
			if (sourceModel()->data(index).toString().contains((*it).m_value) == false)
				accept = false;
			break;

To copy to clipboard, switch view to plain text mode 

This is a big switch, for all possible actions. Then i do the same for orList, with the difference that if the data fits the filter pattern, then i set accept to true.
Finally i return accept.

[wysota]

Do you invalidate the filter after changing "ands" and "ors"?

[Khal Drogo]

Yes. I have a setFilter method, where i pass an AdvancedFilter and set the currentFilter in my proxy model to that one, afterwards i call invalidate.

[wysota]

Disable filtering (make filterAcceptsRow return true for every index) and see if the application starts behaving correctly. If so, the problem must be within the filtering method. In that case post the whole method here.

[Khal Drogo]

Yes, it does, so here it is:

bool AdvancedSortFilterProxyModel::filterAcceptsRow(int sourceRow, const QModelIndex &sourceParent) const
{
	bool accept = true;
	const QList<Filter>& andList = m_currentFilter.andList();
	const QList<Filter>& orList = m_currentFilter.orList();
	QList<Filter>::const_iterator it;
	QModelIndex index;
 
	for (it = andList.begin(); it != andList.end(); it++)
	{
		index = sourceModel()->index(sourceRow, (*it).m_column, sourceParent);
		switch ((*it).m_action)
		{
		case FilterAction_Has:
			if (sourceModel()->data(index).toString().contains((*it).m_value) == false)
				accept = false;
			break;
		case FilterAction_Is:
			if ((sourceModel()->data(index).toString() == (*it).m_value.pattern()) == false)
				accept = false;
			break;
		case FilterAction_LessThan:
			if ((sourceModel()->data(index).toDouble() < (*it).m_value.pattern().toDouble()) == false)
				accept = false;
			break;
		case FilterAction_MoreThan:
			if ((sourceModel()->data(index).toDouble() > (*it).m_value.pattern().toDouble()) == false)
				accept = false;
			break;
		default:
			break;
		}
	}
 
	for (it = orList.begin(); it != orList.end(); it++)
	{
		index = sourceModel()->index(sourceRow, (*it).m_column, sourceParent);
		switch ((*it).m_action)
		{
		case FilterAction_Has:
			if (sourceModel()->data(index).toString().contains((*it).m_value))
				accept = true;
			break;
		case FilterAction_Is:
			if ((sourceModel()->data(index).toString() == (*it).m_value.pattern()))
				accept = true;
			break;
		case FilterAction_LessThan:
			if ((sourceModel()->data(index).toDouble() < (*it).m_value.pattern().toDouble()))
				accept = true;
			break;
		case FilterAction_MoreThan:
			if ((sourceModel()->data(index).toDouble() > (*it).m_value.pattern().toDouble()))
				accept = true;
			break;
		default:
			break;
		}
	}
	return accept;
}

bool AdvancedSortFilterProxyModel::filterAcceptsRow(int sourceRow, const QModelIndex &sourceParent) const
{
	bool accept = true;
	const QList<Filter>& andList = m_currentFilter.andList();
	const QList<Filter>& orList = m_currentFilter.orList();
	QList<Filter>::const_iterator it;
	QModelIndex index;

	for (it = andList.begin(); it != andList.end(); it++)
	{
		index = sourceModel()->index(sourceRow, (*it).m_column, sourceParent);
		switch ((*it).m_action)
		{
		case FilterAction_Has:
			if (sourceModel()->data(index).toString().contains((*it).m_value) == false)
				accept = false;
			break;
		case FilterAction_Is:
			if ((sourceModel()->data(index).toString() == (*it).m_value.pattern()) == false)
				accept = false;
			break;
		case FilterAction_LessThan:
			if ((sourceModel()->data(index).toDouble() < (*it).m_value.pattern().toDouble()) == false)
				accept = false;
			break;
		case FilterAction_MoreThan:
			if ((sourceModel()->data(index).toDouble() > (*it).m_value.pattern().toDouble()) == false)
				accept = false;
			break;
		default:
			break;
		}
	}

	for (it = orList.begin(); it != orList.end(); it++)
	{
		index = sourceModel()->index(sourceRow, (*it).m_column, sourceParent);
		switch ((*it).m_action)
		{
		case FilterAction_Has:
			if (sourceModel()->data(index).toString().contains((*it).m_value))
				accept = true;
			break;
		case FilterAction_Is:
			if ((sourceModel()->data(index).toString() == (*it).m_value.pattern()))
				accept = true;
			break;
		case FilterAction_LessThan:
			if ((sourceModel()->data(index).toDouble() < (*it).m_value.pattern().toDouble()))
				accept = true;
			break;
		case FilterAction_MoreThan:
			if ((sourceModel()->data(index).toDouble() > (*it).m_value.pattern().toDouble()))
				accept = true;
			break;
		default:
			break;
		}
	}
	return accept;
}

To copy to clipboard, switch view to plain text mode 

I know it could be better, i.e. to use QString in the filter and convert it to QRegExp if needed, not the other way around, also, the if's themselves are not quite needed.
But i don't really see, what can cause that behaviour that i described previously...

[Khal Drogo]

Any ideas fresh in the morning perhaps?

[wysota]

The code seems ok, provided those two for loops work as you expect them to work. Which version of Qt are you using? I remember there were some issues about missing items related to the proxy back around Qt 4.2.

BTW. Your and-or mechanism is a bit ambigous. Does X and Y or Z mean (X and Y) or Z or X and (Y or Z)? Because you can only represent the first one using your approach. Storing everything in a tree-like structure would be more flexible. You could then even mix ands and ors - (A and (B or C)) or (D and E).

[Khal Drogo]

I am using 4.3.2.

( About the implementation, you are absolutely right. Using a tree with and/or as nodes and the filters themselves as leaves, then using inorder traversing to interpret the advanced filter would be a much better approach, and one i will definitely consider.

However, sadly, i dont think this influences the problem which i described. )

[wysota]

I suggest you insert some qDebug() statements here and there and use them to pinpoint what exactly happens. If you know what happens, it'll be easier to find a solution.

[Khal Drogo]

I will most certainly try Wysota, though i dont really know what to watch out for.

Thanks for your assistance, i appreciate it.