QString :: is there some kind of "named place markers" (like in python string) ?

[lexar]

Hi Folks,

I would like to load preformated strings from some persistence and apply current parameters to them.
But the simple numbered parameter markers can be mixed easily while creating string patterns. Thus i would like to address these parameter markers by name. So:


Is there a way to name the parameter markers like python allows (instead of %1, %2, ...) ?
What is the best way (workaround) to do this ?


Details:
So I want something like the python string functionality:


>>> "Hello %(first_name)s %(last_name)s!" % {"first_name":"Max", "last_name":"Power"}
'Hello Max Power!'

[Lykurg]

see QString::arg()

[lexar]

please read the question before you push the "reply" button...

[Lykurg]

please read the question before you push the "reply" button...
Can you guess that because of the tonns of dumb newbie questions one only fly over such questions and answer it. Normaly one is 99% right with the answer. In your case I am not. Go and kill me. And sorry for trying to help you.
And as to your question:
QString str = "Go and read %censored manual aka documentation and try to use your %whatsoever";
str.replace("%censored","fucking").replace("%whatsoever","brain");
qWarning() << str;

[high_flyer]

please read the question before you push the "reply" button...
Where do you people get the nerve to be so obnoctious, and that towards someone who is trying to help you??
And THAT is what you have to write with your 4th!! post on this forum?
Have a look at how many posts Lykurg hast on this forum and to how many people he helped already!
Offending those who are trying to help you will not do you much good!

[lexar]

It should be hard to help other people under such pressure, keep cool! ;)

Of course i have noticed the member "replace(..)", the problem here is that on replacing a whole bunch of parameters we get pretty much temporary strings, what is pretty inefficient. (Sorry, I didn't mention this point in the initial post -- my fault, it's your turn to "Go and kill me" ).

I was looking for a more elegant solution like python does: a single call with a dictionary...

[high_flyer]

what is pretty inefficient.
In what way it is inefficient?
That your line is longer to type?

I also didn't understand the problem exactly... an example would help.

[lexar]

Where do you people get the nerve to be so obnoctious, and that towards someone who is trying to help you??
And THAT is what you have to write with your 4th!! post on this forum?
Have a look at how many posts Lykurg hast on this forum and to how many people he helped already!
Offending those who are trying to help you will not do you much good!

I'm sorry, this sounds some kind of aggressive to you, was not my intention!

I was trying to give enough information (repeating stuff, and making examples) so it should be clear, that i'm familiar with the "%N" notation and the basic stuff. But seems that I have to practice a bit more...

[high_flyer]

I was trying to give enough information (repeating stuff, and making examples) so it should be clear,
no, I respoded to THIS post:

please read the question before you push the "reply" button...

[lexar]

In what way it is inefficient?
That your line is longer to type?

Assume we have a text with, let say 30 placeholders. So calling 30 times the replace() function on a each time newly created temporary string looks pretty inefficient to me.

Seems that I have to write the required functionality by my self. Receiving a string-pattern and a hash-map, exchanging each next found placeholder by a value from the map in a single buffer.

But if you have a better Idea, please let me know...

[lexar]

no, I respoded to THIS post:

I know, I tried to explain why I replied it this way.

[Lykurg]

Of course i have noticed the member "replace(..)", the problem here is that on replacing a whole bunch of parameters we get pretty much temporary strings, what is pretty inefficient.
Not sure what you exactly mean by "temporary strings", but replace gives you an const reference back, so it is working on its own data directly without creating a working copy of it content:
QString &QString::replace(const QString &before, const QString &after, Qt::CaseSensitivity cs)
{
return replace(before.constData(), before.size(), after.constData(), after.size(), cs);
}

QString &QString::replace(const QChar *before, int blen,
const QChar *after, int alen,
Qt::CaseSensitivity cs)
{
if (d->size == 0) {
if (blen)
return *this;
} else {
if (cs == Qt::CaseSensitive && before == after && blen == alen)
return *this;
}
if (alen == 0 && blen == 0)
return *this;

QStringMatcher matcher(before, blen, cs);

int index = 0;
while (1) {
uint indices[1024];
uint pos = 0;
while (pos < 1023) {
index = matcher.indexIn(*this, index);
if (index == -1)
break;
indices[pos++] = index;
index += blen;
// avoid infinite loop
if (!blen)
index++;
}
if (!pos)
break;

replace_helper(indices, pos, blen, after, alen);

if (index == -1)
break;
// index has to be adjusted in case we get back into the loop above.
index += pos*(alen-blen);
}

return *this;
} The only thing is that every time a QStringMatcher is created. But that's normally not such a big thing. (if you don't work on an app werer every 0,0001 ms counts.)

EDIT: ...and what I forgot to mention/paste: replace_helper() also don't create a temporary copy.

[high_flyer]

Assume we have a text with, let say 30 placeholders. So calling 30 times the replace() function on a each time newly created temporary string looks pretty inefficient to me.
I would tend to say that working like that with a strings is inefficient on its self, and if you have to replace 30 place holders, then the syntax doesn't change much, you still have to resolve 30 place holder, is some form, be it calling a function 30 times, or some other operations that will do the same job - but the job has to be done.
I would be interested to see your implementation, and see if it really will be more efficient.
But enough philosophy.

One way, which may be more efficient (if its not the way Qt is doing it in replace()) is to use QRegExp to look for the place holders and use QMap<QString,QString> as a dictionary.

[lexar]

Lykurg, thanks not giving me up

Nice solution, undoubted. But the replace_helper helps Qt to avoid temporary strings only for a single replace call.
So it helps in this case:

QString("one abd one lakgje one lakgjes one").replace("one", "atomic");

but it does not help for this case:

QString("one two three four five, ...").replace("one","1").replace("two","2").replace("three","3").rep...

[lexar]

I would tend to say that working like that with a strings is inefficient on its self, and if you have to replace 30 place holders, then the syntax doesn't change much, you still have to resolve 30 place holder, is some form, be it calling a function 30 times, or some other operations that will do the same job - but the job has to be done.
I would be interested to see your implementation, and see if it really will be more efficient.
But enough philosophy.

One way, which may be more efficient (if its not the way Qt is doing it in replace()) is to use QRegExp to look for the place holders and use QMap<QString,QString> as a dictionary.

Yea, unfortunately strings are inefficient, but until humans cannot read bitstreams... ;)

Seems like the QHash class is the right dictionary object, but also it seems, that I have to write the string parsing and buffer filling by my own, cause I want to avoid expensive heap allocation calls for temporary strings with partial replacements...

Thanks All!

[high_flyer]

Yes probably QHash is better then QMap.
However I still think that QRegExp in conjunction with QHash will be the best way to go.



Here we've passed the QRegExp to QString's replace() function to replace the matched text with new text.


QRegExp rx("&(?!amp;)"); // match ampersands but not &amp;
QString line1 = "This & that";
line1.replace(rx, "&amp;");
// line1 == "This &amp; that"
QString line2 = "His &amp; hers & theirs";
line2.replace(rx, "&amp;");
// line2 == "His &amp; hers &amp; theirs"


So you could do a similar think in a loop that iterates through your QHash.

[lexar]

Yes probably QHash is better then QMap.
However I still think that QRegExp in conjunction with QHash will be the best way to go.

Then I probably do not get the point?!?
As far as I understand QRegExp is able to find a placeholder pattern, but it cannot help to replace them (especially a bunch of them) -- or do I miss something?


Ah, in your just inserted example code, we still have the mentioned 30 calls to the replace(), resulting in 30 temporary strings. That was the "inefficient" point...

[Lykurg]

Ah, in your just inserted example code, we still have the mentioned 30 calls to the replace(), resulting in 30 temporary strings. That was the "inefficient" point...
I am still strugelling on understanding why and where Qt produces temporary strings if you use replace multiple on a string. But if you have a loop you only call it once per each "replaceword". So for that there is no temporary string created.
But that leads to: You have to make it by yourown and Qt does not provide a function for that.

QString yourString;
QHash<QString, QString> searchReplace;
//fill it
QHash<QString, QString>::const_iterator i = searchReplace.constBegin();
while (i != hash.constEnd()) {
yourString.replace(i.key(), i.value());
++i;
}

[high_flyer]

As far as I understand QRegExp is able to find a placeholder pattern, but it cannot help to replace them
Read the RegExp docs:

Search and Replace A regexp can replace all occurrences of a substring with a different substring, e.g., replace all occurrences of & with &amp; except where the & is already followed by an amp;.


Ah, in your just inserted example code, we still have the mentioned 30 calls to the replace(), resulting in 30 temporary strings. That was the "inefficient" point...
But if you call replace once for each iteration you wrote:

Nice solution, undoubted. But the replace_helper helps Qt to avoid temporary strings only for a single replace call.
So by calling it in a loop you are not getting any temps.

[lexar]

I am still strugelling on understanding why and where Qt produces temporary strings if you use replace multiple on a string.

Each call to replace() (in replace_helper) creates a new heap buffer for the new string instance (assumed the replacements are bigger, then the markers) see "resize(newLen);" call in replace_helper().
Okay, the hull of the string remains the same, but we have a heap allocation, and that is the expensive part!



So by calling it in a loop you are not getting any temps.

Can U make an example? Still no idea how it should work all together meeting my point: no temporary strings with partial replacements.

[lexar]

So, I think we are at the point to agree, there is no such thing like:


QString("%(one) %(two) %(three) %(four) ...").arg(qhash_dictionary);

Creating only a single new heap object for the new string content.

1:0 for python ;)

[high_flyer]

So, I think we are at the point to agree, there is no such thing like:

I didn't know that it was the debate ;-)

The question is, what is python doing behind the scenes.


Creating only a single new heap object for the new string content.
Well, not quite, since the dictionary has to have strings in it...
In the solution I offered you, you create a QHash, and a QRegExp, and that's it, so its not that much of difference in terms of memory usage.
From here on it s mostly splitting hairs.

[lexar]

I'm also not familiar with the python internals, the "1:0" is for the API.

Unfortunately your solution using regex does not fit the requirements: replacing bunch of different named-place-holders in a single string, avoiding temporary objects. (The place-holder strings (the keys) should not be counted, they exist in each case.)

Thanks anyway.

[high_flyer]

Unfortunately your solution using regex does not fit the requirements:
Why not?
Where do temporary objects are being created?

[lexar]

Where do temporary objects are being created?

each replace() call calls replace_helper(), which calls resize() with allocates a NEW, bigger heap block, for the growing string content.
Of course if you are lucky, and growing successes, you are done. It depends on memory fragmentation and system activity... See "void QString::resize(int size)".

[high_flyer]

each replace() call calls replace_helper(), which calls resize() with allocates a NEW, bigger heap block, for the growing string content.
That is simply not true.
That is only true (the resizing), if the resulting string is larger then the original string.
This can happen if your place holders are smaller then the string that replaces them. (line 18,41)
In such a case, more memory will be allocated for the resulting sting in any case.
Also note, that afterBuffer gets deleted at the end of the function, which means for each call not more then alen bytes will be allocated, and released (lines 61,65), and that too, only happens in the cases specified in the comment.

Here is the code:

void QString::replace_helper(uint *indices, int nIndices, int blen, const QChar *after, int alen)
{
// copy *after in case it lies inside our own d->data area
// (which we could possibly invalidate via a realloc or corrupt via memcpy operations.)
QChar *afterBuffer = const_cast<QChar *>(after);
if (after >= reinterpret_cast<QChar *>(d->data) && after < reinterpret_cast<QChar *>(d->data) + d->size) {
afterBuffer = static_cast<QChar *>(qMalloc(alen*sizeof(QChar)));
Q_CHECK_PTR(afterBuffer);
::memcpy(afterBuffer, after, alen*sizeof(QChar));
}

QT_TRY {
detach();
if (blen == alen) {
// replace in place
for (int i = 0; i < nIndices; ++i)
memcpy(d->data + indices[i], afterBuffer, alen * sizeof(QChar));
} else if (alen < blen) {
// replace from front
uint to = indices[0];
if (alen)
memcpy(d->data+to, after, alen*sizeof(QChar));
to += alen;
uint movestart = indices[0] + blen;
for (int i = 1; i < nIndices; ++i) {
int msize = indices[i] - movestart;
if (msize > 0) {
memmove(d->data + to, d->data + movestart, msize * sizeof(QChar));
to += msize;
}
if (alen) {
memcpy(d->data + to, afterBuffer, alen*sizeof(QChar));
to += alen;
}
movestart = indices[i] + blen;
}
int msize = d->size - movestart;
if (msize > 0)
memmove(d->data + to, d->data + movestart, msize * sizeof(QChar));
resize(d->size - nIndices*(blen-alen));
} else {
// replace from back
int adjust = nIndices*(alen-blen);
int newLen = d->size + adjust;
int moveend = d->size;
resize(newLen);

while (nIndices) {
--nIndices;
int movestart = indices[nIndices] + blen;
int insertstart = indices[nIndices] + nIndices*(alen-blen);
int moveto = insertstart + alen;
memmove(d->data + moveto, d->data + movestart,
(moveend - movestart)*sizeof(QChar));
memcpy(d->data + insertstart, afterBuffer, alen*sizeof(QChar));
moveend = movestart-blen;
}
}
} QT_CATCH(const std::bad_alloc &) {
if (afterBuffer != after)
qFree(afterBuffer);
QT_RETHROW;
}
if (afterBuffer != after)
qFree(afterBuffer);
}

[lexar]

That is simply not true.
That is only true (the resizing), if the resulting string is larger then the original string.

And in my world that is the most probably case: when the replacement is bigger then the place-marker (simplified):

1: QString("%(placeholder_for_streetname) %(placeholder_for_cityname)").arg({...:"sun street", ...:"suncity"})
2: QString("%(street) %(city)").arg({...:"simpsons road", ...:"springfeeld"})

// note: arg() is a fictive construct receiving a dictionary (just to demonstrate the dict content)

Thus this is _the_ true, at least for me...

[high_flyer]

Thus this is _the_ true, at least for me...
Please note exactly what I wrote in response to your:

regex does not fit the requirements:...avoiding temporary objects.
I showed you that no temp objects are being created.

What you wrote:

when the replacement is bigger then the place-marker (simplified):
Is about the resizing of the string, and this happens in any case/language/api, and has nothing to do with temp objects, in the case you are using too.

[lexar]

Forget it. Seems our understanding of efficiency is just too different.

Thanks anyway...