libcurl under Windows does not write data after realloc

[Ishmael]

I'm trying to get a libcurl script to run under Windows. I know about QNetworkRequest, but I would just like to get this working, if possible. It works fine under Linux.

For some reason, the first time the WriteMemoryCallback function is called, data is written as expected. However, on subsequent call to the same function, new data does not get appended to the array after a realloc. Any idea what is going on? Thanks for your help!

Copying straight from the example provided by the libcurl folks:

curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, WriteMemoryCallback);  // Rather than writing to a file, keep data in memory
curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, (void *)&chunk);

curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, WriteMemoryCallback);  // Rather than writing to a file, keep data in memory
curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, (void *)&chunk);

To copy to clipboard, switch view to plain text mode 

size_t WriteMemoryCallback(void *ptr, size_t size, size_t nmemb, void *data) {
 
    size_t realsize = size * nmemb;
    struct MemoryStruct *mem = (struct MemoryStruct *)data;
 
    mem->memory = (char *)realloc(mem->memory, mem->size + realsize + 1);
    if (mem->memory == NULL) {
        printf("not enough memory (realloc returned NULL)\n");
        exit(EXIT_FAILURE);
    }
 
    memcpy(&(mem->memory[mem->size]), ptr, realsize);
    mem->size += realsize;
    mem->memory[mem->size] = 0;
 
    // THIS IS CORRECT
    qDebug()<<"Bytes received:";
    qDebug()<<QString::fromUtf8((char *)ptr);
 
    // NEW DATA DID NOT GET APPENDED!
    qDebug()<<"Bytes in memory:";
    qDebug()<<QString::fromUtf8(mem->memory);
 
    return realsize;
}

size_t WriteMemoryCallback(void *ptr, size_t size, size_t nmemb, void *data) {

    size_t realsize = size * nmemb;
    struct MemoryStruct *mem = (struct MemoryStruct *)data;

    mem->memory = (char *)realloc(mem->memory, mem->size + realsize + 1);
    if (mem->memory == NULL) {
        printf("not enough memory (realloc returned NULL)\n");
        exit(EXIT_FAILURE);
    }

    memcpy(&(mem->memory[mem->size]), ptr, realsize);
    mem->size += realsize;
    mem->memory[mem->size] = 0;

    // THIS IS CORRECT
    qDebug()<<"Bytes received:";
    qDebug()<<QString::fromUtf8((char *)ptr);

    // NEW DATA DID NOT GET APPENDED!
    qDebug()<<"Bytes in memory:";
    qDebug()<<QString::fromUtf8(mem->memory);

    return realsize;
}

To copy to clipboard, switch view to plain text mode 

In my .h file, I have:

struct MemoryStruct {
    char *memory;
    size_t size;
};
static size_t WriteMemoryCallback(void *ptr, size_t size, size_t nmemb, void *data);
 
class MainWindow : public QMainWindow
{
    Q_OBJECT
    etc.

struct MemoryStruct {
    char *memory;
    size_t size;
};
static size_t WriteMemoryCallback(void *ptr, size_t size, size_t nmemb, void *data);

class MainWindow : public QMainWindow
{
    Q_OBJECT
    etc.

To copy to clipboard, switch view to plain text mode 

[high_flyer]

To be sure you don't have an issue with charachter encoding and similar related problems that just make the qDebug() string not visible, put a breakpoint on that line and look in to the mem->memory memory in the debugger.

[Ishmael]

Thanks for your reply! You got me on the right track. This appears to be a display bug with Qt Creator. There is some sort of buffer in the debug output that causes a delay. The first thing I did was replace qDebug() with fprintf(stderr...), printing one character at a time. Doing this, I noticed that the output to each fprintf() call was truncated, but that subsequent calls to fprintf() completed the previous call's output. This does not happen using qDebug (it must clear the buffer each time).

In any case, the good news is that the webpage does seem to be downloaded correctly, as verified by a) writing the output to a file b) converting to a QString and using a regular expression to look for the terminating </html>. The only problem is, I can't figure out how to display it within Qt Creator (using the Watch menu doesn't work either - all I see is a small image of a musical eighth note. Charming.).

[high_flyer]

Is this a console application (QCoreApplication) or a gui application (QApplication)?
Are you in release or debug mode?
Do you want the output only for debug purposses or do you want it as an application feature?
qDebug() is only a debug output.
If this is a GUI app you probably should output it to a widget, or if you really want it on the console, you should output it to std output.

[Ishmael]

This is a GUI application. I'm in debug mode, and I just wanted to see the output for debugging. I finally found a simple workaround, which is to use split("\n") to break the webpage (one long string) into multiple lines, then use a loop to print each line. Thanks for your help!