Custom Datatypes Causing Undefined Behavior Among Global Variable References

[brogrammer]

To give you a background, I have developed in C (and some C++ when it's called for) for the past several years, along with x86 assembly (inline and standalone). I am new to Qt and am using QT Creator 5.1 with MinGW compiler (Windows XP/no ASLR). My problem, in summary, is that I cannot use a global variable across different namespaces, as somehow the memory changes its location, even if dynamically allocated. I have tested this problem on another platform (VC++ and Linux/Clang) without using Qt and it works just fine.



The best way I can describe this problem is whenever I am using a user-defined datatype (struct serverUpdateInformation_t) within a namespace (NS_A) function, if I use a different variable of the same type (struct serverUpdateInformation_t) within a namespace (NS_B) which calls NS_A earlier in program flow, the offsets/fields within my struct change. Even using pre-processing directives like #pragma pack... do not work in keeping data uniform within memory.

Needless to say it is frustrating. I am trying to have just one copy of my variable available between "compilation units", but after trying several tricks the best I can get is the variable to have its memory address changed between functions, even though there is no explicit reassignment of the pointer location.

As an aside, these functions are not being compiled with the Q_OBJECT macro in their respective header files. I'm just trying to do "standard" C++, if not standard C. However, other "compilation units" are using the Q_OBJECT macro, which does end up calling some of these functions via my GUI code.

Example:

Assume variable "static serverLoginResponse_t serverLoginResponse" is defined within a "GlobalVARS.h" header file among two or more namespaces with functions which access the same variable.

namespace NS_A
{
   ...
   int NS_A::RecvUpdateInformation()
   {
      // Whenever I try to use the global variable "serverLoginResponse" directly in NS_A::FuncA, if I try to again use it in NS_A::FuncB or NS_B::FuncA, the compiler makes
      // another "copy" of the static global variable, thus negating the point of a global variable.
      // Thus, I use a pointer, which at least lets me keep the global variable in the same place in memory.
      Credentials::serverUpdateInformation_t* pServerUpdateInformation = &Credentials::serverUpdateInformation;
      ...
      // I allocate and setup fields below for a function in namespace NS_B to deal with...
      return status;
}

namespace NS_A
{
   ...
   int NS_A::RecvUpdateInformation()
   {
      // Whenever I try to use the global variable "serverLoginResponse" directly in NS_A::FuncA, if I try to again use it in NS_A::FuncB or NS_B::FuncA, the compiler makes
      // another "copy" of the static global variable, thus negating the point of a global variable.
      // Thus, I use a pointer, which at least lets me keep the global variable in the same place in memory.
      Credentials::serverUpdateInformation_t* pServerUpdateInformation = &Credentials::serverUpdateInformation;
      ...
      // I allocate and setup fields below for a function in namespace NS_B to deal with...
      return status;
}

To copy to clipboard, switch view to plain text mode 

The interesting thing is, when I try to achieve the same reference in an NS_B function, it alters the offsets of the struct fields in NS_A. Yes, I know! It will still compile fine, yet the offsets of the struct's fields change between "compilation units".

For example, if I use this following code in NS_B

namespace NS_B
{
   ...
   int Login()
   {
      ...
      // If we don't uncomment the following line(s), our offsets in referencing the struct get messed up, leading to improper computation.
      Credentials::serverUpdateInformation_t* pServerUpdateInformation = &Credentials::serverUpdateInformation;
      Credentials::serverUpdateInformation serverUpdateInformation_LocalCopy = Credentials::serverUpdateInformation;  // This line will cause offset problems as well, as an example.
      ...
      return status;
   }
}

namespace NS_B
{
   ...
   int Login()
   {
      ...
      // If we don't uncomment the following line(s), our offsets in referencing the struct get messed up, leading to improper computation.
      Credentials::serverUpdateInformation_t* pServerUpdateInformation = &Credentials::serverUpdateInformation;
      Credentials::serverUpdateInformation serverUpdateInformation_LocalCopy = Credentials::serverUpdateInformation;  // This line will cause offset problems as well, as an example.
      ...
      return status;
   }
}

To copy to clipboard, switch view to plain text mode 

This has been frustrating beyond belief. I know this is possible in standard C without any problems. It feels like I'm fighting against the framework in order to do basic low-level C/C++ within QT.
My current workaround solution was to serialize this datatype into a file after I initially referenced it in the NS_A function. Sadly, whenever I introduce a local variable of the same datatype within a different "compilation unit", it affects the memory layout of the global variable of the same type, even though they both should not interfere with each other!

Please help!

[ChrisW67]

The in-memory layout of a structure under C/C++ is controlled by the declaration and choices made by the compiler and has nothing at all to do with Qt. If your two separate compilation units are accessing the internals of a struct differently then either they were given different declarations from which to work, or they were built with different compiler options (e.g. affecting packing or alignment).

However, I think your problem is not with internal packing of structure but simply with multiple instance of the structure. You say we should assume a static global is defined in a header. Every place this header is included in a CPP file will create a separate instance of that variable: the values in these instances are independent and pointers to them will be different. The header should be written:

namespace Credentials {
 
struct serverUpdateInformation_t {
    // stuff
};
 
// This tells the C++ compiler the variable exists somewhere
extern serverUpdateInformation_t serverUpdateInformation;
 
}

namespace Credentials {

struct serverUpdateInformation_t {
    // stuff
};

// This tells the C++ compiler the variable exists somewhere
extern serverUpdateInformation_t serverUpdateInformation;

}

To copy to clipboard, switch view to plain text mode 

and in ONE cpp file:

#include "header.h"
// This is where the storage actually is and the linker will find it
Credentials::serverUpdateInformation_t serverUpdateInformation;

#include "header.h"
// This is where the storage actually is and the linker will find it
Credentials::serverUpdateInformation_t serverUpdateInformation;

To copy to clipboard, switch view to plain text mode 

[brogrammer]

Thank you for the response Chris.

However, even after changing "static" to "extern" and declaring within the .cpp file globally, accessing the variable either through reference or direct, will modify the program execution and data received from my server.

If need be, I can upload the .cpp and .h files. However, I'll copy and paste the snippets below:


Credentials.h

#ifndef CREDENTIALSTYPES_H
#include "CredentialsTypes.h"
#endif
 
namespace Credentials
{
    static loginSend_t                         loginSend;               // #1
    static serverLoginResponse_t               serverLoginResponse;     // #2
    static clientVersion_t                     clientVersion;           // #3
    extern serverUpdateInformation_t           serverUpdateInformation; // #4
 
   ...

#ifndef CREDENTIALSTYPES_H
#include "CredentialsTypes.h"
#endif

namespace Credentials
{
    static loginSend_t                         loginSend;               // #1
    static serverLoginResponse_t               serverLoginResponse;     // #2
    static clientVersion_t                     clientVersion;           // #3
    extern serverUpdateInformation_t           serverUpdateInformation; // #4

   ...

To copy to clipboard, switch view to plain text mode 

Credentials.cpp

namespace Credentials
{
    serverUpdateInformation_t   serverUpdateInformation;
 
    int Login()
    {
        // Establish a new ServerSession().
        QString dns = QLatin1String("login.xpextend.com");
        QString ipString = QLatin1String("192.168.56.102");
        QStringList ipNotation(ipString.split(QLatin1String(".")));
        quint32 ipAddress;
 
        Credentials::serverUpdateInformation_t* pServerUpdateInformation = &Credentials::serverUpdateInformation;
        // [B]Leaving in the above line, as you'll see below, ANYWHERE in this compilation unit will affect the ability of the QByteArray to receive information from the server[/B]
 
        ipAddress = ((ipNotation.at(0).toLong()) << 24) | ((ipNotation.at(1).toLong()) << 16) |
                    ((ipNotation.at(2).toLong()) << 8) | ((ipNotation.at(3).toLong()) << 0);
        ServerSession s(ipString);
 
        if (s.Connect() == ERROR_SUCCESS)
        {
            // Successfully connected.  Client successfully sent the whole loginResponse_t structure to the server.
            // Now we must read the serverLoginResponse_t structure.
            int loginResponse = s.ParseLoginResponse();
 
            if (loginResponse == ERROR_SUCCESS)
            {
                // Goal:  Determine if we need to download any new patches or client.
 
                // Attempt to send the server our version number.
                if (s.SendVersionNumber() != ERROR_SUCCESS)
                {
                    // There was an error sending our version number.
                }
 
                // Attempt to read the server's update information.
                if (s.RecvUpdateInformation() != ERROR_SUCCESS)
                {
                    // There was an error receiving update information.
                }
 
                // At this point we can parse the structure the serverUpdateInformation_t
                //Credentials::serverUpdateInformation_t* pServerUpdateInformation = &Credentials::serverUpdateInformation;
                // [B]Why does the above ^^^ and/or below screw up ServerSession's variable in the call to s.RecvUpdateInformation()?[/B]
                //Credentials::serverUpdateInformation_t serverUpdateInformation_Copy = Credentials::serverUpdateInformation;
 
                // Check to see if we need to update our client or not.
                if (CURRENT_VERSION < clientUpdate.version)
                ...

namespace Credentials
{
    serverUpdateInformation_t   serverUpdateInformation;

    int Login()
    {
        // Establish a new ServerSession().
        QString dns = QLatin1String("login.xpextend.com");
        QString ipString = QLatin1String("192.168.56.102");
        QStringList ipNotation(ipString.split(QLatin1String(".")));
        quint32 ipAddress;

        Credentials::serverUpdateInformation_t* pServerUpdateInformation = &Credentials::serverUpdateInformation;
        // [B]Leaving in the above line, as you'll see below, ANYWHERE in this compilation unit will affect the ability of the QByteArray to receive information from the server[/B]

        ipAddress = ((ipNotation.at(0).toLong()) << 24) | ((ipNotation.at(1).toLong()) << 16) |
                    ((ipNotation.at(2).toLong()) << 8) | ((ipNotation.at(3).toLong()) << 0);
        ServerSession s(ipString);

        if (s.Connect() == ERROR_SUCCESS)
        {
            // Successfully connected.  Client successfully sent the whole loginResponse_t structure to the server.
            // Now we must read the serverLoginResponse_t structure.
            int loginResponse = s.ParseLoginResponse();

            if (loginResponse == ERROR_SUCCESS)
            {
                // Goal:  Determine if we need to download any new patches or client.

                // Attempt to send the server our version number.
                if (s.SendVersionNumber() != ERROR_SUCCESS)
                {
                    // There was an error sending our version number.
                }

                // Attempt to read the server's update information.
                if (s.RecvUpdateInformation() != ERROR_SUCCESS)
                {
                    // There was an error receiving update information.
                }

                // At this point we can parse the structure the serverUpdateInformation_t
                //Credentials::serverUpdateInformation_t* pServerUpdateInformation = &Credentials::serverUpdateInformation;
                // [B]Why does the above ^^^ and/or below screw up ServerSession's variable in the call to s.RecvUpdateInformation()?[/B]
                //Credentials::serverUpdateInformation_t serverUpdateInformation_Copy = Credentials::serverUpdateInformation;

                // Check to see if we need to update our client or not.
                if (CURRENT_VERSION < clientUpdate.version)
                ...

To copy to clipboard, switch view to plain text mode 

ServerSession.cpp

...
int ServerSession::RecvUpdateInformation()
{
    // Check to make sure the socket is currently connected.
    if (this->tcpSocket->ConnectedState == QAbstractSocket::ConnectedState)
    {
        // Populate magic values for PATCH_UPDATE_t and CLIENT_UPDATE_t.
        snprintf(Credentials::serverUpdateInformation.patchUpdate.magic, strlen(PATCH_UPDATE_MAGIC), "%s", PATCH_UPDATE_MAGIC);
        snprintf(Credentials::serverUpdateInformation.clientUpdate.magic, strlen(CLIENT_UPDATE_MAGIC), "%s", CLIENT_UPDATE_MAGIC);
 
        QByteArray  inputArray;
        char*       contents;
        int         indexContents = 0;
 
        while (!inputArray.contains(END_UPDATE_MAGIC))
        {
            this->tcpSocket->waitForReadyRead();
            inputArray += this->tcpSocket->readAll();
        }
 
        contents = inputArray.data();       /* Pointer to data including NULL byte */
        // "contents" is free()'d when QByteArray goes out of scope.
 
        // Populate Credentials::serverUpdateInformation.
        Credentials::serverUpdateInformation_t* pServerUpdateInformation = &Credentials::serverUpdateInformation;
        memcpy(pServerUpdateInformation, contents, sizeof(Credentials::serverUpdateInformation.patchUpdate) -
               sizeof(Credentials::serverUpdateInformation.patchUpdate.lists));
        indexContents += sizeof(Credentials::serverUpdateInformation.patchUpdate) -
                sizeof(Credentials::serverUpdateInformation.patchUpdate.lists);
        ...

...
int ServerSession::RecvUpdateInformation()
{
    // Check to make sure the socket is currently connected.
    if (this->tcpSocket->ConnectedState == QAbstractSocket::ConnectedState)
    {
        // Populate magic values for PATCH_UPDATE_t and CLIENT_UPDATE_t.
        snprintf(Credentials::serverUpdateInformation.patchUpdate.magic, strlen(PATCH_UPDATE_MAGIC), "%s", PATCH_UPDATE_MAGIC);
        snprintf(Credentials::serverUpdateInformation.clientUpdate.magic, strlen(CLIENT_UPDATE_MAGIC), "%s", CLIENT_UPDATE_MAGIC);

        QByteArray  inputArray;
        char*       contents;
        int         indexContents = 0;

        while (!inputArray.contains(END_UPDATE_MAGIC))
        {
            this->tcpSocket->waitForReadyRead();
            inputArray += this->tcpSocket->readAll();
        }

        contents = inputArray.data();       /* Pointer to data including NULL byte */
        // "contents" is free()'d when QByteArray goes out of scope.

        // Populate Credentials::serverUpdateInformation.
        Credentials::serverUpdateInformation_t* pServerUpdateInformation = &Credentials::serverUpdateInformation;
        memcpy(pServerUpdateInformation, contents, sizeof(Credentials::serverUpdateInformation.patchUpdate) -
               sizeof(Credentials::serverUpdateInformation.patchUpdate.lists));
        indexContents += sizeof(Credentials::serverUpdateInformation.patchUpdate) -
                sizeof(Credentials::serverUpdateInformation.patchUpdate.lists);
        ...

To copy to clipboard, switch view to plain text mode 

Basically, if I create or try to reference ANY variable of type serverUpdateInformation_t within Credentials::Login() it will effect the (QByteArray) inputArray receiving data from the server. Depending on where in Credentials::Login() I declare a variable of type serverUpdateInformation_t I have seen it lead to no socket data being read or socket data being interpreted incorrectly by the QByteArray data type, which expects the size of the QByteArray as the first qint16 (WORD) of data sent.

Why would declaring, for example, a completely-unrelated variable of type serverUpdateInformation_t affect what is going on within a different compilation unit, especially AFTER the function has been called before a declaration is made?!!!

[ChrisW67]

It doesn't. The only ways this can occur are outlined in my first paragraph or the result of programmer error (e.g. overrunning a buffer accessed through a pointer). Do a complete rebuild and take the allocation of storage space out of the header.

If your problem persists you need to provide a small, complete, compilable example that demonstrates the problem.