system-independent C++ data types

[magland]

Anyone know a good way in C++ of defining numeric variables in system-independent manner such as:
32-bit integer
64-bit real
16-bit unsigned integer,
etc.

... purpose is reading and writing binary data to file so that, for example, 32-bit Windows user will always be able to read the file I wrote with 64-bit Linux, and vice versa.

[wysota]

Some libraries (like Qt) define their own platform independent types. For instance qint32, qint64, etc.

[magland]

Thanks for the info Wysota... Qt is amazing has so many things, yet always so simple...

Any ideas for 32- and 64- bit floating point? I noticed there is no qreal32 or qfloat32 for example in Qt.

[wysota]

First question would be - do you really need to know if a double is 64bit or 128bit long? I sincerily doubt you'll ever need a 128bit precision

[magland]

First question would be - do you really need to know if a double is 64bit or 128bit long? I sincerily doubt you'll ever need a 128bit precision

I see your point, and agree in most situations. However the pitfall is reading and writing to file.

For example, suppose I have the following code:

void write_raw_data(FILE *outf,double *data, long N) {
  fwrite(data, sizeof(double),N,outf);
}
 
void read_raw_data(FILE *inf,double *data, long N) {
  fread(data, sizeof(double),N,inf);
}

void write_raw_data(FILE *outf,double *data, long N) {
  fwrite(data, sizeof(double),N,outf);
}

void read_raw_data(FILE *inf,double *data, long N) {
  fread(data, sizeof(double),N,inf);
}

To copy to clipboard, switch view to plain text mode 

Then, you may not be able to read the file I wrote, and vice versa.

[wysota]

In that case the type size won't help you. If I have a big endian machine and you have a little endian machine (for instance a PPC Mac and a x86 PC) you'll get completely different values from the same binary representation. When you want to store a value in a file, you should always store it the same way, so that if you read it, you can be sure it is read properly - for example using a big endian 32 bit long U2 encoded representation of an integer. Then it's only a matter of reading the value the exact same way and interpretting it correctly.

[magland]

In that case the type size won't help you. If I have a big endian machine and you have a little endian machine (for instance a PPC Mac and a x86 PC) you'll get completely different values from the same binary representation. When you want to store a value in a file, you should always store it the same way, so that if you read it, you can be sure it is read properly - for example using a big endian 32 bit long U2 encoded representation of an integer. Then it's only a matter of reading the value the exact same way and interpretting it correctly.

Okay, so lets say I have an integer ( int x=5; ). What C++ commands would I need to perform in order to write it to a file using say "big endian 32-bit long U2 representation". Would I have to create the individual bytes one by one?

[wysota]

use the htonl() function, it converts a "host" integer to "network" integer.

[fullmetalcoder]

use the htonl() function, it converts a "host" integer to "network" integer.

What the point here??? Wasn't the question about "file" writing? In this case QDataStream and QTextStream are basically everything one needs.

[magland]

What the point here??? Wasn't the question about "file" writing? In this case QDataStream and QTextStream are basically everything one needs.

Yes, thanks. I think that's right... I will use QDataStream for writing raw data. It has everything I want including "setByteOrder".

Now my only remaining question is does the following code always write a 64-bit quantity, even on systems where double is 128 byte?

QDataStream & operator<< ( double f )

QDataStream & operator<< ( double f )

To copy to clipboard, switch view to plain text mode 

According to the Qt documentation, this should always be 64-bit output, but I'm skeptical... does anyone have an idea about it?

[fullmetalcoder]

Now my only remaining question is does the following code always write a 64-bit quantity, even on systems where double is 128 byte?

QDataStream & operator<< ( double f )

QDataStream & operator<< ( double f )

To copy to clipboard, switch view to plain text mode 

According to the Qt documentation, this should always be 64-bit output, but I'm skeptical... does anyone have an idea about it?

1. Do you know many systems where double occupy 128 bytes?
2. If the docs clearly state that a 64 byte float is written then why being skeptical... Qt docs are by far the best I've ever seen in a C++ library so lets trust or, if you're still unsure, ask the Trolls (or check out the sources)...
3. After verification it seems that there is a *STANDARD* for double type : "IEEE 754", according to the docs. What about googling around for some specs? http://en.wikipedia.org/wiki/IEEE_754

[wysota]

What the point here??? Wasn't the question about "file" writing? In this case QDataStream and QTextStream are basically everything one needs.

Provided that one uses Qt. The question is in the "General Programming" forum. If one doesn't use Qt, htonl, ntohl, htons and ntohs are nice macros to convert between host and network (big endian) ordered integers.

Do you know many systems where double occupy 128 bytes?

Probably every 64b system has 128b long doubles. They are not called "doubles" for nothing.

[fullmetalcoder]

Provided that one uses Qt. The question is in the "General Programming" forum. If one doesn't use Qt, htonl, ntohl, htons and ntohs are nice macros to convert between host and network (big endian) ordered integers.

True enough but it seems Qt fits everywhere...

Probably every 64b system has 128b long doubles. They are not called "doubles" for nothing.

I don't think so... I mean, not every 64b system has 128b doubles... They aren't indeed called "doubles" for nothing... Yet there also exist "quads" (though not widely used right now because brought by a recent revision of the aforementioned IEEE standard : http://en.wikipedia.org/wiki/IEEE_754r). Ain't the world so nice?

[jacek]

Probably every 64b system has 128b long doubles. They are not called "doubles" for nothing.

(128b long) doubles or 128b (long doubles)?

#include <iostream>
 
int main()
{
   std::cout << "int = " << sizeof( int ) << std::endl;
   std::cout << "int * = " << sizeof( int * ) << std::endl;
   std::cout << "float = " << sizeof( float ) << std::endl;
   std::cout << "double = " << sizeof( double ) << std::endl;
   std::cout << "long double = " << sizeof( long double ) << std::endl;
   return 0;
}

#include <iostream>

int main()
{
   std::cout << "int = " << sizeof( int ) << std::endl;
   std::cout << "int * = " << sizeof( int * ) << std::endl;
   std::cout << "float = " << sizeof( float ) << std::endl;
   std::cout << "double = " << sizeof( double ) << std::endl;
   std::cout << "long double = " << sizeof( long double ) << std::endl;
   return 0;
}

To copy to clipboard, switch view to plain text mode 

Result:

$ ./a.out
int = 4
int * = 8
float = 4
double = 8
long double = 16

64 bits in 64 bit systems have nothing to do with floating point values, they don't have much to do with integers either --- they're needed to enlarge the address space.

[wysota]

64 bits in 64 bit systems have nothing to do with floating point values, they don't have much to do with integers either --- they're needed to enlarge the address space.

I don't agree. Not only the address bus is enlarged to 64 bits, but also the data bus, so it's not only about increasing the address space.

[jacek]

Not only the address bus is enlarged to 64 bits, but also the data bus, so it's not only about increasing the address space.

I was referring to the programming model, nevertheless you can widen the data bus without changing the address length (but it depends whether you treat the cache and MMU as parts of a CPU or as external components).