Read 12-bits values from file

[franco.amato]

Good morning,
I have to read values from an input file where each number represents a 12-bits value.
How can I do this in C++?
I created a

struct value_struct
{
  unsigned value: 12;
};

struct value_struct
{
  unsigned value: 12;
};

To copy to clipboard, switch view to plain text mode 

Then I used ifstream to read the value from the file

value_struct data;
ifstream input;
input.open("in.txt");
if (!input)
{
  //show error
}
 input >> data.value;  //gives a compiler error

value_struct data;
ifstream input;
input.open("in.txt");
if (!input)
{
  //show error
}
 input >> data.value;  //gives a compiler error

To copy to clipboard, switch view to plain text mode 

[d_stranz]

The error is because ifstream has no operator>>() that can handle a 12-bit value.

Are these 12-bit values packed in the file? That is, is the file 12bits-12bits-12bits, or are the 12 bits the first 12 of a 16-bit "frame"? And is the value little or big endian?

If the 12-bit values are packed, then you will probably have to read them one unsigned char at a time, then construct an integer from that.

union value_struct
{
     unsigned char charVal[ 4 ];
     uint_16  intVal;
};
 
value_struct data;
data.intVal = 0;  // Ensures all bytes contain zero
input >> data.charVal[0] >> data.charVal[1] >> data.charVal[2];
 
// use data.intVal ...

union value_struct
{
     unsigned char charVal[ 4 ];
     uint_16  intVal;
};

value_struct data;
data.intVal = 0;  // Ensures all bytes contain zero
input >> data.charVal[0] >> data.charVal[1] >> data.charVal[2];

// use data.intVal ...

To copy to clipboard, switch view to plain text mode 

If there are endian issues, you may need to swap the order in which you read in the bytes so that when you access the integer value, it is a correct number.

Don't worry about the efficiency in reading a byte at a time. The OS will probably read a large chunk if not the entire file into an in-memory buffer and access it from there, not read it a byte at a time from disk.

[franco.amato]

Hi,
the file contains 12bits 12bits 12bits ... little endian.
I didn't get your example, why do you create a vector of 4 bytes? And how to use data.intVal if it contains only 0?

[d_stranz]

why do you create a vector of 4 bytes?

I see I made a mistake - somehow I confused myself into thinking a byte was 4 bits. Sorry. What I proposed will not work at all for your problem.

You need to read the file in 24-bit chunks - three 8-bit bytes at a time - to match two 12-bit "words". So read your data into an unsigned char [ 3 ] array. You then chop this into two 16-bit integers by taking all of the first byte and the first half of the second byte and putting it into one integer, then the second half of the second byte and all of the third byte and putting it into the second integer.

Use a union like this:

unsigned char inData[ 3 ];
 
union SplitData
{
    unsigned char bytes[ 2 ];
    uint_16 intVal;
}
 
// Need to check for end of file, just in case there is an odd number of 12-bit values.  
// In that case, don't read the third byte.
input >> inData[ 0 ] >> inData[ 1 ] >> inData[ 3 ];
 
uint_16 value1;
uint_16 value2;
 
SplitData splitData;
 
 
splitData.bytes[ 0 ] = inData[ 0 ];
splitData.bytes[ 1 ] = inData[ 1 ] & 0xF0;  // take first four bits
 
// Might have to do this bit shifting instead of just straight assignment
value1 = splitData.intVal >> 4;
 
splitData.bytes[ 0 ] = inData[ 1 ] & 0x0F; // take second four bits
splitData.bytes[ 1 ] = inData[ 2 ];
 
// Here it is OK because the first 4 bits have already been masked off to zero
value2 = splitData.intVal;

unsigned char inData[ 3 ];

union SplitData
{
    unsigned char bytes[ 2 ];
    uint_16 intVal;
}

// Need to check for end of file, just in case there is an odd number of 12-bit values.  
// In that case, don't read the third byte.
input >> inData[ 0 ] >> inData[ 1 ] >> inData[ 3 ];

uint_16 value1;
uint_16 value2;

SplitData splitData;


splitData.bytes[ 0 ] = inData[ 0 ];
splitData.bytes[ 1 ] = inData[ 1 ] & 0xF0;  // take first four bits

// Might have to do this bit shifting instead of just straight assignment
value1 = splitData.intVal >> 4;

splitData.bytes[ 0 ] = inData[ 1 ] & 0x0F; // take second four bits
splitData.bytes[ 1 ] = inData[ 2 ];

// Here it is OK because the first 4 bits have already been masked off to zero
value2 = splitData.intVal;

To copy to clipboard, switch view to plain text mode 

If there is an odd number of 12-bit values in the file, then ignore value2 for the last read.

You might have to swap bytes and masking around because of endianess, but the basic idea is the same.

[franco.amato]

I am still not understanding.
Why did you write:

input >> inData[ 0 ] >> inData[ 1 ] >> inData[ 3 ];

input >> inData[ 0 ] >> inData[ 1 ] >> inData[ 3 ];

To copy to clipboard, switch view to plain text mode 

and not

input >> inData[ 0 ] >> inData[ 1 ] >> inData[ 2 ];

input >> inData[ 0 ] >> inData[ 1 ] >> inData[ 2 ];

To copy to clipboard, switch view to plain text mode 

And why are you reading 3 elements at time? In my file I have rows of 64 items. Each row is separated by new line character

[d_stranz]

Why did you write:

Sorry again, it is a typo. Should be inData[ 2 ] as you said.

And why are you reading 3 elements at time? In my file I have rows of 64 items. Each row is separated by new line character

You said you had 12 bit integers in your file, so I assumed that the file is binary. Now you are seeming to say that the file is actually a text file, since it has newlines in it. If it is a text file, then how can it contain 12-bit integers? And what is an "item" - a 12 bit integer or something else?

What EXACTLY is the format of these files? Text, binary, what? If it is binary, then is it 12-bit integers, packed 2 integers to every 3 bytes (which it would be if the format is 12 bits - 12 bits - 12 bits, like you said - every three 8-bit bytes holds two 12-bit numbers), or is it something else? The code I wrote takes every 3 bytes and unpacks them into two integers, 1 1/2 bytes (12 bits) into one, 1 1/2 bytes into the next.

If it is a text file, then copy and paste a few lines into your reply so we can see what it looks like.

[franco.amato]

Hi,
the file is a text file where each value in each row is a number representing a 12 bits value.
Below 5 rows as example:

2 0 0 1 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 639 0 239 0 3614 1 3614 1 0 0 0 0 160 1 160 0 0 16 384 1190 3367 2051 67 268 1311 660 1636 2048 0 2048 2048 134 59 37 86 196 101 54 16 248 0 0 246 53 63 32 130 248 0 0 246 53 63 32 130 40 119 11 19 3861 0 3860 3861 3861 3862 3862 3861 3860 3861 3860 0 0 0 0 0 0 0 0 0 3862 0 3697 3610 3523 3435 3347 3259 3172 3083 2995 0 0 0 0 0 0 0 0 0 1520 1 0 11 0 0 0 0 1636 660 0 8 4095 7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
2 0 0 2 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 639 0 239 0 3614 1 3614 1 0 0 0 0 160 1 160 0 0 16 384 1190 3369 2051 67 268 1310 660 1636 0 0 2048 0 134 59 37 86 196 101 54 16 248 0 0 246 53 63 32 130 248 0 0 246 53 63 32 130 40 119 11 19 3861 0 3863 3861 3862 3862 3861 3862 3861 3862 3862 0 0 0 0 0 0 0 0 0 3860 0 3700 3611 3522 3436 3348 3259 3172 3085 2997 0 0 0 0 0 0 0 0 0 1520 1 0 11 0 0 0 0 1636 660 0 8 4095 7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
2 0 0 3 27 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 639 0 239 0 3614 1 3614 1 0 0 0 0 160 1 160 0 0 16 384 1190 3371 2051 67 268 1311 661 1636 0 2048 0 2048 134 59 37 86 196 101 54 16 248 0 0 246 53 63 32 130 248 0 0 246 53 63 32 130 40 119 11 19 3861 0 3862 3860 3862 3860 3862 3860 3860 3861 3860 0 0 0 0 0 0 0 0 0 3861 0 3697 3611 3523 3435 3347 3258 3171 3084 2995 0 0 0 0 0 0 0 0 0 1504 1 0 11 0 0 0 0 1636 661 0 8 4095 7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Each value in each row is separated by space and each row is separated by new-line character

[Lesiok]

So it is not a file with 12-bit values, but a text file divided into lines and each line contains integers separated by a space.

[d_stranz]

So in other words, your file is ASCII, and each line contains 64 ASCII numbers whose values range from 0 - 4095. If you had provided that information at the beginning, it could have saved a lot of time and confusion.

QFile inFile( "data.txt" );
if ( !inFile.open( QIODevice::ReadOnly | QIODevice::Text ) )
{
    // error, do something with it.
}
 
// Assuming no error:
QTextStream input( &inFile );
while ( !input.atEnd() )
{
    // Read each line
    QString line = input.readLine();
 
    // Split it into individual items based on the 'space' delimiter
    QStringList numbers = line.split( ' ' );
 
    // For each item, convert it to an integer
    for ( QString numberStr : numbers )
    {
        // Here's your "12 bits" number, do something with it
        int number = numberStr.toInt();
    }
}

QFile inFile( "data.txt" );
if ( !inFile.open( QIODevice::ReadOnly | QIODevice::Text ) )
{
    // error, do something with it.
}

// Assuming no error:
QTextStream input( &inFile );
while ( !input.atEnd() )
{
    // Read each line
    QString line = input.readLine();

    // Split it into individual items based on the 'space' delimiter
    QStringList numbers = line.split( ' ' );
    
    // For each item, convert it to an integer
    for ( QString numberStr : numbers )
    {
        // Here's your "12 bits" number, do something with it
        int number = numberStr.toInt();
    }
}

To copy to clipboard, switch view to plain text mode 

[franco.amato]

I am sorry for the confusion.
Yes, you are right, and I have to represent each read number into a 12 bits value, this was my problem from the beginning

[d_stranz]

I have to represent each read number into a 12 bits value

What does this mean? Do you have to output the number (in binary) to some device that uses a 12-bit wide channel?

[franco.amato]

Yes it's exactly that