Create an array of values from different text files in C

Question

I'm working in C on 64-bit Ubuntu 14.04.

I have a number of .txt files, each containing lines of floating point values (1 value per line). The lines represent parts of a complex sample, and they're stored as real(a1) \n imag(a1) \n real(a2) \n imag(a2), if that makes sense.

In a specific scenario there are 4 text files each containing 32768 samples (thus 65536 values), but I need to make the final version dynamic to accommodate up to 32 files (the maximum samples per file would not exceed 32768 though). I'll only be reading the first 19800 samples (depending on other things) though, since the entire signal is contained in those 39600 points (19800 samples).

A common abstraction is to represent the files / samples as a matrix, where columns represent return signals and rows represent the value of each signal at a sampling instant, up until the maximum duration.

What I'm trying to do is take the first sample from each return signal and move it into an array of double-precision floating point values to do some work on, move on to the second sample for each signal (which will overwrite the previous array) and do some work on them, and so forth, until the last row of samples have been processed.

Is there a way in which I can dynamically open files for each signal (depending on the number of pulses I'm using in that particular instance), read the first sample from each file into a buffer and ship that off to be processed. On the next iteration, the file pointers will all be aligned to the second sample, it would then move those into an array and ship it off again, until the desired amount of samples (19800 in our hypothetical case) has been reached.

I can read samples just fine from the files using fscanf:

rx_length = 19800;
int x;
float buf;

double *range_samples = calloc(num_pulses, 2 * sizeof(range_samples));

for (i=0; i < 2 * rx_length; i++){
    x = fscanf(pulse_file, "%f", &buf);
    *(range_samples) = buf;
}

All that needs to happen (in my mind) is that I need to cycle both sample# and pulse# (in that order), so when finished with one pulse it would move on to the next set of samples for the next pulse, and so forth. What I don't know how to do is to somehow declare file pointers for all return signal files, when the number of them can vary inbetween calls (e.g. do the whole thing for 4 pulses, and on the next call it can be 16 or 64).

If there are any ideas / comments / suggestions I would love to hear them.

Thanks.

Is the list of file known at the beginning of program or is it real time processing with new files being created while the program runs ? — Serge Ballesta
– Serge Ballesta, Commented Oct 2, 2014 at 14:07
Yes, the files are rx_1_out.txt where 1 represents the pulse number. — user2742907
– user2742907, Commented Oct 2, 2014 at 17:15

Carl · Accepted Answer · 2014-10-02 13:50:46Z

1

I would make the code you posted a function that takes an array of file names as an argument:

void doPulse( const char **file_names, const int size )
{
  FILE *file = 0;

  // declare your other variables

  for ( int i = 0; i < size; ++i )
  {
    file = fopen( file_names[i] );

    // make sure file is open
    // do the work on that file

    fclose( file );
    file = 0;
  }
}

answered Oct 2, 2014 at 13:50

Carl

9918 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Serge Ballesta · Accepted Answer · 2014-10-02 14:55:50Z

What you need is a generator. It would be reasonably easy in C++, but as you tagged C, I can imagine a function, taking a custom struct (the state of the object) as parameter. It could be something like (pseudo code) :

struct GtorState {
    char *files[];
    int filesIndex;
    FILE *currentFile;
};

void gtorInit(GtorState *state, char **files) {
    // loads the array of file into state, set index to 0, and open first file
}

int nextValue(GtorState *state, double *real, double *imag) {
    // read 2 values from currentFile and affect them to real and imag
    // if eof, close currentFile and open files[++currentIndex]
    // if real and imag were found returns 0, else 1 if eof on last file, 2 if error
}

Then you main program could contain :

GtorState state;
// initialize the list of files to process
gtorInit(&state, files);

double real, imag);
int cr;
while (0 == (cr = nextValue(&state, &real, &imag)) {
    // process (real, imag)
}
if (cr == 2) {
    // process (at least display) error
}

Alternatively, your main program could iterate the values of the different files and call a function with state analog of the above generator that processes the values, and at the end uses the state of the processing function to get the results.

user2742907 · Accepted Answer · 2014-10-03 09:28:37Z

Tried a slightly different approach and it's working really well.

In stead of reading from the different files each time I want to do something, I read the entire contents of each file into a 2D array range_phase_data[sample_number][pulse_number], and then access different parts of the array depending on which range bin I'm currently working on.

Here's an excerpt:

#define REAL(z,i) ((z)[2*(i)])
#define IMAG(z,i) ((z)[2*(i)+1])

for (i=0; i<rx_length; i++){
    printf("\t[%s] Range bin %i. Samples %i to %i.\n", __FUNCTION__, i, 2*i, 2*i+1);
    for (j=0; j<num_pulses; j++){
        REAL(fft_buf, j) = range_phase_data[2*i][j];
        IMAG(fft_buf, j) = range_phase_data[2*i+1][j];
    }
    printf("\t[%s] Range bin %i done, ready to FFT.\n", __FUNCTION__, i);

    // do stuff with the data
}

This alleviates the need to dynamically allocate file pointers and in stead just opens the files one at a time and writes the data to the corresponding column in the matrix.

Cheers.

Collectives™ on Stack Overflow

Create an array of values from different text files in C

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related