How can I read lines with different number of data entries from text file into array in Matlab?

Question

I have a text file containing message and timestamps:

 {193220,message,ucontroller9,controller1,lowalarm,success},
 {193220,message,controller1,changer,{change,down},success},
 {193220,controlaction,tapchange_down,{oldsetting,0.975},{newsetting,0.95}},
 {193220,modelupdate_start,changeup,changer,193220},
 {14430,modelupdate_complete},
 {278480,message,ucontroller6,controller1,highalarm,success},
 {278480,message,controller1,changer,{change,up},success},
 ...

I would like to read these lines into an array and re-order the array based on the first column. My approach so far has been as follows:

fid = fopen('messagestore.txt')    % Open file
n=1    
while 1
string = fgetl(fid);               % get line
if ~ischar(string), break, end     % break if end of file
string = strrep(string,'[','');    %
string = strrep(string,']','');    %
string = strrep(string,'{','');    % strip out unwanted characters
string = strrep(string,'}','');    %
string = strrep(string,',',' ');   %
string = string(2:end);            % remove space at start

MessageArray(:,n) = textscan(string,'%f%s%s%s%s%s');  % format and save into array
n=n+1;
end
fclose(fid)

This partially works but I am having a problem with discriminating between different message types (i.e. number of fields). I'm guessing that I could use the matlab sort function if I could just get the data into an array properly.

Is there a way to do this in matlab?

If you only need to order them, you don't actually need to parse the entire string and can just stop after the first field. — Dennis Jaheruddin
– Dennis Jaheruddin, Commented Nov 15, 2012 at 17:15

Sam · Accepted Answer · 2012-11-15 17:46:06Z

Dennis is right in saying that you probably should do some preprocessing, since you have multiple sets of braces and such.

What you may be looking for is some way of basically sorting the lines of the file. If you're not worried about capturing the individual string fields, but just in re-ordering lines, you can read and sort according to the first field, which you know to be numbers. This is pretty hackish, but maybe you're looking for something like (this is essentially what Dennis suggests in the comments above):

clear all, clc
fid = fopen('messagestore.txt');
n=1;
nums = [];
while 1
    string = fgetl(fid);               % get line
    if ~ischar(string), break, end     % break if end of file

    for k = 1:length(string)
        if string(k) == ','
            break
        end
    end
    nums = [nums str2num(string(2:k-1))];
    MessageArray{n} = string;
    n=n+1;
end
[nums,I] = sort(nums);
MessageArray = MessageArray(I);
fclose(fid);

% Print some output
for k = 1:length(nums)
    disp(MessageArray{k})
end

which produces

{14430,modelupdate_complete},
{193220,message,ucontroller9,controller1,lowalarm,success},
{193220,message,controller1,changer,{change,down},success},
{193220,controlaction,tapchange_down,{oldsetting,0.975},{newsetting,0.95}},
{193220,modelupdate_start,changeup,changer,193220},
{278480,message,ucontroller6,controller1,highalarm,success},
{278480,message,controller1,changer,{change,up},success}

Now, I am not entirely sure how your timestamps work, here I literally sort the numerical value. But you should be able to get the basic idea from the code -- read the numbers from the first field into a vector and use that vector to sort the lines themselves. Whatever type of sorting your timestamps require can be performed by changing [nums,I] = sort(nums)

Dennis Jaheruddin · Accepted Answer · 2012-11-15 17:12:18Z

1

Here are some steps that should work:

0: Find an upper bounds for the number of fields in each line

1: Make sure you have nice rectangular data. Look at the number of commas present in each line minus the number of { brackets. (don't deduct the } or you will be counting doubles). This gives you the number of fields you have. Then add commas untill the maximum is achieved for each line.

2: Change the { and } into a standard text indicator like "

3: Import the result using xlsread() or similar.

answered Nov 15, 2012 at 17:12

Dennis Jaheruddin

21.6k8 gold badges73 silver badges133 bronze badges

1 Comment

Dennis Jaheruddin Over a year ago

if your dataset contains nasty constructions such as { that are not seperators, or if " occurs in the file you will need to do some smart pre-processing by substituting them

Collectives™ on Stack Overflow

How can I read lines with different number of data entries from text file into array in Matlab?

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related