1

I want to make a tokenization sub-program which will work like this :

  • Read user input
  • Find tokens
  • Use a substring function to get each token
  • Store each token in a struct

The idea was simple ( i think ) but I came up with a difficult problem. When making the substring function, I realized that it was a memory leak hole. The only way i can think of making a substring function is this :

    char* sub = ( char* ) malloc ( ( some_length + 1 ) * sizeof( char ) );

    for ( i = start_index ; i < some_length ; i++ )
    {
         sub[ i - start_index ] = source_string[i];
    }

    sub[ some_length ] = '\0'

    return sub;

But the problem is that when using the substring function, i won't have the ability to free that memory afterwards.

    // Example usage

    TokenStruct* MyToken = CreateToken( substring( input , start , length ) );

Some may suggest that i should free memory inside the CreateToken function but this seems like a VERY bad idea to me because it will make CreateToken's code very dependent on the substring function. Also, the substring function might be used in many other functions.

I had an idea of keeping a table of pointers and free them before terminating the program but seems kind of sketchy...

What do you think guys? What is the best way to deal with this kind of problem?

Thanks in advance!

14
  • Let TokenStruct store the token by reference (char *). And define substring() as a source/creator, which's result needs to be deleted, like strdup() for example. Commented May 25, 2013 at 18:21
  • You may want to consider using lex as an indirect means of solving your problem. And #include <string.h> and use strncpy() instead of your for-loop. Commented May 25, 2013 at 18:24
  • 1
    You don't need to cast the return value of malloc in a C program. And sizeof(char) is 1. Commented May 25, 2013 at 18:25
  • 1
    @meaning-matters, no you're not right when talking about the language specification. I quoted it, even. Commented May 25, 2013 at 18:35
  • 1
    @meaning-matters, that site must be lying to you. My quotation is from C99, but it holds for older standards, and even for (most? all?) pre-standardized C. There are plenty of questions & answers on Stack Overflow if you care to read more. Commented May 25, 2013 at 18:43

1 Answer 1

3

This is a very common problem in C and other languages of the same type. There are basically three solutions to this:

  1. Either do the call separately and then free the memory afterwards.
  2. Keep the pointer in the object (MyToken in your case), and free the string when you free the object.
  3. Use an array, and pass it in (together with a maximum length) to the function.

Since the created pointer have to be "live" for the life of your object (MyToken) anyway, then I suggest method number two.

Sign up to request clarification or add additional context in comments.

6 Comments

I just commented above about your second suggestion that this will prevent me from initializing MyToken with a variable that i want to change afterwards. Not all TokenStructs will be initialized using substring. Do you think that i should give away that "freedom" i described above or do something else?
@GiwrgosTsopanoglou For those occasions where you don't initialize the structure with substring then you can use e.g. strdup, and you know that the pointer in the structure is always allocated on the heap.
@JoachimPilebord I fail to understand how strdup will help me. A quick search on google showed me that it is used to duplicate a string... can you please explain?
@GiwrgosTsopanoglou The Strdup allocates memory for the duplicate string on the heap, just like malloc. If the string is always allocated on the heap, then you know you can always free it then you free the object.
I have been programming for many hours and my mind is kind of blurred so i can't understand "where" in the program shall i use strdup?
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.