Delete a string phrase from a data frame column and replace it python

Question

So, I have two dataframes. the first dataframe is dataset conatians several columns, what i will use in this dataframe is the dataset['text_msg'], this columns contains text data.

The second Dataframe sentences_to_exclude contains the data which type is text type.

The column that i will use in this dataframe is sentences_to_exclude['sentences'].

What i need to do is to verify if there are sentences from sentences_to_exclude['sentences'] in the first dataframe and remove the whole sentence.
I have tried a function but it didn't work for me: Here is the function i've used ==>

  def remove_words(data):
    words_to_remove = sentences_to_exclude['sentences'].lower().split(" ")
    text_body = dataset['text_msg']
    for word in words_to_remove:
        text_body = text_body.replace(word,'' )
    return text_body

Here's an exemple of sentences_to_exclude['sentences']

pour un traitement optimal de votre demande, veuillez indiquer les informations ci-dessous

and for the fisrt data frame here's an example of the dataset['text_msg']:

pour un traitement optimal de votre incident, nous vous prions de renseigner les informations ci-dessous : - code transaction : - numero de facture / commande client : - criteres dexecution et message derreur (a attacher en pj) description detaillee de votre demande

Hope that my request is clear Thank you for help in advance

Example Data

sentences = ['code transaction', 'Pour un traitement efficace']
text = [ ' i should delete code transaction ', ' i am trying to delete Pour un traitement efficace only from this sentence ' ]

df1 = pd.DataFrame({'Sentences ': sentences })
df2 = pd.DataFrame({'Text': text})

Please provide example data and an example output also in dataframe format. — Erfan
– Erfan, Commented Feb 28, 2019 at 14:49
This is not clear at all, I suggest you look at other questions to understand how a good question looks like. — Erfan
– Erfan, Commented Feb 28, 2019 at 15:06

Erfan · Accepted Answer · 2019-02-28 17:04:47Z

Still don't understand your question correctly, I will try to help you, but please next time you have to include example data.

To answer your question I will give example dataset and explain how to remove words or sentences from other text:

# This is our example data
sentences = ['code transaction', 'Pour un traitement efficace']
text = [ ' i should delete code transaction ', ' i am trying to delete Pour un traitement efficace only from this sentence ' ]

df1 = pd.DataFrame({'Sentences': sentences})
df2 = pd.DataFrame({'Text': text})

# df1

    Sentences
0   code transaction
1   Pour un traitement efficace

# df2
    Text
0   i should delete code transaction
1   i am trying to delete Pour un traitement effi...

Next we want to harmonize our data so we wont have mismatches, so we convert to uppercase:

df1['Sentences'] = df1.Sentences.str.upper()
df2['Text'] = df2.Text.str.upper()


    Sentences
0   CODE TRANSACTION
1   POUR UN TRAITEMENT EFFICACE


    Text
0   I SHOULD DELETE CODE TRANSACTION
1   I AM TRYING TO DELETE POUR UN TRAITEMENT EFFI...

Now our data is in the right format, we can remove the text from one dataset to another

df2['Text_cleaned'] = df2.Text.str.replace('|'.join(df1.Words), '')


    Text                                                Text_cleaned
0   I SHOULD DELETE CODE TRANSACTION                    I SHOULD DELETE
1   I AM TRYING TO DELETE POUR UN TRAITEMENT EFFI...    I AM TRYING TO DELETE ONLY FROM THIS SENTENCE

What does '|'.join(df1.Sentences) do?
It returns a string delimited by |

'|'.join(df1.Words)

'CODE TRANSACTION|POUR UN TRAITEMENT EFFICACE'

Hope this helps you and answers your question.
You can now apply this logic to your own data.

Thank u for your response. i have tried what you did but it didn't work i will give an exemple of the data that i am using so it can be much clear for you.

Collectives™ on Stack Overflow

Delete a string phrase from a data frame column and replace it python

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related