word frequency in python not working

Question

I am trying to count frequencies of words in a text file using python.

I am using the following code:

openfile=open("total data", "r")

linecount=0
for line in openfile:
    if line.strip():
        linecount+=1

count={}

while linecount>0:
    line=openfile.readline().split()
    for word in line:
        if word in count:
            count[word]+=1
        else:
            count[word]=1
    linecount-=1

print count

But i get an empty dictionary. "print count" gives {} as output

I also tried using:

from collections import defaultdict
.
.
count=defaultdict(int)
.
.
     if word in count:
          count[word]=count.get(word,0)+1

But i'm getting an empty dictionary again. I dont understand what am i doing wrong. Could someone please point out?

After you count all of the lines the input from the file is exhausted, you need to re-open it. — Hunter McMillen
– Hunter McMillen, Commented Jul 2, 2013 at 13:17

Ashwini Chaudhary · Accepted Answer · 2013-07-02 13:22:33Z

9

This loop for line in openfile: moves the file pointer at the end of the file. So, if you want to read the data again then either move the pointer(openfile.seek(0)) to the start of the file or re-open the file.

To get the word frequency better use Collections.Counter:

from collections import Counter
with open("total data", "r") as openfile:
   c = Counter()
   for line in openfile:
      words = line.split()
      c.update(words)

edited Jul 2, 2013 at 13:22

answered Jul 2, 2013 at 13:17

Ashwini Chaudhary

252k60 gold badges478 silver badges519 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

eduffy · Accepted Answer · 2013-07-02 13:18:54Z

1

Add openfile.seek(0) right after you initialize count. That'll put the read pointer to the beginning of the file

answered Jul 2, 2013 at 13:18

eduffy

40.5k14 gold badges99 silver badges94 bronze badges

Comments

Inbar Rose · Accepted Answer · 2013-07-02 13:21:27Z

1

This is a much more direct way of counting the word frequency in a file:

from collections import Counter

def count_words_in_file(file_path):
    with open(file_path) as f:
        return Counter(f.read().split())

Example:

>>> count_words_in_file('C:/Python27/README.txt').most_common(10)
[('the', 395), ('to', 202), ('and', 129), ('is', 120), ('you', 111), ('a', 107), ('of', 102), ('in', 90), ('for', 84), ('Python', 69)]

answered Jul 2, 2013 at 13:21

Inbar Rose

43.7k24 gold badges91 silver badges137 bronze badges

Collectives™ on Stack Overflow

word frequency in python not working

3 Answers 3

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related