Finding strings in a large text file in Python

Question

The following is my code:

with open("WinUpdates.txt") as f:
    data=[]
    for elem in f:
        data.append(elem)

with open("checked.txt", "w") as f:
    check=True
    for item in data:
        if "KB2982791" in item:
            f.write("KB2982791\n")
            check=False
        if "KB2970228" in item:
            f.write("KB2970228\n")
            check=False
        if "KB2918614" in item:
            f.write("KB2918614\n")
            check=False
        if "KB2993651" in item:
            f.write("KB2993651\n")
            check=False
        if "KB2975719" in item:
            f.write("KB2975719\n")
            check=False
        if "KB2975331" in item:
            f.write("KB2975331\n")
            check=False
        if "KB2506212" in item:
            f.write("KB2506212\n")
            check=False
        if "KB3004394" in item:
            f.write("KB3004394\n")
            check=False
        if "KB3114409" in item:
            f.write("KB3114409\n")
            check=False
        if "KB3114570" in item:
            f.write("KB3114570\n")
            check=False

    if check:
        f.write("No faulty Windows Updates found!")

The "WinUpdates.txt" file contains a lot of lines like these:

http://support.microsoft.com/?kbid=2980245 RECHTS Update
KB2980245 NT-AUTORITÄT\SYSTEM 8/18/2014
http://support.microsoft.com/?kbid=2981580 RECHTS Update
KB2981580 NT-AUTORITÄT\SYSTEM 8/18/2014
http://support.microsoft.com/?kbid=2982378 RECHTS Security Update KB2982378 NT-AUTORITÄT\SYSTEM 9/12/2014
http://support.microsoft.com/?kbid=2984972 RECHTS Security Update KB2984972 NT-AUTORITÄT\SYSTEM 10/17/2014
http://support.microsoft.com/?kbid=2984976 RECHTS Security Update KB2984976 NT-AUTORITÄT\SYSTEM 10/17/2014
http://support.microsoft.com/?kbid=2984981 RECHTS Security Update KB2984981 NT-AUTORITÄT\SYSTEM 10/16/2014
http://support.microsoft.com/?kbid=2985461 RECHTS Update
KB2985461 NT-AUTORITÄT\SYSTEM 9/12/2014
http://support.microsoft.com/?kbid=2987107 RECHTS Security Update KB2987107 NT-AUTORITÄT\SYSTEM 10/17/2014
http://support.microsoft.com/?kbid=2990214 RECHTS Update
KB2990214 NT-AUTORITÄT\SYSTEM 4/16/2015
http://support.microsoft.com/?kbid=2991963 RECHTS Security Update KB2991963 NT-AUTORITÄT\SYSTEM 11/14/2014
http://support.microsoft.com/?kbid=2992611 RECHTS Security Update KB2992611 NT-AUTORITÄT\SYSTEM 11/14/2014
http://support.microsoft.com/?kbid=2993651 RECHTS Update
KB2993651 NT-AUTORITÄT\SYSTEM 8/29/2014
http://support.microsoft.com/?kbid=2993958 RECHTS Security Update KB2993958 NT-AUTORITÄT\SYSTEM 11/14/2014

But when I execute my code, it says that it has not found any of those updates? Even though I know that it should find 4. I wrote the "data" list into a new text file, but there it seems everything alright?

Why do you think my code does not work?

Same as @Math; if I manually add additional "KBxxxxxxx" to the WinUpdates.txt file, I get multiple results in checked.txt. What output are you getting, and what output do you want? — TheSchwa
– TheSchwa, Commented Feb 18, 2016 at 12:47
The output is "No faulty Windows Updates found!" in the "checked.txt" file, even though it should write four of the checked updates. I wonder if it is the file, the exported WinUpdates.txt that is causing the issues. — mrg
– mrg, Commented Feb 18, 2016 at 12:58
Same as @PM2Ring when I copy and paste the code you supplied in the question I get "KB2993651" in checked.txt. — TheSchwa
– TheSchwa, Commented Feb 18, 2016 at 13:00
Hmm. With new data I get nothing. If I print the file as it's read in I get [Decode error - output not utf-8]. So, it looks like a unicode problem. I'm running Python2. @PM2Ring: I'm guessing you're on Python3? — SiHa
– SiHa, Commented Feb 18, 2016 at 13:13
@SiHa: No, I'm on Python 2.6.6, but I'm using Linux & I pasted the data into a file using the cat command in a shell which uses UTF-8 encoding. — PM 2Ring
– PM 2Ring, Commented Feb 18, 2016 at 13:21

PM 2Ring · Accepted Answer · 2016-02-18 13:52:44Z

FWIW, your code can be written in a more compact way that doesn't require a zillion if statements. Also, since the (new) data file is only 63342 bytes you can read the whole thing into a single string, rather than into a list of strings.

kb_ids = (
    "KB2982791",
    "KB2970228",
    "KB2918614",
    "KB2993651",
    "KB2975719",
    "KB2975331",
    "KB2506212",
    "KB3004394",
    "KB3114409",
    "KB3114570",
)

with open("WinUpdates.txt") as f:
    data = f.read()

check = True
with open("checked.txt", "w") as f:
    for kb in kb_ids:
        if kb in data:
            f.write(kb + "\n")
            check = False

    if check:
        fout.write("No faulty Windows Updates found!\n")

Contents of checked.txt, using the linked data:

KB2970228
KB2918614
KB2993651
KB2506212
KB3004394

Note that this code prints the found kbids in the order that they're defined in kb_ids, rather than the order they occur in "WinUpdates.txt".

Searching through the whole file as a string for each kbid is probably not a good idea if the file is large, eg, more than a megabyte or so; you might want to run some timing tests (using timeit) to see which strategy works best on your data.

If you want to read a file into a list there's no need to use a for loop, you can just do this:

with open("WinUpdates.txt") as f:
    data = f.readlines()

Alternatively, you can process the file line by line without reading it into a list:

kb_ids = (
    "KB2982791",
    "KB2970228",
    "KB2918614",
    "KB2993651",
    "KB2975719",
    "KB2975331",
    "KB2506212",
    "KB3004394",
    "KB3114409",
    "KB3114570",
)

check = True
with open("WinUpdates.txt") as fin:
    with open("checked.txt", "w") as fout:
        for data in fin:
            for kb in kb_ids:
                if kb in data:
                    fout.write(kb + "\n")
                    check = False

        if check:
            fout.write("No faulty Windows Updates found!\n")

On more modern versions of Python the two with statements can be combined into a single line.

Mark Skelton · Accepted Answer · 2016-02-18 12:44:23Z

I added and fixed what you were missing check the two comments to see what I mean. This worked for me so it should work for you. Have a great day!

with open("WinUpdates.txt", "r") as f:  #you forgot to put the "r" option to read the file
    data = f.read()  #no reason to put the data into a list a string will do fine

with open("checked.txt", "w") as f:
    check=True
    if "KB2982791" in data:
        f.write("KB2982791\n")
        check=False
    if "KB2970228" in data:
        f.write("KB2970228\n")
        check=False
    if "KB2918614" in data:
        f.write("KB2918614\n")
        check=False
    if "KB2993651" in data:
        f.write("KB2993651\n")
        check=False
    if "KB2975719" in data:
        f.write("KB2975719\n")
        check=False
    if "KB2975331" in data:
        f.write("KB2975331\n")
        check=False
    if "KB2506212" in data:
        f.write("KB2506212\n")
        check=False
    if "KB3004394" in data:
        f.write("KB3004394\n")
        check=False
    if "KB3114409" in data:
        f.write("KB3114409\n")
        check=False
    if "KB3114570" in data:
        f.write("KB3114570\n")
        check=False

    if check:
        f.write("No faulty Windows Updates found!")

Collectives™ on Stack Overflow

Finding strings in a large text file in Python

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related