Recursive Regex with Joined Results

Question

My input files are html files with no extension. Desired output is regex matched URLs from all files from the root_dir and results joined in single file. My regex works and I can output results from a single file.

import re
with open('/Users/files/filename') as f:
    for line in f:
        urls = re.findall (r"([\w%~\+-=]*\.mp3)", line);
        print (*urls)

I could use glob but unsure how to:

import glob
import re
root_dir = '/Users/files/'
for filename in glob.iglob(root_dir + '**/*.*', recursive=True):
        urls = re.findall (r"([\w%~\+-=]*\.mp3)", line);
        print (*urls)

Your first code sample works on opening a directory open('/Users/files/')?? Your second code sample misses an open(filename) statement. — Han-Kwang Nienhuys
– Han-Kwang Nienhuys, Commented Jun 12, 2020 at 22:09
@Han-KwangNienhuys no it does not open directory, i fixed the code to show that. — vanziro
– vanziro, Commented Jun 12, 2020 at 22:53
@Edward so you are saying there is no way to recursively use regex in python? — vanziro
– vanziro, Commented Jun 12, 2020 at 22:54

Ryszard Czech · Accepted Answer · 2020-06-15 15:09:04Z

1

Use

import re, glob                                 # Import the libraries

root_dir = r'/Users/files'                      # Set root directory
save_to_file = r'/Users/urls_extracted.txt'     # File path to save results to
all_files = glob.glob("{}/*".format(root_dir))  # Get a glob with filepaths

with open(save_to_file, 'w') as fw:             # Open stream to write to
  for filename in all_files:                    # Iterate over the files
    with open(filename, 'r') as fr:             # Open file to read from  
      for url in re.findall(r"[\w%~+\-=]*\.mp3", fr.read()): # Get all matches and iterate over them
        fw.write("{}\n".format(url))            # Write each URL to write stream

Note that the dash must be escaped in the regular expression if you meant a - character and not a range.

edited Jun 15, 2020 at 15:09

answered Jun 13, 2020 at 20:55

Ryszard Czech

18.7k4 gold badges27 silver badges39 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

vanziro Over a year ago

Thank you, this worked well. Is there any way you could include some comment describing the code above or is it too cumbersome?

Ryszard Czech Over a year ago

@x0rz0r Added comments

Collectives™ on Stack Overflow

Recursive Regex with Joined Results

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related