Document distance in Python

Most of your code can be greatly reduced.

Doing string addition is often a bad idea in Python, especially if you are doing it in a loop. Also note that a + "" == a for any string a. I would replace your mix of global code and the word_processor function with this short function:

import string

WHITELIST = set(string.ascii_letters + string.digits + string.whitespace)

def get_words(file_name):
    with open(file_name) as f:
        for line in f:
            line = "".join(c for c in line.casefold() if c in WHITELIST)
            yield from line.split()

This filters each line of the file to only contain ASCII letters and whitespace and splits it by whitespace afterwards (the default behavior of str.split). The yield from effectively flattens the output into a stream of words (the whole function is a generator, so you can consume the output one at a time without keeping the whole list in memory). It also uses the with keyword to ensure your files are closed, even in the event of an exception interrupting the program. I used str.casefold instead of str.lower, as recommended by @AJNeufeld in the comments.

You should put the actual code into a function as well. I would call it similarity, since it does not actually measure distance, but similarity (from 0 to 100). For this you can use collections.Counter instead of using list.count repeatedly (each of which is \$\mathcal{O}(n)\$).

from collections import Counter
from math import sqrt

def similarity(words1, words2):
    words1, words2 = Counter(words1), Counter(words2)
    dot = sum(words1[k] * words2[k] for k in words1.keys() & words2.keys())
    norm1 = sqrt(sum(v**2 for v in words1.values()))
    norm2 = sqrt(sum(v**2 for v in words2.values()))
    return dot / (norm1 * norm2)

Here I used the fact that Counter can take any iterable (like the output of get_words) and counts how often each item occurs (it returns a kind of dictionary). Also, dictionary keys behave like sets, so we can do a set intersection so we don't have to check for every key if it is also in the second words counter. I would also leave scaling the similarity to the caller.

Use it within a if __name__ == "__main__": guard to allow importing from the module. You might want to add a simple CLI interface with sys.argv (or do it properly and use argparse if you want more functionality). Throw in a f-string to make formatting the output to the user easier (and make it a percent with not too many digits shown).

import sys

if __name__ == "__main__":
    file1, file2 = sys.argv[1:3]
    sim = similarity(get_words(file1), get_words(file2))
    print(f"Similarity of {file1} and {file2}: {sim:.2%}")

Usage:

python3 similarity.py file.txt file_old.txt
# Similarity of file.txt and file_old.txt: 92.19%

For these two small files (21KiB), your code takes 0.3s, whereas my code only takes 0.05s on my machine.