How to modify file content using python?

Question

The following code reads a file, uses syntax tree to append fullstop to docstrings of that file. How can I save changes made in called file? I understand present code doesn't change content in original file but the local variables accessing it. Can you suggest changes, if possible provide learning resource as well?

astcode.py

import ast
import sys
import os

filename = sys.argv[1]


# """Getting all the functions"""
ast_filename = os.path.splitext(ast.__file__)[0] + '.py'

with open(filename) as fd:
    file_contents = fd.read()

module = ast.parse(file_contents)

# with open(filename, 'w') as file:
# module level
if(isinstance(module.body[0], ast.Expr)):
    docstr = module.body[0].value.s
    if(module.body[0].value.s not in '.'):
        docstr += '.'
        # ast.dump(module, include_attributes=True)
    print(docstr)


# function level
function_definitions = [node for node in module.body if isinstance(node, ast.FunctionDef)]

for function in function_definitions:
    # next_node = function_definitions[idx].body
    next_node = function.body
    for new_node in next_node:
        if(isinstance(new_node, ast.Expr)):
            if(isinstance(new_node.value, ast.Str)):
                # Docstring stored in docstr variable.
                docstr = new_node.value.s

                if(docstr[-1] not in '.'):
                    new_node.value.s += '.'
                
                # astString = ast.dump(new_node, annotate_fields=True, include_attributes=True)
                # print(astString)
                # compile(astString, filename, 'eval')
                # print(exec(astString))
                print(new_node.value.s)

    # for line in module:
        # file.write(line)

Example

testfile.py

def readDictionaryFile(dictionary_filename):
    """readDictionaryfile doc string"""
    return []

def readTextFile(text_filename):
    """readTextfile doc string"""
    return []

$ python3 astcode.py testfile.py

Expected

testfile.py

def readDictionaryFile(dictionary_filename):
    """readDictionaryfile doc string."""
    return []

def readTextFile(text_filename):
    """readTextfile doc string."""
    return []

Note: Fullstop(.) appended.

I'm not familiar with the AST library, but I believe the way to write it back to a file is with the dump(...) method, which can convert the entire AST into a single string. docs.python.org/3/library/ast.html — user1324109
– user1324109, Commented Mar 26, 2021 at 13:18
@user1324109 dump function returns formatted dump of the tree in node and helps in debugging, the string you are referring shows the edited content with row/column numbers if passed with optional arguments. I believe there's something to do with compile function of ast library itself, I am leaving a link to it... AST Compile. I hope you would like it. My question is not solved yet... do share your thoughts if you find something. Thanks — Naman Jain
– Naman Jain, Commented Mar 26, 2021 at 14:18
I ran this on a sample python file and got an attribute error on value.s Perhaps a small input and expected output would make it more clear. Is the question about the actual output, or just how to write to a file? — Kenny Ostrom
– Kenny Ostrom, Commented Mar 26, 2021 at 14:30

Kenny Ostrom · Accepted Answer · 2021-03-26 17:33:43Z

2

Looking at the documentation link, I notice there's a NodeVisitor and NodeTransformer with a code example. I looked at how they unparse a function def, and it's basically the same as you've done in your original question, so I used that.

# https://docs.python.org/3/library/ast.html#ast.NodeTransformer
class MyDocstringTransformer(ast.NodeTransformer):
    def visit_FunctionDef(self, node):
        if len(node.body):
            if isinstance(node.body[0], ast.Expr):
                if isinstance(node.body[0].value, ast.Constant):
                    if isinstance(node.body[0].value.value, str):
                        docstring = node.body[0].value.value
                        node.body[0].value.value = docstring + '.'
        return node

Using python 3.9's ast module gets us https://docs.python.org/3/library/ast.html#ast.unparse which is about as close as we can get to changing the ast node and then rewriting the original file.

tree = ast.parse(file_contents)
new_tree = MyDocstringTransformer().visit(tree)
print(ast.unparse(new_tree))

Instead of just overwriting to the same filename, you may want to write to a temp file, then let the OS attempt to delete the old file and rename the temp file to the old name, thus performing the replace in the OS.

answered Mar 26, 2021 at 17:33

Kenny Ostrom

5,9913 gold badges26 silver badges34 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Naman Jain Over a year ago

Thanks @KennyOstrom. Why shouldn't I overwrite to the same file?

Collectives™ on Stack Overflow

How to modify file content using python?

Example

Expected

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

Example

Expected

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related