Can migrations.RunPython run arbitrary python code?

Question

I want to include the processing of many many raw SQL files (functions, triggers ...) at the end of my migrations.

I thought to write my own Special operation, but I've been told to use the django.db.migrations.RunPython() command instead and put some django.db.migrations.RunSQL() commands in the called function.

As RunPython() takes a callable with 2 instances (an App and a SchemaEditor) I seriously doubt (I reviewed the source a bit) that I can call a function with pure python code, it seems like it could only perform ORM manipulations. Should I use execute_from_command_line() inside the RunPython ? Or is that way doomed to failure ? Or doing things poorly ?

from __future__ import unicode_literals
from django.db import migrations

def load_sql(apps, schema_editor):

    from os.path import normpath, dirname, isfile, join
    from os import listdir

    sql_folder_path = '/backoffice/sql/'

    def load_raw_sql(folder_inside):
        folder_path = join(sql_folder_path, folder_inside)
        sql_files = [join(folder_path, f) for f in listdir(folder_path) if isfile(join(folder_path, f))]
        for sql_file in sql_files:
            with open(sql_file, 'r') as g:
                migrations.RunSQL(g.read())

    folders = ['functions', 'index', 'triggers']

    for folder in folders:
        load_raw_sql(folder)


class Migration(migrations.Migration):
    dependencies = [
        ('app1', '0001_squashed_0018_auto_20150616_0708'),
    ]

    operations = [
        migrations.RunPython(load_sql),
    ]

We are using PostGreSQL. Thanks in advance for your answers.

"I seriously doubt...that I can call a function with pure python code, it seems like it could only perform ORM manipulations." This sentence is really throwing me. What do you mean by pure Python vs only ORM manipulations? All Django is written in Python. What is "impure" about it? — Two-Bit Alchemist
– Two-Bit Alchemist, Commented Jul 2, 2015 at 19:27
Wrong words, but I mean that I've only seen models-like implementationsinside runPython() commands, and I want to know if I could write any python code in it ? — Phylante
– Phylante, Commented Jul 2, 2015 at 19:37
Presumably. Have you tried it? What would stop you? Even those models are just Python code. — Two-Bit Alchemist
– Two-Bit Alchemist, Commented Jul 2, 2015 at 20:56
I've tried the above code, seems like I am doing things the wrong way, I don't get any output if I put print statements, my django_migrations table gets an entry for the migration when I run the code, so it doesn't fail. I can't figure out if i need the link sqlparse package and if i should use RunSQL or manually use a database connection. — Phylante
– Phylante, Commented Jul 3, 2015 at 8:17

Greg · Accepted Answer · 2015-07-02 21:28:41Z

5

I'd have thought it fairly obvious that an operation called RunPython can, er, run python code, but the short answer is yes, you can do whatever you want in a RunPython operation.

The main thing to consider is that the code you write needs to keep working in the future (or until you squash your migrations) because every time you run migrate or makemigrations, all the previous migrations are used to derive the current state of the database.

answered Jul 2, 2015 at 21:28

Greg

10.4k5 gold badges47 silver badges46 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Phylante Over a year ago

Thanks, so I should assume that it's my code that's broken right ?

Phylante · Accepted Answer · 2015-07-03 10:13:12Z

1

django.db.migration.RunPython can accept any python code. The problem here was that I tried to use RunSQL() inside the call I made with RunPython(), which seems not to work in my case because I didn't provide an app and a schema editor. I managed to run my batch of SQL files thanks to a cursor and wrote inside my callable:

from django.db import migrations
cursor = connections['default'].cursor()
sql = "FOO;"
cursor.execute(sql)

answered Jul 3, 2015 at 10:13

Phylante

1482 silver badges10 bronze badges

Collectives™ on Stack Overflow

Can migrations.RunPython run arbitrary python code?

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related