sqlalchemy insert - string argument without an encoding

Question

The code below worked when using Python 2.7, but raises a StatementError when using Python 3.5. I haven't found a good explanation for this online yet.

Why doesn't sqlalchemy accept simple Python 3 string objects in this situation? Is there a better way to insert rows into a table?

from sqlalchemy import Table, MetaData, create_engine
import json

def add_site(site_id):
    engine = create_engine('mysql+pymysql://root:password@localhost/database_name', encoding='utf8', convert_unicode=True)
    metadata = MetaData()
    conn = engine.connect()
    table_name = Table('table_name', metadata, autoload=True, autoload_with=engine)

    site_name = 'Buffalo, NY'
    p_profile = {"0": 300, "1": 500, "2": 100}

    conn.execute(table_name.insert().values(updated=True,
                                    site_name=site_name, 
                                    site_id=site_id, 
                                    p_profile=json.dumps(p_profile)))

add_site(121)

EDIT The table was previously created with this function:

def create_table():
    engine = create_engine('mysql+pymysql://root:password@localhost/database_name')
    metadata = MetaData()

    # Create table for updating sites.
    table_name = Table('table_name', metadata,
        Column('id', Integer, Sequence('user_id_seq'), primary_key=True),
        Column('updated', Boolean),
        Column('site_name', BLOB),
        Column('site_id', SMALLINT),
        Column('p_profile', BLOB))

    metadata.create_all(engine)

EDIT Full error:

>>> scd.add_site(121)
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1073, in _execute_context
    context = constructor(dialect, self, conn, *args)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/default.py", line 610, in _init_compiled
    for key in compiled_params
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/default.py", line 610, in <genexpr>
    for key in compiled_params
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/sql/sqltypes.py", line 834, in process
    return DBAPIBinary(value)
  File "/usr/local/lib/python3.5/dist-packages/pymysql/__init__.py", line 79, in Binary
    return bytes(x)
TypeError: string argument without an encoding

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user1/Desktop/server_algorithm/database_tools.py", line 194, in add_site
    failed_acks=json.dumps(p_profile)))
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1078, in _execute_context
    None, None)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
    exc_info
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/util/compat.py", line 185, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1073, in _execute_context
    context = constructor(dialect, self, conn, *args)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/default.py", line 610, in _init_compiled
    for key in compiled_params
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/default.py", line 610, in <genexpr>
    for key in compiled_params
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/sql/sqltypes.py", line 834, in process
    return DBAPIBinary(value)
  File "/usr/local/lib/python3.5/dist-packages/pymysql/__init__.py", line 79, in Binary
    return bytes(x)
sqlalchemy.exc.StatementError: (builtins.TypeError) string argument without an encoding [SQL: 'INSERT INTO table_name (updated, site_name, site_id, p_profile) VALUES (%(updated)s, %(site_name)s, %(site_id)s, %(p_profile)s)']

Please provide your model declarations and the full stack trace. — univerio
– univerio, Commented Aug 24, 2016 at 0:19
For this, I'm just using the SQL Expression Language, instead of the Object Relational Mapper, so I don't have model declarations. My understanding is that either approach should work. — haudarren
– haudarren, Commented Aug 24, 2016 at 16:42
Added an edit to show how the table was created, if helpful. — haudarren
– haudarren, Commented Aug 24, 2016 at 17:05
You are passing in a str object to site_name but it's expecting a bytes (since it's a BLOB). — univerio
– univerio, Commented Aug 24, 2016 at 19:01

haudarren · Accepted Answer · 2016-08-25 20:11:15Z

7

As univerio mentioned, the solution was to encode the string as follows:

conn.execute(table_name.insert().values(updated=True,
                                    site_name=site_name, 
                                    site_id=site_id, 
                                    p_profile=bytes(json.dumps(p_profile), 'utf8')))

BLOBs require binary data, so we need bytes in Python 3 and str in Python 2, since Python 2 strings are sequences of bytes.

If we want to use Python 3 str, we need to use TEXT instead of BLOB.

edited Aug 25, 2016 at 20:11

answered Aug 24, 2016 at 21:57

haudarren

4451 gold badge4 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

clony Over a year ago

Or we can do: p_profile=json.dumps(p_profile).encode('utf-8'). SQLAlchemy will automatically convert it to byte when it performs the insert.

airiox · Accepted Answer · 2020-01-10 01:19:18Z

0

You simply just need to convert your string to a byte string ex:

                                    site_name=str.encode(site_name), 
                                    site_id=site_id, 
                                    p_profile=json.dumps(p_profile)))```

or

```site_name = b'Buffalo, NY'```

answered Jan 10, 2020 at 1:19

airiox

1

Collectives™ on Stack Overflow

sqlalchemy insert - string argument without an encoding

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related