1

I use Python.SQLAlchemy with MySQL Database. All code bellow normal work for latin symbols in data, but not work for cyrilic:

UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-17: character maps to

I added "encoding='utf8', convert_unicode=True" in engine constructor but is nothing to change

Charset/Collation in MySQL for Table: utf8 / utf8-bin

Code:

Database connect

    def DB_alchemy(self, category, db="mysql://user:pass@localhost/all_gid_2"):
    self.sql_engine = sql.create_engine(db, echo=True, encoding='utf8', convert_unicode=True)

    metadata = sql.MetaData(self.sql_engine)

    sql_tbl_name_products = category+'_products'
    sql_tbl_name_class = category + '_classes'

    self.tbl_products = sql.Table(sql_tbl_name_products, metadata, autoload=True)
    self.tbl_classes = sql.Table(sql_tbl_name_class, metadata, autoload=True)

    self.connection = self.sql_engine.connect()

....

Insert

def Insert_df_to_SQL(self, df, tbl):

        dict_insert = df.to_dict(orient='records')

        insert_qry = tbl.insert()
        self.connection.execute(insert_qry, dict_insert)

The echo of SQLAlchemy engine in cyrrilic data:

2020-08-02 22:07:05,839 INFO sqlalchemy.engine.base.Engine INSERT INTO Nb_classes (type, class_subtype, text, explanation, name) VALUES (%s, %s, %s, %s, %s) 2020-08-02 22:07:05,840 INFO sqlalchemy.engine.base.Engine ('CL', 'Производительность', 'Расширенная функциональность', 'Стандартный процессор, внешняя графика начального уровня, мультимедиа', 'CL_discret_lite') 2020-08-02 22:07:05,840 INFO sqlalchemy.engine.base.Engine ROLLBACK

    Traceback (most recent call last):
  File "C:/Users/shulya403/Shulya403_works/all_gid_2/Database/db_insert_pd.py", line 259, in <module>
    FillDB.Classes_to_SQL(df_new=FillDB.df_Classes.head(3))
  File "C:/Users/shulya403/Shulya403_works/all_gid_2/Database/db_insert_pd.py", line 241, in Classes_to_SQL
    self.Insert_df_to_SQL(df_select, self.tbl_classes)
  File "C:/Users/shulya403/Shulya403_works/all_gid_2/Database/db_insert_pd.py", line 166, in Insert_df_to_SQL
    self.connection.execute(insert_qry, dict_insert)
  File "C:\Users\shulya403\Shulya403_works\all_gid_2\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1014, in execute
    return meth(self, multiparams, params)
  File "C:\Users\shulya403\Shulya403_works\all_gid_2\venv\lib\site-packages\sqlalchemy\sql\elements.py", line 298, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "C:\Users\shulya403\Shulya403_works\all_gid_2\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1133, in _execute_clauseelement
    distilled_params,
  File "C:\Users\shulya403\Shulya403_works\all_gid_2\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1318, in _execute_context
    e, statement, parameters, cursor, context
  File "C:\Users\shulya403\Shulya403_works\all_gid_2\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1515, in _handle_dbapi_exception
    util.raise_(exc_info[1], with_traceback=exc_info[2])
  File "C:\Users\shulya403\Shulya403_works\all_gid_2\venv\lib\site-packages\sqlalchemy\util\compat.py", line 178, in raise_
    raise exception
  File "C:\Users\shulya403\Shulya403_works\all_gid_2\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1278, in _execute_context
    cursor, statement, parameters, context
  File "C:\Users\shulya403\Shulya403_works\all_gid_2\venv\lib\site-packages\sqlalchemy\engine\default.py", line 593, in do_execute
    cursor.execute(statement, parameters)
  File "C:\Users\shulya403\Shulya403_works\all_gid_2\venv\lib\site-packages\MySQLdb\cursors.py", line 199, in execute
    args = tuple(map(db.literal, args))
  File "C:\Users\shulya403\Shulya403_works\all_gid_2\venv\lib\site-packages\MySQLdb\connections.py", line 280, in literal
    s = self.string_literal(o.encode(self.encoding))
  File "C:\Users\shulya403\AppData\Local\Continuum\anaconda3\lib\encodings\cp1252.py", line 12, in encode
    return codecs.charmap_encode(input,errors,encoding_table)
UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-17: character maps to <undefined>

What can i do?

2
  • 2
    Try adding ?charset=utf8mb4 to the end of your connection URI. Commented Aug 2, 2020 at 19:26
  • YES, It works! ?charset=utf8mb4, Gord Thompson, Reagrds! Commented Aug 2, 2020 at 19:37

1 Answer 1

8

Try adding ?charset=utf8mb4 to the end of your connection URI. – Gord Thompson

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.