25

I am having this error when translating thousands of text data in an iteration:

Expecting value: line 1 column 1 (char 0)

My code for translating big amounts of text:

translatedList = []
for index, row in df.iterrows():
    newrow = copy.deepcopy(row)
    try:
        # translate the 'text' column
        translated = translator.translate(row['text'], dest='en')
        newrow['translated'] = translated.text
    except Exception as e:
        print(str(e))
        continue
    translatedList.append(newrow)

I receive this error after translating about 2-3k rows.

Improve this question

11 Answers 11

Reset to default
29

I kind of figured out the problem. I think that this is about Google API's request limit.

I solved this by reinitializing the translator API on every iteration:

import copy
from googletrans import Translator

translatedList = []
for index, row in df.iterrows():
    # REINITIALIZE THE API
    translator = Translator()
    newrow = copy.deepcopy(row)
    try:
        # translate the 'text' column
        translated = translator.translate(row['text'], dest='en')
        newrow['translated'] = translated.text
    except Exception as e:
        print(str(e))
        continue
    translatedList.append(newrow)
Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

I could find two older questions (here and here) which actually relate this to a character limit. Weirdly enough, they mention a 15k limit which I cannot find in relevant docs (The questions are more recent than latest updates to quotas. In any case this is not an informative error message...
Whether it is related to the 15k limit or not, the error does not provide any useful information as you say. However you are not blocked by any limit if you initialize the API each time.
9

This is what I had to do to bypass their API call restriction... I use a VPN, specifically Nord-Vpn, so to do it the way I did you would need to be able to connect/disconnect from/to a VPN through the terminal...

    def translate_text(text, dest_language="en"):
        # Used to translate using the googletrans library
        import json
        translator = googletrans.Translator()
        try:
            translation = translator.translate(text=text, dest=dest_language)
        except json.decoder.JSONDecodeError:
            # api call restriction
            process = subprocess.Popen(["nordvpn", "d"], stdout=subprocess.PIPE)
            process.wait()
            process = subprocess.Popen(["nordvpn", "c", "canada"], stdout=subprocess.PIPE)
            process.wait()
            return Process_Data.translate_text(text=text, dest_language=dest_language)
        return translation
Improve this answer

4 Comments

This is a very interesting comment. I'm having the same issue, could you please go deeper on this method Im new on all this things. I do not how to use the vpn you mention
The googletrans api limits an IP address to a certain amount of calls within a certain amount of time. Because I needed to make a lot of calls, I decided to use my VPN which allows me to change my IP address. My code above detects when googletrans's API restricts a call, "except json.decoder.JSONDecodeError", and using the library "import subprocess" I execute commands to change my VPN server and then try an make the API call again, now with a different IP address.
That's great, unfortunately I don't have the experience to make this. What would I need in order to make your code work for me? Is it necessary to install/configurate anything?
You'll need to install a VPN on your Windows/macOS/Linux system that can be controlled through the terminal. Once you have done that you are golden. Just switch out ["nordvpn", "d"] and ["nordvpn", "c", "canada"] for the correct commands that you would execute in your terminal to disconnect and connect to your specific VPN. I use NordVPN hence the "nordvpn" command.
5

Google may be blocking your IP, use a VPN and it should work.

Improve this answer

Comments

4

There could be 2 reasons for this:
1. IP address is temporarily blocked.
2. You have reached the character limit.

I faced the same issue and ended up using another package called translate and it works flawlessly. The syntax is pretty similar too. You can find it here or do pip install translate

Improve this answer

2 Comments

Thanks, it worked for me to translate from Russian to English but I think its not good enough.
I tried this module, it's easier to use but the limit is so small
3

In my case, it's caused by emoji in strings. I removed them and everything works well.

Improve this answer

Comments

3

In my case, the error was caused by too many requests in a short time period and my IP address was temporarily blocked. I tried it the next day again and everything worked well.

Improve this answer

Comments

3

I will give a modified answer of Austin Marino, here a solution that just worked for me in a 2000 word list (would work in bigger list too).

first you need to install NordVPN and add it to the system path check this link :

https://support.nordvpn.com/Connectivity/Windows/1350897482/Connect-to-NordVPN-app-on-Windows-using-the-Command-Prompt.htm

The goal is so you can connect/disconnect and chose servers with CMD (you can do the same thing in linux) so you can cantrole theses NordVPN CMD commands through Python code.

Here is the function (please import the libraries) : 

   import random

listofservers = ["South Africa", "Egypt" , "Australia", "New Zealand",  "South Korea", "Singapore", "Taiwan", "Vietnam", "Hong Kong", "Indonesia", "Thailand", "Japan", "Malaysia", "United Kingdom", "Netherlands", "Germany", "France", "Belgium", "Switzerland", "Sweden","Spain","Denmark", "Italy", "Norway", "Austria", "Romania", "Czech Republic", "Luxembourg", "Poland", "Finland", "Hungary", "Latvia", "Russia", "Iceland", "Bulgaria", "Croatia", "Moldova", "Portugal", "Albania", "Ireland", "Slovakia","Ukraine", "Cyprus", "Estonia", "Georgia", "Greece", "Serbia", "Slovenia", "Azerbaijan", "Bosnia and Herzegovina", "Macedonia","India", 'Turkey', 'Israel', 'United Arab Emirates', 'United States', 'Canada','Mexico'
,"Brazil", "Costa Rica", "Argentina", "Chile"]

def SelectServer(l):
    return random.choice(l)

def translate_text(text, dest_language="en"):  
    # Used to translate using the googletrans library
    translator = googletrans.Translator()
    try:

        translation = translator.translate(text=text, dest=dest_language)

    except json.decoder.JSONDecodeError:
        # api call restriction

        print("exception !! déconection du VPN ")
        process = subprocess.Popen(["nordvpn", "-d"], shell = True ,stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        process.wait()

        time.sleep(5)

        srv = SelectServer(listofservers)

        print("sélection du serveur  : "+ srv + " et connexion")

        process = subprocess.Popen(["nordvpn", "-c", "-g", srv ], shell = True ,stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        process.wait()
        time.sleep(60)

        return translate_text(text=text, dest_language=dest_language)

    return translation.text


    #translate to EN and remove EN stopwords 
    ListCapTranslated = []
    for row  in ListCaptionsCleanFiltred:
        # REINITIALIZE THE API
        newrow = translate_text(row, dest_language="en")
        ListCapTranslated.append(newrow)

ListCapTranslated

Before running the code, please add NordVPN to System path and test connecting/disconnecting on servers through CMD so you make sure everything works.

Cheers.

Improve this answer

Comments

2

I have also faced this problem. In my case, it was due to translating text (in english) to english.

As a workaround I have used another package langdetect to route the non-english text to be translated using google translate.

some snippet from the code:

from langdetect import detect
lang = detect(title)
if lang == 'en':
    temp_dict['title'] = title
else:
    temp_dict['title'] = translator.translate(title, dest='en').text
Improve this answer

Comments

1

This happens due to the translation limit. You can either use VPNs or Tor to bypass the limitation. However, you can circumvent this by using the translingual python package. Also, the language codes can be accessed here, language codes.

from translingual import translate
# example
trans = translate.translate(data=['hello world', 'the world is yours', 'whatever you do, whatever I do', '2b or not 2b'],tolang='es',fromlang='en',thread=3)
print(trans.translate())
Improve this answer

Comments

0

It's due to API Limit. The translation limit for each Initialization. So Reinitialize the Translator after the limit by breaking down the code.

from googletrans import Translator

translator = Translator()

Improve this answer

Comments

0

The problem lies with the requests that one device can make with the same IP. Changing the VPN resolves the problem. A free and simple alternative to the NordVPN is the TunnelBear. You can download it from here.

For me, the maximum requests to the googletranslate API are 200. So, every 200 requests I change manually the VPN and then, I continue with the next requests.

Unfortunately, there is some manual work here, since you need every time to change the VPN connection. However, it is helpful in case you want to have fast results and avoid programming the VPN change in Python.

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.