GAE error with task queues. Is something wrong with my setup? (python)

Question

I'm trying task queues for the first time and while I think I have everything set up correctly, I am getting an error.

First, here is my queue.yaml file:

total_storage_limit: 500M
queue:
- name: loader
  rate: 1/s
  bucket_size: 1

Second, here is my code to call my task queue (it's called load.py). It needs to run every day, so I have this particular script run as a cron job.

for file in archiveList:
        taskqueue.add(queue_name='loader',url='/tasks/loadworker',params = {'ID':file[:-4],'XML':str(file)})

My loadworker.py file is basically this:

class MainPage(webapp2.RequestHandler):
    def post(self):
        ID = self.request.get('ID')
        XML = self.request.get('XML')
        tmp = trialDatabase.get_or_insert(ID)
        #REST OF CODE GOES BELOW

Here is the error I am seeing:

WARNING  2014-04-11 15:24:41,156 taskqueue_stub.py:1974] Task task936 failed to execute. This task will retry in 0.400 seconds
INFO     2014-04-11 15:24:41,156 module.py:627] loadandprocess: "POST /tasks/loadworker HTTP/1.1" 404 -

Off the top of my head, I'm wondering: do the task queue "caller" and the task queue "worker" need to be in the same module? I have my task queue worker in my default app.yaml file, and my loader is in the loadandprocess.yaml file. How does it work for usage, like...would each task worker call a B4 instance class as specified in my loadandprocess.yaml file?

Thanks!

Edit:

Here is the relevant part of app.yaml:

- url: /tasks/loadworker
  script: loadworker.application
  login: admin

Do you have a handler in your app.yaml file for /tasks/loadworker pointing to loadworker.py ? — IanGSY
– IanGSY, Commented Apr 11, 2014 at 15:53
Do you have something like application = webapp2.WSGIApplication([('/tasks/loadworker', MainPage)]) at the bottom of your loadworker.py file? The error being returned is 404, page not found, which would indicate the request is not being handled. — IanGSY
– IanGSY, Commented Apr 11, 2014 at 16:17
Figured it out -- the worker and handler need to be in the same module. I had my worker in app.yaml and my handler in loadandprocess.yaml. Thank you, for the replies, though, I appreciate it. — user3058197
– user3058197, Commented Apr 11, 2014 at 18:58
They can be in separate modules. You need to specify the 'target' parameter when you create the task. — Greg
– Greg, Commented Apr 11, 2014 at 20:34

GAEfan · Accepted Answer · 2014-04-11 18:07:32Z

2

Loadworker is returning a 404. A taskqueue task must return an http code 200, else it will continue to fail and retry.

Make sure loadworker returns an http response, something like:

return HttpResponse("Did it", mimetype='text/plain')

edited Apr 11, 2014 at 18:07

answered Apr 11, 2014 at 17:25

GAEfan

11.4k2 gold badges20 silver badges34 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

user3058197 Over a year ago

The end of my page has: application = webapp2.WSGIApplication([('/tasks/loadworker', MainPage)],debug=True). I guess I need to add a return statement somewhere in my def post(self): method?

IanGSY Over a year ago

I do not think this is the problem here, the request will return 200 status code by default if the page is run successfully, you do not need to set it yourself.

user3058197 Over a year ago

Figured it out -- the worker and handler need to be in the same module. I had my worker in app.yaml and my handler in loadandprocess.yaml. Thank you, for the replies, though, I appreciate it.

Collectives™ on Stack Overflow

GAE error with task queues. Is something wrong with my setup? (python)

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related