0

I have a task that was working fine on my local server but when I pushed it to Heroku, nothing happens. there are no error messages. I am a newbie when it comes to this and locally I would start the worker by doing

celery worker -A blog -l info. 

So I'm guessing that's the issue may have to do with that. because I don't know to do this. I doubt I'm supposed to do this in my app. heres my code

celery.py

import os

from celery import Celery

from django.conf import settings

# set the default Django settings module for the 'celery' program.
os.environ.setdefault(
    'DJANGO_SETTINGS_MODULE', 'gettingstarted.settings'
)

app = Celery('blog')

# Using a string here means the worker will not have to
# pickle the object when using Windows.
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

my tasks.py

import requests
import random
import os

from bs4 import BeautifulSoup
from .celery import app
from .models import Post
from django.contrib.auth.models import User


@app.task
def the_star():
    def swappo():
        user_one = ' "Mozilla/5.0 (Windows NT 6.0; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0" '
        user_two = ' "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5)" '
        user_thr = ' "Mozilla/5.0 (Windows NT 6.3; Trident/7.0; rv:11.0) like Gecko" '
        user_for = ' "Mozilla/5.0 (Macintosh; Intel Mac OS X x.y; rv:10.0) Gecko/20100101 Firefox/10.0" '

        agent_list = [user_one, user_two, user_thr, user_for]
        a = random.choice(agent_list)
        return a

    headers = {
        "user-agent": swappo(),
        "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
        "accept-charset": "ISO-8859-1,utf-8;q=0.7,*;q=0.3",
        "accept-encoding": "gzip,deflate,sdch",
        "accept-language": "en-US,en;q=0.8",
    }

    # scraping from worldstar
    url_to = 'http://www.worldstarhiphop.com'
    html = requests.get(url_to, headers=headers)
    soup = BeautifulSoup(html.text, 'html5lib')
    titles = soup.find_all('section', 'box')
    name = 'World Star'

    if os.getenv('_system_name') == 'OSX':
        author = User.objects.get(id=2)
    else:
        author = User.objects.get(id=3)

    def make_soup(url):
        the_comments_page = requests.get(url, headers=headers)
        soupdata = BeautifulSoup(the_comments_page.text, 'html5lib')
        comment = soupdata.find('div')
        para = comment.find_all('p')
        kids = [child.text for child in para]
        blu = str(kids).strip('[]')
        return blu

    cleaned_titles = [title for title in titles if title.a.get('href') != 'vsubmit.php']
    world_entries = [{'href': url_to + box.a.get('href'),
                      'src': box.img.get('src'),
                      'text': box.strong.a.text,
                      'comments': make_soup(url_to + box.a.get('href')),
                      'name': name,
                      'url': url_to + box.a.get('href'),
                      'embed': None,
                      'author': None,
                      'video': False
                      } for box in cleaned_titles][:10] # The count

    for entry in world_entries:
        post = Post()
        post.title = entry['text']
        title = post.title
        if not Post.objects.filter(title=title):
            post.title = entry['text']
            post.name = entry['name']
            post.url = entry['url']
            post.body = entry['comments']
            post.image_url = entry['src']
            post.video_path = entry['embed']
            post.author = entry['author']
            post.video = entry['video']
            post.status = 'draft'
            post.save()
            post.tags.add("video", "Musica")
    return world_entries

my views.py

def shopan(request):
    the_star.delay()
    return redirect('/')

I have multiple instances of REDIS_URL

soI ran

heroku redis:promote REDIS_URL

and that's what I use in my environment variable which you can see above. How can i make this work?

1 Answer 1

1

You need to add an entry in your Procfile to tell Heroku to start the Celery worker:

worker:celery worker -A blog -l info
Sign up to request clarification or add additional context in comments.

2 Comments

I just did what you said and pushed it to my server but nothing happened. I went back to the docs and searched for procfile and it said heroku ps:scale worker=1, I tried to do that and it said Couldn't find that formation
I had to space it out so there was space between the colon and celery like this worker: celery worker -A blog -l info and then I had to run heroku ps:scale worker=1 and ot worked

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.