Scheduler makes a python script run but doesn't produce a csv file

Question

I've written a script in python to scrape some links from a webpage and write them to a csv file. My script does this in the right way when run it from an IDE.

When I run the same using windows task scheduler, I can see that a command prompt pops up and the script runs and prints result there as well but I don't get any csv file when the task is done.

Am I missing something?

What possible change should I bring about to get a csv file when the script is run via .bat file from windows task scheduler?

.bat contains:

@echo off
"C:\Users\WCS\AppData\Local\Programs\Python\Python36-32\python.exe" "C:\Users\WCS\AppData\Local\Programs\Python\Python36-32\examplescript.py"

examplescript.py contains:

import csv
import requests
from urllib.parse import urljoin
from bs4 import BeautifulSoup

url = "https://www.janglo.net/component/option,com_sobi2/"

def get_info(link):
    res = requests.get(url)
    soup = BeautifulSoup(res.text,"lxml")
    for items in soup.select("#sobi2CatListSymbols .sobi2SubcatsListItems a[title]"):
        if items.text=="Tutors":
            ilink = f"{urljoin(url,items.get('href'))}"
    return ilink

def get_links(tlink):
    linklist = []
    res = requests.get(tlink)
    soup = BeautifulSoup(res.text,"lxml")
    for item in soup.select(".sobi2ItemTitle a"):
        linklist.append(urljoin(url,item.get("href")))
    return linklist

if __name__ == '__main__':
    with open("task_scheduler.csv","a",newline="") as infile:
        writer = csv.writer(infile)
        item = get_info(url)
        for nlink in get_links(item):
            print(nlink)
            writer.writerow([nlink])

I've clearly specified the location of the ".bat" file below:

"C:\Users\WCS\AppData\Local\Programs\Python\Python36-32\test_schedule.bat"

You don't need a single-line batch file to run the Python script as a task. I don't know why you think you need it. — Eryk Sun
– Eryk Sun, Commented Dec 2, 2018 at 14:47
That is because I could not find any way to save the csv file in the python directory without executing .bat file @eryksun. — SIM
– SIM, Commented Dec 3, 2018 at 11:02
Put the fully-qualified path of "python.exe" in the "program" field; the fully-qualified and quoted path of the script in the "arguments" field; and the fully-qualified working directory in the "start in" field. That said, you shouldn't save data files in the Python directory or in application directories, for various reasons, including the fact that in general your program may not have permission to create files there. For user-visible data, use well-known user data folders such as "Documents". — Eryk Sun
– Eryk Sun, Commented Dec 3, 2018 at 14:38

Martijn Pieters · Accepted Answer · 2018-12-02 15:33:44Z

3

Your code writes to the current working directory, because it opens a filename without a path. The location of the .bat file does not determine the current working directory of code being run.

You didn't specify a working directory; I'd expect that to be set with the Start in field in the scheduler, so the file is being created in the default location, %Windir%\System32\.

Either set a directory in the Start in field for the CSV file to be created in, have your batch script use cd to move the current working directory to the right location, or use an absolute path when opening the file.

For example, if your batch script first runs cd /d %~dp0 then the working directory is changed to the directory of the batch script, and relative paths in Python are resolved against that location.

edited Dec 2, 2018 at 15:33

answered Dec 2, 2018 at 13:57

Martijn Pieters

1.1m326 gold badges4.2k silver badges3.4k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

SIM Over a year ago

Right you are @sir Martijn Pieters!! I found a csv file containing those required fields in that default directory (system32). However, when I specify this C:\Users\WCS\Desktop address right next to Start in (optional): inputbox (visible in the image above), I still get the csv file in the default directory. Did I follow your instruction properly?

Martijn Pieters Over a year ago

@asmitu: I don't know. I don't use Windows myself, and my research shows that that option should set the current working directory for the command you are running. However, in case that the batch script CWD doesn't carry forward, I added another option: cd /d %~dp0.

Collectives™ on Stack Overflow

Scheduler makes a python script run but doesn't produce a csv file

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related