Passing a variable in URL?

Question

I'm new in Python. I have a file which has a bunch of ids (integer values) written in them. It's a text file.

Now I need to pass each id inside the file into a url.

For example "https://example.com/[id]"

It will be done in this way

A = json.load(urllib.urlopen("https://example.com/(the first id present in the text file)"))
print A

What this will essentially do is that it will read certain information about the id present in the above URL and display it. I want this to work in a loop format where in it will read all the ids inside the text file and pass it to the URL mentioned in 'A' and display the values continuously..is there a way to do this?

pyfunc · Accepted Answer · 2012-06-21 07:03:40Z

31

Old style string concatenation can be used

>>> id = "3333333"
>>> url = "https://example.com/%s" % id
>>> print url
https://example.com/3333333
>>>

The new style string formatting:

>>> url = "https://example.com/{0}".format(id)
>>> print url
https://example.com/3333333
>>>

The reading for file as mentioned by avasal with a small change:

f = open('file.txt', 'r')
for line in f.readlines():
    id = line.strip('\n')
    url = "https://example.com/{0}".format(id)
    urlobj = urllib.urlopen(url)
    try:
        json_data = json.loads(urlobj)
        print json_data
    except:
        print urlobj.readlines()

edited Jun 21, 2012 at 7:03

answered Jun 21, 2012 at 5:17

pyfunc

67k15 gold badges155 silver badges139 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

user1452759 Over a year ago

Thank you very much for helping me out. I'm facing a new problem now. Suppose I try to read a text file with say, just a handful of ids, it has no problem in executing. But my text file from where I'm reading the ids has a LOT of them. And the code fails to execute. What is the reason for this? Is there a way to get around this?

pyfunc Over a year ago

@user1452759: What is the error that you are getting? It would be nice to start from there. The solution should work for even if you have many ids. Chances are that not all the url fetch is returning data that can be loaded into json

pyfunc Over a year ago

@user1452759: Try the solution as I have mentioned in edited example. It will check if data can be loaded as json or else will print the output of what was fetched.

xvatar · Accepted Answer · 2012-06-21 05:29:00Z

5

lazy style:

url = "https://example.com/" + first_id

A = json.load(urllib.urlopen(url))
print A

old style:

url = "https://example.com/%s" % first_id

A = json.load(urllib.urlopen(url))
print A

new style 2.6+:

url = "https://example.com/{0}".format( first_id )

A = json.load(urllib.urlopen(url))
print A

new style 2.7+:

url = "https://example.com/{}".format( first_id )

A = json.load(urllib.urlopen(url))
print A

edited Jun 21, 2012 at 5:29

answered Jun 21, 2012 at 5:19

xvatar

3,25919 silver badges20 bronze badges

Comments

Jainik · Accepted Answer · 2021-02-08 16:15:35Z

3

Python 3+

New String formatting is supported in Python 3 which is a more readable and better way to format a string. Here's the good article to read about the same: Python 3's f-Strings

In this case, it can be formatted as

url = f"https://example.com/{id}"

Detailed example

When you want to pass multiple params to the URL it can be done as below.

name = "test_api_4"
owner = "[email protected]"

url = f"http://localhost:5001/files/create" \
f"?name={name}" \
f"&owner={owner}" \

We are using multiple f-string here and they can be appended by ''. This will keep them in the same line without inserting any new line character between them.

For values which have space

For such values you should import from urllib.parse import quote in your python file and then quote the string like: quote("firstname lastname")

This will replace space character with %20.

edited Feb 8, 2021 at 16:15

answered Oct 17, 2019 at 18:10

Jainik

2,4721 gold badge22 silver badges27 bronze badges

1 Comment

Kevin Lemaire Over a year ago

And any clue for value with / in it? Like dataset ids in Freenas?

Haldean Brown · Accepted Answer · 2012-06-21 05:21:31Z

2

The first thing you need to do is know how to read each line from a file. First, you have to open the file; you can do this with a with statement:

with open('my-file-name.txt') as intfile:

This opens a file and stores a reference to that file in intfile, and it will automatically close the file at the end of your with block. You then need to read each line from the file; you can do that with a regular old for loop:

  for line in intfile:

This will loop through each line in the file, reading them one at a time. In your loop, you can access each line as line. All that's left is to make the request to your website using the code you gave. The one bit your missing is what's called "string interpolation", which allows you to format a string with other strings, numbers, or anything else. In your case, you'd like to put a string (the line from your file) inside another string (the URL). To do that, you use the %s flag along with the string interpolation operator, %:

url = 'http://example.com/?id=%s' % line
A = json.load(urllib.urlopen(url))
print A

Putting it all together, you get:

with open('my-file-name.txt') as intfile:
  for line in intfile:
    url = 'http://example.com/?id=%s' % line
    A = json.load(urllib.urlopen(url))
    print A

answered Jun 21, 2012 at 5:21

Haldean Brown

12.8k5 gold badges45 silver badges60 bronze badges

2 Comments

user1452759 Over a year ago

Thank you very much for helping me out. I'm facing a new problem now. Suppose I try to read a text file with say, just a handful of ids, it has no problem in executing. But my text file from where I'm reading the ids has a LOT of them. And the code fails to execute. What is the reason for this? Is there a way to get around this?

Haldean Brown Over a year ago

Could you post the error message? It's hard to say without knowing what it says the problem is.

Collectives™ on Stack Overflow

Passing a variable in URL?

4 Answers 4

3 Comments

Comments

1 Comment

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

3 Comments

Comments

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related