Python Downloading twitter video using python (without using twitter api)

Question

I want to download twitter video without using twitter’s official API . So when i open an icognito and chrome dev tool in chrome and request any video like “https://twitter.com/KTHopkins/status/1248140219490209792” i see twitter requests two important headers to server the video 1.‘authorization’ 2. ‘x-guest-token’

which i seem to cannot get where these tokens are generated from ? Belown is my python request which i am sending .

import requests

headers = {

    'authority': 'api.twitter.com',

    'dnt': '1',

    'x-twitter-client-language': 'en',

    # 'x-csrf-token': '6089ceeab3324243e7b952679b2b7851',

    'authorization': 'Bearer AAAAAAAAAAAAAAAAAAAAANk3DgEAAAAAB0pZx4xjgXBOoalj%2FRbagurxD2M%3DG8634UVlBud8LrLG4nGo7FpN2RCO2xul5BuPKHuejUAV14O0KG',

    'user-agent': 'Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36',

    'sec-fetch-dest': 'empty',

    'x-guest-token': '1248286947669237760',

    'x-twitter-active-user': 'yes',

    'accept': '*/*',

    'origin': 'https://twitter.com',

    'sec-fetch-site': 'same-site',

    'sec-fetch-mode': 'cors',

    'accept-language': 'en-US,en;q=0.9,hi;q=0.8',

    # 'cookie': 'personalization_id="v1_tWyK8Fn5ofSPjSAEKsnyrw=="; guest_id=v1%3A158644503604220835; ct0=6089ceeab3324243e7b952679b2b7851; _twitter_sess=BAh7CSIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNo%250ASGFzaHsABjoKQHVzZWR7ADoPY3JlYXRlZF9hdGwrCDI2fl9xAToMY3NyZl9p%250AZCIlNmM2YjZiYTU4MzdhY2FkNDQwZjcwMGU1NDliNzEzN2Y6B2lkIiViOWUx%250AYzM5MDk3ZTQ0YzMyZDRkMGU3YTdkM2FlMGY2YQ%253D%253D--223c07ac4708a9bec30dec1e0e9c3d52544b310c; _ga=GA1.2.162154316.1586445033; _gid=GA1.2.1445748635.1586445033; gt=1248286947669237760',

}

response = requests.get('https://api.twitter.com/2/timeline/conversation/1248293309950255107.json')

print(response.text)

so help me in geeting those two tokens .

Looks like its generated by java script. You might try using selenium link. — Brandon Campbell
– Brandon Campbell, Commented Apr 10, 2020 at 13:11
@BrandonCampbell so if i am using selenium then how do i get those two tokens ? — pankaj roy
– pankaj roy, Commented Apr 10, 2020 at 13:15
Selenium runs a web browser. You give it commands to load pages, and interact with them. The generation would work just as it did in your browser. — Brandon Campbell
– Brandon Campbell, Commented Apr 10, 2020 at 16:48
@BrandonCampbell i have used selinium for automation , but i was asking how to get those two tokens . some coding please . — pankaj roy
– pankaj roy, Commented Apr 10, 2020 at 17:57

Alexander Adler · Accepted Answer · 2020-06-09 10:52:14Z

6

You can get a fresh x-guest-token via

curl -skL https://twitter.com/ -H 'User-Agent: Firefox' --compressed | grep -o 'gt=[0-9]*' | sed s.gt=..

i. e., by download the website's content and parse its JavaScript. Note: The User-Agent needs to be set to something that Twitter believes to understand JavaScript. Otherwise, they send a legacy version of the website which doesn't contain the token.

answered Jun 9, 2020 at 10:52

Alexander Adler

611 silver badge4 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

pankaj roy Over a year ago

And what about authorization token ? How do i fetch that ?

Alexander Adler Over a year ago

Has been the same in the last few years: authorization: Bearer AAAAAAAAAAAAAAAAAAAAANRILgAAAAAAnNwIzUejRCOuH5E6I8xnZz4puTs%3D1Zv7ttfk8LF81IUq16cHjhLTvJu4FA33AGWWjCpTnA

pankaj roy Over a year ago

But in my above snippet it can be seen that authorization code is different .

Alexander Adler Over a year ago

That's correct. I got both the Bearer and guest-token from interacting with the website. The website uses the API itself, especially the endpoint api.twitter.com/2/timeline/profile/. I cannot find documentation for this endpoint, but it returns (given some parameters) the past few tweets by that user. It requires both an Authorization and an x-guest-token header.

Alexander Adler Over a year ago

You're right, @JasonBaumgartner. But curl -vkL twitter.com/ > /dev/null 2>&1 | grep -o 'gt=[0-9]*' | sed s.gt=.. (getting the cookies) still works. I fear this method is slightly brittle and needs constant fidgeting when the website changes.

inteoryx · Accepted Answer · 2021-10-25 18:09:18Z

6

This github repo contains a python script that will download twitter videos. You can look at the source to see exactly how to do it.

The high level of what you need to do is:

Get the bearer token (The html of the twitter link you go to links a file called main.some random numbers.js. Within that javascript file is the bearer token.
Take the bearer token and call https://api.twitter.com/1.1/guest/activate.json using the bearer token as an authorization header

curl 'https://api.twitter.com/1.1/guest/activate.json' -X POST -H 'authorization: Bearer AAAAAAAAAAAAAAAAAAAAANRILgAAAAAAnNwIzUejRCOuH5E6I8xnZz4puTs%3D1Zv7ttfk8LF81IUq16cHjhLTvJu4FA33AGWWjCpTnA'

This should return json containing a valid guest token.
With the bearer token and guest token you can then make requests as if you were an unauthenticated user. To download the video you'll need to find the list of mp4 and m4s files that compose the video (in the resolution you want) and then download them and reconstitute them into an mp4 file. As I mentioned the linked github project has the source for all this. If you want the details look there!

edited Oct 25, 2021 at 18:09

answered Oct 25, 2021 at 18:03

inteoryx

8055 silver badges10 bronze badges

2 Comments

parvij Over a year ago

it doesn't work

inteoryx Over a year ago

Can you be more descriptive? Or open a github issue?

Collectives™ on Stack Overflow

Python Downloading twitter video using python (without using twitter api)

2 Answers 2

5 Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related