97 questions
0
votes
1
answer
59
views
How do I convert a browse link to a playlist link
how to I convert a browse link to a playlist link
https://music.youtube.com/browse/MPREb_xFN2XG22AXC ---> https://music.youtube.com/playlist?list=OLAK5uy_nQY-UERFpsL1d5UTPVMjX7mtnVlKg7D4w
0
votes
1
answer
223
views
IOT based queue management system with nodeMcu (esp8266)
I want to create a queue management system used on counters in shops or banks as my data-structure project. I know HTML, CSS, some JavaScript and C++. This is for my DSA project in my institute. I am ...
0
votes
1
answer
35
views
Python Selenium Dropdown list Clickable
I want to scrap investing.com website. Firefox Webdriver closes when clicking on one of the dropdown list item.
Dropdown menu is hiidden. I want to choose "BİST Ulusal Tüm" item. What can i ...
- 1
0
votes
1
answer
163
views
Python Scrapy 999 error meaning. Linkedin company data scraping
I am trying to scrape company data on Linkedin.
In order to get to the company page, login is required.
I am using the following Scrapy Spider:
import scrapy
from scrapy import Spider
from scrapy.http ...
- 487
0
votes
0
answers
69
views
issues with selenium, not finding by xpath on iframe
I'm doing web scraping with Selenium in python and I need to click a button to extract some data this button is inside of an iframe which I already switched (switch.to_frame()) and Selenium still ...
1
vote
1
answer
203
views
How to extract html input value by id with Selenium in Python
How can I extract the value 4 using elementId maxId in Python.
<input type="hidden" id="maxId" value="4">
Python: 3.11.1, ChromeDriver: 109.0.5414.74
- 183
1
vote
1
answer
96
views
Element is found but not clickable
I'm trying to find an element by it's id, click on it and download a file.
driver.get(url);
driver.implicitly_wait(60);
time.sleep(3)
element = WebDriverWait(driver, 20).until(EC....
- 7,409
0
votes
0
answers
163
views
How to capture note screenshot from web page using typescript
I wanted to write code in typesript to capture image from web page
Like all browser have option in inspect element -> select element -> right click -> capture note screenshot
As show in ...
- 13
1
vote
1
answer
1k
views
How to check if the specific text on a website changed using python script
I'm trying to write a python script to check the status's display text for a specific country (ie. Ecuador)
on this website:
https://immi.homeaffairs.gov.au/what-we-do/whm-program/status-of-country-...
- 19
0
votes
2
answers
68
views
Grab information from inside the form using web scraping - python 3
I'm creating a project using web scraping , I'm having trouble extracting information from an Iframe form, when I try to extract the values of the name , position and company field.
Code I'm testing:...
- 3
1
vote
1
answer
35
views
When importing HTML into Google Sheets, there are issues with the order by and select col functions as well as the header
I am using this function =QUERY(INDEX(SUBSTITUTE(IMPORTHTML (I2 & "?refresh=" & I6,I3,I4), "*", )), "SELECT Col1, Col2, Col3, Col4 ORDER BY Col1 ASC")
This is ...
- 11
0
votes
2
answers
79
views
is it possible to receive Dom in google webscript?
I receive from google sheet some html tags to be displayed but it shows as raw string. here is how
Index.html
<? var BlogData = getSheetData("Blogs"); ?>
<? for(var i = 0; i <...
- 39
1
vote
2
answers
143
views
Selenium Python: Invalid Selector in find_element_by_css_selector (Error)
I'm doing an automation to search a certain process on a website and collect information. When requesting data collection, it returns the following error:
"Selenium Python: Invalid Selector in ...
- 27
0
votes
2
answers
1k
views
Web scripting invalid syntax in URL
I am a beginner of web scripting.
I was following a tutorial on Edureka: A Beginner’s Guide to learn web scraping with python!.
There is a syntax error shown inside the URL of my script:
driver.get('&...
- 1
0
votes
1
answer
276
views
BeautifulSoup gives an empty list
I am trying to scrape website with Beautiful Soup. After printing the container, It gives me an empty list. How can I fix this?
import requests
from bs4 import BeautifulSoup
import lxml
URL = 'https:/...
- 53
0
votes
1
answer
443
views
Want to convert my google script web app into custom menu pop up
I made a google sheet script web app but I want to create a custom menu from where I can get my web app as pop up on the same page so that I don't need to always go to web app link or if any ...
-2
votes
1
answer
698
views
Prevent default action with a programatically dispatched event
In this talk at 32:50 the speaker shows this code:
const nextClick = new Promise(resolve => {
link.addEventListener('click', resolve, { once: true });
});
nextClick.then(event => {
...
- 1,201
1
vote
2
answers
438
views
Extract text from dynamic Web page using R
I am working on a data prep tutorial, using data from this article: https://www.nytimes.com/interactive/2021/01/19/upshot/trump-complete-insult-list.html#
None of the text is hard-coded, everything is ...
- 10.5k
-2
votes
1
answer
2k
views
How do I fill in and submit a form on another website with axios?
I was making a shoe raffle bot with Node.js and was originally using headless Puppeteer to automate the process of filling out the raffle form and submitting it. I was told that Puppeteer is very CPU-...
- 384
1
vote
0
answers
101
views
How would you recommend I automate website data entry?
I'm looking for a simple script that I can use to automate website data entry.
Trigged when the value of a cell in a gsheet changes, the script will login to the website, navigate to the correct url, ...
- 11
1
vote
1
answer
203
views
how to modify html using javascript after the page loads
I am new to js and learning how to modify html using js ,i want to modify:
<video controls="" controlslist="download" src="//storage.googleapis.com/media-session/caminandes/short.mp4#t=80">&...
- 23
1
vote
0
answers
133
views
Why is my VBA program not getting all of the tag names that are equal to "path" from an website?
I am wondering if anyone would please be able to help me with this issue I am having. I am trying to make a program that will find a certain icon on a web page and then click on all occurrences of it ...
- 11
0
votes
1
answer
69
views
Beautifulsoup find element
hi i was trying to find an element i have found the first one (soup1_title) easily but i had trouble to find next one! i just need book author for example for the first one my favor output is:"by J. G....
0
votes
1
answer
783
views
Get Updated Gas Price from EthGas Station in be used in a python application
Is there any way to know the updated Gas Price from EthGasStation to be used in python. I want to regularly know the average transaction confirmation time by gas price before send a transaction to the ...
- 467
1
vote
1
answer
1k
views
How to pull the vedio link name imbeded in a web page along with vedio name
I am trying to pull all the video link reference from the web page along with video names, i have tried below code.
#!/usr/bin/python3
from bs4 import BeautifulSoup
import requests
import urllib
url =...
- 488
0
votes
3
answers
3k
views
Why request.get() returning wrong page content?
I have been trying to parse a webpage using BeautifulSoup. When I import urlopen fromm urllib.request and open https://pbejobbers.com it returns following instead of webpage itself:
<html>
&...
- 27
0
votes
1
answer
496
views
Web scraping company details from LinkedIn --- not able to get body tag inside
I am trying to get company information from a LinkedIn account but I am not able to get any content inside body. Can you please tell me what is wrong?
I need to get
company
website
industry
...
- 68
0
votes
1
answer
2k
views
Web Scraping - Requests ConnectionError: ('Connection aborted.', OSError("(60, 'ETIMEDOUT')",))
I'm trying to access to a webpage. I tried 'UserAgent' to add headers, however, got timeout error:
my new codes:
from fake_useragent import UserAgent
import requests
url = "https://www.bestbuy....
- 45
1
vote
1
answer
850
views
Image doesn't download fully with aiohttp
I am using the following to get an image, but it's not properly downloaded and only a small top portion of image is seen. The file size is less than 256 kB. What's wrong and how could I fix it?
async ...
- 305
0
votes
2
answers
64
views
Preserve multi-line addresses separated with `<br/>`
How could I remove extra blank line between address line? I am
using Beautifulsoup to scraping from a web page.
I know that <br/> generates a new line. However, If I were to use
replace to ...
- 33
2
votes
1
answer
3k
views
How to run a script after a pull-request on GitHub?
Good morning, everyone,
I want to create a script which automatically update an issue on RedMine when someone make a pull-request on our GitHub based on the pull-request comment.
I wrote a script ...
- 145
0
votes
0
answers
94
views
Form filling and web scraping from a website
Not a good way to start, but pardon me if this if off-topic, it seems like a programming question though...
From an ASP.NET website I want to open a page in a new browser that has a toolbar at top ...
- 671
0
votes
0
answers
179
views
How to fix continuous reloading of a webpage by tamper monkey script?
I'm trying to build a script that will click 3 buttons in succession but after clicking the first button, the script goes into infinite reloading.
The site I'm trying to click buttons on: JDoodle
You ...
- 347
0
votes
3
answers
2k
views
ElementNotVisibleException error in Python/Selenium webdriver
I write this code for web scraping work:
browser.find_element_by_class_name('open_all_j').click()
This code line gives me an error:
selenium.common.exceptions.ElementNotVisibleException: Message: ...
0
votes
1
answer
632
views
Parse HTML into text with Div level in R
library(XML)
html <- read_html("https://www.sec.gov/Archives/edgar/data/1011290/000114036105007405/body.htm")
doc.html = htmlTreeParse(html, useInternal = TRUE)
doc.text = unlist(xpathApply(doc....
- 237
0
votes
3
answers
287
views
Can I use a format string in a URL to navigate to web page, replacing the specified portion of the URL?
I'm interested in creating some sort of simple C# application takes a user string and passes it into a target portion of the URL. For example, since user query is visible in the page URL DuckDuckGo
...
- 1
2
votes
0
answers
325
views
How many request for scraping can handle ScrapyRT
I have a single project with the web interface where I should download from 3000 to 20000 urls per week (month). I use tickets system for showing a progress: what is downloaded, what is pending for ...
- 1,825
2
votes
1
answer
143
views
Add in 'if' clause using beautifulSoup
I'm using BeautifulSoup to scrape a company site job positions (I have permission). The below code is able to run and the output is the url for the job posting, however I want to add in a criteria ...
- 99
-1
votes
2
answers
2k
views
How to use Web Script in Alfresco as a developer?
I am new to Alfresco and using Alfresco 5.2. I started learning it as an ECM and now I can understand the Share interface to some extent. But I need to master Alfresco as a developer, for which I ...
0
votes
1
answer
60
views
Can't create appropriate selector to grab names
How can I grab all the names under "Followed by" category from the below elements. The way the name have been laid out is kinda foreign to me that is why I can't get all of them. I have already used a ...
- 22.5k
1
vote
1
answer
204
views
Read page source before POST
I want to know if there is a way to POST parameters after reading the page source. Ex: read captcha before posting ID#
My current code:
import requests
id_number = "1"
url = "http://www....
- 13
4
votes
9
answers
4k
views
scraping website with login page
I currently login to the time from website using the following script.
browser = webdriver.Chrome('E:/Shared Folders/Users/runnerjp/chromedriver/chromedriver.exe')
browser.get("https://www.timeform....
- 779
0
votes
1
answer
487
views
Unable to fill form using splinter
I am trying to fill out a login form on https://login.alibaba.com/?spm=a2700.8293689.0.0.EKYnH1&tracelog=hd_signin. However i am unable to fill out the login form. Here is my current code.
...
- 528
0
votes
1
answer
342
views
rvest web content scraping issue / car trading website
Question
I wanted to rvest specific parts of the websites (car sales platform).
The CSS is frankly too confusing for me to figure out what's wrong on my own.
#### scraping the website www.otomoto....
1
vote
2
answers
61
views
While parsing any URL DOM is not loading
$ie = new-object -com internetexplorer.application
$ie.visible=$true
$ie.navigate('https://google.co.in')
while($ie.ReadyState -ne 4)
{
Write-Host "dom is loading"
$ie.ReadyState -eq 4
...
0
votes
1
answer
867
views
Get Data From Website With Web Scraping
I want to get all image from a website where the data store in a inner div, how can i fetch all image. I tried but it can't work. Here is my code
<?php
$html = file_get_contents('http://en....
- 33
1
vote
2
answers
352
views
Which is better web script or cmis in alfresco
I am curently developing a java/jee application using Alfresco as ECM. I want to know what is the best way to communicate with alfresco in a Java/jee app.Which is better using the web script or cmis ?
- 460
-1
votes
2
answers
5k
views
Alfresco webscript to get the list of all files and folder with their size
Need a js based alfresco webscript to get the list of all folders and files recursively alond with their size.
- 49
0
votes
1
answer
1k
views
javascript - switch in a function with 2 variables
I try to make a game but in following code I have defined a function with a for-loop wherein I puted an switch-statement. Now I see with console.log() that all worked fine but the switch-statement is ...
- 133
0
votes
1
answer
142
views
How can make my function rotate 360 and move up and down?
Hey guys so I'm working on animating my java script for a project, and being a newbie at java-script I have only figured out how to make my image move up and down in a set area. How can make my ...
user5327466