python-requests

Fix Python – Download and save PDF file with Python requests module

by

I am trying to download a PDF file from a website and save it to disk. My attempts either fail with encoding errors or result in blank PDFs.
In [1]: import requests

In [2]: url = ‘http://www.hrecos.org//images/Data/forweb/HRTVBSH.Metadata.pdf’

In [3]: response = requests.get(url)

In [4]: with open(‘/tmp/metadata.pdf’, ‘wb’) as f:
…: f…..

Fix Python – Why doesn’t requests.get() return? What is the default timeout that requests.get() uses?

by

In my script, requests.get never returns:
import requests

print (“requesting..”)

# This call never returns!
r = requests.get(
“http://www.some-site.example”,
proxies = {‘http’: ‘222.255.169.74:8080’},
)

print(r.ok)

What could be the possible reason(s)? Any remedy? What is the default timeout that get uses?
….

Fix Python – Difference between data and json parameters in python requests package

by

What is the difference between the data and json parameters in the python Requests package?
It is unclear from the documentation
Does this code:
import requests
import json
d = {‘a’: 1}
response = requests.post(url, data=json.dumps(d))

Note that we convert the dict to JSON here ☝️ !
Do anything different than:
import requests
import json
d = {‘a’….

Fix Python – zsh: no matches found: requests[security]

by

I am trying to run a python urllib2 script and getting this error:

InsecurePlatformWarning: A true SSLContext object is not available.
This prevents urllib3 from configuring SSL appropriately and may cause
certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning….

Fix Python – How to “log in” to a website using Python’s Requests module?

by

I am trying to post a request to log in to a website using the Requests module in Python but its not really working. I’m new to this…so I can’t figure out if I should make my Username and Password cookies or some type of HTTP authorization thing I found (??).
from pyquery import PyQuery
import requests

url = ‘http://www.locationary.com/home/in….

Fix Python – Log all requests from the python-requests module

by

I am using python Requests. I need to debug some OAuth activity, and for that I would like it to log all requests being performed. I could get this information with ngrep, but unfortunately it is not possible to grep https connections (which are needed for OAuth)
How can I activate logging of all URLs (+ parameters) that Requests is accessing?
….

Fix Python – Python Requests and persistent sessions

by

I am using the requests module.
I have figured out how to submit data to a login form on a website and retrieve the session key, but I can’t see an obvious way to use this session key in subsequent requests.
Can someone fill in the ellipsis in the code below or suggest another approach?
>>> import requests
>>> login_data = {‘formPosted’: ‘1’, ‘lo….

Fix Python – How to send cookies in a post request with the Python Requests library?

by

I’m trying to use the Requests library to send cookies with a post request, but I’m not sure how to actually set up the cookies based on its documentation. The script is for use on Wikipedia, and the cookie(s) that need to be sent are of this form:
enwiki_session=17ab96bd8ffbe8ca58a78657a918558e; path=/; domain=.wikipedia.com; HttpOnly

However, t….