Python - Requests

Python Requests 

Introduction to Python Requests

The Python Requests library is one of the most widely used tools for sending HTTP requests in Python. It provides a simple, elegant, and human-readable syntax for interacting with web services and APIs. Whether you are performing web scraping, working with RESTful APIs, or automating web tasks, the Requests library makes it easy to send HTTP/HTTPS requests and handle responses.

Python Requests is a third-party library that simplifies HTTP requests. Unlike the built-in urllib module, Requests provides a cleaner API and abstracts many complexities like connection handling, cookies, headers, and sessions.

Some of the key features of Python Requests include:

• Sending HTTP GET, POST, PUT, DELETE, PATCH requests.
• Handling query parameters, headers, and form data.
• Working with cookies and sessions.
• Automatic content decoding (JSON, XML, HTML, etc.).
• Timeouts, retries, and error handling.
• Streaming large files.

Installing Python Requests

Before using the Requests library, it must be installed. You can install it using pip:

pip install requests

Once installed, you can import it in your Python scripts:

import requests

Making HTTP Requests

GET Requests

The GET method is used to request data from a server. Requests makes it simple to send GET requests:

import requests

url = "https://jsonplaceholder.typicode.com/posts/1"
response = requests.get(url)

print("Status Code:", response.status_code)
print("Response Body:", response.text)

Here, response.status_code gives the HTTP status code, while response.text returns the response body as a string.

POST Requests

The POST method is used to send data to the server. Requests allow sending form data or JSON payloads easily:

import requests

url = "https://jsonplaceholder.typicode.com/posts"
data = {
    "title": "Python Requests",
    "body": "Python Requests library is very powerful.",
    "userId": 1
}

response = requests.post(url, json=data)
print("Status Code:", response.status_code)
print("Response JSON:", response.json())

PUT Requests

The PUT method is used to update existing resources:

import requests

url = "https://jsonplaceholder.typicode.com/posts/1"
data = {
    "id": 1,
    "title": "Updated Title",
    "body": "Updated body content",
    "userId": 1
}

response = requests.put(url, json=data)
print("Status Code:", response.status_code)
print("Updated Data:", response.json())

DELETE Requests

DELETE requests remove resources from the server:

import requests

url = "https://jsonplaceholder.typicode.com/posts/1"
response = requests.delete(url)

print("Status Code:", response.status_code)
print("Response Text:", response.text)

Handling Query Parameters

You can pass query parameters to GET requests using the params argument:

import requests

url = "https://jsonplaceholder.typicode.com/posts"
params = {"userId": 1}

response = requests.get(url, params=params)
print("Response JSON:", response.json())

Working with Headers

Custom headers can be sent in requests using the headers parameter. This is useful for authentication, content type, and user-agent specifications:

import requests

url = "https://jsonplaceholder.typicode.com/posts"
headers = {"User-Agent": "MyApp/1.0"}

response = requests.get(url, headers=headers)
print("Response Status Code:", response.status_code)
print("Headers:", response.headers)

Handling Cookies

Requests can handle cookies automatically or manually:

import requests

url = "https://httpbin.org/cookies"
cookies = {"session_id": "123456"}

response = requests.get(url, cookies=cookies)
print("Response JSON:", response.json())

Using Sessions

Sessions allow you to persist cookies and certain parameters across multiple requests:

import requests

session = requests.Session()
session.headers.update({"User-Agent": "MyApp/1.0"})

# First request
response1 = session.get("https://httpbin.org/cookies/set/session_id/123456")
print("First Response:", response1.text)

# Second request retains cookies
response2 = session.get("https://httpbin.org/cookies")
print("Second Response:", response2.json())

Timeouts and Error Handling

Requests support specifying timeouts to prevent hanging requests:

import requests

url = "https://jsonplaceholder.typicode.com/posts"

try:
    response = requests.get(url, timeout=5)
    response.raise_for_status()  # Raise exception for HTTP errors
    print("Response JSON:", response.json())
except requests.Timeout:
    print("Request timed out")
except requests.HTTPError as err:
    print("HTTP error occurred:", err)
except requests.RequestException as err:
    print("Other error occurred:", err)

Downloading Files

Requests can be used to download files from the internet efficiently:

import requests

url = "https://www.example.com/sample.pdf"
response = requests.get(url, stream=True)

with open("sample.pdf", "wb") as f:
    for chunk in response.iter_content(chunk_size=1024):
        if chunk:
            f.write(chunk)

Uploading Files

Files can be uploaded using the files parameter:

import requests

url = "https://httpbin.org/post"
files = {"file": open("sample.txt", "rb")}

response = requests.post(url, files=files)
print("Response JSON:", response.json())

Handling JSON Responses

Most APIs return data in JSON format. Requests makes it easy to parse JSON:

import requests

url = "https://jsonplaceholder.typicode.com/posts/1"
response = requests.get(url)

# Convert JSON response to Python dictionary
data = response.json()
print("Title:", data["title"])
print("Body:", data["body"])

Advanced Usage

Authentication

Requests support various authentication methods including Basic Auth, OAuth, and Token Auth:

import requests
from requests.auth import HTTPBasicAuth

url = "https://httpbin.org/basic-auth/user/pass"
response = requests.get(url, auth=HTTPBasicAuth("user", "pass"))
print("Status Code:", response.status_code)
print("Response JSON:", response.json())

Proxies

You can send requests through a proxy:

import requests

url = "https://jsonplaceholder.typicode.com/posts"
proxies = {
    "http": "http://10.10.1.10:3128",
    "https": "https://10.10.1.10:1080",
}

response = requests.get(url, proxies=proxies)
print("Response Status Code:", response.status_code)

Custom SSL Verification

Requests allow you to disable SSL verification or provide custom certificates:

import requests

url = "https://self-signed.badssl.com/"
response = requests.get(url, verify=False)  # Not recommended for production
print("Response Status Code:", response.status_code)

 Using Python Requests

• Always handle exceptions like Timeout, HTTPError, and RequestException.
• Use sessions when making multiple requests to the same server to reuse TCP connections.
• Set timeouts to avoid hanging requests.
• Use streaming when downloading large files.
• Never disable SSL verification in production.
• Respect API rate limits to avoid being blocked.
• Use descriptive headers and user-agent strings for better server compatibility.

The Python Requests library is a powerful tool for making HTTP requests. Its simplicity and ease of use make it ideal for beginners and professionals alike. From sending GET and POST requests to handling sessions, cookies, headers, and JSON responses, Requests simplifies web interaction in Python. By following best practices, you can build robust, efficient, and secure applications that interact with web services and APIs seamlessly.