Here is how to make a webpage request in Python.


# Import the requests library
import requests
  
# Visit a page with Python
response = requests.get('https://example.com')

# Get the content of visited page
content = response.content
print(content) 

Output

The output is, of course, the source code of the webpage. In our case, the printed source code would look something like this:

b'<!doctype html>\n<html>\n<head>\n    <title>Example Domain</title>\n\n    <meta charset="utf-8" />\n    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />\n    <meta name="viewport" content="width=device-width, initial-scale=1" />\n    <style type="text/css">\n    body {\n        background-color: #f0f0f2;\n        margin: 0;\n        padding: 0;\n        font-family: -apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;\n        \n    }\n    div {\n        width: 600px;\n        margin: 5em auto;\n        padding: 2em;\n        background-color: #fdfdff;\n        border-radius: 0.5em;\n        box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n    }\n    a:link, a:visited {\n        color: #38488f;\n        text-decoration: none;\n    }\n    @media (max-width: 700px) {\n        div {\n            margin: 0 auto;\n            width: auto;\n        }\n    }\n    </style>    \n</head>\n\n<body>\n<div>\n    <h1>Example Domain</h1>\n    <p>This domain is for use in illustrative examples in documents. You may use this\n    domain in literature without prior coordination or asking for permission.</p>\n    <p><a href="https://www.iana.org/domains/example">More information...</a></p>\n</div>\n</body>\n</html>\n'


Explanation

Note: You need to install the Python requests library with pip install requests

The Python code above makes an HTTP request to the given URL just like browsers do. The server that hosts the webpage returns a response object which contains the content of that webpage. Python will extract the content of the webpage using the content property. The content is actually the webpage source code and inside that source code there are the data which you may want to scrape. To scrape the data you need to add additional code that extracts the data from the source code. A good library to scrape data with Python is BeautifulSoup.



Related HowTos
Create matplotlib graphs
Rename a file
Get today's date and convert to string
Create a text file and write text on it
Read a text file with Python
Scrape a Wikipedia page
Install dependencies
Flush the output of the print function
Prettyprint a JSON file
Create a dictionary with list comprehension
Select multiple columns in a Pandas dataframe
Profile a script
Reverse a string
Convert two lists into a dictionary
Convert an integer into a string
Generate random strings with upper case letters and digits
Extract extension from filename
Print to stderr in Python
Generate random integers between 0 and 9
Use a pythonic way to create a long multi-line string
Measure elapsed time in Python
Install specific package versions with pip
Read from stdin (standard input)
Get the class name of an instance
Check if a string is empty
Pad zeroes to a string
Delete an element from a dictionary
Check if a string is a float
Count the occurrences of an item in a list
Remove an element from a list by index
Use static methods
Remove a trailing newline
Print literal curly-brace characters in a string and also use .format on it
Exit/deactivate a virtualenv
Determine the type of an object
Limit floats to two decimal points
Know if an object has an attribute
Select an item from a list randomly
Read a file line-by-line into a list
Call a function of a module by using its name
Get the number of elements in a list
Print without a newline or space
Sort a list of dictionaries by a value of the dictionary
Remove a key from dictionary
Rename column names with Pandas
Lowercase a string
Upgrade all Python packages with pip
Get a substring of a string
Get the last element of a list
Parse a string to a float or integer
Convert a string into datetime format
Access environment variable values
Print coloured text on the terminal
Find current directory of a file
Change the size of figures drawn with Matplotlib
Manually raise an exception
Delete a file or folder
Split a list into evenly sized parts
Select rows from a DataFrame based on column values with Pandas
Install pip on Windows
Check if a given key already exists in a dictionary
Iterate over rows in a DataFrame for Pandas
Make function decorators and chain them together
Pass a variable by reference
Make a time delay
Convert bytes to a string
Copy a file
Concatenate two lists
Add new keys to a dictionary
Catch multiple exceptions in a single 'except' block
Check if a list is empty
Get the current time
Sort a dictionary by value
Use global variables in a function
List all files of a directory
Iterate over dictionaries using for loops
Check if a string contains a substring
Find the index of an item in a list
Understand how slice notation works
Make a flat list out of a list of lists
Accesses the index in for loops
Safely create a nested directory
Merge two dictionaries in a single expression
Check whether a file exists without exceptions
Convert a dictionary into a list
Convert a list into a dictionary
Duplicate a file
Append text in a text file
Use for loops
Deploy a web app to Heroku
Schedule a Python script for execution at a specific time every day
Store Python passwords securely on Windows, Mac, and Linux
Do dictionary comprehension
Do list comprehension
Create a virtual environment
Create a new file
Merge two lists
Extract items from two different lists into one list
Check if a text file is empty
Randomly select an item from a list
Generate a random integer
Break a while loop
Create a pandas DataFrame from a dictionary
Create a pandas DataFrame from a list
Get the last item of a list
Delete a column from a pandas dataframe
Access a column of a pandas dataframe
Create a class
Get the first two characters of a string
Loop through two lists at the same time
Recommended Course

Python Mega Course: Learn Python in 60 Days, Build 20 Apps
Learn Python on Udemy completely in 60 days or less by building 20 real-world applications from web development to data science.