Scraping real estate data with Python to find opportunities
In this tutorial, we'll explore how to scrape real estate data from an API using Python's requests library. We'll also learn how to apply filters to retrieve potential bargain properties whose prices have recently dropped. Introduction When hunting for great real estate opportunities, one of the best indicators can be a recent price drop. Having a tool that quickly shows you only these properties can save you tons of time—and might help you scoop up a deal before everyone else notices! In this post, we’ll: Discuss the basics of using requests to interact with a real estate API. Learn how to filter results using query parameters—particularly focusing on price variation queries. Parse and display the returned data in a concise format. Requirements Python 3 installed A terminal or command-line prompt Basic familiarity with the Python requests library An API key (if required by the API) Step 1: Understanding the API The API we use might respond with data such as: Property ID Title or Address Price Location Historical price changes Other relevant information Key Query Parameters This API supports several query parameters that help us filter the results: Parameter Type Description includedDepartments[] array Filter by department(s). Example: departments/77 fromDate date Only retrieve properties listed (or updated) after this date. propertyTypes[] array Filter by property type. Example: 0 for apartments, 1 for houses, etc. transactionType string 0 for sale, 1 for rent, etc. withCoherentPrice bool Only retrieve properties whose price is coherent with the market. budgetMin number Minimum budget threshold. budgetMax number Maximum budget threshold. eventPriceVariationFromCreatedAt date Date from which an event of type price is created — inclusive. eventPriceVariationMin number Minimum percentage of price variation (negative or positive). We’ll especially focus on the eventPriceVariation parameters to find properties that have decreased in price. Step 2: Crafting the Request Below is a sample script using Python's requests library to query the endpoint. Adjust the parameters and headers as needed, especially if an X-API-KEY is required. import requests import json # 1. Define the endpoint URL url = "https://api.stream.estate/documents/properties" # 2. Create the parameters params = { 'includedDepartments[]': 'departments/77', 'fromDate': '2025-01-10', 'propertyTypes[]': '1', # 1 might represent 'apartment' 'transactionType': '0', # 0 might represent 'sale' 'withCoherentPrice': 'true', 'budgetMin': '100000', 'budgetMax': '500000', # Focusing on price variation 'eventPriceVariationFromCreatedAt': '2025-01-01', # since the beginning of the year 'eventPriceVariationMin': '10', # at least a 10% drop } # 3. Define headers with the API key headers = { 'Content-Type': 'application/json', 'X-API-KEY': '' } # 4. Make the GET request response = requests.get(url, headers=headers, params=params) # 5. Handle the response if response.status_code == 200: data = response.json() print(json.dumps(data, indent=2)) else: print(f"Request failed with status code {response.status_code}") Explanation of Important Parameters eventPriceVariationMin = '-10' This means you’re looking for at least a 10% price decrease. eventPriceVariationMax = '0' Setting this to 0 ensures you don’t include properties that have had any price increase or any variation above 0%. Essentially, you’re capturing negative or zero changes.
In this tutorial, we'll explore how to scrape real estate data from an API using Python's requests library. We'll also learn how to apply filters to retrieve potential bargain properties whose prices have recently dropped.
Introduction
When hunting for great real estate opportunities, one of the best indicators can be a recent price drop. Having a tool that quickly shows you only these properties can save you tons of time—and might help you scoop up a deal before everyone else notices!
In this post, we’ll:
- Discuss the basics of using
requeststo interact with a real estate API. - Learn how to filter results using query parameters—particularly focusing on price variation queries.
- Parse and display the returned data in a concise format.
Requirements
- Python 3 installed
- A terminal or command-line prompt
- Basic familiarity with the Python
requestslibrary - An API key (if required by the API)
Step 1: Understanding the API
The API we use might respond with data such as:
- Property ID
- Title or Address
- Price
- Location
- Historical price changes
- Other relevant information
Key Query Parameters
This API supports several query parameters that help us filter the results:
| Parameter | Type | Description |
|---|---|---|
| includedDepartments[] | array | Filter by department(s). Example: departments/77
|
| fromDate | date | Only retrieve properties listed (or updated) after this date. |
| propertyTypes[] | array | Filter by property type. Example: 0 for apartments, 1 for houses, etc. |
| transactionType | string |
0 for sale, 1 for rent, etc. |
| withCoherentPrice | bool | Only retrieve properties whose price is coherent with the market. |
| budgetMin | number | Minimum budget threshold. |
| budgetMax | number | Maximum budget threshold. |
| eventPriceVariationFromCreatedAt | date | Date from which an event of type price is created — inclusive. |
| eventPriceVariationMin | number | Minimum percentage of price variation (negative or positive). |
We’ll especially focus on the eventPriceVariation parameters to find properties that have decreased in price.
Step 2: Crafting the Request
Below is a sample script using Python's requests library to query the endpoint. Adjust the parameters and headers as needed, especially if an X-API-KEY is required.
import requests
import json
# 1. Define the endpoint URL
url = "https://api.stream.estate/documents/properties"
# 2. Create the parameters
params = {
'includedDepartments[]': 'departments/77',
'fromDate': '2025-01-10',
'propertyTypes[]': '1', # 1 might represent 'apartment'
'transactionType': '0', # 0 might represent 'sale'
'withCoherentPrice': 'true',
'budgetMin': '100000',
'budgetMax': '500000',
# Focusing on price variation
'eventPriceVariationFromCreatedAt': '2025-01-01', # since the beginning of the year
'eventPriceVariationMin': '10', # at least a 10% drop
}
# 3. Define headers with the API key
headers = {
'Content-Type': 'application/json',
'X-API-KEY': 'Explanation of Important Parameters
eventPriceVariationMin = '-10'
This means you’re looking for at least a 10% price decrease.
eventPriceVariationMax = '0'
Setting this to 0 ensures you don’t include properties that have had any price increase or any variation above 0%. Essentially, you’re capturing negative or zero changes.
