HTTP conditional requests are a neat trick to optimize web communication by letting the client ask the server, “Hey, has this resource changed since I last downloaded it?” If the resource is unchanged, the server replies with a 304 Not Modified status, saving bandwidth and improving load times.
Two common HTTP headers power this mechanism: If-Modified-Since and If-None-Match. The first one checks if the resource has been updated since a specific date/time, while the second compares an ETag (a unique identifier assigned by the server) against the client’s cached version.
Think of If-Modified-Since as a timestamp check. The client says, “Give me the resource only if it’s newer than this date.” If the resource hasn’t changed, the server responds with 304 Not Modified, and the client can keep using its cached copy.
If-None-Match works a bit differently. The server provides an ETag with each response, a sort of fingerprint for that version of the content. On subsequent requests, the client sends back this ETag in the If-None-Match header. If the resource is unchanged, the server returns 304 again.
Using conditional requests is a win-win: clients avoid unnecessary downloads, and servers reduce bandwidth and processing load. This is especially valuable for APIs serving large JSON payloads or static assets like images and scripts.
One nuance is that not all servers or proxies implement ETags consistently, and clock skew can mess with If-Modified-Since. For robust caching, many systems combine both headers, falling back gracefully if one fails.
Here’s a quick rundown of the request and response flow:
Client sends GET /resource with: If-Modified-Since: Wed, 21 Oct 2015 07:28:00 GMT If-None-Match: "686897696a7c876b7e" Server checks resource state: - If unchanged, respond with 304 Not Modified (no body) - If changed, respond with 200 OK and updated resource + new ETag and Last-Modified headers
When you build clients that respect these headers, you’re not just being polite; you’re making your apps faster and more scalable. It’s the difference between downloading megabytes repeatedly and fetching only what’s new.
Remember, conditional requests aren’t magic. If your server doesn’t send Last-Modified or ETag headers in its responses, the client can’t leverage them. So, check your server or API documentation before expecting this to work out of the box.
On the flip side, if you control the server, setting strong ETags or accurate Last-Modified timestamps is crucial. Weak or missing validators lead to cache misses, negating performance gains.
Also, beware of how your backend handles dynamic content. Sometimes, even a minor change in whitespace or metadata can change the ETag, causing unnecessary cache invalidations. Designing your ETag generation thoughtfully helps avoid this.
Conditional requests also play well with HTTP methods beyond GET. For example, when using PUT or DELETE, you can send an If-Match header with an ETag to ensure you only modify or delete a resource if it hasn’t changed since you last fetched it. This helps prevent lost updates in concurrent environments.
This mechanism is the backbone of HTTP caching strategies and RESTful API design. Skipping it means you’re throwing away one of the few built-in ways HTTP helps you optimize network usage without reinventing the wheel.
Next, let’s see how to harness this power in Python using the requests library, where sending and handling these headers becomes a matter of a few lines of code. It’s surprisingly straightforward—
Seagate Portable 1TB External Hard Drive HDD – USB 3.0 for PC, Mac, PlayStation, & Xbox, 1-Year Rescue Service (STGX1000400) , Black
9% OffImplementing conditional requests with the requests library
To implement conditional requests with the requests library, you simply need to add the appropriate headers to your request. If you have a cached version of a resource, grab its ETag or Last-Modified value from the previous response and include it in the headers.
Here’s a basic example that uses If-None-Match with an ETag you’ve stored from a prior request:
import requests
url = "https://api.example.com/data"
etag = '"686897696a7c876b7e"' # Previously saved ETag from server response
headers = {
"If-None-Match": etag
}
response = requests.get(url, headers=headers)
if response.status_code == 304:
print("Resource not modified. Use cached data.")
else:
print("Resource updated. Process new data.")
print(response.text)
# Save new ETag for next request
new_etag = response.headers.get("ETag")
if new_etag:
etag = new_etag
Similarly, if you prefer to use If-Modified-Since, you’d do something like this:
import requests
url = "https://api.example.com/data"
last_modified = "Wed, 21 Oct 2015 07:28:00 GMT" # Last-Modified from previous response
headers = {
"If-Modified-Since": last_modified
}
response = requests.get(url, headers=headers)
if response.status_code == 304:
print("Resource not modified since", last_modified)
else:
print("Resource updated. Process new data.")
print(response.text)
# Save new Last-Modified for next request
new_last_modified = response.headers.get("Last-Modified")
if new_last_modified:
last_modified = new_last_modified
Combining both headers is straightforward and often recommended to cover more cases:
import requests
url = "https://api.example.com/data"
etag = '"686897696a7c876b7e"'
last_modified = "Wed, 21 Oct 2015 07:28:00 GMT"
headers = {
"If-None-Match": etag,
"If-Modified-Since": last_modified
}
response = requests.get(url, headers=headers)
if response.status_code == 304:
print("Resource not modified. Use cached version.")
else:
print("Resource updated.")
print(response.text)
etag = response.headers.get("ETag", etag)
last_modified = response.headers.get("Last-Modified", last_modified)
Handling the 304 Not Modified response is the key part here. The server intentionally sends no body, so you shouldn’t try to parse response.text or response.json() when you get a 304. Instead, rely on your cached copy.
When you’re writing a client for an API or a static asset server that supports conditional requests, it’s a good idea to wrap this logic inside a function or a class that manages headers and caching transparently. For example:
import requests
class ConditionalRequester:
def __init__(self):
self.etag = None
self.last_modified = None
def get(self, url):
headers = {}
if self.etag:
headers["If-None-Match"] = self.etag
if self.last_modified:
headers["If-Modified-Since"] = self.last_modified
response = requests.get(url, headers=headers)
if response.status_code == 304:
print("Not modified, using cached data.")
return None # Signal to use cache
elif response.status_code == 200:
self.etag = response.headers.get("ETag", self.etag)
self.last_modified = response.headers.get("Last-Modified", self.last_modified)
return response.text
else:
response.raise_for_status()
# Usage:
client = ConditionalRequester()
data = client.get("https://api.example.com/data")
if data is not None:
print("Fetched new data:", data)
else:
print("Using cached data.")
That example encapsulates the conditional request pattern and updates headers automatically. You can extend this with persistent storage of etag and last_modified values to survive application restarts.
One subtlety to watch out for: some servers may return ETag values with weak validators (prefix W/), which indicate semantically equivalent but not byte-identical resources. The requests library doesn’t parse or interpret these—you just send whatever the server gave you back in the If-None-Match header verbatim.
Also, when dealing with APIs that paginate or filter data, conditional requests can sometimes be tricky. Make sure you use the right URL and headers for each distinct resource representation to avoid cache confusion.
Source: https://www.pythonlore.com/handling-http-conditional-requests-with-requests/
