A browser makes Google Search look like a page. For a program, the useful representation is a structured response that preserves the visible text, destination URLs, result order, and pagination state.

Reserp accepts a complete Google Search URL and returns its visible result blocks as JSON. The example below calls that HTTP endpoint using only Python's standard library.

Store the API key outside the script

Put the key in an environment variable rather than source code, notebooks, command history, or the Google URL:

export RESERP_API_KEY="your-api-key"

The key belongs in the Authorization header. Keeping it out of the submitted URL also prevents it from appearing in URL logs and saved query datasets.

Build the Google Search URL

Use urlencode instead of joining query strings manually. This handles spaces, punctuation, and non-English text correctly.

from urllib.parse import urlencode


def google_search_url(
    query: str,
    country: str = "us",
    language: str = "en",
    start: int = 0,
) -> str:
    if start < 0 or start % 10 != 0:
        raise ValueError("start must be a non-negative multiple of 10")

    parameters = {
        "q": query,
        "gl": country,
        "hl": language,
    }
    if start:
        parameters["start"] = start

    return "https://www.google.com/search?" + urlencode(parameters)

The q parameter contains the query. The gl and hl parameters make the country context and interface language explicit. They influence the search context but should not be treated as strict country or content-language filters.

Send the HTTP request

The request body contains one field: the complete Google Search URL.

import json
import os
from urllib.error import HTTPError
from urllib.request import Request, urlopen

API_URL = "https://api.reserp.ai/v1/serp"
API_KEY = os.environ["RESERP_API_KEY"]


def fetch_search(google_url: str) -> dict:
    request = Request(
        API_URL,
        data=json.dumps({"url": google_url}).encode("utf-8"),
        headers={
            "Authorization": f"Bearer {API_KEY}",
            "Content-Type": "application/json",
        },
        method="POST",
    )

    try:
        with urlopen(request, timeout=30) as response:
            payload = json.load(response)
    except HTTPError as error:
        raw_body = error.read().decode("utf-8", errors="replace")
        try:
            problem = json.loads(raw_body)
        except json.JSONDecodeError:
            problem = {"error": f"http_{error.code}", "retryable": False}

        raise RuntimeError(
            f"Search failed: {problem.get('error', 'unknown_error')}"
        ) from error

    if payload.get("ok") is not True:
        raise RuntimeError("Search returned an unusable response")

    return payload


page = fetch_search(
    google_search_url(
        "renewable energy storage research",
        country="gb",
        language="en",
    )
)

A successful response includes:

  • ok: whether the operation completed successfully
  • url: the normalized submitted search URL
  • finalUrl: the URL after Google redirects
  • results: visible result blocks in their returned order
  • pagination: the current and next Google offsets
  • billed: whether billing settled for the request

Read result blocks without losing nested links

Each result can contain visible text, a destination url, and nested children. Some visible modules and sitelinks contain useful URLs below the top level, so iterate recursively when you need every destination.

def walk_results(results):
    for result in results:
        yield result
        yield from walk_results(result.get("children", []))


for result in walk_results(page["results"]):
    url = result.get("url")
    text = result.get("text", "")
    if url:
        print(url)
        print(text[:160].replace("\n", " "))
        print()

Do not assume that every block has both fields. A text-only block can be meaningful, while a parent block may organize child destinations.

Preserve the raw response before transforming it

If the results will become research data, save the original payload before filtering, flattening, or deduplicating it:

from datetime import datetime, timezone

snapshot = {
    "collectedAt": datetime.now(timezone.utc).isoformat(),
    "query": "renewable energy storage research",
    "country": "gb",
    "language": "en",
    "response": page,
}

with open("search-snapshot.json", "w", encoding="utf-8") as output:
    json.dump(snapshot, output, ensure_ascii=False, indent=2)

Keeping the raw tree makes later decisions reversible. You can always derive a flat URL table from the tree; you cannot reconstruct parent-child relationships after discarding them.

Fetch the next page

Use the supplied pagination.nextUrl:

second_page = fetch_search(page["pagination"]["nextUrl"])

Do not calculate the next offset from len(page["results"]). A page can contain ordinary listings, rich modules, and nested links while Google's pagination still advances in organic offsets of ten.

Handle failures deliberately

Documented error responses contain a stable error code together with retryable and billed. An invalid request should be corrected rather than repeated. A transient response should be retried only within a bounded policy, normally with exponential backoff and jitter.

At minimum, production code should:

  • Use a finite timeout
  • Check the HTTP status and ok field
  • Parse error bodies without logging credentials or complete result payloads
  • Retry only when the response says the failure is retryable
  • Limit both attempts and total elapsed time
  • Preserve valid empty result arrays separately from failed requests

That is enough to move from an interactive search to a usable JSON collection without introducing a browser or an additional client library. The Google Search API documentation contains the current request and response schema.