I'm encountering inconsistent search results between Google's Custom Search JSON API, the public CSE URL, and regular Google searches despite identical queries and configuration. Here's my setup:
CSE Configuration:
"Search the entire web" enabled
"All regions" selected
No site restrictions
Tested with VPN to rule out geo-location issues
Behavior Observed:
google.com search (browser): Results A
Public CSE URL (https://cse.google.com/cse?cx=[my-engine-id]): Matches Results A
Custom Search JSON API: Returns Results B (different from A)
Code Implementation (Python):
def google_search(query, num_results=10, news=False):
"""Perform Google search using Custom Search API"""
try:
logging.info(f"Starting Google search for query: '{query}'")
# Get random credentials
credential = get_credential()
api_key = credential["api_key"]
cx_id = credential["search_engine_id"]
logging.info(f"Starting Google search for query: '{query}' with API key: {api_key[:8]}...")
url = "https://www.googleapis.com/customsearch/v1"
params = {
'key': api_key,
'cx': cx_id,
'q': query,
'num': num_results
}
response = requests.get(url, params=params)
response.raise_for_status()
results = response.json()
if "items" not in results:
logging.warning(f"No results found in Google search for query: '{query}'")
return "No results found."
formatted_results = []
for i, item in enumerate(results["items"], start=1):
title = item.get('title', 'No title')
link = item.get('link', 'No URL')
snippet = item.get('snippet', 'No description available')
formatted_results.append(
f"{i}. Title: {title}\n URL: {link}\n Description: {snippet}"
)
logging.info(f"Google search successful for query: '{query}'. Found {len(results['items'])} results.")
return "\n\n".join(formatted_results)
Troubleshooting Attempts:
Verified CSE config has "Search entire web" enabled
Compared results using public CSE URL (matches regular Google)
Added gl, hl, lr parameters to match my location
Set filter=0 and safe="off"
Tested with/without VPN
Different result sets persist across various queries
Why does the JSON API return different results than the public CSE interface (which matches regular Google) when both use the same CSE configuration? How can I make the API return results consistent with regular Google searches?
