API key with search engine to avoid HTTP error 429: too many requests

I am automating a search for keywords on some websites with about 400 searches in total. The search is similar to keyword site:domain.com. I am using Google Search but could use another service.

I get urllib.error.HTTPError: HTTP Error 429: Too Many Requests after about 20 requests. I could change my IP with a VPN or wait for the timeout specified in the Retry-after header (as in this StackOverflow thread), but I prefer to request an API key.

In Google Cloud Platform, I found how to request an API key for Custom Search, but that is for indexing a website, rather than searching for keywords on several websites.

I tried some workarounds:

• using ImportXML() in Google Sheets, as in this page, but the cells display N/A

• a custom search that searches the whole web, as in this page, but that feature seems to have been disabled; I could include the manually the around 100 websites I am targeting, but that is tedious.

• calling AppleScript from Python to open the page in Safari, writing the results to disk, and parsing the HTML, but that is super clunky and prone to errors

Does Google or another search engine provide an API with a key to access their search results?