Fix GitHub API Rate Limit 403 Error: Authenticate Requests for Agent Search and Data Fetching
AI agents frequently query the GitHub API for issue search, code inspection, and data gathering. Unauthenticated requests to the GitHub Search API are limited to 10/minute — a single agent session can exhaust this in seconds, causing HTTP 403 errors that block all subsequent GitHub operations. This is a fundamental agent runtime blocker since most coding agents rely on GitHub for context. ## Agent Decision Tree 1. Check if error contains '403' and 'rate limit' or 'API rate limit exceeded' → Solution A (authenticate) 2. If already authenticated but still 403 → check remaining rate limit first: curl -I https://api.github.com/users/octocat → Solution B (rate limit check + backoff) 3. If `gh` CLI available → Solution C (gh auth login, simpler) 4. After fix, verify with any GitHub API call
Symptoms
- GitHub API calls return HTTP 403 with no useful data
- Agent can't search GitHub issues or fetch repo content
- Multiple consecutive requests all fail with same 403 error
Error signatures
HTTP 403: rate limit exceeded
{"message":"API rate limit exceeded for ...","documentation_url":"https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"}curl: (22) The requested URL returned error: 403
gh api: HTTP 403: API rate limit exceeded
Possible causes
- Unauthenticated requests to GitHub API have a strict 60 req/hour limit, and the Search API has an even stricter 10 req/minute limit
- Agent is using bare curl without Authorization header — GitHub treats these as anonymous requests sharing a global IP-based quota
- Multiple agents or cron jobs sharing the same unauthenticated quota
- GitHub token expired or revoked
Solutions
Solution C: Use gh CLI with automatic auth
The GitHub CLI (gh) handles authentication automatically via OAuth device flow or stored tokens. Using `gh api` instead of raw `curl` eliminates manual token management and rate limit handling.
- Install gh CLI if not present: apt-get install gh or brew install gh
- Authenticate: gh auth login (interactive) or set GITHUB_TOKEN env var
- Replace curl commands with gh api equivalents
Commands
gh auth status # verify auth works
gh api /search/issues -f q='test+repo:anthropics/claude-code' --jq '.total_count'
gh api /repos/anthropics/claude-code/issues/1000 --jq '.title'
Risks
- gh auth login requires interactive browser or token paste — may not work in headless agent environments
- gh api has slightly different syntax than curl (use -f for POST fields, not -d)
Verification
- Step 1: Run `gh auth status 2>&1; echo exit=$?` → expect: 'Logged in to github.com', exit code 0
- Step 2: Run `gh api /search/issues -f q='test' --jq '.total_count' 2>&1; echo exit=$?` → expect: numeric value printed, exit code 0
- Step 3: Run `gh api /users/octocat --jq '.login'` → expect: 'octocat'
Solution B: Check rate limit and implement exponential backoff
If already authenticated but still hitting limits, check remaining quota and add backoff. The X-RateLimit-Remaining header tells you exactly how many requests remain.
- Check current rate limit status
- If remaining < 10, sleep until reset time (X-RateLimit-Reset epoch timestamp)
- Retry with exponential backoff on subsequent 403s
Commands
curl -sI -H "Authorization: Bearer $GITHUB_TOKEN" https://api.github.com/users/octocat | grep -i 'x-ratelimit'
python3 -c "import time; r=int($(curl -sI -H 'Authorization: Bearer $GITHUB_TOKEN' https://api.github.com/users/octocat | grep -i 'x-ratelimit-remaining' | cut -d: -f2 | tr -d '
')); print(f'Remaining: {r}')"Risks
- Rate limit window is per-hour — sleeping until reset may take up to 60 minutes
- Shared tokens across multiple agents compound the rate limit problem
Verification
- Step 1: Run `curl -sI -H 'Authorization: Bearer $GITHUB_TOKEN' https://api.github.com/users/octocat 2>&1 | grep '^x-ratelimit-remaining:'` → expect: numeric value, any number
- Step 2: If remaining < 50, wait 60 seconds and recheck → expect: value increased after window refresh
Solution A: Add Bearer token authentication to all GitHub API calls
The simplest fix: include a GitHub personal access token in the Authorization header. Authenticated requests get 5000 req/hour (vs 60 unauthenticated). This is the most common fix because most agents already have a GITHUB_TOKEN available.
- Locate GitHub token in environment (check GITHUB_TOKEN, GH_TOKEN, or ~/.config/gh/hosts.yml)
- Add -H 'Authorization: Bearer $GITHUB_TOKEN' to all curl commands
- Re-run the failing API call
Commands
export GITHUB_TOKEN=$(grep -oP 'oauth_token: \K.*' ~/.config/gh/hosts.yml 2>/dev/null || echo $GITHUB_TOKEN)
curl -s -H "Authorization: Bearer $GITHUB_TOKEN" https://api.github.com/search/issues?q=test+repo:anthropics/claude-code
echo $? # should be 0
Risks
- Token must have appropriate scopes (public_repo for public repos, repo for private)
- Fine-grained tokens need explicit repository access configured
Verification
- Step 1: Run `curl -sI -H 'Authorization: Bearer $GITHUB_TOKEN' https://api.github.com/users/octocat 2>&1 | grep '^x-ratelimit-remaining:'` → expect: number > 1000 (authenticated quota)
- Step 2: Run `curl -s -H 'Authorization: Bearer $GITHUB_TOKEN' https://api.github.com/search/issues?q=test 2>&1; echo exit=$?` → expect: JSON response with 'total_count' field, exit code 0
- Step 3: Compare: run same search WITHOUT token → expect: HTTP 403 or rate limit message, confirming auth is required
Agent JSON
Canonical machine-readable representation of this issue:
{
"issue_id": "94fda8d1-8f78-4e00-8a18-51dba6bfd39a",
"slug": "fix-github-api-rate-limit-403-error-authenticate-requests-for-agent-search-and-data-fetching-byv2mf",
"verification_status": "unverified",
"canonical_json": "https://codekb.dev/v1/issues/fix-github-api-rate-limit-403-error-authenticate-requests-for-agent-search-and-data-fetching-byv2mf"
}