Quick answer
A 500 error (HTTP 500 Internal Server Error) is the status code a server returns when an unexpected condition on its side stopped it from completing the request. It says the server failed but not why, so the cause is found in server logs, not in the browser.
Formula: 5xx share = crawler fetches with a 5xx status ÷ all crawler fetches
What is a 500 error?
A 500 error (HTTP 500 Internal Server Error) is the status code a web server returns when something went wrong on its side and it could not complete the request. RFC 9110, the HTTP standard, defines it as the server having "encountered an unexpected condition that prevented it from fulfilling the request". MDN describes it as a generic catch-all, used when no more specific 5xx code fits.
Put plainly, a 500 is the server saying "I made a mistake, but I am not telling you what it was." The browser, the crawler and the person looking at the error page all get the same three digits and nothing else of use. The real reason (an uncaught exception, a failed database query, a broken configuration line) is written only in the server's own logs, if it is written anywhere. That is why troubleshooting a 500 starts on the server, not in the browser.
The same logic applies to search. Whether a 500 hurts your rankings depends on how long it lasts: Google's crawlers slow down and retry when they meet server errors, so a short burst usually leaves no trace, while URLs that keep returning errors are eventually dropped from the index.
The 5xx class as a whole means the server knows it has erred or cannot perform the request. That distinguishes a 500 from 4xx codes such as a 404, where the request itself was the problem (a page that does not exist). Clearing the browser cache or trying another browser rarely fixes a true 500, because nothing on the visitor's side caused it.
What a 500 error looks like
Servers, frameworks and hosting platforms phrase the same status in many ways. All of these are the same code:
- "500 Internal Server Error" or "HTTP Error 500", the default pages of most web servers.
- "Internal Server Error. The server encountered an internal error or misconfiguration and was unable to complete your request."
- A blank white page with status 500 in the browser's developer tools, common when error display is turned off.
- "Request failed with status code 500", the message HTTP client libraries such as Axios produce when an API call gets a 500 back. The bug is still on the server that answered.
- A branded error page from a CDN or proxy, which may show the origin's 500 or a 502 if the origin did not answer properly.
500 vs 502, 503 and 504
| Code | Name | What it means (RFC 9110) | Typical cause | Is a retry likely to work? |
|---|---|---|---|---|
| 500 | Internal Server Error | Unexpected condition prevented fulfilling the request | Code exception, bad configuration, failed query | Only if the cause was transient |
| 501 | Not Implemented | Server does not support the functionality required | Unsupported request method | No |
| 502 | Bad Gateway | A gateway or proxy got an invalid response from the server behind it | App server crashed or returned garbage | Often, once the app server recovers |
| 503 | Service Unavailable | Temporarily unable to handle the request due to overload or maintenance | Planned maintenance, traffic spike, worker pool full | Yes, after a delay; may send Retry-After |
| 504 | Gateway Timeout | A gateway or proxy did not get a timely response from upstream | Slow query, upstream API hanging | Sometimes; the slow part needs fixing |
The practical difference: 502 and 504 point at the connection between a proxy and the application, 503 says "come back later" on purpose, and 500 says the application itself failed. For planned downtime, a 503 with a Retry-After header is the correct signal; returning 500 during maintenance tells clients and crawlers something broke.
What causes a 500 error
Because 500 is a catch-all, the list of causes is long, but most fall into five groups:
- Unhandled exceptions in application code. A null value, a type error, a missing file. Many frameworks convert any uncaught exception into a 500 response.
- A recent deployment or configuration change. New code, an updated dependency, a changed environment variable, a syntax error in a server configuration or rewrite file.
- A failing dependency. The database is down or out of connections, an external API returns errors, a cache server is unreachable.
- Resource limits. Memory exhausted, disk full, a process killed for exceeding its time limit.
- Permissions. The server process cannot read a file or write to a directory it needs.
Groups 1 and 2 cover most real incidents. If a 500 appears minutes after a deploy, the deploy is the first suspect.
How to troubleshoot a 500 error
The order below works for any stack, because it starts from evidence instead of guesses.
- Note the exact URL, time and request. Which page or API endpoint, at what time (with timezone), with what method and parameters. Is it every request, or only some users or some inputs?
- Read the server's error log at that timestamp. The application log or the web server's error log holds the stack trace or error message the 500 page hides. This step usually answers the question outright.
- Check what changed. Compare the time the errors began with the last deployment, configuration change or dependency update. If they line up, roll back first and investigate after.
- Check dependencies and resources. Database connections, external API status, memory, disk space and process limits around the time of the error.
- Check where the 500 comes from. Behind a CDN or load balancer, confirm whether the origin returned 500 or whether the proxy produced the error. The access log on the origin shows what the application actually sent.
- Reproduce and fix, then watch the error rate. Confirm the fix by seeing 5xx counts return to their usual level in the access logs, not by loading the page once.
If you are a visitor rather than the site owner, the honest advice is short: reload once, try again later, and contact the site if it persists. Nothing on your device caused it.
How 500 errors affect SEO
Google's documentation on HTTP status codes describes the behaviour for 500, 502 and 503 in three steps:
- Crawling slows down. 5xx and 429 responses make Google's crawlers temporarily reduce the crawl rate for the site, in proportion to the number of URLs returning server errors.
- Indexed pages are kept for a while. Already indexed URLs are preserved, but URLs that persistently return a server error are eventually dropped from the index. Google does not state a fixed number of days.
- Recovery is gradual. Once the server returns 2xx again, the crawl rate increases gradually.
One URL is special: robots.txt. If it returns a 5xx, Google's robots.txt specification says Google stops crawling the whole site for the first 12 hours while it keeps retrying the file, then uses the last good copy for up to 30 days. A broken robots.txt therefore pauses crawling of every page, not just one.
The rule that follows is about duration. A few minutes of errors during a deploy are retried and forgotten. Errors that repeat day after day on the same URLs, or a site that is down for several days, lose pages from the index and slow the crawl of everything else, which also wastes crawl budget. A page that shows an error message but returns status 200 is a different problem, a soft 404, and is not handled as a server error at all. Other crawl problems are covered in crawl errors and, for missing pages, how to fix 404 errors.
Worked example: one week of crawler fetches
Sample data: a site's access logs show 4,200 fetches by Googlebot in one week.
| Status | Fetches | Share of all fetches |
|---|---|---|
| Success (status below 400) | 3,990 | 3,990 ÷ 4,200 = 95.0% |
| 404 | 84 | 84 ÷ 4,200 = 2.0% |
| 5xx | 126 | 126 ÷ 4,200 = 3.0% |
| Total | 4,200 | 100% |
A 3.0% 5xx share looks alarming on its own. The decision depends on where and when the 126 errors happened:
- By time: 104 of the 126 (82.5%) fell on Tuesday between 02:00 and 03:00, the hour of a deployment. The other 22 are spread over six days.
- By URL: 98 of the 126 (77.8%) were on internal search result URLs (
/search?q=…), which return 500 for some special characters in the query. The remaining 28 hit 20 different product pages, all inside the Tuesday window.
So there are two different problems. The Tuesday burst lasted an hour and the same product URLs returned 200 afterwards; Google retries those, and no action is needed beyond preventing the next one with a 503 and Retry-After during deploys. The search URLs fail every day, on every fetch with the problem characters. Those are persistent errors on low-value URLs: fix the input handling so the page returns a proper response, and consider keeping crawlers out of internal search pages altogether. Without the split by time and URL, the team would have treated a one-hour incident and a permanent bug as the same 3.0%.
Common mistakes
- Debugging in the browser. The browser only sees the status code. The cause is in the server logs.
- Returning 500 for maintenance. Use 503 with Retry-After so clients and crawlers know it is temporary.
- Showing an error page with status 200. Crawlers index the error text as content and visitors' tools cannot detect the failure.
- Judging by one page load. A 500 that hits 1% of requests passes a manual check. Count 5xx in the access logs.
- Treating all 5xx as one number. Split by time and URL, as in the example, before deciding how urgent it is.
- Forgetting robots.txt. A 5xx on that one file pauses crawling of the whole site for Google.
Spotting 500s in TapCub
The question behind the example is how you find out that crawlers met a 500 in the first place. In TapCub that answer depends on server-side records, because the page script runs in the browser and cannot see a status code.
- Import your access logs. Server events & log import accepts Nginx or Apache combined-format log lines (up to 5,000 lines per call); the request records it keeps are the ones classified as bots, which is how crawlers that never run JavaScript become visible. See import server logs. Without this step, crawl success looks close to 100% because script records carry no status code.
- Read crawl health by engine. Monitoring › Crawl health shows total fetches, success rate (status below 400), distinct pages, 404s and 5xx, with the trend split by search engine. That is the table in the worked example; open an engine's row for the pages it fetched most. See check crawl health.
- Check the status code per crawler. Web analytics › Bots, on the All bots tab, lists each crawler's requests, pages visited and most common status code, and search engines have a small health table with fetches, success rate, 404s and 5xx. See bots and AI crawlers.
- Get warned when crawling drops. Turn on the crawl anomaly notification to be alerted when one engine's fetches in 24 hours fall below 30% of its 7-day daily average, which is the slowdown Google describes after persistent server errors.
TapCub does not tell you which human visitors received a 500: imported log lines keep only bot requests, and Monitoring › Errors collects front-end errors from the page script, not server status codes. The cause of a 500 is still in your own server logs.
See also: What is a JavaScript error? Types, impact and how to triage them.
Sources
- IETF, RFC 9110: HTTP Semantics, section 15.6.1 500 Internal Server Error.
- MDN Web Docs, 500 Internal Server Error.
- Google for Developers, How HTTP status codes affect Google's crawlers.
- Google for Developers, How Google interprets the robots.txt specification.
Was this explanation clear?
We use your answers to fix and expand this page.
Couldn’t send. Please try again in a minute.
Thanks for the feedback. We’ll use it to improve this page.You’ve already sent feedback on this page. Thanks.

Data Engineer
Writes about event design, data quality, and collection workflows, with verifiable implementation methods for reliable analytics.