Indexing Tests: X robots noindex link to indexable page
This page has an x robots noindex tag (http header)
Will it crawl the following page which is only linked to from this page?
This test was inspired by this tweet
Check Google index for this page
| Header | Value |
|---|---|
| IP Address or XFF | 3.238.70.175 |
| Host | seotest.dev |
| User-agent | CCBot/2.0 (https://commoncrawl.org/faq/) |
| Accept-language | en-US,en;q=0.5 |
| O/S | |
| Device Screen height | |
| Device screen width | |
| Viewport height | |
| Viewport width | |
| Cookies enabled | |
| Referer |