Hopefully someone more knowledgeable than me can assist us with our SiteMap / Google Crawling issue.
website: ez-robot.com
I tried submitting the sitemap to Google's Search Console and got a "couldn't fetch" error.
I then attempted to have GSC do a URL inspection of my home page and other pages on our site. Initially GSC said it could see those pages, so I requested indexing. This prompted GSC to do a test, and it said the URLs weren't accessible to Google.
Further details said:
Crawl allowed? error No: blocked by robots.txt
Page fetch error Failed: Blocked by robots.txt
Within our SEO tools (on Weebly), I have ensured that the "Hide from Search Engines was deactivated". I saved and repiublished the site. I checked the /robots.text for the site and am getting this, which seems standard, and like the website shouldn't be blocking GSC:
Sitemap: https://www.ez-robot.com/sitemap.xml
User-agent: NerdyBot
Disallow: /
User-agent: *
Disallow: /ajax/
Disallow: /apps/
Is there somewhere else that I need to make changes?
Also, the website had already been verified by GSC through the Domain Hosting Company (GoDaddy) so I don't see a place to try to get the HTML code to paste in, as an alternate form of verification... I don't think the verification is the issue though.
I have Google Analytics currently working on the site, if that's beneficial info.
Any help would be appreciated. We've recently ported our site to Weebly, and have changed a huge percentage of our URLs and content, so I'd love to get crawled by Google quickly, so we don't get lost.
Thanks,
