Meta Tags, Sitemaps and Robots.txt: A Beginner's Checklist
The basic technical pieces every small website needs, explained without jargon.
You can write the best page in the world and still lose out because the basics are missing. Here is a short checklist for a new site, with what each piece does.
A good title and description
The title is what shows as the blue link in search results. Keep it under about 60 characters and put what the page is about near the front. The meta description is the grey text underneath. Search engines may rewrite it, but a clear 120 to 155 character summary often gets used. Use the Meta Tag Generator to draft both.
Open Graph tags for sharing
When someone pastes your link into WhatsApp or LinkedIn, these tags decide the title, text and picture of the preview. Without them, the preview can look empty. The Open Graph generator writes them for you. Use an image around 1200 by 630 pixels.
A sitemap
A sitemap is a list of the URLs you want crawled. It does not force a search engine to index them, but it helps it find them. Put it at /sitemap.xml and submit it in Google Search Console. The Sitemap Generator builds one from a list of URLs.
Robots.txt
This tiny text file at the root of your site tells crawlers which areas to stay out of, such as admin pages or internal search. It is a polite request, not a lock, so never use it to hide private content. Build one with the Robots.txt Generator and include a line pointing to your sitemap.
Do not skip the boring part
- Each page needs its own unique title and description.
- Pick one canonical version of every URL and stick to it.
- Make sure the site loads quickly on a phone.
- Write for people first. Markup supports good content, it does not replace it.