Make this site properly indexable by search engines and readable by AI answer engines (ChatGPT, Claude, Perplexity, Google AI Overviews). Work through the steps in order. **Do not skip step 0** — most of the value is in what it finds, and adding metadata to a site whose content is invisible achieves nothing. Adapt everything to this repo's actual stack. If it is a framework, use that framework's metadata and routing conventions rather than hand-writing tags. ## Step 0 — audit first, and show me the numbers Before changing anything, measure. Report a table of findings. The single most important check: **load each page with JavaScript disabled and count the words and key elements that survive.** Use a headless browser (`javaScriptEnabled: false` in Playwright) rather than guessing. ```js const ctx = await browser.newContext({ javaScriptEnabled: false }); const page = await ctx.newPage(); await page.goto(url); // textContent, not innerText — innerText omits collapsed
and // anything not currently rendered, which crawlers do still read. const words = await page.evaluate(() => { const c = document.body.cloneNode(true); c.querySelectorAll('script,style').forEach(n => n.remove()); return c.textContent.replace(/\s+/g, ' ').trim().split(/\s+/).length; }); ``` Anything built at runtime from a JS array, fetched client-side, or rendered only after hydration is **invisible to most AI crawlers**, which do not execute JavaScript. Google renders JS; GPTBot, ClaudeBot and PerplexityBot largely do not. Also report, per page: - `` and its length, meta description, canonical - Open Graph and Twitter tags — present at all? absolute URLs? - JSON-LD blocks and whether each parses - `h1` count and text; heading hierarchy - Is the **phone number, email or address present as readable text**, or only inside `href="tel:"`, `mailto:` or a WhatsApp link? An answer engine asked "how do I contact X" can only quote text it can read. - Word count with and without JavaScript - Internal link count - Whether `robots.txt`, `sitemap.xml` and `llms.txt` exist - How many distinct URLs are actually indexable ## Step 1 — get the content into the HTML Fix whatever step 0 found. This matters more than every other step combined. If lists, catalogues, cards or galleries are built client-side, **render them into the HTML** and let the script replace them with identical markup on load. Keep the two templates byte-identical or the page visibly flickers. If items are reachable only through `#fragments` or a modal, they share one URL. Search engines cannot rank a fragment. Make each item a real link to a real page — see step 7. ## Step 2 — per-page metadata Every page gets its own, never a site-wide copy: - `<title>` under ~60 characters, describing the page, front-loading the term a buyer would actually search. "Full Collection" tells nobody what is collected. - `<meta name="description">`, 140–160 characters - `<link rel="canonical">` with the absolute URL - Open Graph: `og:type`, `og:site_name`, `og:title`, `og:description`, `og:url`, `og:image`, `og:image:width`, `og:image:height`, `og:locale` - `twitter:card = summary_large_image`, plus title, description, image Three things decide whether previews actually work: 1. **Absolute URLs for `og:image`.** Relative paths are the usual reason a preview silently fails — scrapers do not resolve them. 2. **JPEG or PNG for the share image, 1200×630**, even if the rest of the site is WebP. WhatsApp and several older scrapers handle WebP previews inconsistently. 3. **Declare width and height**, so the preview renders full-size immediately. ## Step 3 — structured data (JSON-LD) Use a single `@graph` per page so entities can cross-reference by `@id`. - Home: `Organization` or `LocalBusiness` (name, address, telephone, logo, image, `areaServed`), plus `WebSite` - Any listing page: `CollectionPage` + `ItemList` - Any item page: `Product` (or `Service`, `Article`, `Recipe` — whatever fits) - Anywhere with questions and answers: `FAQPage` - Any embedded video: `VideoObject` - Deeper than one level: `BreadcrumbList` Validate that every block parses as JSON before committing. Afterwards, check the live URLs against Google's Rich Results Test — syntactic validity is not the same as Google accepting it. ## Step 4 — crawler files **`robots.txt`** — allow everything, and name the AI crawlers explicitly, since several are blocked by default at CDN or platform level: `GPTBot`, `OAI-SearchBot`, `ChatGPT-User`, `ClaudeBot`, `Claude-SearchBot`, `anthropic-ai`, `PerplexityBot`, `Perplexity-User`, `Google-Extended`, `Applebot-Extended`, `meta-externalagent`, `Amazonbot`, `Bytespider`, `CCBot`. End with the absolute `Sitemap:` line. **`sitemap.xml`** — every indexable URL. The namespace is `http://www.sitemaps.org/schemas/sitemap/0.9`. **`sitemap.org` without the `s` is a typo that makes search engines reject the file**, and nothing warns you. Parse the file and assert the namespace after writing it. **`llms.txt`** at the site root — a plain-Markdown summary for answer engines: what the organisation does, where it is, how to contact it, what each page covers, and any disambiguation ("X, Y and Z all refer to the same company"). Keep it factual; this is the file an LLM is most likely to quote verbatim. ## Step 5 — contact details as readable text Put the phone number, address and any email in visible text, with `tel:` and `mailto:` links, in the contact area and the footer. Also in the `LocalBusiness` schema. A number that exists only inside a WhatsApp URL cannot be quoted by anything. ## Step 6 — an FAQ Q&A is the shape answer engines quote: a page carrying the exact question someone asks is what gets cited. Write 6–10 real buyer questions with substantive answers, and emit `FAQPage` schema. **Generate the visible section and the schema from one source** so they cannot drift apart. Use `<details>`/`<summary>` if you want it collapsible — that content is in the DOM and is indexed. ## Step 7 — one URL per thing worth ranking If the repo has a catalogue, portfolio, team, or locations list where each item has real content, give each item its own page, generated from whatever data already exists. Each needs its own title, description, canonical, OG tags, schema, and internal links to siblings. Make the listing cards real `<a href>` elements. If a modal is wanted, have JS call `preventDefault()` on a plain left click and let modified clicks through, so ctrl/cmd-click still opens the page. Without JavaScript the listing must be a set of real links. Add every new URL to the sitemap. ## Rules **Never invent business facts.** Price, minimum order, lead time, founding date, certifications, geo coordinates, headcount, client names. If it is not already published on the site or given to you, ask me. Wrong facts in structured data propagate into answer engines and are very hard to retract. Where a fact is missing, leave the field out and list it for me at the end. **Verify, do not assert.** Every claim you make about the result should be something you measured. Re-run the step 0 audit at the end and show the before/after table. **Do not regress the visitor experience.** Everything above should be invisible to a human using the site normally. ## Traps that cost time - `innerText` omits collapsed `<details>` and anything not rendered. Use `textContent` when modelling what a crawler reads. - Sitemap namespace: `sitemaps.org`, not `sitemap.org`. - Reusing an existing DOM `id` fails silently — `getElementById` returns the first match and your assignment goes nowhere. Check for duplicate ids. - Relative `og:image` URLs silently produce no preview. - WebP share images fail on some scrapers, WhatsApp included. - Over `file://`, font preloads log CORS errors that do not happen over HTTPS. Serve with a real HTTP server before believing them. - In Playwright, `response.body()` silently fails for `file://` responses, so byte counters undercount badly. Resolve requests to files and stat them. ## Report back 1. The before/after audit table 2. Every file added or changed, and why 3. Facts you need from me before anything else can be asserted 4. What to do next off-site: verify in Google Search Console, submit the sitemap, import into Bing Webmaster Tools, and — if the business has a physical location — set up a Google Business Profile, which for a local business usually outperforms everything on this list