SEO and AEO best practices.
Every claim carries its source.
A working reference for the two surfaces that now decide whether anyone finds you. Search engines still rank pages. Answer engines quote passages. The rules below cover both, and every number on this page links to the primary document it came from.
Each heading is a question somebody actually asks. Each answer beneath it stands on its own, so lifting one paragraph out of this page loses nothing.
What are the numbers worth memorising?
Eight signals carry most of the technical weight on a page, and each one settles on a number rather than an opinion. The sections underneath explain where every figure comes from.
| Signal | The rule | The number |
|---|---|---|
| Title element | One per page, subject first, no shared prefix across a template | 70 characters rendered |
| Meta description | One per URL, written as a snippet candidate rather than an answer | 100 to 160 characters |
| Canonical | One absolute URL, matched by every link, sitemap row and JSON-LD url | 1 URL form, slash included |
| Heading ladder | One h1, no skipped levels, question-shaped h2 headings | 1 h1 per page |
| Open Graph | og:title, og:type, og:image and og:url on every page | 4 required properties |
| XML sitemap | Canonical, indexable, 200-only URLs with a lastmod you can defend | 50,000 URLs and 50MB per file |
| robots.txt | Separate groups for training crawlers and retrieval agents | 500 KiB parse limit |
| JSON-LD | One graph per page, one @id per entity, identifying fields inlined | 1 script tag |
How long should a page title be, and what belongs in it?
A page title should name the one thing the page is about, put the distinguishing words first, and stay short enough to survive truncation in a result list. Google publishes no character limit and says it builds the title link from the title element, on-page headings, and anchor text, rewriting the tag whenever it reads as generic, stuffed, or duplicated across a site. Pick a working ceiling and enforce it in the build. This site hard-fails its own build above 70 rendered characters, which leaves 53 for the page-specific half once a 17-character brand suffix is appended.
The failure mode is rarely a long title. It is a set of titles that all open with the same six words, because the first thirty characters are the only part a person scans and the part a retriever weights hardest. Front-load the noun that separates this page from every other page on the domain, then delete the pipe-delimited keyword tail.
What makes a meta description worth writing?
A meta description is not a ranking input. It is one candidate for the snippet, and Google picks between it and text lifted straight from the page depending on what the searcher typed, which is why a page with no description still gets a snippet. Write one per URL at between 100 and 160 characters, describe what the page decides rather than what it covers, and hold the answer itself back so the snippet earns a click instead of replacing one.
Google asks for descriptions that are unique per page and accurate about the content. A single description repeated across a template is worse than none at all, because it teaches the engine that the tag is boilerplate and it stops using the tag anywhere on the site.
Which URL should the canonical point at, and does the trailing slash matter?
The canonical should carry one absolute URL on the production origin, and every internal link, sitemap entry, og:url and JSON-LD url on that page should match it character for character. The trailing slash matters because /guide and /guide/ are two separate URLs to a crawler unless one redirects to the other. Choose one form, send the other to it with a 301, and never advertise a URL in a sitemap that answers with a redirect.
Google treats rel=canonical as a strong signal rather than an instruction, and it selects its own canonical whenever the signals on a site disagree. Disagreement is the thing to remove. A page that links to itself without the slash while its sitemap lists it with the slash is arguing against itself, and the engine settles that argument on its own terms.
How many H1 elements should a page have?
One. The HTML specification permits any number and Google tolerates several, so this is a discipline rule rather than a penalty rule, and the discipline pays off downstream. A single h1 that states the subject hands a screen reader, a summariser and a retrieval chunker the same unambiguous document title. Every heading below it should then descend one level at a time, because a jump from h2 to h4 breaks the outline that assistive technology and passage-level retrieval both walk.
Treat the heading ladder as the table of contents a model will actually use. A retriever splits a long page on heading boundaries, so an h2 phrased as a real question and followed by a self-contained answer is the unit that gets stored, scored and quoted. A heading that reads Overview produces a chunk no query can match.
Which Open Graph tags actually matter?
Four. The Open Graph protocol names og:title, og:type, og:image and og:url as required for every page, and everything past those four is optional. Add og:description because most platforms render it, and generate a 1200 by 630 pixel card per route rather than reusing one site-wide image, because a shared link carrying a generic card converts worse than one that names the page it opens.
og:url is the property teams forget and the one that matters most for consolidation, since the protocol defines it as the canonical URL that serves as the object's permanent identifier in the graph. Point it at the exact string the canonical tag carries, trailing slash included.
What does an honest XML sitemap look like?
It lists only canonical, indexable URLs that answer 200, it stays under 50,000 URLs and 50MB uncompressed per file, and its lastmod values are true. Google states plainly that it ignores priority and changefreq, and that it uses lastmod only when the value is consistently and verifiably accurate, for instance by comparing it against the actual last modification of the page.
A pipeline that stamps today's date on every URL at deploy time is worse than a sitemap carrying no lastmod at all, because one dishonest column teaches the crawler to distrust the whole file. Reserve lastmod for a real change to the main content, the structured data or the links, and keep the copyright year out of it.
What does robots.txt control, and what does it not?
robots.txt controls crawling and nothing else. A disallowed URL can still be indexed from links elsewhere on the web, which is why a noindex robots meta tag rather than a Disallow line is the tool for keeping a page out of the index. The file has to be UTF-8 at the root of the origin, Google enforces a 500 kibibyte size limit and ignores every rule past it, and RFC 9309 requires any conforming parser to read at least that same 500 kibibytes.
The snippet controls belong to the same family and deserve a deliberate setting. max-snippet, max-image-preview and nosnippet govern how much of a page an engine may display, and Google's own guidance points site owners at those controls rather than at robots.txt when they want to limit what shows up inside AI features on Search.
Which AI crawler tokens should robots.txt name?
Nine tokens cover the engines that matter today, and they fall into two groups that behave nothing alike. Training crawlers such as GPTBot, ClaudeBot, Google-Extended and CCBot collect content that may feed model training, and disallowing them costs a site nothing on any live answer surface. Retrieval agents such as OAI-SearchBot, PerplexityBot, Claude-User, Perplexity-User and ChatGPT-User fetch a page to answer a question someone is asking right now, and disallowing those removes the site from the answer itself.
OpenAI states the split outright, noting that a site can allow OAI-SearchBot to appear in search results while disallowing GPTBot to stay out of foundation model training, and that each setting works independently of the others. Google describes Google-Extended as a standalone control with no effect on Google Search. Perplexity and Anthropic both document a crawler and a user-triggered agent under separate tokens for the same reason. One caveat sits inside that split. Perplexity states that Perplexity-User generally ignores robots.txt rules because a person asked for the fetch, so a disallow line there is a request rather than a gate.
| robots.txt token | Operator | What it does | What blocking costs |
|---|---|---|---|
| Google-Extended | Gemini training and grounding, controlled separately from Search | No effect on Google Search | |
| GPTBot | OpenAI | Crawls content for foundation model training | Training exclusion only |
| OAI-SearchBot | OpenAI | Builds the index behind ChatGPT search results | Removes you from ChatGPT search |
| ChatGPT-User | OpenAI | Fetches a page when a person asks ChatGPT about it | Blocks live answers about your site |
| PerplexityBot | Perplexity | Surfaces and links sites in Perplexity search results | Removes you from Perplexity results |
| Perplexity-User | Perplexity | Visits a page to answer a question a person just asked | Little. Perplexity documents that this fetcher generally ignores robots.txt |
| ClaudeBot | Anthropic | Collects web content that may contribute to model training | Training exclusion only |
| Claude-User | Anthropic | Retrieves a page in response to a Claude user's question | Blocks live answers about your site |
| CCBot | Common Crawl | Builds the open corpus most public models train on | Training exclusion only |
How should a JSON-LD entity graph be structured?
One script tag per page, one @graph inside it, and one @id per real-world entity that every other node references. The point of the graph is identity rather than decoration. A page that ships a Person node in one script and an Article pointing at it from a second script hands a validator two documents to read separately, and the validator reports a thin duplicate entity instead of merging the two into one.
Inline the identifying fields of anything you reference. A crawler arriving at a single page in isolation cannot dereference a cross-page pointer, so an author declared only by a bare @id publishes no author name at all when the home page was never fetched. Google also requires that structured data represent content visible to the reader, so every claim in the markup needs a matching claim in the copy.
Do FAQPage and BreadcrumbList markup still earn anything?
BreadcrumbList still earns the trail Google prints in place of a raw URL, so it belongs on every page below the root. FAQPage no longer earns a rich result for most sites. Google retired the FAQ rich result in August 2023 and its documentation now states the feature appears only for well-known, authoritative government and health websites, which means a general B2B site marking up an FAQ gets no accordion and no extra pixels.
Ship the FAQPage node anyway, for a different reason. It encodes a clean question and answer pair in machine-readable form, which is the exact shape a retrieval system stores and returns. The rich result was never the whole value, and the node costs one object inside a graph the page already emits.
What are E-E-A-T signals, and how do you show them?
E-E-A-T stands for Experience, Expertise, Authoritativeness and Trustworthiness. It comes from Google's Search Quality Rater Guidelines rather than from any single input in the ranking system, so no meta tag sets it. You show it by naming a real author with a linked profile, dating the last significant revision honestly, writing first-hand detail only someone who did the work would know, and citing primary sources by link rather than by assertion.
The cheapest version of this is a number with a source attached. A sentence stating that sitemaps cap at 50,000 URLs and 50MB per file, credited to the sitemaps.org protocol, survives being lifted into an answer somewhere else. A sentence telling the reader to keep sitemaps small does not, because it carries nothing a model can attribute.
What is llms.txt, and does publishing one do anything?
llms.txt is a proposed convention for a markdown file at the root of a site that hands an agent a curated index of the pages worth reading. Jeremy Howard published it on 3 September 2024 and it now sits at a second revision. No major engine has committed to reading it, so nobody should promise rankings from it. It costs one route, it doubles as a human-auditable inventory of what a site claims to offer, and keeping it in sync is close to free.
Be honest about what it buys. Google states outright that you do not need to create new machine-readable files, AI text files, or special schema.org markup to appear in its AI features, so nobody should sell llms.txt as the thing that gets a site cited. Pair the index with a fuller variant and keep both true. This site serves /llms.txt as the index and /llms-full.txt as the expanded corpus, and a test fails the build whenever a URL appears in one file and not the other, because an index that drifts away from the site it describes is worse than no index.
When do you actually need hreflang?
Only when the same content exists in more than one language or regional variant. A single-language site needs none, and adding it invents a maintenance burden with no upside. Where it does apply, every URL in a set must reference every other URL in that set including itself, the annotations have to agree in both directions, and an x-default entry should name the page a visitor lands on when no variant matches their language.
The usual failure is a one-way declaration, where the English page names the Arabic variant and the Arabic page names nothing back. Google may ignore annotations that lack return references, which drops the site back where it started while leaving extra markup behind to maintain.
How do you measure AI citations instead of guessing at them?
Fix a query set, run it against the live engines on a schedule, and record whether the domain appears and where. Guessing is the default because the surfaces feel opaque, yet half the data already sits in Search Console. Google states that sites appearing in AI features such as AI Overviews and AI Mode are counted inside overall search traffic, reported in the Performance report under the Web search type. The other half of the picture comes from running the queries yourself against real engines.
This site does that with a harness in packages/tooling/scripts/ai-visibility, which queries live ChatGPT and Perplexity engines through searchapi.io and records the citation rate against a fixed query set. Measurement is the whole point. Google states there are no additional requirements to appear in AI Overviews or AI Mode beyond normal SEO practice, so the only honest way to know whether a change moved anything is to have measured the same queries before the change shipped.
Where were these rules tested?
On this domain, with the numbers written down before and after. Everything below is measured on pooyagolchian.com rather than borrowed from a case study, and the honest part is the third card.
0.32% to 0.86%
Sitewide click-through rate
Measured across the August 2026 work on this domain, on flat impressions, with average position moving from 11.9 to 8.7 over the same window.
ChatGPT and Perplexity
Engines the harness queries
Citation rate is measured against live engines through searchapi.io rather than inferred from rankings, and this site is cited more often than it ranks.
16 results
Total results for the brand phrase
Google reports 16 results for the exact phrase pooya golchian, with this domain at position 1, zero sitelinks and no knowledge panel. Entity density off-site is the gap, not markup on-site.
Questions this reference gets asked
Short answers to the arguments that come up whenever a team starts optimizing for answer engines as well as search engines.
Mostly the same work with a sharper output format. Google states there are no additional requirements to appear in AI Overviews or AI Mode beyond normal SEO practice, so crawlability, canonical hygiene and honest structured data still carry the weight. What changes is the writing. An answer engine quotes a passage rather than ranking a page, so a heading phrased as a question followed by a complete self-contained answer with a number in it is the unit that gets retrieved.
Block the training crawlers if the policy calls for it, and leave the retrieval agents open. GPTBot, ClaudeBot, Google-Extended and CCBot collect content for model training, and disallowing them costs nothing in any live answer surface. OAI-SearchBot, PerplexityBot, Claude-User, Perplexity-User and ChatGPT-User fetch pages to answer a question someone is asking right now, and blocking those removes the site from the answer. OpenAI documents the split explicitly and says each setting is independent of the others.
Weeks rather than days, and the first thing that moves is click-through rate rather than position. On this domain the August 2026 pass moved sitewide click-through rate from 0.32% to 0.86% and average position from 11.9 to 8.7 while impressions stayed flat, which is the signature of better titles and snippets against an unchanged crawl. Split the Search Console window at the deploy date, because a 90-day average blends the before and after into something that reads as a broken site.
No. A knowledge panel comes from an entity Google is confident about, and confidence is built from independent sources that describe the same entity the same way. Markup helps a crawler reconcile what it already found elsewhere. This domain ranks at position 1 for its own brand phrase with a complete Person graph and still shows no panel and no sitelinks, against 16 total results for the phrase, which is a volume and off-site density problem rather than a schema problem.
Rewriting the first paragraph under each heading so it stands on its own. Most pages open a section with a transition sentence that means nothing once it is lifted out of context, and that is the sentence a retriever stores. Name the subject, state the claim, attach a number, and cite the source. It costs nothing in build time and it is the difference between a page a model can quote and a page it has to summarise from scratch.
Not for most sites. Google retired the FAQ rich result in August 2023 and its documentation now limits the feature to well-known, authoritative government and health websites. FAQPage markup is still worth shipping because it encodes question and answer pairs in the exact shape a retrieval system stores, but nobody should expect an accordion in the search result from it.
Engagement
Want these rules checked against your own site?
The audit runs this list end to end, then measures citation rate on live ChatGPT and Perplexity engines so the fix list is ranked by lift rather than by habit.
SEO and AEO audits start at $6k.
One fixed-scope pass over technical SEO, the entity graph, and measured AI citation rate. You get a findings document, a ranked fix list with file-level locations, and a re-measure at 60 days. Delivery runs 2 to 3 weeks.