Optimizing content for AI agents means writing pages whose individual claims survive being lifted off the page, because the reader is increasingly a machine assembling an answer rather than a person scanning a results list. That is a different brief from classic search optimisation, which rewards earning a click. This post makes the case using predictions published by people who have no answer engine optimization business to protect, then gets specific about what actually changes in a content operation and what the work costs.
The 30-second rule
Between 9 and 11 December 2025, three a16z teams that do not sell answer engine optimization published versions of the same prediction. Growth said people will interface with the web through agents. Infrastructure said the traffic those agents generate is recursive, bursty and massive. Speedrun said products stop being mass-produced. The convergence matters more than any single call, because these are investors with no product line riding on the answer. Eight months on, the cheap half of the response has aged well and the expensive half has not. Rewrite your openings so a machine can lift them, put the source and date inside the sentence carrying the number, name entities in full, and measure citation share on a fixed prompt set. Defer the infrastructure spend until you have measured a real block.
The 30-second version: between 9 and 11 December 2025, three separate a16z teams published versions of the same claim in the firm's Big Ideas 2026 series. The Growth team said people will reach the web through agents. The Infrastructure team said the traffic those agents generate is recursive, bursty and massive. The Speedrun team said products stop being mass-produced. None of the three needed the others to be right. Eight months later, the cheap half of the response has aged well and the expensive half has not, which is a useful result on its own.
Almost everything written about optimizing for answer engines is written by someone who sells answer engine optimization. We sell it too, so this post starts from a source that does not. A venture firm publishing its investing theses has a different incentive: it is allocating capital against these calls, so being right matters more than being persuasive. When the same structural claim shows up three times in one series, from three teams solving unrelated problems, it is worth reading carefully before deciding what to do about it.
Operator noteFORKOFF has no commercial relationship with a16z and is not a portfolio company. The series is cited as an external source.
What did a16z actually predict about AI agents?
a16z predicted, across three separate teams in its Big Ideas 2026 series, that agents become the layer through which people reach the web. Stephenie Zhang on the Growth team wrote a prediction titled "Creating for agents, not humans", stating that "In 2026, people will start interfacing with the web through their agents. And what mattered for human consumption won't matter the same way for agent consumption." That is the sentence this entire post hangs on, and it was written by someone whose job is spotting growth patterns, not selling schema markup.
The second thread comes from the Infrastructure team. Malika Aubakirova's prediction, titled "Agent-native infrastructure becomes table stakes", describes the mechanical consequence: "We're shifting from human-speed traffic that's predictable and low concurrency to 'agent-speed' workloads that're recursive, bursty, and massive." She also frames where the pressure comes from, writing that "In 2026, the biggest infrastructure shock won't come from outside companies, but from within."
The third comes from Speedrun. Joshua Lu's prediction is titled "The year of me", and argues that "2026 will become 'the year of me': the moment when products stop being mass-produced and start being made for you", concluding that "the biggest companies of the next century will win by finding the individual inside the average." Lu makes the same argument a second time in the speedrun companion piece, which is how a personalization prediction ends up mattering to a content team.
a16z
@a16z
We asked a16z's investors for their takes on the biggest problems builders will tackle in 2026. Here's part 1 of Big Ideas 2026:
Operator noteEvery a16z quote in this post was re-verified against the live a16z.com pages on 2026-08-04, not taken from a summary.
Why do three independent predictions matter more than one?
Three teams reaching the same premise separately is stronger evidence than any one of them stating it forcefully, because the teams were not solving the same problem. Growth was thinking about how users find products. Infrastructure was thinking about what breaks under load. Speedrun was thinking about consumer product design. Each prediction stands on its own and would still be interesting if the other two were withdrawn. That is what makes the shared assumption underneath them worth taking seriously.
The three a16z Big Ideas 2026 predictions that converge, verified against source 2026-08-04
| Partner | Team | Prediction title | The claim underneath it |
|---|---|---|---|
| Stephenie Zhang | Growth | Creating for agents, not humans | People will reach the web through agents, so what mattered for human consumption stops mattering the same way |
| Malika Aubakirova | Infrastructure | Agent-native infrastructure becomes table stakes | Traffic shifts from predictable and low-concurrency to recursive, bursty and massive |
| Joshua Lu | Speedrun | The year of me | Products stop being mass-produced, and winners find the individual inside the average |
| Marc Andrusko | Apps | Prompt-free and proactive applications arrive | The prompt box dies for mainstream users and AI becomes invisible scaffolding |
| Sean Neville | Crypto | From know your customer to know your agent | Non-human identities already outnumber human employees 96-to-1 |
Sourced from a16z Big Ideas 2026 parts 1, 2 and 3, published 9, 10 and 11 December 2025. Every quoted fragment was re-verified against the live a16z.com pages on 2026-08-04. FORKOFF has no commercial relationship with a16z.
The convergence is not limited to those three either. In part 2, Marc Andrusko on the Apps team predicted that "2026 marks the death of the prompt box for mainstream users...AI becomes invisible scaffolding woven through every workflow." Anish Acharya, in the same instalment, framed the distribution consequence, writing that "ChatGPT's 900M user audience...new distribution channel is set to kick off a once-in-a-decade gold rush in consumer tech in 2026." Seema Amble added that enterprises "will need systems of coordination: new layers to manage multi-agent interactions...across autonomous workflows."
Read together, these are not five predictions about AI. They are five descriptions of the same intermediary appearing between a person and the thing they are trying to reach. If the prompt box disappears into scaffolding, if distribution runs through an assistant with hundreds of millions of users, and if enterprises are coordinating fleets of agents, then the number of moments where a human personally reads your page while deciding something goes down, and the number of moments where a machine reads it on their behalf goes up.
The value here is who is not saying it
Every agency selling answer engine optimization argues that answer engines matter. That argument is worth very little on its own, because the person making it is paid to make it. What makes the a16z series useful is that a venture firm has no answer engine optimization product line, no retainer to protect, and a strong incentive to be right rather than persuasive, because they are allocating money against these calls. When a claim arrives from someone with nothing to sell you, it earns a different weight. That is the only reason this post is anchored on their predictions instead of ours.
Source: FORKOFF editorial position, stated so readers can discount it appropriately
The identity number that makes the direction concrete
In part 3, Sean Neville's prediction on moving from know your customer to know your agent contains the most quotable figure in the series: "non-human identities now outnumber human employees 96-to-1". His point is that these identities remain unbanked, meaning they transact without the identity rails humans have. It is worth being precise about what that number is and is not, because it is the kind of statistic that gets misquoted into something it never claimed. The a16z crypto companion post carries the same predictions with the attributions intact, which is how we confirmed them.
It is a claim about identities inside organisations, service accounts, tokens and machine credentials, not a claim about web traffic composition. Nobody should recycle it as a statistic about what fraction of web visitors are bots, because it never measured that and the two things are not related. What it does establish is a direction of travel that is already very far along inside the enterprise, before the consumer-facing version of the same shift has really started. The distinction matters for anyone building a business case, because an identity ratio inside a company tells you nothing at all about how many machines are reading your marketing pages this month. The same instalment carries Jeremy Zhang's figure that "Stablecoins accounted for an estimated 46 trillion dollars in transaction volume last year", another number that only makes sense in a world where a great deal of activity has no human in the loop at the moment it happens.
Elizabeth Harkavy's prediction in the same part, titled "The invisible tax on the open web", names the uncomfortable half of this directly: AI agents extract data from ad-supported sites while systematically bypassing the revenue streams those sites depend on. That is a real cost, and it is the reason a chunk of the web is actively trying to keep agents out rather than optimise for them. Any honest version of this argument has to hold both facts at once.
a16z
@a16z
The 3rd and final installment of a16z's Big Ideas 2026, this time featuring takes from our Crypto team (and some special guests) looking ahead to the new year:
What changes about the path between a question and a buyer?
The practical consequence is that the discovery path gained a step, and the new step is where the decision now happens. In the classic path, a person typed a query, saw ten results, chose one based on how it looked, and arrived on a page that then had to convince them. Every optimisation instinct most teams have was trained on that sequence. In the agent-mediated path, a person asks a question, a machine reads many sources, and the person receives a synthesised answer that either names you or does not.
The difference is not that traffic moved. The difference is that the persuasion moment moved upstream, into a place you cannot design. You do not get to control the layout, the ordering, or the framing of the answer the buyer sees. The only thing you control is whether your page produced a claim clean enough to be carried into it with your name attached. Our own guide on how AI Overviews decide which brands to cite goes deeper on the selection mechanics; the point here is that selection is now the whole game.
This is also why the usual framing of "AI is taking our clicks" is the wrong complaint. The clicks were never the asset. The clicks were a proxy for having been chosen, and the proxy is being replaced by something more direct and much less forgiving.
The citation is the impression now
Teams keep trying to value answer engine work in sessions, and the numbers look bad, because the referral click often never happens. That accounting misses what actually changed. On a results page you earned a chance to be considered among ten options. In a synthesised answer you are either named as the source or you are absent, and being named reads to the buyer as a recommendation rather than a listing. Fewer events, much higher value per event. Measuring it in sessions will tell you to stop doing the thing that is working.
Source: FORKOFF AI Citation Index 2026 methodology notes
What does machine-legible content actually mean?
Machine-legible content means a page whose claims still make sense after being removed from the page. That is the whole definition, and it is more demanding than it sounds, because most well-written pages are built on accumulated context. A good human-facing article sets something up in paragraph two and pays it off in paragraph nine. An extraction process arriving at paragraph nine gets a sentence about "this approach" with no idea what the approach is.
The same page, written for a human reader and written to be extractable
| Element | Written for a human on a results page | Written so a machine can lift it |
|---|---|---|
| Opening line | A hook that creates curiosity and rewards scrolling | A self-contained answer that survives being pasted somewhere else with no context |
| A statistic | The number in the sentence, the source in a footer or a link further down | The number, the source and the date in the same sentence |
| The subject | Named once, then "it", "this" and "the company" for the rest of the page | Named in full at the start of each section that makes a claim about it |
| A comparison | A visually styled grid that reads well on screen | A real table with real headers, so the relationship survives being parsed |
| Structured data | Added for rich-result eligibility, loosely matched to the copy | Agreeing exactly with the visible text, because a mismatch is a reason to distrust both |
| The conclusion | A summary that depends on everything above it | A restatement that stands alone, because it is often the only part quoted |
The left column is not wrong. It is what a decade of optimising for a human choosing between ten blue links correctly taught teams to do. The point is that the two columns are no longer the same brief.
Working through that table on real pages produces a short list of habits that do almost all of the work. Name the subject in full at the start of any section that makes a claim about it, rather than relying on the reader having read the heading. Put the source and the date inside the sentence that carries the number, not in a footer, because a footer does not travel with an extracted sentence. Use real tables with real headers when you are expressing a relationship, since a styled grid of divs reads as decoration. Make your structured data agree exactly with the visible text, which is covered properly in our schema markup for AEO breakdown.
None of this requires writing worse prose for humans. That is the part teams get wrong when they first try. The instruction is not "write like a robot", it is "stop depending on the reader having scrolled". Those are different, and the second one has been good writing advice since long before any of this.
The answer capsule is the highest-leverage single change
An answer capsule is a self-contained direct answer of roughly 40 to 80 words, placed immediately after the heading it answers and before any list, table or image. It is the highest-leverage change available because it is the exact unit an answer engine can quote without editing, and because almost no page has one. Most pages open a section with a transition sentence, a bit of context, and then arrive at the answer three paragraphs later, by which point the extractable moment has passed.
The test is simple and slightly brutal: copy the first eighty words after any heading on your site, paste them into an empty document, and read them. If they reference something that came earlier, if they start with "this" or "it" or "that said", if they need the heading to be comprehensible, they will not be quoted. Fixing this on an existing page takes a few minutes per section and requires no new research, which is why it is the first thing we do on any answer engine optimization engagement.
Every H2 on this post follows the rule, which you can verify by scrolling back and reading the first paragraph under any of them in isolation. That is not a stylistic accident, and it is the cheapest thing on this entire list.
How AI Agents Will Transform in 2026 (a16z Big Ideas)
a16z
a16z walking through how they expect AI agents to change in 2026, from the same Big Ideas series
The three layers underneath extractability
Extractability has three layers, and skipping the bottom one is why a lot of schema work produces nothing measurable. The top layer is the standalone claim, the capsule described above. The middle layer is evidence, meaning every number carrying its source and date inline so the claim can be trusted without a second lookup. The bottom layer is agreement, meaning your structured data, your visible copy and your page title all say the same thing.
Teams tend to start at the bottom because it feels technical and therefore serious, adding markup to pages whose prose is not quotable. That produces perfectly valid structured data describing a page with nothing liftable in it. The order that works is the reverse: fix the claims, then the evidence, then make the markup agree. Our 12 structural patterns piece covers the specific formats, and the ranking factors guide covers the selection side.
The agreement layer deserves one specific warning. A mismatch between markup and copy is worse than having no markup, because it gives a reason to distrust both. If your FAQ schema contains answers that are not on the page, you have not added a signal, you have added a contradiction.
What changes about measurement when the click never happens?
Measurement changes from counting arrivals to counting mentions, and this is genuinely harder rather than merely different. A session is unambiguous and instant. A citation is probabilistic, varies between runs of the same prompt, differs across engines, and has no equivalent of a referrer header you can rely on. Anyone who tells you this is a solved measurement problem in August 2026 is describing a product roadmap rather than a current capability.
What works today is unglamorous. Fix a prompt set that represents how your buyers actually ask, run it on a schedule against each engine you care about, record which sources get named, and track your share of those citations over time. The absolute number on any single run is noise. The trend across a fixed set is signal. FORKOFF's own AI Citation Index ran a 50-prompt buyer-intent cluster across five engines and arrived at a 34 percent average cite rate, with Perplexity leading the index at 48 percent. The most useful finding was not the average but the spread underneath it, and the fact that 14 of the 50 prompts returned zero citations on any engine at all.
That divergence has a direct operational consequence: optimising for one engine and assuming the others follow is a mistake, which is the argument in our Perplexity versus Google AI Overviews comparison. The method for building the baseline itself is in how to measure your share of AI citations, and if you want a single reading before committing to a programme, the AI search visibility checker will give you one.
There is also a vocabulary problem worth clearing up before anyone builds a reporting line around this, because answer engine optimization and generative engine optimization are used interchangeably by people who mean different things, and the distinction matters once you are choosing what to actually measure. We separate the two in the AEO versus GEO guide.
a16z just dropped their Big Ideas list. two of them hit me differently as someone actually building in AI
a16z put out their big ideas for 2026 and most of it is the usual VC futurism stuff but two ideas actually connected to problems I deal with every day running an AI startup. first one is collaborative AI tools. right now every AI tool is basically single player. a16z… Show more
The infrastructure half nobody in marketing owns
Aubakirova's prediction matters to a content team for one reason: a page that cannot be fetched cannot be quoted, and the failure is completely silent. Her description of agent workloads as "recursive, bursty, and massive" is a statement about traffic shape, not traffic volume. A single agent working a task can fan out into many parallel requests in milliseconds and then recurse on the results, producing a pattern that looks nothing like a human reading a page.
Rate limiters, bot management rules and edge protection are tuned for the human shape. Presented with the agent shape, they do exactly what they were built to do and block it. Nothing in a marketing dashboard reports this. Human sessions look normal, uptime is green, and the only symptom is an absence in a channel you were probably not measuring anyway.
The failure mode here is silent
If an answer engine's crawler is being rate-limited or challenged by your edge protection, nothing in your analytics will say so. Human sessions look normal, uptime is green, and the only symptom is an absence you were never measuring. This is the specific reason Aubakirova's infrastructure prediction belongs in a content conversation at all. A page can be perfectly extractable and still never be read, and the team will conclude the content work failed when the content work never ran.
Source: FORKOFF technical audit findings across client properties
The check is cheap even though the fix might not be. Look at what your edge is actually serving to the named answer-engine crawlers, confirm they are getting the same content a human gets rather than a challenge page or a stripped variant, and confirm they are not being rate-limited into uselessness. That is an afternoon of work with your infrastructure team and it either finds something or it does not. Re-architecting for agent-speed concurrency is the expensive version, and almost nobody reading this needs it yet.
Is this just SEO with new vocabulary?
The strongest argument against everything above is that AI referral volume is still small, and it is a good argument. For most businesses we work with, organic search remains the larger channel by a wide margin in August 2026, and a team that abandons search to chase citations will be worse off in a year. Any version of this pitch that skips past that is selling rather than analysing.
Operator noteShipped as a citation asset. Six candidate head terms all measured below the tenant volume floor on 2026-08-04.
This is not a replacement for search, yet
Nothing in the a16z series claims search disappears in 2026, and neither does this post. Organic search remains the larger channel for almost every business we work with, and a team that abandons it to chase citations will be worse off in twelve months. The honest framing is additive. The same page can be built to rank and to be quoted, the two briefs overlap more than they conflict, and the cost of covering both is mostly a matter of how the first eighty words of each section are written.
Source: FORKOFF client programme observations, 2026
There are four more objections worth stating properly, and the first one is the sharpest. It is that this is just search optimisation with new vocabulary, and the people making it are not cranks. An SEO consultant posted a 2017 Search Engine Land article about winning featured snippets in December 2025 with the observation that reading it eight years later, "it sounds exactly like modern AEO/GEO advice." That is a fair hit. Structuring content into extractable, directly-answering blocks is not a 2026 invention, it is what snippet optimisation asked for most of a decade ago, and anyone presenting it as new is overselling.
The honest counter is narrower than the usual one. Much of the tactical advice genuinely is old, and the parts that are old are the parts most worth doing. What changed is not the technique but the payoff structure: a featured snippet was one slot on a page that still showed nine other results, whereas a synthesised answer frequently names three sources and nothing else. Same craft, different concentration. Our AEO versus SEO breakdown works through where the two briefs actually diverge, and Google's own guidance on generative AI features is notably unexcited about anything exotic, which is itself evidence for the skeptics on this specific point.
The second objection is that a chunk of this field is straightforwardly oversold. In a widely-read r/SEO thread, one practitioner argued that when people claim "AI/LLMs have their own trust models, prefer short content, need a special structure, use schema and build their own indices", that is "a campaign of disinformation." We do not agree with all of it, and we sell this service, so discount us accordingly. But the specific claims being attacked there are ones we would also not make, which is why nothing in this post asks you to buy a file format or a proprietary index.
The third is that the engines will change their selection logic and invalidate the work. Likely true in the specifics, less true in the fundamentals, since "make a clear, sourced, self-contained claim" is not a trick that gets patched. The incumbent guidance currently holding the featured snippet for this query says much the same thing, which is mild evidence that the durable advice has already converged.
The fourth is structural, and it is the one we find hardest to dismiss. It is Harkavy's open-web tax read from the publisher's side: if being read produces no traffic, the incentive to publish erodes. A Hacker News commenter made this argument in May 2023, long before any of these predictions, asking "What incentive does a website have to produce content if it receives no traffic?" and predicting owners "will wall off any search engine that doesn't give them anything in return". Two and a half years later that is visibly happening. A web where the best sources are behind a wall is a worse outcome for everyone in this argument, us included, and nothing in the a16z series resolves it.
One last disclosure belongs here: a16z has a portfolio interest in agents mattering. Readers should weight the source accordingly. That interest is exactly why this post leans on their descriptive claims about mechanism, which are checkable, rather than their forecasts about magnitude, which are not.
The most overlooked a16z Speedrun play: building tools for AI-native startups, not for humans using AI
a16z Speedrun's portfolio has a pattern most founders miss: selling tools to AI companies, not to humans using AI. Six themes dominate: agent-native infrastructure, AI-native vertical services, prompt-free consumer apps, voice agents, AI tools for AI companies, and vertical research-as-a-service. Most indie hackers are building for the first four. Almost… Show more
What has actually held up eight months on?
Eight months after publication, the interface prediction has aged best and the timeline predictions have aged worst, which is the normal shape of forecasting. The claim that people would increasingly reach information through an assistant rather than a results page is visibly more true in August 2026 than in December 2025. The claim that this would arrive as a wholesale replacement, which the series does not actually make but many readers took from it, has not happened.
Operator noteFiled as opinion rather than trending. The series is eight months old, so this is a thesis argument, not a news reaction.
a16z put out their big ideas for 2026 and most of it is the usual VC futurism stuff but two ideas actually connected to problems I deal with every day running an AI startup.
The commentary since publication has been more discerning than the usual reaction cycle. Operators picked out the specific predictions that matched problems they already had rather than treating the list as prophecy, and the most interesting public reactions have been about the second-order implication rather than the headline, that the customer and the reader are increasingly machine-shaped.
Most indie hackers are building for the first four. Almost nobody is building specifically for category five.
For our own purposes the useful scoring is narrower. The cheap structural work has produced measurable movement in citation share across client programmes and cost effectively nothing, because it improved the pages for human readers at the same time. The expensive work, meaning dedicated tooling and infrastructure changes, has mostly not been justified yet on the evidence available. That asymmetry is the actual finding, and it is the one that determines sequencing.
Split the work by cost of being wrong
The response to this thesis divides cleanly into work that is free if the thesis is wrong and work that is expensive if the thesis is wrong. Rewriting an opening paragraph so it stands alone, moving a source and a date into the sentence that carries the number, and naming the subject instead of writing "it" all make the page better for human readers too. If agents never arrive, you have lost nothing. Re-architecting for bursty machine traffic, buying dedicated tooling and restructuring a team are the opposite. Do the first category on conviction and the second only on measurement.
Source: FORKOFF answer engine optimization engagements, sequencing used across client programmes
2. llms.txt Setup
The cheapest structural change on this list is also the one most teams skip, because it looks optional rather than load-bearing. llms.txt is a proposed convention, a plain-text file at the site root that tells an AI system which pages matter and gives it a clean, markdown version of each one to read instead of parsing rendered HTML. It is not a ranking signal Google reads and it is not a substitute for structured data. It is a map an agent can follow when it is trying to decide what a site is and which page answers a specific question, without spending its own compute stripping navigation chrome out of your HTML first.
FORKOFF runs its own llms.txt and llms-full.txt at forkoff.xyz, generated from the same content that ships to human readers rather than hand-maintained as a separate document, because a second copy of the site's content is a second thing that drifts. The setup that matters is narrow: a per-route markdown mirror kept in parity with the rendered page (not a summary, not a stale snapshot), a root llms.txt that lists every real route rather than every route that existed six months ago, and a build step that regenerates both from source so the map cannot silently fall behind the site. The failure mode we have measured directly is not having none of this; it is having a hand-written version from launch day that nobody has touched since, which reads to an agent as confidently wrong rather than absent.
Optimize for speed
Page speed matters for AI crawlers for a different reason than it matters for Google's Core Web Vitals ranking factor, and conflating the two leads to optimizing the wrong thing. A ranking crawler tolerates a slow page because the payoff for indexing it is high and the crawl is amortized over time. An agent answering a live user question inside a bounded prompt-and-tool-call budget does not have that luxury: if fetching and parsing a page takes long enough to blow the agent's own latency budget, the page gets skipped in favor of a faster source, even a worse one, and the agent never learns what it missed. Slow beats absent for a human with a browser tab open. Slow loses to fast-and-mediocre for an agent that has to answer in one turn.
The practical fix is the same discipline that already helps human readers: server-rendered content over client-side-only rendering (an agent that does not execute JavaScript sees nothing on a client-rendered page, which is a harder failure than slow), a lean first response that does not make the crawler wait on non-essential scripts, and keeping the markdown mirror small enough that reading it costs an agent less than reading the full rendered page would. None of this requires new tooling if the site already treats CWV as real; it requires treating the AI-agent path as a second, stricter reader rather than assuming a fast page for Google is automatically a fast page for a tool-calling agent on someone else's compute budget.
What is agentic commerce?
Agentic commerce is the case where an AI agent does not just answer a question about a product, it completes the transaction: comparing options, applying a stated budget or constraint, and executing the purchase or booking through a tool call rather than handing the human a link to click. It is the logical endpoint of the same shift the a16z series describes, an intermediary reading and now acting on a page's content on a buyer's behalf, and it is why the answer-capsule discipline in this piece is not only about being quoted, it is about being the option an agent's tool call actually selects when several sources satisfy the same query.
The structural implication for a seller's own pages is that price, availability, and terms need to be machine-extractable in the same place the human-readable claim about them lives, the same "structure agrees with substance" rule Google already states for FAQ and Article schema. A page whose visible copy says "starting at $X" while a separate, unsynced pricing table is the actual source of truth is a page an agent cannot safely transact against, and an agent that cannot verify a price will route around the page entirely rather than risk quoting a stale one. This is early enough that no team should over-invest in agentic-commerce-specific infrastructure yet, per the same phase-one-before-phase-three sequencing above, but the underlying hygiene, one place where the number lives, rendered in both the page a human reads and the structured data an agent reads, is worth having regardless of when or whether transactional agents become a meaningful channel.
AI Training vs. AI Search, what's the difference?
These are two different crawlers with two different jobs, and treating them as one is how a site ends up either invisible to answer engines or unknowingly feeding a training set it never agreed to. An AI TRAINING crawler (the bots that build a foundation model's pretraining corpus) visits once, on its own schedule, months before any model ships, and what it collects shapes model weights that cannot be un-learned by blocking the crawler afterward. An AI SEARCH crawler (the bots a live product like Perplexity or ChatGPT's browsing mode dispatches at query time) visits when a real user's question needs an answer right now, and blocking it has an immediate, visible effect: your page stops being citable starting the moment the block takes effect. robots.txt can and should treat these differently, allowing the search-time crawlers this article already covers while a site that has a real reason to opt out of training (a wire-service or subscription publisher, mostly) can block the training-specific user agents by name without touching search visibility at all.
An ai-friendly robots.txt, and how blocking crawlers can make you invisible
The failure mode here is the same shape found earlier in this piece: a robots.txt written for 2023's crawlers goes stale and starts blocking the ones that matter now, silently. The rule is not "allow everything," it is "know which agents you are blocking and why." Concretely: name the crawlers you want (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, and whichever others match your traffic sources) with explicit Allow rules rather than relying on a bare wildcard, and audit the Disallow list for anything that accidentally catches build output or API routes those crawlers need to render or verify your page, the exact class of bug this piece already covered under crawler-access checks. A site that blocks an agent by accident reads to that agent as a site with nothing to say, not a site with a policy, and there is no error message telling you it happened, only the absence of citations you would otherwise expect.
Accessible APIs, RSS, or feeds
An agent that has to render a full page to extract one fact is doing more work than an agent that can hit a feed or an API and get the fact directly, and every extra step is a place the agent can time out, misparse, or simply give up. Where one exists, a JSON API, an RSS or Atom feed, or a documented endpoint is a faster, more reliable path to the same information than the rendered page, and worth exposing even when the primary audience is still human readers on the page itself.
Submit a sitemap.xml
A submitted, current sitemap.xml is the cheaper, more universal version of the same idea as a feed: it tells any crawler, training or search, exactly which URLs exist without it having to discover them by link-following, and a sitemap that has not been resubmitted since a site restructure is one more stale-map problem alongside the robots.txt and llms.txt versions already covered here.
Indicate content freshness
An agent deciding which of several similar sources to cite has almost no way to tell a page updated last week from one abandoned two years ago unless the page says so explicitly. A visible, accurate last-updated date next to the claim it applies to, not just in a page footer nobody reads, is a citation-worthiness signal the same way the answer-capsule structure is: it tells the agent this number is current enough to repeat. The failure that costs more than having no date at all is a stale date that is wrong in the other direction, a "last updated" stamp bumped by a template change with no corresponding content edit, because an agent (or a human) that catches the mismatch once stops trusting every date on the site afterward.
Data accuracy
Data accuracy is the same discipline applied to the numbers themselves: a stat that was correct when written and never re-verified is a liability the moment the underlying reality changes, which is the entire argument for the audit-ledger practice this piece already describes for FORKOFF's own claims. An agent that catches a site publishing one stale number stops trusting every other number on that domain, the same way a human reader does after catching one factual error.
The optimization ladder
Treat the items in this piece as a ladder, not a checklist to complete in one pass: crawler access and an accurate sitemap and robots.txt are the bottom rung, because nothing above them matters if the agent cannot reach the page at all; the answer-capsule and entity-naming discipline is the middle rung, because it is what turns a reachable page into a citable one; and llms.txt, API access, and freshness signals are the top rung, refinements that compound once the first two rungs are solid. Climbing the ladder out of order, building agent-specific infrastructure before the basic crawl-and-cite path works, is the exact inversion this piece's 90-day sequencing exists to prevent.
API design for AI agents and real-time content adaptation
Beyond a read-only feed, some sites now expose an API surface built explicitly for agent consumption, structured, versioned, documented the way a developer-facing API would be, rather than an afterthought scrape target. This matters most for sites whose core value is data rather than narrative (comparison tools, pricing calculators, live inventories), where an agent that can query the API directly gets a more reliable answer than one parsing a rendered table. Real-time content adaptation, serving a version of a page tuned for the requesting agent rather than one fixed document for everyone, is the more speculative end of this, and it carries a real risk the a16z thesis does not dwell on: a page that shows different facts to an agent than to a human reader is a page that cannot be held to one standard of truth, and the safer default for most sites is one accurate document read by both, not an agent-specific variant that can drift from what a human sees.
Clean, structured text wins
Clean, structured text wins over decorative markup for the same reason a well-formed answer capsule wins over a buried one: an agent parsing a page for a fact does not benefit from a visually rich layout that adds no extractable structure, and every div of decoration between the agent and the sentence it needs is one more place parsing can go wrong.
Metadata and semantic matter more
Metadata and semantic HTML matter more than they did when the only reader was a ranking crawler willing to wait out a messy document. A heading that is actually a <h2>, a list that is actually a <ul>, and a title tag and meta description that state the page's real subject rather than a marketing tagline all cost nothing to get right and each one is a small, cumulative signal about what a page is and how trustworthy its structure is, both for a search-time agent and for whatever eventually gets pulled into a training run.
TL;DR: a quick AI-optimization checklist
Speed and simplicity are critical here in the literal sense: an agent working inside a bounded turn rewards a page that is fast to fetch and cheap to parse over one that is merely comprehensive. Run this list against a page before spending on anything more elaborate: llms.txt and a current markdown mirror in place; robots.txt allowing the search-time agents by name; a submitted, current sitemap.xml; a visible, accurate freshness date beside the claim it covers; an answer capsule in the first eighty words; and a fixed prompt set run against five engines so the next change is measured against a real baseline instead of a guess.
The 90-day version of this work
The 90-day version has three phases ordered by cost of being wrong rather than by interest. Phase one is measurement, because everything after it is unfalsifiable without a baseline. Phase two is the structural work that improves pages regardless of whether the thesis holds. Phase three is the infrastructure and tooling spend, which should only start once phase one has produced evidence that it is needed.
In the first thirty days, build the prompt set and the citation baseline, and separately run the crawler-access check described above. Both are cheap and both are prerequisites. In the next thirty, rewrite the opening eighty words of every section on your highest-intent pages to stand alone, move sources and dates inline, and replace pronoun subjects with named entities. Start with money pages and the pages already ranking, not with new content, because you are converting existing visibility into citability. The answer engine optimization playbook covers the page-by-page sequence, and the AEO checker will tell you which pages fail before you start.
In the final thirty, re-run the baseline and compare. Only now does the infrastructure question deserve budget, and only if the access check found a real problem. Teams that invert this order spend the most money on the least certain part, which is how a reasonable thesis produces an unreasonable invoice. For B2B specifically, the sequencing differs slightly and is covered in our AEO for B2B SaaS guide.
Where this leaves a founder deciding what to do
A founder should do the cheap half of this work now on conviction and the expensive half later on evidence. That is the entire recommendation, and it holds whether a16z turns out to be early, right on time, or wrong about magnitude. The structural writing work pays for itself in human readability alone. The measurement work is what converts an argument into a decision. Everything beyond those two is a bet that should wait for a number.
The reason to take the a16z series seriously is not that venture predictions are reliable, because individually they are not. It is that three teams with different mandates described the same intermediary appearing between people and information, and one of them was precise about the mechanism rather than the vibe. Zhang's sentence about what mattered for human consumption not mattering the same way for agent consumption is a testable claim about page structure, and it can be acted on cheaply.
What we would push back on is the implied urgency that this kind of series always generates. Nothing in the source material says search collapses this year. The predictions describe a direction, and the correct response to a direction is to move in it at a cost proportional to your confidence, which for most teams means a fortnight of editing rather than a re-platforming project. If you want the version of this scoped against your own pages, that is what our answer engine optimization work is, and the founder funnel side covers the demand half once the visibility half is working.
AI in 2026: 3 Predictions For What's To Come (a16z Big Ideas)
a16z
The three AI predictions a16z chose to put on video, published at the end of December 2025
a16z
@a16z
Part 2 of Big Ideas 2026, with takes from our American Dynamism and Apps teams on what next year holds for tech:
The last word belongs to the counter-argument, because it deserves it. If in twelve months AI referral volume is still a rounding error and citation share has not moved anything commercial, the people who spent a fortnight tightening their opening paragraphs will have lost a fortnight and gained clearer pages. The people who re-architected will have a harder conversation. Sequence accordingly.
