Content Gap Automation
A gap is only a gap when demand exists and the site could credibly answer it. Everything else is a longer list. Ivan Sp builds agents that compare a site’s existing coverage against real demand and against competitor URL sets. Output marks each topic as covered, thin, or missing, which turns a vague sense of falling behind into a specific build list.
What Gets Compared
The agent works from three inputs. The client’s own URL set establishes current coverage. A competitor sitemap or URL list, supplied rather than guessed, establishes what rivals publish. Query data establishes what people actually search.
Each topic then gets marked. Covered topics have a page that genuinely addresses them. Thin topics have a page that mentions them without answering. Missing topics have nothing.
Why Only Pasted URLs
Competitor comparison runs against URL lists supplied as input. The agent never estimates competitor traffic, competitor rankings, or the value of a competitor page.
URL coverage is fully verifiable, and traffic estimates are not. A deliverable claiming a competitor earns a specific number of visits from a specific page presents a guess as a finding, and a client can act on it before discovering the difference.
Expected Vocabulary Checks
Coverage runs deeper than topic presence. For each subject, the agent lists the terms a knowledgeable reader expects, then checks whether the page genuinely uses them.
A term appearing in a list without being used in a sentence counts as a failure rather than a pass. A self-storage page mentioning unit sizes, access hours, and gate access in a bulleted list, without explaining any of them, has the vocabulary and none of the coverage.
Tiered Analysis Depth
Analysis scales to the size of the opportunity rather than running at full depth on every term:
- Quick pass. Difficulty estimate, weaker competitors identified, winning content format, intent match, and three immediate actions.
- Standard pass. Minimum viable depth from the strongest ranking pages, commonly required sections, trust signals in use, and technical or schema opportunities.
- Planning pass. Findings from the first two stages rolled into a 30-day, 90-day, and six-month plan with defined success measures.
Reading Results Pages Without Visiting Every Site
Most competitive signal comes from the results page itself. Titles, descriptions, publish and update dates, rich snippet presence, answer block content, and format hints in URL structure are all available without visiting a single competitor page.
Site visits get reserved for what results pages cannot supply, meaning real word count, heading structure, internal linking patterns, and author expertise signals. Turning gaps into briefs runs through automated content brief generation.
What the Agent Compares
- The site’s coverage
- Established by crawling the site rather than by asking anyone.
- Real demand
- Query data from Search Console, plus the visible prompts on a results page.
- Competitor coverage
- Taken from URL lists or sitemaps supplied as input, rather than estimated.
- Why only pasted URLs
- A tool’s estimate of competitor rankings is a sample. A sitemap is evidence.
- The three verdicts
- Covered, thin, or missing, assigned per topic.
- Expected vocabulary
- The terms a credible answer contains, drawn from the ranking set.
- Analysis depth
- Three tiers, matched to how much opportunity a term represents.
- What the agent will not output
- A gap with no demand data behind it.
Filter One Gap Report Yourself
Ivan Sp suggests a manual version of the filtering.
- Open your most recent gap report and count the total rows.
- Delete every row containing a competitor’s brand name.
- Delete every row your business could not credibly write about.
- Count what remains.
Most reports lose over half to those two passes alone. What survives is closer to a plan than what you started with.
Where the Standard Gap Analysis Advice Breaks Down
How many rows of your last gap report actually became pages?
- The usual advice
- Run a gap report against your competitors and work through everything they rank for and you do not.
- What holds up in practice
- A competitor’s keyword set includes accidental rankings, their branded queries, and topics with no value to you. An unfiltered gap report looks like a plan and produces months of low-value work.
Consider a team handed 1,400 gap rows from a keyword tool. The data is accurate and nobody knows where to start.
Nine hundred rows are near-duplicates, two hundred are the competitor’s brand, and two hundred more are topics the business cannot credibly discuss.
I filter to demand, commercial relevance, and credibility before anything reaches a content team. What survives is usually under forty rows.
An unfiltered gap report is worse than none. Work gets done, the rows that mattered stay buried, and a quarter disappears.
A 1,400-row report nobody can act on quietly costs you the quarter it was meant to plan.
Four Things Commonly Assumed About Gap Automation
Which of these describes your last gap analysis?
I meet these four assumptions whenever a tool export is treated as a plan.
| Commonly assumed | What actually holds |
|---|---|
| Everything a competitor ranks for is an opportunity. | Competitors rank accidentally for plenty of terms. Demand plus credibility is what makes a gap real. |
| Tool estimates of competitor rankings are data. | Estimates are samples of an index. A supplied sitemap or URL list is evidence rather than an inference. |
| More rows means a more thorough analysis. | A long list postpones the decision. The value of the analysis is what it removes. |
| Gaps should all be closed. | Deliberately declining topics is what gives a site shape. A site answering everything adjacent is authoritative on nothing. |
| The agent decides what to build. | The agent produces a filtered, scored candidate list. Which rows become pages is a commercial judgement. |
The rows you decline matter as much as the rows you build. An agent that only adds candidates has automated the easy half of the job.
A Worked Example: What the Filters Remove
Consider a raw gap export processed through the agent. The filtering below is illustrative, written to show how much survives. No client engagement is described here.
| Filter stage | Rows removed | Rows remaining |
|---|---|---|
| Raw export | None | 1,400 |
| Remove competitor branded queries | 210 | 1,190 |
| Collapse near-duplicates by ranking overlap | 870 | 320 |
| Remove topics with no commercial relevance | 195 | 125 |
| Remove topics with no credible answer available | 62 | 63 |
| Score and cap to budget | 26 | 37 |
I hand over the final row rather than the first. Thirty-seven scored rows is a plan, and 1,400 rows is a spreadsheet.
An unfiltered gap report is a supermarket receipt presented as a meal plan. Everything on it is real and it tells you nothing about what to cook.
The third row does most of the work. Collapsing on ranking overlap removes duplicates that wording-based grouping keeps as separate opportunities.
How the Agent Runs
I do the filtering in code, so the same input produces the same shortlist every time.
- Crawl the site and map which existing page, if any, addresses each candidate topic.
- Pull real demand data from Search Console rather than from an estimate.
- Read the results page for each candidate, capturing prompts and related searches.
- Take competitor coverage from supplied URL lists or sitemaps rather than from tool estimates.
- Assign each topic a verdict of covered, thin, or missing.
- Remove competitor branded queries, which are never an opportunity.
- Collapse near-duplicates by ranking overlap rather than by wording similarity.
- Remove topics with no commercial relevance and topics with no credible answer available.
- Score the survivors on demand, value, and effort, then cap the list to the available budget.
- Refuse to output any row with no supporting demand data behind it.
How This Engagement Is Scoped
I scope gap automation by the number of competitors compared and the size of the seed set.
- What arrives
- A running agent, a filtered and scored candidate list, and the reason recorded per removal.
- Typical duration
- Four to six weeks to build, including a parallel running period.
- What the client supplies
- Search Console access, competitor URLs or sitemaps, and commercial priorities.
- Included
- Crawl integration, demand pull, filtering rules, scoring, and output formatting.
- Not included
- Deciding which rows to build, and writing the pages.
- What changes the estimate
- Several service lines, several markets, or more than four competitors.
- How it is quoted
- After a short scoping call, once the competitor set is agreed.
Tools and Primary Sources
The references below supply the demand and comparison data. Each opens in a new tab.
- Search Console API supplies the demand evidence the agent filters against.
- Ahrefs Content Gap compares keyword sets across domains, which is one input among several.
- Semrush Keyword Gap performs the same comparison from a different index, useful as a cross-check.
- Google Trends separates a rising topic from a declining one before anything is scored.
Common Questions
Where do competitor URLs come from?
You supply them, usually as a sitemap or an exported list. Supplying them keeps the comparison verifiable and lets you choose which competitors genuinely matter.
Does a gap always mean we should build the page?
No. A gap only matters where demand exists and the business can answer credibly. Some gaps exist because the topic is not worth covering.
How often should gap analysis run?
Quarterly suits most sites, and more often in fast-moving categories. Running it continuously produces noise, since coverage changes slowly.
Why not use a tool’s competitor keyword estimates?
Estimates are samples of a third-party index and they include accidental rankings. A supplied sitemap or URL list is evidence of what a competitor actually publishes.
Discuss This Work
The first call is a free 30-minute consultation with no pitch attached. You receive an honest assessment of where the site stands and what can realistically be built.