Automated Sprawl Reports
========================

A Sprawl report is a comprehensive automated analysis of a website. Content Chimera
crawls the site, analyzes the data, and produces a multi-section report with charts,
screenshots, and written analysis — all generated automatically. Instead of spending
hours building a report by hand, you get a ready-made starting point that covers the
most important aspects of a site's content.

.. contents:: On this page
   :local:
   :depth: 2


What the Report Includes
-------------------------

A Sprawl report is built from the same data you would use to construct a manual report,
but the system assembles it for you. A typical Sprawl report includes:

- **Site structure overview** — How content is distributed across folders and subfolders,
  with charts showing where the bulk of pages live.
- **Content type distribution** — A breakdown of the types of content found on the site
  (HTML pages, PDFs, images, etc.).
- **Page template analysis** — Which page templates are used most frequently, helping
  identify patterns in how the site is built.
- **Key findings and recommendations** — Written analysis highlighting notable patterns,
  potential issues, and areas worth investigating further.
- **Supporting charts and screenshots** — Visual evidence backing up the written
  findings.

The report is assembled as a standard Content Chimera report, so you can edit it after
generation — adding your own sections, removing components that aren't relevant, or
refining the written analysis.


How to Generate a Sprawl Report
---------------------------------

Sprawl reports are currently available through the MCP interface. The pipeline crawls
the site, processes the data, and generates the report in one end-to-end run.

.. admonition:: MCP
   :class: tip

   Ask your AI assistant to generate a Sprawl report:

      *"Run a Sprawl report for https://example.com"*

   If the site has already been crawled and you want to regenerate the report without
   re-crawling:

      *"Re-run the Sprawl report for this extent, skip the crawl"*

   Your AI assistant will show a confirmation prompt before proceeding, because the
   pipeline includes LLM analysis which has an associated cost.

   .. raw:: html

      <details>
      <summary>Tool reference</summary>
      <p>Tool: <code class="docutils literal notranslate">run-long-pipeline</code> with <code class="docutils literal notranslate">pipeline: "sprawl"</code></p>
      </details>

.. important::

   The Sprawl pipeline requires confirmation before it begins. This is because the
   pipeline includes AI-powered analysis steps that incur cost. The confirmation prompt
   tells you what will happen so you can decide whether to proceed.


What to Expect
--------------

The Sprawl pipeline runs through several stages:

1. **Crawl** — The site is crawled to gather all pages and their content.
2. **Process** — Pages are enhanced with extracted metadata (folders, file types, etc.)
   and the data is optimized for analysis.
3. **Analyze** — The graph database and semantic database are loaded, and the crawl is
   summarized.
4. **Generate report** — Charts are created, written analysis is produced, and everything
   is assembled into a report.

For a moderately-sized site, this process can take anywhere from a few minutes to over an
hour depending on the number of pages.

You can track progress at any time using the job status.

.. admonition:: MCP
   :class: tip

   Ask your AI assistant to check on progress:

      *"What's the status of my Sprawl report?"*

   .. raw:: html

      <details>
      <summary>Tool reference</summary>
      <p>Tool: <code class="docutils literal notranslate">job-status</code> with the <code class="docutils literal notranslate">job_chain_id</code> returned when the pipeline started.</p>
      </details>

When the pipeline completes, the report appears in the **Reports** section of your
extent. From there you can view it, edit it, present it, or print it to PDF.