Server side search integration

Read the Docs provides server side search (SSS) in replace of the default search engine of your site. To accomplish this, Read the Docs parses the content directly from your HTML pages [*].

If you are the author of a theme or a static site generator you can read this document, and follow some conventions in order to improve the integration of SSS with your theme/site.

Indexing

The content of the page is parsed into sections, in general, the indexing process happens in three steps:

Identify the main content node.
Remove any irrelevant content from the main node.
Parse all sections inside the main node.

Read the Docs makes use of ARIA roles and other heuristics in order to process the content.

Tip

Following the ARIA conventions will also improve the accessibility of your site. See also https://webaim.org/techniques/semanticstructure/.

Main content node

The main content should be inside a <main> tag or an element with role=main, and there should only be one per page. This node is the one that contains all the page content to be indexed. Example:

<html>
   <head>
      ...
   </head>
   <body>
      <div>
         This content isn't processed
      </div>

      <div role="main">
         All content inside the main node is processed
      </div>

      <footer>
         This content isn't processed
      </footer>
   </body>
</html>

If a main node isn’t found, we try to infer the main node from the parent of the first section with a h1 tag. Example:

<html>
   <head>
      ...
   </head>
   <body>
      <div>
         This content isn't processed
      </div>

      <div id="parent">
         <h1>First title</h1>
         <p>
            The parent of the h1 title will
            be taken as the main node,
            this is the div tag.
         </p>

         <h2>Second title</h2>
         <p>More content</p>
      </div>
   </body>
</html>

If a section title isn’t found, we default to the body tag. Example:

<html>
   <head>
      ...
   </head>
   <body>
      <p>Content</p>
   </body>
</html>

Irrelevant content

If you have content inside the main node that isn’t relevant to the page (like navigation items, menus, or search box), make sure to use the correct role or tag for it.

Roles to be ignored:

navigation
search

Tags to be ignored:

nav

Special rules that are derived from specific documentation tools applied in the generic parser:

.linenos, .lineno (line numbers in code-blocks, comes from both MkDocs and Sphinx)
.headerlink (added by Sphinx to links in headers)
.toctree-wrapper (added by Sphinx to the table of contents generated from the toctree directive)

Example:

<div role="main">
   ...
   <nav role="navigation">
      ...
   </nav>
   ...
</div>

Sections

Sections are stored in a dictionary composed of an id, title and content key.

Sections are defined as:

h1-h7, all content between one heading level and the next header on the same level is used as content for that section.
dt elements with an id attribute, we map the title to the dt element and the content to the dd element.

All sections have to be identified by a DOM container’s id attribute, which will be used to link to the section. How the id is detected varies with the type of element:

h1-h7 elements use the id attribute of the header itself if present, or its section parent (if exists).
dt elements use the id attribute of the dt element.

To avoid duplication and ambiguous section references, all indexed dl elements are removed from the DOM before indexing of other sections happen.

Here is an example of how all content below the title, until a new section is found, will be indexed as part of the section content:

<div role="main">
   <h1 id="section-title">
      Section title
   </h1>
   <p>
      Content to be indexed
   </p>
   <ul>
      <li>This is also part of the section and will be indexed as well</li>
   </ul>

   <h2 id="2">
      This is the start of a new section
   </h2>
   <p>
      ...
   </p>

   ...

   <header>
      <h1 id="3">This is also a valid section title</h1>
   </header>
   <p>
      Thi is the content of the third section.
   </p>
</div>

Sections can be contained in up to two nested tags, and can contain other sections (nested sections). Note that the section content still needs to be below the section title. Example:

<div role="main">
   <div class="section">
      <h1 id="section-title">
         Section title
      </h1>
      <p>
         Content to be indexed
      </p>
      <ul>
         <li>This is also part of the section</li>
      </ul>

      <div class="section">
         <div id="nested-section">
            <h2>
               This is the start of a sub-section
            </h2>
            <p>
               With the h tag within two levels
            </p>
         </div>
      </div>
   </div>
</div>

Note

The title of the first section will be the title of the page, falling back to the title tag.

Other special nodes

Anchors: If the title of your section contains an anchor, wrap it in a headerlink class, so it won’t be indexed as part of the title.

<h2>
   Section title
   <a class="headerlink" title="Permalink to this headline">¶</a>
</h2>

Code blocks: If a code block contains line numbers, wrap them in a linenos or lineno class, so they won’t be indexed as part of the code.

<table class="highlighttable">
   <tr>
      <td class="linenos">
         <div class="linenodiv">
            <pre>1 2 3</pre>
         </div>
      </td>

      <td class="code">
         <div class="highlight">
            <pre>First line
Second line
Third line</pre>
         </div>
      </td>
   </tr>
</table>

Overriding the default search

Static sites usually have their own static search index, and search results are retrieved via JavaScript. Read the Docs overrides the default search for Sphinx projects only, and provides a fallback to the original search in case of an error or no results.

Sphinx

Sphinx’s basic theme provides the static/searchtools.js file, which initializes search with the Search.init() method. Read the Docs overrides the Search.query method and makes use of Search.output.append to add the results. A simplified example looks like this:

var original_search = Search.query;

function search_override(query) {
   var results = fetch_resuls(query);
   if (results) {
      for (var i = 0; i < results.length; i += 1) {
         var result = process_result(results[i]);
         Search.output.append(result);
      }
   } else {
      original_search(query);
   }
}

Search.query = search_override;

$(document).ready(function() {
   Search.init();
});

Highlights from results will be in a span tag with the highlighted class (This is a <span class="highlighted">result</span>). If your theme works with the search from the basic theme, it will work with Read the Docs’ SSS.

Other static site generators

All projects that have HTML pages that follow the conventions described in this document can make use of the server side search from the dashboard or by calling the API.

Supporting more themes and static site generators

All themes that follow these conventions should work as expected. If you think other generators or other conventions should be supported, or content that should be ignored or have an especial treatment, or if you found an error with our indexing, let us know in our issue tracker.