Apply Sphinx’s Functionality to Create Documentation for Your Next Data Science Project

Well-written documentation is crucial for almost any data science project as it enhances clarity, facilitates collaboration, and ensures reproducibility. Clear and concise documentation provides context for the project’s objectives, methodologies, and findings, making it easier for other team members (especially, newbies) and stakeholders to understand the meaning behind work done. Furthermore, documentation serves as a reference for future enhancements or troubleshooting, reducing time spent on re-explaining or even refreshing the main concepts.

Sounds attractive, isn’t it? But do you know that you can create functional documentation via Sphinx documentation tool in a consistent style simply by using docstrings? If you don’t know too much about Sphinx’s functionality yet, this post can help you to figure it out.

Few words about docstrings
Docstrings are the comment blocks that appear in any class, class method, and function within the code.

Three main docstring formats are officially supported via Sphinx: Google [1], NumPy [2], and reStructuredText (reST) [3]. Which one to choose is up to you, but in this post I’ll work with the reST format, because of its versatility.

In this article, I’ll introduce you to three most impressive functionalities of Sphinx’s tool, which can automatically generate documentation for Python modules. Before considering these three cases I assume that you’ve already created a documentation directory and installed Sphinx on your machine. If not, please, read the TDS article on how to install and set up Sphinx first [4].

After installing Sphinx, create a new Sphinx project by command `sphinx-quickstart`. Follow the prompts to set up your project. This will populate your directory with several files, including `conf.py` and `index.rst`.

Case 1. Use cross-references for quick navigation

According to the official website of Sphinx, one of its most useful features is creating automatic cross-references through semantic cross-referencing roles. Cross-references can be used to link to functions, classes, modules, and even sections within your documentation.

For instance, the cross reference to an object description, such as :func:`.name`, will create a link to the place where name() is documented.

Let’s examine how it is in practice. Imagine that we have a simple Python module called mymodule.py with two basic functions inside.

First function is about summing two numbers:

def add(a: int, b: int) -> int:
    """
    Add two numbers.

    :param a: First number.
    :param b: Second number.
    :return: Sum of a and b.
    """
    return a + b

Second is about subtracting one number from the other:

def subtract(c: int, d: int) -> int:
    """
    Subtract two numbers.

    :param c: First number.
    :param d: Second number.
    :return: Subtracting d from c.
    """
    return c - d

It’s possible to use :func: to create cross-references to these two functions within the documentation (:func:.add, :func:.subtract). Let’s create another file (main.py), which will use the functions from mymodule.py. You can add docstrings here if you want to document this file as well:

from mymodule import add, subtract
def main():
   """
   Main function to demonstrate the use of two functions.

   It uses :func:`.add` and :func:`.subtract` functions from mymodule.py.
   """
   # Call the first function
   first = add(2,3)
   print(first)

   # Call the second function
   second = subtract(9,8)
   print(second)

if __name__ == "__main__":
   main()

To automatically generate documentation from your code, you can enable the autodoc extension in your conf.py file. Add 'sphinx.ext.autodoc' to the extensions list:

extensions = ['sphinx.ext.autodoc']

Make sure to include the path to your module so that Sphinx can find it. Add the following lines at the top of conf.py:

import os
import sys
sys.path.insert(0,  os.path.abspath('../src')) # mymodule.py and main.py are located in src folder in documentation directory

Then we need to generate .rst files of our Python packages. They are Sphinx’s own format and need to be generated before making HTML-files. It’s faster to use the apidoc command to deal with .rst. Run in the terminal:

sphinx-apidoc -o source src

Here -o source defines the directory to place the output files, and src sets the location of Python modules we need to describe. After running this command, newly generated .rst files will appear in your folder.

Finally, navigate to your documentation’s folder and run:

make html

This will generate HTML documentation in the _build/html directory. Open the generated HTML files in a web browser. You should see your documentation with cross-references to the add and subtract functions:

Click here on the function names and you will be taken to a page with their description:

Case 2. Add links to external resources

In addition to the ability to insert cross-references, Sphinx allows you to add links to external resources. Below is an example of how you can create a function in mymodule.py file that utilizes the built-in abs() function to demonstrate how it’s possible to add a link to the official Python documentation in its docstrings:

def calculate_distance(point1, point2):
   """
   Calculate the distance between two points in a 2D space.

   This function uses the built-in `abs()` function to compute the absolute     
   differences in the x and y coordinates of the two points.

   For more details, see the official Python documentation for `abs()`:
   `abs() `_.
   """
   a, b = point1
   c, d = point2

   # Calculate the differences in x and y coordinates
   delta_x = abs(c - a)
   delta_y = abs(d - b)

   # Calculate the Euclidean distance using the Pythagorean theorem
   distance = (delta_x**2 + delta_y**2) ** 0.5
   return distance

Running make html command for this case provide you the following output:

Case 3. Create special directives and examples for better visual effects

In Sphinx you can create short paragraphs with different admonitions, messages, and warnings, as well as with concrete examples of obtained results. Let’s enrich our module with a note directive and example.

def calculate_distance(point1, point2):
   """
   Calculate the distance between two points in a 2D space.

   This function uses the built-in `abs()` function to compute the absolute
   differences in the x and y coordinates of the two points.

   For more details, see the official Python documentation for `abs()`:
   `abs() `_.

   Example:
       >>> calculate_distance((1, 2), (4, 6))
       5.0

   .. note::
       There is a function that calculates the Euclidean distance directly - `math.hypot() `_.
   """
   a, b = point1
   c, d = point2

   # Calculate the differences in x and y coordinates
   delta_x = abs(c - a)
   delta_y = abs(d - b)

   # Calculate the Euclidean distance using the Pythagorean theorem
   distance = (delta_x**2 + delta_y**2) ** 0.5
   return distance

And the resulting HTML page looks as follows:

Therefore, for adding any example within the docstrings you need to use >>>. And to specify a note there, just use .. note::. A good thing is that you might add links to external resources inside the note.

Conclusion

Thorough documentation allows others not only to better understand the subject of reading, but to deeply interact with it, which is essential for technical and scientific documentation. Overall, good documentation promotes efficient knowledge transfer and helps maintain the project’s longevity, ultimately contributing to its success and impact.

In this post we considered how to create a simple, yet well-written documentation using Sphinx documentation tool. Not only did we learn how to create a Sphinx project from scratch, but also realized how to use its functionality, including cross-references, links to external resources, and special directives. Hope, you found this knowledge helpful for yourself!

Note: all images in the article were made by author.