Loop, Recursion and Memory Tracing

This page describes two additional PythonTA features:

Print-based debugging for loops and recursion: This feature makes tracing easier by printing the state of each loop iteration or recursive function call in a nicely-formatted table using the tabulate library.
Snapshot-based debugging for visualizing the memory diagram: This feature makes it easier to visualize the Python memory model by leveraging the MemoryViz library.

These additional features are found in the python.debug submodule.

Loop tracing with `AccumulationTable`

The following section will focus on the debugging of while and for loops. This feature uses the python_ta.debug.AccumulationTable as a context manager wrapping a loop. Let’s start with two examples.

Example 1: for loop

# demo.py
from python_ta.debug import AccumulationTable


def calculate_sum_and_averages(numbers: list) -> list:
    """Return the running sums and averages of the given numbers.
    """
    sum_so_far = 0
    list_so_far = []
    avg_so_far = None
    with AccumulationTable(["sum_so_far", "avg_so_far", "list_so_far"]):
        for number in numbers:
            sum_so_far = sum_so_far + number
            avg_so_far = sum_so_far / (len(list_so_far) + 1)
            list_so_far.append((sum_so_far, avg_so_far))

    return list_so_far


if __name__ == '__main__':
    calculate_sum_and_averages([10, 20, 30, 40, 50, 60])

When this file is run, we get the following output:

$ python demo.py
iteration    number    sum_so_far    avg_so_far    list_so_far
-----------  --------  ------------  ------------  ---------------------------------------------------------------------------
          N/A       0             None          []
          10        10            10.0          [(10, 10.0)]
          20        30            15.0          [(10, 10.0), (30, 15.0)]
          30        60            20.0          [(10, 10.0), (30, 15.0), (60, 20.0)]
          40        100           25.0          [(10, 10.0), (30, 15.0), (60, 20.0), (100, 25.0)]
          50        150           30.0          [(10, 10.0), (30, 15.0), (60, 20.0), (100, 25.0), (150, 30.0)]
          60        210           35.0          [(10, 10.0), (30, 15.0), (60, 20.0), (100, 25.0), (150, 30.0), (210, 35.0)]

Example 2: while loop

To use AccumulationTable with while loops, you need to pass in the name of the loop variable when initializing the table.

# demo.py
from python_ta.debug import AccumulationTable


def calculate_sum_up_to_target(target: int) -> list:
    """Return the running sums of the given target.
    """
    number = 0
    sum_so_far = 0
    list_so_far = []
    with AccumulationTable(["number", "sum_so_far", "list_so_far"]):
        while number <= target:
            sum_so_far = sum_so_far + number
            list_so_far = list_so_far + [number]
            number += 1

    return list_so_far


if __name__ == '__main__':
    calculate_sum_up_to_target(5)

When this file is run, we get the following output:

$ python demo.py
iteration    number    sum_so_far    list_so_far
-----------  --------  ------------  ------------------
          0         0             []
          1         0             [0]
          2         1             [0, 1]
          3         3             [0, 1, 2]
          4         6             [0, 1, 2, 3]
          5         10            [0, 1, 2, 3, 4]
          6         15            [0, 1, 2, 3, 4, 5]

API

AccumulationTable.__init__(accumulation_names: list[str], output: None | str = None) → None

Initialize an AccumulationTable context manager for print-based loop debugging.

Parameters:: accumulation_names – a list of the loop accumulator variable names to display.

The AccumulationTable class has the following instance attributes you can access after the with statement.

AccumulationTable.loop_variables: dict[str, list]: A dictionary mapping loop variable variable name to its values across all loop iterations.

AccumulationTable.loop_accumulators: dict[str, list]: A dictionary mapping loop accumulator variable name to its values across all loop iterations.

For example:

from python_ta.debug import AccumulationTable


def calculate_sum_and_averages(numbers: list) -> list:
    """Return the running sums and averages of the given numbers.
    """
    sum_so_far = 0
    list_so_far = []
    avg_so_far = None
    with AccumulationTable(["sum_so_far", "avg_so_far", "list_so_far"]) as table:
        for number in numbers:
            sum_so_far = sum_so_far + number
            avg_so_far = sum_so_far / (len(list_so_far) + 1)
            list_so_far.append((sum_so_far, avg_so_far))

    print(table.loop_accumulators)
    return list_so_far

You also have the option to pass in a file path as an attribute to the AccumulationTable object. In this case, the table will be appended to the file instead of being written the console. For example:

from python_ta.debug import AccumulationTable


def calculate_sum_and_averages(numbers: list) -> list:
    """Return the running sums and averages of the given numbers.
    """
    sum_so_far = 0
    list_so_far = []
    avg_so_far = None
    output_file = 'output.txt'
    with AccumulationTable(["sum_so_far", "avg_so_far", "list_so_far"], output_file) as table:
        for number in numbers:
            sum_so_far = sum_so_far + number
            avg_so_far = sum_so_far / (len(list_so_far) + 1)
            list_so_far.append((sum_so_far, avg_so_far))

    return list_so_far

Current limitations

The AccumulationTable is a new PythonTA feature and currently has the following known limitations:

AccumulationTable uses sys.settrace to update variable state, and so is not compatible with other libraries (e.g. debuggers, code coverage tools).
The AccumulationTable context manager can only log the execution of one for loop. To log the state of multiple for loops, each must be wrapped in a separate with statement and fresh AccumulationTable instance.

Recursion tracing with `RecursionTable`

This section will discuss the debugging of recursive functions. This feature uses the python_ta.debug.RecursionTable class as a context manager wrapping a recursive function.

Example usage

# demo.py
from python_ta.debug import RecursionTable

def factorial(n: int) -> int:
    """Calculate the factorial of n."""
    if n == 0:
        return 1
    return n * factorial(n - 1)

def trace_factorial(number: int) -> None:
    "Trace a recursively defined factorial function using RecursionTable."
    with RecursionTable("factorial"):
        factorial(number)

if __name__ == '__main__':
    trace_factorial(4)

When this file is run, we get the following output:

$ python demo.py
n    return value    called by
---  --------------  ------------
4    24              N/A
3    6               factorial(4)
2    2               factorial(3)
1    1               factorial(2)
0    1               factorial(1)

API

RecursionTable.__init__(function_name: str) → None: Initialize a RecursionTable context manager for print-based recursive debugging of <function_name>.

The RecursionTable class has the following methods you can access after the with statement.

RecursionTable.get_recursive_dict() → dict[str, list]: Use the instance variables that define the table to create a final dictionary which directly represents the table.

For example:

# demo.py
from python_ta.debug import RecursionTable

def factorial(n: int) -> int:
    """Calculate the factorial of n."""
    if n == 0:
        return 1
    return n * factorial(n - 1)

def trace_factorial(number: int) -> None:
    "Trace a recursively defined factorial function using RecursionTable."
    with RecursionTable("factorial") as table:
        factorial(number)

    traced_data = table.get_recursive_dict()
    print(traced_data)

Tracing with user-defined classes

Both AccumulationTable and RecursionTable call str on objects to display table entries. If you plan to use instances of a user-defined class in these tables (for example, a Tree class when tracing a recursive Tree method), we recommend implementing the __str__ method in your class to ensure a meaningful output is displayed.

Current limitations

The RecursionTable is a new PythonTA feature and currently has the following known limitations:

RecursionTable uses sys.settrace to update variable state, and so is not compatible with other libraries (e.g. debuggers, code coverage tools).
Only one function can be traced per use of RecursionTable, and so mutually-recursive functions are not supported.

Tracing the Python Memory Model

The following section will focus on tracing the Python memory model. This feature uses the python_ta.debug.SnapshotTracer as a context manager to visualize program memory.

Example usage

# demo.py
from python_ta.debug import SnapshotTracer


def func_multi_line(output_directory: str = None) -> None:
    """
    Function for testing SnapshotTracer
    """
    with SnapshotTracer(
        output_directory=output_directory,
        include_frames=("func_multi_line",),
        exclude_vars=("output_directory",),
        memory_viz_args=MEMORY_VIZ_ARGS,
    ):
        num = 123
        some_string = "Hello, world"
        num2 = 321
        arr = [some_string, "string 123321"]


if __name__ == '__main__':
    func_multi_line()

When this function runs, the variables within func_multi_line are captured, and memory models are outputted to output_directory for each line of code. For the expected output, refer to the snapshots in tests/test_debug/snapshot_tracer_testing_snapshots/func_multi_line.

API

SnapshotTracer.__init__(output_directory: str | None = None, webstepper: bool = False, **kwargs) → None

Initialize a context manager for snapshot-based debugging.

Parameters:

output_directory – The directory to save the snapshots, defaulting to the current directory. Note: Use this argument instead of the –output flag in memory_viz_args to specify the output directory. The directory will be created if it does not exist.
webstepper – Opens a MemoryViz Webstepper webpage to interactively visualize the resulting memory diagrams.
**kwargs – All other keyword arguments are passed to python.debug.snapshot. Refer to the snapshot function for more details.

Current Limitations

The SnapshotTracer has the following limitations:

Due to differences in Python interpreters, this context manager only works with Python versions >= 3.10.
The context manager does not step into any function calls. Calling functions within the traced function may lead to undefined behavior.
SnapshotTracer uses sys.settrace to update variable states, and therefore is not compatible with other libraries (e.g., debuggers, code coverage tools).

Loop, Recursion and Memory Tracing

Loop tracing with AccumulationTable

Example 1: for loop

Example 2: while loop

API

Current limitations

Recursion tracing with RecursionTable

Example usage

API

Tracing with user-defined classes

Current limitations

Tracing the Python Memory Model

Example usage

API

Current Limitations

Loop tracing with `AccumulationTable`

Recursion tracing with `RecursionTable`