TermStreamer

`TermStreamer`

TermStreamer()

A term streamer that can handle large expressions, by streaming terms to and from disk.

Methods

Name	Description
`__add__`	Add two term streamers together, returning the result.
`__iadd__`	Add another term streamer to this one.
`__new__`	Create a new term streamer with a given path for its files, the maximum size of the memory buffer and the number of cores.
`clear`	Clear the term streamer.
`fits_in_memory`	Check if the term stream fits in memory.
`get_byte_size`	Get the byte size of the term stream.
`get_num_terms`	Get the number of terms in the stream.
`load`	Load terms and their state from a binary file into the term streamer
`map`	Apply a transformer to all terms in the stream.
`map_single_thread`	Apply a transformer to all terms in the stream using a single thread.
`normalize`	Sort and fuse all terms in the stream.
`push`	Push an expression to the term stream.
`save`	Export terms and their state to a binary file
`to_expression`	Convert the term stream into an expression

`add`

TermStreamer.__add__(other: TermStreamer) -> TermStreamer

Add two term streamers together, returning the result.

`iadd`

TermStreamer.__iadd__(other: TermStreamer) -> None

Add another term streamer to this one.

`new`

TermStreamer.__new__(
    path: Optional[str] = None,
    max_mem_bytes: Optional[int] = None,
    n_cores: Optional[int] = None,
) -> TermStreamer

Create a new term streamer with a given path for its files, the maximum size of the memory buffer and the number of cores.

`clear`

TermStreamer.clear() -> None

Clear the term streamer.

`fits_in_memory`

TermStreamer.fits_in_memory() -> bool

Check if the term stream fits in memory.

`get_byte_size`

TermStreamer.get_byte_size() -> int

Get the byte size of the term stream.

`get_num_terms`

TermStreamer.get_num_terms() -> int

Get the number of terms in the stream.

`load`

TermStreamer.load(filename: str, conflict_fn: Optional[Callable[[str], str]] = None) -> int

Load terms and their state from a binary file into the term streamer. The number of terms loaded is returned.

The state will be merged with the current one. If a symbol has conflicting attributes, the conflict can be resolved using the renaming function conflict_fn.

A term stream can be exported using TermStreamer.save.

`map`

TermStreamer.map(f: Transformer, stats_to_file: Optional[str] = None) -> TermStreamer

Apply a transformer to all terms in the stream.

Parameters

f (Transformer) The transformer to apply.
stats_to_file (str, optional) If set, the output of the stats transformer will be written to a file in JSON format.

`map_single_thread`

TermStreamer.map_single_thread(f: Transformer, stats_to_file: Optional[str] = None) -> TermStreamer

Apply a transformer to all terms in the stream using a single thread.

Parameters

f (Transformer) The transformer to apply.
stats_to_file (str, optional) If set, the output of the stats transformer will be written to a file in JSON format.

`normalize`

TermStreamer.normalize() -> None

Sort and fuse all terms in the stream.

`push`

TermStreamer.push(expr: Expression) -> None

Push an expression to the term stream.

`save`

TermStreamer.save(filename: str, compression_level: int = 9) -> None

Export terms and their state to a binary file. The resulting file can be read back using TermStreamer.load or by using Expression.load. In the latter case, the whole term stream will be read into memory as a single expression.

`to_expression`

TermStreamer.to_expression() -> Expression

Convert the term stream into an expression. This may exceed the available memory.