Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: add test for multiple interrupts and tasks #2941

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 58 additions & 0 deletions libs/langgraph/tests/test_pregel.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import enum

Check notice on line 1 in libs/langgraph/tests/test_pregel.py

View workflow job for this annotation

GitHub Actions / benchmark

Benchmark results

......................................... fanout_to_subgraph_10x: Mean +- std dev: 61.5 ms +- 1.2 ms ......................................... fanout_to_subgraph_10x_sync: Mean +- std dev: 53.5 ms +- 0.7 ms ......................................... fanout_to_subgraph_10x_checkpoint: Mean +- std dev: 74.9 ms +- 0.8 ms ......................................... fanout_to_subgraph_10x_checkpoint_sync: Mean +- std dev: 95.8 ms +- 1.6 ms ......................................... fanout_to_subgraph_100x: Mean +- std dev: 604 ms +- 22 ms ......................................... fanout_to_subgraph_100x_sync: Mean +- std dev: 522 ms +- 12 ms ......................................... fanout_to_subgraph_100x_checkpoint: Mean +- std dev: 749 ms +- 10 ms ......................................... fanout_to_subgraph_100x_checkpoint_sync: Mean +- std dev: 963 ms +- 16 ms ......................................... react_agent_10x: Mean +- std dev: 30.8 ms +- 0.7 ms ......................................... react_agent_10x_sync: Mean +- std dev: 22.9 ms +- 0.3 ms ......................................... react_agent_10x_checkpoint: Mean +- std dev: 38.2 ms +- 0.7 ms ......................................... react_agent_10x_checkpoint_sync: Mean +- std dev: 36.9 ms +- 0.3 ms ......................................... react_agent_100x: Mean +- std dev: 343 ms +- 6 ms ......................................... react_agent_100x_sync: Mean +- std dev: 273 ms +- 3 ms ......................................... react_agent_100x_checkpoint: Mean +- std dev: 636 ms +- 6 ms ......................................... react_agent_100x_checkpoint_sync: Mean +- std dev: 620 ms +- 6 ms ......................................... wide_state_25x300: Mean +- std dev: 23.3 ms +- 0.5 ms ......................................... wide_state_25x300_sync: Mean +- std dev: 15.3 ms +- 0.1 ms ......................................... wide_state_25x300_checkpoint: Mean +- std dev: 249 ms +- 13 ms ......................................... wide_state_25x300_checkpoint_sync: Mean +- std dev: 245 ms +- 13 ms ......................................... wide_state_15x600: Mean +- std dev: 27.4 ms +- 0.6 ms ......................................... wide_state_15x600_sync: Mean +- std dev: 17.8 ms +- 0.2 ms ......................................... wide_state_15x600_checkpoint: Mean +- std dev: 429 ms +- 14 ms ......................................... wide_state_15x600_checkpoint_sync: Mean +- std dev: 424 ms +- 13 ms ......................................... wide_state_9x1200: Mean +- std dev: 27.2 ms +- 0.5 ms ......................................... wide_state_9x1200_sync: Mean +- std dev: 17.8 ms +- 0.1 ms ......................................... wide_state_9x1200_checkpoint: Mean +- std dev: 278 ms +- 13 ms ......................................... wide_state_9x1200_checkpoint_sync: Mean +- std dev: 274 ms +- 12 ms

Check notice on line 1 in libs/langgraph/tests/test_pregel.py

View workflow job for this annotation

GitHub Actions / benchmark

Comparison against main

+-----------------------------------------+---------+-----------------------+ | Benchmark | main | changes | +=========================================+=========+=======================+ | fanout_to_subgraph_100x | 631 ms | 604 ms: 1.04x faster | +-----------------------------------------+---------+-----------------------+ | fanout_to_subgraph_100x_checkpoint | 776 ms | 749 ms: 1.04x faster | +-----------------------------------------+---------+-----------------------+ | react_agent_100x_checkpoint_sync | 641 ms | 620 ms: 1.03x faster | +-----------------------------------------+---------+-----------------------+ | react_agent_100x_checkpoint | 652 ms | 636 ms: 1.02x faster | +-----------------------------------------+---------+-----------------------+ | wide_state_25x300_checkpoint_sync | 250 ms | 245 ms: 1.02x faster | +-----------------------------------------+---------+-----------------------+ | fanout_to_subgraph_10x_checkpoint_sync | 97.7 ms | 95.8 ms: 1.02x faster | +-----------------------------------------+---------+-----------------------+ | wide_state_9x1200_checkpoint_sync | 279 ms | 274 ms: 1.02x faster | +-----------------------------------------+---------+-----------------------+ | fanout_to_subgraph_100x_checkpoint_sync | 978 ms | 963 ms: 1.02x faster | +-----------------------------------------+---------+-----------------------+ | wide_state_15x600_checkpoint_sync | 430 ms | 424 ms: 1.01x faster | +-----------------------------------------+---------+-----------------------+ | react_agent_10x_sync | 23.1 ms | 22.9 ms: 1.01x faster | +-----------------------------------------+---------+-----------------------+ | wide_state_15x600_sync | 17.9 ms | 17.8 ms: 1.01x faster | +-----------------------------------------+---------+-----------------------+ | react_agent_10x_checkpoint | 38.5 ms | 38.2 ms: 1.01x faster | +-----------------------------------------+---------+-----------------------+ | react_agent_100x_sync | 275 ms | 273 ms: 1.01x faster | +-----------------------------------------+---------+-----------------------+ | wide_state_9x1200_sync | 17.9 ms | 17.8 ms: 1.00x faster | +-----------------------------------------+---------+-----------------------+ | wide_state_25x300_sync | 15.4 ms | 15.3 ms: 1.00x faster | +-----------------------------------------+---------+-----------------------+ | react_agent_10x_checkpoint_sync | 37.0 ms | 36.9 ms: 1.00x faster | +-----------------------------------------+---------+-----------------------+ | Geometric mean | (ref) | 1.01x faster | +-----------------------------------------+---------+-----------------------+ Benchmark hidden because not significant (12): wide_state_9x1200_checkpoint, wide_state_25x300_checkpoint, wide_state_15x600_checkpoint, fanout_to_subgraph_100x_sync, react_agent_100x, fanout_to_subgraph_10x, wide_state_9x1200, wide_state_15x600, wide_state_25x300, react_agent_10x, fanout_to_subgraph_10x_checkpoint, fanout_to_subgraph_10x_sync
import json
import logging
import operator
Expand Down Expand Up @@ -5300,3 +5300,61 @@
{"node_a": [{"foo": "a1"}, {"foo": "a2"}]},
{"node_b": {"foo": "b"}},
]


def test_falsy_return_from_task() -> None:
"""Test with a falsy return from a task."""
checkpointer = MemorySaver()

@task
def falsy_task() -> bool:
return False
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

triggers exception on any falsy value when used in conjunction with HIL


@entrypoint(checkpointer=checkpointer)
def graph(state: dict) -> dict:
"""React tool."""
task_result = falsy_task().result()

Check failure on line 5316 in libs/langgraph/tests/test_pregel.py

View workflow job for this annotation

GitHub Actions / cd libs/langgraph / lint #3.12

Ruff (F841)

tests/test_pregel.py:5316:9: F841 Local variable `task_result` is assigned to but never used
human_value = interrupt("test")

Check failure on line 5317 in libs/langgraph/tests/test_pregel.py

View workflow job for this annotation

GitHub Actions / cd libs/langgraph / lint #3.12

Ruff (F841)

tests/test_pregel.py:5317:9: F841 Local variable `human_value` is assigned to but never used

configurable = {"configurable": {"thread_id": uuid.uuid4()}}
graph.invoke({"a": 5}, configurable)
graph.invoke(Command(resume="123"), configurable)


def test_multiple_interrupts_imperative() -> None:
"""Test multiple interrupts with an imperative API."""
from langgraph.checkpoint.memory import MemorySaver
from langgraph.func import entrypoint, task

checkpointer = MemorySaver()
counter = 0

@task
def double(x: int) -> int:
"""Increment the counter."""
nonlocal counter
counter += 1
return 2 * x

@entrypoint(checkpointer=checkpointer)
def graph(state: dict) -> dict:
"""React tool."""

values = []

for idx in [1, 2, 3]:
values.extend([double(idx).result(), interrupt({"a": "boo"})])

return {"values": values}

configurable = {"configurable": {"thread_id": uuid.uuid4()}}
graph.invoke({}, configurable)
# Currently fails when double accepts a variable
graph.invoke(Command(resume="a"), configurable)
graph.invoke(Command(resume="b"), configurable)
result = graph.invoke(Command(resume="c"), configurable)
# `double` value should be cached appropriately when used w/ `interrupt`
assert result == {
"values": [2, "a", 4, "b", 6, "c"],
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fails currently

}
assert counter == 3
Loading