Replay & Debugging

AI agent instructions

STOP. Do not read past this section until you have read and followed /quickstart.md.Goal: produce a reliable SDK automation script. The intended authoring flow is generate then edit, not write browser automation from scratch.Do not hand-write SDK code with element IDs, selectors, or form fields from memory. Element IDs, selectors, and field mappings must come from a live observe() call, CLI notte page observe output, or generated workflow code. Guessed browser targets fail on real pages.Use the Notte skill and CLI to inspect the live site, validate actions/selectors, handle auth/session state, and export workflow code with notte sessions workflow-code before using SDK docs or SDK code.SDK reference pages are for understanding, running, or editing generated workflow code. They are not the starting point for manually authoring the initial browser automation. SDK-first code is guesswork on real-world pages with dynamic selectors, auth state, CAPTCHAs, and anti-bot behavior.

Debug agent executions with MP4 replays that show exactly what the agent saw and did during its run.

Agent Replay

Get a visual replay of the agent’s execution. The session.replay() method returns a ReplayResponse with a presigned URL you can use to download the MP4 file:

from notte_sdk import NotteClient

client = NotteClient()

with client.Session() as session:
    agent = client.Agent(session=session)
    result = agent.run(task="Navigate and extract data")

# Get MP4 replay (returns presigned URL)
replay = session.replay()

# Download to file
replay.download("agent_run.mp4")

What’s Included

Agent replays capture:

Visual recording of every page the agent visited
Actions taken by the agent (clicks, fills, navigations)
Page transitions and loading states
Element highlights showing what the agent interacted with
Timestamps for each action

Viewing Replays

In Jupyter Notebook

Display the replay inline:

view_in_notebook.py

with client.Session() as session:
    agent = client.Agent(session=session)
    agent.run(task="Complete task")

replay = session.replay()
replay.download("notebook_replay.mp4")

Save to File

Save for later viewing or sharing:

save_to_file.py

with client.Session() as session:
    agent = client.Agent(session=session)
    agent.run(task="Complete task")

replay = session.replay()
replay.download("debug_run.mp4")

# With custom path
output_dir = Path("./replays")
output_dir.mkdir(exist_ok=True)
replay.download(output_dir / f"agent_{agent.agent_id}.mp4")

In Browser

Use the Notte Console to view replays:

view_in_browser.py

result = agent.run(task="Complete task")

# Get replay URL
print(f"View replay: https://console.notte.cc/sessions/{session.session_id}")

Debugging Workflow

1. Run Agent

Execute your agent task:

run_agent.py

agent = client.Agent(session=session, max_steps=15)
result = agent.run(task="Find and click the pricing button")

2. Check If Failed

Identify failures:

check_if_failed.py

agent = client.Agent(session=session)
result = agent.run(task="Complete task")

if not result.success:
    print(f"Agent failed: {result.answer}")
print(f"Completed {len(result.steps)} steps")

3. Inspect Steps

Review what the agent did:

from notte_sdk import NotteClient

client = NotteClient()

with client.Session() as session:
    agent = client.Agent(session=session)
    result = agent.run(task="Complete task")

    for i, step in enumerate(result.steps):
        print(f"Step {i + 1}: {step['action']}")
        print(f"  Success: {step['success']}")
        if not step["success"]:
            print(f"  Error: {step['message']}")

4. Watch Replay

See the visual execution:

watch_replay.py

with client.Session() as session:
    agent = client.Agent(session=session)
    agent.run(task="Complete task")

replay = session.replay()
replay.download("debug_replay.mp4")

# Watch where it failed
# Identify if element selectors were wrong
# Check if page loaded correctly

5. Fix and Retry

Adjust based on findings:

from notte_sdk import NotteClient

client = NotteClient()

with client.Session() as session:
    # Maybe increase max_steps
    agent = client.Agent(session=session, max_steps=25)

    # Or adjust the task
    result = agent.run(task="Click the 'View Pricing' button in the main navigation")

Common Issues

Agent Can’t Find Element

Symptoms:

Agent fails with “Element not found”
Replay shows element exists but wasn’t clicked

Debug:

from notte_sdk import NotteClient

client = NotteClient()

with client.Session() as session:
    agent = client.Agent(session=session)
    result = agent.run(task="Complete task")

replay = session.replay()
replay.download("missing_element.mp4")

# Watch replay and check:
# - Did page finish loading?
# - Is element visible/in viewport?
# - Is element covered by another element?

Fix:

fix_missing_element.py

agent = client.Agent(session=session)
# Be more specific in task
task = "Wait for the page to load, then click the blue 'Submit' button at the bottom"

# Or use session actions first
session.execute(type="goto", url="https://example.com")
time.sleep(2)  # Wait for load
result = agent.run(task="Click submit button")

Agent Takes Wrong Action

Symptoms:

Agent clicks wrong button
Agent fills wrong form field

Debug:

from notte_sdk import NotteClient

client = NotteClient()

with client.Session() as session:
    agent = client.Agent(session=session)
    result = agent.run(task="Fill the email field")

    # Check what agent saw
    for step in result.steps:
        print(f"Action: {step['action']}")
        print(f"Reasoning: {step.get('reasoning', 'N/A')}")

Fix:

from notte_sdk import NotteClient

client = NotteClient()

with client.Session() as session:
    # Be more specific
    task = "Fill the email input field in the signup form with user@example.com"

    # Or disable vision if causing confusion
    agent = client.Agent(session=session, use_vision=False)

Agent Exceeds Max Steps

Symptoms:

Agent stops before completing task
len(result.steps) == max_steps

Debug:

from notte_sdk import NotteClient

client = NotteClient()

with client.Session() as session:
    agent = client.Agent(session=session)
    result = agent.run(task="Complete task")

    print(f"Steps taken: {len(result.steps)}")
    print(f"Max steps: {agent.request.max_steps}")

replay = session.replay()
replay.download("debug_replay.mp4")  # See where it got stuck

Fix:

from notte_sdk import NotteClient

client = NotteClient()

with client.Session() as session:
    # Increase max_steps
    agent = client.Agent(session=session, max_steps=30)

    # Or break task into smaller steps
    result1 = agent.run(task="Navigate to products page")
    result2 = agent.run(task="Search for laptops")

Advanced Debugging

Step-by-Step Analysis

Inspect each step in detail:

step_by_step_analysis.py

result = agent.run(task="Complex task")

for i, step in enumerate(result.steps):
    print(f"\n=== Step {i + 1} ===")
    print(f"Action: {step.get('action')}")
    print(f"Success: {step.get('success')}")
    print(f"Page URL: {step.get('url')}")
    print(f"Message: {step.get('message')}")

Comparing Runs

Compare successful vs failed runs:

from notte_sdk import NotteClient

client = NotteClient()

# Successful run
with client.Session() as session_1:
    agent_1 = client.Agent(session=session_1)
    result_success = agent_1.run(task="Working task")
replay_success = session_1.replay()
replay_success.download("success.mp4")

# Failed run
with client.Session() as session_2:
    agent_2 = client.Agent(session=session_2)
    result_fail = agent_2.run(task="Failing task")
replay_fail = session_2.replay()
replay_fail.download("failure.mp4")

# Compare side-by-side
# Identify where behavior diverges

Agent Status During Execution

Monitor agent in real-time:

monitor_status.py

with client.Session() as session:
    agent = client.Agent(session=session)
    agent.start(task="Long running task")

    while True:
        status = agent.status()

        if status.status == "closed":
            break

        print(f"Steps: {len(status.steps)}")

        time.sleep(5)

# Get replay after completion
# replay = session.replay()
# replay.download("replay.mp4")

Best Practices

1. Always Review Failed Agents

Don’t ignore failures - watch the replay:

review_failed_agents.py

with client.Session() as session:
    agent = client.Agent(session=session)
    result = agent.run(task="Critical task")

if not result.success:
    replay = session.replay()
    replay.download(f"failure_{agent.agent_id}.mp4")
    print(f"Saved replay for debugging: failure_{agent.agent_id}.mp4")

2. Save Replays for Production Issues

Keep replays of production failures:

save_production_replays.py

    result = agent.run(task="Production task")
    if not result.success:
        replay = session.replay()
        replay.download(f"prod_failure_{agent.agent_id}.mp4")
        logger.error(f"Agent failed, replay saved: prod_failure_{agent.agent_id}.mp4")
except Exception as e:
    logger.error(f"Agent exception: {e}")

3. Use Replays for QA

Verify agent behavior before deploying:

replays_for_qa.py

with client.Session() as session:
    agent = client.Agent(session=session)
    # Test run
    result = agent.run(task="Test checkout flow")

# Review replay before production
replay = session.replay()
replay.download("qa_replay.mp4")

# Verify:
# - All steps completed correctly
# - No unexpected behavior
# - Performance is acceptable

Save and share for collaboration:

share_with_team.py

with client.Session() as session:
    agent = client.Agent(session=session)
    agent.run(task="Feature test")

replay = session.replay()
replay.download(f"feature_test_{agent.agent_id}.mp4")

# Share file with team
# Or share console link
print(f"Console replay: https://console.notte.cc/sessions/{session.session_id}")

Limitations

Replays are currently:

✅ Available for all completed agents
✅ Full video recording of execution
✅ Include element highlights
⚠️ Generated after completion (not real-time)
⚠️ May be large files for long runs

Next Steps

Lifecycle

Understand agent execution

Configuration

Optimize agent settings

Best Practices

Build reliable agents

Live View

Watch sessions in real-time

Getting Started

Sessions

Agents

Functions

Agent Tools

Scraping

Guides

Integrations

Replay & Debugging

Agent Replay

What’s Included

Viewing Replays

In Jupyter Notebook

Save to File

In Browser

Debugging Workflow

1. Run Agent

2. Check If Failed

3. Inspect Steps

4. Watch Replay

5. Fix and Retry

Common Issues

Agent Can’t Find Element

Agent Takes Wrong Action

Agent Exceeds Max Steps

Advanced Debugging

Step-by-Step Analysis

Comparing Runs

Agent Status During Execution

Best Practices

1. Always Review Failed Agents

2. Save Replays for Production Issues

3. Use Replays for QA

Limitations

Next Steps

Lifecycle

Configuration

Best Practices

Live View

​Agent Replay

​What’s Included

​Viewing Replays

​In Jupyter Notebook

​Save to File

​In Browser

​Debugging Workflow

​1. Run Agent

​2. Check If Failed

​3. Inspect Steps

​4. Watch Replay

​5. Fix and Retry

​Common Issues

​Agent Can’t Find Element

​Agent Takes Wrong Action

​Agent Exceeds Max Steps

​Advanced Debugging

​Step-by-Step Analysis

​Comparing Runs

​Agent Status During Execution

​Best Practices

​1. Always Review Failed Agents

​2. Save Replays for Production Issues

​3. Use Replays for QA

​4. Share Replays with Team

​Limitations

​Next Steps

Lifecycle

Configuration

Best Practices

Live View

Agent Replay

What’s Included

Viewing Replays

In Jupyter Notebook

Save to File

In Browser

Debugging Workflow

1. Run Agent

2. Check If Failed

3. Inspect Steps

4. Watch Replay

5. Fix and Retry

Common Issues

Agent Can’t Find Element

Agent Takes Wrong Action

Agent Exceeds Max Steps

Advanced Debugging

Step-by-Step Analysis

Comparing Runs

Agent Status During Execution

Best Practices

1. Always Review Failed Agents

2. Save Replays for Production Issues

3. Use Replays for QA

4. Share Replays with Team

Limitations

Next Steps