Skip to content

Device Reference

This document provides a reference for the abstract base classes (blueprints) that define the core functionalities for different types of devices in Pymordial. Each blueprint serves as a contract for concrete device implementations.

Bridge Device

Bases: ABC

Abstract base class for device bridges (e.g., ADB, Win32).

Defines the low-level commands to control the device hardware or emulator.

Source code in src/pymordial/core/blueprints/bridge_device.py 
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
class PymordialBridgeDevice(ABC):
    """Abstract base class for device bridges (e.g., ADB, Win32).

    Defines the low-level commands to control the device hardware or emulator.
    """

    @abstractmethod
    def connect(self):
        """Connects to the device bridge."""
        pass

    @abstractmethod
    def disconnect(self):
        """Disconnects from the device bridge."""
        pass

    @abstractmethod
    def run_command(self):
        """Executes a raw shell command on the device."""
        pass

    @abstractmethod
    def open_app(self):
        """Opens an application on the device."""
        pass

    @abstractmethod
    def is_app_running(self):
        """Checks if an application is currently running."""
        pass

    @abstractmethod
    def show_recent_apps(self):
        """Shows the recent apps overview."""
        pass

    @abstractmethod
    def close_app(self):
        """Closes an application on the device."""
        pass

    @abstractmethod
    def tap(self):
        """Simulates a tap/click event."""
        pass

    @abstractmethod
    def type_text(self):
        """Simulates text input."""
        pass

    @abstractmethod
    def go_home(self):
        """Simulates pressing the Home button."""
        pass

    @abstractmethod
    def press_enter(self):
        """Simulates pressing the Enter key."""
        pass

    @abstractmethod
    def press_esc(self):
        """Simulates pressing the Escape/Back key."""
        pass

    @abstractmethod
    def capture_screenshot(self):
        """Captures a single frame from the device."""
        pass

    @abstractmethod
    def start_stream(self):
        """Starts a persistent video/screen stream."""
        pass

    @abstractmethod
    def stop_stream(self):
        """Stops the persistent video/screen stream."""
        pass

    @abstractmethod
    def get_latest_frame(self):
        """Retrieves the most recent frame from the stream."""
        pass

capture_screenshot() abstractmethod

Captures a single frame from the device.

Source code in src/pymordial/core/blueprints/bridge_device.py 
73
74
75
76
@abstractmethod
def capture_screenshot(self):
    """Captures a single frame from the device."""
    pass

close_app() abstractmethod

Closes an application on the device.

Source code in src/pymordial/core/blueprints/bridge_device.py 
43
44
45
46
@abstractmethod
def close_app(self):
    """Closes an application on the device."""
    pass

connect() abstractmethod

Connects to the device bridge.

Source code in src/pymordial/core/blueprints/bridge_device.py 
13
14
15
16
@abstractmethod
def connect(self):
    """Connects to the device bridge."""
    pass

disconnect() abstractmethod

Disconnects from the device bridge.

Source code in src/pymordial/core/blueprints/bridge_device.py 
18
19
20
21
@abstractmethod
def disconnect(self):
    """Disconnects from the device bridge."""
    pass

get_latest_frame() abstractmethod

Retrieves the most recent frame from the stream.

Source code in src/pymordial/core/blueprints/bridge_device.py 
88
89
90
91
@abstractmethod
def get_latest_frame(self):
    """Retrieves the most recent frame from the stream."""
    pass

go_home() abstractmethod

Simulates pressing the Home button.

Source code in src/pymordial/core/blueprints/bridge_device.py 
58
59
60
61
@abstractmethod
def go_home(self):
    """Simulates pressing the Home button."""
    pass

is_app_running() abstractmethod

Checks if an application is currently running.

Source code in src/pymordial/core/blueprints/bridge_device.py 
33
34
35
36
@abstractmethod
def is_app_running(self):
    """Checks if an application is currently running."""
    pass

open_app() abstractmethod

Opens an application on the device.

Source code in src/pymordial/core/blueprints/bridge_device.py 
28
29
30
31
@abstractmethod
def open_app(self):
    """Opens an application on the device."""
    pass

press_enter() abstractmethod

Simulates pressing the Enter key.

Source code in src/pymordial/core/blueprints/bridge_device.py 
63
64
65
66
@abstractmethod
def press_enter(self):
    """Simulates pressing the Enter key."""
    pass

press_esc() abstractmethod

Simulates pressing the Escape/Back key.

Source code in src/pymordial/core/blueprints/bridge_device.py 
68
69
70
71
@abstractmethod
def press_esc(self):
    """Simulates pressing the Escape/Back key."""
    pass

run_command() abstractmethod

Executes a raw shell command on the device.

Source code in src/pymordial/core/blueprints/bridge_device.py 
23
24
25
26
@abstractmethod
def run_command(self):
    """Executes a raw shell command on the device."""
    pass

show_recent_apps() abstractmethod

Shows the recent apps overview.

Source code in src/pymordial/core/blueprints/bridge_device.py 
38
39
40
41
@abstractmethod
def show_recent_apps(self):
    """Shows the recent apps overview."""
    pass

start_stream() abstractmethod

Starts a persistent video/screen stream.

Source code in src/pymordial/core/blueprints/bridge_device.py 
78
79
80
81
@abstractmethod
def start_stream(self):
    """Starts a persistent video/screen stream."""
    pass

stop_stream() abstractmethod

Stops the persistent video/screen stream.

Source code in src/pymordial/core/blueprints/bridge_device.py 
83
84
85
86
@abstractmethod
def stop_stream(self):
    """Stops the persistent video/screen stream."""
    pass

tap() abstractmethod

Simulates a tap/click event.

Source code in src/pymordial/core/blueprints/bridge_device.py 
48
49
50
51
@abstractmethod
def tap(self):
    """Simulates a tap/click event."""
    pass

type_text() abstractmethod

Simulates text input.

Source code in src/pymordial/core/blueprints/bridge_device.py 
53
54
55
56
@abstractmethod
def type_text(self):
    """Simulates text input."""
    pass

Vision Device

Bases: ABC

Abstract base class for vision-based device interaction.

Defines the contract for visual operations such as screen scaling, pixel color checking, and text extraction/finding.

Source code in src/pymordial/core/blueprints/vision_device.py 
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
class PymordialVisionDevice(ABC):
    """Abstract base class for vision-based device interaction.

    Defines the contract for visual operations such as screen scaling,
    pixel color checking, and text extraction/finding.
    """

    @abstractmethod
    @abstractmethod
    def scale_img_to_screen(self) -> Any:
        """Scales the reference image to match the current screen resolution.

        Returns:
            A PIL Image object resized to the device screen dimensions.
        """
        pass

    @abstractmethod
    def check_pixel_color(self) -> bool | None:
        """Verifies if a specific pixel matches a target color.

        Returns:
            True if color matches within tolerance, False if not, None if error.
        """
        pass

    @abstractmethod
    def where_element(self) -> tuple[int, int] | None:
        """Finds the coordinates of a visual element.

        Returns:
            A tuple of (x, y) coordinates if found, None otherwise.
        """
        pass

    @abstractmethod
    def where_elements(self) -> tuple[int, int] | None:
        """Finds the coordinates of one of multiple visual elements.

        Returns:
            A tuple of (x, y) coordinates if any element is found, None otherwise.
        """
        pass

    @abstractmethod
    def find_text(self) -> tuple[int, int] | None:
        """Finds the coordinates of specific text on the screen.

        Returns:
            A tuple of (x, y) coordinates if text is found, None otherwise.
        """
        pass

    @abstractmethod
    def check_text(self) -> bool:
        """Checks if specific text is visible on the screen.

        Returns:
            True if text is found, False otherwise.
        """
        pass

    @abstractmethod
    def read_text(self) -> list[str]:
        """Reads all text from the current screen.

        Returns:
            A list of strings representing the text lines found on screen.
        """
        pass

check_pixel_color() abstractmethod

Verifies if a specific pixel matches a target color.

Returns:

Type Description
bool | None

True if color matches within tolerance, False if not, None if error.
Source code in src/pymordial/core/blueprints/vision_device.py 
25
26
27
28
29
30
31
32
@abstractmethod
def check_pixel_color(self) -> bool | None:
    """Verifies if a specific pixel matches a target color.

    Returns:
        True if color matches within tolerance, False if not, None if error.
    """
    pass

check_text() abstractmethod

Checks if specific text is visible on the screen.

Returns:

Type Description
bool

True if text is found, False otherwise.
Source code in src/pymordial/core/blueprints/vision_device.py 
61
62
63
64
65
66
67
68
@abstractmethod
def check_text(self) -> bool:
    """Checks if specific text is visible on the screen.

    Returns:
        True if text is found, False otherwise.
    """
    pass

find_text() abstractmethod

Finds the coordinates of specific text on the screen.

Returns:

Type Description
tuple[int, int] | None

A tuple of (x, y) coordinates if text is found, None otherwise.
Source code in src/pymordial/core/blueprints/vision_device.py 
52
53
54
55
56
57
58
59
@abstractmethod
def find_text(self) -> tuple[int, int] | None:
    """Finds the coordinates of specific text on the screen.

    Returns:
        A tuple of (x, y) coordinates if text is found, None otherwise.
    """
    pass

read_text() abstractmethod

Reads all text from the current screen.

Returns:

Type Description
list[str]

A list of strings representing the text lines found on screen.
Source code in src/pymordial/core/blueprints/vision_device.py 
70
71
72
73
74
75
76
77
@abstractmethod
def read_text(self) -> list[str]:
    """Reads all text from the current screen.

    Returns:
        A list of strings representing the text lines found on screen.
    """
    pass

scale_img_to_screen() abstractmethod

Scales the reference image to match the current screen resolution.

Returns:

Type Description
Any

A PIL Image object resized to the device screen dimensions.
Source code in src/pymordial/core/blueprints/vision_device.py 
15
16
17
18
19
20
21
22
23
@abstractmethod
@abstractmethod
def scale_img_to_screen(self) -> Any:
    """Scales the reference image to match the current screen resolution.

    Returns:
        A PIL Image object resized to the device screen dimensions.
    """
    pass

where_element() abstractmethod

Finds the coordinates of a visual element.

Returns:

Type Description
tuple[int, int] | None

A tuple of (x, y) coordinates if found, None otherwise.
Source code in src/pymordial/core/blueprints/vision_device.py 
34
35
36
37
38
39
40
41
@abstractmethod
def where_element(self) -> tuple[int, int] | None:
    """Finds the coordinates of a visual element.

    Returns:
        A tuple of (x, y) coordinates if found, None otherwise.
    """
    pass

where_elements() abstractmethod

Finds the coordinates of one of multiple visual elements.

Returns:

Type Description
tuple[int, int] | None

A tuple of (x, y) coordinates if any element is found, None otherwise.
Source code in src/pymordial/core/blueprints/vision_device.py 
43
44
45
46
47
48
49
50
@abstractmethod
def where_elements(self) -> tuple[int, int] | None:
    """Finds the coordinates of one of multiple visual elements.

    Returns:
        A tuple of (x, y) coordinates if any element is found, None otherwise.
    """
    pass

Emulator Device

Bases: ABC

Abstract interface for controlling emulator software (e.g., BlueStacks).

Attributes:

Name Type Description
state

StateMachine tracking the emulator's lifecycle.
Source code in src/pymordial/core/blueprints/emulator_device.py 
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
class PymordialEmulatorDevice(ABC):
    """Abstract interface for controlling emulator software (e.g., BlueStacks).

    Attributes:
        state: StateMachine tracking the emulator's lifecycle.
    """

    def __init__(self):
        self.state = StateMachine(
            current_state=EmulatorState.CLOSED,
            transitions=EmulatorState.get_transitions(),
        )

    @abstractmethod
    def open(self):
        """Opens the emulator software.

        Returns:
            True if the launch command was successful, False otherwise.
        """
        pass

    @abstractmethod
    def wait_for_load(self):
        """Waits for the emulator to reach a ready state.

        Returns:
            True if proper loading was detected within timeout, False otherwise.
        """
        pass

    @abstractmethod
    def is_ready(self):
        """Checks if the emulator is currently ready to accept commands.

        Returns:
            True if the emulator is ready, False otherwise.
        """
        pass

    @abstractmethod
    def close(self):
        """Closes or terminates the emulator software.

        Returns:
            True if the close command was successful, False otherwise.
        """
        pass

close() abstractmethod

Closes or terminates the emulator software.

Returns:

Type Description

True if the close command was successful, False otherwise.
Source code in src/pymordial/core/blueprints/emulator_device.py 
71
72
73
74
75
76
77
78
@abstractmethod
def close(self):
    """Closes or terminates the emulator software.

    Returns:
        True if the close command was successful, False otherwise.
    """
    pass

is_ready() abstractmethod

Checks if the emulator is currently ready to accept commands.

Returns:

Type Description

True if the emulator is ready, False otherwise.
Source code in src/pymordial/core/blueprints/emulator_device.py 
62
63
64
65
66
67
68
69
@abstractmethod
def is_ready(self):
    """Checks if the emulator is currently ready to accept commands.

    Returns:
        True if the emulator is ready, False otherwise.
    """
    pass

open() abstractmethod

Opens the emulator software.

Returns:

Type Description

True if the launch command was successful, False otherwise.
Source code in src/pymordial/core/blueprints/emulator_device.py 
44
45
46
47
48
49
50
51
@abstractmethod
def open(self):
    """Opens the emulator software.

    Returns:
        True if the launch command was successful, False otherwise.
    """
    pass

wait_for_load() abstractmethod

Waits for the emulator to reach a ready state.

Returns:

Type Description

True if proper loading was detected within timeout, False otherwise.
Source code in src/pymordial/core/blueprints/emulator_device.py 
53
54
55
56
57
58
59
60
@abstractmethod
def wait_for_load(self):
    """Waits for the emulator to reach a ready state.

    Returns:
        True if proper loading was detected within timeout, False otherwise.
    """
    pass

OCR Device

Bases: ABC

Abstract base class for OCR engines.

All OCR implementations must inherit from this class and implement the extract_text method.

Source code in src/pymordial/core/blueprints/ocr_device.py 
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
class PymordialOCRDevice(ABC):
    """Abstract base class for OCR engines.

    All OCR implementations must inherit from this class and implement
    the extract_text method.
    """

    @abstractmethod
    def extract_text(self, image_path: Path | bytes | str | Any) -> str:
        """Extracts text from an image.

        Args:
            image_path: The image source (file path, bytes, or numpy array).

        Returns:
            The raw text extracted from the image.

        Raises:
            ValueError: If the image format is unsupported or cannot be processed.
        """
        pass

    @abstractmethod
    def find_text(self, search_text: str, image_path: Any) -> tuple[int, int] | None:
        """Finds the coordinates (center) of the specified text in the image.

        Args:
            search_text: The text string to search for.
            image_path: The image source (file path, bytes, or numpy array).

        Returns:
            A tuple of (x, y) coordinates for the center of the found text,
            or None if the text is not found.
        """
        pass

    def contains_text(
        self, search_text: str, image_path: Path | bytes | str | Any
    ) -> bool:
        """Checks if image contains specific text.

        Args:
            search_text: The text string to search for.
            image_path: The image source (file path, bytes, or numpy array).

        Returns:
            True if the search text is found (case-insensitive), False otherwise.
        """
        extracted = self.extract_text(image_path)
        return search_text.lower() in extracted.lower()

    def extract_lines(self, image_path: Path | bytes | str | Any) -> list[str]:
        """Extracts text as individual lines.

        Args:
            image_path: The image source (file path, bytes, or numpy array).

        Returns:
            A list of non-empty text lines extracted from the image.
        """
        text = self.extract_text(image_path)
        return [line.strip() for line in text.split("\n") if line.strip()]

contains_text(search_text, image_path)

Checks if image contains specific text.

Parameters:

Name Type Description Default
search_text str

The text string to search for.

 required
image_path Path | bytes | str | Any

The image source (file path, bytes, or numpy array).

 required

Returns:

Type Description
bool

True if the search text is found (case-insensitive), False otherwise.
Source code in src/pymordial/core/blueprints/ocr_device.py 
44
45
46
47
48
49
50
51
52
53
54
55
56
57
def contains_text(
    self, search_text: str, image_path: Path | bytes | str | Any
) -> bool:
    """Checks if image contains specific text.

    Args:
        search_text: The text string to search for.
        image_path: The image source (file path, bytes, or numpy array).

    Returns:
        True if the search text is found (case-insensitive), False otherwise.
    """
    extracted = self.extract_text(image_path)
    return search_text.lower() in extracted.lower()

extract_lines(image_path)

Extracts text as individual lines.

Parameters:

Name Type Description Default
image_path Path | bytes | str | Any

The image source (file path, bytes, or numpy array).

 required

Returns:

Type Description
list[str]

A list of non-empty text lines extracted from the image.
Source code in src/pymordial/core/blueprints/ocr_device.py 
59
60
61
62
63
64
65
66
67
68
69
def extract_lines(self, image_path: Path | bytes | str | Any) -> list[str]:
    """Extracts text as individual lines.

    Args:
        image_path: The image source (file path, bytes, or numpy array).

    Returns:
        A list of non-empty text lines extracted from the image.
    """
    text = self.extract_text(image_path)
    return [line.strip() for line in text.split("\n") if line.strip()]

extract_text(image_path) abstractmethod

Extracts text from an image.

Parameters:

Name Type Description Default
image_path Path | bytes | str | Any

The image source (file path, bytes, or numpy array).

 required

Returns:

Type Description
str

The raw text extracted from the image.

Raises:

Type Description
ValueError

If the image format is unsupported or cannot be processed.
Source code in src/pymordial/core/blueprints/ocr_device.py 
15
16
17
18
19
20
21
22
23
24
25
26
27
28
@abstractmethod
def extract_text(self, image_path: Path | bytes | str | Any) -> str:
    """Extracts text from an image.

    Args:
        image_path: The image source (file path, bytes, or numpy array).

    Returns:
        The raw text extracted from the image.

    Raises:
        ValueError: If the image format is unsupported or cannot be processed.
    """
    pass

find_text(search_text, image_path) abstractmethod

Finds the coordinates (center) of the specified text in the image.

Parameters:

Name Type Description Default
search_text str

The text string to search for.

 required
image_path Any

The image source (file path, bytes, or numpy array).

 required

Returns:

Type Description
tuple[int, int] | None

A tuple of (x, y) coordinates for the center of the found text,
tuple[int, int] | None

or None if the text is not found.
Source code in src/pymordial/core/blueprints/ocr_device.py 
30
31
32
33
34
35
36
37
38
39
40
41
42
@abstractmethod
def find_text(self, search_text: str, image_path: Any) -> tuple[int, int] | None:
    """Finds the coordinates (center) of the specified text in the image.

    Args:
        search_text: The text string to search for.
        image_path: The image source (file path, bytes, or numpy array).

    Returns:
        A tuple of (x, y) coordinates for the center of the found text,
        or None if the text is not found.
    """
    pass