You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The bug
After execute serve run test:app in terminal, it raise a validation error about object_store_memory in ray_actor_options.
Expected behavior
The deployment should start up successfully.
Useful information
Following are logs from terminal:
2024-05-14 12:29:58,511 INFO scripts.py:499 -- Running import path: 'test:app'.
2024-05-14 12:29:58,543 INFO worker.py:1564 -- Connecting to existing Ray cluster at address: 0.0.0.0:6379...
2024-05-14 12:29:58,546 INFO worker.py:1740 -- Connected to Ray cluster. View the dashboard at http://0.0.0.0:8265
Traceback (most recent call last):
File "/home/user/.pyenv/versions/3.11.9/lib/python3.11/site-packages/ray/serve/scripts.py", line 548, in run
serve.run(app, blocking=should_block, name=name, route_prefix=route_prefix)
File "/home/user/.pyenv/versions/3.11.9/lib/python3.11/site-packages/ray/serve/api.py", line 578, in run
handle = _run(
^^^^^
File "/home/user/.pyenv/versions/3.11.9/lib/python3.11/site-packages/ray/serve/api.py", line 492, in _run
deployments = pipeline_build(target._get_internal_dag_node(), name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.pyenv/versions/3.11.9/lib/python3.11/site-packages/ray/serve/_private/deployment_graph_build.py", line 81, in build
serve_root_dag = ray_dag_root_node.apply_recursive(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.pyenv/versions/3.11.9/lib/python3.11/site-packages/ray/dag/dag_node.py", line 297, in apply_recursive
return fn(
^^^
File "/home/user/.pyenv/versions/3.11.9/lib/python3.11/site-packages/ray/dag/dag_node.py", line 282, in __call__
self.cache[node._stable_uuid] = self.fn(node)
^^^^^^^^^^^^^
File "/home/user/.pyenv/versions/3.11.9/lib/python3.11/site-packages/ray/serve/_private/deployment_graph_build.py", line 82, in <lambda>
lambda node: transform_ray_dag_to_serve_dag(node, node_name_generator, name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.pyenv/versions/3.11.9/lib/python3.11/site-packages/ray/serve/_private/deployment_graph_build.py", line 162, in transform_ray_dag_to_serve_dag
deployment_shell: Deployment = schema_to_deployment(deployment_schema)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.pyenv/versions/3.11.9/lib/python3.11/site-packages/ray/serve/deployment.py", line 651, in schema_to_deployment
replica_config = ReplicaConfig.create(
^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.pyenv/versions/3.11.9/lib/python3.11/site-packages/ray/serve/_private/config.py", line 517, in create
config = cls(
^^^^
File "/home/user/.pyenv/versions/3.11.9/lib/python3.11/site-packages/ray/serve/_private/config.py", line 429, in __init__
self._validate()
File "/home/user/.pyenv/versions/3.11.9/lib/python3.11/site-packages/ray/serve/_private/config.py", line 438, in _validate
self._validate_ray_actor_options()
File "/home/user/.pyenv/versions/3.11.9/lib/python3.11/site-packages/ray/serve/_private/config.py", line 563, in _validate_ray_actor_options
ray_option_utils.validate_actor_options(self.ray_actor_options, in_options=True)
File "/home/user/.pyenv/versions/3.11.9/lib/python3.11/site-packages/ray/_private/ray_option_utils.py", line 340, in validate_actor_options
actor_options[k].validate(k, v)
File "/home/user/.pyenv/versions/3.11.9/lib/python3.11/site-packages/ray/_private/ray_option_utils.py", line 32, in validate
raise TypeError(
TypeError: The type of keyword 'object_store_memory' must be (<class 'int'>, <class 'NoneType'>), but received type <class 'float'>
2024-05-14 12:30:00,038 ERR scripts.py:585 -- Received unexpected error, see console logs for more details. Shutting down...
(ProxyActor pid=846212) INFO 2024-05-14 12:30:00,022 proxy 0.0.0.0 proxy.py:1161 - Proxy starting on node b2280301862051dba174650bd75aa47d6b9749aefb6fe43f795f2e53 (HTTP port: 8000).
The text was updated successfully, but these errors were encountered:
aRyBernAlTEglOTRO
added
bug
Something that is supposed to be working; but isn't
triage
Needs triage (eg: priority, bug/not-bug, and owning component)
labels
May 14, 2024
I think this bug is caused by this line of code, after I change the value from _counting_option("object_store_memory", False) to _resource_option("object_store_memory") the error fixed.
I think this line of code is the reason that cause the bug, which convert the value of object_store_memory to float, I think change it from object_store_memory: float to object_store_memory: int will fix this bug.
What happened + What you expected to happen
After execute
serve run test:app
in terminal, it raise a validation error aboutobject_store_memory
inray_actor_options
.The deployment should start up successfully.
Following are logs from terminal:
Versions / Dependencies
Python Version: 3.11.9
Ray Version: 2.21.0
FastAPI Version: 0.111.0
OS: Ubuntu 22.04.4 LTS
Reproduction script
# bash script serve run test:app
Issue Severity
Low: It annoys or frustrates me.
The text was updated successfully, but these errors were encountered: