(serve-api)=
(core-apis)=
.. module:: ray
.. autosummary::
:nosignatures:
:toctree: doc/
serve.deployment
:noindex:
serve.ingress
serve.batch
.. autosummary::
:nosignatures:
:toctree: doc/
:template: autosummary/class_without_init_args.rst
serve.Deployment
serve.Application
serve.handle.RayServeHandle
serve.handle.RayServeSyncHandle
.. autosummary::
:nosignatures:
:toctree: doc/
serve.get_replica_context
.. autosummary::
:nosignatures:
:toctree: doc/
serve.run
serve.delete
serve.start
serve.shutdown
(serve-cli)=
.. click:: ray.serve.scripts:cli
:prog: serve
:nested: full
(serve-rest-api)=
PUT "/api/serve/deployments/"
Declaratively deploys the Serve application. Starts Serve on the Ray cluster if it's not already running. See single-app config schema for the request's JSON schema.
Example Request:
PUT /api/serve/deployments/ HTTP/1.1
Host: http://localhost:52365/
Accept: application/json
Content-Type: application/json
{
"import_path": "fruit.deployment_graph",
"runtime_env": {
"working_dir": "https://github.com/ray-project/serve_config_examples/archive/HEAD.zip"
},
"deployments": [
{"name": "MangoStand", "user_config": {"price": 1}},
{"name": "OrangeStand", "user_config": {"price": 2}},
{"name": "PearStand", "user_config": {"price": 3}}
]
}
Example Response
HTTP/1.1 200 OK
Content-Type: application/json
GET "/api/serve/deployments/"
Gets the config for the application currently deployed on the Ray cluster. This config represents the current goal state for the Serve application. See single-app config schema for the response's JSON schema.
Example Request:
GET /api/serve/deployments/ HTTP/1.1
Host: http://localhost:52365/
Accept: application/json
Example Response:
HTTP/1.1 200 OK
Content-Type: application/json
{
"import_path": "fruit.deployment_graph",
"runtime_env": {
"working_dir": "https://github.com/ray-project/serve_config_examples/archive/HEAD.zip"
},
"deployments": [
{"name": "MangoStand", "user_config": {"price": 1}},
{"name": "OrangeStand", "user_config": {"price": 2}},
{"name": "PearStand", "user_config": {"price": 3}}
]
}
GET "/api/serve/deployments/status"
Gets the Serve application's current status, including all the deployment statuses. See status schema for the response's JSON schema.
Example Request:
GET /api/serve/deployments/ HTTP/1.1
Host: http://localhost:52365/
Accept: application/json
Example Response
HTTP/1.1 200 OK
Content-Type: application/json
{
"app_status": {
"status": "RUNNING",
"message": "",
"deployment_timestamp": 1855994527.146304
},
"deployment_statuses": [
{
"name": "MangoStand",
"status": "HEALTHY",
"message": ""
},
{
"name": "OrangeStand",
"status": "HEALTHY",
"message": ""
},
{
"name": "PearStand",
"status": "HEALTHY",
"message": ""
},
{
"name": "FruitMarket",
"status": "HEALTHY",
"message": ""
},
{
"name": "DAGDriver",
"status": "HEALTHY",
"message": ""
}
]
}
DELETE "/api/serve/deployments/"
Shuts down Serve and the Serve application running on the Ray cluster. Has no effect if Serve is not running on the Ray cluster.
Example Request:
DELETE /api/serve/deployments/ HTTP/1.1
Host: http://localhost:52365/
Accept: application/json
Example Response
HTTP/1.1 200 OK
Content-Type: application/json
PUT "/api/serve/applications/"
Declaratively deploys a list of Serve applications. If Serve is already running on the Ray cluster, removes all applications not listed in the new config. If Serve is not running on the Ray cluster, starts Serve. See multi-app config schema for the request's JSON schema.
Example Request:
PUT /api/serve/applications/ HTTP/1.1
Host: http://localhost:52365/
Accept: application/json
Content-Type: application/json
{
"applications": [
{
"name": "fruit_stand",
"route_prefix": "/fruit",
"import_path": "fruit.deployment_graph",
"runtime_env": {
"working_dir": "https://github.com/ray-project/serve_config_examples/archive/HEAD.zip"
},
"deployments": [
{"name": "MangoStand", "user_config": {"price": 1}},
{"name": "OrangeStand", "user_config": {"price": 2}},
{"name": "PearStand", "user_config": {"price": 3}}
]
},
{
"name": "calculator",
"route_prefix": "/math",
"import_path": "conditional_dag.serve_dag",
"runtime_env": {
"working_dir": "https://github.com/ray-project/test_dag/archive/HEAD.zip"
},
"deployments": [
{"name": "Multiplier", "ray_actor_options": {"num_cpus": 0.5}},
{
"name": "Adder",
"ray_actor_options": {"env_vars": {"override_increment": "5"}}
},
]
}
]
}
Example Response
HTTP/1.1 200 OK
Content-Type: application/json
GET "/api/serve/applications/"
Gets cluster-level info and comprehensive details on all Serve applications deployed on the Ray cluster. See metadata schema for the response's JSON schema.
GET /api/serve/applications/ HTTP/1.1
Host: http://localhost:52365/
Accept: application/json
Example Response (abridged JSON):
HTTP/1.1 200 OK
Content-Type: application/json
{
"proxy_location": "HeadOnly",
"http_options": {
"host": "127.0.0.1",
"port": 8000
},
"deploy_mode": "MULTI_APP",
"applications": {
"fruit_stand": {
"name": "fruit_stand",
"route_prefix": "/fruit",
"docs_path": null,
"status": "RUNNING",
"message": "",
"last_deployed_time_s": 1679952253.748111,
"deployed_app_config": "...",
"deployments": {
"fruit_app_MangoStand": {
"name": "fruit_app_MangoStand",
"status": "HEALTHY",
"message": "",
"deployment_config": "...",
"replicas": [
{
"replica_id": "fruit_app_MangoStand#bSkrHK",
"state": "RUNNING",
"pid": 59350,
"actor_name": "...",
"actor_id": "...",
"node_id": "...",
"node_ip": "...",
"start_time_s": 1679952254.3458009
}
]
},
}
},
}
}
DELETE "/api/serve/applications/"
Shuts down Serve and all applications running on the Ray cluster. Has no effect if Serve is not running on the Ray cluster.
Example Request:
DELETE /api/serve/applications/ HTTP/1.1
Host: http://localhost:52365/
Accept: application/json
Example Response
HTTP/1.1 200 OK
Content-Type: application/json
(serve-rest-api-config-schema)=
.. currentmodule:: ray.serve
.. autosummary::
:toctree: doc/
schema.ServeDeploySchema
schema.HTTPOptionsSchema
schema.ServeApplicationSchema
schema.DeploymentSchema
schema.RayActorOptionsSchema
(serve-rest-api-response-schema)=
.. autosummary::
:toctree: doc/
schema.ServeStatusSchema
.. autosummary::
:toctree: doc/
schema.ServeInstanceDetails
schema.ApplicationDetails
schema.DeploymentDetails
schema.ReplicaDetails