index.rst 1.4 KB

1234567891011121314151617181920212223242526272829303132
  1. .. _workflows:
  2. Ray Workflows: Durable Ray Task Graphs
  3. ======================================
  4. .. warning::
  5. Ray Workflows is available as **alpha** in Ray 2.0+. Expect rough corners and
  6. for its APIs and storage format to change. Please file feature requests and
  7. bug reports on GitHub Issues or join the discussion on the
  8. `Ray Slack <https://forms.gle/9TSdDYUgxYs8SA9e8>`__.
  9. Ray Workflows implements high-performance, *durable* application workflows using
  10. Ray tasks as the underlying execution engine. It enables task-based Ray jobs
  11. to seamlessly resume execution even in the case of entire-cluster failure.
  12. Why Ray Workflows?
  13. ------------------
  14. **Flexibility:** Combine the flexibility of Ray's dynamic task graphs with
  15. strong durability guarantees. Branch or loop conditionally based on runtime
  16. data. Use Ray distributed libraries seamlessly within workflow tasks.
  17. **Performance:** Ray Workflows offers sub-second overheads for task launch and
  18. supports workflows with hundreds of thousands of tasks. Take advantage of the
  19. Ray object store to pass distributed datasets between tasks with zero-copy
  20. overhead.
  21. You might find that Ray Workflows is *lower level* compared to engines such as
  22. `AirFlow <https://www.astronomer.io/blog/airflow-ray-data-science-story>`__
  23. (which can also run on Ray). This is because Ray Workflows focuses more on core
  24. durability primitives as opposed to tools and integrations.