zhuwenxing a08000cfbf enhance:[skip e2e]update one pod mode resource (#34196) | 3 月之前 | |
---|---|---|
.. | ||
chaos_objects | 2 年之前 | |
config | 2 年之前 | |
scripts | 11 月之前 | |
testcases | 5 月之前 | |
README.md | 2 年之前 | |
chaos_commons.py | 1 年之前 | |
chaos_test.sh | 2 年之前 | |
checker.py | 4 月之前 | |
cluster-values.yaml | 6 月之前 | |
conftest.py | 5 月之前 | |
constants.py | 8 月之前 | |
nats-standalone-values.yaml | 1 年之前 | |
one-pod-standalone-values.yaml | 3 月之前 | |
requirements.txt | 9 月之前 | |
run.sh | 2 年之前 | |
standalone-values.yaml | 6 月之前 | |
test_chaos.py | 10 月之前 | |
test_chaos_apply.py | 1 年之前 | |
test_chaos_apply_to_coord.py | 1 年之前 | |
test_chaos_apply_to_determined_pod.py | 7 月之前 | |
test_chaos_bulk_insert.py | 1 年之前 | |
test_chaos_data_consist.py | 2 年之前 | |
test_chaos_memory_stress.py | 10 月之前 | |
test_chaos_multi_replicas.py | 2 年之前 | |
test_load_with_checker.py | 10 月之前 |
Chaos tests are designed to check the reliability of Milvus.
For instance, if one pod is killed:
Chaos tests run in pytest framework, same as e2e tests.
Please refer to Run E2E Tests
Kill pod every 5s
Two direction(to and from) network isolation between a pod and the rest of the pods
Set the pod(querynode, indexnode and datanode)as multiple replicas, make one of them failure, and test milvus's functionality
Limit the memory resource of pod and generate plenty of stresses over a group of pods
standalone pod is killed
minio pod is killed
chaos_objects
ALL_CHAOS_YAMLS
in constants.py
would be parsed as a parameter and be passed into test_chaos.py
testcases.yaml
locates in folder chaos_objects
test_chaos.py
Run a single test scenario manually(take query node pod is killed as instance):
update ALL_CHAOS_YAMLS = 'chaos_querynode_podkill.yaml'
in constants.py
run the commands below: ```bash cd /milvus/tests/python_client/chaos
pytest test_chaos.py --host ${Milvus_IP} -v
Run multiple test scenario in a category manually(take network partition chaos for all pods as instance):
1. update `ALL_CHAOS_YAMLS = 'chaos_*_network_partition.yaml'` in `constants.py`
2. run the commands below:
```bash
cd /milvus/tests/python_client/chaos
pytest test_chaos.py --host ${Milvus_IP} -v
Run test scenario automatically:
chaos_test.sh
run the commands below:
cd /milvus/tests/python_client/chaos
# in this step, script will install milvus with replicas_num and run testcase
bash chaos_test.sh ${pod} ${chaos_type} ${chaos_task} ${replicas_num}
# example: bash chaos_test.sh querynode pod_kill chaos-test 2
still in planning
chaos_${component_name}_${chaos_type}.yaml
. Make sure kubectl apply -f ${your_chaos_yaml_file}
can take effecttestcases.yaml
. You should figure out the expectation of milvus during the chaosManually
above and check whether it as your expectation