Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Perform a scheduling simulation in dry-run mode. #1584

Open
zhifei92 opened this issue Dec 25, 2024 · 6 comments · May be fixed by #1585
Open

Perform a scheduling simulation in dry-run mode. #1584

zhifei92 opened this issue Dec 25, 2024 · 6 comments · May be fixed by #1585
Assignees
@zhifei92
Labels
kind/feature Categorizes issue or PR as related to a new feature.

Comments

@zhifei92
Copy link
Contributor

Is your feature request related to a problem? Please describe.

Scenario: in dry-run mode.If the descheduler determines that 10 pods in the cluster can be evicted, and each of these pods requires 8 cores and 16 GB of memory (8c/16g), while the total available resources in the cluster are sufficient, but only one node meets the 8c/16g requirement. Under the current logic, all pods are considered evictable.

In dry-run mode, no resources are deducted because the pods are not actually evicted and rescheduled, nor is scheduling simulated.

loged:

klog.V(1).InfoS("Evicted pod in dry run mode", "pod", klog.KObj(pod), "reason", opts.Reason, "strategy", opts.StrategyName, "node", pod.Spec.NodeName, "profile", opts.ProfileName)

Using dry-run to preview which pods can be rescheduled may mislead users and cause unexpected issues.

Describe the solution you'd like

In dry-run mode, pod scheduling can be simulated to find fitting nodes. This enhances the accuracy of the dry-run results.
Describe alternatives you've considered

What version of descheduler are you using?
master
descheduler version:

Additional context
@zhifei92 zhifei92 added the kind/feature Categorizes issue or PR as related to a new feature. label Dec 25, 2024
@zhifei92 zhifei92 linked a pull request Dec 25, 2024 that will close this issue
Open
@zhifei92
Copy link
Contributor Author

/assign

@googs1025
Copy link
Member

IIUC: Descheduler dry run mode

  • checks and reports which Pods can be evicted, but does not actually perform these operations.
  • No resource deduction: Because Pods are not actually evicted or rescheduled, resources in the cluster are not actually deducted.
  • No simulation of scheduling: In dry run mode, Descheduler does not simulate the rescheduling process, that is, it does not try to find a new suitable node for the evicted Pod.

So I understand the original mechanism to be reasonable.

@zhifei92
Copy link
Contributor Author

Here is an example:

  • In a cluster with four nodes (A, B, C, D), after a certain rescheduling strategy calculation, it is determined that two Pods with resource requirements of 2 CPU cores and 4 GB of memory need to be evicted from node A.
  • Node B can accommodate one 2c/4g Pod, while nodes C and D can only handle Pods with requirements of 1.5 CPU cores and 3 GB of memory.

In a non-dry-run scenario, only one Pod on node A can be evicted because the second Pod cannot be successfully rescheduled onto another node. However, in dry-run mode, both Pods on node A are reported as evictable, which is inaccurate since it does not consider the feasibility of rescheduling these Pods onto other nodes.

@zhifei92
Copy link
Contributor Author

No simulation of scheduling

Indeed, we don't need to simulate scheduling in a dry-run scenario.

More accurately, we should reserve the resources for the Pods that can be evicted in a dry-run scenario, ensuring the accuracy of Pod evictions in dry-run mode. This means that when identifying which Pods could be evicted, the Descheduler should account for the resource usage of those Pods as if they were actually being evicted and not rescheduled, thus providing a more accurate reflection of what would happen in a non-dry-run operation.

@zhifei92
Copy link
Contributor Author

In real-world usage scenarios, users would first use the dry-run mode to confirm how many Pods and which specific Pods can be evicted. If in practice only 1 Pod can actually be evicted, but the dry-run mode reports that 10 Pods can be evicted, this is unacceptable. Users rely on the accuracy of dry-run outputs to plan and understand the impact of eviction actions, so the dry-run results must realistically reflect what would happen in an actual operation.

@googs1025
Copy link
Member

After the offline talking , I understood the background and I think that this issue was reasonable. 🤔

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@zhifei92 zhifei92

Labels
kind/feature Categorizes issue or PR as related to a new feature.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Perform a scheduling simulation in dry-run mode. zhifei92/descheduler
2 participants
@zhifei92 @googs1025