Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: add IgnorePodsWithResourceClaims parameter for configuration #1603

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

feature: add IgnorePodsWithResourceClaims parameter for configuration #1603

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 14 additions & 13 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,19 +133,20 @@ These are top level keys in the Descheduler Policy that you can use to configure

The Default Evictor Plugin is used by default for filtering pods before processing them in an strategy plugin, or for applying a PreEvictionFilter of pods before eviction. You can also create your own Evictor Plugin or use the Default one provided by Descheduler. Other uses for the Evictor plugin can be to sort, filter, validate or group pods by different criteria, and that's why this is handled by a plugin and not configured in the top level config.

| Name |type| Default Value | Description |
|---------------------------|----|---------------|-----------------------------------------------------------------------------------------------------------------------------|
| `nodeSelector` |`string`| `nil` | limiting the nodes which are processed |
| `evictLocalStoragePods` |`bool`| `false` | allows eviction of pods with local storage |
| `evictSystemCriticalPods` |`bool`| `false` | [Warning: Will evict Kubernetes system pods] allows eviction of pods with any priority, including system pods like kube-dns |
| `ignorePvcPods` |`bool`| `false` | set whether PVC pods should be evicted or ignored |
| `evictFailedBarePods` |`bool`| `false` | allow eviction of pods without owner references and in failed phase |
| `labelSelector` |`metav1.LabelSelector`|| (see [label filtering](#label-filtering)) |
| `priorityThreshold` |`priorityThreshold`|| (see [priority filtering](#priority-filtering)) |
| `nodeFit` |`bool`|`false`| (see [node fit filtering](#node-fit-filtering)) |
| `minReplicas` |`uint`|`0`| ignore eviction of pods where owner (e.g. `ReplicaSet`) replicas is below this threshold |
| `minPodAge` |`metav1.Duration`|`0`| ignore eviction of pods with a creation time within this threshold |
| `ignorePodsWithoutPDB` |`bool`|`false`| set whether pods without PodDisruptionBudget should be evicted or ignored |
| Name | type | Default Value | Description |
|--------------------------------|------------------------|-------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------|
| `nodeSelector` | `string` | `nil` | limiting the nodes which are processed |
| `evictLocalStoragePods` | `bool` | `false` | allows eviction of pods with local storage |
| `evictSystemCriticalPods` | `bool` | `false` | [Warning: Will evict Kubernetes system pods] allows eviction of pods with any priority, including system pods like kube-dns |
| `ignorePvcPods` | `bool` | `false` | set whether PVC pods should be evicted or ignored |
| `evictFailedBarePods` | `bool` | `false` | allow eviction of pods without owner references and in failed phase |
| `labelSelector` | `metav1.LabelSelector` || (see [label filtering](#label-filtering)) |
| `priorityThreshold` | `priorityThreshold` || (see [priority filtering](#priority-filtering)) |
| `nodeFit` | `bool` | `false` | (see [node fit filtering](#node-fit-filtering)) |
| `minReplicas` | `uint` | `0` | ignore eviction of pods where owner (e.g. `ReplicaSet`) replicas is below this threshold |
| `minPodAge` | `metav1.Duration` | `0` | ignore eviction of pods with a creation time within this threshold |
| `ignorePodsWithoutPDB` | `bool` | `false` | set whether pods without PodDisruptionBudget should be evicted or ignored |
| `ignorePodsWithResourceClaims` | `bool` | `false` | set whether pods with resource claims should be evicted or ignored. |

### Example policy

Expand Down
3 changes: 3 additions & 0 deletions pkg/apis/componentconfig/types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,9 @@ type DeschedulerConfiguration struct {
// IgnorePVCPods sets whether PVC pods should be allowed to be evicted
IgnorePVCPods bool

// IgnorePodsWithResourceClaims sets whether pods using resource claims should be allowed to be evicted
IgnorePodsWithResourceClaims bool

// Tracing specifies the options for tracing.
Tracing TracingConfiguration

Expand Down
3 changes: 3 additions & 0 deletions pkg/apis/componentconfig/v1alpha1/types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,9 @@ type DeschedulerConfiguration struct {
// IgnorePVCPods sets whether PVC pods should be allowed to be evicted
IgnorePVCPods bool `json:"ignorePvcPods,omitempty"`

// IgnorePodsWithResourceClaims sets whether pods using resource claims should be allowed to be evicted
IgnorePodsWithResourceClaims bool `json:"ignorePodsWithResourceClaims,omitempty"`

// Tracing is used to setup the required OTEL tracing configuration
Tracing TracingConfiguration `json:"tracing,omitempty"`

Expand Down
2 changes: 2 additions & 0 deletions pkg/apis/componentconfig/v1alpha1/zz_generated.conversion.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 6 additions & 5 deletions pkg/descheduler/policyconfig.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,11 +101,12 @@ func setDefaultEvictor(profile api.DeschedulerProfile, client clientset.Interfac
newPluginConfig := api.PluginConfig{
Name: defaultevictor.PluginName,
Args: &defaultevictor.DefaultEvictorArgs{
EvictLocalStoragePods: false,
EvictSystemCriticalPods: false,
IgnorePvcPods: false,
EvictFailedBarePods: false,
IgnorePodsWithoutPDB: false,
EvictLocalStoragePods: false,
EvictSystemCriticalPods: false,
IgnorePvcPods: false,
EvictFailedBarePods: false,
IgnorePodsWithoutPDB: false,
IgnorePodsWithResourceClaims: false,
},
}

Expand Down
9 changes: 9 additions & 0 deletions pkg/framework/plugins/defaultevictor/defaultevictor.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,15 @@ func New(args runtime.Object, handle frameworktypes.Handle) (frameworktypes.Plug
})
}

if defaultEvictorArgs.IgnorePodsWithResourceClaims {
ev.constraints = append(ev.constraints, func(pod *v1.Pod) error {
if utils.IsPodWithResourceClaims(pod) {
return fmt.Errorf("pod has a ResourceClaim and descheduler is configured to ignore ResourceClaim pods")
}
return nil
})
}

return ev, nil
}

Expand Down
69 changes: 51 additions & 18 deletions pkg/framework/plugins/defaultevictor/defaultevictor_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ import (
"k8s.io/apimachinery/pkg/util/uuid"
"k8s.io/client-go/informers"
"k8s.io/client-go/kubernetes/fake"
"k8s.io/utils/ptr"
"sigs.k8s.io/descheduler/pkg/api"
podutil "sigs.k8s.io/descheduler/pkg/descheduler/pod"
frameworkfake "sigs.k8s.io/descheduler/pkg/framework/fake"
Expand All @@ -37,20 +38,21 @@ import (
)

type testCase struct {
description string
pods []*v1.Pod
nodes []*v1.Node
pdbs []*policyv1.PodDisruptionBudget
evictFailedBarePods bool
evictLocalStoragePods bool
evictSystemCriticalPods bool
ignorePvcPods bool
priorityThreshold *int32
nodeFit bool
minReplicas uint
minPodAge *metav1.Duration
result bool
ignorePodsWithoutPDB bool
description string
pods []*v1.Pod
nodes []*v1.Node
pdbs []*policyv1.PodDisruptionBudget
evictFailedBarePods bool
evictLocalStoragePods bool
evictSystemCriticalPods bool
ignorePvcPods bool
priorityThreshold *int32
nodeFit bool
minReplicas uint
minPodAge *metav1.Duration
result bool
ignorePodsWithoutPDB bool
ignorePodsWithResourceClaims bool
}

func TestDefaultEvictorPreEvictionFilter(t *testing.T) {
Expand Down Expand Up @@ -802,6 +804,36 @@ func TestDefaultEvictorFilter(t *testing.T) {
},
ignorePvcPods: false,
result: true,
}, {
description: "ignorePodsWithResourceClaims is not set, pod with ResourceClaims, evicts",
pods: []*v1.Pod{
test.BuildTestPod("p15", 400, 0, n1.Name, func(pod *v1.Pod) {
pod.ObjectMeta.OwnerReferences = test.GetNormalPodOwnerRefList()
pod.Spec.ResourceClaims = []v1.PodResourceClaim{
{
Name: "test-name",
ResourceClaimName: ptr.To("test-resourceclaim"),
},
}
}),
},
ignorePodsWithResourceClaims: false,
result: true,
}, {
description: "ignorePodsWithResourceClaims is set, pod with ResourceClaims, evicts",
pods: []*v1.Pod{
test.BuildTestPod("p15", 400, 0, n1.Name, func(pod *v1.Pod) {
pod.ObjectMeta.OwnerReferences = test.GetNormalPodOwnerRefList()
pod.Spec.ResourceClaims = []v1.PodResourceClaim{
{
Name: "test-name",
ResourceClaimName: ptr.To("test-resourceclaim"),
},
}
}),
},
ignorePodsWithResourceClaims: true,
result: false,
},
}

Expand Down Expand Up @@ -900,10 +932,11 @@ func initializePlugin(ctx context.Context, test testCase) (frameworktypes.Plugin
PriorityThreshold: &api.PriorityThreshold{
Value: test.priorityThreshold,
},
NodeFit: test.nodeFit,
MinReplicas: test.minReplicas,
MinPodAge: test.minPodAge,
IgnorePodsWithoutPDB: test.ignorePodsWithoutPDB,
NodeFit: test.nodeFit,
MinReplicas: test.minReplicas,
MinPodAge: test.minPodAge,
IgnorePodsWithoutPDB: test.ignorePodsWithoutPDB,
IgnorePodsWithResourceClaims: test.ignorePodsWithResourceClaims,
}

evictorPlugin, err := New(
Expand Down
6 changes: 6 additions & 0 deletions pkg/framework/plugins/defaultevictor/defaults.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,4 +52,10 @@ func SetDefaults_DefaultEvictorArgs(obj runtime.Object) {
if !args.NodeFit {
args.NodeFit = false
}
if !args.IgnorePodsWithoutPDB {
args.IgnorePodsWithoutPDB = false
}
if !args.IgnorePodsWithResourceClaims {
args.IgnorePodsWithResourceClaims = false
}
}
59 changes: 31 additions & 28 deletions pkg/framework/plugins/defaultevictor/defaults_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,47 +33,50 @@ func TestSetDefaults_DefaultEvictorArgs(t *testing.T) {
name: "DefaultEvictorArgs empty",
in: &DefaultEvictorArgs{},
want: &DefaultEvictorArgs{
NodeSelector: "",
EvictLocalStoragePods: false,
EvictDaemonSetPods: false,
EvictSystemCriticalPods: false,
IgnorePvcPods: false,
EvictFailedBarePods: false,
LabelSelector: nil,
PriorityThreshold: nil,
NodeFit: false,
IgnorePodsWithoutPDB: false,
NodeSelector: "",
EvictLocalStoragePods: false,
EvictDaemonSetPods: false,
EvictSystemCriticalPods: false,
IgnorePvcPods: false,
EvictFailedBarePods: false,
LabelSelector: nil,
PriorityThreshold: nil,
NodeFit: false,
IgnorePodsWithoutPDB: false,
IgnorePodsWithResourceClaims: false,
},
},
{
name: "DefaultEvictorArgs with value",
in: &DefaultEvictorArgs{
NodeSelector: "NodeSelector",
EvictLocalStoragePods: true,
EvictDaemonSetPods: true,
EvictSystemCriticalPods: true,
IgnorePvcPods: true,
EvictFailedBarePods: true,
LabelSelector: nil,
NodeSelector: "NodeSelector",
EvictLocalStoragePods: true,
EvictDaemonSetPods: true,
EvictSystemCriticalPods: true,
IgnorePvcPods: true,
EvictFailedBarePods: true,
LabelSelector: nil,
NodeFit: true,
IgnorePodsWithoutPDB: true,
IgnorePodsWithResourceClaims: true,
PriorityThreshold: &api.PriorityThreshold{
Value: utilptr.To[int32](800),
},
NodeFit: true,
IgnorePodsWithoutPDB: true,
},
want: &DefaultEvictorArgs{
NodeSelector: "NodeSelector",
EvictLocalStoragePods: true,
EvictDaemonSetPods: true,
EvictSystemCriticalPods: true,
IgnorePvcPods: true,
EvictFailedBarePods: true,
LabelSelector: nil,
NodeSelector: "NodeSelector",
EvictLocalStoragePods: true,
EvictDaemonSetPods: true,
EvictSystemCriticalPods: true,
IgnorePvcPods: true,
EvictFailedBarePods: true,
LabelSelector: nil,
NodeFit: true,
IgnorePodsWithoutPDB: true,
IgnorePodsWithResourceClaims: true,
PriorityThreshold: &api.PriorityThreshold{
Value: utilptr.To[int32](800),
},
NodeFit: true,
IgnorePodsWithoutPDB: true,
},
},
}
Expand Down
25 changes: 13 additions & 12 deletions pkg/framework/plugins/defaultevictor/types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,16 +25,17 @@ import (
type DefaultEvictorArgs struct {
metav1.TypeMeta `json:",inline"`

NodeSelector string `json:"nodeSelector,omitempty"`
EvictLocalStoragePods bool `json:"evictLocalStoragePods,omitempty"`
EvictDaemonSetPods bool `json:"evictDaemonSetPods,omitempty"`
EvictSystemCriticalPods bool `json:"evictSystemCriticalPods,omitempty"`
IgnorePvcPods bool `json:"ignorePvcPods,omitempty"`
EvictFailedBarePods bool `json:"evictFailedBarePods,omitempty"`
LabelSelector *metav1.LabelSelector `json:"labelSelector,omitempty"`
PriorityThreshold *api.PriorityThreshold `json:"priorityThreshold,omitempty"`
NodeFit bool `json:"nodeFit,omitempty"`
MinReplicas uint `json:"minReplicas,omitempty"`
MinPodAge *metav1.Duration `json:"minPodAge,omitempty"`
IgnorePodsWithoutPDB bool `json:"ignorePodsWithoutPDB,omitempty"`
NodeSelector string `json:"nodeSelector,omitempty"`
EvictLocalStoragePods bool `json:"evictLocalStoragePods,omitempty"`
EvictDaemonSetPods bool `json:"evictDaemonSetPods,omitempty"`
EvictSystemCriticalPods bool `json:"evictSystemCriticalPods,omitempty"`
IgnorePvcPods bool `json:"ignorePvcPods,omitempty"`
EvictFailedBarePods bool `json:"evictFailedBarePods,omitempty"`
LabelSelector *metav1.LabelSelector `json:"labelSelector,omitempty"`
PriorityThreshold *api.PriorityThreshold `json:"priorityThreshold,omitempty"`
NodeFit bool `json:"nodeFit,omitempty"`
MinReplicas uint `json:"minReplicas,omitempty"`
MinPodAge *metav1.Duration `json:"minPodAge,omitempty"`
IgnorePodsWithoutPDB bool `json:"ignorePodsWithoutPDB,omitempty"`
IgnorePodsWithResourceClaims bool `json:"ignorePodsWithResourceClaims,omitempty"`
}
5 changes: 5 additions & 0 deletions pkg/utils/pod.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,11 @@ func IsPodWithPVC(pod *v1.Pod) bool {
return false
}

// IsPodWithResourceClaims returns true if the pod has resource claims.
func IsPodWithResourceClaims(pod *v1.Pod) bool {
return len(pod.Spec.ResourceClaims) != 0
}

// IsPodCoveredByPDB returns true if the pod is covered by at least one PodDisruptionBudget.
func IsPodCoveredByPDB(pod *v1.Pod, lister policyv1.PodDisruptionBudgetLister) (bool, error) {
// We can't use the GetPodPodDisruptionBudgets expansion method here because it treats no pdb as an error,
Expand Down
Loading