github.com/gogf/gf/v2/util/gvalid: valid binary field,a large amount of memory is consumed

[redwolf2019]

Go version

go version go1.23.2 darwin/arm64

GoFrame version

2.7.4

Can this bug be reproduced with the latest release?

Option Yes

What did you do?

package main

import (
	"context"
	"github.com/gogf/gf/v2/frame/g"
	"github.com/gogf/gf/v2/os/gfile"
	"github.com/gogf/gf/v2/os/gproc"
)

func main() {
	in := &Model{Raw: gfile.GetBytes("/Users/redwolf/Downloads/50MB.zip")}

	if err := g.Validator().Data(in).Run(context.Background()); err != nil {
		g.Log().Fatal(context.Background(), err)
	}

	gproc.Listen()
}

type Model struct {
	Raw []byte `v:"required"`
}

When I validate a 50MB []byte field, the actual memory usage approaches 1GB.

What did you see happen?

High memory usage with delayed garbage collection.

What did you expect to see?

It shouldn't consume this much memory.

[stardemo]

debug了一下 看起来是 array = gconv.Interfaces(in.Value)的问题

[wlynxg]

gvalid 包在对 slice 结构做校验时，会使用 gconv.Interfaces() 函数将 slice 转换为 []interface{} 类型，：

gf/util/gvalid/gvalid_validator_check_value.go

Lines 289 to 295 in 80f57d1

	case reflect.Slice, reflect.Array:
	var array []interface{}
	if gjson.Valid(in.Value) {
	array = gconv.Interfaces(gconv.Bytes(in.Value))
	} else {
	array = gconv.Interfaces(in.Value)
	}

gconv.Interface() 在做转换时，对于 []uint8 和 []byte 会展开转换为 []interface{}，由于 []byte 较大，所以在这里会导致内存暴涨：

gf/util/gconv/gconv_slice_any.go

Lines 66 to 74 in 80f57d1

	case []uint8:
	if json.Valid(value) {
	_ = json.UnmarshalUseNumber(value, &array)
	} else {
	array = make([]interface{}, len(value))
	for k, v := range value {
	array[k] = v
	}
	}

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

When the gvalid package verifies the slice structure, it will use the gconv.Interfaces() function to convert slice into []interface{} Type,:

gf/util/gvalid/gvalid_validator_check_value.go

Lines 289 to 295 in 80f57d1

	case reflect.Slice, reflect.Array:
	var array []interface{}
	if gjson.Valid(in.Value) {
	array = gconv.Interfaces(gconv.Bytes(in.Value))
	} else {
	array = gconv.Interfaces(in.Value)
	}

When gconv.Interface() performs conversion, []uint8 and []byte will be expanded and converted into []interface{}, because []byte is larger, so it will cause memory explosion here:

gf/util/gconv/gconv_slice_any.go

Lines 66 to 74 in 80f57d1

	case []uint8:
	if json.Valid(value) {
	_ = json.UnmarshalUseNumber(value, &array)
	} else {
	array = make([]interface{}, len(value))
	for k, v := range value {
	array[k] = v
	}
	}