Go as a compilation target

[lucasavila00]

package main

import (
	"fmt"
)

type Tuple2[T any, U any] struct {
	First  T
	Second U
}
type Tuple3[T any, U any, V any] struct {
	First  T
	Second U
	Third  V
}
type Tuple4[T any, U any, V any, X any] struct {
	First  T
	Second U
	Third  V
	Fourth X
}

// tuple matching
func go_(x Tuple2[int, int]) Tuple2[int, int] {
	if x.First != 0 || x.Second != 2 {
		panic("assignment_no_match")
	}
	return x
}

// assert
func go_1(x int) int {
	if x != 1 {
		panic("assignment_no_match")
	}
	return x
}

// nested_binding
func go_3(x Tuple4[any, Tuple3[any, any, int], any, int]) Tuple4[any, Tuple3[any, any, int], any, int] {
	if x.Second.Third != 2 || x.Fourth != 1 {
		panic("assignment_no_match")
	}

	a := x.First
	_ = a
	t := x.Second
	_ = t
	b := x.Second.First
	_ = b
	c := x.Second.Second
	_ = c

	return x
}

// variable_renaming
func go_4(x Tuple2[int, int], foo func(int) any) Tuple2[int, int] {
	a := 1
	foo(a)
	a_1 := 2
	foo(a_1)
	if x.Second != 3 {
		panic("assignment_no_match")
	}
	a_2 := x.First
	b := a_2
	foo(b)
	c := func() Tuple2[int, int] {
		a_3 := a_2
		return Tuple2[int, int]{a_3, b}
	}()
	foo(a_2)
	x_1 := c

	return x_1
}

// generics & polymorphism

type Animal_Variant_Dog[A any] struct {
	dog_content A
}

func (f Animal_Variant_Dog[any]) inspect() string {
	return fmt.Sprintf("Dog(%v)", f.dog_content)
}

type Animal_Variant_Cat[A any] struct {
	cat_content A
}

func (f Animal_Variant_Cat[any]) inspect() string {
	return fmt.Sprintf("Cat(%v)", f.cat_content)
}

type Animal[A any] struct {
	dog *Animal_Variant_Dog[A]
	cat *Animal_Variant_Cat[A]
}

func receives_animal[A any](animal Animal[A]) {

	if animal.cat != nil {
		fmt.Printf("animal.cat.inspect(): %v\n", animal.cat.inspect())
	} else if animal.dog != nil {
		fmt.Printf("animal.dog.inspect(): %v\n", animal.dog.inspect())
	}

}

func main() {

	cat := Animal_Variant_Cat[string]{cat_content: "meow"}
	an := Animal[string]{cat: &cat}
	receives_animal(an)

	dog := Animal_Variant_Dog[string]{dog_content: "woof"}
	an2 := Animal[string]{dog: &dog}
	receives_animal(an2)

}

[lpil]

Could be cool! Go has a lot of nice things about it.

Before we implement a backend we would want to hand-compile some Gleam code to Go and other options such as C and ChezScheme and benchmark them to see what the performance is like. I suspect Go will be slower as it is optimised for a low number of allocations and lots of mutation, which is the inverse of Gleam

[timjs]

If we'd take that route, it will be more difficult and way more work to compile Gleam performantly in the first place. Wasm gives you a lot for free (GC, functions, modules) and is designed to be performant and easy to compile to.

[lpil]

I believe WASM does not have GC yet, and it will be a one size fits all GC rather than being one optimised for the language. Other compilation targets will have similar advantages but without the disadvantages of no threads, worse performance, less mature tooling, etc.

I'm not convinced there are advantages of going directly to WASM yet.

[leobm]

If you have a Go target, you can compile to WASM afterwards (e.g. either with the native go implementation or tinygo).

[rockwotj]

I actually spent some time looking into this and determined that golamg has annoyances as a codegen target for Gleam

1. The package/module systems are completely different. Golang works on package directories and the files make no difference. I suppose you could generate a directory per Gleam module but that's sort of awkward.
2. A good litmus test is implementing the map type for Gleam IMO, I think that even though go has generics now, creating a map type requires a hash function to be passed in, and that would require a divergence from the erlang and js APIs.
3. Golang has a lot of annoying naming restrictions. Currently Gleam's js backend uses $ to namespace internal symbols from user supplied symbols. Only Unicode letters and digits are valid identifiers in golang. Although I suppose an obscure Unicode letter could suffice here
4. Go visibility works by the convention of uppercase vs lowercase letters, but gleam has different opinions about uppercase being for types and lowercase being for others symbols.

Nothing here is insurmountable, and this could work, but I've been hacking on a C++ backend instead. As part of this work I'm creating a lower level intermediate representation that should greatly reduce the work of a golang port.

[rockwotj]

Tracking of a native backend for the compiler is here #109

[cottand]

👋 I'd like to revive this conversation

TL;DR: I think Go is a very good fit to be a backend for Gleam, and there is a real use-case for it in both lightweight, standalone CLIs and long-running applications with smaller footprints than a Javascript runtime or, maybe, the BEAM. Go library support would also improve Gleam library support, which is a pain-point for any young language.

I think the following are good reasons to want to use Gleam with a Go backend:

• Go has a wealth of existing type-safe libraries that would unlock widespread compatibility. Because Go is so popular, these libraries’ maintainers can be more active than Erlang’s alternatives, or can be official maintainers. Examples that come to mind include libraries around gRPC, Kubernetes, Hashicorp (Vault, Terraform…), etc.

  Because Go’s libraries are type-safe, it is possible to generate type-safe Gleam APIs for Go libraries.

• When not using CGO, a Go backend unlocks static, self-contained, native binaries for Gleam. This would allow us to build Gleam applicatoins that are much easier to package (into debian, arch, or windows binaries) and do not need runtime dependencies (be it a VM in BEAM/JVM-style, or dynamic libs, in C/C++ style). Essentially you get Native or wasm compiler backend for Gleam #109 but easier and better.

Additionally, the Go runtime has features that would facilitate making sure that existing Gleam code targeting Erlang is also compatible with a Go-based compiler:

• Go has an excellent stdlib
• Goroutines are a good replacement for Erlang processes — they are also user-level-managed and more lightweight than threads. They also grow their stack dynamically, which means they can do deep-recursion when calls are not tail-recursive.
• Go channels are a good potential building block for implementing messaging in actor-based frameworks like OTP

Lookig much longer term, further work could unlock value for Gleam:

• A shortcoming of the JS backend is that it does not have the concurrency mechanisms the BEAM has. But Go can compile to WASM, so if Gleam actors were successfully ported to Go, they could be run on a browser too.

  More generally, Go-based WASM allows library harmony between browser vs server because the same underlying libraries can be reused.

• If the Gleam compiler is ever rewritten in Gleam, a Go/Native backend would allow it to be a standalone binary (as opposed to BEAM/JS programs, which require additional dependencies)

I understand these would be a much later steps for Gleam, but it is a good example of what Go could potentially open the door to.

For completeness, I do think there would be some shortcomings in a Go backend for Gleam:

• Go has plenty of memory guarantees, but immutability is not one of them. When reading structs passed to it via a Go library, there is a risk of unexpected mutation in pointer-addressed memory (although I believe there is risk of this in JS too).
• Yet another backend might further splinter the Gleam library ecosystem. I think that to mitigate this, as much as possible should be implemented in a platform-agnostic way in the stdlib directly, as opposed to platform-specific libs like gleam/erlang. I understand this is tricky as long as JS is present and its concurrency remains so fundamentally different.

Addressing @lpil 's comment:

Go will be slower as it is optimised for a low number of allocations and lots of mutation, which is the inverse of Gleam

While true, a lot of work can still be done on Gleam's end to make sure it reduces the number of allocations. Specifically, because Gleam does not mutate, functions can share structs between each other, rather than copying them every time. But I do think we'll never know how bad it is until we run some benchmarks! Right now I am writing a very dumb transpiler based on glance, and hopefully I can gather some numbers to make a stronger case.

I understand all of this is much easier said than done! It's a big shift and it might not be worth it right now. I am not fluent in Rust (my jobs are usually Go and JVM), but if this is a direction Gleam wants to take, I am willing to get up to speed to help with compiler development or help with the library integration side of things.

[lpil]

Unfortunately you cannot work to avoid allocations, FP requires many fast small allocations, which is the opposite of what Go is highly optimised for. Go doesn't have any type safe libraries as its type system isn't powerful enough, but in the context of code generation target type safety tends to be a hindrance rather than a benefit as you have to manage two type systems at once.

A Go target would help us achieve very few of our goals so it is not going to happen officially. Someone could fork the compiler to make an unofficial backend if they wanted, much like the exciting C, C#, wasm, Nix, and Scheme backends.

[cottand]

I understand you'll never be able to make the same amount of allocations in FP as opposed to more imperative languages. What I am hoping is that the tradeoff will still be worth it thanks to AOT compilation.

I know about some of the other compiler forks. The point that I was trying to make is that you'll get many of these for free by just supporting Go: it can be compiled to WASM, and you can use CGO to bind to Nix and other C libraries. On top of that, you can get OTP-like abstractions (goroutines and channels) in these backends kind of for free, too.

I understand that this might not be what you want to focus on right now though!

edit: that these forks exist means there is demand for a Native backend, but this demand can only be fulfilled by the upstream compiler, because using a soft-fork (due to #3352) is a risky and hard-to-maintain process. So in practice these forks are much harder to use in prod in real applications, because they do not have the guarantees the main compiler has.

[lpil]

Go doesn't meet many of Gleam's goals, and the ones it does meet other compilation targets such as C and Scheme meet more successfully with fewer drawbacks. There will not be any official Go related changes to the compiler.