Consequently, the GC has to assume that anything forward of any given slice into the underlying array may become accessible in the future as there is legal Go that can access it. It's still memory safe, but it surprised me.
I had some code that was using my incorrect belief that slices could not be resliced up in size to implement some light security boundaries. Fortunately it was still legal, because the code in question simply didn't slice things larger and it's not like I was allowing arbitrary user-supplied code to run, so it was still correct in what it was doing. But I was expecting the runtime to scream if I did somehow screw it up when in fact it may not, depending on the exact capacity and what happened when.
It's also asymmetric, as far as I know; you can slice forward into the array if there is capacity, but if you've got a slice that starts after index 0 in the backing array you can't use that slice to walk back into the underlying array. That is, with
s := []int{11, 12, 13, 14, 15}
s = s[2:]
Corrections (again not involving "unsafe", it's obvious those two values are still accessible through "unsafe") welcome. I was wrong once, it's easy to believe I could be wrong again.
s = s[:3:3]
> Functions that shrink the size of a slice (Delete, DeleteFunc, Compact, CompactFunc, and Replace) now zero the elements between the new length and the old length.
Things like:
slices.Sort(s) // correct
slices.Compact(s) // incorrect
slices.Delete(s, ...) // incorrect
s := slices.Delete(s, ...) // incorrect if 's' is referenced again in the outer scope
s = slices.Delete(s, ...) // correct
[1] Especially given that the capacity is preserved by default, which contributes to the current confusion. See my older comment: >>39112735
[2] Originally "...are different" but edited for clarity.
Understanding rsc’s “Go Slices” blog is very helpful here. Coming from Java or something, this exposure of underlying storage could be jarring, but coming from C, Go slices are basically built in fat arrays, and this behavior doesn’t surprise me. Maybe it was a design mistake to expose so much of the underlying machinery. Ymmv.
That said, I’d also point out that, while you can more or less replicate the Go example with Rust slices, in Rust it would be more idiomatic to pass around a Vec (or a mutable reference to a Vec) if a callee needs to do something like change the length. And you can’t resize a Vec if there are other references to its contents.
The Go compiler generates an error when you are (silently) ignoring the return value of any function. Or, to put it in other words, every compiler which does allow to (silently) ignore the return value of a function, should not be used at all (C++ has at least `[[nodiscard]]` since 17 and C with C23 - which is "too little and too late", as always).
If Delete or Compact are only available there and it’s modified in place, the problems don’t arise in the first place.
fn compacted<T: ...>(input: Cow<'_, [T]>) -> Cow<'_, [T]> { ... }
After seeing this signature, I think that Go is giving up on it's simpleness principle.
It does not. You can actually infer that from TFA listing cases as problematic which would be caught by such a compilation error, and confirming it by just compiling them:
a := []int{}
// compiler says nothing
slices.Delete(a, 0, 0)
> C++ has at least `[[nodiscard]]` since 17 and C with C23 - which is "too little and too late", as always
You can't even mark your own functions or types as nodiscard in Go. You need third-party tooling even just to ensure you're not unwittingly ignoring error results:
f, err := os.Create("/tmp/filename")
if err == nil {
return
}
// compiler doesn't say anything even though f.Close returns error
f.Close()I have to say that I don't understand the rationale of a compiler that errors on unused variables but lets the user silently ignore function return values. As a solution to explicitly ignore return values already exists in the language.
Even without going the whole 9 yards of the Rust memory model, something like the views in C++, or the java {Array,Linked}Lists containing references to objects and thus being resistant to re-alloc, or even plainly immutable list like in OCaml are all working and simple enough solutions to the issue.
I still can't wrap my mind around how supposedly experienced people like Pike & co. may have designed slices, then went ‶yep, that makes perfect sense for a robust, basic, widely used data structure to create our language around″, outside of a few toy programs.
As a language design though Elm remains remarkably simple with parametric polymorphism (aka Generics) but it needs other design choices to do so. Elm is the Go of Haskell :-)
Both are type-constraining annotations, so why two syntaxes?
slices.Sort(s) // fine
slices.Compact(s) // broken
s = slices.Compact(s) // fine
s := slices.Compact(s) // broken (!!!)
slices.Delete(s, …) // broken
s = slices.Delete(s, …) // fine
How on earth did it take twelve years for this “simple” language to make a function to delete elements from an array, with `s = append(s[:start], s[end:]...)` having to suffice until then? How on earth does the “better” API twelve years later have such a gaping footgun? This is a core type that quite literally every program uses. How have things gone so far off the rails that “setting the obsolete pointers to nil” is such an intractable problem for end users they had to add a new keyword to the language?
For other languages I see posts where weird language corner cases bring up challenging issues that really reinforce the idea that language design is hard. Rust—for example—has unsoundness issues in corner cases of the type system. But for go, it feels like there’s a constant stream of own goals on core areas of the language where the design should have knocked it out of the park. “Simple manipulation of arrays” just should not have this many footguns.
I like the way Rust has it a lot more but it is still imperfect and feels arbitrary in ways.
I find myself unsure where to put the templatization parameters and it doesn't feel perfectly symmetrical.
Unsure if impl MyType<T> where T: X
is comperable to impl MyType<T: X> Sometimes it needs impl<T> MyType<T> where T: X
Always assume that only the return value of slice mutating functions are the valid reference and the old one always invalid. This is not always completely accurate, but it is very useful in that, it is also never "wrong".
So if the type parameter said "[]E" where E is a comparable then "Foo" wouldn't be allowed because Foo is not the same type as "[]int" (for that you'd need a type alias i.e "type Foo = []int". This is by design since it allows you to define new types you don't want to get accidentally used where you don't want them to.
When defining library functions that are truly generic you may want instead to let them be used on the "underlying type". For example, Sort library authors decided that generally you want it to just work on the underlying type and that's why they added ~ in front of the generic type. Otherwise every invocation of Sort you'd need to convert the type e.g. "slices.Sort([]int(myfoo))".
The other type parameter "E comparable" doesn't need the tilde because "comparable" is not an actual type but a type constraint.
A type constraint is an interface that defines the set of permissible type arguments for the respective type parameter and controls the operations supported by values of that type parameter.
You don't need a tilde because any type that implements that constraint will be accepted. Also any type implementing a constraint that embeds that constraint also works
The first "fine" scenario is fine because `slices.Sort` works in place, it doesn't return a value at all.
And the other "fine" versions do essentially what you advocate, by overwriting the invalid reference with the new one.
It must be said that even splitting them properly you'd have issues as long as you mix slices and mutability (e.g. take a full slice out of a vector then delete() on the vector, the slice will see the hole). Though the issues would be less prominent.
slices.Compact(s) // modified slice in the return value is ignored
s = slices.Compact(s) // s now points to the new changed slice
s := slices.Compact(s) // s already exists, this is not valid Go syntax.
slices.Delete(s, …) // modified slice in the return value is ignored
s = slices.Delete(s, …) // s now points to the new changed slice
`func Index(s interface{}, v interface{}) int` both has to deal with incompatible types, & the function body is going to be anything but simple
(sure, without generics most people wouldn't even write that `func Index`, but ultimately there's plenty of libraries deep in reflect that'd greatly benefit from generics)
I've also been dealing with temporal.io using interface{} everywhere, got to a point where I wrote typed wrappers using generics to have type checking for signals
That is not really correct, and is much of the issue: Compact and Delete operate in-place on the backing buffer while copying the slice metadata. This is the same issue as the append() builtin and why it's so fraught (before you get in all the liveness stuff).
> s := slices.Compact(s) // s already exists, this is not valid Go syntax.
s := []int{}
if true {
s := slices.Compact(s)
_ = s
}As that's the case it's indeed hard to defend it. Data structure-wise it still kind of makes sense since as you mentioned the slice metadata is changed, but it basically making the old slice invalid is rather annoying.
For the := example sure, it's a bit far fetched example and likely would not pass any code review, but there are cases where shadowing is indeed valid. So is the `s := slices.Compact(s)` in this example not working as expected then?
EDIT: looking at another reply to the parent the := being broken likely is trying to point that using := also modifies the slice and thus the original slice is broken when one tries to use it in the original scope. That's really bad practice, but true indeed.
Looking at the commit log I found out that `string_list.h` in the project was something that I could use. There's functions and initializers for duplicating and not duplicating the string. So I chose `STRING_LIST_INIT_DUP` and used `string_list_split` on `\n`.[2] Then I could use a regular for-loop since the struct has `nr` which is the count. And then it just worked.
I was pleasantly surprised that there was such a nice library for "list of strings" in C.[2] And the price you pay compared to more modern languages is an extra allocation (or one per sub-string?). And although I have only used it for that one thing I didn't run into any footguns.
Languages designed way after C (like in this century) should keep in mind what they want to give the user the ability to do: just make simple immutable types (like `String` in Java (of course there's `StringBuilder)), give immutable/mutable pairs, and also if they want to give access to no-allocation slices (like Rust). And if they go all the way with this they really need to give a large enough API surface so that all of these different concerns don't get mixed up. And so that users don't have to think about the underlying arrays all the time in order to not do something unintentional.
[1] Apparently `strtok` is not nice to use because it can change the string you pass to it or something. I didn't read a lot about it.
[2] The documentation tells you when you should be using dup/no-dup. And it makes sense to duplicate/allocate in this case, I think. Intuitively to me it seems that I'm paying for an allocation in order to not have to mess up per-char iteration at all. And that was fine for my use of it.
[3] And that the macro stopped me from going down the wrong path with `strtok`!
Agreed, although conversely if we could start Rust development all over again I'd probably argue that must_use on functions (as opposed to on types) should probably be the default behavior. (These days it basically is the default behavior for functions in the standard library.) Though Rust didn't gain the ability to annotate functions with must_use until 1.27. Switching the default could perhaps be a good candidate for an Edition.
I had to dig a little and in fact, once we remember that a slice is a view into an array and that "some" of these methods return slices, it's actually fine.
The only issue is perhaps s:=slices.Compact() But that's not even because of the API. It's because of the short variable assignment that may allow shadowing, unfortunately.
The issue is probably overblown here.
To be even more thorough, I've had the pleasure (lol) to implement a wrapper to have some form of immutable slices so I can say that it is mitigable. Not pleasant but mitigable. (I needed to compare values of a slice stored in a map, value before retrieval from map vs. value when it had been inserted back into the map, so having aliasing was problematic since the value in the map would be updated at the same time the value retrieved was (obviously, duh!) , had to implement a copy-on-write variant).
:)
Only in the meme sense, it does not actually solve or help solve the problem in any way.
It’s ugly. Would not recommend. 0/10
But then it's a problem of understanding what slices are so it does help in practice.
I am more concerned by the difficulty of mutating a slice (deleting and appending element) while iterating over it for instance. Done it and that's more difficult, ceremonious.
By having this weird API on slices, they force you to be explicit on allocating new memory.
No, the problem is what I put at the top, that slices are hopelessly confused, they have two completely different and incompatible purposes and Go does not let you differentiate between the two.
Understanding what slices are does not help, neither in theory nor in practice.
s := []int{}
for i := range 29754 {
s = append(s, i)
}
var s = new List();
foreach(var i in Enumerable.Range(0, 29754)) {
s.Add(i);
}The trick in understanding what they are is to understand that these are not vectors if I try to get closer to your semantics. Once it is viewed purely as a kind of reference type, a view in a backing array, it has only one meaning.
It's easier for a traditional functional language to implement lists and vectors perhaps because they operate on the premise of immutability first. Beware of the memory footprint though.
I admit that it might be easier to think in term of vectors. That's kind of what was done designing the wrapper.
Still, as I understand, slices are just lower-level. Immutability is a higher, language level concept.
This property complicates the semantics of slices.Delete() in an interesting way:
It is out of scope of the current proposal to write anything to the original tail
I also fail to see how this would translate to
> make programmers mindful of the cost of allocating new space on array operations.
anyway. `append` is not taking in an allocator, reporting allocation failures, or reporting reallocations here. Like proper vectors, it's silently reallocating (and over-allocating) as needed.
Not at all. With e.g. `append` being a “maybe I'll allocate, maybe not, take a guess” kind of method, it's basically just like Java's `List.add` with extra steps and much worse egonomics.
> Several new functions (Insert, Replace, Delete, etc.) modify the slice. To understand how they work, and how to properly use them, we need to examine the underlying structure of slices. [My emphasis]
What this is telling us, although perhaps the author doesn't appreciate, is that this isn't an abstraction. If it was an abstraction we don't need to "examine the underlying structure of slices" in order to properly use them, we can just treat them as opaque. But in Go that's not an option, intimate knowledge of the inner workings is crucial to correct programming in Go.
And that's not necessarily a problem for its main authors, who have anyway got such an understanding, but it should have given pause to somebody who thought about how you can effectively teach others to use this language. For Go in particular that's a huge problem because it's intended as a language for software engineering and for that purpose you do want to rely heavily on abstraction.
I don't quite get what you mean by 'broken' here. You know that the length of a slice can't be altered by passing it to a function. So clearly s will still contain the same number of elements as it did before the call to Compact. Similarly for Delete.
You can ignore the return value of the function and that might introduce a bug in your code. But that's something that could happen with all kinds of functions.