Go: Memory leak in map

While browsing Go’s issues I came across this one— runtime: maps do not shrink after elements removal. I decided to reproduce the issue with this code —[1]

runtime.GC() was reducing the memory footprint. So why did the op’s app crash? I started discussion on the above issue and got some interesting insights (I will write another article on how map data structure works in Go).

1. Go zeroes the key and value on delete operation.
2. This means If key/value is a pointer, it will be replaced by nil pointer and the memory pointed will be recollected.
3. However non pointer data structures like primitives, arrays will be replaced by their zero values, not freeing any space. For e.g. [100]int takes same space, with or without values.
4. That means the space previously allocated is never freed as Go simply zeroes it, rather than

maps are always increasing in Go. They can never decrease

So the string values in the buckets of the map are reduced to empty string. All the strings are collected by GC. However the map still takes around 9MB space. This is the space of the buckets(array), Go allocated to old 400K pointers(strings are pointers) for keys and values.

I decided to create an example without pointers. In [2], I used [640]byte array as both key and value for the maps. Since its zero value is same size I expected GC not to affect size, however GC was still able to collect the deleted keys and values. Because —

Go allocates data more than 128 bytes from heap and save pointers in the bucket.

So this time I tried with 120 bytes and it worked. GC failed to collect the deleted keys and values.

Summary

1. Go maps are monotonically increasing, as you add more keys and values, the size increases. Deleting keys however doesn’t shrinks it.
2. Solution is to create new map after you have deleted enough keys (half?). Go will garbage collect the old map as it is not referenced anymore.
3. Go will however free the space pointed by pointer keys and values.

[1] https://play.golang.org/p/pdZls3ljYb9
[2] https://play.golang.org/p/7mMcOruxuG_t