diff --git a/README.md b/README.md index 91ceb1e..febbdff 100644 --- a/README.md +++ b/README.md @@ -2,4 +2,24 @@ I've just started a new job at [Datadog](https://www.datadoghq.com/) to work on [Continuous Profiling](https://www.datadoghq.com/product/code-profiling/) for Go. To make sure that I know what I'm talking about, I'm planning to do an in-depth study of the existing profilers and how they work. I'll try to summarize what I learned in this repository as it might be useful to others. -For now there is nothing worth looking at. Please come back later : ) \ No newline at end of file +For now there is nothing worth looking at. Please come back later : ) + +## External Links + +- Go Docs + + - [Diagnostics](https://golang.org/doc/diagnostics.html): Has a very good overview over the available profiling and tracing facilities but doesn't go into a lot of depth. + - [runtime/pprof](https://golang.org/pkg/runtime/pprof/#Profile): Lists the available profiles and has a little more explanation about what kind of data they produce. + - [runtime](https://golang.org/pkg/runtime/): Has documentation on the various control knobs and pprof facilities, e.g. `MemProfileRate`. + - [net/http/pprof](net/http/pprof): Not a lot of docs, but diving into the code from there shows how the various profilers can be started/stopped on demand. + +- JDB + + - [Profiler labels in Go](https://rakyll.org/profiler-labels/): An introduction to using pprof labels and how they allow you to add additional context to your profiles. + - [Custom pprof profiles](https://rakyll.org/custom-profiles/): Example for using custom profiles, shows tracking open/close events of a blob store and how to figure out how many blobs are open at a given time. + - [Mutex profile](https://rakyll.org/mutexprofile/): Brief intro to the mutex profile. + - [Using Instruments to profile Go programs](https://rakyll.org/instruments/): How to use the macOS Instruments app (I think it's built on dtrace) to profile Go programs. Not clear what the benfits are, if any. + +- [Profiling Go programs with pprof](https://jvns.ca/blog/2017/09/24/profiling-go-with-pprof/) by Julia Evans: A nice tour with a focus on heap profiling and the pprof output format. + + \ No newline at end of file diff --git a/examples/heap.go b/examples/heap.go new file mode 100644 index 0000000..74ca7b1 --- /dev/null +++ b/examples/heap.go @@ -0,0 +1,78 @@ +// +build ignore + +package main + +import ( + "context" + "fmt" + "log" + "net/http" + _ "net/http/pprof" + "os" + "runtime" + "time" + + "golang.org/x/sync/errgroup" +) + +func main() { + if err := run(); err != nil { + fmt.Fprintln(os.Stderr, err) + os.Exit(1) + } +} + +func run() error { + runtime.MemProfileRate = 1 + + g, _ := errgroup.WithContext(context.Background()) + + g.Go(func() error { + addr := "localhost:6060" + log.Printf("Listening on %s", addr) + return http.ListenAndServe(addr, nil) + }) + + //g.Go(allocStuff) + + // Memory leak that leaks 1 kB / sec at a rate of 10 Hz. + g.Go(func() error { return leakStuff(1024, 100*time.Millisecond) }) + g.Go(func() error { return allocStuff(1024, 100*time.Millisecond) }) + g.Go(func() error { return forceGc(time.Second) }) + + return g.Wait() +} + +var leak []*Data + +// leakStuff leaks ~bytes every interval. +func leakStuff(bytes int, interval time.Duration) error { + for { + leak = append(leak, newData(bytes)) + time.Sleep(interval) + } +} + +// allocStuff is allocating things but not leaking them. +func allocStuff(bytes int, interval time.Duration) error { + for { + newData(bytes) + time.Sleep(interval) + } +} + +// forceGc forces a GC to occur every interval. +func forceGc(interval time.Duration) error { + for { + time.Sleep(interval) + runtime.GC() + } +} + +func newData(size int) *Data { + return &Data{data: make([]byte, size)} +} + +type Data struct { + data []byte +} diff --git a/heap.md b/heap.md new file mode 100644 index 0000000..38c97e1 --- /dev/null +++ b/heap.md @@ -0,0 +1,36 @@ +# Go's Heap Profiler + +Totally a work in progress ... !!! Please come back later : ). + +## How does it work? + +Look at the go source code to understand how the data is captured, what role runtime.MemProfileRate plays, etc. + +## How does it fail? + +Think through sampling rate issues, alloc size classes, etc. + +## Data Format + +Figure out how the data ends up in the pprof file. + +## GC Control + +``` +# turn of gc +GOGC=off go run +# print gc events to stdout +GODEBUG=gctrace=1 go run +``` + +## Questions + +- What are the [docs](https://golang.org/pkg/runtime/pprof/#Profile) talking about here? How do I actually use this? + + > Pprof's -inuse_space, -inuse_objects, -alloc_space, and -alloc_objects flags select which to display, defaulting to -inuse_space (live objects, scaled by size). + + A: Those flags are deprecated. Easiest way to select this stuff is via the pprof web ui's sample drop down. + +- The [docs](https://golang.org/pkg/runtime/pprof/#Profile) say I should get some kind of data, even if there is no GC. I can reproduce that, but the data seems to not change? + + > If there has been no garbage collection at all, the heap profile reports all known allocations. This exception helps mainly in programs running without garbage collection enabled, usually for debugging purposes. \ No newline at end of file