some stuff

README.md

@@ -2,4 +2,24 @@
 
 I've just started a new job at [Datadog](https://www.datadoghq.com/) to work on [Continuous Profiling](https://www.datadoghq.com/product/code-profiling/) for Go. To make sure that I know what I'm talking about, I'm planning to do an in-depth study of the existing profilers and how they work. I'll try to summarize what I learned in this repository as it might be useful to others.
 
-For now there is nothing worth looking at. Please come back later : )
+For now there is nothing worth looking at. Please come back later : )
+
+## External Links
+
+- Go Docs
+
+  - [Diagnostics](https://golang.org/doc/diagnostics.html): Has a very good overview over the available profiling and tracing facilities but doesn't go into a lot of depth.
+  - [runtime/pprof](https://golang.org/pkg/runtime/pprof/#Profile): Lists the available profiles and has a little more explanation about what kind of data they produce.
+  - [runtime](https://golang.org/pkg/runtime/): Has documentation on the various control knobs and pprof facilities, e.g. `MemProfileRate`.
+  - [net/http/pprof](net/http/pprof): Not a lot of docs, but diving into the code from there shows how the various profilers can be started/stopped on demand.
+
+- JDB
+
+  - [Profiler labels in Go](https://rakyll.org/profiler-labels/): An introduction to using pprof labels and how they allow you to add additional context to your profiles.
+  - [Custom pprof profiles](https://rakyll.org/custom-profiles/): Example for using custom profiles, shows tracking open/close events of a blob store and how to figure out how many blobs are open at a given time.
+  - [Mutex profile](https://rakyll.org/mutexprofile/): Brief intro to the mutex profile.
+  - [Using Instruments to profile Go programs](https://rakyll.org/instruments/): How to use the macOS Instruments app (I think it's built on dtrace) to profile Go programs. Not clear what the benfits are, if any.
+
+- [Profiling Go programs with pprof](https://jvns.ca/blog/2017/09/24/profiling-go-with-pprof/) by Julia Evans: A nice tour with a focus on heap profiling and the pprof output format. 
+
+  

examples/heap.go

@@ -0,0 +1,78 @@
+// +build ignore
+
+package main
+
+import (
+	"context"
+	"fmt"
+	"log"
+	"net/http"
+	_ "net/http/pprof"
+	"os"
+	"runtime"
+	"time"
+
+	"golang.org/x/sync/errgroup"
+)
+
+func main() {
+	if err := run(); err != nil {
+		fmt.Fprintln(os.Stderr, err)
+		os.Exit(1)
+	}
+}
+
+func run() error {
+	runtime.MemProfileRate = 1
+
+	g, _ := errgroup.WithContext(context.Background())
+
+	g.Go(func() error {
+		addr := "localhost:6060"
+		log.Printf("Listening on %s", addr)
+		return http.ListenAndServe(addr, nil)
+	})
+
+	//g.Go(allocStuff)
+
+	// Memory leak that leaks 1 kB / sec at a rate of 10 Hz.
+	g.Go(func() error { return leakStuff(1024, 100*time.Millisecond) })
+	g.Go(func() error { return allocStuff(1024, 100*time.Millisecond) })
+	g.Go(func() error { return forceGc(time.Second) })
+
+	return g.Wait()
+}
+
+var leak []*Data
+
+// leakStuff leaks ~bytes every interval.
+func leakStuff(bytes int, interval time.Duration) error {
+	for {
+		leak = append(leak, newData(bytes))
+		time.Sleep(interval)
+	}
+}
+
+// allocStuff is allocating things but not leaking them.
+func allocStuff(bytes int, interval time.Duration) error {
+	for {
+		newData(bytes)
+		time.Sleep(interval)
+	}
+}
+
+// forceGc forces a GC to occur every interval.
+func forceGc(interval time.Duration) error {
+	for {
+		time.Sleep(interval)
+		runtime.GC()
+	}
+}
+
+func newData(size int) *Data {
+	return &Data{data: make([]byte, size)}
+}
+
+type Data struct {
+	data []byte
+}

heap.md

@@ -0,0 +1,36 @@
+# Go's Heap Profiler
+
+Totally a work in progress ... !!! Please come back later : ).
+
+## How does it work?
+
+Look at the go source code to understand how the data is captured, what role runtime.MemProfileRate plays, etc.
+
+## How does it fail?
+
+Think through sampling rate issues, alloc size classes, etc.
+
+## Data Format
+
+Figure out how the data ends up in the pprof file.
+
+## GC Control
+
+```
+# turn of gc
+GOGC=off go run <code>
+# print gc events to stdout
+GODEBUG=gctrace=1 go run <code>
+```
+
+## Questions
+
+- What are the [docs](https://golang.org/pkg/runtime/pprof/#Profile) talking about here? How do I actually use this?
+
+  > Pprof's -inuse_space, -inuse_objects, -alloc_space, and -alloc_objects flags select which to display, defaulting to -inuse_space (live objects, scaled by size).
+
+  A: Those flags are deprecated. Easiest way to select this stuff is via the pprof web ui's sample drop down.
+
+- The [docs](https://golang.org/pkg/runtime/pprof/#Profile) say I should get some kind of data, even if there is no GC. I can reproduce that, but the data seems to not change?
+
+  > If there has been no garbage collection at all, the heap profile reports all known allocations. This exception helps mainly in programs running without garbage collection enabled, usually for debugging purposes.