From Source Code to Executable

Originally published in Japanese at https://zenn.dev/ymotongpoo/books/go-ebpf-primer/viewer/15-source_to_binary.

Chapter 2 said that instructions sit in memory and have addresses. The executable file is where that sequence of instructions comes from. This chapter follows your source code through compilation and linking, and then opens the finished file.

Three things carry into the hurdle chapters. go tool objdump turns machine code back into a form you can read. go tool nm shows you the table that maps names to addresses. The third is knowing how an executable divides its own contents.

From source to machine code

When you build Go source code, you get a binary that holds a sequence of machine-code instructions. Machine code is the byte sequence that represents the smallest units of direction a CPU can act on, which Chapter 2 covered. The instructions sit in order from the start of the sequence, and each one has an address. Execution is the repeated cycle of fetching the instruction at the address that PC points to, executing it, and advancing PC.

Somewhere in this transformation, the function that you wrote in main.go becomes a sequence of instructions plus the address that holds them. The world of machine code does not, as a rule, carry over the names of your functions and variables.

Figure 1: Solid arrows show the direction of the transformation. The dashed arrow shows the symbol table looking up an address from a name, which is a reference rather than a transformation.

Disassembly

Turning a sequence of machine-code bytes back into instruction notation that a person can read is disassembly. Go ships go tool objdump for the job.

Assembly is about to appear, so let me say this first. You do not need to write assembly for anything in this book. When disassembly output appears, read the lines that carry an arrow or an annotation, and skip the rest.

A small function makes a good example. //go:noinline tells the compiler to leave the function as a call instead of expanding it, and Chapter 5 explains why that matters.

• Open in the Go Playground

package main

import "fmt"

//go:noinline
func double(n int) int {
	return n * 2
}

func main() {
	fmt.Println(double(21))
}

Build it and disassemble main.double. The output below comes from a binary built with GOOS=linux GOARCH=amd64. Your machine may run macOS or Windows, and these two environment variables still let you observe the same thing.

$ GOOS=linux GOARCH=amd64 go build -o demo main.go
$ go tool objdump -s 'main\.double$' demo
TEXT main.double(SB) .../main.go
  main.go:7   0x49e180   4801c0   ADDQ AX, AX   ← computes n*2
  main.go:7   0x49e183   c3       RET           ← returns from the function

The columns, from left to right, are the source line number, the address of the instruction, the machine-code bytes, and the readable instruction notation. The single Go line n * 2 became one instruction, ADDQ AX, AX, which adds AX to itself and thus doubles it. RET returns from the function.

Look at the column of bytes. 4801c0 is three bytes and c3 is one byte, so the instructions do not share a length. The byte sequence itself also carries no marker that says where one instruction ends and the next begins. A disassembler has no choice but to interpret one instruction at a time, from the start. This property returns in Hurdle 1, as real OBI code.

Figure 2: The arrow shows the direction in which interpretation proceeds. The same four bytes separate into a three-byte instruction and a one-byte instruction only once you interpret them in order from the start. Start reading from a byte in the middle and you get a different interpretation.

The symbol table

Names do not reach machine code, but the binary carries a separate table that maps names to addresses. That table is the symbol table. It holds the fact from the output above, that main.double starts at address 0x49e180, and go tool nm shows it to you.

$ go tool nm demo | grep 'main\.double'
  49e180 T main.double

This table is also why you can hand a function name to go tool objdump and disassemble only that function. When you want to place a hook on a function entry from outside, this table gives you the address from the name.

Static linking and dynamic linking

The compiler works on one component of the source at a time, which in Go means one package at a time. Linking is the step that joins the machine code from each component into one executable and fixes the final address of each function. The program that does the work is the linker.

Two methods join libraries to a program. Static linking pulls the machine code of every library you use into the executable whole. Dynamic linking leaves each library in a separate file, called a shared library, and binds it at run time. The typical C program uses the second method. The machine code of the standard C library (libc) sits outside the executable, and its addresses become fixed at run time.

The Go linker chooses static linking unless you use cgo. On Linux with a C toolchain installed, however, net and os/user pick their cgo-based name-resolution implementations by default, so a program that uses net/http links dynamically. CGO_ENABLED=0 switches them to the pure Go implementations and removes the dependency on shared libraries. Either way, the property that matters to the side that instruments the program does not change. Chapter 6 covers what that property is.

Figure 3: Arrows show the direction of a dependency. A C process resolves its dependencies on shared libraries at run time, and a Go binary has no such step.

ELF sections

An executable file on Linux uses a format called ELF. The file divides into sections: machine code goes into .text, constants into .rodata, and the symbol table from the previous section into .symtab.

A Go binary carries two more things that belong to Go. One is .gopclntab, a table for looking up a function name and a line number from an address. The other is a build-info blob that records the Go version and the list of dependency modules. The Go runtime reads .gopclntab itself, to build a stack trace on panic.

Figure 4: A table of who needs which section. Execution itself needs .text, .rodata, .data, .bss, and .gopclntab. The ones that exist only for whoever reads the file are .symtab, DWARF, and the build info.

How an executable gets loaded into memory

What Chapter 3 called the text segment is the .text section from the previous section. .text therefore exists in two places: inside the file on disk, and in the memory of the running process.

You might expect the kernel to copy the whole file into physical memory at startup, and it does not. All the kernel does at startup is write a note in the mapping table from Chapter 3. The note says which position in the executable to read when something references a given range of virtual addresses. The table works in fixed-size units called pages, normally 4KB on Linux. A page reaches physical memory at the moment the CPU tries to execute an instruction on it¹. A huge executable therefore starts running right away.

Sharing is another result of this scheme. In the Chapter 3 experiment, the two processes started from one executable (PIDs 25 and 31) had independent virtual address spaces. Their .text pages can still share one copy in physical memory. Independent virtual addresses and a shared physical page do not contradict each other. However many copies of a program you start, the .text pages cost one copy of physical memory².

Figure 5: Solid arrows show where the mapping tables point, and the dashed arrow shows a read from the file. The virtual addresses of the two processes are independent, and the .text entries in their mapping tables can still point at the same physical page.

Sharing works because .text pages are read-only. The mapping table records the physical page and a permission alongside it: reading and executing allowed, writing not. A write by one sharer would change execution for every sharer, so this permission is what holds the sharing together. If a program tries to overwrite its own machine code, the kernel stops it on the spot.

Some tools still want to write. A debugger, for one, edits the machine code of a running process to plant breakpoints. The kernel handles this with copy-on-write. It leaves the shared page alone and makes a private copy of the single page to be changed. It writes to the copy, then points the mapping table of that process at the copy. Neither the file on disk nor the original shared page changes.

Figure 6: Solid arrows show where the mapping tables point, and the dashed arrow shows the copy being made. Only the mapping table of the process that wrote now points at the copy, and the original page and the file on disk are unchanged.

Injecting an observation point into a running program from outside also happens on top of this mechanism. Chapter 7 covers the details.

DWARF

Chapter 2 ended with the point that struct field names disappear after compilation, and yet a debugger still shows r.count by name. It can do so because the compiler writes debug information called DWARF into the ELF sections whose names start with .debug_³.

Figure 7: Arrows show references. Running memory holds nothing but a byte sequence. DWARF records how many bytes from the start each field of each type sits at, so the debugger can show that field by name.

What is useful to a debugger is useful to an instrumentation tool that looks in from outside. Such a tool reads .gopclntab or the symbol table to learn where the functions are. It reads DWARF to learn where the fields are, and the build info to learn the version.

The program itself runs without DWARF and without the symbol table. Building with -ldflags="-s -w" drops .symtab and DWARF, and the file gets smaller. Chapter 14 checks what that does to instrumentation.

Key points for the hurdles

• A machine-code byte sequence carries no instruction boundaries, and the only way to interpret it is in order from the start.
• You can find the address of a function from its name through the symbol table or .gopclntab.
• The mapping from field names to offsets survives in DWARF and also in the type information for reflection. -s -w erases only the DWARF copy.
• Machine-code pages are read-only and shared between processes. A write swaps in a copy that is private to one process, which is copy-on-write.

Exercises

1. Build any Go program. Disassemble main.main with go tool objdump -s 'main\.main$' and count the RET instructions.
2. How many of the functions that you wrote, meaning the symbols that start with main., appear in the output of go tool nm? If a small function that you know you wrote is missing, what do you think happened? (Hint: Chapter 5 covers it.)