Repository navigation
Re-enable linker's DCE (to make runc binary 2 MB smaller) - #5506
Conversation
| github.com/seccomp/libseccomp-golang v0.12.0 | ||
| github.com/sirupsen/logrus v1.10.2 | ||
| github.com/urfave/cli/v3 v3.11.0 | ||
| github.com/urfave/cli/v3 v3.14.0 |
There was a problem hiding this comment.
We can wait a week for dependabot to create a PR to update urfave/cli (because of cooldown setting), and then I will rebase this PR.
There was a problem hiding this comment.
Marked as draft; opened a bump PR: #5508
|
@AkihiroSuda @thaJeztah @rata this one is no longer a draft. We can merge #5486 first and I'll do another rebase, or just merge this one (and close #5486) -- I'm fine either way. |
kolyshkin
left a comment
There was a problem hiding this comment.
We can also add a validate job that checks the binary does not grow too big (podman has something like this I believe), but this can be done as a followup.
| .PHONY: verify-dce | ||
| verify-dce: | ||
| set -o pipefail; $(GO_BUILD) -o /dev/null -ldflags=-dumpdep . 2>&1 \ | ||
| | { ! grep '<ReflectMethod>'; } |
There was a problem hiding this comment.
This feels a bit brittle; we'd almost need to have a reverse check to verify the check itself (e.g. if the template build-arg is removed)
There was a problem hiding this comment.
Well, we can try building runc without the urfave_cli_no_template tag and check that <ReflectMethod> is indeed there:
.PHONY: verify-dce
verify-dce:
# Sanity check: without urfave_cli_no_template, DCE must be disabled,
# otherwise the check below is not working.
$(GO) build -tags "$(BUILDTAGS)" -o /dev/null -ldflags=-dumpdep . 2>&1 \
| grep -q '<ReflectMethod>'
set -o pipefail; $(GO_BUILD) -o /dev/null -ldflags=-dumpdep . 2>&1 \
| { ! grep '<ReflectMethod>'; }but if we ever switch to urfave/cli/v4 and they will stop using text/template (they plan to), or if text/template itself is somehow fixed in a future Go version, the check will fail again.
Or, we can quickly build a binary which calls reflect.Value.MethodByName with a non-constant string and check that <ReflectMethod> is there:
tests/dce-canary/main.go:
// dce-canary is a positive control for "make verify-dce". It calls
// reflect.Value.MethodByName with a non-constant argument, which makes
// the linker disable dead code elimination of exported methods, so
// "verify-dce" must detect it. If it does not, the check itself is broken.
package main
import (
"os"
"reflect"
)
type t struct{}
func (t) Foo() {}
func main() {
reflect.ValueOf(t{}).MethodByName(os.Args[0])
}- Makefile:
verify-dce:
+ # Make sure the check works, using a program which disables DCE.
+ $(GO) build -o /dev/null -ldflags=-dumpdep ./tests/dce-canary 2>&1 \
+ | grep -q '<ReflectMethod>'
set -o pipefail; $(GO_BUILD) -o /dev/null -ldflags=-dumpdep . 2>&1 \
| { ! grep '<ReflectMethod>'; }@thaJeztah do you want something like this, or perhaps we can just drop this entirely and add a binary size diff check?
There was a problem hiding this comment.
We could also add a forbidigo linter rule (to not use text/template or call reflect.Value.Method, but
- this check is only for runc itself, i.e. packages we use are not checked (but it would find something like what was fixed in libct: don't use reflect to log CRIU options #5486);
reflect.Value.Method[ByName]is not breaking DCE but forbidigo can't distinguish const vs non-const.
There was a problem hiding this comment.
So it's not a linter job at all...
Maybe use whydeadcode? Nah, it doesn't have exit code, it's just a tool to check why DCE is disabled.
There was a problem hiding this comment.
OK I think the best way is to use a canary to check the check (and a hint to use whydeadcode). Last commit amended; WDYT @thaJeztah ?
There was a problem hiding this comment.
It probably shouldn't be a blocker for this PR (which is great to have), but at least we should give it some thought, because we don't want to silently regress.
What option do you prefer?
There was a problem hiding this comment.
Oh! You updated; let me check the latest version (just stepped away from my keyboard 😅)
| - name: verify linker dead code elimination is not disabled | ||
| run: make verify-dce |
There was a problem hiding this comment.
we could also add a positive check by doing something like ! make RUNC_BUILDTAGS=-urfave_cli_no_template verify-dce here but again this will break once we switch to urfave/cli/v4
thaJeztah
left a comment
There was a problem hiding this comment.
LGTM
left one suggestion / comment
| endif | ||
| # Tags which are always set. Using urfave_cli_no_template enables linker | ||
| # dead code elimination, resulting in a smaller binary. | ||
| BUILDTAGS_ALWAYS := urfave_cli_no_template |
There was a problem hiding this comment.
I was slightly wondering if we needed the make var at all, as there's no reason to override it, but making it a var somewhat implies it can be, and it currently doesn't make things less repetitive (we still need to include $(BUILDTAGS_ALWAYS) in both places.
If we do keep the var, we could consider naming it BUILDTAGS_COMMON, aligning with LDFLAGS_COMMON (shared between static and non-static)
There was a problem hiding this comment.
Makes sense; build tag inlined (hope we won't have any more places 😓 to add it).
Using text/template disables linker dead code elimination of exported methods for the whole program (see golang/go#72895). urfave/cli v3.14.0 added urfave_cli_no_template build tag, which makes it render help without text/template, producing the same output. Since runc does not use custom help templates, we can always use it. Together with the previous commits, this makes runc binary about 2 MB (15%) smaller (13.3 MB -> 11.3 MB on linux/amd64). Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Calling reflect.Value.Method or reflect.Value.MethodByName with a non-constant argument (directly, or indirectly via e.g. text/template) disables the linker's dead code elimination of exported methods, noticeably increasing the binary size. Add verify-dce target to make sure this does not happen again, and run it from CI. To make sure the check itself works, also run it against a small program which disables DCE, and expect it to fail. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>

Go linker dead code elimination (DCE) of exported methods is disabled for the whole program if
reflect.Value.Method[ByName]with a non-constant argument is used anywhere in the binary. One such use is intext/template(see golang/go#72895).This PR (together with a few prior ones) eliminates all such uses:
urfave_cli_no_templatebuild tag (runc does not use custom help templates).make verify-dce(checks there are no<ReflectMethod>symbols in-ldflags=-dumpdepoutput), and run it in CI, so this does not regress.As a result, runc binary is about 15% smaller on all architectures.
NOTE runc binary size is important because when running as rootless, runc has to copy itself.
Supersedes: #5486.