add an "export" command for libraries #369
Comments
|
It's unclear to me if this means we should keep GOPRIVATE or GOGARBLE support around. On one hand, it makes sense to keep GOGARBLE as a configuration option, because On the other hand, would anyone ever set GOGARBLE directly? When building manually, one would (presumably) always want to obfuscate all packages. When exporting a library like the example above, one would just want to obfuscate the library's packages. One possible use case for GOGARBLE would be to emulate the Right now, I'm still leaning towards phasing out GOGARBLE/GOPRIVATE support. |
|
Also, I think this command should be akin to |
In the added test case, "garble -literals build" would fail:
--- FAIL: TestScripts/literals (8.29s)
testscript.go:397:
> env GOPRIVATE=test/main
> garble -literals build
[stderr]
# test/main
Usz1FmFm.go:1: cannot call non-function string (type int), declared at Usz1FmFm.go:1
Usz1FmFm.go:1: string is not a type
Usz1FmFm.go:1: cannot call non-function append (type int), declared at Usz1FmFm.go:1
That is, for input code such as:
var append int
println("foo")
_ = append
We'd end up with obfuscated code like:
var append int
println(func() string {
// obfuscation...
x = append(x, ...)
// obfuscation...
return string(x)
})
_ = append
Which would then break, as the code is shadowing the "append" builtin.
To work around this, always obfuscate variable names, so we end up with:
var mwu1xuNz int
println(func() string {
// obfuscation...
x = append(x, ...)
// obfuscation...
return string(x)
})
_ = mwu1xuNz
This change shouldn't make the quality of our obfuscation stronger,
as local variable names do not currently end up in Go binaries.
However, this does make garble more consistent in treating identifiers,
and it completely avoids any issues related to shadowing builtins.
Moreover, this also paves the way for publishing obfuscated source code,
such as burrowers#369.
Fixes burrowers#417.
In the added test case, "garble -literals build" would fail:
--- FAIL: TestScripts/literals (8.29s)
testscript.go:397:
> env GOPRIVATE=test/main
> garble -literals build
[stderr]
# test/main
Usz1FmFm.go:1: cannot call non-function string (type int), declared at Usz1FmFm.go:1
Usz1FmFm.go:1: string is not a type
Usz1FmFm.go:1: cannot call non-function append (type int), declared at Usz1FmFm.go:1
That is, for input code such as:
var append int
println("foo")
_ = append
We'd end up with obfuscated code like:
var append int
println(func() string {
// obfuscation...
x = append(x, ...)
// obfuscation...
return string(x)
})
_ = append
Which would then break, as the code is shadowing the "append" builtin.
To work around this, always obfuscate variable names, so we end up with:
var mwu1xuNz int
println(func() string {
// obfuscation...
x = append(x, ...)
// obfuscation...
return string(x)
})
_ = mwu1xuNz
This change shouldn't make the quality of our obfuscation stronger,
as local variable names do not currently end up in Go binaries.
However, this does make garble more consistent in treating identifiers,
and it completely avoids any issues related to shadowing builtins.
Moreover, this also paves the way for publishing obfuscated source code,
such as burrowers#369.
Fixes burrowers#417.
In the added test case, "garble -literals build" would fail:
--- FAIL: TestScripts/literals (8.29s)
testscript.go:397:
> env GOPRIVATE=test/main
> garble -literals build
[stderr]
# test/main
Usz1FmFm.go:1: cannot call non-function string (type int), declared at Usz1FmFm.go:1
Usz1FmFm.go:1: string is not a type
Usz1FmFm.go:1: cannot call non-function append (type int), declared at Usz1FmFm.go:1
That is, for input code such as:
var append int
println("foo")
_ = append
We'd end up with obfuscated code like:
var append int
println(func() string {
// obfuscation...
x = append(x, ...)
// obfuscation...
return string(x)
})
_ = append
Which would then break, as the code is shadowing the "append" builtin.
To work around this, always obfuscate variable names, so we end up with:
var mwu1xuNz int
println(func() string {
// obfuscation...
x = append(x, ...)
// obfuscation...
return string(x)
})
_ = mwu1xuNz
This change shouldn't make the quality of our obfuscation stronger,
as local variable names do not currently end up in Go binaries.
However, this does make garble more consistent in treating identifiers,
and it completely avoids any issues related to shadowing builtins.
Moreover, this also paves the way for publishing obfuscated source code,
such as #369.
Fixes #417.
|
I think unidocs obfuscation is cute, I might try to reverse it tomorrow. |
|
I just realised what is the biggest hurdle with But when exporting, we want to obfuscate all files - no matter the GOOS, GOARCH, or any other build tags. If a package has I can think of two options:
A third option would be to obfuscate in some way that doesn't rely on build tags. But this would require giving up |
|
2 - what if we consider each tag as a unique build and put all files obfuscated with this tag under this tag as well? This would highly increase the total size (and obfuscation time), but make the resulting package universal and with unique obfuscation for each tag (tags combination?). |
|
This could multiply the size of Go modules though, so I'd rather avoid it unless there are significant benefits. I don't see any significant benefits, given that I expect that most users would want to publish all the "build tag variants" alongside each other. Spotting the common bits between them wouldn't be that hard. |
|
I want to publish a private Go library without exposing the source code. I really need this feature and I hope for support. |
|
Any news? I really need this feature, too! |
|
This issue is only aspirational no one is actually working on it or planning to. @wubin1989 @jitcor |
|
So garble can realize the effect similar to unipdf? Or java proguard the kind of obfuscation effect, through the obfuscation configuration file to add obfuscation rules. |
|
Yes garble export would generate code in a similar manner how unipdf is obfuscated. |
@lu4p do you mean we just need to add "export" command line subcommand, the feature has already implemented, right? |
|
No this is not implemented |
|
I implemented something (minimal) like this here: https://github.com/yardenlaif/balagan |
|
@yardenlaif stared and forked. Many thanks! |
|
Effective obfuscation looks like a hard problem. I just ran the "obfuscated" code in https://github.com/unidoc/unipdf/blob/master/pdfutil/pdfutil.go through gofmt: It is not very effective at all. Then again, I can't see how anything can be that effective. |
Hi @pjebs, FWIW, just in case unclear, that example is not using garble. For garble, you could take a look at some of the examples with control flow hardening enabled, like: |
|
Hi @mvdan, maybe this is already considered or mentioned, but one quick comment is that collapsing down multiple packages into a single package seems it would be helpful to reduce visibility due to package boundaries and generally allow things to be mixed together more. (In other words, could be somewhat similar to the approach IIRC the go stdlib used to do for importing http2 from golang.org/x/net where it collapsed everything down into a single source file using https://pkg.go.dev/golang.org/x/tools/cmd/bundle, or something along those lines). |
|
@thepudds it's an interesting idea, although it seems relatively orthogonal to a I will say, though, that it's not a particularly easy task. One would have to be careful to maintain...
|
|
Part of the reason it comes to mind for garble export is I'd guess people interested in garble export for commercial reasons might be ok with a build performance hit for incremental builds, especially if it's optional (e.g., maybe not enabled in some cases, but maybe enabled in CI and for final packaging). Also, a commercial library might tend to be smaller in practice than the amount of source code involved with the whole program obfuscation people do today with garble, so maybe the performance hit might tend to be smaller for a garble export use case. I would guess cmd/bundle doesn't tackle anything with init ordering. I would have guessed it dealt with uniqueness of identifiers including for method sets, but maybe not (e.g., maybe not an issue for its real initial targets like http2 and maybe gzip), or maybe whatever it does is not a good fit for garble. I suppose one way to "implement" it for garble export might be to add a ~sentence to the readme or doc like "If desired and if it works for you, you can consider using golang.org/x/tools/cmd/bundle before running garble export". In any event, just a quick thought. 😅 |
|
Ah I see what you mean. It is potentially more interesting for |
In the context of #276 (comment), I've been thinking that there are scenarios where one might want to obfuscate only some packages: when publishing or packaging an obfuscated library, to be consumed as regular Go packages via source code.
We do sort of support this already, via something like:
Then, the obfuscated source code will be under
out/my.corp/some/library.I think this use case is valid, given that it's not possible (or at least not easy) to distribute Go libraries without distributing source code too. One such example is unidoc, where its libraries like unipdf are obfuscated then pushed to GitHub.
We already have much of the machinery here, and garble is pretty advanced as a Go obfuscator, so I think it makes sense to take it one step further and make this easier to do. For example, via:
Some key differences compared to how we currently obfuscate code:
Internal packages such as
my.corp/some/library/internal/foowould still be fully obfuscated, including changing their API names and import paths. That could be a way to tell what packages must remain importable and usable.Otherwise, I think obfuscation could stay the same: stripping unexported names, position information, most comments, etc.
We might have to tweak our obfuscator to collapse newlines, too - right now, it does that via
/*linecomments for the compiler, but newlines in the printed source code remain, since those don't affect binaries. They affect published source code, though - and it's also likely that those compiler directives shouldn't be present in this new mode.The text was updated successfully, but these errors were encountered: