Godep: Dependency Management in Go

Go differs from many other languages in that a wide range of dependency management methods and tools exist for it. The approach endorsed by the Go team involves vendoring dependencies within the project folder and modifying the the import statements to support the new location.

Godep works differently than the endorsed approach… Instead of requiring changes to be made to the source code to support the vendored import path, the GOPATH is modified to use the vendored folder. This is beneficial since the same code will work both within the Godep environment and in a standalone environment using go get to set up the GOPATH. Other dependency management methods and tools exist, so be sure to read up on them and pick one suitable for your project or organization.

For basic usage instructions, check out the Godep documentation. Based on my experience with Godep, I’ll give you a quick overview on the pros and cons of using it, as well as a few guidelines.

A godep save command will copy all imported packages in their entirety from your current GOPATH into a vendored workspace folder in ./Godeps/_workspace. A list of those packages will be stored with relevant version information in a controller file, Godeps/Godeps.json. This is done not just for the packages your project directly imports but also for any imported by your dependencies -- essentially anything required to build your project. You must add this entire folder SCM and it should never be manually altered in any way.

Using Godep is as simple as prepending your normal Go commands like go test or go build with the godep command. This uses a temporarily extended GOPATH which prioritizes the Godep vendor directory.

$ godep save ./…       # saves your GOPATH to the Godep folder
$ godep go build ./…   # builds using the Godep vendored dependencies
$ godep go test ./…    # tests using the Godep vendored dependencies

From here, should you apply a change to your GOPATH, your project will be isolated.

$ go get -u github.com/golang/protobuf/… # update a dependency
$ go build ./…                          # build using standard GOPATH
$ godep go build ./…                     # build using Godep vendored version

There are two methods to apply changes to your Godep folder. Understanding the correct cases for using each will save you a huge amount of time and frustration. godep save and godep update work very differently and alter your Godep in different ways. The key thing to remember here is that if you’re ever in doubt, use godep save.

godep update takes a specific dependency package and updates the vendored instance of that package with the version in your GOPATH. This will update files with changes, add new files, remove old ones, and update the version SHA listed in the Godeps.json file.

$ go get -u github.com/golang/protobuf/…
$ godep update github.com/golang/protobuf/…

This will not add or remove sub-packages from dependency management, nor will it update any other dependencies recursively. Only previously imported packages are listed in the Godeps.json file and only those listed are updated.

Updating the entire package will update any references to sub-packages; however no new packages will be added, nor old ones removed. Similarly, if your dependency update is dependent upon another change elsewhere in your dependency stack, you may run into issues. godep update only touches the packages listed in Godeps.json, which match the provided package pattern.

In contrast, godep save applies the entire relevant GOPATH to the Godeps folder and will add/remove packages as needed. Because it’s based off of your GOPATH, godep save can also check for build errors and non-clean repositories before applying changes, enforcing dependency cohesion.

Given the dangers of using godep update (missing packages and dependencies), it’s much safer to use godep save. The only situation where it’s safe to use godep update are when both of these conditions are satisfied:

• No dependencies of your target dependency need to be updated.

• No imports were added to or removed from your target dependency.

If the dependency is external to your organization, it can be difficult to determine what changes are taking place, so it is safer to never use **godep update` on anything third-party.

Beyond knowing the basic commands to use with Godep, there are a number of gotchas.

In situations where a dependency is versioned from multiple sources, it can easily be represented at different versions across the same Godep environment.

Consider a case where internal codebase ProjectA lists ProjectB and ThirdPartyA as dependencies. Both ProjectB and ThirdPartyA list ThirdPartyB as a dependency. If ThirdPartyA is ever updated to require a new version of ThirdPartyB while ProjectB continues to reference the older version of ThirdPartyB, which version does ProjectA list in Godeps?

ProjectA
 ├-- ProjectB
 | └-- ThirdPartyB
 ├-- ThirdPartyA
 | └-- ThirdPartyB

The answer is simple.

Godep has no internal conflict resolution mechanism and does not resolve Godep requirements of dependencies. The decision about which version to use is left up to the developer; in this case, ProjectA would use whichever version of ThirdPartyB was last set in Godep. Since the last write was from the update of ThirdPartyA, ProjectA would reference the newer version. Any code from ProjectB would reference the newer version when loaded as a dependency.

This is a subtle point, but it highlights the need for caution when updating dependencies, as well as the need for all dependency version changes to be synchronized across projects. Compile errors would be caught when ProjectA was built, but subtle behavioural changes might not be caught till later.

Therefore, a dependency update should be applied to all projects simultaneously. Consider updating dependencies via a looped script, applying the change to all of your codebases at once.

for project in projects
  cd workspace/project
  git checkout master && git pull
  godep restore
  go get -u ThirdPartyB/…
  godep save ./…
  git checkout -b “updating-godeps”
  git commit ./Godeps -m “Updating Godeps”
  git push origin updating-godeps
end

In cases where several internal projects require a synchronized version of a dependency, it’s much simpler to use a forked version of a dependency and push dependency control up to the fork that your organization maintains. This fork becomes the central control for versioning that dependency. However, it’s more likely that an update to that dependency will cause critical breaking changes across multiple projects.

For a list of various dependency management methods and tools for Go, check out the Golang docs on the subject.

Despite Godep providing a versioned GOPATH instance, it still uses your original GOPATH. The vendored workspace is checked for a package before your main GOPATH. Generally this doesn’t cause issues, but it can be troublesome in some cases.

Since your main GOPATH is referenced, missing dependencies may not be caught until later in the development cycle. Any package missing from Godep is most likely present in your GOPATH; this seamless fallback makes it difficult to determine which dependency version a command ends up using. Due to this composite GOPATH, build errors can appear when dependency versions are incompatible. The best way to avoid this issue, and catch missing dependencies early, is to build in a throwaway GOPATH or in a container.

Not every project may require the broad dependency control as provided by Godep.

Unlike import path-based vendoring, Godep vendors the entire set of dependencies regardless of a specific desire to version them. This does mean that no dependencies will ever be updated unless explicitly altered, first through GOPATH and then through Godep.

Should your organization have a large number of common dependencies across different projects, you may want to look into using a forked dependency model. Godep provides a locally controlled, customizable dependency management system. When used with care, this system can support highly versioned and reproducible builds, especially in change resistive environments with few shared dependencies.