Mirroring Go Vulnerability Database for an air-gapped CI environment

At Adyen, security is paramount; we are always on the lookout for tools that enhance the robustness of our infrastructure. 

In the CI/CD(Continuous Integration and Continuous Delivery) team, we were particularly interested in the introduction of govulncheck by the Go security team. The tool offers Go developers the ability to check their project dependencies for known vulnerabilities seamlessly. We attended the Gophercon EU 2023, in Berlin and we had the opportunity to interact with Julie Qiu, Staff engineer at Google and team lead of the Go Security team. She gave an insightful talk on Vulnerability Management for Go, and our discussion with her further fueled our determination to integrate this tool into our CI processes.

However, a significant hurdle lay ahead: Our CI environment is air-gapped… 

To leverage govulncheck, we would require constant access to the Go vulnerability database; a resource usually fetched from https://vuln.go.dev. The challenge posed here is that our CI environment is air-gapped: this means that our Gitlab CI runners do not have direct access to the internet.

To enable the seamless integration of govulncheck, we came up with a solution that consisted of a multi-stage approach:

1. Cloning the Vulnerability Database: We began by setting up a pipeline stage to clone the official vulnerability database.

2. Building the Local Database: Once the official database was cloned, the next stage involved generating our local vulnerability database:

3. Publishing to Internal S3: After building, we took the generated-db and published it to our internal S3 bucket.

Once mirrored and stored internally, developers could utilize the local vulnerability database by pointing govulncheck to our S3 endpoint. To do the same in your project, simply invoke:

Integrating tools in an air-gapped CI environment can often feel like trying to fit a square peg in a round hole. But, with a bit of scripting and automation, we managed to make govulncheck work for us. It's a testament to the versatility of Go and the surrounding tooling. If you're in a similar boat with an isolated environment, we hope our approach gives you a good starting point. Here's to more secure code and fewer vulnerability surprises!