Webinar Q&A: Continuous Delivery and Pipeline Traceability with Jenkins and Chef

Thank you to everyone who joined us on our webinar, the recording is now available. Because of the technical difficulties in recording this webinar, here is another recording of the missing information.

Below are the questions we received during the webinar Q&A:

Q: Has Chef-client output come out on console or have specific things been captured in this traceability case?
A: I just wanted to show the generated report JSON at the very end of the Chef run. By default, the output is usually redirected to /var/log/chef/client.log, but if you run Chef-client manually you’ll see the output on the stdout of course.

Q: On one slide, it was mentioned to use Environments to version recipes. This is a best practice question then, does that slide then suggest that environments should be of type "appName-Version" and applied to nodes? It points to the cookbooks that are versioned. So that would be different than using Environments of like "test", "dev", etc.
A: For clarity, the recommendation is to use Environments to pin specific cookbook versions to a particular subset of nodes. You can still use environment names like “test”, “dev”, etc. For example, I have “mycookbook” version 1.2.3. I want to roll out “mycookbook” version 1.3.0. Set your CI job to first update “dev” with ‘cookbook “mycookbook”, “= 1.3.0”’, while “test” has ‘cookbook “mycookbook”, “= 1.2.3”’. When you are ready to ready to promote this change from dev to test, the CI job that promotes this to test should set the definition for the “test” environment to ‘cookbook “mycookbook”, “= 1.3.0”’.

Q: What licensing restrictions are there on using open-source Chef in the enterprise? Is it just the enterprise features?
A: You can use OSS Chef for free, no restrictions. In fact, you can also demo the Enterprise paid features from OSS. You can use all the Enterprise features for under 25 nodes for free. For more than 25 nodes, you have to license the Enterprise features.

Q: Do we have a cookbook for the Weblogic server ?
A: You can see all the cookbooks shared in Supermarket at http://supermarket.chef.io but you can also find many others not on Supermarket via GitHub. There is a weblogic cookbook currently on the Chef Supermarket site.

Q: Would a simple app code only change result in a change to a cookbook (tweak version of app to deploy?) and also trigger app unit / acceptance tests during the Test Kitchen execution?
A: You don't *have* to structure change that way, but I'd recommend it. I find it's easier for auditing to discover what happened, where and when. You could skip those tests, but I wouldn't recommend it unless you're in some emergency breakfix scenario.

Q: Can you recommend a Jenkins plugin for pipeline?
A: For the build pipeline plugin, I'd recommend Build Flow. You could start with something simpler like the Build Pipelines plugin, but that doesn't allow for concurrency. You might not need concurrency. But you might. :-) So I'd recommend starting there.

Q: How do you decide what version to automatically increment in the Jenkins Build Job?
A: Your cookbooks are tagged with versions in them: X.Y.Z (major.minor.patch). If you submit any new change without rolling X or Y, the pipeline just increments Z based on the last known good version. If you roll X or Y, Z resets to zero and the CI job manages versions from there. This makes Z in whatever source you submit effectively useless. The pipeline owns Z, you own X and Y.

Q: Do you have example cookbooks showcasing this setup available for review somewhere, like a GitHub repo?
A: Sure, you can grab the the scripts, cookbooks and configs used in this webinar from my GitHub repository: https://github.com/woohgit/jenkins-chef-traceability-example

Q: Can you provide some tools/utilities for testing the Chef code? If I am not using Vagrant can I use Test-Kitchen. It would be great if you could some utilities which can be used to test Chef code without using Vagrant.
A: Is there a particular aversion to using Vagrant? You could run tests by hand, I suppose. Recommended practice is to use Vagrant via test-kitchen; it makes your life much easier. You could also try something like minitest and enable the chef-minitest report handler. If there’s a reason you can’t use vagrant, let’s talk about that and figure out a workable approach.

Q: Does Vagrant simply spin up a VM with Chef installed in it?
A: Test-kitchen effectively manages your vagrant files and sets them up to automatically bootstrap chef and grab your cookbook code out of your PWD. You can also use the kitchen config file to pass configs on to vagrant directly if you want addl functions. Check out the bento project for more details on what happens to your vagrant box -- https://github.com/opscode/bento

Q: Can you put together a list of all technologies in this demo (Chef, Jenkins, Ruby, Vagrant, etc.)?
A: I think the best is to check out the source code from my GitHub repository. It has a README which contains the answers for your question. My GitHub repository can be found here: https://github.com/woohgit/jenkins-chef-traceability-example

Q: How does the Jenkins plugin obtain information about the status of the Chef deployment?
A: Chef generates a report at the end of Chef-client run and the Chef-handler-Jenkins gem - which is included in the cookbooks - selects only the file related changes and sends a POST request to a specific Jenkins URL. The Chef Tracking Plugin handles that POST data. To be more precise: the Chef Tracking Plugin exposes an API endpoint: http:///chef/report which is used by Chef to send the reports.

Q: Just saw that Chef mentioned it had deployed a war as a result of the Jenkins job. Is it possible instead to deploy other types of artifacts? E.g. install an .exe in Windows, or apply RPM update packages in SUSE/RHEL?
A: Sure, you can deploy any type of files. There is absolutely no restriction.

Q: Is there a way to trigger deployments from within Jenkins?
A: Sure, you can. You can configure Jenkins to start a build when a commit has been pushed to the repository, or a job A can trigger job B if the build was successful. There are a lot of plugins as well which makes this easier. I’d suggest this one:
https://wiki.jenkins-ci.org/display/JENKINS/Parameterized+Trigger+Plugin

Q: IS there any weblogic cookbook?
A: Answered above.

Q: Does Chef use something like a snapshot to revert any manual changes made to nodes or does it only apply settings outlined in a recipe?
A: Snapshots are not used or recommended. Everything in Chef is explicit: Chef only manages settings outlined in your recipe. Chef is not a magic pony and it does not somehow automagically understand everything that has changed on your system.

Q: We speak about serverspec (which tests the server) but is chefspec also needed? To test the cookbook-recipe code?.. Or is Serverspec good enough of a test to move things into production?
A: There are a number of test frameworks to use and I’m a big fan of chefspec. The proposed workflow was an example of the types of functions we may want. I would recommend both testing your code with chefspec and testing your resulting infrastructure with serverspec at different points in your development lifecycle.

Q: What is the best method of using Chef controller/controller or controller/agent setup?
A: High Availability in Chef is accomplished as active/passive.

Q: How can I perform hotfix configs using Chef?
A: I think the question is how to deploy a hotfix in this type of pipeline? In the proposed workflow, the most obvious way to deploy a hotfix is to merge it straight into controller (bypassing the validation/code review cycles). Merge the code and let your pipeline promote that programatically.

Q: Can we spawn VMs in the cloud instead of using Vagrant?
A: Since version 1.1, Vagrant is no longer tied to VirtualBox and also works with other virtualization software such as AWS EC2. Check for specific vagrant plugins here: http://vagrant-lists.github.io/ , especially the vagrant-aws plugin. Vagrant is only a wrapper around VirtualBox, KVM, AWS EC2, DigitalOcean and so on.

Q: How do you see the future of Chef given the rise of Docker? Do you see that the adoption of Chef will increase or decrease as the adoption of Docker increases?
A: Containers also require configuration. It will be interesting to see where Docker goes in the future. While creating a container is simple with Docker, running complex infrastructure topologies in production is difficult. Complexity never entirely goes away, as engineers we may just move it to different parts of the stack. Configuration management still has a strong role even in a containerized world. I encourage you to look at the work we’ve done with Chef Container for more ways to use Chef with container -- https://docs.chef.io/containers.html .