Nathen Harvey

a blog

DevOp With Me

Come DevOp with me! We’ll explore what DevOps is and what it is not.

Learning Chef - Part 6

Learning Chef Series

  • Part 1 - Introduce the project, configure workstation, and register a node with hosted Chef
  • Part 2 - Download cookbooks from the community site, add MongoDB, Apache, and Passenger to our node
  • Part 3 - Start writing a cookbook to deploy our application
  • Part 4 - Finish the application deploy, introduce roles
  • Part 5 - Multi-node Vagrant
  • Part 6

Engine Yard DevOps Innovator

Last week Engine Yard announced the winners of their inaugural Engine Yard Innovators Awards.

I was both honored and surprised when I learned that I had been chosen as the Engine Yard Innovator in the DevOps category.

Our DevOps category winner is Nathen Harvey. A devops guru who has been travelling from conference to conference evangelizing the use of Chef, Nathen is known for his “Rails With Chef” proficiency. He works to help others understand the importance of backend compatibility.

I was asked to write a post about some my work. This seemed like a good time to reflect on some of the work I’ve been doing for the past few years.

In late 2009, I joined CustomInk to head up the Web Operations team. While there, I was able to help drive the adoption of many DevOps practices including automation, continuous delivery, collaboration across teams, expanded responsibility, and participation in open source communities.

Learning Chef - Part 5

Learning Chef Series

  • Part 1 - Introduce the project, configure workstation, and register a node with hosted Chef
  • Part 2 - Download cookbooks from the community site, add MongoDB, Apache, and Passenger to our node
  • Part 3 - Start writing a cookbook to deploy our application
  • Part 4 - Finish the application deploy, introduce roles
  • Part 5 - Multi-node Vagrant
  • Part 6

Learning Chef - Part 4

Learning Chef Series

  • Part 1 - Introduce the project, configure workstation, and register a node with hosted Chef
  • Part 2 - Download cookbooks from the community site, add MongoDB, Apache, and Passenger to our node
  • Part 3 - Start writing a cookbook to deploy our application
  • Part 4 - Finish the application deploy, introduce roles
  • Part 5 - Multi-node Vagrant
  • Part 6

Part 4 of our Learning Chef tutorial was run as a Google+ Hangout that was streamed to YouTube.

In Part 4, we completed the application deploy and then looked at roles.

I’ll update this post soon with a breakdown of each step we took during this session. In the meantime, you can watch the entire video below.

Also, you can grab the code from the following repositories on github:

What’s Next

In Learning Chef - Part 5 we will move the MongoDB to it’s own VM.

In the meantime, please let us know what you think of this post and these videos! Drop a note in the comments or reach out to @nathenharvey or @mulpat on twitter.

Learning Chef - Part 3

Learning Chef Series

  • Part 1 - Introduce the project, configure workstation, and register a node with hosted Chef
  • Part 2 - Download cookbooks from the community site, add MongoDB, Apache, and Passenger to our node
  • Part 3 - Start writing a cookbook to deploy our application
  • Part 4 - Finish the application deploy, introduce roles
  • Part 5 - Multi-node Vagrant
  • Part 6

Part 3 of our Learning Chef tutorial was run as a Google+ Hangout that was streamed to YouTube.

In Part 3, we added a bunch of cookbooks from the community site including git, application, and application_ruby. After adding these cookbooks, we created a cookbook of our own to deploy a sample Rails application.

The application wasn’t fully deployed by the end of the tutorial but we’ll pick-up from there next time.

I’ll update this post soon with a breakdown of each step we took during this session. In the meantime, you can watch the entire video below.

Also, you can grab the code from the following repositories on github:

What’s Next

In Learning Chef - Part 4 we finish the deployment of the sample application and then explore roles.

In the meantime, please let us know what you think of this post and these videos! Drop a note in the comments or reach out to @nathenharvey or @mulpat on twitter.

Learning Chef - Part 2

Learning Chef Series

  • Part 1 - Introduce the project, configure workstation, and register a node with hosted Chef
  • Part 2 - Download cookbooks from the community site, add MongoDB, Apache, and Passenger to our node
  • Part 3 - Start writing a cookbook to deploy our application
  • Part 4 - Finish the application deploy, introduce roles
  • Part 5 - Multi-node Vagrant
  • Part 6

Part 2 of our Learning Chef tutorial was run as a Google+ Hangout that was streamed to YouTube.

Review of Part 1

Discuss Chef Solo vs. Chef Server

Chef Solo allows you to run Chef Cookbooks without a Chef Server. There are a number of things that you don’t get when using Chef Solo. Check the Chef Solo page on the wiki for more information.

Learning Chef - Part 1

Learning Chef Series

  • Part 1 - Introduce the project, configure workstation, and register a node with hosted Chef
  • Part 2 - Download cookbooks from the community site, add MongoDB, Apache, and Passenger to our node
  • Part 3 - Start writing a cookbook to deploy our application
  • Part 4 - Finish the application deploy, introduce roles
  • Part 5 - Multi-node Vagrant
  • Part 6

In November of 2012, Patrick Mulder posted a request on the Chef mailing list. He was

looking for some 1-1 teaching via skype to help me get going in setting up a basic DB server from scratch, as well as a basic dev server as intermediary step.

I thought this would be an excellent opportunity to feed my recent addiction to Google+ Hangouts. I would provide Patrick some one-on-one tutoring if he would agree to having the sessions broadcast live on YouTube. We had some technical issues getting our first session going in a Google+ Hangout but we were able to meet via Skype and I captured video of the session.

Our goal is to help you get up and running on Chef by following our progress. The intent is to have additional sessions run via Google+ Hangouts that are steamed live to YouTube. This post includes our first session which has been broken into nine short videos. I hope you enjoy these videos and are able to learn something about Chef, too. Both Patrick and I are looking forward to your feedback on this experiment.

You can watch all of the videos in the YouTube playlist or keep reading and watch each video in turn.

Introduction

In this video we introduce ourselves and the project.

MVT: Knife Test and Travis CI

In my last post, MVT: Foodcritic and Travis CI I described the process for having Travis CI look after your cookbooks and run Foodcritic, the cookbook lint tool, on your cookbook after each git push. In this post, we’ll iterate on the “Minimum Viable Test” idea by adding in support for knife’s cookbook testing.

Wait, I’m already running foodcritic, do I really need to run knife cookbook test, too?

I’ll use a very simple example to demonstrate that you do.

Let’s create a very basic cookbook:

1
2
3
4
knife cookbook create very_basic
** Creating cookbook very_basic
** Creating README for cookbook: very_basic
** Creating metadata for cookbook: very_basic

Next, we’ll write a flawed recipe:

cookbooks/very_basic/recipes/default.rb
1
2
3
4
package "flawed" do
  action :nothing
end
end

Now, run foodcritic on this cookbook:

1
foodcritic cookbooks/very_basic

Foodcritic doesn’t throw any errors or find any problem with the cookbook.

Let’s try testing it with knife:

1
2
3
4
5
6
knife cookbook test very_basic
checking very_basic
Running syntax check on very_basic
Validating ruby files
FATAL: Cookbook file recipes/default.rb has a ruby syntax error:
FATAL: /Users/nharvey/projects/chef-hosted/.chef/../cookbooks/very_basic/recipes/default.rb:22: syntax error, unexpected keyword_end, expecting $end

OK, it should now be obvious that knife cookbook test should be included as part of our MVT.

MVT: Foodcritic and Travis CI

One of the big themes that emerged during #ChefConf was that we should be testing our infrastructure code. Software engineers have been practicing test-driven development, behavior-driven development, continuous integration, and many other testing-related practices for a long time. It’s becoming more important for the infrastructure engineers to learn from and apply these practices to our day-to-day workflow. When it comes to testing Chef-driven infrastructure automation, there are a number of tools and practices that are starting to emerge. In this article I’ll look at a “minimum viable testing” (MVT) approach to this problem using Foodcritic and Travis CI. Follow the steps in this article to get your public cookbooks tested after every git push.

Testing with Chef

The idea of building automated tests for your infrastructure code has been getting a lot of traction lately. When it comes to Chef, many tools are starting to emerge.

The first tool in this area to get any significant traction, that I know of, was cucumber-chef. I first learned of this tool when I saw a pre-release copy of Test-Driven Infrastructure with Chef at the O’Reilly booth at Velocity Conf 2011. Stephen Nelson-Smith, the book’s author and framework’s lead developer, proposes an outside-in approach to testing where your tests can also act as monitors that look after the health of your infrastructure. I like the idea of this approach and feel it makes a lot of sense in a greenfield environment. One benefit of this approach is that it blurs the line between testing and monitoring. You can easily hook-up your monitoring system to your cucumber tests.

ChefSpec is another tool for testing your Chef code. It is a gem that makes it easy to write RSpec examples for Chef cookbooks. This style of testing allows you to execute your tests without needing to converge the node that your tests are running on. In other words, you can execute your tests without needing to provision a server. One huge appeal to this style of testing is that the feedback loop is very small. You’ll get feedback about your cookbook changes within seconds or a very few minutes of saving your changes.

Minitest Chef Handler is yet another tool for testing with Chef. This runs a suite of minitest tests as a report handler in your Chef-managed nodes. As you may know, report handlers are run at the end of each chef run, or convergence.

Testing at ChefConf

At the inaugural #ChefConf there were many sessions that included information about many companies’ approach to testing. Here’s a quick list of some of the sessions: