Posts Tagged ‘testing’

Writing a Cucumber Feature

January 2nd, 2010

This is the second post about Cucumber, where we’ll be exploring the creation of a feature. Cucumber centers on testing the expected behavior of your application, rather than the expected behavior of a method. I had someone explain to me once, “unit tests/specs are the methods/implementation you wish you had, Cucumber tests are the application you wish you had.”

User Stories

When thinking about a Cucumber feature you will want to relate it to a user story. If you’re familiar with Agile development stories are nothing new.  For those who are not though it’s not too complicated. A story is simply an expression of value generally in the form of:  ”As {a role} I want/need {to do some function} so that {business value}.” Generally the business value is best expressed in terms of: revenue generation, revenue protection, operational efficiency (cost savings).

For the example we’ll be using this story:

As a workshop co-ordinator I need to manage contact information so I can easily reference individuals and start communications with them.

You’ll notice that this story doesn’t imply implementation.  You generally never want to imply implementation in a story.  Also you’ll notice that the story is simple and comprehensible by pretty much anyone (business user, developer, average Joe Public).

Starting a Feature File

In our Rails application we’ll want to go inside our /features folder.  Create a new file called manage_contact.feature.  At the top of the file type in the following:

Feature:  Manage Contact
  In order to easily reference individuals and start communications with them
  As a workshop co-ordinator
  I want to manage contact information

Now we have our first feature.  The next step is to define some scenarios.  Scenarios express various actions the user might take in relation to the feature.  Below are some of the scenarios we came up with for this feature (these go below your feature entry – indented to be even to the “In order, As a, I want” section).  You don’t need to, but I like listing the scenarios in order of priority, this way a developer working on the feature can just start on top and work downwards:

Scenario: View list of contacts

Scenario: Add a new contact

Scenario: Email a contact from the list

Scenario: Remove an existing contact

Scenario: Update an existing contact

Scenario: Start an IM conversation with a contact from the list

Your finished feature file should look something like this gist. In the next post we’ll start to look at building out our scenario’s with step definitions, that is define what behavior we expect to happen with each scenario we have defined.

No comments »

Posted in Agile, Programming

Tags:

Adding Cucumber to Your Salad

January 2nd, 2010

Cucumber has come a long way since I’ve first started to use it.  My two posts about getting started with BDD have been the most visited pages of my blog.  So it seems only fitting to address the new features and update the material.  I’ll be creating a series of blog posts (so it doesn’t end up being one giant scrolling page) addressing a real-life situation of creating a contact management feature.  We’ll build it from the ground up using Cucumber and RSpec.

Installing Cucumber / RSpec

The first step to using Cucumber (and RSpec) is to install the necessary gems.  This is very straightforward and probably familiar to most people:

sudo gem install rspec rspec-rails cucumber cucumber-rails webrat ZenTest database_cleaner

If you are using Ruby > 1.9 you will also need to install the test-unit gem (or RSpec will not load properly).

sudo gem install test-unit -v=1.2.3

Now if you run into errors installing webrat, know that it requires nokogiri (an awesome XML parser, see my blog post about nokogiri).  In order to build nokogiri though you need to have a few libraries installed ahead of time.  If you run into any errors installing webrat (likely because it couldn’t install the dependency nokogiri gem) try the following (I’m using a Debian -Kbuntu- flavor of Linux, so your specific command might vary):

sudo aptitude install libxml2-dev libxslt1-dev

Once you have installed the two developer libraries for libxml2 and libxslt you can try to reinstall webrat (sudo gem install webrat) and it should properly build.

Adding RSpec and Cucumber to a Rails Project

Having the gems installed is the first step.  To use Cucumber in your Rails project you need to run a few generation scripts which prime your application and make a few config updates. Run the following commands from inside your Rails application root:

script/generate rspec
script/generate cucumber

These scripts create new rake tasks, a spec and feature folder, as well as modify your database.yml (adding a new cucumber environment).  You’ll also notice a new cucumber environment file added to your environments folder already completed with all the necessary config.gem entries, nice.

That’s all there is to it for getting started.  Next post I will talk about outside-in development and starting your first feature!

No comments »

Posted in Agile, Programming

Tags:

LSRC 2009 – Day 3

September 27th, 2009

How Much Testing

This presentation was very interesting.  It looked at the age-old question of “how much should we test and what do we test?”  It started out going over several real-life stories of companies who had failed to properly answer the question.  One in particular had a test suite that took over 15-20 minutes to build, over 5 minutes to start the server once built, and then used “webunit” tests that took forever to run.  Also the tests only covered 21% of the system — four years later the coverage was 24%!  (As a side note: Selenium testing is hard, to do right.)

For testing it’s critical to get feed back as immediate as possible.  So if your test suite it taking hours to run then you have issues.  You also should ask yourself if your tests are actually failing from time to time, are they really catching bugs and providing value?

Tests should influence your design.  If you are not finding that tests are influencing your design, they you’re probably not doing something right.  The speaker didn’t seem to specifically push TDD or BDD, but I could understand what he meant by this.  In order to test code easily you have to have testable code to begin with.  Code that is not easily tested is probably overly complicated and not designed/abstracted well.

“How easy is it for you to write a new feature?”  This begs the answer as to why testing is a good thing.  Well designed code is easier to maintain and change, and with tests you gain confidence that your application is working as you expected despite the changes.  Martin Fowler often explains that modifiability is the more important attribute to your code (even above performance), and you gain this through understandability (code is easy to follow/read) and testability.  Writing tests helps you write well designed, understandable, and testable code.

There are a lot of reasons why folks are not writing tests.  The speaker addressed the concerns of both management and developers.  Surprisingly he explained that managers are no longer the road-block, and that most managers now want testing – but the developers are resisting.  To write tests first requires a shift in thinking, and for many developers who’ve been coding a certain way for their entire careers, find this shift sometimes uncomfortable.

Ultimately the answer to the question “How much testing,” can be answered by asking, “What’s the right level of assurance?”  If you’re a company like Twitter (for example) if you lose tweets, or a search doesn’t come back exactly right is it really a big deal?  If you’re a patient in a hospital and your heart-lung machine software fails, is it a big deal?  Obviously you wouldn’t necessarily worry if Twitter admitted to less than 70% code-coverage, but you might be concerned if your heart-lung machine was anything less than 100% code-coverage.

Be Awesome

This was a very fast-paced talked that just touched on values or principals you should adopt to “be awesome,” taken from the speaker’s personal experience:

  • Pair program (remote paring is hard, but possible)
  • Respond to change
  • Use micro-frameworks and tools
  • Creativity and craftsmanship
  • Monolithic processes are unfun
  • Are your apps a joy to maintain?

Cucumber

I got the pleasure of seeing Joseph Wilk in person!  He gave his usual presentation on Cucumber, although there were some new features thrown in there.  Tagging was one of them.  The slideshow was pretty involved and if you would like to see an example of his presentation go check out this video:

Outside-in Development with Cucumber and Rspec

View more documents from Joseph Wilk.

Herding Tigers

This was the last big talk of the conference.  It was presented by Danny Blitz, who’s a very dynamic and interesting character.  He basically talked about agile teams, which he refers to as “tiger teams.”  He didn’t really talk about Scrum or XP or the more formal Agile methodologies, but instead discussed his own take on the corporate environment.

Book suggestions:  The Passion of Command, Corpus Business

What is a Tiger Team?

Small development team where velocity and quality are the primary drivers.  The team self-improves, has no meetings (or very few and short), test-automation is taken very seriously with actual QA members on the team.

“A lot of companies fake agile, wtf is a 3 hour scrum?!”

Workplace Archetypes as Animals

  • Tiger:  calculating, cautions individuals who avoid fights they can’t win
  • Cow: herd mentality, afraid of risk, have no original thoughts
  • Bear: big, mellow, live and let live attitude, also avoid fights they can’t win
  • Leopard: unmatched in ferocity, not a good team player though (avoid these)
  • Elephant: C level execs, invincible in battle (even a tiger will not take on an elephant)
  • Hyena: scavenger, dangerous weasels, steals ideas, back-stabbing

Leadership Characteristics

  • Faith and Love
  • Hope
  • Success belongs to the team
  • Failure belongs to the leader
  • The buck stops here
  • Fearlessness (moral courage)
  • Listener and learner
  • Protector

Also good leaders should manage by intent (servant leadership), not micromanage.  They should reward failure, people will make mistakes – let them feel comfortable in making them, as long as people learn.  Demand to be questioned, if your team is afraid of you they will not question your decisions and warn you when something is wrong.  Glorify the lower levels.

Good “grunts” show politeness and professionalism.  (They got it easy.)

Agile is a buffet not a dogma.  Methodologies are wrapped around Agile, *it* isn’t a methodology itself.

1 comment »

Posted in Agile, Programming

Tags:

May 2009 Cucumber Presentation

August 2nd, 2009
Below is a presentation I gave at Austin on Rails.  You can also find the audio for it here:  nicholas-cucumber-rails-audio
View more documents from ncancelliere.

No comments »

Posted in Agile, Programming

Tags:

BDD Step-by-Step Example (part 2, refactor)

March 29th, 2009

The cool thing about Cucumber is how it allows you to reuse your step definitions. It allows you to effectively create a DSL whereby you can generate new tests with little or no additional coding. As you’re writing step definitions keep this in mind, and look for patterns that you can dry up.

Below is an example of a step definition that has grown over time. These steps support a user administration feature using restful_authentication and aasm_state.

Given /^a user that is pending$/ do
  pending
end

When /^I activate the user$/ do
  pending
end

Then /^the user should be able to log in$/ do
  pending
end

Given /^a user that is active$/ do
  pending
end

When /^I suspend the user$/ do
  pending
end

Then /^the user should be in a suspended state$/ do
  pending
end

Then /^the user should not be able to log in$/ do
  pending
end

Given /^a user that is suspended$/ do
  pending
end

When /^I unsuspend the user$/ do
  pending
end

Then /^the user should be in a pending state$/ do
  pending
end

When /^I delete the user$/ do
  pending
end

Immediately you should recognize that there are some patterns here: given something about the user’s state, and when the admin does something to the state, and whether or not you can log in.

Let’s see if we can’t boil these down these 20 some steps into just 4-5? First lets aggregate the common Given’s, using the power of regular expression we can do something like this:

Given /^a user that is (.+)$/ do
end

Hmm, interesting — this might work, so lets try finishing it up:

Given /^a user that is (.+)$/ do |state|
  @user = User.create!(:login => 'testuser', :email => 'test@test.gov', :password => 'testme', :password_confirmation => 'testme', :state =>state)
end

Odd, the test is failing still. Well if you understand how restful_authentication works with aasm you’ll soon discover that any user created on the back-end starts out in a passive state. This actually is going to take some thought, so lets come back to it.

How about all the “When I” steps? Those are easily grouped together. Since restful_auth uses nice verbs for state transitions we can leverage this in our step definitions:

When /^I (.+) the user$/ do |state|
  @user.send("#{state}!")
end

Along the same line of thinking we can apply this to our “Then the user” steps:

Then /^the user should not be able to log in$/ do
  visit login_path
  fill_in "login", :with => @user.login
  fill_in "password", :with => @password
  click_button "Log In"
  response.should contain("Couldn't log you in")
end

Then /^the user should be able to log in$/ do
  visit login_path
  fill_in "login", :with => @user.login
  fill_in "password", :with => @password
  click_button "Log In"
  response.should contain("Logged in successfully")
end

Then /^the user should be in a (.+) state$/ do |state|
  @user.send("#{state}?")
end

There’s a lot going on in the log-in steps, we take the same exact steps to log in a user, the only change is the outcome we expect. Ok so lets refactor that out into a method, and we get:

Then /^the user should not be able to log in$/ do
  login_using_form("Couldn't log you in")
end

Then /^the user should be able to log in$/ do
  login_using_form("Logged in successfully")
end

def login_using_form(expectation)
  visit login_path
  fill_in "login", :with => @user.login
  fill_in "password", :with => @password
  click_button "Log In"
  response.should contain(expectation)
end

Ok, better, so what about the given a user? Anything we create on the backend will start out in a state of passive (short of changing how restful_auth works). Also not all the verbs we would use in a step definition equate to a valid transition in aasm (you cannot transition to passive or pending directly). Well it takes a little more thought, but in the end you might end up with a method like the one below:

Given /^a user that is (.+)$/ do |state|
  state_hash = { :active => "activate!", :suspended => "suspend!" }
  @password = 'testme'
  @user = User.create!(:login => 'testuser', :email => 'test@test.gov', :password => @password, :password_confirmation => @password)
  ##
  # Restful_Authentication doesn't provide direct state transistions for passive or pending,
  # so we do a little tweaking to our user object to get it into the desired state.
  #
  @user.register! unless state == 'passive'  # new accounts created through backend start out as passive
  unless state == 'pending'                  # accounts when registered become pending
    @user.send(state_hash[state.to_sym])     # pending can transition easily to any state
  end
end

Putting it all together we have a step definition file now of only 5 definitions and one helper method. These steps can support a wide combination of user starting states, user state transitions and expectations around logging in and user state post transition. Let the business folks go hog wild!

Given /^a user that is (.+)$/ do |state|
  state_hash = { :active => "activate!", :suspended => "suspend!" }
  @password = 'testme'
  @user = User.create!(:login => 'testuser', :email => 'test@test.gov', :password => @password, :password_confirmation => @password)
  ##
  # Restful_Authentication doesn't provide direct state transistions for passive or pending,
  # so we do a little tweaking to our user object to get it into the desired state.
  #
  @user.register! unless state == 'passive'  # new accounts created through backend start out as passive
  unless state == 'pending'                  # accounts when registered become pending
    @user.send(state_hash[state.to_sym])     # pending can transition easily to any state
  end
end

When /^I (.+) the user$/ do |state|
  @user.send("#{state}!")
end

Then /^the user should not be able to log in$/ do
  login_using_form("Couldn't log you in")
end

Then /^the user should be able to log in$/ do
  login_using_form("Logged in successfully")
end

Then /^the user should be in a (.+) state$/ do |state|
  @user.send("#{state}?")
end

# step helpers --------------------------------------#

# this method simulates a login
def login_using_form(expectation)
  visit login_path
  fill_in "login", :with => @user.login
  fill_in "password", :with => @password
  click_button "Log In"
  response.should contain(expectation)
end

A real-life feature that these steps support:

Feature:  User Administration

So that I can control access to the application
As an admin
I want to manage users

Scenario: Activate a pending user
  Given a user that is pending
  When I activate the user
  Then the user should be able to log in

Scenario: Suspend a user that is active
  Given a user that is active
  When I suspend the user
  Then the user should not be able to log in

Scenario: Unsuspend a user that is suspended
  Given a user that is suspended
  When I unsuspend the user
  Then the user should be in a pending state
  And the user should not be able to log in

Scenario: Purge a user that is active
  Given a user that is active
  When I delete the user
  Then the user should not be able to log in

Scenario: Purge a user that is pending
  Given a user that is pending
  When I delete the user
  Then the user should not be able to log in

Scenario: Purge a user that is suspended
  Given a user that is suspended
  When I delete the user
  Then the user should not be able to log in

1 comment »

Posted in Programming

Tags:

What’s TDD?

October 22nd, 2008

TDD is often viewed by managers, like paired programming, as a huge time suck. This is primarily due to a fundamental misunderstanding of what TDD is – it’s not just about testing.  TDD offers more value than simply a passing test.  Time spent wisely now will avoid big time sucks later.

TDD (and BDD, but I’m going to focus on TDD for simplicity; BDD is about behavior but very similar) is an approach to programming that is particularly popular in Agile development circles. Properly done TDD feels something like this:

a)  First you think about what test will best move your project forward.  (This part can take a while, take your time)

b)  Next you go RED, that is you write the smallest amount of test code that fails. (Less than 5 lines, please – should be a min or less of writing)

c)  Go GREEN. Write just enough code to get your test to pass, no more no less. (This helps you focus on not over-engineering your solution – should be a min or less of writing)

d)  Refactor your code and tests. Look for things like duplication and other “code smells” or if something doesn’t read elegantly and can be written better. Remember 90% of the life of this code will be spent in maintenance, so make it easy to maintain! You should continue to stay green as your refactor. (Take as much time as necessary on this one)

e)  Repeat, again and again.  You’ll find that after an hour you’ve done about 25-50 cycles.  TDD can be intensive, and it’s completely normal to cycle through a this pace.

Are You Wasting Time?

It depends what your goals are. If you want to write software with a low confidence of quality then by all means write code without tests. Your users will politely point out the bugs to you when they encounter them, to your embarrassment. You’ll need to take precious resources off existing projects and tasks, get them to research what’s going wrong, determine if it was supposed to be designed that way (maybe you just arn’t handling that expected error in a nice way, or maybe that error shouldn’t be happening to start). Poor customer satisfaction, project delays, and a lower team morale—what is that worth to you?

The Facts

The fact is all code has bugs, some more than others.

How fast can you identify them?

A failing unit test typically will tell you exactly what line in the code is failing. You could also go in manually poking around, but that takes more time. A suite of 700 tests can be exercised in under 7 seconds. Can a human being test 700 operations/functions of an application in that time? Probably not.

Writing tests while you write code doesn’t take any more effort than if you wrote the tests last. The difference is you identify bugs and issues sooner, which allows you to adapt quicker and fix the problems before they become problems. And remember kids, the cheapest time to fix a defect is at the time of the code’s authoring, and the most expensive when it’s in production.

Limitations of Testing

Testing isn’t a silver bullet, and as I said earlier – all code has bugs. Tests will greatly improve your code quality though. There are some things to keep in mind:

a) Developers tend to write “clean tests,” that is they don’t often think about edge-cases or ways in which the software might be utilized

b) Developers tend to have an optimistic view of code coverage. Many think they’re 95% covered, but in reality the best developers are probably around 80% and the worst 30% (McConnell, Code Complete 2). There are tools that can help measure code coverage.

c) Developers tend to skip more sophisticated kinds of test coverage. You might have 100% statement coverage, but it doesn’t mean all the branches are tested (eg. all the true/false, if, ifelse, else, etc).

Final Closing

TDD helps you write better designed code and applications.  It keeps you focused on the task and helps prevent over-engineering.

TDD helps you write code that is testable, after all it IS tested.

TDD helps you identify problems sooner both in design and the code itself (syntax, operations, etc). Identifying these sooner than later equals money saved.

TDD lets you refactor your code with confidence and deploy with confidence.

TDD doesn’t take any more effort than writing tests last, probably less.

No comments »

Posted in Programming

Tags: