Discussion:
[Cucumber:4066] Cucumber vs, RSpec
Ed Howland
2010-04-20 18:33:28 UTC
Permalink
Please forgive the x-post.

I just got back from the Great Lakes Ruby Bash. They had several good
presentations, two specific to BDD and Cucumber. I also talked to
several CEOs and devs afterwards, and the overall takeaway I gathered
was a shift to less RSpec and more Cucumber. Some people even claimed
a 90/10 split (cukes/specs) on current projects.

This was surorising to me and not at all how I worked up to this
point. I was more 20/80. I usually cuked a feature, then spec'ed the
code to make the cuke work. Each release had some new features and
specs for all the underlying code. Apparently, the feeling is that you
should do all your main thrusts with Cucumber and use RSpec for edge
cases. The theory is that you can change out all the underlying code
and the cukes still pass.

What is the communities consensus on this?


Cheers,
Ed

Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Robert Hanson
2010-04-20 18:43:49 UTC
Permalink
With Cucumber, you can't possibly specify tests for all the corner cases. That's what your rspecs (unit tests) are for.

-----Original Message-----
From: cukes-/***@public.gmane.org [mailto:cukes-/***@public.gmane.org] On Behalf Of Ed Howland
Sent: Tuesday, April 20, 2010 1:33 PM
To: rspec-users; cukes-/***@public.gmane.org
Subject: [Cucumber:4066] Cucumber vs, RSpec

Please forgive the x-post.

I just got back from the Great Lakes Ruby Bash. They had several good presentations, two specific to BDD and Cucumber. I also talked to several CEOs and devs afterwards, and the overall takeaway I gathered was a shift to less RSpec and more Cucumber. Some people even claimed a 90/10 split (cukes/specs) on current projects.

This was surorising to me and not at all how I worked up to this point. I was more 20/80. I usually cuked a feature, then spec'ed the code to make the cuke work. Each release had some new features and specs for all the underlying code. Apparently, the feeling is that you should do all your main thrusts with Cucumber and use RSpec for edge cases. The theory is that you can change out all the underlying code and the cukes still pass.

What is the communities consensus on this?


Cheers,
Ed

Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland

--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+***@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+***@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Mike Sassak
2010-04-20 18:57:07 UTC
Permalink
Post by Ed Howland
Please forgive the x-post.
I just got back from the Great Lakes Ruby Bash. They had several good
presentations, two specific to BDD and Cucumber. I also talked to
several CEOs and devs afterwards, and the overall takeaway I gathered
was a shift to less RSpec and more Cucumber. Some people even claimed
a 90/10 split (cukes/specs) on current projects.
This was surorising to me and not at all how I worked up to this
point. I was more 20/80. I usually cuked a feature, then spec'ed the
code to make the cuke work. Each release had some new features and
specs for all the underlying code. Apparently, the feeling is that you
should do all your main thrusts with Cucumber and use RSpec for edge
cases. The theory is that you can change out all the underlying code
and the cukes still pass.
What is the communities consensus on this?
Hi Ed,

I was also at the GLRB, and was a bit aghast at the claim that you
should have a 90/10 split between cukes and rspec. In my experience,
favoring Cucumber so heavily invites developing code that behaves
correctly, but is messy and difficult to change. I would go so far as
to claim there is a positive correlation between over-reliance on
Cucumber features and rampant violations of the SOLID principles.
Cucumber simply doesn't excel at enforcing simple, testable contracts
between the objects in your code base the way RSpec does. The result
is that your code is hard to refactor and change, which from the
developer's point of view is practically the whole reason to maintain
a good test suite in the first place. This isn't the whole of the
story by any means, but I think it's close to the place to start.

$0.02
Mike

P.S. Hello RSpec Group! What's the etiquette here for cross-posting?
Post by Ed Howland
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
David Chelimsky
2010-04-20 19:05:03 UTC
Permalink
Post by Mike Sassak
Post by Ed Howland
Please forgive the x-post.
I just got back from the Great Lakes Ruby Bash. They had several good
presentations, two specific to BDD and Cucumber. I also talked to
several CEOs and devs afterwards, and the overall takeaway I gathered
was a shift to less RSpec and more Cucumber. Some people even claimed
a 90/10 split (cukes/specs) on current projects.
This was surorising to me and not at all how I worked up to this
point. I was more 20/80. I usually cuked a feature, then spec'ed the
code to make the cuke work. Each release had some new features and
specs for all the underlying code. Apparently, the feeling is that you
should do all your main thrusts with Cucumber and use RSpec for edge
cases. The theory is that you can change out all the underlying code
and the cukes still pass.
What is the communities consensus on this?
Hi Ed,
I was also at the GLRB, and was a bit aghast at the claim that you
should have a 90/10 split between cukes and rspec. In my experience,
favoring Cucumber so heavily invites developing code that behaves
correctly, but is messy and difficult to change. I would go so far as
to claim there is a positive correlation between over-reliance on
Cucumber features and rampant violations of the SOLID principles.
Cucumber simply doesn't excel at enforcing simple, testable contracts
between the objects in your code base the way RSpec does. The result
is that your code is hard to refactor and change, which from the
developer's point of view is practically the whole reason to maintain
a good test suite in the first place. This isn't the whole of the
story by any means, but I think it's close to the place to start.
$0.02
Mike
P.S. Hello RSpec Group! What's the etiquette here for cross-posting?
Etiquette, schmetiquette :)

I'd say, in the interest of keeping the thread in one place, post to the rspec list w/ a link to this thread in the cuke group and invite folks to join the convo.

Cheers,
David
Post by Mike Sassak
Post by Ed Howland
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
David Chelimsky
2010-04-20 19:48:02 UTC
Permalink
Post by David Chelimsky
Post by Mike Sassak
Post by Ed Howland
Please forgive the x-post.
I just got back from the Great Lakes Ruby Bash. They had several good
presentations, two specific to BDD and Cucumber. I also talked to
several CEOs and devs afterwards, and the overall takeaway I gathered
was a shift to less RSpec and more Cucumber. Some people even claimed
a 90/10 split (cukes/specs) on current projects.
This was surorising to me and not at all how I worked up to this
point. I was more 20/80. I usually cuked a feature, then spec'ed the
code to make the cuke work. Each release had some new features and
specs for all the underlying code. Apparently, the feeling is that you
should do all your main thrusts with Cucumber and use RSpec for edge
cases. The theory is that you can change out all the underlying code
and the cukes still pass.
What is the communities consensus on this?
Hi Ed,
I was also at the GLRB, and was a bit aghast at the claim that you
should have a 90/10 split between cukes and rspec. In my experience,
favoring Cucumber so heavily invites developing code that behaves
correctly, but is messy and difficult to change. I would go so far as
to claim there is a positive correlation between over-reliance on
Cucumber features and rampant violations of the SOLID principles.
Cucumber simply doesn't excel at enforcing simple, testable contracts
between the objects in your code base the way RSpec does. The result
is that your code is hard to refactor and change, which from the
developer's point of view is practically the whole reason to maintain
a good test suite in the first place. This isn't the whole of the
story by any means, but I think it's close to the place to start.
$0.02
Mike
P.S. Hello RSpec Group! What's the etiquette here for cross-posting?
Etiquette, schmetiquette :)
I'd say, in the interest of keeping the thread in one place, post to the rspec list w/ a link to this thread in the cuke group and invite folks to join the convo.
I answered this before I realized you were already x-posting :)

I think this conversation is relevant to both communities, so let's
keep it going in both.
Post by David Chelimsky
Cheers,
David
Post by Mike Sassak
Post by Ed Howland
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Joseph Wilk
2010-04-20 19:58:17 UTC
Permalink
Post by David Chelimsky
Post by Mike Sassak
Post by Ed Howland
Please forgive the x-post.
I just got back from the Great Lakes Ruby Bash. They had several good
presentations, two specific to BDD and Cucumber. I also talked to
several CEOs and devs afterwards, and the overall takeaway I gathered
was a shift to less RSpec and more Cucumber. Some people even claimed
a 90/10 split (cukes/specs) on current projects.
This was surorising to me and not at all how I worked up to this
point. I was more 20/80. I usually cuked a feature, then spec'ed the
code to make the cuke work. Each release had some new features and
specs for all the underlying code. Apparently, the feeling is that you
should do all your main thrusts with Cucumber and use RSpec for edge
cases. The theory is that you can change out all the underlying code
and the cukes still pass.
What is the communities consensus on this?
Hi Ed,
I was also at the GLRB, and was a bit aghast at the claim that you
should have a 90/10 split between cukes and rspec. In my experience,
favoring Cucumber so heavily invites developing code that behaves
correctly, but is messy and difficult to change. I would go so far as
to claim there is a positive correlation between over-reliance on
Cucumber features and rampant violations of the SOLID principles.
Cucumber simply doesn't excel at enforcing simple, testable contracts
between the objects in your code base the way RSpec does. The result
is that your code is hard to refactor and change, which from the
developer's point of view is practically the whole reason to maintain
a good test suite in the first place. This isn't the whole of the
story by any means, but I think it's close to the place to start.
First up Rspec and Cucumber are just tools, they can be used in many
ways. So this answer belies my personal usage of these tools.

A big issue for me is scaling tests. Cucumber tests tend to be
end-to-end tests so they cut through the whole application stack. This
is great in terms of freeing you up to refactor the heck out of your
code without having to rewrite lots of tests. But end-to-end tests are
slow, now this can be ok if you working on a small project. In smaller
projects I've worked on I've only used Cukes. In others I've only
tended to drop down to Rspec (which I very much use as a specing/unit
testing tool) when there is complexity, I feel the feedback loop is
not fast enough or I need to explore the design more.

If however you are working on an application thats long lived or lives
in a domain where you're dealing with asynchronous issues (such as
javascript or evented systems) I've seen people very quickly hit 1
hour + test build time. Primarily because they have such a heavy focus
on Cucumber or end-to-end tests. So one direction to help avoid this
is to exploring a few good and bad paths with cucumber but having more
detailed spec coverage. This would help you manage better test build
times.

The other option is to not worry about heavy Cukes usage and throw
lots of hardware at the scaling problem. Ok for some, but it does
end-up costing lots.

One other point is that Cucumber for me is part of a process about
facilitating conversations with non techs. So as a developer its not
always a question of how many Cukes do I think I should have. Its a
question of how much does the stakeholders who I'm writing the
software want. Do they want to edit and write the cukes with us? Will
they go back and reference the cukes in the future?

So in conclusion my split on Cucumbers/Rspecs really depends on the
context of the project. An important factor to think about is scaling
when you only use end-to-end tests.

Joseph Wilk
http://blog.josephwilk.net
http://www.songkick.com
+44 (0)7812 816431
Post by David Chelimsky
Post by Mike Sassak
$0.02
Mike
P.S. Hello RSpec Group! What's the etiquette here for cross-posting?
Etiquette, schmetiquette :)
I'd say, in the interest of keeping the thread in one place, post to the rspec list w/ a link to this thread in the cuke group and invite folks to join the convo.
Cheers,
David
Post by Mike Sassak
Post by Ed Howland
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
John Goodsen
2010-04-20 20:12:30 UTC
Permalink
Watch Ben Mabey's slides and talk at Ruby Conf on outside in development
with Cucumber. It positions rspec and cucumber properly
Post by Joseph Wilk
Post by David Chelimsky
Post by Mike Sassak
Post by Ed Howland
Please forgive the x-post.
I just got back from the Great Lakes Ruby Bash. They had several good
presentations, two specific to BDD and Cucumber. I also talked to
several CEOs and devs afterwards, and the overall takeaway I gathered
was a shift to less RSpec and more Cucumber. Some people even claimed
a 90/10 split (cukes/specs) on current projects.
This was surorising to me and not at all how I worked up to this
point. I was more 20/80. I usually cuked a feature, then spec'ed the
code to make the cuke work. Each release had some new features and
specs for all the underlying code. Apparently, the feeling is that you
should do all your main thrusts with Cucumber and use RSpec for edge
cases. The theory is that you can change out all the underlying code
and the cukes still pass.
What is the communities consensus on this?
Hi Ed,
I was also at the GLRB, and was a bit aghast at the claim that you
should have a 90/10 split between cukes and rspec. In my experience,
favoring Cucumber so heavily invites developing code that behaves
correctly, but is messy and difficult to change. I would go so far as
to claim there is a positive correlation between over-reliance on
Cucumber features and rampant violations of the SOLID principles.
Cucumber simply doesn't excel at enforcing simple, testable contracts
between the objects in your code base the way RSpec does. The result
is that your code is hard to refactor and change, which from the
developer's point of view is practically the whole reason to maintain
a good test suite in the first place. This isn't the whole of the
story by any means, but I think it's close to the place to start.
First up Rspec and Cucumber are just tools, they can be used in many
ways. So this answer belies my personal usage of these tools.
A big issue for me is scaling tests. Cucumber tests tend to be
end-to-end tests so they cut through the whole application stack. This
is great in terms of freeing you up to refactor the heck out of your
code without having to rewrite lots of tests. But end-to-end tests are
slow, now this can be ok if you working on a small project. In smaller
projects I've worked on I've only used Cukes. In others I've only
tended to drop down to Rspec (which I very much use as a specing/unit
testing tool) when there is complexity, I feel the feedback loop is
not fast enough or I need to explore the design more.
If however you are working on an application thats long lived or lives
in a domain where you're dealing with asynchronous issues (such as
javascript or evented systems) I've seen people very quickly hit 1
hour + test build time. Primarily because they have such a heavy focus
on Cucumber or end-to-end tests. So one direction to help avoid this
is to exploring a few good and bad paths with cucumber but having more
detailed spec coverage. This would help you manage better test build
times.
The other option is to not worry about heavy Cukes usage and throw
lots of hardware at the scaling problem. Ok for some, but it does
end-up costing lots.
One other point is that Cucumber for me is part of a process about
facilitating conversations with non techs. So as a developer its not
always a question of how many Cukes do I think I should have. Its a
question of how much does the stakeholders who I'm writing the
software want. Do they want to edit and write the cukes with us? Will
they go back and reference the cukes in the future?
So in conclusion my split on Cucumbers/Rspecs really depends on the
context of the project. An important factor to think about is scaling
when you only use end-to-end tests.
Joseph Wilk
http://blog.josephwilk.net
http://www.songkick.com
+44 (0)7812 816431
Post by David Chelimsky
Post by Mike Sassak
$0.02
Mike
P.S. Hello RSpec Group! What's the etiquette here for cross-posting?
Etiquette, schmetiquette :)
I'd say, in the interest of keeping the thread in one place, post to the
rspec list w/ a link to this thread in the cuke group and invite folks to
join the convo.
Post by David Chelimsky
Cheers,
David
Post by Mike Sassak
Post by Ed Howland
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
--
You received this message because you are subscribed to the Google
Groups "Cukes" group.
Post by David Chelimsky
Post by Mike Sassak
Post by Ed Howland
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
Post by David Chelimsky
Post by Mike Sassak
--
You received this message because you are subscribed to the Google
Groups "Cukes" group.
Post by David Chelimsky
Post by Mike Sassak
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
Post by David Chelimsky
--
You received this message because you are subscribed to the Google Groups
"Cukes" group.
Post by David Chelimsky
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
_______________________________________________
rspec-users mailing list
http://rubyforge.org/mailman/listinfo/rspec-users
--
John Goodsen RADSoft / Better Software Faster
jgoodsen-***@public.gmane.org Lean/Agile/XP/Scrum Coaching and Training
http://www.radsoft.com Ruby on Rails and Java Solutions
Joseph Wilk
2010-04-20 20:24:32 UTC
Permalink
Post by John Goodsen
Watch Ben Mabey's slides and talk at Ruby Conf on outside in development
with Cucumber.  It positions rspec and cucumber properly
I used an analogy at Scotland on Rails that might be helpful. I was
thinking about Rspec and Cucumber in terms of planning a driving route
between London and Edinburgh.

Cucumber would test the output, the value:

"Then I am in Edinburgh"

So while driving, knowing I'm not in Edinburgh yet is not really not
that much help for me. Its my end value but the feedback loop is too
big. I could drive to Edinburgh via the USA and still achieve the
value. I like to think of Rspec as a route planner which is giving me
continuous feedback, helping me improve the design of my route. The
feedback loop is fast and it helps me get to my value.

You checkout the presentation:
http://blog.josephwilk.net/ruby/outside-in-development-with-cucumber-and-rspec.html

--
Joseph Wilk
http://blog.josephwilk.net
http://www.songkick.com
+44 (0)7812 816431
Post by John Goodsen
Post by Joseph Wilk
Post by David Chelimsky
Post by Mike Sassak
Post by Ed Howland
Please forgive the x-post.
I just got back from the Great Lakes Ruby Bash. They had several good
presentations, two specific to BDD and Cucumber. I also talked to
several CEOs and devs afterwards, and the overall takeaway I gathered
was a shift to less RSpec and more Cucumber. Some people even claimed
a 90/10 split (cukes/specs) on current projects.
This was surorising to me and not at all how I worked up to this
point. I was more 20/80. I usually cuked a feature, then spec'ed the
code to make the cuke work. Each release had some new features and
specs for all the underlying code. Apparently, the feeling is that you
should do all your main thrusts with Cucumber and use RSpec for edge
cases. The theory is that you can change out all the underlying code
and the cukes still pass.
What is the communities consensus on this?
Hi Ed,
I was also at the GLRB, and was a bit aghast at the claim that you
should have a 90/10 split between cukes and rspec. In my experience,
favoring Cucumber so heavily invites developing code that behaves
correctly, but is messy and difficult to change. I would go so far as
to claim there is a positive correlation between over-reliance on
Cucumber features and rampant violations of the SOLID principles.
Cucumber simply doesn't excel at enforcing simple, testable contracts
between the objects in your code base the way RSpec does. The result
is that your code is hard to refactor and change, which from the
developer's point of view is practically the whole reason to maintain
a good test suite in the first place. This isn't the whole of the
story by any means, but I think it's close to the place to start.
First up Rspec and Cucumber are just tools, they can be used in many
ways. So this answer belies my personal usage of these tools.
A big issue for me is scaling tests. Cucumber tests tend to be
end-to-end tests so they cut through the whole application stack. This
is great in terms of freeing you up to refactor the heck out of your
code without having to rewrite lots of tests. But end-to-end tests are
slow, now this can be ok if you working on a small project. In smaller
projects I've worked on I've only used Cukes. In others I've only
tended to drop down to Rspec (which I very much use as a specing/unit
testing tool) when there is complexity, I feel the feedback loop is
not fast enough or I need to explore the design more.
If however you are working on an application thats long lived or lives
in a domain where you're dealing with asynchronous issues (such as
javascript or evented systems) I've seen people very quickly hit 1
hour + test build time. Primarily because they have such a heavy focus
on Cucumber or end-to-end tests. So one direction to help avoid this
is to exploring a few good and bad paths with cucumber but having more
detailed spec coverage. This would help you manage better test build
times.
The other option is to not worry about heavy Cukes usage and throw
lots of hardware at the scaling problem. Ok for some, but it does
end-up costing lots.
One other point is that Cucumber for me is part of a process about
facilitating conversations with non techs. So as a developer its not
always a question of how many Cukes do I think I should have. Its a
question of how much does the stakeholders who I'm writing the
software want. Do they want to edit and write the cukes with us? Will
they go back and reference the cukes in the future?
So in conclusion my split on Cucumbers/Rspecs really depends on the
context of the project. An important factor to think about is scaling
when you only use end-to-end tests.
Joseph Wilk
http://blog.josephwilk.net
http://www.songkick.com
+44 (0)7812 816431
Post by David Chelimsky
Post by Mike Sassak
$0.02
Mike
P.S. Hello RSpec Group! What's the etiquette here for cross-posting?
Etiquette, schmetiquette :)
I'd say, in the interest of keeping the thread in one place, post to the
rspec list w/ a link to this thread in the cuke group and invite folks to
join the convo.
Cheers,
David
Post by Mike Sassak
Post by Ed Howland
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
--
You received this message because you are subscribed to the Google
Groups "Cukes" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google
Groups "Cukes" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google
Groups "Cukes" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
_______________________________________________
rspec-users mailing list
http://rubyforge.org/mailman/listinfo/rspec-users
--
John Goodsen                 RADSoft / Better Software Faster
http://www.radsoft.com          Ruby on Rails and Java Solutions
_______________________________________________
rspec-users mailing list
http://rubyforge.org/mailman/listinfo/rspec-users
Lucas Prim
2010-04-20 20:30:29 UTC
Permalink
For me, cukes acts more as an acceptance test that tells me when to stop the
development and release a feature as rspec goes down to the internals of the
app in order to make sure everything works ok.
In this context, I think there is no magic recipe that tells you how much
cucumber code you should have on your project in contrast with rspec code.
It all depends on the context you're working. Complex enterprise
applications require extensive unit testing and there i wouldn't wanna miss
rspec.
Im my personal experience, i use cucumber-only for CRUDS and other
down-to-the-basic stuff and whenever i need to develop a challenging chunk
of code i firstly spec it with rspec and then make sure it works with the
rest by running the cucumber feature. Sometimes it's like 500 lines of rspec
and 30 lines of cucumber. It has been working very well!


Lucas Prim
lucas.prim-***@public.gmane.org
+55 (48) 9921-9303
+55 (48) 3334-1979
Post by John Goodsen
Watch Ben Mabey's slides and talk at Ruby Conf on outside in development
with Cucumber. It positions rspec and cucumber properly
Post by Joseph Wilk
Post by David Chelimsky
Post by Mike Sassak
Post by Ed Howland
Please forgive the x-post.
I just got back from the Great Lakes Ruby Bash. They had several good
presentations, two specific to BDD and Cucumber. I also talked to
several CEOs and devs afterwards, and the overall takeaway I gathered
was a shift to less RSpec and more Cucumber. Some people even claimed
a 90/10 split (cukes/specs) on current projects.
This was surorising to me and not at all how I worked up to this
point. I was more 20/80. I usually cuked a feature, then spec'ed the
code to make the cuke work. Each release had some new features and
specs for all the underlying code. Apparently, the feeling is that you
should do all your main thrusts with Cucumber and use RSpec for edge
cases. The theory is that you can change out all the underlying code
and the cukes still pass.
What is the communities consensus on this?
Hi Ed,
I was also at the GLRB, and was a bit aghast at the claim that you
should have a 90/10 split between cukes and rspec. In my experience,
favoring Cucumber so heavily invites developing code that behaves
correctly, but is messy and difficult to change. I would go so far as
to claim there is a positive correlation between over-reliance on
Cucumber features and rampant violations of the SOLID principles.
Cucumber simply doesn't excel at enforcing simple, testable contracts
between the objects in your code base the way RSpec does. The result
is that your code is hard to refactor and change, which from the
developer's point of view is practically the whole reason to maintain
a good test suite in the first place. This isn't the whole of the
story by any means, but I think it's close to the place to start.
First up Rspec and Cucumber are just tools, they can be used in many
ways. So this answer belies my personal usage of these tools.
A big issue for me is scaling tests. Cucumber tests tend to be
end-to-end tests so they cut through the whole application stack. This
is great in terms of freeing you up to refactor the heck out of your
code without having to rewrite lots of tests. But end-to-end tests are
slow, now this can be ok if you working on a small project. In smaller
projects I've worked on I've only used Cukes. In others I've only
tended to drop down to Rspec (which I very much use as a specing/unit
testing tool) when there is complexity, I feel the feedback loop is
not fast enough or I need to explore the design more.
If however you are working on an application thats long lived or lives
in a domain where you're dealing with asynchronous issues (such as
javascript or evented systems) I've seen people very quickly hit 1
hour + test build time. Primarily because they have such a heavy focus
on Cucumber or end-to-end tests. So one direction to help avoid this
is to exploring a few good and bad paths with cucumber but having more
detailed spec coverage. This would help you manage better test build
times.
The other option is to not worry about heavy Cukes usage and throw
lots of hardware at the scaling problem. Ok for some, but it does
end-up costing lots.
One other point is that Cucumber for me is part of a process about
facilitating conversations with non techs. So as a developer its not
always a question of how many Cukes do I think I should have. Its a
question of how much does the stakeholders who I'm writing the
software want. Do they want to edit and write the cukes with us? Will
they go back and reference the cukes in the future?
So in conclusion my split on Cucumbers/Rspecs really depends on the
context of the project. An important factor to think about is scaling
when you only use end-to-end tests.
Joseph Wilk
http://blog.josephwilk.net
http://www.songkick.com
+44 (0)7812 816431
Post by David Chelimsky
Post by Mike Sassak
$0.02
Mike
P.S. Hello RSpec Group! What's the etiquette here for cross-posting?
Etiquette, schmetiquette :)
I'd say, in the interest of keeping the thread in one place, post to the
rspec list w/ a link to this thread in the cuke group and invite folks to
join the convo.
Post by David Chelimsky
Cheers,
David
Post by Mike Sassak
Post by Ed Howland
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
--
You received this message because you are subscribed to the Google
Groups "Cukes" group.
Post by David Chelimsky
Post by Mike Sassak
Post by Ed Howland
To unsubscribe from this group, send email to
.
Post by David Chelimsky
Post by Mike Sassak
Post by Ed Howland
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
Post by David Chelimsky
Post by Mike Sassak
--
You received this message because you are subscribed to the Google
Groups "Cukes" group.
Post by David Chelimsky
Post by Mike Sassak
To unsubscribe from this group, send email to
.
Post by David Chelimsky
Post by Mike Sassak
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
Post by David Chelimsky
--
You received this message because you are subscribed to the Google
Groups "Cukes" group.
Post by David Chelimsky
To unsubscribe from this group, send email to
.
Post by David Chelimsky
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
_______________________________________________
rspec-users mailing list
http://rubyforge.org/mailman/listinfo/rspec-users
--
John Goodsen RADSoft / Better Software Faster
http://www.radsoft.com Ruby on Rails and Java Solutions
_______________________________________________
rspec-users mailing list
http://rubyforge.org/mailman/listinfo/rspec-users
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Ed Howland
2010-04-21 00:35:34 UTC
Permalink
Cool stuff to think about. This pretty much confirms my gut feeling on
the balance between the two. I've worked on big Rails projects in the
past, but before Cucumber existed. Now, I do small stuff and use
Cucumber to plan the gems I am writing. I drop into RSpec to get the
rapid feedback going.

I liked the London -> Edinburgh analogy (partly because I've driven
the route back in the '70s).

I am improving my cukes with Aruba and Capybara. So I learned something useful.

Cheers,
Ed

Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
Post by Lucas Prim
For me, cukes acts more as an acceptance test that tells me when to stop the
development and release a feature as rspec goes down to the internals of the
app in order to make sure everything works ok.
In this context, I think there is no magic recipe that tells you how much
cucumber code you should have on your project in contrast with rspec code.
It all depends on the context you're working. Complex enterprise
applications require extensive unit testing and there i wouldn't wanna miss
rspec.
Im my personal experience, i use cucumber-only for CRUDS and other
down-to-the-basic stuff and whenever i need to develop a challenging chunk
of code i firstly spec it with rspec and then make sure it works with the
rest by running the cucumber feature. Sometimes it's like 500 lines of rspec
and 30 lines of cucumber. It has been working very well!
Lucas Prim
+55 (48) 9921-9303
+55 (48) 3334-1979
Post by John Goodsen
Watch Ben Mabey's slides and talk at Ruby Conf on outside in development
with Cucumber.  It positions rspec and cucumber properly
Post by Joseph Wilk
Post by David Chelimsky
Post by Mike Sassak
Post by Ed Howland
Please forgive the x-post.
I just got back from the Great Lakes Ruby Bash. They had several good
presentations, two specific to BDD and Cucumber. I also talked to
several CEOs and devs afterwards, and the overall takeaway I gathered
was a shift to less RSpec and more Cucumber. Some people even claimed
a 90/10 split (cukes/specs) on current projects.
This was surorising to me and not at all how I worked up to this
point. I was more 20/80. I usually cuked a feature, then spec'ed the
code to make the cuke work. Each release had some new features and
specs for all the underlying code. Apparently, the feeling is that you
should do all your main thrusts with Cucumber and use RSpec for edge
cases. The theory is that you can change out all the underlying code
and the cukes still pass.
What is the communities consensus on this?
Hi Ed,
I was also at the GLRB, and was a bit aghast at the claim that you
should have a 90/10 split between cukes and rspec. In my experience,
favoring Cucumber so heavily invites developing code that behaves
correctly, but is messy and difficult to change. I would go so far as
to claim there is a positive correlation between over-reliance on
Cucumber features and rampant violations of the SOLID principles.
Cucumber simply doesn't excel at enforcing simple, testable contracts
between the objects in your code base the way RSpec does. The result
is that your code is hard to refactor and change, which from the
developer's point of view is practically the whole reason to maintain
a good test suite in the first place. This isn't the whole of the
story by any means, but I think it's close to the place to start.
First up Rspec and Cucumber are just tools, they can be used in many
ways. So this answer belies my personal usage of these tools.
A big issue for me is scaling tests. Cucumber tests tend to be
end-to-end tests so they cut through the whole application stack. This
is great in terms of freeing you up to refactor the heck out of your
code without having to rewrite lots of tests. But end-to-end tests are
slow, now this can be ok if you working on a small project. In smaller
projects I've worked on I've only used Cukes. In others I've only
tended to drop down to Rspec (which I very much use as a specing/unit
testing tool) when there is complexity, I feel the feedback loop is
not fast enough or I need to explore the design more.
If however you are working on an application thats long lived or lives
in a domain where you're dealing with asynchronous issues (such as
javascript or evented systems) I've seen people very quickly hit 1
hour + test build time. Primarily because they have such a heavy focus
on Cucumber or end-to-end tests. So one direction to help avoid this
is to exploring a few good and bad paths with cucumber but having more
detailed spec coverage. This would help you manage better test build
times.
The other option is to not worry about heavy Cukes usage and throw
lots of hardware at the scaling problem. Ok for some, but it does
end-up costing lots.
One other point is that Cucumber for me is part of a process about
facilitating conversations with non techs. So as a developer its not
always a question of how many Cukes do I think I should have. Its a
question of how much does the stakeholders who I'm writing the
software want. Do they want to edit and write the cukes with us? Will
they go back and reference the cukes in the future?
So in conclusion my split on Cucumbers/Rspecs really depends on the
context of the project. An important factor to think about is scaling
when you only use end-to-end tests.
Joseph Wilk
http://blog.josephwilk.net
http://www.songkick.com
+44 (0)7812 816431
Post by David Chelimsky
Post by Mike Sassak
$0.02
Mike
P.S. Hello RSpec Group! What's the etiquette here for cross-posting?
Etiquette, schmetiquette :)
I'd say, in the interest of keeping the thread in one place, post to
the rspec list w/ a link to this thread in the cuke group and invite folks
to join the convo.
Cheers,
David
Post by Mike Sassak
Post by Ed Howland
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
--
You received this message because you are subscribed to the Google
Groups "Cukes" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google
Groups "Cukes" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google
Groups "Cukes" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
_______________________________________________
rspec-users mailing list
http://rubyforge.org/mailman/listinfo/rspec-users
--
John Goodsen                 RADSoft / Better Software Faster
http://www.radsoft.com          Ruby on Rails and Java Solutions
_______________________________________________
rspec-users mailing list
http://rubyforge.org/mailman/listinfo/rspec-users
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+***@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Matt Wynne
2010-04-21 06:59:55 UTC
Permalink
Here's an idea relevant to this discussion that came up last week when
I did a talk on Cucumber to ScotRUG.

Paul Wilson was describing how he used to use Fit, back in the day,
for testing at different levels in the stack rather than just for end-
to-end tests. The idea was to surface the tests at the same customer-
facing level, but steer them at the *appropriate* level in the stack,
depending on what was being tested.

So for example in the canonical problem of roles with different access
rights to a system, you could write the nice readable tabulated test
in Cucumber, but instead of running that cuke against the whole stack,
stepping through the login form etc, you could just run it against
whatever class is responsible for managing roles.

In order for this to work, you'd need to write the Cukes in a
declarative style, avoiding too much detail about exactly how you'd
carry out the step. This allows the same step to be run against the
whole stack or a focussed piece of the system instead. I tend to think
now that this is just good practice anyway.

This isn't something I've tried myself, but I imagine it means you'd
still get the benefit of customer-readable tests that validate the
behaviour of the system, but running potentially much faster than if
they were end-to-end tests. I can see a disadvantage that it could
make the test code more complex and confusing. You'd obviously still
need a few end-to-end tests in order to make sure the whole thing
still wired up together.

On a team with good customer-developer trust, I'd probably just use
RSpec for these kind of tests, but it's interesting to try thinking
about cucumber as The Place where business rules are surfaced.

Anyone tried doing anything like this?
Post by Ed Howland
Please forgive the x-post.
I just got back from the Great Lakes Ruby Bash. They had several good
presentations, two specific to BDD and Cucumber. I also talked to
several CEOs and devs afterwards, and the overall takeaway I gathered
was a shift to less RSpec and more Cucumber. Some people even claimed
a 90/10 split (cukes/specs) on current projects.
This was surorising to me and not at all how I worked up to this
point. I was more 20/80. I usually cuked a feature, then spec'ed the
code to make the cuke work. Each release had some new features and
specs for all the underlying code. Apparently, the feeling is that you
should do all your main thrusts with Cucumber and use RSpec for edge
cases. The theory is that you can change out all the underlying code
and the cukes still pass.
What is the communities consensus on this?
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en
.
cheers,
Matt

http://mattwynne.net
+447974 430184
George Dinwiddie
2010-04-21 15:09:06 UTC
Permalink
Matt,

Yes, I do that and recommend it. Thanks for writing such a clear
description. Dale Emery calls this expressing the /essence/ of the user
story.

- George
Here's an idea relevant to this discussion that came up last week when I
did a talk on Cucumber to ScotRUG.
Paul Wilson was describing how he used to use Fit, back in the day, for
testing at different levels in the stack rather than just for end-to-end
tests. The idea was to surface the tests at the same customer-facing
level, but steer them at the *appropriate* level in the stack, depending
on what was being tested.
So for example in the canonical problem of roles with different access
rights to a system, you could write the nice readable tabulated test in
Cucumber, but instead of running that cuke against the whole stack,
stepping through the login form etc, you could just run it against
whatever class is responsible for managing roles.
In order for this to work, you'd need to write the Cukes in a
declarative style, avoiding too much detail about exactly how you'd
carry out the step. This allows the same step to be run against the
whole stack or a focussed piece of the system instead. I tend to think
now that this is just good practice anyway.
This isn't something I've tried myself, but I imagine it means you'd
still get the benefit of customer-readable tests that validate the
behaviour of the system, but running potentially much faster than if
they were end-to-end tests. I can see a disadvantage that it could make
the test code more complex and confusing. You'd obviously still need a
few end-to-end tests in order to make sure the whole thing still wired
up together.
On a team with good customer-developer trust, I'd probably just use
RSpec for these kind of tests, but it's interesting to try thinking
about cucumber as The Place where business rules are surfaced.
Anyone tried doing anything like this?
Post by Ed Howland
Please forgive the x-post.
I just got back from the Great Lakes Ruby Bash. They had several good
presentations, two specific to BDD and Cucumber. I also talked to
several CEOs and devs afterwards, and the overall takeaway I gathered
was a shift to less RSpec and more Cucumber. Some people even claimed
a 90/10 split (cukes/specs) on current projects.
This was surorising to me and not at all how I worked up to this
point. I was more 20/80. I usually cuked a feature, then spec'ed the
code to make the cuke work. Each release had some new features and
specs for all the underlying code. Apparently, the feeling is that you
should do all your main thrusts with Cucumber and use RSpec for edge
cases. The theory is that you can change out all the underlying code
and the cukes still pass.
What is the communities consensus on this?
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
cheers,
Matt
http://mattwynne.net
+447974 430184
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Ed Howland
2010-04-21 15:59:48 UTC
Permalink
Post by Matt Wynne
So for example in the canonical problem of roles with different access
rights to a system, you could write the nice readable tabulated test in
Cucumber, but instead of running that cuke against the whole stack, stepping
through the login form etc, you could just run it against whatever class is
responsible for managing roles.
In order for this to work, you'd need to write the Cukes in a declarative
style, avoiding too much detail about exactly how you'd carry out the step.
This allows the same step to be run against the whole stack or a focussed
piece of the system instead. I tend to think now that this is just good
practice anyway.
This sounds like a good idea, but I'm confused about how to implement
it. My lack of knowledge about Cucumber is showing through. Would you
have some different step_definition file and how do you tell Cucumber
to use a different step def file? Or do you put some sort of
conditional in each step definition that fires on some external flag
(environment variable)? Or is there someway of accessing tags passed
to Cucumber?

Thanks


Cheers,
Ed

Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
Post by Matt Wynne
Here's an idea relevant to this discussion that came up last week when I did
a talk on Cucumber to ScotRUG.
Paul Wilson was describing how he used to use Fit, back in the day, for
testing at different levels in the stack rather than just for end-to-end
tests. The idea was to surface the tests at the same customer-facing level,
but steer them at the *appropriate* level in the stack, depending on what
was being tested.
This isn't something I've tried myself, but I imagine it means you'd still
get the benefit of customer-readable tests that validate the behaviour of
the system, but running potentially much faster than if they were end-to-end
tests. I can see a disadvantage that it could make the test code more
complex and confusing. You'd obviously still need a few end-to-end tests in
order to make sure the whole thing still wired up together.
On a team with good customer-developer trust, I'd probably just use RSpec
for these kind of tests, but it's interesting to try thinking about cucumber
as The Place where business rules are surfaced.
Anyone tried doing anything like this?
Post by Ed Howland
Please forgive the x-post.
I just got back from the Great Lakes Ruby Bash. They had several good
presentations, two specific to BDD and Cucumber. I also talked to
several CEOs and devs afterwards, and the overall takeaway I gathered
was a shift to less RSpec and more Cucumber. Some people even claimed
a 90/10 split (cukes/specs) on current projects.
This was surorising to me and not at all how I worked up to this
point. I was more 20/80. I usually cuked a feature, then spec'ed the
code to make the cuke work. Each release had some new features and
specs for all the underlying code. Apparently, the feeling is that you
should do all your main thrusts with Cucumber and use RSpec for edge
cases. The theory is that you can change out all the underlying code
and the cukes still pass.
What is the communities consensus on this?
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
cheers,
Matt
http://mattwynne.net
+447974 430184
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
George Dinwiddie
2010-04-21 17:01:05 UTC
Permalink
Ed,
Post by Ed Howland
Post by Matt Wynne
So for example in the canonical problem of roles with different access
rights to a system, you could write the nice readable tabulated test in
Cucumber, but instead of running that cuke against the whole stack, stepping
through the login form etc, you could just run it against whatever class is
responsible for managing roles.
In order for this to work, you'd need to write the Cukes in a declarative
style, avoiding too much detail about exactly how you'd carry out the step.
This allows the same step to be run against the whole stack or a focussed
piece of the system instead. I tend to think now that this is just good
practice anyway.
This sounds like a good idea, but I'm confused about how to implement
it. My lack of knowledge about Cucumber is showing through. Would you
have some different step_definition file and how do you tell Cucumber
to use a different step def file? Or do you put some sort of
conditional in each step definition that fires on some external flag
(environment variable)? Or is there someway of accessing tags passed
to Cucumber?
You can have as many step definition files as you want. The steps can
call any code you want.

- George
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Ed Howland
2010-04-21 21:29:24 UTC
Permalink
Yes, but if you have 2 step_definition files and they each have a
matching regex for a step, how do you distinguish them when running
cucumber?

Thanks,
Ed

Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland



On Wed, Apr 21, 2010 at 1:01 PM, George Dinwiddie
Ed,
Post by Ed Howland
Post by Matt Wynne
So for example in the canonical problem of roles with different access
rights to a system, you could write the nice readable tabulated test in
Cucumber, but instead of running that cuke against the whole stack, stepping
through the login form etc, you could just run it against whatever class is
responsible for managing roles.
In order for this to work, you'd need to write the Cukes in a declarative
style, avoiding too much detail about exactly how you'd carry out the step.
This allows the same step to be run against the whole stack or a focussed
piece of the system instead. I tend to think now that this is just good
practice anyway.
This sounds like a good idea, but I'm confused about how to implement
it. My lack of knowledge about Cucumber is showing through. Would you
have some different step_definition file and how do you tell Cucumber
to use a different step def file? Or do you put some sort of
conditional in each step definition that fires on some external flag
(environment variable)?  Or is there someway of accessing tags passed
to Cucumber?
You can have as many step definition files as you want.  The steps can call
any code you want.
 - George
--
 ----------------------------------------------------------------------
 * George Dinwiddie *                      http://blog.gdinwiddie.com
 Software Development                    http://www.idiacomputing.com
 Consultant and Coach                    http://www.agilemaryland.org
 ----------------------------------------------------------------------
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+***@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Phillip Koebbe
2010-04-21 21:49:25 UTC
Permalink
Post by Ed Howland
Yes, but if you have 2 step_definition files and they each have a
matching regex for a step, how do you distinguish them when running
cucumber?
Thanks,
Ed
You don't. You can have only one matching reg ex. The step definition
would need to make a decision about which path of execution to follow
and call the appropriate method.

Peace,
Phillip
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Gregory Hnatiuk
2010-04-21 21:56:41 UTC
Permalink
Post by Ed Howland
Post by Ed Howland
Yes, but if you have 2 step_definition files and they each have a
matching regex for a step, how do you distinguish them when running
cucumber?
I may be wrong, but I don't think Matt was suggesting a single step behaving
two different ways. Rather, I think the idea is that a step which is a
higher level description affords you more choices in how to implement it,
including skipping the UI entirely. If you use a declarative style in your
scenarios, you don't have to be tied down to clicking certain things or
filling in certain fields, for example.

I'm not sure exactly how having the same step behaving in two different ways
would come into play, at least for this discussion, as it seems easy enough
to just write two different steps.

Greg
Post by Ed Howland
Thanks,
Post by Ed Howland
Ed
You don't. You can have only one matching reg ex. The step definition would
need to make a decision about which path of execution to follow and call the
appropriate method.
Peace,
Phillip
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Matt Wynne
2010-04-22 07:32:19 UTC
Permalink
Post by Ed Howland
Yes, but if you have 2 step_definition files and they each have a
matching regex for a step, how do you distinguish them when running
cucumber?
I may be wrong, but I don't think Matt was suggesting a single step
behaving two different ways. Rather, I think the idea is that a
step which is a higher level description affords you more choices in
how to implement it, including skipping the UI entirely. If you use
a declarative style in your scenarios, you don't have to be tied
down to clicking certain things or filling in certain fields, for
example.
I'm not sure exactly how having the same step behaving in two
different ways would come into play, at least for this discussion,
as it seems easy enough to just write two different steps.
Greg
That's right Greg. Imagine one of my roles / authentication scenarios,
for the requirement that administrators are allowed to delete users'
accounts. There are two ways of writing that up:

Here's the 'imperative' style:

Given there is a User Bob
And I am logged in as an Admin
When I follow 'admin'
And I follow 'Users'
And I follow 'Bob'
And I press 'Delete'
Then the User "Bob" should not be in the database

And here's the declarative style:

Given I am an administrator
When I try to delete another user's account
Then I should be successful

Now writing the scenario in this declarative style not only gives me
the advantage that the scenario is more succinct and readable, but it
also affords an abstraction about how the *test* is implemented. I
could implement those steps through the UI, perhaps using the call-
steps-from-step-defs feature (I wish that had a catchier name!) to
call the same steps as in the imperative-style scenario, or I could
implement them to work directly against the classes that implement
this functionality.

We used the imperative style heavily at Songkick, so I'm not sure how
it would play out in a large test suite, but I can imagine that the
declarative style offers more opportunities for clashes between
features. Has anyone experienced this pain?
Post by Ed Howland
Thanks,
Ed
You don't. You can have only one matching reg ex. The step
definition would need to make a decision about which path of
execution to follow and call the appropriate method.
Peace,
Phillip
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en
.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en
.
cheers,
Matt

http://mattwynne.net
+447974 430184
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Ben Mabey
2010-04-23 03:24:31 UTC
Permalink
Post by Matt Wynne
On Wed, Apr 21, 2010 at 5:49 PM, Phillip Koebbe
Yes, but if you have 2 step_definition files and they each have a
matching regex for a step, how do you distinguish them when running
cucumber?
I may be wrong, but I don't think Matt was suggesting a single step
behaving two different ways. Rather, I think the idea is that a step
which is a higher level description affords you more choices in how
to implement it, including skipping the UI entirely. If you use a
declarative style in your scenarios, you don't have to be tied down
to clicking certain things or filling in certain fields, for example.
I'm not sure exactly how having the same step behaving in two
different ways would come into play, at least for this discussion, as
it seems easy enough to just write two different steps.
Greg
That's right Greg. Imagine one of my roles / authentication scenarios,
for the requirement that administrators are allowed to delete users'
Given there is a User Bob
And I am logged in as an Admin
When I follow 'admin'
And I follow 'Users'
And I follow 'Bob'
And I press 'Delete'
Then the User "Bob" should not be in the database
Given I am an administrator
When I try to delete another user's account
Then I should be successful
Now writing the scenario in this declarative style not only gives me
the advantage that the scenario is more succinct and readable, but it
also affords an abstraction about how the *test* is implemented. I
could implement those steps through the UI, perhaps using the
call-steps-from-step-defs feature (I wish that had a catchier name!)
to call the same steps as in the imperative-style scenario, or I could
implement them to work directly against the classes that implement
this functionality.
At my last job (over a year now) I used Cucumber against domain models
directly with a lot of success. In one particular situation I was
working with the project manager to give me examples of what kinds of
ads should be served based on various rules (associated with the ads)
and what we knew about a user. To get to the point of serving ads to a
person (via the workflow in the app) required a lot of clicking and
submitting. I thought it was just overkill and would make the features
needlessly slow. So, I set up a single happy path that went through the
entire process. I then used a Scenario Outline that hit the main domain
model directly. In the Scenario Outline is where I added all the
examples the customer and I came up with. There were a lot of examples
and we covered a lot of edge cases with them.

This approach worked quite well. The steps were very simple to create
and were highly declarative. It was basically: Given these ads and
rules, When a customer comes along with x attributes, then these adds
should be displayed. I could have easily switched out the domain model
step definitions to the ones that went through the entire process. That
still would have been a waste IMO though. I had a single happy path
scenario that guaranteed that my domain models were getting the info
they needed and that the appropriate ads were served up.

I would recommend this approach as it worked out quite nicely for us and
kept a lot of our scenarios as fast as our specs.

-Ben
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Pat Maddox
2010-04-21 22:42:14 UTC
Permalink
profiles

features/step_definitions/full_stack
features/step_definitions/domain_model

Then set up a full_stack and domain_model profile in cucumber.yml that load those step definition paths
Post by Ed Howland
Yes, but if you have 2 step_definition files and they each have a
matching regex for a step, how do you distinguish them when running
cucumber?
Thanks,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
On Wed, Apr 21, 2010 at 1:01 PM, George Dinwiddie
Ed,
Post by Ed Howland
Post by Matt Wynne
So for example in the canonical problem of roles with different access
rights to a system, you could write the nice readable tabulated test in
Cucumber, but instead of running that cuke against the whole stack, stepping
through the login form etc, you could just run it against whatever class is
responsible for managing roles.
In order for this to work, you'd need to write the Cukes in a declarative
style, avoiding too much detail about exactly how you'd carry out the step.
This allows the same step to be run against the whole stack or a focussed
piece of the system instead. I tend to think now that this is just good
practice anyway.
This sounds like a good idea, but I'm confused about how to implement
it. My lack of knowledge about Cucumber is showing through. Would you
have some different step_definition file and how do you tell Cucumber
to use a different step def file? Or do you put some sort of
conditional in each step definition that fires on some external flag
(environment variable)? Or is there someway of accessing tags passed
to Cucumber?
You can have as many step definition files as you want. The steps can call
any code you want.
- George
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Pat Maddox
2010-04-21 23:05:31 UTC
Permalink
Cucumber features are the best tool I know of for capturing requirements from my customer. RSpec specs are the best tool I know of for communicating intent and gauging code quality among the developer team.

I'm not sure how exactly you're quantifying a 90/10 or 80/20 split. I would expect that there would be a lot of overlap in coverage. That is, any given line of code is likely to have some cucumber and some rspec coverage. Personally I shoot for 100% RSpec coverage, and Cucumber is based entirely on what my customer wants. If we discuss a new feature or bug fix and they feel I know exactly what they're talking about and don't need a cucumber test for it, I don't write the cucumber test. Cukes are for teasing out requirements, which is most important when there's domain complexity that I don't understand, because I'm a programmer and not a domain expert. Every line of code I write gets RSpec coverage. That's how I personally feel confident that my code does what I think it does, and a
lso helps me keep my dependencies under control.

It's true that you can change out all the underlying code and cucumber tests still pass. But you should be able to change out a lot of code and have your specs still pass, as well. If you're changing the API then some specs might no longer be valid, or need to be moved, or whatever. That's just a part of refactoring. Although to be honest I think focused specs help me refactor more than other people, because I take really small steps when I refactor. When most people "refactor" they tear out a bunch of shit and see if it still works.

Cucumber tests tend to take longer because they're written at a much higher level. That requires more setup, and more steps through the codebase. For that reason, Cucumber isn't particularly good at giving me rapid feedback. I want feedback in 10 seconds rather than 10 minutes.

The best mantra I have for using Cucumber & RSpec in harmony is, "RSpec lets me know my code works right, Cucumber lets me know my code is doing the right work."

Pat
Post by Ed Howland
Please forgive the x-post.
I just got back from the Great Lakes Ruby Bash. They had several good
presentations, two specific to BDD and Cucumber. I also talked to
several CEOs and devs afterwards, and the overall takeaway I gathered
was a shift to less RSpec and more Cucumber. Some people even claimed
a 90/10 split (cukes/specs) on current projects.
This was surorising to me and not at all how I worked up to this
point. I was more 20/80. I usually cuked a feature, then spec'ed the
code to make the cuke work. Each release had some new features and
specs for all the underlying code. Apparently, the feeling is that you
should do all your main thrusts with Cucumber and use RSpec for edge
cases. The theory is that you can change out all the underlying code
and the cukes still pass.
What is the communities consensus on this?
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Mike Sassak
2010-04-22 05:26:50 UTC
Permalink
Cucumber features are the best tool I know of for capturing requirements from my customer.  RSpec specs are the best tool I know of for communicating intent and gauging code quality among the developer team.
I'm not sure how exactly you're quantifying a 90/10 or 80/20 split.  I would expect that there would be a lot of overlap in coverage.  That is, any given line of code is likely to have some cucumber and some rspec coverage.  Personally I shoot for 100% RSpec coverage, and Cucumber is based entirely on what my customer wants.  If we discuss a new feature or bug fix and they feel I know exactly what they're talking about and don't need a cucumber test for it, I don't write the cucumber test.  Cukes are for teasing out requirements, which is most important when there's domain complexity that I don't understand, because I'm a programmer and not a domain expert.  Every line of code I write gets RSpec coverage.  That's how I personally feel confident that my code does what I think it does, and also helps me keep my dependencies under control.
It's true that you can change out all the underlying code and cucumber tests still pass.  But you should be able to change out a lot of code and have your specs still pass, as well.  If you're changing the API then some specs might no longer be valid, or need to be moved, or whatever.  That's just a part of refactoring.  Although to be honest I think focused specs help me refactor more than other people, because I take really small steps when I refactor.  When most people "refactor" they tear out a bunch of shit and see if it still works.
Cucumber tests tend to take longer because they're written at a much higher level.  That requires more setup, and more steps through the codebase.  For that reason, Cucumber isn't particularly good at giving me rapid feedback.  I want feedback in 10 seconds rather than 10 minutes.
The best mantra I have for using Cucumber & RSpec in harmony is, "RSpec lets me know my code works right, Cucumber lets me know my code is doing the right work."
I don't think the percentage splits were meant quantitatively, but
rather as a way of saying that integration/full-stack tests with
Cucumber drastically reduce the need for RSpec or unit tests. I'm
skeptical of such claims myself, and not only because I think Cucumber
and RSpec are tools that target very different needs. I'm sure my own
limitations as a developer have a lot to do with feeling this way, but
leaning hard on integration tests reminds me of the Brian Kernighan
quote about debugging being twice as hard as writing the program in
the first place. Full-stack black box tests make it too easy for me to
write overly-complex code and still have a system that satisfies the
business rules. The idea of writing two sets of steps, one for
integration and one for the domain is very intriguing, and one that I
might try in the future, but I think that sidesteps the issue that
prompted Ed's post in the first place.

I think your mantra hits the nail on the head, but then I'm a sucker
for a good chiasmus. :-)

Mike
Pat
Post by Ed Howland
Please forgive the x-post.
I just got back from the Great Lakes Ruby Bash. They had several good
presentations, two specific to BDD and Cucumber. I also talked to
several CEOs and devs afterwards, and the overall takeaway I gathered
was a shift to less RSpec and more Cucumber. Some people even claimed
a 90/10 split (cukes/specs) on current projects.
This was surorising to me and not at all how I worked up to this
point. I was more 20/80. I usually cuked a feature, then spec'ed the
code to make the cuke work. Each release had some new features and
specs for all the underlying code. Apparently, the feeling is that you
should do all your main thrusts with Cucumber and use RSpec for edge
cases. The theory is that you can change out all the underlying code
and the cukes still pass.
What is the communities consensus on this?
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+***@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Ed Howland
2010-04-22 16:14:05 UTC
Permalink
I am in agreement with Mike and Pat here. As the folks I met at GLRB
explained to me, this was meant to increase productivity. As soon as
the Cucumber features pass, you were done. They might write some
corner case unit tests in RSpec. I think that is a little dangerous.
You'd have to be careful to write failure mode features. Those might
not be specified by the customer. And I agree that the API emerges
from the rapid feedback loop that RSpec gives you. I also take very
small steps when writing and also when refactoring. Changing the whole
API has never occurred with me, usually it's small changes to the UI.

I do however think that domain level and full stack tests are an
appropriate use for Cucumber. In my current code, it's just some gems.
I'd like to write some domain level stuff for them. But I can't figure
out how to do that.

Cucumber has tagged hooks, so I wonder ig that is a possibiliy.

Before("@domain") do
@domain_level=true
end

Before("@full_stack") do
@full_stack = true
end

Then in yout step definitions, do a test:
if @dimain
...
else
...
end

But this ugligies my code, IMO. Can you call World() within a Before
hook. That might be a way of pulling in one API over another. You;d
have to write your step defs in a higher level, luke not calling
Webrat directly, but calling "set_transfer(20.00)" that does a fill_in
for Webrat in one Workd() and a domain level set in some other
World().

module Domain
def set_transfer(field, amt)
@transfer.set(amt)
end
end

module FullStack
def set_transfer(field, amt)
fill_in field, amt
end
end

Before("@domain") do
World(Domain)
end
Before("@full_stack") do
World(FullStack)
end

Am I on the right track?

Thanks


Cheers,
Ed

Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
Post by Mike Sassak
Cucumber features are the best tool I know of for capturing requirements from my customer.  RSpec specs are the best tool I know of for communicating intent and gauging code quality among the developer team.
I'm not sure how exactly you're quantifying a 90/10 or 80/20 split.  I would expect that there would be a lot of overlap in coverage.  That is, any given line of code is likely to have some cucumber and some rspec coverage.  Personally I shoot for 100% RSpec coverage, and Cucumber is based entirely on what my customer wants.  If we discuss a new feature or bug fix and they feel I know exactly what they're talking about and don't need a cucumber test for it, I don't write the cucumber test.  Cukes are for teasing out requirements, which is most important when there's domain complexity that I don't understand, because I'm a programmer and not a domain expert.  Every line of code I write gets RSpec coverage.  That's how I personally feel confident that my code does what I think it does, and also helps me keep my dependencies under control.
It's true that you can change out all the underlying code and cucumber tests still pass.  But you should be able to change out a lot of code and have your specs still pass, as well.  If you're changing the API then some specs might no longer be valid, or need to be moved, or whatever.  That's just a part of refactoring.  Although to be honest I think focused specs help me refactor more than other people, because I take really small steps when I refactor.  When most people "refactor" they tear out a bunch of shit and see if it still works.
Cucumber tests tend to take longer because they're written at a much higher level.  That requires more setup, and more steps through the codebase.  For that reason, Cucumber isn't particularly good at giving me rapid feedback.  I want feedback in 10 seconds rather than 10 minutes.
The best mantra I have for using Cucumber & RSpec in harmony is, "RSpec lets me know my code works right, Cucumber lets me know my code is doing the right work."
I don't think the percentage splits were meant quantitatively, but
rather as a way of saying that integration/full-stack tests with
Cucumber drastically reduce the need for RSpec or unit tests. I'm
skeptical of such claims myself, and not only because I think Cucumber
and RSpec are tools that target very different needs. I'm sure my own
limitations as a developer have a lot to do with feeling this way, but
leaning hard on integration tests reminds me of the Brian Kernighan
quote about debugging being twice as hard as writing the program in
the first place. Full-stack black box tests make it too easy for me to
write overly-complex code and still have a system that satisfies the
business rules. The idea of writing two sets of steps, one for
integration and one for the domain is very intriguing, and one that I
might try in the future, but I think that sidesteps the issue that
prompted Ed's post in the first place.
I think your mantra hits the nail on the head, but then I'm a sucker
for a good chiasmus. :-)
Mike
Pat
Post by Ed Howland
Please forgive the x-post.
I just got back from the Great Lakes Ruby Bash. They had several good
presentations, two specific to BDD and Cucumber. I also talked to
several CEOs and devs afterwards, and the overall takeaway I gathered
was a shift to less RSpec and more Cucumber. Some people even claimed
a 90/10 split (cukes/specs) on current projects.
This was surorising to me and not at all how I worked up to this
point. I was more 20/80. I usually cuked a feature, then spec'ed the
code to make the cuke work. Each release had some new features and
specs for all the underlying code. Apparently, the feeling is that you
should do all your main thrusts with Cucumber and use RSpec for edge
cases. The theory is that you can change out all the underlying code
and the cukes still pass.
What is the communities consensus on this?
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+***@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Ed Howland
2010-04-22 16:25:05 UTC
Permalink
Sorry, just raad the posts about profiles in cucumber.yml. But the
RSpec book says you can set tags in the profile. I don't yet see how
to load a particular directory under step_definitions/full_stack and
not load step_definitions/domain

Thanks,

Cheers,
Ed

Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
Post by Ed Howland
I am in agreement with Mike and Pat here. As the folks I met at GLRB
explained to me, this was meant to increase productivity. As soon as
the Cucumber features pass, you were done. They might write some
corner case unit tests in RSpec. I think that is a little dangerous.
You'd have to be careful to write failure mode features. Those might
not be specified by the customer. And I agree that the API emerges
from the rapid feedback loop that RSpec gives you. I also take very
small steps when writing and also when refactoring. Changing the whole
API has never occurred with me, usually it's small changes to the UI.
I do however think that domain level and full stack tests are an
appropriate use for Cucumber. In my current code, it's just some gems.
I'd like to write some domain level stuff for them. But I can't figure
out how to do that.
Cucumber has tagged hooks, so I wonder ig that is a possibiliy.
end
end
   ...
 else
   ...
 end
But this ugligies my code, IMO. Can you call World() within a Before
hook. That might be a way of pulling in one API over another. You;d
have to write your step defs in a higher level, luke not calling
Webrat directly, but calling "set_transfer(20.00)" that does a fill_in
for Webrat in one Workd() and a domain level set in some other
World().
module Domain
 def set_transfer(field, amt)
 end
end
module FullStack
 def set_transfer(field, amt)
   fill_in field, amt
 end
end
 World(Domain)
end
 World(FullStack)
end
Am I on the right track?
Thanks
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
Post by Mike Sassak
Cucumber features are the best tool I know of for capturing requirements from my customer.  RSpec specs are the best tool I know of for communicating intent and gauging code quality among the developer team.
I'm not sure how exactly you're quantifying a 90/10 or 80/20 split.  I would expect that there would be a lot of overlap in coverage.  That is, any given line of code is likely to have some cucumber and some rspec coverage.  Personally I shoot for 100% RSpec coverage, and Cucumber is based entirely on what my customer wants.  If we discuss a new feature or bug fix and they feel I know exactly what they're talking about and don't need a cucumber test for it, I don't write the cucumber test.  Cukes are for teasing out requirements, which is most important when there's domain complexity that I don't understand, because I'm a programmer and not a domain expert.  Every line of code I write gets RSpec coverage.  That's how I personally feel confident that my code does what I think it does, and also helps me keep my dependencies under control.
It's true that you can change out all the underlying code and cucumber tests still pass.  But you should be able to change out a lot of code and have your specs still pass, as well.  If you're changing the API then some specs might no longer be valid, or need to be moved, or whatever.  That's just a part of refactoring.  Although to be honest I think focused specs help me refactor more than other people, because I take really small steps when I refactor.  When most people "refactor" they tear out a bunch of shit and see if it still works.
Cucumber tests tend to take longer because they're written at a much higher level.  That requires more setup, and more steps through the codebase.  For that reason, Cucumber isn't particularly good at giving me rapid feedback.  I want feedback in 10 seconds rather than 10 minutes.
The best mantra I have for using Cucumber & RSpec in harmony is, "RSpec lets me know my code works right, Cucumber lets me know my code is doing the right work."
I don't think the percentage splits were meant quantitatively, but
rather as a way of saying that integration/full-stack tests with
Cucumber drastically reduce the need for RSpec or unit tests. I'm
skeptical of such claims myself, and not only because I think Cucumber
and RSpec are tools that target very different needs. I'm sure my own
limitations as a developer have a lot to do with feeling this way, but
leaning hard on integration tests reminds me of the Brian Kernighan
quote about debugging being twice as hard as writing the program in
the first place. Full-stack black box tests make it too easy for me to
write overly-complex code and still have a system that satisfies the
business rules. The idea of writing two sets of steps, one for
integration and one for the domain is very intriguing, and one that I
might try in the future, but I think that sidesteps the issue that
prompted Ed's post in the first place.
I think your mantra hits the nail on the head, but then I'm a sucker
for a good chiasmus. :-)
Mike
Pat
Post by Ed Howland
Please forgive the x-post.
I just got back from the Great Lakes Ruby Bash. They had several good
presentations, two specific to BDD and Cucumber. I also talked to
several CEOs and devs afterwards, and the overall takeaway I gathered
was a shift to less RSpec and more Cucumber. Some people even claimed
a 90/10 split (cukes/specs) on current projects.
This was surorising to me and not at all how I worked up to this
point. I was more 20/80. I usually cuked a feature, then spec'ed the
code to make the cuke work. Each release had some new features and
specs for all the underlying code. Apparently, the feeling is that you
should do all your main thrusts with Cucumber and use RSpec for edge
cases. The theory is that you can change out all the underlying code
and the cukes still pass.
What is the communities consensus on this?
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+***@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Mike Sassak
2010-04-22 16:31:14 UTC
Permalink
Post by Ed Howland
Sorry, just raad the posts about profiles in cucumber.yml. But the
RSpec book says you can set tags in the profile. I don't yet see how
to load a particular directory under step_definitions/full_stack and
not load step_definitions/domain
See the section in cucumber --help about the -r flag. Using it
disables automatic loading, giving you get much more control over what
gets loaded when and from where.

Mike
Post by Ed Howland
Thanks,
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
Post by Ed Howland
I am in agreement with Mike and Pat here. As the folks I met at GLRB
explained to me, this was meant to increase productivity. As soon as
the Cucumber features pass, you were done. They might write some
corner case unit tests in RSpec. I think that is a little dangerous.
You'd have to be careful to write failure mode features. Those might
not be specified by the customer. And I agree that the API emerges
from the rapid feedback loop that RSpec gives you. I also take very
small steps when writing and also when refactoring. Changing the whole
API has never occurred with me, usually it's small changes to the UI.
I do however think that domain level and full stack tests are an
appropriate use for Cucumber. In my current code, it's just some gems.
I'd like to write some domain level stuff for them. But I can't figure
out how to do that.
Cucumber has tagged hooks, so I wonder ig that is a possibiliy.
end
end
   ...
 else
   ...
 end
But this ugligies my code, IMO. Can you call World() within a Before
hook. That might be a way of pulling in one API over another. You;d
have to write your step defs in a higher level, luke not calling
Webrat directly, but calling "set_transfer(20.00)" that does a fill_in
for Webrat in one Workd() and a domain level set in some other
World().
module Domain
 def set_transfer(field, amt)
 end
end
module FullStack
 def set_transfer(field, amt)
   fill_in field, amt
 end
end
 World(Domain)
end
 World(FullStack)
end
Am I on the right track?
Thanks
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
Post by Mike Sassak
Cucumber features are the best tool I know of for capturing requirements from my customer.  RSpec specs are the best tool I know of for communicating intent and gauging code quality among the developer team.
I'm not sure how exactly you're quantifying a 90/10 or 80/20 split.  I would expect that there would be a lot of overlap in coverage.  That is, any given line of code is likely to have some cucumber and some rspec coverage.  Personally I shoot for 100% RSpec coverage, and Cucumber is based entirely on what my customer wants.  If we discuss a new feature or bug fix and they feel I know exactly what they're talking about and don't need a cucumber test for it, I don't write the cucumber test.  Cukes are for teasing out requirements, which is most important when there's domain complexity that I don't understand, because I'm a programmer and not a domain expert.  Every line of code I write gets RSpec coverage.  That's how I personally feel confident that my code does what I think it does, and also helps me keep my dependencies under control.
It's true that you can change out all the underlying code and cucumber tests still pass.  But you should be able to change out a lot of code and have your specs still pass, as well.  If you're changing the API then some specs might no longer be valid, or need to be moved, or whatever.  That's just a part of refactoring.  Although to be honest I think focused specs help me refactor more than other people, because I take really small steps when I refactor.  When most people "refactor" they tear out a bunch of shit and see if it still works.
Cucumber tests tend to take longer because they're written at a much higher level.  That requires more setup, and more steps through the codebase.  For that reason, Cucumber isn't particularly good at giving me rapid feedback.  I want feedback in 10 seconds rather than 10 minutes.
The best mantra I have for using Cucumber & RSpec in harmony is, "RSpec lets me know my code works right, Cucumber lets me know my code is doing the right work."
I don't think the percentage splits were meant quantitatively, but
rather as a way of saying that integration/full-stack tests with
Cucumber drastically reduce the need for RSpec or unit tests. I'm
skeptical of such claims myself, and not only because I think Cucumber
and RSpec are tools that target very different needs. I'm sure my own
limitations as a developer have a lot to do with feeling this way, but
leaning hard on integration tests reminds me of the Brian Kernighan
quote about debugging being twice as hard as writing the program in
the first place. Full-stack black box tests make it too easy for me to
write overly-complex code and still have a system that satisfies the
business rules. The idea of writing two sets of steps, one for
integration and one for the domain is very intriguing, and one that I
might try in the future, but I think that sidesteps the issue that
prompted Ed's post in the first place.
I think your mantra hits the nail on the head, but then I'm a sucker
for a good chiasmus. :-)
Mike
Pat
Post by Ed Howland
Please forgive the x-post.
I just got back from the Great Lakes Ruby Bash. They had several good
presentations, two specific to BDD and Cucumber. I also talked to
several CEOs and devs afterwards, and the overall takeaway I gathered
was a shift to less RSpec and more Cucumber. Some people even claimed
a 90/10 split (cukes/specs) on current projects.
This was surorising to me and not at all how I worked up to this
point. I was more 20/80. I usually cuked a feature, then spec'ed the
code to make the cuke work. Each release had some new features and
specs for all the underlying code. Apparently, the feeling is that you
should do all your main thrusts with Cucumber and use RSpec for edge
cases. The theory is that you can change out all the underlying code
and the cukes still pass.
What is the communities consensus on this?
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+***@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Zach Moazeni
2010-04-22 16:57:24 UTC
Permalink
I'll jump in here as I was one of the guys who presented a shift in my testing strategies at the Great Lakes Ruby Bash.

To give some context, I've built projects that were very focused on isolation tests that used Rspec and Mocks to assert behavior. I do think there is merit in using BDD as a code-design tool, however I have also maintained large test suites that had more historical context than regression value. I consider this inside-out testing. Developing/Testing at the unit level and then bubbling out to an Acceptance test.

I've been subtly migrating to what I consider outside-in testing, starting with Acceptance tests and then moving to a unit level if necessary. And as a result I've felt much better about the codebases (including the tests). Although I was getting a bit uneasy since I haven't heard of other developers experiencing similar pain points. Talking with a few developers at the Great Lakes Ruby Bash was really refreshing because not only did it sound like they had similar pains, but they were going down the same path I was.

I was one of the guys who did a lightning talk on a library I'm working on (Harvested, a Ruby API wrapper for Harvest http://github.com/zmoazeni/harvested). And I do have a 90/10 split in Acceptance vs Unit with I'm cool with. I wasn't throwing out those numbers as what you should shoot for, but just anecdotal experience.

I recognize that I may be criticized for writing untested code or that I'm disagreeing with BDD/TDD. I'm don't think I am. I feel I've gone way too far on the side of testing, and after reflecting on my experiences with past projects feeling the pendulum swinging back towards a healthy balance.

Some specific opinions I have are: I don't test Controllers or Views or Rails-DSL validations (e.g. validates_presence_of, validates_uniqueness_of). I do test "interesting validations", and instead of testing "interesting controller actions" I prefer to refactor that so the controller is very minimalist. The same goes for views, I pull away anything that looks "interesting" in a view. This is totally subjective, so I don't have a great way to clarify what I mean by "interesting".

I don't disagree with mocks, but I would rather have a "boxed functional test" (sorry running out of language here) that test how multiple objects behave together and use mocks/stubs to form the boundaries of the tests.

Hopefully this doesn't come across as another post of "I don't see the point of testing", or "Mocks are silly". Additionally, I'm a bit nervous posting these thoughts to an audience that are most likely going to disagree with this strategy. So I would welcome any support along with critiques on these ideas.

--
Zach Moazeni
http://simplechatter.com
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+***@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
steve ross
2010-04-22 18:28:16 UTC
Permalink
Post by Zach Moazeni
I'll jump in here as I was one of the guys who presented a shift in my testing strategies at the Great Lakes Ruby Bash.
To give some context, I've built projects that were very focused on isolation tests that used Rspec and Mocks to assert behavior. I do think there is merit in using BDD as a code-design tool, however I have also maintained large test suites that had more historical context than regression value. I consider this inside-out testing. Developing/Testing at the unit level and then bubbling out to an Acceptance test.
I've been subtly migrating to what I consider outside-in testing, starting with Acceptance tests and then moving to a unit level if necessary. And as a result I've felt much better about the codebases (including the tests). Although I was getting a bit uneasy since I haven't heard of other developers experiencing similar pain points. Talking with a few developers at the Great Lakes Ruby Bash was really refreshing because not only did it sound like they had similar pains, but they were going down the same path I was.
I was one of the guys who did a lightning talk on a library I'm working on (Harvested, a Ruby API wrapper for Harvest http://github.com/zmoazeni/harvested). And I do have a 90/10 split in Acceptance vs Unit with I'm cool with. I wasn't throwing out those numbers as what you should shoot for, but just anecdotal experience.
I recognize that I may be criticized for writing untested code or that I'm disagreeing with BDD/TDD. I'm don't think I am. I feel I've gone way too far on the side of testing, and after reflecting on my experiences with past projects feeling the pendulum swinging back towards a healthy balance.
Some specific opinions I have are: I don't test Controllers or Views or Rails-DSL validations (e.g. validates_presence_of, validates_uniqueness_of). I do test "interesting validations", and instead of testing "interesting controller actions" I prefer to refactor that so the controller is very minimalist. The same goes for views, I pull away anything that looks "interesting" in a view. This is totally subjective, so I don't have a great way to clarify what I mean by "interesting".
I don't disagree with mocks, but I would rather have a "boxed functional test" (sorry running out of language here) that test how multiple objects behave together and use mocks/stubs to form the boundaries of the tests.
Hopefully this doesn't come across as another post of "I don't see the point of testing", or "Mocks are silly". Additionally, I'm a bit nervous posting these thoughts to an audience that are most likely going to disagree with this strategy. So I would welcome any support along with critiques on these ideas.
--
Zach Moazeni
http://simplechatter.com
Specs (i.e., unit-level testing) will give you -- and Pat touched on some of it -- a rapid feedback loop with an almost direct pointer to what part of your code isn't working. When you are testing at the Cuke level and there are object interactions, you are multiplying the complexity of what's under test. That means you are decreasing the simplicity of going right to the errant line(s) of code.

It's all, of course, dependent on your needs for a "safety net" to reassure you that your code means what it says and says what it means.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+***@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Pat Maddox
2010-04-23 04:40:54 UTC
Permalink
Post by Zach Moazeni
I'll jump in here as I was one of the guys who presented a shift in my testing strategies at the Great Lakes Ruby Bash.
To give some context, I've built projects that were very focused on isolation tests that used Rspec and Mocks to assert behavior. I do think there is merit in using BDD as a code-design tool, however I have also maintained large test suites that had more historical context than regression value. I consider this inside-out testing. Developing/Testing at the unit level and then bubbling out to an Acceptance test.
I've been subtly migrating to what I consider outside-in testing, starting with Acceptance tests and then moving to a unit level if necessary. And as a result I've felt much better about the codebases (including the tests). Although I was getting a bit uneasy since I haven't heard of other developers experiencing similar pain points. Talking with a few developers at the Great Lakes Ruby Bash was really refreshing because not only did it sound like they had similar pains, but they were going down the same path I was.
I was one of the guys who did a lightning talk on a library I'm working on (Harvested, a Ruby API wrapper for Harvest http://github.com/zmoazeni/harvested). And I do have a 90/10 split in Acceptance vs Unit with I'm cool with. I wasn't throwing out those numbers as what you should shoot for, but just anecdotal experience.
I recognize that I may be criticized for writing untested code or that I'm disagreeing with BDD/TDD. I'm don't think I am. I feel I've gone way too far on the side of testing, and after reflecting on my experiences with past projects feeling the pendulum swinging back towards a healthy balance.
Some specific opinions I have are: I don't test Controllers or Views or Rails-DSL validations (e.g. validates_presence_of, validates_uniqueness_of). I do test "interesting validations", and instead of testing "interesting controller actions" I prefer to refactor that so the controller is very minimalist. The same goes for views, I pull away anything that looks "interesting" in a view. This is totally subjective, so I don't have a great way to clarify what I mean by "interesting".
I don't disagree with mocks, but I would rather have a "boxed functional test" (sorry running out of language here) that test how multiple objects behave together and use mocks/stubs to form the boundaries of the tests.
Hopefully this doesn't come across as another post of "I don't see the point of testing", or "Mocks are silly". Additionally, I'm a bit nervous posting these thoughts to an audience that are most likely going to disagree with this strategy. So I would welcome any support along with critiques on these ideas.
I know several people who have reported success using a similar approach.

Pat
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+***@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Zach Dennis
2010-04-23 14:20:55 UTC
Permalink
Post by Zach Moazeni
I'll jump in here as I was one of the guys who presented a shift in my
testing strategies at the Great Lakes Ruby Bash.
To give some context, I've built projects that were very focused on
isolation tests that used Rspec and Mocks to assert behavior. I do think
there is merit in using BDD as a code-design tool, however I have also
maintained large test suites that had more historical context than
regression value. I consider this inside-out testing. Developing/Testing at
the unit level and then bubbling out to an Acceptance test.
I've been subtly migrating to what I consider outside-in testing, starting
with Acceptance tests and then moving to a unit level if necessary. And as a
result I've felt much better about the codebases (including the tests).
Although I was getting a bit uneasy since I haven't heard of other
developers experiencing similar pain points. Talking with a few developers
at the Great Lakes Ruby Bash was really refreshing because not only did it
sound like they had similar pains, but they were going down the same path I
was.
I was one of the guys who did a lightning talk on a library I'm working on
(Harvested, a Ruby API wrapper for Harvest
http://github.com/zmoazeni/harvested). And I do have a 90/10 split in
Acceptance vs Unit with I'm cool with. I wasn't throwing out those numbers
as what you should shoot for, but just anecdotal experience.
I recognize that I may be criticized for writing untested code or that I'm
disagreeing with BDD/TDD. I'm don't think I am. I feel I've gone way too far
on the side of testing, and after reflecting on my experiences with past
projects feeling the pendulum swinging back towards a healthy balance.
As you know per our discussion after GLRB, I've shared a very similar
experience. I started with no testing, then got bit by the test bug, and
then went way too far on the side of trying to do all TDD, before I had the
experience or knowledge to make good decisions on how to do it, when to do
it, etc. As I mentioned in my other post on this thread (the really long
one), if a unit level example isn't providing value beyond what you are
driving with Cucumber, then I don't see any problems with what you're doing.
You're the best judge of that with the code you're working with so you know
where the value is.
Post by Zach Moazeni
Some specific opinions I have are: I don't test Controllers or Views or
Rails-DSL validations (e.g. validates_presence_of, validates_uniqueness_of).
I do test "interesting validations", and instead of testing "interesting
controller actions" I prefer to refactor that so the controller is very
minimalist. The same goes for views, I pull away anything that looks
"interesting" in a view. This is totally subjective, so I don't have a great
way to clarify what I mean by "interesting".
I don't disagree with mocks, but I would rather have a "boxed functional
test" (sorry running out of language here) that test how multiple objects
behave together and use mocks/stubs to form the boundaries of the tests.
I like mocks in certain situations. I also like them when writing examples
against other systems/services that I do not have access to run tests
against. If the service is available I like to have integration example(s)
in place either using Cucumber or a even a tool like RSpec. Based on my code
I may or may not have additional unit level examples, it depends on if
they're adding an additional level of value.
Post by Zach Moazeni
Hopefully this doesn't come across as another post of "I don't see the
point of testing", or "Mocks are silly". Additionally, I'm a bit nervous
posting these thoughts to an audience that are most likely going to disagree
with this strategy. So I would welcome any support along with critiques on
these ideas.
Don't be nervous. There is no one right way. We're all continuously learning
and pushing the boundaries of how to efficiently write working software that
is maintainable. Sharing success stories and pain points of different
approaches and workflows helps the community grow and forces everyone to
think.

I agree with a lot of what you said, you didn't say "unit level examples are
stupid". You said that you reached a point to where you were writing
examples that cost more than they were worth and you adapted your approach,
and you shared a concrete example of using it successfully. I applaud this,
and it seems very in line with the 2nd principle of BDD.
--
Zach Dennis
http://www.continuousthinking.com (personal)
http://www.mutuallyhuman.com (hire me)
http://ideafoundry.info/behavior-driven-development (first rate BDD
training)
@zachdennis (twitter)
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Ed Howland
2010-04-23 20:31:49 UTC
Permalink
I am curious about "boxed functional tests". Can you expand on that further?

Overall, I've learned from this thread to be more disciplined about
driving outside-in through Cucumber and then down to RSpec. I had
started doing that approach already. I have gotten a better feel for
the balance.

My pain points have been writing step definitions. Cucumber encourages
reusable step defs and I need to get batter about packaging them for
use across projects.

Thanks all for your input, it was very valuable. Hope to meet some
(more) of you at upcoming conferences. (I am the guy with the
walker/cane)

I might post about my experience with writing a generic steps and
using a domain set of step defs and a full stack set. Check my
tweetfeed.

Cheers,
Ed

Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
Post by Zach Dennis
Post by Zach Moazeni
I'll jump in here as I was one of the guys who presented a shift in my
testing strategies at the Great Lakes Ruby Bash.
To give some context, I've built projects that were very focused on
isolation tests that used Rspec and Mocks to assert behavior. I do think
there is merit in using BDD as a code-design tool, however I have also
maintained large test suites that had more historical context than
regression value. I consider this inside-out testing. Developing/Testing at
the unit level and then bubbling out to an Acceptance test.
I've been subtly migrating to what I consider outside-in testing, starting
with Acceptance tests and then moving to a unit level if necessary. And as a
result I've felt much better about the codebases (including the tests).
Although I was getting a bit uneasy since I haven't heard of other
developers experiencing similar pain points. Talking with a few developers
at the Great Lakes Ruby Bash was really refreshing because not only did it
sound like they had similar pains, but they were going down the same path I
was.
I was one of the guys who did a lightning talk on a library I'm working on
(Harvested, a Ruby API wrapper for Harvest
http://github.com/zmoazeni/harvested). And I do have a 90/10 split in
Acceptance vs Unit with I'm cool with. I wasn't throwing out those numbers
as what you should shoot for, but just anecdotal experience.
I recognize that I may be criticized for writing untested code or that I'm
disagreeing with BDD/TDD. I'm don't think I am. I feel I've gone way too far
on the side of testing, and after reflecting on my experiences with past
projects feeling the pendulum swinging back towards a healthy balance.
As you know per our discussion after GLRB, I've shared a very similar
experience. I started with no testing, then got bit by the test bug, and
then went way too far on the side of trying to do all TDD, before I had the
experience or knowledge to make good decisions on how to do it, when to do
it, etc.  As I mentioned in my other post on this thread (the really long
one), if a unit level example isn't providing value beyond what you are
driving with Cucumber, then I don't see any problems with what you're doing.
You're the best judge of that with the code you're working with so you know
where the value is.
Post by Zach Moazeni
Some specific opinions I have are: I don't test Controllers or Views or
Rails-DSL validations (e.g. validates_presence_of, validates_uniqueness_of).
I do test "interesting validations", and instead of testing "interesting
controller actions" I prefer to refactor that so the controller is very
minimalist. The same goes for views, I pull away anything that looks
"interesting" in a view. This is totally subjective, so I don't have a great
way to clarify what I mean by "interesting".
I don't disagree with mocks, but I would rather have a "boxed functional
test" (sorry running out of language here) that test how multiple objects
behave together and use mocks/stubs to form the boundaries of the tests.
I like mocks in certain situations. I also like them when writing examples
against other systems/services that I do not have access to run tests
against. If the service is available I like to have  integration example(s)
in place either using Cucumber or a even a tool like RSpec. Based on my code
I may or may not have additional unit level examples, it depends on if
they're adding an additional level of value.
Post by Zach Moazeni
Hopefully this doesn't come across as another post of "I don't see the
point of testing", or "Mocks are silly". Additionally, I'm a bit nervous
posting these thoughts to an audience that are most likely going to disagree
with this strategy. So I would welcome any support along with critiques on
these ideas.
Don't be nervous. There is no one right way. We're all continuously learning
and pushing the boundaries of how to efficiently write working software that
is maintainable. Sharing success stories and pain points of different
approaches and workflows helps the community grow and forces everyone to
think.
I agree with a lot of what you said, you didn't say "unit level examples are
stupid".  You said that you reached a point to where you were writing
examples that cost more than they were worth and you adapted your approach,
and you shared a concrete example of using it successfully. I applaud this,
and it seems very in line with the 2nd principle of BDD.
--
Zach Dennis
http://www.continuousthinking.com (personal)
http://www.mutuallyhuman.com (hire me)
http://ideafoundry.info/behavior-driven-development (first rate BDD
training)
@zachdennis (twitter)
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+***@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Zach Moazeni
2010-04-23 22:50:30 UTC
Permalink
Post by Ed Howland
I am curious about "boxed functional tests". Can you expand on that further?
For instance, if I have 3 objects that work with each other, I used to test each in isolation (using mocks) which would then have overlapping coverage in an acceptance test. Now I will start with the top level acceptance, and if the composition of the 3 objects are "interesting" enough, I will have a different series of tests that stress edge cases on the 3 objects working together (treating them like a composition). Often, compositions like those are going to rely on other objects, so I'll use mocks/stubs to form a boundary around them and not worry about everything else in the system.

You could use Cucumber for those composition tests, nothing says Cucumber has to only be used for top level acceptance tests. Or you could use other unit testing frameworks if that makes more sense.

The key point for me now is that I don't _start_ with testing objects in isolation.

--
Zach Moazeni
http://simplechatter.com
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+***@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Zach Dennis
2010-04-22 19:32:12 UTC
Permalink
Post by Ed Howland
I am in agreement with Mike and Pat here. As the folks I met at GLRB
explained to me, this was meant to increase productivity.
Writing things that are unnecessary increase productivity. To that extent I
agree with that statement. I like to minimize writing pointless examples
that drive nothing, and increase the cost of maintenance over time.
Post by Ed Howland
As soon as
the Cucumber features pass, you were done.
This is specifically something I mentioned @ GLRB that I did not agree
with. Cucumber features help identify when the feature is working, but for
it to be done you need to ensure you leave the code in a well-factored
state. Which means if something spiked out an implementation and found it
made the feature pass, then they can stop their spike, and being the real
implementation driving it with examples as necessary. At least that's how I
roll.
Post by Ed Howland
They might write some
corner case unit tests in RSpec. I think that is a little dangerous.
You'd have to be careful to write failure mode features. Those might
not be specified by the customer.
And I agree that the API emerges
Post by Ed Howland
from the rapid feedback loop that RSpec gives you. I also take very
small steps when writing and also when refactoring. Changing the whole
API has never occurred with me, usually it's small changes to the UI.
Small steps is often relative. How big are your feet? :)

Given Joseph's example in his reply on this thread about driving from from
Edinburgh to London (or vise-versa). I like his analogy, however, if I am
walking from my front-door to my mailbox, the context has changed and what I
need to do to prepare for that trip and make sure I successfully make it to
my mailbox is quite different then traveling cross-country.
Post by Ed Howland
I do however think that domain level and full stack tests are an
appropriate use for Cucumber. In my current code, it's just some gems.
I'd like to write some domain level stuff for them. But I can't figure
out how to do that.
I like to keep features that are full-stack very very separate from features
that are not. Since these examples serve different purposes I like having a
clear separation between the two and their step definitions. I have not done
this on any production sites just playground practice.

Zach
Post by Ed Howland
Cucumber has tagged hooks, so I wonder ig that is a possibiliy.
@domain_level=true
end
@full_stack = true
end
...
else
...
end
But this ugligies my code, IMO. Can you call World() within a Before
hook. That might be a way of pulling in one API over another. You;d
have to write your step defs in a higher level, luke not calling
Webrat directly, but calling "set_transfer(20.00)" that does a fill_in
for Webrat in one Workd() and a domain level set in some other
World().
module Domain
def set_transfer(field, amt)
@transfer.set(amt)
end
end
module FullStack
def set_transfer(field, amt)
fill_in field, amt
end
end
World(Domain)
end
World(FullStack)
end
Am I on the right track?
Thanks
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
Post by Mike Sassak
Post by Pat Maddox
Cucumber features are the best tool I know of for capturing requirements
from my customer. RSpec specs are the best tool I know of for communicating
intent and gauging code quality among the developer team.
Post by Mike Sassak
Post by Pat Maddox
I'm not sure how exactly you're quantifying a 90/10 or 80/20 split. I
would expect that there would be a lot of overlap in coverage. That is, any
given line of code is likely to have some cucumber and some rspec coverage.
Personally I shoot for 100% RSpec coverage, and Cucumber is based entirely
on what my customer wants. If we discuss a new feature or bug fix and they
feel I know exactly what they're talking about and don't need a cucumber
test for it, I don't write the cucumber test. Cukes are for teasing out
requirements, which is most important when there's domain complexity that I
don't understand, because I'm a programmer and not a domain expert. Every
line of code I write gets RSpec coverage. That's how I personally feel
confident that my code does what I think it does, and also helps me keep my
dependencies under control.
Post by Mike Sassak
Post by Pat Maddox
It's true that you can change out all the underlying code and cucumber
tests still pass. But you should be able to change out a lot of code and
have your specs still pass, as well. If you're changing the API then some
specs might no longer be valid, or need to be moved, or whatever. That's
just a part of refactoring. Although to be honest I think focused specs
help me refactor more than other people, because I take really small steps
when I refactor. When most people "refactor" they tear out a bunch of shit
and see if it still works.
Post by Mike Sassak
Post by Pat Maddox
Cucumber tests tend to take longer because they're written at a much
higher level. That requires more setup, and more steps through the
codebase. For that reason, Cucumber isn't particularly good at giving me
rapid feedback. I want feedback in 10 seconds rather than 10 minutes.
Post by Mike Sassak
Post by Pat Maddox
The best mantra I have for using Cucumber & RSpec in harmony is, "RSpec
lets me know my code works right, Cucumber lets me know my code is doing the
right work."
Post by Mike Sassak
I don't think the percentage splits were meant quantitatively, but
rather as a way of saying that integration/full-stack tests with
Cucumber drastically reduce the need for RSpec or unit tests. I'm
skeptical of such claims myself, and not only because I think Cucumber
and RSpec are tools that target very different needs. I'm sure my own
limitations as a developer have a lot to do with feeling this way, but
leaning hard on integration tests reminds me of the Brian Kernighan
quote about debugging being twice as hard as writing the program in
the first place. Full-stack black box tests make it too easy for me to
write overly-complex code and still have a system that satisfies the
business rules. The idea of writing two sets of steps, one for
integration and one for the domain is very intriguing, and one that I
might try in the future, but I think that sidesteps the issue that
prompted Ed's post in the first place.
I think your mantra hits the nail on the head, but then I'm a sucker
for a good chiasmus. :-)
Mike
Post by Pat Maddox
Pat
Post by Ed Howland
Please forgive the x-post.
I just got back from the Great Lakes Ruby Bash. They had several good
presentations, two specific to BDD and Cucumber. I also talked to
several CEOs and devs afterwards, and the overall takeaway I gathered
was a shift to less RSpec and more Cucumber. Some people even claimed
a 90/10 split (cukes/specs) on current projects.
This was surorising to me and not at all how I worked up to this
point. I was more 20/80. I usually cuked a feature, then spec'ed the
code to make the cuke work. Each release had some new features and
specs for all the underlying code. Apparently, the feeling is that you
should do all your main thrusts with Cucumber and use RSpec for edge
cases. The theory is that you can change out all the underlying code
and the cukes still pass.
What is the communities consensus on this?
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
--
You received this message because you are subscribed to the Google
Groups "Cukes" group.
Post by Mike Sassak
Post by Pat Maddox
Post by Ed Howland
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
Post by Mike Sassak
Post by Pat Maddox
--
You received this message because you are subscribed to the Google
Groups "Cukes" group.
Post by Mike Sassak
Post by Pat Maddox
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
Post by Mike Sassak
--
You received this message because you are subscribed to the Google Groups
"Cukes" group.
Post by Mike Sassak
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
--
Zach Dennis
http://www.continuousthinking.com (personal)
http://www.mutuallyhuman.com (hire me)
http://ideafoundry.info/behavior-driven-development (first rate BDD
training)
@zachdennis (twitter)
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Zach Dennis
2010-04-22 19:45:07 UTC
Permalink
Post by Zach Dennis
Post by Ed Howland
I am in agreement with Mike and Pat here. As the folks I met at GLRB
explained to me, this was meant to increase productivity.
Writing things that are unnecessary increase productivity. To that extent I
agree with that statement. I like to minimize writing pointless examples
that drive nothing, and increase the cost of maintenance over time.
Bah typo...

*Not * writing things that are unnecessary increase productivity.....
<continue next sentence>
--
Zach Dennis
http://www.continuousthinking.com (personal)
http://www.mutuallyhuman.com (hire me)
http://ideafoundry.info/behavior-driven-development (first rate BDD
training)
@zachdennis (twitter)
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Pat Maddox
2010-04-27 18:14:28 UTC
Permalink
My other contribution to this thread is that if your code is tested primarily via Cucumber, you probably have SRP violations all over the place. That's what I've noticed in my own code, anyway.

The tricky thing is that SRP violations are not necessarily crippling, particularly in Rails apps (where SRP violations are pretty much built-in). They just make your test suite a bit slower, limit reuse, etc... things that are valuable to us but typically not as valuable as just shipping.

Pat
Post by Ed Howland
Post by Pat Maddox
Cucumber features are the best tool I know of for capturing requirements from my customer. RSpec specs are the best tool I know of for communicating intent and gauging code quality among the developer team.
I'm not sure how exactly you're quantifying a 90/10 or 80/20 split. I would expect that there would be a lot of overlap in coverage. That is, any given line of code is likely to have some cucumber and some rspec coverage. Personally I shoot for 100% RSpec coverage, and Cucumber is based entirely on what my customer wants. If we discuss a new feature or bug fix and they feel I know exactly what they're talking about and don't need a cucumber test for it, I don't write the cucumber test. Cukes are for teasing out requirements, which is most important when there's domain complexity that I don't understand, because I'm a programmer and not a domain expert. Every line of code I write gets RSpec coverage. That's how I personally feel confident that my code does what I think it does, an
d also helps me keep my dependencies under control.
Post by Ed Howland
Post by Pat Maddox
It's true that you can change out all the underlying code and cucumber tests still pass. But you should be able to change out a lot of code and have your specs still pass, as well. If you're changing the API then some specs might no longer be valid, or need to be moved, or whatever. That's just a part of refactoring. Although to be honest I think focused specs help me refactor more than other people, because I take really small steps when I refactor. When most people "refactor" they tear out a bunch of shit and see if it still works.
Cucumber tests tend to take longer because they're written at a much higher level. That requires more setup, and more steps through the codebase. For that reason, Cucumber isn't particularly good at giving me rapid feedback. I want feedback in 10 seconds rather than 10 minutes.
The best mantra I have for using Cucumber & RSpec in harmony is, "RSpec lets me know my code works right, Cucumber lets me know my code is doing the right work."
Post by Ed Howland
Please forgive the x-post.
I just got back from the Great Lakes Ruby Bash. They had several good
presentations, two specific to BDD and Cucumber. I also talked to
several CEOs and devs afterwards, and the overall takeaway I gathered
was a shift to less RSpec and more Cucumber. Some people even claimed
a 90/10 split (cukes/specs) on current projects.
This was surorising to me and not at all how I worked up to this
point. I was more 20/80. I usually cuked a feature, then spec'ed the
code to make the cuke work. Each release had some new features and
specs for all the underlying code. Apparently, the feeling is that you
should do all your main thrusts with Cucumber and use RSpec for edge
cases. The theory is that you can change out all the underlying code
and the cukes still pass.
What is the communities consensus on this?
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Curtis Schofield
2010-04-27 18:38:54 UTC
Permalink
Personally SRP violations are making my work hell right now on a 2 year old
rails system. My first concern when re-factoring is putting test in place -
there isn't much coverage when i started.

I've been doing that for the past 2 weeks and I've found that cucumber is
doing well for getting higher level coverage of 'it works' or 'it needs to
work like this' - but even so - the real guts of the problem is far and away
from Cucumber and so i've been using Cucumber to assist my design by
covering basic behaviors and then writing tests for the places where new
things must live and then moving them in while keeping the cukes functioning
an extracting responsibilities as the rspec tests require.

It's quite a job - now testing the views is another thing entirely - cuke
sorta works for this - but not so much - since i'm in a codebase that has
degenerated to rhtml bog in many ways - it's a bit of a trick and i have no
really good way to ensure coverage or confidence.

Great thread.
Post by Pat Maddox
My other contribution to this thread is that if your code is tested
primarily via Cucumber, you probably have SRP violations all over the place.
That's what I've noticed in my own code, anyway.
The tricky thing is that SRP violations are not necessarily crippling,
particularly in Rails apps (where SRP violations are pretty much built-in).
They just make your test suite a bit slower, limit reuse, etc... things
that are valuable to us but typically not as valuable as just shipping.
Pat
Post by Ed Howland
Post by Pat Maddox
Cucumber features are the best tool I know of for capturing requirements
from my customer. RSpec specs are the best tool I know of for communicating
intent and gauging code quality among the developer team.
Post by Ed Howland
Post by Pat Maddox
I'm not sure how exactly you're quantifying a 90/10 or 80/20 split. I
would expect that there would be a lot of overlap in coverage. That is, any
given line of code is likely to have some cucumber and some rspec coverage.
Personally I shoot for 100% RSpec coverage, and Cucumber is based entirely
on what my customer wants. If we discuss a new feature or bug fix and they
feel I know exactly what they're talking about and don't need a cucumber
test for it, I don't write the cucumber test. Cukes are for teasing out
requirements, which is most important when there's domain complexity that I
don't understand, because I'm a programmer and not a domain expert. Every
line of code I write gets RSpec coverage. That's how I personally feel
confident that my code does what I think it does, and also helps me keep my
dependencies under control.
Post by Ed Howland
Post by Pat Maddox
It's true that you can change out all the underlying code and cucumber
tests still pass. But you should be able to change out a lot of code and
have your specs still pass, as well. If you're changing the API then some
specs might no longer be valid, or need to be moved, or whatever. That's
just a part of refactoring. Although to be honest I think focused specs
help me refactor more than other people, because I take really small steps
when I refactor. When most people "refactor" they tear out a bunch of shit
and see if it still works.
Post by Ed Howland
Post by Pat Maddox
Cucumber tests tend to take longer because they're written at a much
higher level. That requires more setup, and more steps through the
codebase. For that reason, Cucumber isn't particularly good at giving me
rapid feedback. I want feedback in 10 seconds rather than 10 minutes.
Post by Ed Howland
Post by Pat Maddox
The best mantra I have for using Cucumber & RSpec in harmony is, "RSpec
lets me know my code works right, Cucumber lets me know my code is doing the
right work."
Post by Ed Howland
Post by Pat Maddox
Post by Ed Howland
Please forgive the x-post.
I just got back from the Great Lakes Ruby Bash. They had several good
presentations, two specific to BDD and Cucumber. I also talked to
several CEOs and devs afterwards, and the overall takeaway I gathered
was a shift to less RSpec and more Cucumber. Some people even claimed
a 90/10 split (cukes/specs) on current projects.
This was surorising to me and not at all how I worked up to this
point. I was more 20/80. I usually cuked a feature, then spec'ed the
code to make the cuke work. Each release had some new features and
specs for all the underlying code. Apparently, the feeling is that you
should do all your main thrusts with Cucumber and use RSpec for edge
cases. The theory is that you can change out all the underlying code
and the cukes still pass.
What is the communities consensus on this?
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
--
You received this message because you are subscribed to the Google
Groups "Cukes" group.
Post by Ed Howland
Post by Pat Maddox
Post by Ed Howland
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
Post by Ed Howland
Post by Pat Maddox
--
You received this message because you are subscribed to the Google
Groups "Cukes" group.
Post by Ed Howland
Post by Pat Maddox
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
Post by Ed Howland
--
You received this message because you are subscribed to the Google Groups
"Cukes" group.
Post by Ed Howland
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Curtis Schofield
2010-04-27 18:39:32 UTC
Permalink
Single responsibility Principle

On Tue, Apr 27, 2010 at 11:38 AM, Curtis Schofield <
Post by Curtis Schofield
Personally SRP violations are making my work hell right now on a 2 year old
rails system. My first concern when re-factoring is putting test in place -
there isn't much coverage when i started.
I've been doing that for the past 2 weeks and I've found that cucumber is
doing well for getting higher level coverage of 'it works' or 'it needs to
work like this' - but even so - the real guts of the problem is far and away
from Cucumber and so i've been using Cucumber to assist my design by
covering basic behaviors and then writing tests for the places where new
things must live and then moving them in while keeping the cukes functioning
an extracting responsibilities as the rspec tests require.
It's quite a job - now testing the views is another thing entirely - cuke
sorta works for this - but not so much - since i'm in a codebase that has
degenerated to rhtml bog in many ways - it's a bit of a trick and i have no
really good way to ensure coverage or confidence.
Great thread.
Post by Pat Maddox
My other contribution to this thread is that if your code is tested
primarily via Cucumber, you probably have SRP violations all over the place.
That's what I've noticed in my own code, anyway.
The tricky thing is that SRP violations are not necessarily crippling,
particularly in Rails apps (where SRP violations are pretty much built-in).
They just make your test suite a bit slower, limit reuse, etc... things
that are valuable to us but typically not as valuable as just shipping.
Pat
Post by Ed Howland
Post by Pat Maddox
Cucumber features are the best tool I know of for capturing
requirements from my customer. RSpec specs are the best tool I know of for
communicating intent and gauging code quality among the developer team.
Post by Ed Howland
Post by Pat Maddox
I'm not sure how exactly you're quantifying a 90/10 or 80/20 split. I
would expect that there would be a lot of overlap in coverage. That is, any
given line of code is likely to have some cucumber and some rspec coverage.
Personally I shoot for 100% RSpec coverage, and Cucumber is based entirely
on what my customer wants. If we discuss a new feature or bug fix and they
feel I know exactly what they're talking about and don't need a cucumber
test for it, I don't write the cucumber test. Cukes are for teasing out
requirements, which is most important when there's domain complexity that I
don't understand, because I'm a programmer and not a domain expert. Every
line of code I write gets RSpec coverage. That's how I personally feel
confident that my code does what I think it does, and also helps me keep my
dependencies under control.
Post by Ed Howland
Post by Pat Maddox
It's true that you can change out all the underlying code and cucumber
tests still pass. But you should be able to change out a lot of code and
have your specs still pass, as well. If you're changing the API then some
specs might no longer be valid, or need to be moved, or whatever. That's
just a part of refactoring. Although to be honest I think focused specs
help me refactor more than other people, because I take really small steps
when I refactor. When most people "refactor" they tear out a bunch of shit
and see if it still works.
Post by Ed Howland
Post by Pat Maddox
Cucumber tests tend to take longer because they're written at a much
higher level. That requires more setup, and more steps through the
codebase. For that reason, Cucumber isn't particularly good at giving me
rapid feedback. I want feedback in 10 seconds rather than 10 minutes.
Post by Ed Howland
Post by Pat Maddox
The best mantra I have for using Cucumber & RSpec in harmony is, "RSpec
lets me know my code works right, Cucumber lets me know my code is doing the
right work."
Post by Ed Howland
Post by Pat Maddox
Post by Ed Howland
Please forgive the x-post.
I just got back from the Great Lakes Ruby Bash. They had several good
presentations, two specific to BDD and Cucumber. I also talked to
several CEOs and devs afterwards, and the overall takeaway I gathered
was a shift to less RSpec and more Cucumber. Some people even claimed
a 90/10 split (cukes/specs) on current projects.
This was surorising to me and not at all how I worked up to this
point. I was more 20/80. I usually cuked a feature, then spec'ed the
code to make the cuke work. Each release had some new features and
specs for all the underlying code. Apparently, the feeling is that you
should do all your main thrusts with Cucumber and use RSpec for edge
cases. The theory is that you can change out all the underlying code
and the cukes still pass.
What is the communities consensus on this?
Cheers,
Ed
Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland
--
You received this message because you are subscribed to the Google
Groups "Cukes" group.
Post by Ed Howland
Post by Pat Maddox
Post by Ed Howland
To unsubscribe from this group, send email to
.
Post by Ed Howland
Post by Pat Maddox
Post by Ed Howland
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
Post by Ed Howland
Post by Pat Maddox
--
You received this message because you are subscribed to the Google
Groups "Cukes" group.
Post by Ed Howland
Post by Pat Maddox
To unsubscribe from this group, send email to
.
Post by Ed Howland
Post by Pat Maddox
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
Post by Ed Howland
--
You received this message because you are subscribed to the Google
Groups "Cukes" group.
Post by Ed Howland
To unsubscribe from this group, send email to
.
Post by Ed Howland
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group, send email to
.
For more options, visit this group at
http://groups.google.com/group/cukes?hl=en.
--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to cukes+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
Zach Dennis
2010-04-22 19:14:42 UTC
Permalink
Post by Pat Maddox
Cucumber features are the best tool I know of for capturing requirements
from my customer. RSpec specs are the best tool I know of for communicating
intent and gauging code quality among the developer team.
I'm not sure how exactly you're quantifying a 90/10 or 80/20 split. I
would expect that there would be a lot of overlap in coverage. That is, any
given line of code is likely to have some cucumber and some rspec coverage.
Personally I shoot for 100% RSpec coverage, and Cucumber is based entirely
on what my customer wants. If we discuss a new feature or bug fix and they
feel I know exactly what they're talking about and don't need a cucumber
test for it, I don't write the cucumber test. Cukes are for teasing out
requirements, which is most important when there's domain complexity that I
don't understand, because I'm a programmer and not a domain expert. Every
line of code I write gets RSpec coverage. That's how I personally feel
confident that my code does what I think it does, and also helps me keep my
dependencies under control.
It's true that you can change out all the underlying code and cucumber
tests still pass. But you should be able to change out a lot of code and
have your specs still pass, as well. If you're changing the API then some
specs might no longer be valid, or need to be moved, or whatever. That's
just a part of refactoring. Although to be honest I think focused specs
help me refactor more than other people, because I take really small steps
when I refactor. When most people "refactor" they tear out a bunch of shit
and see if it still works.
Cucumber tests tend to take longer because they're written at a much higher
level. That requires more setup, and more steps through the codebase. For
that reason, Cucumber isn't particularly good at giving me rapid feedback.
I want feedback in 10 seconds rather than 10 minutes.
The best mantra I have for using Cucumber & RSpec in harmony is, "RSpec
lets me know my code works right, Cucumber lets me know my code is doing the
right work."
+1
--
Zach Dennis
http://www.continuousthinking.com (personal)
http://www.mutuallyhuman.com (hire me)
http://ideafoundry.info/behavior-driven-development (first rate BDD
training)
@zachdennis (twitter)
Zach Dennis
2010-04-22 19:07:56 UTC
Permalink
Post by Ed Howland
I just got back from the Great Lakes Ruby Bash. They had several good
presentations, two specific to BDD and Cucumber. I also talked to
several CEOs and devs afterwards, and the overall takeaway I gathered
was a shift to less RSpec and more Cucumber. Some people even claimed
a 90/10 split (cukes/specs) on current projects.
I spoke at the GLRB, specifically reflecting on my experience with BDD and
outside-in development over the past few years, but I didn't attend the
first half of the conference, so I'm not familiar with the other talks or
comments made in those talks about a 90/10 split, or some of the other
things Ed mentioned in original post.

Ed, if you got any of this impression from me, please contact me off list.

Below are my thoughts in general...
Post by Ed Howland
This was surorising to me and not at all how I worked up to this
point. I was more 20/80. I usually cuked a feature, then spec'ed the
code to make the cuke work. Each release had some new features and
specs for all the underlying code. Apparently, the feeling is that you
should do all your main thrusts with Cucumber and use RSpec for edge
cases.
For me, Cucumber and RSpec fill different roles in my approach to software
development. It's not really an either/or proposition. I do want a tool that
supports high level plain text customer readable language which describes
the application. I also want a tool to drive behaviour, interaction, and
design of underlying components. These tools have different audiences and
provide different types of value.

There are times when I will use a Cucumber feature to drive the behaviour of
the application and not drop down to RSpec. And there are times when I add
change behaviour at a low-level where I do not expose a new Cucumber
feature. This all depends on the system, what's currently in-place, what's
being changed or added, etc.

In my experience there are several parts of webapps which are extremely
shallow and simplistic. There's not rich behaviour, interesting
interactions, or really any complexities. It's just monotonous work (like
CRUD, writing a basic RSS feed, etc.)

In many cases, after an application has evolved to a certain point a lot of
new features are simply re-using existing functionality you've already
built. It's just being packaged differently to present information in
another way or to provide downloadable reports, etc. This is another area
where I may not drop down to RSpec, and I let Cucumber drive this behaviour.
But I don't do this for everything. I do this when it makes sense.

I'm not in the business of writing examples just for the sake of writing
examples. They need to provide value. Example-driving code usually help
drive behaviour because they support emergent design, provide regression,
readability/maintainability. If these things are already handled, then I
need to ask myself why I am writing the example.

This is the case for a lot of vanilla Rails stuff. Rails has made decisions
for you on the design (ie: how associations are defined), so you don't
receive that value by example driving an association at a low level. If you
are driving the feature from the outside-in with Cucumber then Cucumber will
provide regression to ensure that association exists in the future. And
since a declarative statement like "belongs_to" is not complex, does not
drive design, and adding an example doesn't offer additional readability,
why are people still writing specs around the association? I don't care how
easy shoulda makes it to write, it doesn't mean it should be written.

Knowing when to not use a tool is just as important as knowing how to use
the tool.

One thing I mentioned in my talk @ GLRB was that Cucumber provides
implementation flexibility. This was in relation to my observation that
example-driving code well (with tools like RSpec/etc) requires experience,
courage, self-reflection, and willingness to change. A lot of folks don't
have that experience. They want to do things well, but they're kind of
guessing as they go. While I love apprenticeship, mentoring, and truly
engaged/collaborative/thought-provoking teams, I realize not everyone is in
these environments. I want to help folks to get some experience before
making a commitment to a decision, I encourage folks to utilize spikes.
Cucumber happens to act as a wonderful umbrella for folks to spike on lower
level details so they can get feedback on if what they're thinking is moving
them in the right direction.

Spikes provide learning, they help folks obtain some more experience, and
folks are ultimately able to make better decisions since spikes are focused
on problems being solved now. It may not be the best decision in the
universe, but making a better decision is usually much much better than
making simply a uninformed decision which can easily lead to commitment.
Spikes aren't only tied to implementation either, they can be an exploration
for how you would go about writing a good example to describe and drive some
behaviour.

My intent for the statement about "Cucumber provides flexibility" wasn't to
encourage folks to drop unit-level examples because they have supreme
flexibility with Cucumber. It was to encourage folks to take a step back and
explore with a spike what they're attempting to accomplish, so they can come
back and make a better decision moving forward.

I would love to rid the world of unnecessary examples, brittle examples,
impossible to read/maintain examples. Spikes put folks in a better position
to make bette decisions in that regard so I feel like my encouragement and
observations are still accurate and relevant.

And as Joseph said it's all project, client, and context specific.

My 2 cents. :)
--
Zach Dennis
http://www.continuousthinking.com (personal)
http://www.mutuallyhuman.com (hire me)
http://ideafoundry.info/behavior-driven-development (first rate BDD
training)
@zachdennis (twitter)
Loading...