test the integration of my adapter with the live service. This is
Automation Code (https://leanpub.com/EvolutionaryAnatomy/).
test. I need to check the happy path and that errors are reported
correctly. I might need several checks on different classes of errors,
e.g., no response, error response, unintelligible response....
things are wired up correctly.
Post by Marit van DijkGeorge, (and others :))
For instance we have pages (or data) in our Portal that user are /
aren't allowed to see depending on their role, or a contract they may or
may not have. These I can mock in local tests.
On the test environment I do still want to see all the components
(including message queues and other services maintained by other teams)
*actually* working together, as the other services may have changed,
rendering my mocking of them incorrect. It's sort of a early indication
of breaking changes. The company iI work for s looking to close this gap
by Contract Testing, which imho still doesn't fully cover the actual
dependencies but at least gets you a bit closer. It's all about fast and
early feedback!
Regards,
Marit
Marit,
Thank you for mentioning "very important unhappy flows." I don't have an
example of that in my Equine Horoscope project, but that's an important
thing to remember. I use UI level checks to make sure that what the
end-user sees is appropriate even when things go haywire on the back
end. If I've designed the code well, I don't have to check all such
haywire conditions, knowing that they will all be caught and handled
similarly.
All in all, my rule of thumb is to test any desired functionality at the
lowest level of abstraction possible. The reporting of problems to the
user is one of those things that can only be tested at the UI level.
- George
Post by Marit van Dijk@Vikram,
if business logic is extensively tested on a lower/faster level
(unit,
Post by Marit van Dijkapi) we do not check all cases again at a (slower) UI level.
Instead we
Post by Marit van Dijkdo more of a smoke test on that level, including only some happy
flows
Post by Marit van Dijk(or very important unhappy flows).
Hope this helps!
Marit
Vikram,
> Hi George,
>
> Thanks for sharing the project.
>
> Can you please clarify per below doubts from my last 1
year of
Post by Marit van Dijk > experiments with BDD ?
>
> 1. My role is SDET and following BDD since last 1 year.
> 2. I am mainly involved automating UI & API
> 3. How & at which stage do you decide if user story or
part of it
Post by Marit van Dijk can be
> automated at unit level ?
If the business rule that a scenario is checking is
well-isolated, I
Post by Marit van Dijk would wire up the scenario to just the class or module that
implements
Post by Marit van Dijk that business rule. Other scenarios will likely depend on the
business
Post by Marit van Dijk rule because the behavior they are checking has that
dependency, but I
Post by Marit van Dijk can thoroughly check the business rule by itself.
In such cases, though my scenario is expressed in business
terms, it's
Post by Marit van Dijk testing below the official API at a level that a unit test
might.
Post by Marit van Dijk > 4. Lets take an example of e-commerce application again
>
> As a user I would like to order Tesla Model X from
mygreenplanet.com <http://mygreenplanet.com>
<http://mygreenplanet.com>
Post by Marit van Dijk >
> 1.1 From my experience with BDD & Cucumber I will automate
api which
Post by Marit van Dijk > does search, payment, order confirmation, user auth etc
> 1.2 But I will also automate this whole flow at UI level
to make
Post by Marit van Dijk sure UI
> layer has parsed info properly from api and shown to user in
correct way.
> 1.3 As this is "business critical" flow, I will run api test
cases every
> few minutes with jenkins ( to make sure whole backend
infrastructure is
> up & running all the time) also these are cheap + fast
tests but UI
Post by Marit van Dijk > test cases only when there is new change in UI master.
That sounds excellent, to me. I'm inclined to also run UI
tests on a
Post by Marit van Dijk periodic basis, nightly or weekly, for extra security. And in
some
Post by Marit van Dijk contexts, capture screenshots of every screen so someone can
detect
Post by Marit van Dijk visual problems (usually CSS clashes, IME) without having to
traverse
Post by Marit van Dijk the app.
>
> Please let me know from your experiences, how do you map
"Test
Post by Marit van Dijk Pyramid"
> with user stories discovered during 3 Amigos session ?
When I was building this project, I wasn't thinking of the
test pyramid
Post by Marit van Dijk at the time. Instead, I was using heuristics to decide what
to do, and
Post by Marit van Dijk the test pyramid was a result of that. This was a new "aha"
for me. The
Post by Marit van Dijk test pyramid doesn't seem like a useful guide at the moment of
development, but a good yardstick for questioning the result.
I was programming this by myself (and what a drag it is to
not have a
Post by Marit van Dijk pair for discussing ideas), but in a team situation I would
think that
Post by Marit van Dijk the programmer would dive into the unit test level as soon as
they
Post by Marit van Dijk start
working. I also TDD the test support code I need along the
way. I've
Post by Marit van Dijk encountered situations where there was a bug in the test
support code,
Post by Marit van Dijk so the tests weren't checking what people thought they were.
I have not watched it, yet, but Agile India 2017 videoed my
talk on
Post by Marit van Dijk this
evolution. The talk doesn't include everything that the book
does (due
Post by Marit van Dijk to time constraints), but it might be helpful.
http://youtu.be/JbPBNyc1E6o
http://youtu.be/JbPBNyc1E6o
Post by Marit van Dijk http://youtu.be/JbPBNyc1E6o
http://youtu.be/JbPBNyc1E6o
Post by Marit van Dijk I'd love to hear your feedback to this material.
- George
>
> Thanks & Regards,
> Vikram
>
>
> On Wednesday, August 16, 2017 at 1:07:18 AM UTC+2, George
>
> Robert, Björn, Vikram,
>
> Perhaps you'd be interested in looking at
> https://github.com/gdinwiddie/EquineHoroscope
<https://github.com/gdinwiddie/EquineHoroscope>
Post by Marit van Dijk <https://github.com/gdinwiddie/EquineHoroscope
<https://github.com/gdinwiddie/EquineHoroscope>>
Post by Marit van Dijk > <https://github.com/gdinwiddie/EquineHoroscope
<https://github.com/gdinwiddie/EquineHoroscope>
Post by Marit van Dijk <https://github.com/gdinwiddie/EquineHoroscope
<https://github.com/gdinwiddie/EquineHoroscope>>> which demonstrates
Post by Marit van Dijk > acceptance testing mostly at the API level, unit
testing to
Post by Marit van Dijk support
> that, and, in this example, only a single GUI test to
ensure
Post by Marit van Dijk things are
> wired up correctly. There's an accompanying book on
LeanPub
Post by Marit van Dijk if you're
> interested in that.
>
> - George
>
> > Hi Bjorn,
> >
> > On Friday, August 11, 2017 at 2:06:34 AM UTC-7,
80Vikram
Post by Marit van Dijk > >
> > Hi Björn,
> >
> > I went through the talk but the concept was pretty
abstract
> and I
> > didn't fully understood it.
> >
> > - Do you have any open source project where you
tried
Post by Marit van Dijk this
> approach ?
> >
> > - Per my understanding it only worked for Nat
as their
Post by Marit van Dijk > application
> > was based on "Ports-and-Adaptors" architecture,
not
Post by Marit van Dijk sure if this
> > will work in other scenario lets say
"e-commerce ordering
Post by Marit van Dijk > flow" or
> > mobile native app like Uber.
> >
> > - In my project, we do follow BDD and discuss &
write
Post by Marit van Dijk user
> stories
> > and try to automate as much at unit level
automation (
Post by Marit van Dijk sdk /
> > application code ) by developers.
> >
> > - QA is involved in doing all of exposed API
automation and
> ofcourse
> > UI level functional automation as well which is
not
Post by Marit van Dijk covered
> at unit
> > / api level ( Cucumber + Appium ) .
> >
> > Thanks in advance.
> >
> > Regards,
> > Vikram
> >
> >
> > I echo Vikram's thoughts. I went through Nat's
presentation (he
> did a
> > wonderful job) -- not once, but 3 times -- and
picked up
Post by Marit van Dijk minor
> tidbits
> > here and there. However, I guess I'm missing the big
picture and
> some
> > context. Is there a github example that you might
know of
Post by Marit van Dijk that lays
> > this approach out with concrete examples ??
> >
> > Regards,
> > Robert
> >
> >
> > On Wednesday, August 9, 2017 at 4:02:14 PM
UTC+2, Björn
Post by Marit van Dijk > >
> >
> > Hi Aslak,
> >
> > I didn't understand "...This allows you to
implement
> step
> > definitions by making direct
method/function
Post by Marit van Dijk calls to
> the
> > domain logic, avoiding the UI...."
> >
> > If I'm automating ordering flow in an
e-commerce website
> > with Selenium, then I need to do all
the calls (
Post by Marit van Dijk > selenium
> > method calls on web elements in
ordering flow
Post by Marit van Dijk ) at UI
> level
> > right ?
> >
> >
> > If you are executing BDD specifications
(feature
Post by Marit van Dijk files)
> for a
> > e-commerce system, you have a choice of
which access
Post by Marit van Dijk > > - browser
> > - http service
> > - direct call to the domain layer
> > By using different sets of step definition
it is
Post by Marit van Dijk possible to
> > execute the very same feature file using
any of these
Post by Marit van Dijk > access points.
> > Nat Pryce did a presentation "End to end
functional tests
> that
> > can run in milliseconds"
http://youtu.be/Fk4rCn4YLLU on
> > CukeUp! 2017 about this.
> >
> > Regards
> > Björn
> >
> >
> > Thanks,
> > Vikram
> >
> > On Tuesday, August 8, 2017 at 4:23:03 PM
UTC+2, Aslak
> >
> > Always pick a Cucumber
implementation in
Post by Marit van Dijk the same
> > language as the application (in
this case,
Post by Marit van Dijk Behat).
> >
> > This allows you to implement step
definitions by
> making
> > direct method/function calls to the
domain
Post by Marit van Dijk logic,
> > avoiding the UI.
> >
> > This is the only way to achieve a test
pyramid, and
> > avoid an ice cream cone: Slow,
expensive
Post by Marit van Dijk and brittle
> > tests/scenarios.
> >
> > Aslak
> >
> > On Wed, 12 Jul 2017 at 05:36,
Deepak Sh
Post by Marit van Dijk > >
> > We want to implement Cucumber
in our new
Post by Marit van Dijk > project as
> > we want to follow BDD approach.
The
Post by Marit van Dijk project
> will be
> > developed using Symfony 3.0 and
Angular 4.0.
> >
> > I am new in automation testing.
I have
Post by Marit van Dijk > exposure in
> > Python with Selenium and little
bit in
Post by Marit van Dijk Java with
> > Selenium.
> >
> > Should I go with the
"in-process" or
Post by Marit van Dijk > "out-of-process" ?
> >
> > I am confused between PHP( using
BEHAT) and
> > Java/Python and Selenium
Framework.
Post by Marit van Dijk > >
> > What will be the best solution
here as
Post by Marit van Dijk I am
> looking
> > for the career prospective also.
Please help me.
> >
> >
> >
> >
> >
> >
> >
> >
> > --
> >
> >
> http://cukes.info/posting-rules.html
<http://cukes.info/posting-rules.html>
Post by Marit van Dijk <http://cukes.info/posting-rules.html
<http://cukes.info/posting-rules.html>>
Post by Marit van Dijk > <http://cukes.info/posting-rules.html
<http://cukes.info/posting-rules.html>
Post by Marit van Dijk <http://cukes.info/posting-rules.html
<http://cukes.info/posting-rules.html>>>
<http://cukes.info/posting-rules.html
<http://cukes.info/posting-rules.html>
Post by Marit van Dijk <http://cukes.info/posting-rules.html
<http://cukes.info/posting-rules.html>>
Post by Marit van Dijk > <http://cukes.info/posting-rules.html
<http://cukes.info/posting-rules.html>
Post by Marit van Dijk <http://cukes.info/posting-rules.html
<http://cukes.info/posting-rules.html>>>>
Post by Marit van Dijk > >
> >
> > ---
> >
> >
> > You received this message
because you are
Post by Marit van Dijk > subscribed
> > to the Google Groups "Cukes"
group.
Post by Marit van Dijk > >
> >
> > To unsubscribe from this group
and stop
Post by Marit van Dijk > receiving
> > emails from it, send an email to
> >
> >
> > For more options, visit
> > https://groups.google.com/d/optout
<https://groups.google.com/d/optout>
Post by Marit van Dijk <https://groups.google.com/d/optout
<https://groups.google.com/d/optout>>
Post by Marit van Dijk > <https://groups.google.com/d/optout
<https://groups.google.com/d/optout>
Post by Marit van Dijk <https://groups.google.com/d/optout
<https://groups.google.com/d/optout>>>
<https://groups.google.com/d/optout
<https://groups.google.com/d/optout>
Post by Marit van Dijk <https://groups.google.com/d/optout
<https://groups.google.com/d/optout>>
Post by Marit van Dijk > <https://groups.google.com/d/optout
<https://groups.google.com/d/optout>
Post by Marit van Dijk <https://groups.google.com/d/optout
<https://groups.google.com/d/optout>>>>.
Post by Marit van Dijk > >
> >
> > --
> > Posting rules: http://cukes.info/posting-rules.html
<http://cukes.info/posting-rules.html>
Post by Marit van Dijk <http://cukes.info/posting-rules.html
<http://cukes.info/posting-rules.html>>
Post by Marit van Dijk > <http://cukes.info/posting-rules.html
<http://cukes.info/posting-rules.html>
Post by Marit van Dijk <http://cukes.info/posting-rules.html
<http://cukes.info/posting-rules.html>>>
Post by Marit van Dijk > > ---
> > You received this message because you are
subscribed to
Post by Marit van Dijk the Google
> > Groups "Cukes" group.
> > To unsubscribe from this group and stop receiving
emails
Post by Marit van Dijk from it,
> send
> > For more options, visit
https://groups.google.com/d/optout <https://groups.google.com/d/optout>
Post by Marit van Dijk <https://groups.google.com/d/optout
<https://groups.google.com/d/optout>>
Post by Marit van Dijk > <https://groups.google.com/d/optout
<https://groups.google.com/d/optout>
Post by Marit van Dijk <https://groups.google.com/d/optout
<https://groups.google.com/d/optout>>>.
----------------------------------------------------------------------
Post by Marit van Dijk > * George Dinwiddie * http://blog.gdinwiddie.com
> Software Development http://www.idiacomputing.com
> Consultant and Coach http://www.agilemaryland.org
>
>
----------------------------------------------------------------------
Post by Marit van Dijk >
> --
> Posting rules: http://cukes.info/posting-rules.html
<http://cukes.info/posting-rules.html>
Post by Marit van Dijk <http://cukes.info/posting-rules.html
<http://cukes.info/posting-rules.html>>
Post by Marit van Dijk > ---
> You received this message because you are subscribed to
the Google
Post by Marit van Dijk > Groups "Cukes" group.
> To unsubscribe from this group and stop receiving emails
from it,
Post by Marit van Dijk send
> For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>
Post by Marit van Dijk <https://groups.google.com/d/optout
<https://groups.google.com/d/optout>>.
----------------------------------------------------------------------
Post by Marit van Dijk * George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------
Post by Marit van Dijk--
Posting rules: http://cukes.info/posting-rules.html
<http://cukes.info/posting-rules.html>
Post by Marit van Dijk---
You received this message because you are subscribed to the Google
Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it,
send
Post by Marit van DijkFor more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.