A Testers Guide To the Galaxy

A Testers Guide To the Galaxy

Who am I to write this blog?

I've been in the testing game for app. 12 years now and in the agile game for more then 14 years, started out as a part time tester in internal test projects, moved on to a role as full time tester doing automation, took my ISEB foundation, moved on the consulting world, now as test coordinator, became part of an agile project, added certified SCRUM Master to my CV, got my ISTQB ATM exam, moved on to role as Test Manager. Recently I have moved more and more into the agile game, adding CAT Certified Agile Tester, ISTQB CTFL Agile Tester Extension and SAFe Agilist to the list of certifications. So now I act more as a trainer (CAT, ISTQB CTFL & CTFL AT) and agile coach!

I simply want to share my thoughts on agile and testing and all thats connected with it.
Feel free to comment on my thoughts!

Agile, a way to fail faster

MethodsPosted by Søren Wassard Wed, August 02, 2017 14:32:03
Which companies are going agile?
Is it the companies which know exactly what they are doing? The ones which have all their processes in place, and follow them? The ones that have a solid way of collecting requirements and documenting them? Or actually the ones that have a good documentation standard and follows it?
Or is it the companies who lives in caos, have no or few processes, or at least are following them? The ones that have no requirements documented (or no documentation at all)?
I argue that many companies that goes agile today, and have for the past few years are the ones that fall into that last group.
Companies that lack everything looks for the easy fix, and for some reason agile has become that apparent easy fix.
But is it? Is it an easy fix? will it solve the problems listed above? Not at all, what they will end up with is a new documenation issue, new processes that are probably twisted to fit the company, requirements caos, dev trying to do agile, rest of company continuing doing like they used to, resulting in even more clashes then previously.
Is this because of agile, is it in its nature? No, it is not, because many companies have succesfully implemented one or more agile approaches/flavours.
To me one of the differences between those who succeed and those who fail, is how disciplined, well organized, well structured, well documented, they were before the agile transformation.
A company that is structured and know what they are doing can often handle the disruption which an agile (or indeed any) transformation is, whereas others will suffer and fail.
The reason for this is because they try to fix all of their problems by changing the methodology whereas the right approach in my perspective is taking babysteps, Fix one problem at a time;
-Get your (just enough) documentation in place
-Establish good processes and follow them
-Collect and document requirements sufficiently
-Get everyone involved - from the board of directors to the piccolos, get them involved, start to get them to embrace the thoughts behind agile
And then you can implement your agile methodology, and your chance of succes is much higher.
So if done wrongly, agile is just another way to fail, your bad results will be visible a lot faster then before, in essence, you will fail faster.
Done correctly it will take time, a lot of time, and commitment from the entire organization but you will end up with a company that can react to the disruptions that happen all the time and which will increase in numbers and frequency over time, and come out stronger each time.

  • Comments(0)//tg2g.wassard.eu/#post11

Test is dead - RIP!

Thoughts on testingPosted by Søren Wassard Wed, June 21, 2017 10:34:49
Did I get your attention? I'm a for real? Do I actually mean that?
Yes, I will state that testing is dead!
You probably disagree; How can test be dead, if we do not test how can we ensure a high product quality, how can we validate that the system works as intended, how can we verify that the functionality works?
Well, the statement "test is dead" is based on the context in which we normally would test; re-actively! Testing is re-actively verifying and validating a solution. "But we have shifted left, we do reviews, we are involved early in the process, whenever something is ready for either static or dynamic testing" - exactly my point - when something allready has been created = a re-active action.
Shift-left is simply not enough, we must enter the scene way earlier, we must be present when the solution is being formed, being discussed, being detailed from idea level to Release theme, features, epics, user stories, so not reviewing, but being part of the creation.
My thesis is based on the power of 3, the 3 amigos concept; At least 3 different roles / archetypes must be involved in the creation of the requirements (what ever form they are in) to ensure that as many angles to the requirement / solution are taken into consideration. If 1 person alone writes a requirement there will be undocumented assumptions, expected levels of knowledge, tendencies to "happy path" the solution. The more roles you involve, the more angles you get to a given solution, hence assumptions and other issues are identified before the solution is being described and created.
One of these roles is of cause the "tester" who really shouldn't spend time on testing, but rather must spend time on facilitating and securing quality in the process and in the description of the solution (requirement, epic, user story - what ever), and that is why I state that test is dead.
We must be involved as soon as something starts, we must ensure that we wander of the happy path and challenge assumptions. If we do that right, if we ensure, or rather contribute to the quality assurance of the product up front, we do not have to test re-actively, or at least we can minimize the amount of testing needed.
Statement; Prevention over detection! Be pro-active over being re-active!
So, testers are no longer testers! I'd prefer the title Quality Facilitator, which in my mind makes so much more sense. It really states what we should do, instead of what we have been doing.
It makes sense in the agile context where the team is responsible for the quality of the product, it makes sense in the context of Acceptance Test Driven Development, Behaviour Driven Development, Test Driven Development where the test cases that we create simply are an outcome of the conversation that we have about writing / building the right requirement / solution, and we can of casuse benefit from these test cases to re-actively ensure that we have build it correct, that we can state when we are done, but neither ATDD, BDD or TDD's main focus is to create test cases, it is to create the conversation - pro-actively.
So test is maybe really not dead, but the attention needs to shift beyond left, tests must become a byproduct of the conversation of doing things right from the beginning

  • Comments(0)//tg2g.wassard.eu/#post10

AgileTestingDays day 3 - Highlights

ConferencesPosted by Søren Wassard Mon, November 17, 2014 12:15:14

Don’t put me in a box

Seems that the hot topic this year is soft skills, something that I emphasize!

Anthony talks about not limiting our self by a title, for instance “tester”.

It was a very inspiring talk with examples of his life experience on how he went outside the box, and to do something different he had no supporting slides :o) – just a man on a stage with a microphone!

He gave us a lot of different examples. One of them was from Special Forces. He told how he went through a course on how to enter a house in a hostage situation. 5 persons were needed; one to break down (or just open) the door, two to make a first entry and secure the room and two to backup/support from behind. Once in a room they would re-group so that everyone was back in their position before entering the next room. This made the 2 persons in front most in risk of getting hit during entry, and the specialist for opening doors was essential. Now, it took some time to re-group and by doing so they might lose their momentum and an enemy would have time to take countermeasures. A way of mitigating that was to change how they entered a house in a possible hostage situation. They learned the READ system (reading the situation in order to know how to act!). Still 5 persons going in, but once through the first door, which was most likely to be the one that needed a specialist for blowing it up or opening it otherwise, the first two persons who reached the next door would now be the once to open it and make the first entry and the remaining team would support. By doing so, the roles changed all the time, so they would have to adapt.

Does it sound familiar? Sounds almost like agile, doesn’t it?

Anthony also referred to LEAN, especially the Waste types. And not to the 7 types of waste, no, to the 8 types of waste!! Yes, and number 8 is important: “Skills – Underutilizing capabilities, delegating tasks with inadequate training”.

How can that be put into context of adapting your role? It simply means that even though you have an area of expertise it doesn’t mean that you don’t have other areas of knowledge that can add value whenever your expertise isn’t of use. On the other hand, do not waste time by working on tasks for which you have no skills unless no other persons with that specific skill available.

-You are a Terminator, right?
-Yes. Cyberdyne Systems, Model 101
-… you're like a machine underneath, right? But sort of alive outside?
-I'm a cybernetic organism. Living tissue over a metal endoskeleton.

Science fiction? Yes! Impossible to think of? No!

Referring back to the keynote by Lisa and Janet Tuesday morning; They came on the scene with a presentation about the future, dressed up like Captain Kirk and Mr. Spock from the original Star Trek series first aired back in 1966 – that’s now 50 years ago. Back then, the technology in Star Trek was amazing, it was so unreal. But is it? It was, for sure, but actually it is basically just the transporter (you know; “Beam me up, Scotty!) that is unreal today. Most of the devices have been superseded by the current technology!

That is one of the points in the talk by Daniël Maslyn (last year did the famous “wrote my presentation on hotel stationary last night, took pictures of it with my cellphone and put it in PowerPoint” presentation).

Daniël’s talk; “We are the robots; Agile Testing for future robots” had a lot of cool references to Blade Runner (in the Directors Cut Version there is a unicorn appearing).

The reason for this is also the fact that the future (Blade Runner is supposed to reflect the world anno 2019) may not be that far away. Different people are construction autonomous robots, are constructing parts for them, for instance eyes that we do today, not for robots, but as spare parts for humans.

And this poses a challenge in testing. These spare parts and robots are becoming increasingly complex, and most of them must be tested in the live, they cannot be simulated! Of cause you can do some testing on the software and hardware devices individually, but only when put together you’ll get the real result!

One of the tests natural to run on robots is the Turing test. For those who don’t know the Turing test it is a test of a machine's ability to exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human (definition from; http://en.wikipedia.org/wiki/Turing_test).

Now are we at a point where the Turing test is needed? Maybe not, but we are getting there.

That’s it for the highlights. The rest of day 3 was dedicated to CAT Trainer Day.

I really hope to be back next year for Agile Testing Days in Potsdam :o)

  • Comments(0)//tg2g.wassard.eu/#post9

AgileTestingDays day 2 - Highlights

ConferencesPosted by Søren Wassard Thu, November 13, 2014 11:05:10

Simply the best key ever…

Yesterday Lisa and Janet gave a good keynote, but todays keynote by Joe Justice from Scrum Inc. (Twitter: @JoeJustice0, @ScrumInc) was amazing!

Joe is the inventor of XM. Isn’t that a Citroën from the eighties you may ask? Well, yes, but it is also the abbreviation for eXtreme Manufactoring, taking Scrum from software development to hardware manufacturing. He is also the founder of the car company Wikispeed.

The initial purpose of Wikispeed was to win the X Prize Challenge, a competition to build a road safe car that can run 100 Mpg or in the European consumption standard; 1,5 liters of fuel per 100 km!

So how do you do that? Well, you start by creating the user story:

As a car manufacturer
I would like to build a car
So that I can win the X Prize Challenge

And then you start building a car using scrum and XP and very important, you implement Test First!

Did he win? No, but came into a 10th place by building a car in Joes garage!

That impressed quite a few people and XM was born.

XM has now been implemented by Joe and his team at customers like; Lockhead-Martin, Boeing, John Deere, X-box, HP, TomTom.

Joe gave an example of how one of his customers, John Deere, after 16 months performed 7.2 times better than before they implemented XM! All employees are also happier than before XM (Scrum) and work less hours.

So Scrum is not limited to software development!

By the way, Scrum Inc. has gathered metrics regarding performane and team size and found that a size of 4,6 persons (yes, that is an average) performs the best!

But Scrum can be used in other areas as well, for example education. A new school projects EduScrum has emerged. It seems that the students learn faster and get better grades! #eduscrum – check it out!!!!

It also seems that the financial world has got their eyes opened for Scrum. Pictet is a privately held bank which demands from its clients to deposit a minimum of 75.000.000 Euro in order to open an account in the bank, so people with accounts here are powerful people. They have decided to create a Scrum Fond who will only invest in companies which successful run Scrum. The reason? Because Joe proved to them that the risk is much less, and the profit higher than with companies run traditionally!

The whole point is that by going with Scrum and doing test first, we know where we are heading, can change as things change around us and thereby deliver faster, better, cheaper!

So people; Go spread the word – be Agile Evangelists!!

So, I was really looking forward to being a part of this, I couldn’t wait to get my hands on that car!

But where is the car??

The car, the Wikispeed car, would come in late, very late, which meant I had to change my plan, but unfortunately the session I’d liked to go to were full, as in people were standing up outside the open door trying to catch a glimpse of what was going on inside.

So I ended up going to other less crowded sessions which unfortunately didn’t leave much of an impression.

Second keynote

Insights from happy change agents

Alexander Schwartz and Fanny Pittack

You go to a conference, attend a course, get an idea, and now that must be implemented, because you think it is amazingly great….

But, how did your team, your colleagues feel about that change? Did they embrace the change? Or did they on the contrary go against it?
What could you have done to implement the change easier?
There is not one answer, but these advices are worth considering;

- Talk it through with your team beforehand!
- Make sure to adapt your ideas to the context in which it must be implemented

Both advices point to the fact that it is much easier to achieve a goal if you work together, if you have a common goal, and agree on how to get there!

So the takeaways are:
- Try it, do not avoid changes
- Listen to feedback and yourself
- Learn!
- Forget about it
- Try again

Every change is a leap of faith!

After lunch I thought that I was going to do some Black Ops testing, but then finally, yes, the Wikispeed car arrived. In order to get the build started we had to get the car of the truck and into the lobby – in parts!

So the professional racing crew from Team Wikispeed.de startet, and with help from a us delegates, we managed to get the car disassembled and carried into the lobby in approximately an hour

And then the event started – first sprint started at 16:00. Up to 25 persons including the 4 guys from Team Wikispeed.de was to form 5 teams with 5 persons at the maximum. As there was only 4 professional mechanics the question was raised who had the most car building experience from the remaining persons, and it turned out to yours truly – yes – I can now put yet another area of expertise to my CV; Car Building Expert ;O)

Joe Justice gave an introduction, teams picked a Product Owner and a Scrum Master and then the Product Owner ran off and picked a story from the back log.

Now the whole exercise was about how to do construction, Scrum style. The expert on each team explained what had to be done in order to complete the story, team decided how to divide the work, Scrum Master facilitated (got tools, parts, etc.) based on the needs of the team. This was the sprint planning. As a sprint was roughly 30 minutes it took approximately 2 minutes to do the planning, and then we “sprinted”, fulfilled the picked up user story. It quickly turned out that there were dependencies, and the scrum masters did scrum of scrums to mitigate these dependencies in order for all to reach the goals.

After the sprint a short retrospective was held, and the next 25 people were going to build in the next sprint.

4 sprints later a car, or at least a rolling chassis with a body, should be done.

There were also a number of requirements, which basically added up to the fact that the car ultimately should conform to German race car standards as this car ultimately will run on the Nürnburg Ring. The fact that these requirements existed was not highlighted that clearly, so yes, it did lead to errors.

One requirement was that all bolts should have just the length needed for 2 washers and a locknut to fit.

Now, when the car was taken apart most bolts for the suspension was kept in place, so when the reassembly started, the bolts which were initially used were simply reused.

As some of the nuts were placed inside the frame of the chassis it wasn’t possible to see if we met the requirement for the bolts, it turned out, we didn’t L

So as we started out on sprint 2 we had to take the suspension apart again and start over, but it quite fast became clear that the bolts were all 5 mm short of meeting the requirements. So we spiked, tried out different solutions, shorter bolts, way longer bolts, but failed, we could not meet the requirement.

Joe took a C(3?)PO decision and decided to go with the “almost long enough” bolts for sprint 3 in order for the car to get done at all.

And finally after a very long (1½ hour) 4th sprint it was done! The car was build, using a few subject matter experts and a lot of willing hands, and working agile! eXtreme Manufacturing at its best!

It was tremendous fun, THE best agile exercises I’ve ever participated in!!

Some fun facts about the car; Wikispeed.de bought the rolling chassis. They are going to rebuild it, make it race legal, add a BMW M3 race engine to it, and the goal is that within 2 years they are able to drive it in an endurance race on the Nürnburg Ring!!! You can follow the team at www.wikispeed.de

eXtreme Manufacturing rocks!!!

  • Comments(0)//tg2g.wassard.eu/#post8

AgileTestingDays day 1 - Highlights

ConferencesPosted by Søren Wassard Wed, November 12, 2014 11:25:28

Live long and prosper…

No, it’s not Captain Kirk and Mr. Spock but is might as well have been ;o)

It’s Janet Gregory and Lisa Crispin delivering the first keynote @AgileTD, and of cause they do it the way they always do; with a great show.

They’ve left the Enterprise for a couple of days to be with us and talk about the future; Welcome to the future! Preparing for our agile testing journey…

Janet and Lisa stated that we do not know what the future will bring, but that we will get hit by it faster than we imagine, and the speed in which we get hit is increasing.

So how can we as testers mitigate that? By adapting to change! Or as Gunnery Sergeant Highway states it; Improvise, adapt and overcome! We need to change the way we work, we need to be willing to update our skills set, not be afraid to step on to uncharted territory. Lisa and Janet came with a metaphor; Be a Rubik’s cube – get it? Make sure that you have enough skills to match the current context!

They also talked about communication. We are getting lazy, and that has an impact on our ability to communicate, which again leads to misinterpretations and misunderstandings. Too many people call colleagues or writes e-mails to them even thou they are co-located in the same building! The message is; Avoid time consuming and potential misleading communication by getting on your feet, go to your co-worker and talk to them face to face!

Over the years they have also through many projects picked up that even though the projects have been agile, not the right things were done – why? Because the user stories weren’t aligned! So by introducing Impact mapping and Story mapping you can mitigate that, and make sure that only stories which contribute to the final goal are done.

Next up was Kristoffer Nordström who talked about “The struggle of my identity and how I got developers to start testing”.

Kristoffer gave a good talk about how he his entire work life has struggled to be confident in what he was doing. He has always felt that when he entered a new company, project, group, team, whatever, he found himself uncertain whether he could do the job, could match the apparently cool and intelligent people he was surrounded by. This feeling of having to fake something and the fear of being revealed as a fake has been hunting him. But it turns out that approximately 75% of all people feel the same!

So it’s something deep inside us that does this. Kristoffer’s advice was to be true to yourself, remember who you are, in an agile team; pick up that test flag and carry it proudly!

In the same stream followed Allessandra Moreira. She has always been working in traditional waterfall projects with the classic “over the wall” hand over process. So she was a classic tester, with a classic testers mindset when she first joint an agile team, and had to re-invent herself completely. She now faced the challenge of having to take on new skills, become more of a testing coach then a tester, and let go of the “quality gate keeper” equivalent to Gandalf; “You shall not pass”, because suddenly it was okay to “pass” as long as we knew what we were passing into production!

But she overcame, change her mindset and some 18 months later they were running agile!


- Technical skills are NOT the most important skills – soft skills are more important

- A testers job is NOT to ensure quality – a tester is a skilled investigator which can highlight risks and provide information about quality

Last session I want to highlight is the TenKod workshop on automated mobile app testing.

Emil Simeonov gave a good intro to their eclipse based product EZ TestApp, and it’s definitely something worth looking into. Its capture/replay on both simulated as well as real devices! You can even capture on a simulation and replay on a real device – cool!

So that’s something I really want to dig further into!!!

  • Comments(0)//tg2g.wassard.eu/#post7

Other blogs worth visiting

LinksPosted by Søren Wassard Mon, November 26, 2012 10:57:33
Test Evangelist Gitte Ottosens thoughts on testing:
Godtesen - Thoughts on Software Testing

  • Comments(0)//tg2g.wassard.eu/#post6

Worksoft Certify - the verdict

ToolsPosted by Søren Wassard Wed, November 02, 2011 14:04:42
First time I heard about Worksoft Certify was a short introduction to it in a presentation that showed the new trends within software testing.

The promise of "scriptless automated testing" seemed like a good portion of sales grease, because I've heard of "scriptless" testing before, and as you might have guessed, they were not scriptless at all, but simply capture/replay tools.

I then saw a video, and yes, it seemed that it was true - no scripting - not a single line of code!

Even then I was skeptical – could it really be true?

Then last week, finally, I got my hands on the product as I attended a Worksoft Technical enablement course. There it was the mystical product that kept stating that no lines of code were used / generated.

And yes, it is true – someone heard my prayers smiley

Worksoft Certify does not generate code, you don’t need to know Java, VB scripting, C++ or whatever language regular automation tools are using for their scripts.

That being said, I will emphasize that knowing just a slight bit about programming is definitely helpful.

So how does it work?

Worksoft Certify is a “fat client” which needs to be installed on the machine on which you will create and execute your test connecting to the product database in which all the objects, processes etc. are stored.

From that client you create your process. “Proces? I thought we talked about testing, creating scriptless testscripts?” – well – I had to swallow that one as well. Proces is testscript! This naming, and here I’m guessing, are named due to the fact that Worksoft Certify was built for testing SAP solutions. So they use SAP terminology.

But back to the product. You create a new process, name it, save it, go to steps, right click in the steps window, choose New, in the field Application Version you choose Select using LiveTouch. And then you are off and away – easy as that. From here of you record your movements in your SUT (System Under Test), after recording your actions and your checkpoints you then modify your “script” steps by adding variables where needed, set up test data that you need for your variables, and then you can run your automated test.

After a few hours of working with the product I could create quite advanced test scripts, which will work every time, now, and in a year. I was amazed!

Yes, it is originally made for SAP but you can test HTML, .Net applications and a lot of other stuff, as long as it is GUI based.

Being built for SAP it “plugs” in to SAP Solution Manager, you can import requirements and test scripts from SAP and export reports back to the solution manager. If you set it up together with RQM you can actually export the SAP blueprints, get your requirements in automatically, get test plans and test cases created automatically, then you can add your test scripts (sorry – processes), run them – report back to SAP solution manager, log defects in the SAP solution manager incident handling etc. – AWSOME!

A very smart trick in Certify is that it knows the SAP objects, and it knows them by version, so if a new version has update, added or deleted objects Certify knows that, and will inform you about it! We are talking about semi-automated script maintenance. How would a changed object be handled in an old-school test automation tool? You would have to find every single script where the object is used and modify the lines of code. Here Certify automatically updates the object and then you again can run your script – and it still works!

Another thing. You can reuse your processes, use them as child-processes. For instance Log in is the same for all functionality in my application, and I can therefore create a process that runs the log in and add that as a child process to my other processes, creating easy to maintain end to end test. Because, if something changes in my log in porcess, I simply change the one process and all of the processes where this process is a child to will now be running by the new changed process!

So my verdict? well, I think you already know that;

I’m impressed! It’s by far the most innovative tool I’ve seen in many years!

  • Comments(1)//tg2g.wassard.eu/#post5

Why is automation so hard?

ToolsPosted by Søren Wassard Fri, October 07, 2011 08:19:51
In the early eighties I got a Sinclair ZX Spectrum. A fine little piece of hardware.
On that machine I wrote my first lines of code - created my own space invaders like arcade game. Approximately at the same time I started using Comal 80 and for a few years I spend quite some time on this. In the beginning of the nineties I was introduced to TurboPascal, and for a year or so I used that language.
Then I started at Copenhagen Business School studying logistics, and I didn't spend an hour on coding before I in 2004 started working with automated test, using TestPartner. Back then it was a Compuware product, today its being sold by MicroFocus.
TestPartner used VBA scripting to generate the code. I didn't know VBA, so I did capture/replay and got a little help from the developers (has to say that I was the only person in the "testing department").
So I did create automated test scripts. Were they any good ? I think so, yes. But not in version 1.0, and not in 2.0 or 3.0.
I spend ages re-writing, splitting up scripts, making error handling better, continuous improvements, which of cause had an impact on my ability to expand the suite of test scripts.
I have since then worked with automation tools such as Quick Test Professional (QTP), Rational Functional Tester (RFT) and Selenium. They are all capture/replay tools, but the scripts needs to be tweeked, error handling must be added, "intelligence" must be build in etc. All tasks that needs a developers involvement, or at least very good knowledge on the programming language by the tester.
So, do all testers by default have a developer background ? No, they don't! Do all testers feel an impelling urge to learn how to write code ? Probably not - if so they would most likely be developers and not testers.
That leaves us with a few specialized persons, making it hard for an organization which would like to do automated testing. They either don't have the knowledge in-house, or can not afford to call in those specialist who actually can do automation.
Isn't that a problem ?
I think so - I think it's a huge problem!
Then just the other day I came across a product that was new to me; WorkSoft Certify.
Never heard of it before, but here it seemed that the solution to my problem was.
A scriptless automation tool, simply point click, modify attributes in a GUI, and error handling, comparison, etc., all in a GUI smiley Excellent !!
Now, I haven't had the chance to test the tool just yet, but I will as soon as I can, and I'll get back with my thoughts on it.
But still, QTP and RFT are the 2 most commenly used automation tools. They are good tools, but the need the touch of an expert to give you value for your money.
Could we be so lucky that they also in a not so far from now future would be fitted with a usefull GUI that will enable us "non-developers" to do good automated scripting as well ?????

  • Comments(0)//tg2g.wassard.eu/#post4
Next »