Experimentation supports the culture of learning and innovation. It is important that we continuously explore and validate hypotheses in small batches, for example to determine the effect of people, process or product changes, or to determine the effect of framework or architecture improvements.


This is one of my rambling and reflection posts. It's based on observations and random thoughts that are intended to provoke a disccussion, not to be used as a formal definition or guidance.

Why are "continuously" and "small" in bold?

It is important to instill and reinforce that we are on a journey of continuous learning and improvement, with a destination we will never quite get to. It is also important that we are empowered to focus on a small number of work in progress experiments which fit within our heartbeat (cadence). Most of us have a limited capacity, degrade with context switching, and need to prioritize adding value and delighting our customers.

Do not try a big-bang, one EXPERIMENT to rule them all, or boil-the-ocean transformation approach!

Example Experiments

We recently ran an EXPERIMENT to determine when we should scan for vulnerabilities using WhiteSource Bolt and how to tokenize your VSTS pipeline the easy way!.

The EXPERIMENTATION and culture of learning is pivotal for growth and positively affecting our organizational culture.

Typical On-boarding Example Experiment(s)

When on-boarding new engineering teams to their collaboration solution, we are faced with common questions that warrant EXPERIMENTATION:

  • Should we use Scrum or Kanban?
  • Should we break down our work into User Stories and Tasks?
  • Should we focus on planning, development, build, release, and/or monitoring?

Learn from others

It is important to share and learn from other research, for example the DevOps at Microsoft journey. Although dated, the Managing Agile OSS Projects with Microsoft VSO VSTS eBook gives an insight into how the ALM | DevOps Rangers EXPERIMENTED with Scrum, Scaled Scrum, and Kanban.

Understand the fine line between alignment and autonomy

Engineering teams must understand and align with the business definition of organization, roles, teams, cadence, and taxonomy. For example, a common cadence (sprints, iterations), creates a mutual heartbeat, which helps the organization and engineering teams to plan, track, and collaborate more effectively.



As an engineering team, it is important to continuously EXPERIMENT and innovate. Here's a few thoughts on some of the questions, which you need to answer within your context through EXPERIMENTATION and learning.

  • Should we use Scrum or Kanban?
    EXPERIMENT with Scrum if you need to ship value on a regular, time-boxed basis. If you have a continuous stream of tasks that need to be worked on concurrently EXPERIMENT with Kanban. Some teams may be more comfortable and productive with one, while others may adopt both. It's worth an EXPERIMENT to decide what works best for your team.
  • Should we break down our work into User Stories and Tasks?
    The user story describes a unit of work (HOW), including the acceptance criteria, linking back to the business requirements (WHAT, WHY). Some teams break their user stories down into detailed tasks, some use tasks as a lightweight TO DO list, while others do not use them at all. It is worth another EXPERIMENT to determine which strategy *works best for your team.
  • Should we focus on planning, development, build, release, and/or monitoring?
    It feels natural to also EXPERIMENT with planning. The sequence and concurrency of other EXPERIMENTS depends on the maturity of the engineering team and recommendations from self assessments.

You will notice that EXPERIMENT is mentioned a few times in this post. EXPERIMENTS are important to validate hypotheses and guide us on our infinite journey of continuous learning and improvement.


  • Is pivotal for continuous innovation and learning
  • Needs investment and support from everyone
  • Needs a culture of fail fast, without fear of retribution

Do you agree?