Continuous (Pre) Planning

Introduction

In this post I want to introduce the concept of continuously planning. A process flow that we established in my team.

Pre Planning – Short Description

One of the key meetings in a scrum cycle is the Pre-Planning.

Usually it’s a long meeting and after that, there are hours of writing subtasks and time estimations.

In this meeting the team members understand the user stories, challenge the PMs and do estimations. Later the developers will decide on solutions and break them into subtasks.

Background About the Team

Team Structure

My team is a cross-functional one. I have frontend and backend developers, as well as more DevOps oriented engineers.

It means that developers sometimes have independent tasks, which are related only to one or two of them.

Products We Own

We are responsible for three major products, and some minor ones.

In one sprint we work on different products. Those products are usually unrelated to each other. Meaning independent tasks.

Team Culture

In the beginning the team was small, as well as the company.

We had a “startup culture”.

Deliver fast, no planning, no estimations.

People hate long, boring meetings. Feel waste of time.

The Problem

As it turned out, there were many problems in the way we tried to work.

There were common issues that were raised in our retrospectives, as well as my personal observations.

Retrospective

Here’s a list of the main issues we noticed

  • During planning, there were long discussions that were relevant to one engineer and the PM (as result of different products and cross-functional team). People were bored and lost concentration
  • Many user stories were not even started because we discovered that the PMs did not clarify them. We discovered that in the middle of the sprint.
  • User stories were not done because lack of planning
  • Overestimated / underestimated sprints content
  • Lack of ownership. Features were not done because dependencies issues were not thought in advance.

Solution – Continuous Planning

At some point we started doing things differently. After several iterations, it became the continuous planning.

The idea was simple. I asked our PM to provide user stories of next Sprint when current Sprint starts.

Then I checked them and assigned to the relevant developers or pushed back to the PM.

The developers started looking at the user stories, understood them and added planning using subtasks and time estimations.

So basically we had almost two weeks to plan for the next sprint.

How It Works

User Stories are ready in advance

The crucial part here, is the timing, when are the user story ready?

Let’s say the Sprint duration is two weeks and it starts on Wednesday.

The idea is simple: On Wednesday, when Sprint 18 starts, then PM should already provide all user stories for Sprint 19.

By provide I mean have them:

  1. In backlog, under Sprint 19 (we use JIRA, so we create sprints in advance)
  2. Well defined (Description, DoD, etc.)

First Check

The user stories are assigned to me and I go over them. I verify that they are clear with proper Definition of Done. I am identifying dependencies between the user stories. Then I either assign the user story to a developer or reassign it to the PM, challenging about the content or priority.

Discovery and Planning By Each Engineer

From now on, each engineer, during current Sprint, will start planning the assigned user stories for next sprint. He will talk directly to the PM for clarifications and will identify dependencies.

He has full authority to challenge the user story for not being clear. He can reassign it back to me or the PM.

Collaboration

If there is dependency within the team, like FE and BE, then the engineers themselves will talk about it and assign the relevant subtasks in the user stories.

Estimations

The estimations are part of the planning.

Each developer will add his own subtasks with time estimation.

Timing Goal – It’s All About “When”

The key element for success in this process is timing. Our goal is that all user stories for next sprint will be fully provided in the beginning of current sprint.

We also aim to have all user stories planned (subtasks + estimations) 1-2 days before next sprint starts.

Observations and conclusion

Our process is still improving. Currently around 10%-20% of the user stories still come at the last minute, violating the timing goal.

We also encounter dependencies, which we didn’t find while planning.

Here’s a list of pros and cons I already see

Pros

Responsibilities and Ownership

One of the outcomes, which I didn’t anticipate, is that each developer has much more responsibility and ownership on the user stories.

The engineer must think of the requirement, then design and find dependencies, besides just the execution.

On boarding new team members

New team members arrived and had a user story assigned to them on their first day in the office. So “they jumped into the cold water”, and started understanding the feature, system and code almost immediately.

Collaboration

As each team member is responsible of the entire feature, it increased the collaboration between the team members. There is constant discussion between the team members.

It has also increased the collaboration and communication between developers and PMs.

PM Work

As the PM works harder (see cons), the continuously planning forces him to have better planning ahead.

This process “forced” the PM to have clear vision of 3-4 sprints in advance. This clear vision is transparent to everyone, as it is reflected in the JIRA backlog board.

Visibility and Planning Ahead

The clear vision of the PM is reflected in the backlog (JIRA board in our case), make it more transparent, As the board is usually filled with backlog, which is divided to sprints the visibility of future plan is much better.

Challenges (cons)

PM Work

It seems that the PM has more work. User stories should be ready in 1-2 weeks in advance.

The PM needs to work on future sprints (plural) while answering questions about next Sprint and verifying current sprint status.

Questionable Capacity

When there is a dedicated meeting / day for the preplanning, it’s easier to measure the capacity of the team.

It’s harder to understand the real capacity of the team while the developers spend time on planning next sprint during current one.

Architectural and design decisions

Everyone needs to be much more careful in architectural and design of the system. As each developer plan his part, he needs to be more aware of plans of other developers.

This where the manager / lead should assist. Checking that everyone is aligned and make sure there’s good communication.

Lose Control

The lead / manager has less control. Meaning, not everything passes through him.
If you’re micromanager, you will need to let go.

We identified points were the manager (me) must be involved.

  • Dependencies within the team and / or with other teams
  • Architectural / design decisions
  • System behavior

Conclusion

We established a well understood, simple to follow, clear process.

This process is good for our team. It may be good for other teams, perhaps with some adjustments.

As described above, if

  1. There are different roles in the team (frontend, backend)
  2. The team works on different products / projects in the same sprint
  3. People feel that the pre planning meeting is a waste of time

Then perhaps continuous planning is a good approach.

This post was originally published in our company's tech blog:
 http://techblog.applift.com/continuous-pre-planning

Linkedin Twitter facebook github

The Foreman Role in a Team

There is a lot of discussion about the need for a foreman role in a software team.
Robert C. Martin wrote about it in Where is the Foreman?

I recently read a post by David Tanzer who disagrees with Uncle Bob’s point: We don’t need a foreman

The way I see it, a foreman role is important, but perhaps not as extreme as Uncle Bob describes.
Let’s start by quoting the foreman’s role as Uncle Bob describes (in construction and in software):

The foreman on a construction site is the guy who is responsible for making sure all the workers do things right. He’s the guy with the tape-measure that goes around making sure all the walls are placed properly. He’s the guy who examines all the struts, joists, and beams to make sure they are installed correctly, and don’t have any significant defects. He’s the guy who counts the screws in the flooring to make sure it won’t squeak when you walk on it. He’s the guy — the guy who takes responsibility — the guy who makes sure everything is done right.

What would the foreman do on software project? He’d do the same thing he does on a construction project. He’d make sure everything was done, done right, and done on time. He’d be the only one with commit rights. Everybody else would send him pull requests. He’d review each request in turn and reject those that didn’t have sufficient test coverage, or that had dirty code, or bad variable names, or functions that were too long. He’d reject those that, in his opinion, did not meet the level of quality he demands for the project.

I think that the commit rights is crucial point for many critics of this idea.

I would like to suggest a middle way.
A foreman role, but also team’s responsibility.

In real life the team is diverse. Some are seniors, some juniors. Some are expert in a specific field, others in another field. Some are expert in TDD, some in design, some in DB and SQL etc.

Here are the key points as I see it:
A diverse team.
A foreman who’s responsible for the quality and delivery.
The foreman will sometimes make the final call. Even if there is still a disagreement.
The foreman is also responsible that everyone works at the standards he introduced (with the help of the team).
Everyone can commit. Not just the foreman.
Everyone can suggest anything.

Here are some reasons why it might work.

  1. Everyone can commit and everyone can see others’ commits. This means that there is trust between the team members. It also gives each member more responsibility. The foreman in this case will still look for all the things that Uncle Bob describes. But when he sees something wrong (missing test? A code that is not well designed?) then he will approach the person who committed the code and discuss what went wrong. The foreman will have an opportunity to mentor other team members and pass his knowledge.
  2. The foreman can be the peer, with more responsibilities. If Fred notices that people make mistakes, he will discuss it. The foreman has more responsibility. He needs to know to listen. He needs to explain and not blame.
  3. The foreman does not have to be the most experienced developer in everything. He can’t be. He may be most experienced in one or two or three fields. But not all. So if Alice is the most experienced DB developer, Fred the foreman should see that she helps other team members with SQL related stuff. He will still remind Alice about the procedures and code of the whole system.
  4. Sometimes the foreman will need to make decisions. Sometimes not everyone will agree. The foreman needs to know when to stop an argument and give the call.
  5. The foreman doesn’t need to have the sole responsibility for quality. But he’s the one that the management should approach. This is a tricky part. It’s hard to achieve. The team is responsible for the quality and delivery of the code. The foreman is responsible that the team achieve this. The foreman is responsible that everyone practices good coding (and everything that implies). The foreman is the one who needs to mentor team members who do not know how to bring quality code.
  6. The team is responsible for the architecture and design. As I mentioned before, the foreman will sometimes need to stop the discussion and make a decision. Each member of the team should have the opportunity to come forward with suggestions. Sometimes the foreman will bring the best ideas (after all, he’s supposed to be the most experienced), but more than once, other member will introduce the correct design. The foreman needs to listen.
  7. During the planning the team will estimate effort (E.g. will give points to user stories). Then, if the whole team is responsible for the design and architecture, the members will create the tasks with some time estimations. The foreman’s responsibility would be to see that everyone understands the priorities. He should be part of the team while designing and lead the design. If the team did not understand the priorities and didn’t bring quality code, it’s his responsibility. But also the team’s.
  8. The foreman should introduce new technologies to the team. The foreman should introduce the team coding practices. The foreman must pair with other members. Juniors and seniors. While pairing with juniors he actually mentors them. The foreman must see that the team does pair-programming with each other. The foreman is the one that establish code review habits in the team. As a foreman he can ask to review code even it was already done by another person. Sometimes it brings some antagonism, but as mentioned before, he has the responsibility and he’s the one that needs to answer the management.

Uncle Bob suggested a rather extreme approach.
Perhaps it suits in some extreme cases. He describes an open source project that actually work with several foremen: A Spectrum of Trust
On the other side, David Tanzer shows correctly why this approach may deteriorate the team spirit and trust.

I think that it’s possible to have a middle way.
I think that a team can have a foreman, a person who’s in charge. But still let everyone be involved. Have trust, spirit and motivation.

Linkedin Twitter facebook github

Agile Mindset During Programming

I’m Stuck

Recently I found myself in several situations where I just couldn’t write code. Or at least, “good code”
First, I had “writer’s block”. I just could not see what was going to be my next test to write.
I could not find the name for the class / interface I needed.
Second, I just couldn’t simplify my code. Each time I tried to change something (class / method) to a simpler construction, things got worse. Sometimes to break.

I was stuck.

The Tasks

Refactor to Patterns

One of the situation we had was to refactor a certain piece in the code.
This piece of code is the manual wiring part. We use DI pattern in ALL of our system, but due to some technical constraints, we must do the injection by hand. We can live with that.
So the refactor in the wiring part would have given us a nice option to change some of the implementation during boot.
Some of the concrete classes should be different than others based on some flags.
The design patterns we understood we would need were: Factory Method and Abstract Factory
The last remark is important to understand why I had those difficulties.
I will get to it later.

New Module

Another task was to create a new module that gets some input items, extract data from them, send it to a service, parse the response, modify the data accordingly and returns items with modified data.
While talking about it with a peer, we understood we needed several classes.
As always we wanted to have high quality code by using the known OOD principles wherever we could apply them.

So What Went Wrong?

In the case of refactoring the wiring part, I constantly tried to immediately create the end result of the abstract factory and the factory method that would call it.
There are a-lot of details in that wiring code. Some are common and some needed to be separated by the factory.
I just couldn’t find the correct places to extract to methods and then to other class.
Each time I had to move code from one location and dependency to another.
I couldn’t tell what exactly the factory’s signature and methods would be.

In the case of the new module, I knew that I want several classes. Each has one responsibility. I knew I want some level of abstraction and good encapsulation.
So I kept trying to create this great encapsulated abstract data structure. And the code kept being extremely complicated.
Important note: I always to test first approach.
Each time I tried to create a test for a certain behavior, it was really really complicated.

I stopped

Went to have a cup of coffey.
I went to read some unrelated stuff.
And I talked to one of my peers.
We both understood what we needed to do.
I went home…

And then it hit me

The problem I had was that I knew were I needed to go, but instead of taking small steps, I kept trying to take one big leap at once.
Which brings me to the analogy of Agile to good programming habits (and TDD would be one of them).

Agile and Programming Analogy

One of the advantages in Agile development that I really like is the small steps (iteration) we do in order to reach our goal.
Check the two pictures below.
One shows how we aim towards a far away goal and probably miss.
The other shows how we divide to iterations and aim incrementally.

Aiming From Far

Aiming From Far


Aiming Iterative and Incremental

Aiming Iterative and Incremental

Develop in Small Incremental Iterations

This is the moral of the story.
Even if you know exactly how the structure of the classes should look like.
Even if you know exactly which design pattern to use.
Even if you know what to do.
Even if you know exactly how the end result should look like.

Keep on using the methods and practices that brings you to the goal in the safest and fastest way.
Do small steps.
Test each step.
Increment the functionality of the code in small chucks.
TDD.
Pair.
Keep calm.

Refactor Big Leap

Refactor Big Leap


Refactor Small Steps

Refactor Small Steps