Agile design and development principles meet enterprise IT infrastructure: The irresistible force meets the immovable object!

From enterprise data warehouses to OLAP cubes to Cassandra and mongo dB analytics platforms, we are not in Kansas anymore. Agile teams and practices are being pushed and prodded and pleaded and threatened by business units with unrealistic time to market needs, CIOs with enterprise class legacy system obsolescence and frustrated and overwhelmed architecture VPs – how do we apply agile principles and practices to these super-sized work efforts?

Do 2 week iterations apply to our work, or can we make them longer?  How much documentation is appropriate for this class of work?  How do we create vertically slices stories that can be implemented in iteration?  How do we test these technical or infrastructure stories?  How do you express infrastructure work as business user stories, and how should the narrative be written? How can we support iterative product development and still design our platforms and services for high availability, scalability, maintainability and reasonable TCO (total cost of ownership)?  The questions never stop, and these challenges are real and these folks need help; and they need it now!

Product management leaders and executives want to understand how agile methods integrate with traditional UCD, UX, Ideation and Discovery work.

QA leaders want to understand the agile approach to automated unit testing, manual and automated functional testing, integration testing, stress and load testing and user acceptance testing.  And how do all these levels of testing work together, what should be completed in a specific iteration, and who owns what anyway?

The life of an enterprise agile coach or agile program manager is never boring, and it is rarely simple.  In the coming weeks this blog will explore practical thoughts and solutions to address these “upper division” agile topics and challenges.

Cheers and Happy Monday,

Paul

Technical Debt: Pay it now, or pay it later!

The concept of software complexity and deferred refactoring as debt was originally coined by Ward Cunningham in an experience report for OOPSLA ‘92. It has since become an industry wide concept and is a commonly referenced and applied when practicing scrum and agile.
Technical debt is deferred work not directly related to new user story features, but necessary for the overall quality of the system. Examples of technical debt include delaying upgrades to necessary tools and frameworks, delaying the refactoring of overly complex or crude system components, etc. etc.

The problem with technical debt is that it will reduce the velocity of the team, and the quality of the software produced on a greater basis the longer it is ignored and / or deferred (just like only making the minimum payment on a credit card debt).
So now that we have an understanding about technical dept, and the problems caused by it, we can discuss an approach to solving this age old application development and maintenance issue.

Since Scrum teams are supposed to operate in a transparent manner, and the product owner is a full partner with the scrum team, the details of the technical debt that exists and is accruing in a product needs to be visible to and understood by the product owner.
Key technical debt items need to exist in the product backlog. Additionally, the ongoing impact to product quality and scrum team velocity by continuing to defer their resolution needs to be clear to the product owner so that the product owner may rightly prioritize these items in the product backlog.

Additionally some team adopt the notion that in each iteration or predefined number of iterations, a technical debt tax of some number of story points will be included during the sprint planning process. This assures that the most important technical debt items will be addressed when they exist in the backlog, and hopefully the “technical debt balance” will never get to the point where it takes too big of a bite out of a particular iteration or set of iterations.

Also, once the concepts of identifying technical debt and continuing to pay it down on a regular basis are in place, the scrum team becomes very sensitive to these issues, and very cleaver and effective in how it deals with them.

Therefore consistent and thoughtful application of these best practices associated with the management of technical debt will result in better software quality, improved team velocity, a happier and better informed product owner and a less stressed out scrum team! All in all, not a bad deal really.

As always, Paul

Scrum 201: Mastering agile development and continuous integration and testing processes.

As scrum teams develop and mature, it becomes very clear that some aspects of agile and scrum are much easier to learn and master then others.

Sprint Planning, the daily stand up, and writing and sizing user stories, etc. can be learned and well practiced by teams within a reasonable number off sprints.

Other concepts and skills required to effectively implement scrum have a longer learning curve and require substantial changes in development and testing approach, technique and what is considered possible or normal!

This issue cuts right to the heart of what is different between waterfall and agile, and even requires smarter, more creative developers and testers that are actually will and able to communicate the details of their approach, work and progress to their team members on a regular basis.

To further complicate things, many large enterprise computing platforms have become very complex with many loosely to not so loosely coupled tiers.  (OS, DB, App server, Web Server, BUS or Information Broker message routing, Security brokers, WSDL’s and other web services components, etc. etc.).  The number of tiers and the complexity associated with their end to end execution can overwhelm and intimidate many testers, and make it more difficult for them to understand how to approach functional testing before the entire solution is delivered in its entirety.

Success in these environments requires more sophisticated tools and test harness software for automated regression testing, web services testing, etc.

Patience, technical leadership and a real understanding by the team that we “succeed as a team, and fail as a team” is required before real progress can be made.   Testers, developers, designers and architects must work effectively together to devise effective “white box” tests that can chip away at the validation of the overall solution while some or many components are still under development and not yet delivered.    This stuff is the heavy lifting required by scrum team to significantly improve velocity once the low hanging fruit has been picked.  In future posts I will share some techniques I have appreciated to help teams accomplish these goals.

 

End of Iteration Demos: In Scrum the measure of progress is working software!

Modern Project Management defines project progress in many interesting, complex, sometimes useful and sometimes erroneous ways.  From earned value to estimate to complete there is no shortage of thoughts on this always pertinent project management topic.  One of the many simple and effective aspects of the Scrum process is that the key measure of project progress is measured by working software.  This may seem as basic as to almost be taken for granted.  Nevertheless it is remarkable what clear focus can produce, and what a lack of clear focus also produces.

Therefore sprint progress is continuously evaluated thru ongoing software demos by the scrum team.  This simple and transparent activity eliminates surprises, facilitates user story acceptance and provides immediate feedback from the product owner and SME’s to the scrum team.

There are many practices associated with ongoing sprint demos, but the following best practices have been successful in the past and are worth considering:

  • Periodic and / or weekly demos:  Having a re-occurring and predictable demo schedule helps the whole team focus on what is being produced each period, as well has helps the product owners keep up with what is being produced and helps them and the team with the acceptance process.
  • Demo scripts and dry runs: For each sprint scenario, the team develops a step by step script (like a script in a play) then will help define all aspects of the final iteration scripted demo. These scripts will include test the details of the test scenarios, each step in the processing and checking in the demo, who will execute the demo, etc. etc. In preparation for the final demo, the team will perform “dry runs” or rehearsals, just like practicing for the opening night of the play.  These dry runs are performed as many times as required, until they are completed end to end without any errors or issues.
  • Demo Day: Finally the big day has arrived and the excitement that the whole team feels the day of the demo is palatable.  In fact the “opening night” analogy is a good one!
  • Celebrating a successful Demo:  After a successful demo, the team is in a mood to celebrate, and celebrate they should!  Whether it is a pizza lunch, a group hug, dinner offsite or what ever, nothing helps the team building and bonding process like celebrating a team success together as a team. This is very cool and should not be overlooked or minimized!

Nothing breeds success like success and nothing builds credibility and trust with product owners and stakeholders like working software! It is so cool seeing the faces and reactions of product owners as they begin to consistently see working software produced demo after demo and sprint after sprint.  Before long these product owners become the biggest supporters of the scrum team and the scrum process. As they adjust to the rhythm of a sprint (sprint planning, daily stand ups, periodic demos, dry runs and then demo day), they become super productive and much more confident and satisfied with their own role and the results produced for their efforts!

Cheers and regards, PR