After supporting and observing several Proof of Concept activities in various departments, I realized that I could do more to guide a team in how PoCs are planned and executed. Increasing the chances that the activity results into an actually working and productive system.
This document shall collect the lessons learned and it distills a set of quality attributes that we should pay attention to, so that we can guide teams better towards the desired outcome: A productive solution or product.
What are the goals of a PoC and why it should be called differently
The term “Proof of Concept” in the domain of software engineering typically means that a team (or individual) has made an assumption on technology and designed a concept to build a certain solution based on that assumption. Before starting a larger and costlier development this team wants to validate the assumption and so executes a PoC. Optimally, the PoC positively validates the designed concept and so leads to a decision to start the actual implementation and bring it to production.
The more general Wikipedia definition is also a good read: “Proof of concept (PoC) is a realization of a certain method or idea in order to demonstrate its feasibility, or a demonstration in principle with the aim of verifying that some concept or theory has practical potential. A proof of concept is usually small and may or may not be complete.”
So, keeping in mind that the execution of the PoC is mainly a means of risk reduction in developing a larger solution or product, the PoC itself is just a starting point of a longer journey. This is one of the reasons, why many teams struggle with the right scoping of the PoC (where does it end), since the end goal is always a working solution / product. This is why, in the optimal constellation, the artifacts (documentation, code, scripts) that are created to execute the PoC can directly be used to build out the final solution/product and it should not be thrown away.
Software Architects are therefore (sometimes) trained to not use the term PoC and better use “Walking Skeleton”. The metaphor here is about building a large dinosaur and you start with the bones in the form of early validation of assumptions, technology selections and key designs. Once you have enough bones collected to get the skeleton to walk, you can add flesh and skin to the bones and build out the final product / solution.
Quality Attribute #1: We should never think of a PoC of a throw-away activity, it should have product quality code, documentation and shall be executed like actual product development with a backlog, milestones, reviews and a team that has reserved capacity to do the work.
Quality Attribute #2: The goal of an individual PoC activity should be to validate a set of well-defined assumptions. Optimally one assumption per activity.
Elements to consider to increase chances towards production
The base assumption behind each project is, that the team that executes the PoC has a clear picture why that work is required and how it generates value for their business. So business goals and an allover strategy must be clear. In addition, there should be a business owner who sponsors and supports the activity as key stakeholder. Finally, the project that sits behind the PoC activity must have a clear path to market with a business model, a G2M plan and a monetarization concept. You need to validate these business elements and if there are not there yet, work out the elements with the team. Specifically, if there is no clear idea how a product shall go to market or what its business model will be, chances are that even after a successful PoC execution, there is no way forward.
Quality Attribute #3: The PoC lives in a context of a product / solution development that has a clear business plan, a G2M strategy, a business owner as a sponsor and a good picture of the business requirements at current stage.
A typical assumption validation activity starts with a few open question and then generates much more open questions on the way. Or in other words, while the team learns about the new technology or is testing out its key design concepts, they come across new questions they did not think of before. With that, the potential amount of work in the PoC increases naturally. Here it is important to maintain a backlog and note down all work items or questions-to-be-investigated into it – and then prioritize the work again. It is important to time-box the activity from the beginning to keep the pressure up for the team to constantly re-evaluate what needs to be done first to answer the initial question and validate the key assumptions. The learnings from the PoC will help to understand what’s required to build towards production and need to be persisted.
Quality Attribute #4: Assuminga PoC is done in the context of a larger project it must be time-boxed (for good-enough results) to not jeopardize allover product delivery. Elements to work on are prioritized continuously and so scope of work is limited by time and prioritized by value (to help validate the assumption).
Quality Attribute #5: One result artifact of the PoC activity is a backlog of work items that was collected during the time-boxed activity but was not prioritized at the point in time. This serves as input for the next iteration and project phase.
Software architecture and so the decision making that a team is doing around technology selection and concepts is often times around operational qualities like security, performance, scalability, robustness, operations-support and other system quality attributes. So, beside the key functional checks (does it generally work), a set of key system quality attributes should be in scope.
Quality Attribute #6: The scope definition of a PoC should be done around interactive scenarios that are testable. The executed tests shall be measureable with a clear understanding when the test was successful.
It is important here, that the team has defined the criteria that will influence the decision to go forward or stop after the PoC activity. This set of criteria might change during the execution (be agile) based on their value to validate the original assumption. But, the team should have a clear picture on test cases and test acceptance criteria for all test scenarios that are in scope of the PoC.
Quality Attribute #7: Each assumption that is to be validated has a set of executable test scenarios associated and each test scenario has a clear and measureable set of acceptance criteria.
Finally, the team should exercise a brainstorming session about the “what else”. As mentioned before, the team will fill a backlog with open items just naturally while they learn and experiment. Beside the natural path of learning, the team should do a brainstorming session at the end of each iteration (e.g. each sprint) about all other possible risks that might come in based on the current knowledge. The ideas and concerns go into the backlog as research items. Here, latest at the end of the PoC activity, the team should explicitly consider the key operational qualities that would lead to a production ready solution: Build out a full feature set, scaling, make it really secure and robust, clean code and interfaces, document as needed, build out the testing solution, implement operational support (logging, metering, diagnostics, monitoring, updating, …). With this brainstorming, it will generate better understanding what it will take to evolve the walking skeleton towards production.
Quality Attribute #8: The team discussed the elements and work tasks to do reach production quality in order to allow stakeholder expectation management, plan next steps and be clear on what’s important as next step.
The goal of a PoC activity is to reduce risk in building a product / solution. When it is clear what the decision criteria is and what else needs to be done, it is straightforward to make decisions. Also, when all team members and senior management have this information, it makes conversations about the scope, goals, results and next steps much easier. After the allocated time has elapsed, there should be a result documentation / presentation that is discussed with all stakeholders about the base assumptions, key learnings, remaining backlog and so remaining risk and work, test results for the scenarios and allover decision GO or NO-GO.
Quality Attribute #9: Each PoC activity shall have a scope document and a result document that is aligned around the assumptions to be tested, the test scenarios and the outcomes. This shall be shared with all stakeholders during a) a Kick-Off meeting and a b) Results & Decision meeting.
One additional element in the discussion around next steps should be around the G2M. While the technical side of evolving the skeleton towards a fully grown dinosaur, we should have or develop a clearer picture of how the dinosaur is brought to market and sold.
Summary
Here is a list of quality attributes you might to check for in your next PoC activity:
- There is a business owner who sponsors the activity and has a business interest
- The team is clear about the business requirements and is clear on the business context
- There is a clear goal for the surrounding project to go to market and how its monetized (business model)
- A PoC shall be executed in production quality and in an iterative, incremental process
- All PoC artifacts shall be crafted so that they can be used directly to build a final solution
- Make sure the team has the required capacity for execution
- The scope of a PoC shall be around the validation of well-defined assumptions
- The scope of work is done around interactive scenarios that are testable.
- The executed tests shall be measureable with some clear test acceptance criteria
- A PoC is time boxed and time is not extended
- Elements to work on are prioritized continuously and prioritized by value
- A backlog is used to collect all new ideas, risks and questions that rise during the learning
- The team discussed the elements and work tasks to do, to reach production quality
- Each PoC activity shall have a scope document and a result document which is agreed with all stakeholders
- You have 2 meetings with Stakeholders at minimum: a) a Kick-Off meeting and a b) Results & Decision meeting.
- Optimally you work in sprints, including sprint reviews with stakeholder