Tricks or Treats? Depends on Cost Benefit Analyses

As you probably observed weekend, maximizing treats and minimizing tricks is a primary objective of the annual Halloween ritual; as is the case when making decisions. A properly performed Cost Benefit Analysis (CBA) cost benefitmaximizes the high value information you take in and minimize the low value information to increase the odds of making sound decisions, which can lead to subsequent achievement of desired outcomes. If not handled correctly, CBAs can cause poor decisions that can turn your financial house into a house of horrors.  Having a credible, sound CBA can prevent decision makers from holding a bag full of rocks, like our friend Charlie Brown had to endure.

Based on our experiences, here are seven considerations and best practices for CBAs.

Best Practices

  1. “Big Picture” for clarity – The late Yogi Berra once said “If you don’t know where you’re going, you’ll wind up somewhere else.” To that end, it’s critical to start from 10,000 feet up to grasp the big picture of the CBA. A sound CBA communicates immediately identifies the main parts of the action or decision that is the subject of the CBA. The strategic business objectives the project aligns to, the reason for the CBA, and the selection criteria and their relevance to the purpose of the CBA need to be sufficiently considered, defined, and specified before identifying costs and benefits.
  2. Consistent and accurate Time Line – I’ve performed CBAs that look into the future across weeks, years, and even a couple of decades. Stakeholders need to see the timing and variation of the individual benefits and costs across the time period. Further, the period of analysis should be consistent among alternatives, and discrepancies clearly and fully reconciled and explained. When results of the CBA are seen along a timeline, decisions can be better made to maximize ROI through cost reduction, cost postponement or avoidance, gain increase, or acceleration.
  3. Breadth and depth of assumptions and methods – Numbers alone don’t tell the full story. Decision makers and stakeholders need to see the assumptions and methods, which is the basis of estimate, behind the CBA. They need to be able to discern the data and the manner in which it was collected, normalized, and examined, to ensure proper meaning.
  4. Comprehensive inclusion of benefits and costs – Many CBAs lack validity or creditability because they omit important benefits or costs from the scope of the study. So called “soft benefits” such as strategic value, morale, or customer experience can be more difficult to defend, to the extent that many don’t include many of them in the study. This can be a mistake. Since they sometimes are more applicable or meaningful in a study. The key is to quantify them using a rationale, systematic scale for measurement and tangibility. Also, implementing the scale thoughtfully through a carefully composed IPT can help ensure meaningful results.
  5. Does the CBA discuss critical success factors? – Assumptions identified during the study that can be managed or influenced to some extent are critical success factors (CSFs), which affect the results of the selected project or alternative. It’s best to move beyond simply stating the obvious (“benefits exceed costs”) and include “recommendations” that identify the CSFs to manage through implementation plan.
  6. Are risks and opportunities identified and quantified? – Point estimates (i.e. most likely outcomes) are rarely reliable. Therefore, it’s critical to assess a range of potential outcomes. Further, a deeper understanding of how a change in the values of specific drivers (i.e. market size, software size, economic trends) can impact the overall estimate is needed. Because these assumptions are all uncertain to some degree, performing sensitivity and risk analyses can yield greater insight into the uncertainties. Sensitivity analysis reveals which assumptions are most important in impacting overall results; risk analysis reveals the probability of other outcomes.
  7. Is the estimate documented and portrayed clearly and fully using appropriate tools? – A well-documented cost estimate is a characteristic of high-quality CBAs. To ensure a CBA can be easily updated and replicated, it is important to create a basis of estimate (BOE) that defines the source data used and their significance, the calculations performed and their results, and the rationale for choosing a particular estimating method or reference. There are many commercial tools available in the marketplace to facilitate the development of the estimate, including MS Office and ACE IT. More information on cost estimates can be found on the GAO website.

What tricks or treats have you experienced with CBAs?

What Is Quality and How Is It Achieved In a Service Environment – Part Two

In Part I of my blog, I discussed the need to be able to measure performance to ensure  the product performs as required and processes are functioning properly to correct prior deficiencies.  I teed up the challenges associated with measuring performance in a service environment with the simple example of a restaurant dining experience.  The issue is – when there are no clear objective performance criteria to measure, how does one identify, assess, and measure the subjective criteria?   In this part, I’ll talk about two key factors that can help overcome the challenges and contribute to achieving quality in a service environment.  These factors are communication and relationship building.  First, let’s start with some important questions to ask around an illustrative scenario.

What are the important aspects of performance you want to measure and are there ways to measure them?  I’ll use an example pertaining to professional services to frame the challenge.  My customer hires me to pcommunication for blog postrepare and provide reports assessing the progress of sub-Components in completing tasks for which he has provided funding.  Pretty simple for him – he gives another government office money to do something and he gives me money to report the office’s progress to him monthly.  How do I know what constitutes a quality product on my part?  How has he defined and quantified that in my contract?  Do my reports have to comply with a specific format?  Am I allowed only a certain number of grammatical or spelling errors per page?  It’s very possible that I can comply with all observable and quantifiable requirements and still not deliver a product that meets his needs.

Communication is a key factor of success in this case.  How well has he communicated the requirements to me and how well have I responded to those requirements.  The communication needs to be constant, open, and positive.  What do I mean by this?  Webster defines communication as a process by which information is exchanged between individuals.  Our communication must be constant – we should continually be exchanging thoughts and ideas so that I have better insight into what’s important to him.  Our communication about the topic at hand must be open and honest, based on mutual trust..  And finally, our communication must be positive so that the process doesn’t break down.

We leverage our open communication to build our relationship.  We should strive to treat the business relationship like a partnership.  Granted, we each have our own goals that may initially seem incompatible, but through open communication, we will be able to find that overlap area that allows us both to succeed.  Both sides must be focused on mutually achieving their goals and willing to work together to do so.  An adversarial relationship or a “gotcha” environment will not foster open communication and will likely not lead to an ongoing partnership.

With that partnership established and open communication practiced, the customer is able to explain what she needs to be successful.  I’m able to describe what I will provide that meets her needs, complies with the terms of our contract, and allows me to meet my goals as well.  After I’ve delivered my product, we’ll go through it to ensure it meets her needs.  We’ll do interim updates and frequent status checks to facilitate delivery of an on-target product.  If we don’t get it right the first time, through honest feedback and a positive approach, we fix it, together.  She will provide regular feedback and I will adjust accordingly.  With communication and collaboration throughout the activity, my understanding of what quality means to her grows, and I’m better able to deliver that high quality product.  (And going back to Part I – she’ll come back to my restaurant!)

This sounds easy, and it’s easy to describe here.  But it’s not often easy to achieve.  But if both parties to the contract are willing to truly work together and establish this approach, they will find that their chances of meeting their goals and objectives (mutual and individual) are drastically enhanced.

What Is Quality and How Is It Achieved In a Service Environment? – Part One

Quality“Quality” may be one of the more over-used and least-thought-about words in our business vernacular.  We’ve gone through cycles where we used different terms to characterize our attempts to “do” quality better.  Remember Total Quality Management?  How about the focus on ISO 9000?  And more recently, “Six Sigma,” “Lean,” and “Agile” have emerged as concepts to address shortfalls in performance and the inability to consistently deliver high-quality capability.  Through this evolution, we’ve continued to pursue methods to reduce mistakes and track process enhancements to ensure we’ve fixed the problem.  We’ve figured out we need to quantify and measure what we’re doing so we can confirm the changes we’ve made to address problems, have actually worked.

In the federal space, there is an increased emphasis on writing more Performance Work Statements (PWS) and Quality Assurance Surveillance Plans (QASP) into contracts as organizations attempt to shift focus from “how” to “what.”  Instead of a Statement of Work that stipulates how a contractor will perform the required task, a PWS defines the product or services to be delivered and gives the contractor latitude to find a better way to provide the product or service.  The QASP documents the government customer’s approach to measuring key aspects of contractor performance.  This sounds great in theory and has worked well where performance can truly be measured.

But how do you measure performance in a service environment?  There are metrics that can be tracked, but do those metrics directly relate to the overall quality of the service?  When you go to a restaurant, how do you assess the quality of your dining experience?  Was the food tasty?  Was the meat cooked to the right temperature?  Was the service prompt and courteous?  Did the waitress keep your water glass full?  How expensive was the meal?  How long did you have to wait to be seated?  Some of these factors can be measured directly.  Some are yes/no questions.  Some are measurable on a sliding scale.  Now let’s double those criteria.  How did your dinner partner rate those same service features?  These all factor into your assessment of the dinner and your ultimate decision to return to that restaurant again.

These are just some performance criteria that factor into a decision on which restaurant to choose for dinner.  Now let’s take it one step further.  How do you measure performance when those services are professional services an agency needs to support its day-to-day operations?  This quickly gets more complicated than assessing a restaurant’s performance in providing a dining experience.

What are the important aspects of performance you want to measure and are there ways to measure them?  Stay tuned for Part II of this blog, as I’ll use a simple example to frame the challenge and talk about two key factors, namely communication and relationship building, which contribute to achieving quality in a service environment.

Seven Steps to Implement Standardization in Contract Management

seven steps photoIf you want to make your contract management more efficient and see greater returns on your investment of time and effort, standardization could be the solution you’re looking for. Contract management systems and technologies, standardized contract language, templates, best practices, and lessons learned can yield positive, measurable results in effectively and efficiently crafting, administering and managing contracts.

In Part One of this series, I discussed the benefits of standardization and debunked some common misconceptions. In Part Two, I will discuss the importance of maintaining flexibility and offer some tips for implementation. [Read more…]

How Standardization Can Improve Contract Management

standarization photoCan you think of a time when you said to yourself, “there’s got to be a better way to do this?” I know I can. We’re always looking for ways to make our business practices and our lives more efficient. Standardization provides an answer. It can help increase productivity and efficiency by defining expectations, formalizing processes, and creating accountability.

In government contracting – crafting, administering and managing contracts – standardization can be implemented through contract management systems and technologies, standardized contract language, templates, best practices, conducting and applying lessons learned studies, deliverable acceptance and receipts, invoice reviews, etc… Doing so can yield positive, measurable results with effectiveness and efficiency.

At the National Contract Management Association (NCMA) 2015 World Congress, I will lead a presentation on driving efficiency through flexible standardization in contract management. In this first of a two-part series, I’ll look at how standardization and innovation can work together, as well as at the benefits and myths of standardization in contract management. In Part Two, I’ll discuss why it’s still important to be flexible and what to do before, during, and after the implementation of a standardization solution. [Read more…]

EVM – Have You Communicated Its Value to Acquisition and Project Management?

EVM Communication 1Being able to efficiently and effectively diagnose the health of an organization’s portfolio of contracts takes time and persistence. Earned Value Management (EVM), a project management technique for measuring project performance and progress, makes it possible. However, those in the federal sector, including EVM experts, need to do more to communicate its value for increased adoption.

EVM is a novel or complex concept for many stakeholders across the acquisition life cycle process to fully understand, especially technical or non-acquisition personnel, sometimes even within the acquisition and program management communities themselves. However, this lack of understanding is to the detriment of all who have a tremendous stake in the acquisition process, including how to ensure they are getting value for their investments, receiving the capabilities or functionality they’ve paid for, and in a timely way. EVM as a best practice is not only a requirement for many contracts, but a valuable business tool to assess the health of projects, contracts, and portfolios, in addition to enhancing management decisions.

How to help others understand and appreciate the value of EVM is a topic I and my colleague Bill McMahon will tackle in a presentation at the 2015 EVM Practitioners’ Forum Training and Symposium. Set against the backdrop of a particular case, we examine our journey to help a Health and Human Services (HHS) acquisition team, including those stakeholders who may not consider themselves to be part of an “acquisition” team, but are instrumental to successful delivery of project and mission outcomes. Specifically, we talk about how we’ve partnered to establish and use a framework, including policy, consulting, analysis, reporting, training, and tool implementation to communicate EVM information to key medical/technical stakeholders (those that don’t speak EVM or project management for that matter) to facilitate contract portfolio management and decision making. [Read more…]

Acquisition Workforce Training – Rethink Your Approach

Rethink Acquisition Training imageFacing an aging workforce, the question of how to train the next generation of acquisition professionals is a challenge that federal leaders and industry managers have yet to solve. The need for training is great, but budgets and time are often limited.

Senior government procurement executives and practitioners say agencies have cut training dollars even though employees with less experience now have responsibilities that exceed their abilities, according to the Professional Services Council’s (PSC) 2014 Acquisition Policy Survey. And some said those who do receive training may just be “checking the box” with courses that don’t test critical thinking or include innovative practices.

So, how do you face the conundrum that experiential or case-based training, the very type that many people need, can seem cost-prohibitive given tight budget constraints? Integrity Matters talked with Mike Ipsaro, Technical Director at Integrity Management Consulting, who has developed and delivered training for multiple agencies in a wide range of formats. We asked him about how to deliver training that is cost- effective while also being customized. Ipsaro says modular delivery, simulation, and adaptation to emerging or customer-specific needs are all key. [Read more…]

WHY CPIC Matters More than Ever to Cybersecurity

CPIC benefits to Cybersecurity imageFederal CIOs are on the hot seat over cybersecurity after the revelation that the personal information of 4.2 million current and former federal employees was hacked. US Chief Information Officer (CIO) Tony Scott said this week that agencies need to rethink how they fund cybersecurity.

Scott said funding should not be an overall percentage of the IT budget, according to a report by Federal News Radio’s Jason Miller: “I think that’s the wrong way to think about security,” he said. “The right way to think about it is on a risk based analysis. We’ve got threats. We’ve got risks. Just like insurance, that has to be the equation when we are thinking about how much money we should spend on cybersecurity.”

How do federal agencies start building the business cases for greater assets to fund these programs not only today, but in the coming years?” The best way to justify return on investment in cybersecurity programs is the development of a business case through the Capital Planning and Investment Control (CPIC) process. Here’s how CPIC can kick-start the cyber budget process, reveal gaps in coverage and agency understanding, and help with forward planning. [Read more…]

How to Practice Good Cyber Hygiene

protect yourself imageIn this digital age, we depend on our computers and devices for so much that we need to be constantly proactive to protect against cyber threats and attacks. Multiple studies show that government, corporations, and individuals are being increasingly targeted. The Office of Management and Budget (OMB) for example, said federal cybersecurity incidents were up 15% in 2014 from the previous year, hitting a record high of 70,000 in FY2014.

Whether at work in the public or private sector, or on personal computers and devices, there is a greater need than ever to be vigilant. In order to accomplish this, we must practice good cyber hygiene daily. How do you implement cyber hygiene practices today and what can you do to prevent the vulnerabilities of tomorrow? Below are several best practice strategies for strengthening defenses that you can do on personal computers and devices or that public and private sector IT departments are doing behind the scenes: [Read more…]

Successful Task Order Management – Start Early!

start early imageEmploying best practices in Task Order Solicitation Management processes can help Government respond to constant challenges throughout the acquisition life cycle, particularly the acquisition planning phase.  The primary reason for using a coordinated effort to apply best practices among Government Program Managers, Contracting Officers, program officials, and industry stakeholders is to increase the probability of mutual maximum results. In this blog, I’ll share with you some best practices that I’ve learned and experienced from a Government and industry perspective.

Let’s start with task order solicitation best practices for government. They include but are not limited to the following:

  • Directing market research to establish a baseline in organizing technical, functional, and business domains mapped to government requirements/gaps. Once a baseline has been established, the technical, functional, and business domains serve as a framework for driving the creation of the SOO/SOW with clarity and precision.
  • Next, specified consistent roles within each of the domains identified in your baseline are captured and defined, culminating with a packaged release of your RFP.
  • Underpinning this entire best practice is frequent and even collaborative interactivity and development with industry throughout the process.

Again, working with your market research industry partners, and a clearly identified and precisely written baseline requirements document, shortens the release time and produces a focused and accurate product, lessening the risk of costly rework, and even mission degradation.

The objective of Task Order Solicitation for contractors is to identify the best opportunity fit for their company, followed by capturing the task order award, and then successfully delivering task order accomplishment. Usually, this accomplishment is closely related to and supports Government mission accomplishment. You could think of it as a synergetic relationship. By employing best practices early in the acquisition life cycle, both Government and industry can prime the pump for successful performance through the task order life cycle. [Read more…]