Performance-Based Planning and Programming Guidebook

Event ID: 2239712
Event Started: 10/29/2013 12:50:08 PM ET


Please stand by for real time captions.


Ladies and gentlemen, thank you for standing by and welcome to the transportation planning information exchange webinar about the new FHWA and FTA Performance-Based Planning Guidebook. At this time all participants are in a listen-only mode. We will conduct Performance-Based Planning and Programming a question-and-answer session later and instructions will be given at that time. If you should require assistance during the call, please press star then zero. Now like to turn the conference over to your host --


Ken Petty – Opening Remarks from FHWA

  • Good morning and good afternoon everyone, and welcome to the information exchange webinar. As mentioned, the topic today is the Performance-Based Planning and Programming Guidebook. Today's webinar will showcase the new FHWA/ FTA Performance-Based Planning and Programming Guidebook and three presenters will provide an overview of key elements of using this approach in relationship to their existing planning and programming processes. To achieve desired performance outcomes for their transportation system -- it will highlight the preferences being applied by the three agencies.
  • The next two webinars in this series will reflect various aspects of Performance-Based Planning and Programming with the focus on performance consideration in the long-range transportation plan and linking planning and programming in decision making processes of agencies. Each webinar will include presentations from state DOT's, MPOs and transit agency speakers who will discuss effective processes and lessons learned.
  • The next webinar is scheduled for Friday, November 15. If you have not already registered for this event, please wait until at least Friday to sign up since we need to provide a different call in number than is currently provided. For those who have registered, expect to receive an e-mail from us in the next few days with the new call in number. Subsequently, the final in the series of webinars will take place on Thursday, December 5. In terms of additional update from the office of planning, the fall version of the planning update newsletter will be published in November. Stay tuned for that.
  • FHWA, FTA, the National Highway Institute, and the National Transit Institute are working together to update many course offerings to ensure they reflect the changes resulting from the passage of MAP-21. Some of these courses will be easier to update than others; we are hopeful that within the next year the courses will better reflect current roles. In terms of other information, we recently completed updates on resources for financial planning, fiscal constraint, and transportation planning.
  • There are a set of tools on the planning capacity website -- planning.dot.gov, in which a series of spreadsheets allows staff to clearly demonstrate financial constraints or to develop transportation revenue and expenditure scenarios. Also included is a brief overview of the tools in a document that describes steps agencies can take to cooperatively develop a revenue forecasting process, and a literature scan of noteworthy financial planning practices. The tool can be assessed on the financial planning focus area page or using the drop-down menu under peer learning. Again, the website is the planning capacity program at planning.dot.gov.
  • Also, on October 5, we will hold a webinar on scenario planning techniques. The planning information exchange webinar series will host a webinar that explores the diverse tools available to scenarios planning today. The webinar focuses on tools and techniques that facilitate interpretation, analysis, and interpretation of scenarios. Three presenters will provide an overview and use scenarios tools and techniques. You can register for that event on the National Highway Institute event calendar. I will turn it over for FTA news.

Dwayne Weeks – Opening Remarks from FTA

I am the director of the office of planning and environment with the Federal Transit Administration. Some key initiatives we have underway include:

  • On September 30, FTA issued joint draft policy guidance for Metropolitan Planning Organization representation. The comment period closes on October 30. Basically, that is tomorrow. Please review that the guidance, it is only five pages long and send your comments to www.regulations.gov.
  • Also in September, FTA published an advanced notice of proposal rulemaking on the national progress safety plan, a public transportation agency plan, the public transportation safety certification training program, and transit asset management. There is a 90-day comment period that closes in January, and they are seeking comment on 115 questions related to safety and transit asset management. We need that input to draft an advanced notice of proposed rulemaking. Given the number of questions, I would start reading that now. Be prepared to send your responses to the docket. We are also working jointly with FHWA and others on proposed rulemaking for metropolitan and Statewide planning and we are trying to move that through various decision levels. Keep your eyes open, it is slow going but we hope to have that out soon.
  • We continue to work on MAP-21 changes to the major capital investment program and core capacity program. We recently made available the simplified shift on project software available on our public website. STOPS is a tool that can be used to develop estimates and trips on major capital proposed. It is located on our new travel forecast page and the website contains user files and provides links to send this information needed to run the STOPS program. We also have help desk and contractor support. Those of you who don't have advanced travel forecast tools will find STOPS useful for your purposes.
  • We have also updated our public website for transportation planning and opened that up last summer. It is much easier to find information related to metropolitan and Statewide planning and other key initiatives as we finish them. We are also working with FHWA and FTA field offices to update our internal memorandum of agreement. We are working closely on ways to improve the effectiveness of the certification review process and our transportation capacity building program.
  • In addition, we're working to update all of the National Transit Institute courses to reflect MAP-21.
  • In a nutshell, that is some of our key initiatives underway that we are working together with FHWA. I will turn this over to Egan Smith for the remainder the webinar.

Egan Smith, Moderator

Thank you, Dwayne and thank you all for attending this Performance-Based Planning and Programming Guidebook webinar. My name is Egan Smith and I'm with Federal Highway Administration’s Office of Planning. This webinar provides highlights of the Performance-Based Planning and Programming guidebook and is intended to address questions about how the guidebook can be used as a resource. It also includes key good practices from state DOTs, MPOs and transit agency as well.


In order to address the challenges for planning and programming and to influence more performance-based decision-making, FHWA and FTA have supported dialogue to help transportation agencies learn from each other and share information on best practices. Activities have included a series of national and regional multistate workshops and coordination with industry stakeholders and the development of resources focused on performance-based processes such as this Performance-Based Planning and Programming Guidebook, the congestion management process guidebook, and other performance related outreach efforts.


The guidebook is designed as a practical resource to help State DOTs, MPOs, and transit agencies understand what the key elements of PBPP processes are and how they fit within existing planning and programming. It expands upon existing resources and outreach efforts that we have had in the past. It highlights examples of effective practices from State DOTs, MPOs, and transit agencies on their long-range plans, and some of the other planning process elements as well.


Apart from the outreach efforts indicated on the last slide, a stakeholders committee of practitioners from State DOTs, MPOs, and transit agencies guided the development and provided significant input to develop this guidebook. The guidebook is designed to highlight good practices and not to interpret MAP-21 requirements.  Instead, it will identify processes that can be useful in the requirements for performance-based planning.


Performance-Based Planning and Programming attempts to ensure transportation investment decisions are made for long-term planning and short-term programming of projects. They are based on the ability to meet established goals. It includes a wide range of activities and products as part of a cooperative continuing and comprehensive process. The guidebook is organized around the basic elements of Performance-Based Planning and Programming process starting with an executive summary that attempts to capture the overall process and can be shared as a stand-alone document for transportation executives.


Section 1 and 2 introduces basic principles of Performance-Based Planning and Programming, including an overall framework and common themes that are features in successful performance-based approaches. Section 3 to Section 9 gives a detailed explanation of the key components in the PBPP process from goals through system monitoring. Section 10 highlights tips and success factors for an effective process. Section 11 provides four detailed case studies of agencies using PBPP approaches, and  Section 12 provides a glossary and links to relevant reference documents and relevant websites.

Throughout the document multiple short case study bubbles are included to booster the arguments made.


The guidebook leads off with an executive summary that attempts to capture all of the nuances of Performance-Based Planning and Programming. The executive summary defined PBPP including distinctions between performance management and Performance-Based Planning and Programming. It identifies the plans and activities that encompassed the process. It presents this framework diagram indicated on this slide and key Performance-Based Planning and Programming steps and identifies key lessons for effective implementation of the PBPP approach.


To step through the process, we start with the basic characteristics. Strategic Direction basically indicates:

  • Where do we want to go?
  • Analysis (How are we going to get there?)
  • Programming (What will it take?)
  • Implementation and Evaluation (How did we do?)

Under strategic direction, you will have goals, and maybe quantifiable objectives that guide decision making, and performance measures that capture these goals  and objectives.


Next under analysis:
Targets can be set. Though it’s important to recognize the challenge in defining meaningful targets in all areas and tracking progress over time or performance monitoring, identifying and evaluating strategies will guide target setting, investment, and policy decisions made by transportation agencies.


Making the link to programming requires understanding. The set of actions, strategies, and investments that you can use will improve performance. The expected impact of those strategies are for resource constrained achievable targets to be set so the planning process should lead to the development of investment priorities that can feed into an investment plan. Based on the resources that are available, this can lead to a program of projects that produces the outcomes desired.


Finally, successful performance-based planning requires thinking about implementation, reporting, and monitoring, which not only feeds back into the planning process, but also informs the programming of projects as well.

Section One presents the general purpose and overview of the process itself. PBPP builds on existing practices. For instance, you have the strategic highway safety plan or an existing performance-based product that States are required to implement. This requires safety indicators, analyzing data, and identifying strategies. Furthermore, MPO's in areas with populations greater than 200,000, called Transportation Management Areas, are required to have a performance-based congestion management process.  This requires the identification of congestion objectives, election of performance metrics, monitoring of system performance, analysis and selection of strategies, and evaluation of effectiveness. These efforts should be part of the PBPP process along with efforts by State DOTs to develop their transportation asset management plans and other performance-based plans as well.


PBPP involves integrating performance management concepts into the existing federally-required transportation planning and programming processes.

Section Two presents an overview of PBPP key concepts and some key reasons and terminology related to PBPP. A performance-based approach improves return on investment and resource allocation since information about past performance and expected future performance feeds into decisions about the best use of public funds, thus increasing the return on investments made with increasingly scarce resources. It also increases accountability and transparency by providing clear documentation about why transportation dollars were spent in a certain manner and what were the performance results.


This gives the public a greater understanding and faith that transportation dollars are being spent wisely to solve the most pressing problems

Terms such as “goal” and “objective” are often used interchangeably, however, the terminologies are clearly defined in the Guidebook in order to have a common understanding of the PBPP process.


In terms of cooperation and coordination between agencies, it is important that the state DOTs, MPOs, RTPOs, and transit agencies align their goals, objectives, measures, and targets with one another. Traditional and non-traditional planning partners in PBPP should include local governments, the business community, freight communities, law enforcement, housing agencies, economic development organizations, health community, education community, resource agencies, and others.


With that I will turn it over to Michael to get into the details of the process.
We will have a short survey in between.

Michael Grant is a Principal at ICF International in Washington, DC.  He brings more than 18 years of experience focusing on advancing sustainable transportation decision-making practices, including work related to transportation planning, project development, and the environment. He has provided training and technical assistance to State DOTs, metropolitan planning organizations, and transit agencies on topics such as the congestion management process, air quality conformity, and transportation demand management. For FHWA and FTA, he recently led development of the Performance-based Planning and Programming Guidebook, and is currently developing a guidebook on Model Long Range Transportation Plans to support Performance-based Planning. He holds a M.S. in Public Policy & Management from Carnegie Mellon University.


Michael Grant – Overview of PBPP Guidebook

Thank you, Egan. I am seeing from our poll that it looks like folks are learning that all of the themes of collaboration, coordination, are important in PBPP processes. I am going to step through the other chapters in the guidebook and provide some highlights of each section of the guidebook. I will start with goals and objectives, which is Section three of the document.


In discussing goals and objectives, the Performance-Based Planning and Programming guidebook identifies and defines a goal as a broad statement that describes the desired end state. For instance, a goal for a region or for a State may include having a safe transportation system. In developing goals it is important to recognize that goals do not have to fall solely under the control of the transportation agency. A goal may address societal goals, things that are important to the public including economic vitality, mobility, the environment, sustainability and other issues. In thinking about goals, these goals may address a range of different factors including the federally required planning f actors, national recognized goal areas under MAP-21, and other issues of importance to a community. The agency should consider the influence of factors such as land-use decisions, economy, vehicle technologies and other issues when thinking about the goals for their transportation system, and goals for society building on a vision for the region or state.


An objective is defined as a more specific, measurable statement that helps support achievement of a goal. For instance, an example may be to reduce pedestrian fatalities. This is a way to more clearly define what we mean by a broad goal such as a safe transportation system, and gets closer to defining what is really important to a community. There may be multiple objectives under a single goal area. And in a Performance-Based Planning and Programming process, a good objective should include or lead to development of a performance measure that can be tracked over time and used to assess different levels of investment or different policy alternatives.


Objectives may be defined as smart or specific measurable agreed-upon realistic and time bound. In an ideal example, a SMART objective includes some element of these various aspects of having a target and being tied d own. There are various types of objectives that may be used including outcome based, output, or activity oriented objective. It is critical to involve the public in developing goals, because the goals and objectives really serve as a strategic foundation for a performance-based planning approach. This should be a key effort under agencies public involvement strategy.


Section Four of the document speaks to the key element of performance measures in a Performance-Based Planning and Programming approach. Performance measures serve critical purposes in a performance-based process. They can be used during strategy analysis to compare different investment and policy alternatives, and can be used to track actual performance over time. As noted on the slide, they can also be used to more clearly define what we mean by a goal and objective. How are we measuring what we are trying to achieve? As I said, to monitor and track performance over time and serve as a reference for setting targets using performance measure to determine when targets are met.  Then used to identify the baseline along with the measures provided through the federal rulemaking.  Areas really should consider the measures that are of most concern to them as an agency and as a community. That is why it is important for agencies to start developing their planning process to incorporate measures into that process to address outcomes that are desired by users.


They should also build on public concerns in selecting those measures. Defining objectives and associated performance measures may include issues such as livability, quality of life, and economic vitality. These measures may be unique to each state and region. Measures will be selected involving the public and also understanding what kind of data is available and other factors that are listed on this slide.


Section Five talks about the starting point of the analysis phase of identifying trends and targets. This is the part of planning that looks at what are we trying to achieve in terms of performance. In defining trends and targets, a target can be used as a basis for comparing progress over time. It can also help in making decisions about investments and policies. The guidebook identifies a range of different ways in which preferred transit targets can be developed. Firstly, one may develop a directional target or identify a desired trend. This sets the basis for identifying a specific direction of impact. For instance, we may consider reducing the number of fatalities or improving the on-time performance of public transportation. Targets lay out a direction for where we want to head.   An aspirational target is developed to reflect a policy priority and to signal the importance of an issue or to reflect a broad societal target.


Even if the agency may not be able to achieve directly or have control over achieving that target, for instance, fatalities is an example of an aspirational target that has been used by agencies to demonstrate the belief that even one fatality on our transportation system is too many, and this is a key target for our system over time.


A realistic target is a target that takes into account a range of factors, including available financial resources, historical trends, risks, other competing objectives, and the many factors that affect performance in order to try to define something that is specific and can be realistically attained within a plan. Or at least with the investments and resources available to an agency. There are many timeframes that can be considered in looking at targets, including long-range, midrange, and short range. A long-range timeframe typically relates to the timeframe of the long-range transportation plan and maybe 25 years or more. A midrange target might be a ten-year period, and short-range targets might relate to a five-year period. For setting targets, it is important to define a baseline and that baseline can be developed and assessed using historical data as well as assumptions and tools to forecast what the future would look like. Understanding what the future may look like will help in defining where we have challenges in the future and where we may be able to achieve a more specific target.


Section Six of the document speaks to the process of identifying strategies and analyzing alternatives in order to meet those targets. The document identifies a number of common themes. In particular, strategy identification, analysis and selection determine how the targets will be achieved. There are a number of challenges that often agencies face in selecting strategies. These often have to do with conducting analysis of a wide range of different types of investment and capital strategies. It may also include looking across a range of different timeframes. The common themes also relate to considering a full range of different strategy options including near-term and long-term strategies. This may include capital investments and operational improvements, land use strategies and other types of approaches.


We also want to look at analyzing capacity and non-capacity related strategies. Scenarios with planning and analysis are a key tool used to compare packages of different investments and strategies as well. Historical data can be used to understand past and existing performance, and forecasting tools can be used to analyze forward-looking analysis to assess anticipated performance of the transportation system and to assess how different future investment scenarios will affect that performance.


Economic analysis tools can also be used to support trade-off analyses and selecting the strategies to move forward with. There are a number of examples in the guidebook that highlight this and one example is from Washington's State DOT where they have identified a target to reduce fatalities and have used data to look at the source of fatalities to understand the role of different reasons including speed, impairment, run off the road, collisions, and using data to help define strategies to address challenges and to help in achieving their targets.


This slide shows another example from the Dallas area with the North central Texas Council of Governments where as part of their long-range planning process and congestion management process in particular, they conducted an analysis using their travel model to forecast congestion levels in the future. The model is used as a basis to understand how performance of the system is likely to change in the future and to assess alternative investment scenarios and how that might affect congestion in the future.


As we see, using data and tools is an important part of identifying and analyzing strategies in a Performance-Based Planning and Programming process. Planning analysis can also be an important component of this. Scenario planning is an analytical approach that provides a comprehensive framework for evaluating how various combinations of strategies, or scenarios, may affect system performance at the statewide or metropolitan level.


The approach involves identifying various packages of strategies or scenarios and comparing performance against a baseline projection. The scenarios planning approach helps to visualize and articulate both qualitatively and quantitatively how various strategies would help to meet performance targets. In the document, it highlights the number of examples of different ways in which scenario analysis can be conducted. For example, some areas have looked at alternative transportation and land-use policies. By looking at a combination of alternative land use and transportation policies, you can assess how the region performs if various transportation metrics were changed under those different policies.


Another option is to look at anticipated performance at a variety of different funding levels to assess how funding for transportation affects performance of the system, as well as looking at different scenarios for how the funding is used in terms of preservation, under different asset management scenarios, as well as investments in new strategies.


A number of areas have examples, at the North Jersey Transportation Planning Authority where they have looked at as part of their long range plan projected outcomes under a baseline scenarios of trends under their plan scenarios as well as alternative or aspirational scenarios to compare performance and assess how they help to achieve various projected outcomes they have set out for themselves.


Finally, as a result of this process we get into developing the investment priorities that appear in the long-range transportation plan. Section seven of the document speaks to these issues. In developing investment priorities in the long-range plan, the long-range plan elements include investment priorities made through policy discussions about what packages of investments will be adopted and supported by implementation. These policy discussions are native to the development of a long-range transportation plan within the state and metropolitan landing process. The decision-making process is iterative and may involve going back to technical staff to answer questions, re-examine options, and evaluate hybrid scenarios.


Regardless of how a plan is organized, Performance-Based Planning and Programming typically include some basic elements. It identifies a set of goals performance measures and desire targets, taken together they form the basis for selecting investments and policies in the long-range transportation plan. It also would include a status report of current conditions that provide an analysis of the existence systems performance as a baseline for performance within the plan. The baseline condition analysis also can include tracking progress over multiple years to show historical trends and this informs meaningful discussion of targets as well as goals and visions of the plan.


A plan also would include an assessment of needs that are composed of different sets of information, including the shortfall our backlog between existing conditions and target conditions, as well as looking at how anticipated trends in population, employment, land use and other factors will affect performance and the needs for the agency and for the system in a investments required to bring the system to the stated performance desired in those plans.


Finally, it includes identification of investment policies and strategies which are delineated in the plan. Optimal system conditions may be in excess of targets in the plan, given that there is often a backlog of needed projects and investments.  The document should identify investments and strategies to be implemented to help meet performance targets that support progress toward goals.  Priorities should have a clear link to the goals and objectives stated earlier in the plan.  For plans that do not select individual projects, the plan should identify the amount and mix of funding within individual program areas and lay out a package of major investments and strategies.


Here is one example of a long-range plan showing performance scenarios from the Colorado State transportation plan. It is a broad policy plan which includes more detail information available in the technical report but the plan addresses the funding performance linked by analyzing three investment scenarios. Colorado DOT estimated or looked at forecasted revenue scenarios and looked at pavement conditions, as well as other factors under these different scenarios.


They have been straightforward about determining and documenting in the plan about how it needs to make difficult trade-offs, and the impact of investment levels on system performance.


Finally, Section Eight speaks to the programming process which includes investment plans, resource allocation, and coming up with a program of projects. This is what it will take to achieve the targets in the plan.


Linking the planning process to the program remains a challenge for many transportation agencies. In developing the guidebook, there was a bit of discussion about effective ways to really make the link between investment priorities in the long-range plan and the transportation improvement program or STIPs/ TIPs within the State and MPO. Some of the project prioritization methods may include developing scoring techniques or other quantitative approaches that combine multiple attributes to make project decisions. Utilizing economic analysis tools support analysis of alternatives. Other approaches use prioritizing or optimization techniques that identified projects that maximize performance against a range of different constraints. An investment plan may include identification of investment needs to assess system performance and regional and community priorities.


Typically it includes projections of expected revenues and prioritization of investment balancing various goals and given revenue constraints. Mitigating the connections to performance, a STIP/ TIP document can account for tracking consistency of projects in the STIP/ TIP with investment levels identified in the long-range plan. Providing quantitative information on those connections, using project selection criteria to help select and score and rank projects, and it provides information to enable a summary of projects of different types, and how those support different goals of the plan.


Once projects and strategies are prioritized based on project selection criteria, the next step is to match and compare to make investment decisions. As an example, the Atlantic regional commission developed and implemented a project prioritization method in its long-range plan and TIP which evaluates capacity expansion projects, (transit, roadway, and HOV lanes) based on their environmental impact, support for regional land use policies, and ability to reduce congestion.  All system expansion projects in the region’s RTP were evaluated with this method based on intersection with six critical environmental areas, and data for each area was mapped and compared with the proposed projects.


Finally in the process we talked about implementation, evaluation, assessing how the plan is doing, and how we are doing through monitoring, evaluation, and performance reporting. There are a number of reasons to monitor and evaluate a plan, including enhancing understanding of system performance in which strategies are effective, determining whether the objectives have been met through attainment of targets, and ultimately informing the adjustment of projects and programs over time. It supports providing information that will help support future cycles of planning. As we know, planning is an iterative process with a cyclical nature to the process and information for monitoring and evaluation can help to inform ongoing investment decision-making.


In the guidebook we make a distinction between monitoring and evaluation. Monitoring is the process of tracking performance of the system in terms of goals, objectives, and how the system is doing in meeting targets. Evaluation is moving a step further to the process of interpreting the results to understand the impact that investments and policies have made on performance. There may be situations in which performance does not improve or deteriorate because of external factors, population growth, and/or other factors beyond the control of the agency. Being able to understand how the system is doing but also evaluating how the investments in the transportation system are benefiting or helping to affect system performance are both important component of the PBPP process. We look at system-level performance, evaluation, and program level evaluation as well.


An important element in reporting performance is providing information about system performance, but also being able to tell a story about how their performance is changing over time. In the case of the public and policymakers, visuals in dashboards are helpful in communicating information that the public and decision-makers can understand, rather than simply reporting data. The concept of performance journalism has been used by some agencies, notably the Washington State Department of Transportation.


It involves using a combination of quantitative data using charts, tables and measurements along with telling a story about why that performance has changed over time. This is particularly important given the range of factors that may affect system performance over time.


We have in the guidebook a number of examples of reporting tools. There are examples from various agencies for example Virginia DOT has an operational dashboard which shows evaluation of key metrics. As we noted, performance reporting in this element is not only about dashboard, but about understanding how the investments in the system and the policies and changes in planning decisions have affected performance over time.


Finally, the document identifies some keys to success. I will go through all of these, but note there are a few key highlights to focus on measuring what matters. Focus on outcomes that are important to the public. Select a limited number of measures, rather than trying to measure hundreds or dozens of measures. It can be preferable to identify a few key measures that are useful. Build on existing performance-based planning processes. There are a range of processes, strategic highway safety plans, congestion management process, transit agency management plans, and other documents all look at system performance or different aspects of performance.


These can be built into a Performance-Based Planning and Programming approach. Coordination, collaboration, and working together are key elements of success. Finally, the document ends with a number of useful case studies. We will hear from two of these areas today in the presentations from Minnesota Department of Transportation and the Washington Metropolitan Area Transit Authority. The other case studies provide some useful highlights from MPO's and the document includes a glossary as well as a number of other references.


With that, here is some contact information and I will turn it back to Egan Smith.


Thank you, Michael. With that I think there is one survey question, but then we will jump into the case study presentations this afternoon.

Ryan Wilson is a Planning Engineer in the Minnesota Department of Transportation’s Office of Capital Programs and Performance Measures.  In this role, he focuses on investment analysis and highway planning.  In 2013, he served as the project manager for the Minnesota State Highway Investment Plan which guides the investment of $15 billion over a 20-year period.  Mr. Wilson is AICP-certified and holds a B.S. in Civil Engineering from the University of Wisconsin-Madison and Master of Urban and Regional Planning and M.S. in Civil Engineering degrees from the University of Minnesota-Twin Cities.


Ryan Wilson- MnDOT

Great, thank you very much. Appreciate the opportunity to be here. I will let that survey complete. My presentation focuses on a number of different items, spanning many of the points that were spoken to by Michael.


Michael, I believe this graphic was shown earlier.  At a high level, I wanted to start off by saying at Minnesota DOT we talk about Performance-Based  Planning and Programming, the establishment of goals and objectives in the agency's Center on to periodic long-range document, document, our fifty-year vision for transportation as well is our 20-year Statewide multimodal transportation plan. The rest of the work in the Performance-Based Planning and Programming cycle centers on three documents. One that would be a periodic update of our 20-year State highway investment land, which I will talk about in this presentation and was supposed to be adopted within the last few weeks and it is hung up in a few legislative things. And that we have the important annual piece which is the annual update of our tenure work plan. Our STIP plus six years beyond that STIP as well is the annual performance moderating cycle which is critical to ensuring advancement towards our performance goals and objectives.


The literal translation of policy objectives from our long-range plans into investment strategies and projects is accomplished through identifying those investments and expenditures in 10 categories, those categories are shown here on the screen that range from Asset Management of the existing road and bridges to safety investments, too improving connections for many modes as well as community regional projects that benefit quality of life, economic development, and advanced other objectives in our communities.


The remaining slides focus on an example of how we have recently materialized our planning program and process, and advancement I would say in the MnSHIP landing development process. The six steps that guided this plan development process will be the focus and organization of the remaining slides of this presentation. What we think about gathering information, this is an important step that we all do when we develop these long-range plans. We are taking information about our current investment practices, revenue, looking at the policy that guides our agency, in our case we have a state law guiding development of MnSHIP long-range development plan in addition to meeting certain expectations and components identified in MAP-21. We also take into consideration the system performance considerations and projections as well as other technical information in the agency. Something new we did this year it is not that we have never done scenarios planning, but have never undertaking scenario planning  as detailed as the way we did this planning effort. We formed a number of internal technical workgroups and these technical workgroups work with the 10 investment categories I talked a few slides ago, and we developed this performance level concept. It is similar to the level of service concept you have seen through some of the information from FHWA and others.


The idea behind this concept is that for each of those 10 categories, we establish a list of minimum performance level. The idea behind this is that we as start in the agencies, with what  is the least amount of money we could spend in the category to keep the roads safe and operable. In some cases such as payment in bridge, that number was not zero. But you can imagine the talent required to get experts in the area to think about what is the minimum amount of money they could spend in their asset category or particular investment class.


You build successive performance levels. The idea is you have a minimum level at zero and at that level zero you identified which is related to meeting your goals and objectives, identified in your long-range plan. At each successive performance level you begin to invest more, undertake different strategies, and doing each are a combination of those two things begins to mitigate those risks. Ultimately you move up to the highest performance level and that is associated with meeting all your goals and objectives, or meeting your aspirational performance targets.


One thing I wanted to note on this, I suppose it was a bit of luck, coordination, and foresight with federal partners being involved in this, but Minnesota is one of the Pilot asset management plan states along with Louisiana and New York. The performance levels documentation of them has been instrumental in forming the background for a number of pieces of works associated with that transportation asset management plan.


The subsequent step was to group those categories are performance levels across the 10 investment categories into the scenarios that we shared with the public. We group these various performance level in two approaches A and B reflecting different versions of what the system could look like in 20 years. Approach B focused on MnDOT approaches. Approach A was focused on asset driven an entire system and approach C was prioritize systems and multimodal capacity. It was built upon an assumption of fiscal constraint which is considerably different from past practices where we went out and took an approach asking people what they wanted from the transportation system, which generally always resulted in more and greater capacity, where is with this discussion -- people got into the weeds and chose all the approaches to varying degrees.


We had related information about these approaches and why people wanted to invest and what they were willing to trade off to meet their individual goals and objectives. A note regarding risk, we did not mix risk into the public former discussion. Our internal evaluation of the scenarios did include a more formal and higher-level discussion of risk and how we were going to mitigate the most important and significant risks the agency is facing.


We undertook similar efforts using an online interactive scenarios tool. We wanted to broaden our public anticipation and did this pretty last minute in the grand scheme of things for a long range plan. We had a pretty good rate of success given the last-minute nature of these efforts and translated the same ABC concept into an electronic tool.


All led early work, the results from the public, the key capital investment risks identified via internal processes, the performance targets and objectives that came through the performance level concepts, were all key components of establishing our investment priorities, which was how we were going to spend money for the next 20 years. A summary of the rationale is to mitigate those key capital investment risks and the performance objectives that came through the process. We talked about maintaining flexibility wherever possible.  We are a fairly decentralized agency historically, and MAP-21 and some other things as well as some internal things ongoing have caused us to reevaluate how we make decisions. Our goal ultimately was toknow that our districts are well connected to our local constituents. We want to be sure they are able to do projects with those constituents the best they can.


I will come back to the question on what is relevant. I will take any questions if there is time at the end. What we talk about priorities for our investment plan, another key change with the actual breakdown of our investment in these categories into two different periods. This might not seem significant, but I would say it is for two reasons. One, you can see this shift in priorities in the first 10 years we have a reasonably balanced approach, the green pie pieces equating to investment in existing roads and bridges. The second is 90% plus expenditure in maintaining existing roads and bridges. The reason for the shift was the identification of risk in which there are key capital risks we have learned through our process.


The other key benefit of doing this approach is, we knew as an agency, given the fact that we are going to have about a 60% reduction in buying power over the next 20 years.  We see increasing challenges and costs associated with maintaining and reconstructing our assets. This was a key demonstration piece to the public that ultimately the stewardship of the system is central and important for MnDOT.


Given the current track there is significant changes looming. We are undertaking a funding campaign at the state level, at our commissioners are going out and speaking with constituents and stakeholders. This is a key message that he has carried forward from this plan into the public forum.


There was a question in the sidebar talk about how we identified the risks. There are three broad components of this and any deeper discussion would require a conversation or half of a webinar in itself. I would be happy to chat with you further. The three summary points I would make is we did very much for each of those investment categories, undertook a risk register associated with spending money at performance level zero.


If we were spending limited money on safety, what risks are associated with that approach? Each of the workgroups do the same thing for the category. We then had a crosscutting agency work that identified and evaluated the risks across all these categories. Ultimately, we did all the detail risk information and took the greatest risk, you see the seven most significant crosscutting risks across all the categories, and these are the risk that we carried forward to the leadership. That is important because as you know, this can get complicated quickly.


We felt strongly that we needed a quick way to communicate the message of how well each scenarios or set of possible strategies was mitigating these risks. For instance, look at the third one down. That is misalignment with our own fifty-year vision and statewide multimodal transportation plans result in a loss of public trust, because the public has been extensively involved in those efforts. We believe our strategies in the first 10 years mitigate the risk fairly well, because we consider an approach similar into the years. Versus in the second 10 years, roughly speaking one third of the public was interested  in approach similar today, and one third was interested in an expansion oriented approach.


We are misaligned to some degree with our vision and long-range plan. That is a concern. You can see that bridges is the only one that got two checks in the second 10-year period, and reflects a continued sensitivity in the agency towards bridge investments and ensuring that our bridges are safe and sound for many years to come.


I talked a little bit about maintaining district accessibility. Up until this long-range plan, we have largely distributed most of our federal resources to our district, at our eight districts. Now, we have two different approaches and it does look like a peace symbol, although that was not intentional. This was the necessary amount of money to accomplish our data objectives in each bucket. This achieves certain performance targets and manages risks associated with statewide travel.


For this audience, that would be applicable with managing investment on the National Highway System, which is a collaborative process between districts and specialty offices and others here in central office.


We have the district management program which would be managing risk not associated with existing roads and bridges on the National Highway System. That is again maximizing the flexibility of districts to make investments that are most important for the regions.


Two quick notes on project selection and I am wrapped up. One is that MnSHIP as a planning process directly influenced the projects. Directly we put into STIP. For the first time the agency has made an effort to identify projects in years five through seven, those are planned investments beyond a three-year STIP. We have taken a performance driven allocation to make reference towards targets for the National Highway System as part of the statewide performance program. I talked about how 40% -- we have an allocation process by the District risk management program. The remaining funds are distributed based on a formula based on performance as well as system size and use and districts select projects to address their greatest risks and needs in consultation with their local stakeholders and partners.


A few notes on future steps. I think it is important to think about things we are working on moving forward. We had a successful area transportation partnership which we used to prioritize federal funds via Area Transportation Partnerships (ATPs) to bring in regional development commission, Metropolitan Planning Organization, city and county engineers, and others to prioritize those funds.


Since we have evolved our own processes, we need to think about how we use those groups. We need to find an equilibrium on decision-making, because we have made some adjustments to our decision-making processes. We heard loud and clear that our local partners want to be involved in our technical teams, that is important. And, of course, just as MAP-21 rule-making concludes and maybe continues should be the right word.


Finally, the planned strategy piece is critically important to be mentioned we have an annual update of our tenure work plan. It is critical we have the annual update to move projects from the planning years five through seven into our STIP and move projects towards actual construction. I mentioned we have annual performance management cycle which is a critical component of that tenure work plan update. That allows us to consistently and annually track progress towards all of our goals and objectives.


Alternately, it is a component of implementing these plans strategies. With that, I thank you for the chance to be here and present. I look forward to some question and answers. Feel free to get in touch with me if you would like to discuss any of the slides are information further. The plan is 99.9% done. If you Google MnDOT MnSHIP you should easily come across the plan and my contact information. I would love to hear from you. Thank you very much for your time.


Thank you, Ryan. Our next speaker is David Vautin.

David is a Transportation Planner at the Metropolitan Transportation Commission (MTC) in Oakland, California, leading the agency’s efforts in the field of performance assessment. His analytical work informs regional policy decisions by monitoring adherence to adopted goals and targets and by identifying high-performing transportation investments that achieve the region’s sustainability objectives. David is also extensively involved in regional land use and travel modeling, transportation policy analysis, and public outreach for MTC’s long-range planning efforts. He received his Master of Science degree in Civil Engineering from the University of California, Berkeley and his Bachelor of Science degree in Civil Engineering from Cornell University.. Thank you, David.


David Vautin- MTC

Thanks, Egan. Good afternoon everyone. My presentation will provide the San Francisco Bay Area perspective on the opportunities and challenges associated with Performance-Based Planning and Programming. For some background, MTC is the nine-county San Francisco Bay Area MPO a region of 7 million residents which spans from the skyscrapers of San Francisco to the office parks of Silicon Valley and the vineyards of Napa and Sonoma counties.


There are three components to my presentation this morning, first I will provide an overview of MTC's performance-based planning approach, including how we use consistent performance measures to evaluate planning scenarios, assess individual project product, and monitor regional performance. Second, we'll delve into our project performance assessment work to see how that process affected regional decision-making. Thirdly, discuss the challenges we face as well as the opportunities in the MAP-21 era of performance-based planning.


There are five phases to our performance-based planning process here at MTC and they are linked to our four-year regional transportation planning or RTP process. First we establish performance measures and associated numeric targets, then we evaluate both project and scenarios performance determining a set of packages of land-use and transportation investments we may want to pursue.


This process helps identify a preferred scenario and policymakers can work to achieve a hybrid that gets to the most optimal performance for each target and refining our package of transportation investment. We also do performance-based planning throughout our EIR process evaluations the performance of each EIR alternative demonstration our commitment to vetting alternatives to the proposed plan. Finally, we monitor performance of the adopted plan. Given our recent adoption of our latest RTP we are currently in the monitoring phase, focused on rebooting our performance, monitoring program to reflect the new era of sustainability focused planning and a greater emphasis on integrated regional planning.


To provide some brief history of MTC performance-based planning work, it is certainly something that did not happen overnight, it was the result of many years of evolution. We have gone from evaluating simple packages of transportation projects to combinations of land-use patterns, innovative policies and programs and transportation investment strategies and infrastructure investments.


Our performance targets focused on transportation now span issues from social equity, environmental stability etc. Project level have gone from an element of our work to a central component linking directly to the adoptive performance targets are relying on a rigorous cost analysis.


The number of projects subjected to our evaluation have grown as the exemption for projects to avoid these evaluations have been r educed. Let me talk about plan and the performance-based planning process. This is our latest RTP.  As a result of California Senate Bill 375, California RTP's have been merged with a sustainable communities strategy process. This substantially affects the performance process for us. Adopted in July, the Bay Area plan is the first integrated regional plan that addresses land use, housing, and transportation while the same time considering environmental, economic and equity issues.


This slide shows the adoptive performance measures for plan Bay Area. It is a broad array of regional goals and has an associated numeric target. What is important is what is not here. The sustainability of plan Bay Area is clearly apparent when you look here in CA lack of any traditional mobility oriented measures, measure of traffic congestion that often appear in RTP's's. Our plan focuses much more on mode shift and the VMT reduction, trying to address issues of traffic congestion, but really focusing on strategies of mode shift and bringing origins and destinations closer together.


This is a result of deliberate policy direction from our board and speaks volumes about how transportation planning priorities have changed in our region. Performance assessment was conducted at two levels, scenarios and projects. Scenarios are packages of land-use patterns and transportation projects that were evaluated against the high-level performance targets of the plan allowing us to identify which strategies best achieve the regions goals. Over the course of planning process we evaluated over a dozen different packages of transportation and land-use using regional models to forecast these target result today I would like to focus here on the transportation project level -- which had greater impact on policy decisions related to the plan. There were two key components to our work a qualitative the valuation and eight while attentive evaluation each with its own strengths. Qualitatively we did a targets-assessment and looked at the impact of the target for each of the uncommitted projects in our plan.


We also did benefit/cost assessment focusing on the most expensive and significant projects that increase capacity approximately 100 major projects transit expansion projects, efficiency projects, and highway projects as well.


There we did a benefit/cost analysis using our regional product model. We were able to, once we analyze these projects, highlight their performance. We created this set of bubble charts, which highlighted all of our major regional investments. So you understand how to read it, the horizontal access, there is -- we are showing the support for regional targets, plus 10 means you strongly support all 10 of our target, minus 10 means you have a strong adverse impacts of the targets. On the vertical axis is in the benefit/cost ratio and obviously want to be higher than one. The size of the bubble represents the scale of the project benefit.


You can see where we were able to highlight individual projects. I will skip the individual projects, you can take a look at those. For the sake of time, I want to highlight this overall chart which shows how general strategies performed. You can see we were able to highlight some key points for our policymakers. First, we highlighted the fact that the urban projects -- like pricing or idea improvement in the urban quarter perform strongly were cost effective. We are able to highlight how transit investments had investment at a much greater level of support than building Express Lane or widening highways. You can see those had adverse impacts on the regions goals.


The project data was useful in crafting a preferred scenario for a long range plan. Given the plan only has limited funding for expansion projects, nearly 90% of the plan expenditures go towards O and M of the existing system, prioritizing high-performance projects and reconsidering which performance projects were essential to maximize benefits for travelers.


We focused on the outlier projects, and 13 highest performing projects that had eight ratios greater than five and strongly supported targets. You can see examples of those here. We also try to discourage investment in low performing projects, projects with the benefit/cost relationships less than one or having adverse results. These encourage sprawl or are not cost effective.


We required them to submit a compelling case and they would be removed from the plan unless there are compelling cases approved by policymakers. This requirement was successful in removing some of these low performance projects from our plan. Only eight of the 34 projects identified as low performing had their compelling case approved. Most of these were due to their support of mobility objectives for low income or minority neighborhoods.


Twelve projects were immediately withdrawn by sponsors. Thirteen projects were subsequently re-scoped and only allowed to resume environmental studies which are required for these projects to be fully funded using local dollars which is quite a high bar for any project. By the time we reached our committee meeting, only one project remained with staff recommendations rejected and at the 11th hour that sponsor agreed to pursue solely environmental studies. This shows how performance-based performance can be effective in creating policy.


For example, transit expansion projects may be more cost effective in urban areas, but a regional transit expansion would not highlight this very important difference.  Performance assessment for long-range plans has generally been successful. We certainly experienced our own share of challenges. First, we do experience policy challenges. Our policymakers have sometimes unrealistic expectations about what is a really achievable goal.


Three decades for a long-range plan may seem like a long time, but it's hard to change behavior of existing residence. We must continually remind policymakers about the challenge of achieving changes in a relatively mature region. The fact that most funding only sustains the system rather than implementing new projects that may help move the needle on some of our targets, only exacerbates the problem.


Second, policymakers are often more concerned about non- transportation issues and how transportation projects help achieve those goals. Many of you have to face this challenge and you're asked about the economic impacts of the transportation investment or support for social equity objectives. It comes back to reminding policymakers that other policy decisions outside of transportation can help solve these problems. For example, affordability of low income neighborhoods might best be achieved by local jurisdictions and similarly air-quality goals may best be achieved by vehicle regulations as opposed to transportation or land-use strategies.


Policymakers were concerned about the evaluation of projects funded by local sales tax measures approved by voters. We reminded them that many of the sales tax measures rarely provide full funding for projects. Often, just a small sliver of funding for a given project is allocated from local sources. We were to reassure policymakers that performance assessment should be undertaken for all projects.


Second, we have technical challenges an ongoing challenge we are working to addresses is the lack of monitoring data for many of the performance measures we highlighted and care about. It is certainly more difficult to get consistent data for issues outside of our field of expertise in transportation.


We also find it is pushing the limits of regional travel models including our new activity base model and conducting all of these analyses is time-consuming, but we believe it is worthwhile. MAP-21 presents challenges and opportunities for metropolitan planning. For us, it provides an impetus to accelerate our performance monitoring effort. I think it will encourage thoughtful dialogue about how we integrate regional and national objectives. It also presents challenges for an MPO like ours. Potential measures need to be selected carefully. A poorly chosen national measure may be worse than no performance measure at all. And states will be challenged to select realistic targets. There are so many elements of the transportation system outside of their control, it can be hard to establish targets and actually achieve them, especially when they're outside economic factors that distort so many measures. For example, traffic congestion is often affected vary significantly by economic trends and job growth.


How do we deal with inconsistencies? That is a question that remain to be answered in our region. This diagram shows what we face at MTC. The goals of our region and the national goals have limited overlap on issues like health, safety, and maintenance. Our remaining goals emphasize environmental, land use, social equity and objectives, rather than mobility aims highlighted in M AP-21. How we bridge this divide going forward is certainly going to be a challenge for our performance-based planning team.


To highlight a couple of the other initiatives we are working on before we begin our next cycle of long-range planning next year, we are continuing to look forward on Performance-Based Planning and Programming. We are conducting research to try to implement some federal work on the state of good repair, so we can conduct consistent model -based benefit cost analysis for state of good repair projects. That is just as we do for capital expansion project where implementing enhanced stream -based approach rather than a horizon oriented and is it/cost methodology. We are working on ways to better audit local agencies cost estimates for projects as we all have seen there is often persistent underestimation of project cost. That is an issue that is not unique to the Bay Area.


We are rebooting our performance monitoring program in light of upcoming MAP-21 requirements. A few concluding thoughts for today. One, try to make performance assessment as meaningful as possible. Often it can be frustrating to demonstrate impacts for every region or state given the funding needs to maintain existing systems. Two, smart measures should have achievable goals. Project level assessment can help you highlight some of the more localized benefits of projects and spur conversation about smarter packages of projects in an effort to constrain budgets.


Make sure to inform your policymakers about strengths and weaknesses of potential measures and challenges related to achieving targets. Finally, I would urge all MPOs, transit agencies and states to get involved in the MAP-21 performance measure selection process and target setting processes. The selection of performance measures really has the ability to shape policy decisions and perhaps even in future years under future legislation, effect funding allocation. Smart performance measures could support smart decision-making moving forward. Prepare leaders in your organization as well as policymakers to help with target setting process going forward.


With that, like to thank you for your time and I'd be happy to take any questions.


Egan Smith

Thank you, David. There were a couple of questions in the chat pod. You may want to jump into a couple of them right now.


David Vautin

Sure, I will take the first. What factors are included in the benefit cost analysis especially for non- auto mode? We try to include the widest array of benefits that we could calculate using our model. We look at everything from travel time savings, travel cost savings, in missions, public health benefits associated with getting more people walking and biking. We look at noise impact, auto ownership cost and try to capture as many issues as possible. We look at all modes.


When we are analyzing our project we look at its mode shift impact, and of course the benefits of building a transit project may include walking and biking to the stations. We try to be as thorough as possible and ran our entire regional module for every product individually to simulate how 7 million people in our region would be affected by each individual assessment.


How were projects proposed and by whom for that analysis?  With this question, the projects were proposed by a local agency. Each of our nine counties is responsible for compiling a list of performers with proposed projects from their cities and transit agencies. They submitted to us over 1,000 projects and we went through the process of analyzing them in a consistent manner.


So, as opposed to having each county analyze their project separately and having inconsistencies, we use the exact same scoring rubric and model for the entire region.


Finally, a question from Cheyenne MPO. Can you address the project packaging process? We had to parallel processes. We had the traditional approach of coming up with scenarios based off of larger names develop IR planning staff. That was before we had done the project assessment and occurring at the same time. We also did individual project assessment and we combined this scenario and project level work to create preferred scenarios. We created a hybrid scenario oh and added and removed projects from that adding in all the high performing process projects and removing the low performing projects.


While the scenarios were developed based at the project level result, that is something we might want to do in the future, we did use both those analyses to inform our preferred scenario.


Egan Smith

Thank you, David. We will jump into the last speaker of the day,

Patricia Hendren is the Director of the Office of Performance at the Washington Metropolitan Area Transit Authority (WMATA). In this office, her focus has been on expanding the organization’s ability to use performance information to guide decisions, promote accountability and improve results. Prior to this position, she developed the agency’s $11 billion capital needs investment.  She has also worked in the private sector and federal government in statewide and multimodal transportation planning, performance measurement, and transportation finance.  Patricia is the Chair of the TRB Statewide Multimodal Transportation Planning Committee and member of the TRB Performance Measurement Committee.  Patricia holds a Ph.D. in Transportation Technology and Policy from the University of California, Davis, a M.S. in Agricultural and Resource Economics from University of California, Davis and a B.A. in English from Duke University.   Thank you.


Patricia Hendren- WMATA

Today I wanted to share some perspective from WMATA. Given I had the lucky position to go as the last speaker of the day. I want to make sure you are awake. We have a polling question about the five main components of a performance-based approach and what areas you find the most challenging. You can post those results so we can see from our fellow practitioners what we are struggling with. It looks like goals is not the hard one. As she pulls that up, I will introduce WMATA. If you have visited DC and used our services, Metro is the second-largest rail provider in the country, the six largest bus provider and we deliver over one million trips today. Next time you come to DC, you may be more interested in using our services.


As you can see of the five main components up on your screen, related to performance-based approach, folks out there in Internet world are saying that goals are an area that is not challenging. Easiest, I've not got a rigged question, good thing I have a room full of people here with me. Goals are the easiest, and then it looks like performance-based resource allocation is in the rear. Good news is you agree with what I'm thinking.


Let's dive in. Turning PBPP into reality, there are three main takeaways, I am hoping you will leave this talk with. First is we had the term evolution in some prior presentation comments. Performance-based is an evolution that is true with the area that you identified is challenging which is linking your resources, programming and budget to goals and measures and targets.


Second, monitoring is about driving results. We can create nice dashboards, we can create nice reports, but if it is not impacting results and decision-making, what is the point. The whole guide is to drive results. What has surprised me the most is the cultural resistance. At the end of the happy performance-based passed there is acceptance. I will get into that a little bit later.


Who is the Office of Performance? We are a small group that was formed in 2010 with existing resources. As mentioned in the introduction, the mission of our office is to inform, promote, and unify. We mean we want to use or expand our agency's ability to use information to inform decisions, promote Metro benefits to the region and unify employees around, goals. As you can see in this picture, we are not the performance police.


I bring this up, many agencies think about how to dive into a PBPP, it is important to check your cop at the door and this helps the agency get into a performance approach. This is not to catch people speeding down the road, this is to drive better decision-making.


There have been different graphics of what PBPP looks like. This is a simplified version than in the guidebook. At Metro, we have more simplified graphs, but it is in essence the same concept. A Performance-Based Planning and Programming approach creates your map. Through the first step it defines where you want to go, that your goals and objective. Performing measures is how you track if you're making progress. Your target setting that is how you define success and the area is the question which we started out on is the area where we start up this is where we try to link where the dollars go, where your staff hours go, where your projects get funded and how that ties to the goals, measures, and targets used. That is one of the most challenging aspects of Performance-Based Planning and Programming.


I have a young child and we are now watching “super why” and the power of why is amazing. If you ask why something happens, it is amazing the answers you get. The power of why. Looking at the first step in focusing on the easier part of doing a performance-based approach. We have been doing this for awhile. We have gotten much better at reaching out to stakeholders, the public and asking what you want from us? What transit system do you want? What kind of highway system do you want?


We had board adopted the goals that you see here that were approved in 2012. On the performance measures, a performance-based approach. In the upper left is the GMs/CEO business plan, and that has about 12 key measures. Those measures are what our Board of Directors will hold him accountable for on an annual basis. Those measures are very important to the agency. If he is held accountable to them, you can be sure that the rest of us are as well. The vital signs report is our monthly performance report where we do focus the report on asking the question why.


The overlap between the GM plan and the vital signs report are more tightly depicted than this graphic shows, but it also has 12 measures. There are some measures for different audiences. The point I want to make is the department business plan should be a lot bigger circle, because there are a lot of measures in there. There are going to be sub measures that you need to focus on.


You may have to focus on a key measure. I think everyone knows where the sweet spot of this graphic is, it is the area where they all intersect. An example, for that sweet spot is escalator availability. Everyone's favorite topic when we discuss Metro and how we are performing. Escalator availability will be in that sweet spot. In the department business plan for elevator/escalator, we find meantime to repairs, meantime between failure, preventative maintenance compliance, and a lot more sub-measures that are critical in order to drive that sweet spot measure.


As you develop measures, you want to have a small group that you are focused on, but as you go farther interagency that will be sub measures that support that. On target setting, this is something I'm proud to say we have gotten better at. One reason is time to get more familiar and comfortable with what target setting is about. Michael mentioned baselines, getting the trend analysis, having an understanding of what your prior performance has been is critical to understand how you set your targets for the future.


We have gotten better at combining the trend analysis with future actions. What we plan on doing next year the following year that will impact results? Being honest about barriers as far as lessons learned rather than throwing your hands up and say we can't control that, take that into consideration and adjust target settings versus throwing your hands up consider the lesson learned. It would be great if we had more information about peers. Everyone measures things slightly differently, but encourage people. The purpose of the target is critical when you're setting targets. Michael mentioned if you have aspirational targets, zero fatality for example, are you trying to have attainable targets. It is important to think of the role of your targets.


I have the example of bus fleet reliability to show. If you focus on the results and trend analysis, you can see how we have had continual improvement and tougher targets year in and year out, which is quite an accomplishment by our bus folks.


Here is the tough part. This is linking day-to-day activity that folks do and your strategic goals. We do that link and create that link between what people do and what we are all trying to accomplish with the same common outcomes through business plan and budget allocation.


In the business plan, this touches on the evolution. In 2010 we had two departments that had business plans. In 2011, there were eight. In 2012, it jumped up to 27.  In 2013, I'm happy to say all departments have business plans. These are pretty simple business plans, Excel spreadsheets, but the content is the important part. We have the goals, measures, actions people take and owner and who is responsible for driving that action to completion.


The general manager sent out a notice between 2011 and 2012. In that he said you will create a business plan for your department. It's amazing what comes from that. The linkage between resource allocation, programming, or budgetary process, project selection etc. is where we are trying to get better at. In Fiscal Year 2013 to show evolution, we looked at our new initiatives, additional funds we were asking the regions to support the agency.


In their request we said if you give us additional funding, we will demonstrate that progress -- performance will improve in this area. We will show that we will make or link it to our safety goal or too our deliver quality service call. That was our first attempt to connect our request for funding with the specific outcomes. It was only about less than 1% of the budget. It was a good attempt, but we clearly need to do more. In Fiscal Year 2014, our presentation of our budget to the regions or stakeholders and funders was our entire $2.7 billion budget. For that budget, we talked more clearly about what you would get for each of our strategic goals.


In the past development of our budget we are going one step farther. I'm hoping you can take away from the slide that we are getting more details about the results the region can expect with the funding they are giving us. We are more specific about how our budget requests are linking to the actions, the dollars to the actions to the results.


A second takeaway that I want you to have is on the fifth component of Performance-Based Planning and Programming, which is the monitoring. It's on our own scorecard, so I don't want to say scorecards and dashboards are not useful. But if you stop there, then you have a problem. That is where you lose the story. You need to start driving results. The monitoring progress is about a disciplined, rigorous, consistent effort. It is not a one-time-a-year annual report. It is not the flashy dashboard, but something you need to ask yourself continually? Why is performance changing? What do we do differently than last year? What do we put in place in the five-year program? We need to ask ourselves the wonderful question, why? You can see we've been releasing the vital signs report for 38 consecutive months. That is one reason why I have more gray hair now. We also have monthly meetings. The GM holds a monthly meeting with executives and asks “what progress are you making on the 12 measures that they are holding me accountable?”


We have annual target setting sessions, keeping the focus on what steps you are taking to drive progress. We have these two questions. I cannot emphasize enough the power of asking why did performance change? What actions are you going to take about making it better? Simple, but important questions.


I think we throw up another polling question, here. As I go into the story, how do you think about at your own agency, and this is about driving those results. If you are collecting data and reporting on it, I'm not suggesting anyone out there is doing this, and you are reporting it and walking away, then you're missing an opportunity. I'm seeing some yeses and no’s. What I mean by driving results in every month, every day, we look at our real on-time performance and ask why is the rail on-time performance increasing or decreasing? Given the continual asking of the question why, our rail on-time performance has improved significantly over the past two years.


The reason why is because our fleet is 66% more reliable. Cars are breaking down less and we are able to keep them in service for longer and have less disruption to our customers. The topic of balancing goals has come up. We are an aging system so we have a lot of track work. We are doing things that have an impact on our service delivery during the day. By looking at the performance review and asked them what drives them and make adjustments, the different approaches in order to attain those targets that you set for yourself.


There are some stories out there of how they have used performance to really drive results. Kudos to you out there and everyone keep up the good work. The last take away is on this cultural change. We are in a similar challenge situation where we have aging infrastructure, limited funding, we are asked to do many different things as transportation providers, and we are also under increased scrutiny, asking why you are spending the money the way you did, because you need to prove to me that you are a good steward of public dollars.


Those are all valid questions. In performance-based management, and I link this to the five stages of grief, you are familiar with that Kubler Ross model, but let me explain what I'm talking about. For nose-diving into Performance-Based Planning and Programming, you will reach out to a partner and say “we need to start monitoring what we are doing and collect data and ask ourselves how is this enabling us to achieve goals?” “How are we linking our decisions to where we want to go?” Everyone says, no, I've got this. Go away. I don't need help. I know what I'm doing and you are annoying.


Then you say “actually, we really need to do this.” There are federal requirements and hopefully you will have something in your leadership like we do, with our general manager, that says this is not a toy. You will do this and, wow, did they get mad. They said they didn't have time and they got very angry.


Then we go into the bargaining stage. You say we will start monitoring and looking at our performance results if you only look at peak period the period of performance.  Don't look at rush hour or what happens in non- rush-hour and you get an amazing bargaining stage. Then we go into depression. This is when everybody says “oh my gosh, those are the result.  Okay, this is not good.” We need to figure out how to do this thing differently. I promise you at the end of that tunnel, the light we get acceptance and people see the benefits of asking the question why and the benefits of trying to have that unified approach to the common destination or goal.


Here are three takeaways from the last few minutes in the presentation. This is an evolution. The whole purpose of looking at Performance-Based Planning and Programming approach is to guide decisions and push better decisions. Finally, you will get the acceptance. It does take time, but don't give up. I'll leave with the quote from our general manager and he says that performance management takes patience. When you have bad results, this is okay, if you can outline what you will do to fix it and deliver. That has happened to us time and time again. People understand and are willing to work with you if you explain what is going on and what you do to make it better.


Egan Smith
Thank you, Patricia. There is one question.


It looks like from Eric Hill, are your department business plans available for review?

Patricia Hendren

I get this question a lot, and typically, no. But come I think given I get it so often, I will make an effort to talk to one of our more superstar people in our organizations to see if they are willing to share. They are internal documents and that is why I hesitate. I think it would be beneficial and I will make a solid request to share some of the internal documents with you.


With that, we will open up the room for questions. There is nothing in the chat pod at the moment, but we can open up the lines.


Please open up the phone lines for questions.


Phone Operator

If you wish to ask a question, please press “*” then “1” on your telephone keypad. You will hear a tone indicating you have been placed into the queue. A voice prompt on the phone line will indicate when your line has been open. Once again, it is “*””1” if you have a question at this time.


[ Silence ]


Egan Smith

There is nothing in that chat pod and no questions right now, but we had some very stimulating discussion during this session. One of the issues that came up was the question of risk and assessing risk. This is something we hope we can dive into somewhat in the future. We have been discussing it with our acting director of the office of planning.  How can we go about developing a strategy to look at the risk, especially risk in terms of Performance-Based Planning and Programming.


We have a question over the phone.


We were interested in the lady who just got through having her PowerPoint to give us her contact information, name title and the number, e-mail address.


I can scroll back up to my first slide, it is like future going backward, were flying back in time. I apologize for not putting that at the end. My phone number is 202 902 2677.


Texas appreciates you.


Today's presentations are now available in the file share box. They are PDF, consequently you can find Patricia's information in the sample of that slide.


There is a question in the chat pod as well from James Andrew. “Are there any available tools that any of the presenters used or are using to conduct performance measurement?” I will open that up to the presenters.


Patricia Hendren

I will jump in first. We have used in the past a prioritization tool called Decision Lens and you may have heard of this tool. It is a beneficial tool, because it fosters dialogue and collaboration at an agency. We used it when we were prioritizing our $11 Billion ten year capital needs, and we use it to prioritize our $500 million and ARRA funds. Hopefully folks on the phone are aware of the HERS model and the TERM model these are more related to cost-benefit ratio and understanding your backlog for capital needs. Those are some tools that our agency has used. I will pass it over to my fellow presenters.


Egan Smith

Thank you, Patricia.  Ryan and David?


This is Ryan Wilson from MnDOT. There is a number of different tools. We've used Decision Lens from an evaluation stand point.  From the lens of conducting performance measurement, and we rely for the majority of our investments on our pavement and bridge management software and their outputs. The paving one has been used extensively for a while. Our bridge management software was recently reevaluated and retooled following a 35W bridge collapse. It includes an element of risk that considers length of detour and other elements from a consequence standpoint.


Those models have matured a lot and we rely on them heavily. We have a number of other tools that are related to speed targets for corridors and some more nuance things from a safety perspective, but nothing substantially or too holistic. From the tools that influence our long-range plan and decision-making, it's really very mixed. There is no one tool we use to bring everything together outside of Microsoft Excel.


Dave Vautin

This is Dave. We often time will build our own tools. For example, we built a benefit/cost component to our activity based travel model. We use the UrbanSim land use modeling tool to do a lot of our analysis for Greenfield impacts for long-range plans. We also use the EMFAC air quality model to measure the air pollutant impacts that are specific to California. We use TREDIS for forecasting gross regional product for different transportation and land use scenarios that is our economic impact tool. We have incorporated the World Health Organization active transportation health benefits to our regional travel model and, as was earlier mentioned, we use both TERM and a regional pavement tool known as StreetSaver  to look at our state of good repair.


Egan Smith

On the federal highway side, we've been working with a couple of groups at federal highway to develop a spreadsheet that actually has a list of a number of different tools. Hopefully, we can use this e-mail list as a result of the session to provide that information to the folks who have tuned into the webinar today.


There is also an effort under the FHWA Office of Planning as well, by the modeling team, looking at the developing of a detailed database of different tools available for modeling.


Patricia Hendren

When do transit providers begin engaging with the MPOs in engaging the planning process? You may know that I am going to say right away.  But we know that in the real world, sometimes we do that more within our own agency and then reach out and partner after. It is something we could do better.


Dwayne Weeks

I want to back up on that comment. Folks should be communicating now between the various transit providers and their MPO and with the State DOT's. The key area is going to be on data and data collection and data sharing. Make sure folks have comparable information. Likewise, transit providers should be looking at their transit asset management plan and what level of information they have available, and what level of information they need to do. To complete their transit asset management plan, I would say don't wait. I would start right now.


Ken Petty

Earlier there was a question.  “How do you evaluate risk as part of your performance-based process?”


David Vautin

To the West, the big issue of risk is the project cost information. We have examined the last 20 years of transportation products in our region and found projects are going up much faster than ever before. This is a real challenge for us when we come up with the benefit cost ratio we developed these great tools to assess the performance, hours of delay, we can calculate that. But, we have a lot bigger challenge trying to figure out what the real cost of the project is going to be when it is delivered. We are putting some extra effort into that over the next couple of years to improve our next long-range plan.


Patricia Hendren

For WMATA we are currently engaged in an enterprise risk assessment. The motivation for that is to make sure when we prioritize our capital program that we are addressing our high-risk areas. Again, that is one of the areas as an industry we have a lot of work to do. I think folks on the phone are lucky to have MnDOT as a presenter. For DOT and transit, they are on the leading edge looking at risk into the planning and programming process. I would say talk to MnDOT if you want to leapfrog some of the challenges that we are hoping to get a better handle on and do an even better job of making sure we are addressing risk of this enterprise risk assessment effort.


Ken Petty

Operator, can you check the line one more time?


Phone Operator

Once again, if you have a question over the phone, then please press “*” “1” at this time.


[ Silence ]


Ken Petty

Okay. We would like to thank our participants as well as our speakers for today. Thank you all for participating in the webinar Performance-Based Planning and Programming Guidebook. My name is Ken Patty and I am the acting director for the FHWA Office of Planning.

Stay tuned for the next two series of webinars on this topic. The next one is on the integration of performance consideration into the transportation plans long-range and linking performance-based planning programs. And we will be sharing and posting it on our transportation website. Stay tuned. Thanks everyone for your participation.


Phone Operator

That does conclude our conference for today. Thank you for your participation. You may now disconnect.