Minutes of the First Meeting of the
Network Reliability and Interoperability Council V
Summary of Meeting
March 20, 2000
Kent Nilsson, the Council's Designated Federal Officer, opened the meeting at 10:00 a.m. by welcoming the members and introducing the Council's Chairman James Q. Crowe, Chief Executive Officer of Level 3 Communications.
In addition to Kent Nilsson, the following FCC personnel were in attendance: Commissioner Michael K. Powell Dale Hatfield, Chief, Office of Engineering and Technology Douglas Sicker, Chief, Network Technology Division
The following NRIC member companies were represented:
AT&T (P.J. Aduskevicz) AT&T (John Pasqua) Bell Atlantic (Marie Breslin) Cisco (Arthur Reilly) COMSAT (Mark Neibert) Copper Mountain (Joe MarKee) Covad (Bob Knowling) CTIA (Edward Hall) CWA (George Kohl) Elastic Networks (Guy Gill) GTE (Rudy Verdes) Hughes Electronics (Gene Cacciamani) ICA (Ruth Michalecki) Level 3 (James Q. Crowe, Chairman) Level 3 (Pawan Jaggi ) Level 3 (Jack Waters) Lucent (Karl Rausher) MCI/WorldCom (Karen Johnson) Motorola (Glenn Grotefeld) NARUC (Brad Ramsey) NCS (Peter Fonash) NCTA (Bill Check) Nortel (Jonathon Homa) Paradyne (William Cobb) PCIA (Harold Salters) SBC (Donna Geisler) Sprint (Jerry Wade) Telcordia (Rick Harrison) Telco Year 2000 Forum (Gene Chiappetta) USTA (Roy Neel) US West (Eldridge Stafford)
After being introduced, Chairman Crowe emphasized the importance of participation, thanked those who had been involved in NRIC IV and congratulated them on their success in avoiding Y2K problems. Chairman Crowe noted that NRIC V's agenda continued to include Y2K matters. Chairman Crowe stated that NRIC V will be oriented towards the future. He said that a revolution in information technology is taking place, a revolution we were only beginning to understand but one that has made the United States' economy the envy of the world. During the first twenty years, this revolution had been driven by advances in computing and in information storage technology. In the last five years the revolution had spread to telecommunications. At its heart, it is an economic revolution that rapidly lowered the costs of moving information. It will have a broad effect on Americans and American business.
Chairman Crowe said that there were many issues involved in this revolution, but the issues in the NRIC V Charter dealt, first, with reliability and availability, terms that take on new meaning with interconnections among networks with different technologies. He said that both reliability and interoperability are now much more complex issues than they once were. Reliability had traditionally been measured separately for individual companies. Now it is much more complex. Chairman Crowe expressed hope that the NRIC members will clarify these matters and determine the right questions to ask and find the right answers to those questions. The second major issue described in the charter was interoperability, which traditionally dealt only with protocols and SS7. Now, a single call can involve dozens of companies and various technologies, making interoperability much more complex. The role of the marketplace needs to be considered, too, because it was such a great factor in determining the direction of technology and was part of the reason the information revolution had moved so rapidly. The degree to which the NRIC would succeed in addressing these issues would be directly proportional to the extent to which members participated in addressing these topics. Chairman Crowe then introduced Commissioner Michael K. Powell.
Commissioner Powell recalled his dismay when Michael Armstrong, Chairman of the previous NRIC, had stated that there were only 443 days left until the year 2000 and that his engineers had estimated that it would take about 60,000 years to test all possible problems arising from Y2K. The NRIC, nevertheless, had managed to deal with the Y2K problem by working together, non-competitively, with exceptional skill and energy. The NRIC demonstrated that every now and then joint government/industry efforts can produce spectacular results. He said the biggest problem the FCC had was explaining why nothing happened when the year 2000 arrived.
He thanked Chairman Armstrong for agreeing to Chair NRIC IV and for putting so much effort into making it successful. He thanked Frank Ianna, the Vice Chairman, and John Pasqua and the leaders of the focus groups. He especially thanked Gene Chiapetta, Gerry Roth and Marsha MacBride.
Commissioner Powell stated that he was thrilled to be involved in NRIC V and thrilled that John Crowe had agreed to chair NRIC V. Commissioner Powell stated that the NRIC is not a policy making group but that an understanding of the technical issues bearing on reliability and interoperability is essential to the Commission's efforts in making informed policy decisions in a world in which rules and technology are changing quite rapidly, especially in the areas of interconnectivity and convergence. He said the NRIC, along with the Technological Advisory Council, will be crucial in helping the Commission make intelligent decisions. Commissioner Powell stated that the NRIC would likely have a substantial impact on the Commission's decisions over the next few years, and hoped it would help the Commission understand what future issues were likely to arise.
Chairman Crowe agreed that the NRIC's job was not to decide policy matters but rather to provide a common framework to the Commission for its policy decisions. He then introduced Jack Waters, Vice Chairman of NRIC V, and asked the other members to introduce themselves, which they proceeded to do. Chairman Crowe then introduced John Pasqua.
John Pasqua said he would discuss NRIC IV transition matters. He then described the character of the NRIC. The NRIC is a federal advisory committee. Its members are telecommunications industry service providers, customers and suppliers. It makes decisions by consensus. Council meetings are public and require 15 days notice. Working subcommittees are not public and don't require notice. Meeting minutes are available at http://www.nric.org. Mr. Pasqua then proceeded to summarize the history of the NRIC.
NRIC began in 1992. It was originally called the Network Reliability Council. Its first chairman was Paul Henson of United Telecom. It's job was to study the causes of service outages and develop recommendations to reduce outages and reduce their impact on consumers. It produced a final report to the Commission and the Industry called "Network Reliability, a Report to the Nation." It also initiated the Network Reliability Steering Committee (NRSC) under ATIS to collect and analyze network-related incident data.
The 1994 iteration of NRIC was chaired by Richard Notebaert of Ameritech. Its task was to study the regional and demographic variations of network reliability, network interconnection, changing technologies, and essential communication and telecommuting capabilities during emergencies. It's final recommendations are contained in a volume entitled "Network Reliability:The Path Forward."
In 1996, the NRIC was rechartered. Its chairman was Ivan Seidenberg of NYNEX (now Bell Atlantic). Its task was to advise the FCC on the implementation of section 256 of the Telecommunications Act of 1996 and assess the Commission's role in the development of telecommunications standards. Its recommendations are contained in "NRIC Network Interoperability. The Key to Competition."
NRIC IV was rechartered in 1998 to assess the impact of the year 2000 date change on networks and to study the current status of network reliability. It produced quarterly reports on the NRIC web site, testing guidelines and results, contingency plan templates and a final NRIC report in January, 2000. About sixty-five companies participated. Meetings were held quarterly. The final report dealt with leap year as well as year 2000 issues. The charter asked the NRIC to assure optimal relaibility, interoperability and interconnectivity of, and accessibility to, the public telecommunications networks. It studied: (1) year 2000 impact on networks; (2) impact on access to telecommunications networks and services, i.e., CPE; and (3) the current status of network reliability. There were three focus groups corresponding to these three objectives. The first group had three subcommittees dealing with Y2K readiness, Y2K testing, and Y2K contingency planning. The second group had two subcommittees dealing with Y2K readiness and testing, and Y2K contingency planning. The third group had two subcommittees, one to review industry best practices and the other for data analysis. There was also a steering committee to review the progress of the working groups and deal with administrative matters. Leaders of the focus groups and subcommittees included P. Sahni of AT&T, Gerry Roth of GTE, Lou Scerbo of Telcordia, Ronnie Lee Bennet of Lucent, Bill Blatt of Nortel, Vaho Rebassoo of Boeing, Mike Cooke of Nortel, Ray Albers of Bell Atlantic, Rick Harrison of Telcordia and P.J. Aduskevicz of AT&T.
Mr. Pasqua said that the results of NRIC IV were as good as could have been imagined. There were only tiny Y2K glitches, which are still occuring. The countries that NRIC IV was most worried about because of a lack of information had few problems. During the transition to the year 2000, there were higher calling volumes around the globe but the overall volume wasn't much greater than the year before. There were no major problems in the financial sectors, transportation industry, energy industry or government. The same held true for leap year, although, as a result of a Y2K leap year glitch, the weather service in Japan indicated heavy rainfall when, in fact, it was sunny. Mr. Pasqua said that concern for Y2K problems has to continue. There are still systems that are not Y2K compliant that need to be replaced. There also needs to be regression testing to ensure that programs are Y2K compliant.
Mr. Pasqua itemized the lessons learned as a result of the Y2K experience.The lessons are that we are far more interdependent than we ever imagined; networks are complex, extensive in scope and quite robust; the NRIC was an indispensable forum for sharing experiences and leveraging knowledge; networking and information cooperation worked; a non-traditional approach was required to achieve better than traditional results; program management was essential; objective dashboards are critical as a measure of progress, the Y2K challenge substantiated the ongoing need for testing and independent validation and verification programs; consistent and continuous communications on Y2K progress were critical across the industry; and Y2k proved a useful pruning and organizing opportunity. In addition, many companies used Y2K to bring their information technology under better control and reaped additional benefits, including accurate inventories, up to date contingency plans, improved communications with partners and suppliers, retirement of millions of lines of codes and components, upgrades to the latest releases for IT and network platforms, systems and software, and better appreciation of the effectiveness of software tools.
Mr. Pasqua then introduced P.J. Aduskevicz.
Ms. Aduskevicz reported first on the NRSC report which monitored and analyzed outage reports. The NRSC provides quarterly and annual reports. Additionally, the NRSC reports on any adverse trends it finds in its analysis. Under the NRSC, there is also a facilities solution team. Facilities problems are the number one cause of network outages. There is also a procedural team. Cutting across all the outage categories, the NRSC discovered an increasing trend in procedural outages.
The fourth quarter report for 1999 showed that all failure categories were within the green range, i.e., the statistical margins that approximate the baseline year, which is the year outages were first reported. DCS and facility outages, however, were in the yellow range, indicating an impact higher than the baseline year. Total outages in the most recent three quarters are the second highest since the baseline year, and the median outage index measuring the impact of outages was higher than the baseline year value. There were 43 outages in total for the fourth quarter of 1999. Facilities outages, as usual, were the most frequent, followed by local switch outages, then tandem switch outages, than CCS outages, then DCS outages and finally, central office power outages. The procedural outages team concluded that the number of procedural outages is beginning to decrease. The report, for those who want more detail, is on the ATIS web site.
Ms. Aduskevicz stated that the NRSC was also asked by NRIC V to report on the reliability of public telecommunications network services in the United States and determine whether "best practices" previously recommended should be modified or supplemented, and develop a proposal for future consideration relative to extending these best practices to other industry segments not presently included in current practices.
Ms. Aduskevicz said she would continue her report with the work of the NRSC subcommittee that undertook the matter of data analysis and future considerations. She said the team included a large number of participants from a wide variety of industry segments. The group met twenty or thirty times to formulate their final recommendations. The group recommended a voluntary outage reporting trial with participation by service providers of CMRS (Commercial Mobile Radio Service), satellite, cable, data networking and ISPs. The companies participating in the trial would be asked to alert NCS/NCC of outages that are likely to have significant public impact. The group determined that industry associations should provide an informational notice to their membership to inform them of the voluntary outage reporting trial and encourage their participation. The group concluded that analysis of the data from the voluntary trial should be done by a neutral party. This analysis should be similar to the analysis conducted on wireline carrier segments. At the completion of the voluntary trial period (minimum 1 year) an evaluation of the usefulness of the data to participants and the FCC should be undertaken. The group decided that data from the voluntary trial should be held confidential. A FOIA exemption may be needed in order to facilitate participation in the voluntary trial. A process for reporting data during the voluntary trial, including the information that should be included in reports and the time frames for filing reports, can be found in the group's final report (see http://www.nric.org/fg/index.html).
Ms. Aduskevicz said that the group also concluded that carriers should utilize NRSC Wireline Outage Reporting Guidelines (Revised 1999) in complying with Section 63.100. These guidelines reflect updated reporting requirements affecting the 30,000 customer threshold, fire-related incidents, E911 and major airports. The revised Guidelines also clarify who should report and under what conditions reporting is required. A NRSC reporting template that provides a job aid for completing FCC Service Disruption Reports should be utilized by industry segments currently reporting in compliance with 63.100. This template should also be used by industry segments that will report as part of the voluntary trial of outage reporting for CMRS, satellite, cable, data networking, and ISPs. The template is available on the ATIS web site. Industry communication of NRIC "best practices" to CMRS, satellite, cable, ISP, and data networking service providers is recommended. The group recommended that as technology continues to evolve and consumers increasingly have multiple paths for communications, reporting processes should be reviewed with an eye to eliminating redundant or non-value-adding reporting requirements
Ms. Aduskevicz showed a template to guide companies participating in the voluntary outage reporting. The template showed the industry category, the reporting criteria as applied to that category and gave examples of outages reportable under those criteria.
Finally, Ms. Aduskevicz outlined the steps leading to the voluntary trial for non-wireline carriers. She said that the final report's recommendations had been accepted by NRIC IV, and that NRIC V was the "Kick Off" of the voluntary trial for non-wireline service providers. She said the NRIC V members needed to endorse the voluntary trial recommendation and agree to proceed with the implementation process. The NCS/NCC had accepted its proposed role in the voluntary data reporting process. NRIC V needed to charter a focus group to implement the reporting process for the voluntary trial. The industry associations needed to notify their members.
Next Rick Harrison gave a report on NRIC V's focus group 3, subcommittee 1. The report of the subcommittee has been submitted for posting on the FCC and ATIS web sites. The subcommittee identified 232 best practices grouped in five categories: power best practices, essential services best practices, network elements best practices, procedural best practices and supplier best practices. These are a compilation of recommendations from NRC I through NRIC IV. The subcommittee evaluated them and recommended changes. The recommendations are based on the subcommittee members' own expertise, industry survey data and a review of FCC reportable outages. Many on the subcommittee had been involved with earlier NRICs and had previously performed various reviews of the best practices. Power and procedural best practices were chosen for review because there had been increasing outage problems in these areas. After the subcommittee released its list of power best practices, there was improvement in that area. Essential services were chosen as a category because the FCC was particularly interested in that. The subcommittee regrouped, recategorized and consolidated the best practices that had existed since NRC I. The practices were also reworded to make them more generic. The original best practice categories had included switching, SS7, and DCS. The subcommittee found that the best practices for switching, SS7 and DCS were often the same. They decided to regroup those practices under a single new category, "network elements." The subcommittee thought that some of the newer entrants in telecommunications could better relate to such a category, which could include routers, for example, which hadn't been a concern in NRC I. Some of the newer industry members on the subcommittee found many of the best practices applicable to such elements as routers. Procedural outages were discovered throughout all the old categories; so these were also regrouped under a new category.
The subcommittee did determine, by reviewing outage reports, that outages are prevented and mitigated by implementation of best practices. The outage reports often showed that had a best practice been followed, the outage would not have occurred or would have been mitigated to such an extant that it would not have been reportable. A survey of carriers showed that alternate best practices submitted by some companies were either not best practices or not new. There are no new best practices. There are, however new applications for old best practices and new alternatives for implementing old best practices. Diversity best practices, for example, can be implemented with newer kinds of equipment. The subcommittee found that most of the best practices applied to newer equipment. In some cases, however, carriers were not protecting that equipment as they would switches. There are some best practices that have been rendered obsolete by equipment that renders technician involvement unnecessary, creating less opportunity for error.
The subcommittee found that carriers often do not pay enough attention to power equipment. The power plants are typically older. Technicians are somewhat afraid of power equipment. There is a lack of training and hands-on experience. Recent power outages often involved malfunctioning or lack of attention to alarms and lack of, or failure to follow, established methods of procedure for installation and cable mining.
Mr. Harrison concluded by stating that NRIC V should increase the awareness of existing best practices, should continue to update old best practices to make them applicable to new technology and understood by new technicians, and should continue to monitor outages for best practice applications.
Chairman Crowe then introduced Pawan Jaggi, Chairman of the NRIC V Steering Committee, to talk about the structure of NRIC V.
Mr. Jaggi said it was an honor for him to chair the steering committee for NRIC V. He said that the purpose of NRIC V is to provide recommendations to the FCC to ensure optimal reliability and interoperability of telecommunications networks, including packet-switched networks. Its term is from March 20, 2000 to January 6, 2002. It will have three focus groups: Focus Group 1 dealing with Y2K issues, Focus Group 2, dealing with network reliability, and Focus Group 3 dealing with interoperability. The three groups would be chaired, respectively, by John Pasqua of AT&T, Brian Moir of ICA and Ed Eckert of Nortel.
Focus Group 1 will have two subcommittees. The first will be chaired by Pete Spring of AT&T and will consider network issues. The second will be chaired by Bill Blatt of Nortel and will be concerned with CPE issues. Focus Group 1 will continue the work related to the year 2000 date rollover, review the effectiveness of the work done prior to the date change and analyze the impact of the date change on networks.
Focus Group 2 will have four subcommittees. The first will be chaired by Rick Harrison of Telcordia and will deal with best practices on traditional networks. The second will deal with best practices as applied to packet switching. Mr. Jaggi said that no Chairman for this second subcommittee had been designated and invited nominations after the meeting for this position, and for any other empty spots in the organizational structure. The third subcommittee will be led by P.J. Aduskevicz of AT&T and will deal with data reporting and analysis. The fourth subcommittee, also without a designated chairman, will report on data reporting and analysis for packet switching.
Focus Group 2 will evaluate and report on the reliability of the PSTN including the reliability of packet switched networks; evaluate guidelines that were intended to improve the quality of outage reporting for carriers required to report outages and refine these guidelines as necessary; monitor the process of voluntary reporting, analyze data and report on the efficiency of this process; and evaluate existing network outage reporting requirements and make recommendations for improvement. Focus Group 2 will also build on the work of the previous committee as appropriate and refine best practices as necessary; and evaluate and report on how well the best practices and applicable ANSI standards are being followed by carriers.
Focus Group 3 will have two subcommittees. The first, led by Massimo Sorbara of Globespan, will deal with spectrum compatibility. The second, led by Phil Kyees of Paradyne, will deal with spectrum management.
Focus Group 3 will make recommendations concerning technical standards to ensure spectral compatibility in wire-line networks, make recommendations concerning the development of spectrum management processes within the wire-line networks, and recommend and advise the industry and FCC on new interoperability issues that may arise as a result of convergence and digital packet networks.
Mr. Jaggi thanked the NRIC's returning member companies and welcomed the new members and said he hoped for more new participants, asking for volunteers after the meeting. He said that additional focus groups might be added. He said that the next step for NRIC V was to finalize the proposed organization and focus groups, identify the necessary subcommittees under the focus groups; solicit members for subcommittees; establish milestones and timelines for each focus group; and hold the next steering committee meeting on Monday April 10th.
Chairman Crowe asked for questions. Ed Hall of CTIA noted that the Network Interoperability Focus Group description mentioned only wireline carriers and asked if wireless carriers shouldn't be included. Chairman Crowe noted that the charter was given by the FCC, but he said he saw no reason why it couldn't be expanded and asked Commissioner Powell if that was correct. Commissioner Powell agreed that wireless networks could and should be included, especially in view of the changing dynamics of telecommunications.
Chairman Crowe said that economists continue to be puzzled by the enormous improvements in the economy and by the differences between the economy here in the United States and in other industrialized nations. He said the United States' economy was the result of the improvements in information technology over the last twenty years, the first fifteen of which were spent learning the new technologies. In the last five years, however, the payoff became visible. He said that NRIC V had a clear and meaningful contribution to make to the process, but it was going to depend on the involvement of the members. NRIC V could help ensure that the networks continued to benefit from rapidly changing technologies. Chairman Crowe asked Commissioner Powell if he had any comments.
Commissioner Powell said that he wanted to encourage the non-outage reporting carriers to become part of the voluntary reporting trial discussed by Ms. Aduskevicz. He said the whole purpose was to share experience in a way that would increase the welfare of the industry over all. It is not about some back door way for the Commission to get into your reliability business. He also wanted to encourage the members who hadn't been at the center of the industry, such as wireless and cable carriers, to become centrally involved in NRIC V by taking some of the chairmanships that were vacant in the focus group subcommittees.
Chairman Crowe asked for additional comments or questions from the members. There were none. The meeting was adjourned at 11:25.