*************************************************** NOTICE *************************************************** This document was converted from Word97 to ASCII Text format. Content from the original version of the document such as headers, footers, footnotes, endnotes, graphics, and page numbers will not show up in this text version. All text attributes such as bold, itallic, underlining, etc. from the original document will not show up in this text version. Features of the orginal document layout such as columns, tables, line and letter spacing, pagination, and margins will not be preserved in the text version. If you need the complete document, download the Word97, or Adobe Acrobat versions, if available. The path and name of the Word97 and Acrobat files will be the same as the ASCII Text file except that they will end with the letters wp, doc, or pdf respectively, instead of the letters txt. ***************************************************** Before the Federal Communications Commission Washington, D.C. 20554 In the Matter of Application by Bell Atlantic New York for Authorization Under Section 271 of the Communications Act To Provide In-Region, InterLATA Service in the State of New York ) ) ) ) ) ) ) ) ) CC Docket No. 99-295 MEMORANDUM OPINION AND ORDER Adopted: December 21, 1999 Released: December 22, 1999 By the Commission: Chairman Kennard and Commissioners Ness and Powell issuing separate statements; Commissioner Furchtgott-Roth concurring and issuing a statement. TABLE OF CONTENTS Paragraph I. INTRODUCTION AND OVERVIEW 1 II. BACKGROUND 17 A. STATUTORY FRAMEWORK 17 B. HISTORY OF THIS APPLICATION 21 C. NEW YORK COMMISSION AND DEPARTMENT OF JUSTICE EVALUATIONS 23 III. ANALYTICAL FRAMEWORK 29 A. ABSENCE OF UNBUNDLING RULES 29 B. SCOPE OF EVIDENCE IN THE RECORD 32 1. Procedural Framework 32 2. Motions To Strike 38 3. Ex Parte Submissions 41 C. FRAMEWORK FOR ANALYZING COMPLIANCE WITH STATUTORY REQUIREMENTS 43 1. Legal Standard 44 2. Evidentiary Case 47 IV. COMPLIANCE WITH SECTION 271 (C)(1)(A) 61 A. BACKGROUND 61 B. DISCUSSION 62 V. COMPLIANCE WITH CHECKLIST 63 A. CHECKLIST ITEM 1 - INTERCONNECTION 63 1. Non-Pricing Aspects of Interconnection 63 2. Pricing of Collocation 77 B. CHECKLIST ITEM 2 - UNBUNDLED NETWORK ELEMENTS 81 1. Operations Support Systems 82 2. Combinations of Unbundled Network Elements 229 3. Pricing of Network Elements 237 C. CHECKLIST ITEM 3 - POLES, DUCTS, CONDUITS, AND RIGHTS-OF-WAY 263 1. Background 263 2. Discussion 265 D. CHECKLIST ITEM 4-UNBUNDLED LOCAL LOOPS 268 1. Background 268 2. Discussion 273 E. CHECKLIST ITEM 5 -- UNBUNDLED LOCAL TRANSPORT 337 1. Background 337 2. Discussion 338 F. CHECKLIST ITEM 6 - UNBUNDLED LOCAL SWITCHING 343 1. Background 343 2. Discussion 346 G. CHECKLIST ITEM 7 349 1. 911 and E911 Access 349 2. Directory Assistance/Operator Services 351 H. CHECKLIST ITEM 8 - WHITE PAGES DIRECTORY LISTINGS 357 1. Background 357 2. Discussion 360 I. CHECKLIST ITEM 9 - NUMBERING ADMINISTRATION 362 1. Background 362 2. Discussion 364 J. CHECKLIST ITEM 10 - DATABASES AND ASSOCIATED SIGNALING 365 1. Background 365 2. Discussion 366 K. CHECKLIST ITEM 11 - NUMBER PORTABILITY 367 1. Background 367 2. Discussion 369 L. CHECKLIST ITEM 12 - LOCAL DIALING PARITY 372 1. Background 372 2. Discussion 374 M. CHECKLIST ITEM 13 -- RECIPROCAL COMPENSATION. 375 1. Background 375 2. Discussion. 376 N. CHECKLIST ITEM 14 - RESALE 378 1. Background 378 2. Discussion 381 3. Provisioning 400 VI. SECTION 272 COMPLIANCE 401 A. BACKGROUND 401 B. DISCUSSION 403 1. Structural, Transactional, and Accounting Requirements of Section 272 404 2. Nondiscrimination Safeguards of Section 272 417 3. Joint Marketing Requirements of Section 272 419 VII. PUBLIC INTEREST ANALYSIS 422 A. OVERVIEW 422 B. COMPETITION IN LOCAL EXCHANGE AND LONG DISTANCE MARKETS 425 1. Impact on Local Competition 426 2. Impact on Long Distance Competition 428 C. ASSURANCE OF FUTURE COMPLIANCE 429 1. Summary of Performance Reporting and Enforcement Mechanisms 431 2. Key Elements of the Enforcement Plan 433 D. OTHER ARGUMENTS 444 VIII. SECTION 271(D)(6) ENFORCEMENT AUTHORITY 446 IX. CONCLUSION 454 X. ORDERING CLAUSES 455 APPENDIX A: LIST OF COMMENTERS APPENDIX B: STATISTICAL METHODOLOGY APPENDIX C: ANALYSIS OF AVERAGE COMPLETED INTERVALS FOR NON- DISPATCH ORDERS USING CARRIER TO CARRIER AND GERTNER/BAMBERGER STUDY DATA I. INTRODUCTION AND OVERVIEW 1. In this Order, we grant Bell Atlantic's application to enter the interLATA long distance market in New York State based on our conclusion that Bell Atlantic has taken the statutorily required steps to open its local exchange and exchange access markets to competition. The market opening actions by the New York Commission and Bell Atlantic underlying our decision bring the telecommunications industry one step closer to realization of the full pro- competitive goals of the 1996 Telecommunications Act, and promise substantial benefits for consumers in the form of lower rates and innovative service packages. Bell Atlantic filed the application addressed in this Order with the Commission on September 29, 1999. Fifty-seven parties filed comments on the application on October 18, 1999. Of these, more than twenty parties supported grant of the application. Twenty-five parties filed reply comments on November 8, 1999. 2. Our decision today approving Bell Atlantic's application represents the culmination of extensive federal and state efforts implementing the Telecommunications Act of 1996. This action builds on the experience that this Commission has gained from reviewing prior section 271 applications and developing rules to implement section 251 of the Communications Act. Significantly, it also builds on the tireless efforts of the New York Commission, which has worked long and hard with Bell Atlantic and competitive local exchange companies (LECs) to ensure that local markets in New York are open to competition. 3. In enacting the telephony provisions of the 1996 Act, Congress envisioned fundamental pro-competitive changes in the then-existing telecommunications environment. To this end, Congress took the momentous step of requiring that the incumbent LECs open the traditionally non-competitive local exchange and exchange access markets to competition in order to foster the entry of alternative service providers. Once the Bell Operating Companies (BOCs) have opened their local markets to competition, the 1996 Act permits them to enter the in-region, interLATA toll market, thereby increasing competition in the long distance telecommunications market. 4. Unfortunately, implementation of this congressional vision of increased telecommunications competition has, in many instances, not proceeded swiftly or smoothly. For example, some of the section 271 applications that we have reviewed to date have fallen far short of the statutory requirements. Moreover, some carriers attacked sections 271-275 of the Act on constitutional grounds arguing that each constitutes an impermissible bill of attainder. The court roundly rejected this challenge, stating that these provisions "are constitutionally sound." We believe that the instant application represents a turning point in the process of implementing the 1996 Act, with a new focus by the BOCs on taking the steps necessary to open the local exchange and exchange access markets to competition. 5. While this is the first section 271 application to receive Commission approval, our decision here reflects the fundamental principles adopted in our prior section 271 orders. Thus, we apply the general standards developed in prior orders in evaluating section 271 compliance - whether the BOC is providing service to competitors at parity with its retail offerings or, when there is no analogous retail activity, whether the BOC's performance would allow an efficient competitor a meaningful opportunity to compete. Based on our growing experience in addressing issues involving the development of local exchange competition, we also apply these standards in a pragmatic fashion, thus building on our prior decisions. For example, we consider the overall picture presented by the record, rather than focusing on any one aspect of performance. 6. It is no coincidence that this historic first is recorded in New York, a state that has been a leader in opening local markets to competition for over fifteen years, and a state with one of the most rigorous, expert commissions in the nation. Without the dedicated work and unfailing persistence of the New York Commission over the past several years, it is unlikely that this application would have reached a point at which it merits approval. It is also noteworthy that New York State has some of the most intensely competitive local exchange and exchange access markets in the nation. This track record of successful competition places the present application in a different context from prior filings. For the first time, we can evaluate compliance with the requirements of section 271 in a market context, rather than relying solely on predictive judgment. 7. We applaud the dedicated efforts of the New York Commission, beginning shortly after passage of the 1996 Act, to work with Bell Atlantic and competitive LECs to ensure that Bell Atlantic would achieve compliance with section 271. A number of the parties to this proceeding also praise the work of the New York Commission. Even AT&T, which strongly opposes the application, agrees that the New York Commission has significantly advanced Bell Atlantic's progress toward compliance with section 271. MCI states that "[a]t the insistence of the New York State Public Service Commission . . . BA-NY has done much to open its local markets . . . " Nextlink, one of the competitors supporting the application, also cites with approval the "open, collaborative process that included independent third party testing, numerous industry workshops, and staff solicitation and review of detailed public comments." 8. The section 271 process in New York exemplifies the way in which rigorous state proceedings can contribute to the success of a section 271 application. There are a number of elements that were particularly important to the success of this process in opening local markets to competition consistent with the terms of the 1996 Act. These include: (1) full and open participation by all interested parties; (2) extensive independent third party testing of Bell Atlantic's operations support systems (OSS) offering; (3) development of clearly defined performance measures and standards; and (4) adoption of performance assurance measures that create a strong financial incentive for post-entry compliance with the section 271checklist by Bell Atlantic. While we accord applicants flexibility in demonstrating compliance with section 271, these elements played a vital role in the success of this application. 9. First, under the auspices of the New York Commission, both competitive LECs and Bell Atlantic participated fully in collaborative sessions and technical workshops to clarify or resolve issues. This ensured broad-based industry participation throughout the proceeding. 10. Second, extensive third party testing of Bell Atlantic's OSS in New York was also critical to the success of these proceedings. The OSS testing was conducted in two phases. Phase I consisted of development of a detailed and comprehensive plan to evaluate and test the OSS interfaces and the adequacy of Bell Atlantic's processes, procedures, and documentation to allow competitive LECs to access and use these systems. Phase II of the test involved: (1) building the interface and assessing the ease or complexity of developing interface software; and (2) executing the test plan using a pseudo-competitive LEC. The rigorous, comprehensive third party testing in New York identified numerous shortcomings in Bell Atlantic's OSS performance that were subsequently corrected and re-tested. KPMG released its final report on August 6, 1999, concluding that Bell Atlantic's OSS was commercially available and sufficient to handle reasonable, anticipated commercial volumes. 11. Third, the New York Commission developed, and continues to refine, inter-carrier performance measures and service quality standards in its Carrier-to-Carrier proceeding. For example, the New York Commission has instituted collaborative proceedings to address xDSL issues and is developing xDSL specific performance measures and standards. This effort represents an ongoing process as a number of additional standards remain under development. To ensure that the company's performance data or "metrics" are reported reliably in accordance with the New York Commission's definitions, New York staff and KPMG reviewed the adequacy of internal controls surrounding the data collection process. In addition, the New York Commission's staff verifies on a monthly basis that Bell Atlantic's reported results conform to the definitions developed in the Carrier-to-Carrier proceeding. The definitions and standards developed in that proceeding have done much to foster the development of consistent and meaningful data concerning Bell Atlantic's performance. This gives us greater confidence that our decision is based on performance data that accurately measures Bell Atlantic's actual performance. 12. Fourth, the New York Commission has adopted Bell Atlantic's proposal for self- effectuating performance assurance plans that will provide significant financial incentives for Bell Atlantic to maintain an open market and prevent "backsliding" in the future provision of service by Bell Atlantic to competitive LECs. It is important that these plans are designed to function automatically without imposing administrative and regulatory burdens on competitors. It is also significant that the New York Commission is committed to supervising the implementation of these plans. 13. The well established pro-competitive regulatory environment in New York in conjunction with recent measures to achieve section 271 compliance has, in general, created a thriving market for the provision of local exchange and exchange access service. Competitors in New York are able to enter the local market using all three entry paths provided under the Act. These new entrants are serving both residential and business customers in geographic areas throughout the state, although competition is most intense for business customers in urban areas, especially in New York City. As a result, the extent of competition in New York greatly exceeds that in the other states for which BOCs have filed section 271 applications. 14. Bell Atlantic estimates that competitors serve at least 1,118,180 lines in New York. According to Bell Atlantic, competitors serve at least 651,793 lines using their own facilities, 152,055 lines using the UNE platform, and 314,332 lines through resale. Bell Atlantic states that competitive LECs serve both residential and business customers. Bell Atlantic estimates that competitors in New York serve at least 35,753 residential lines over their own facilities. In addition, Bell Atlantic estimates that competitive LECs in New York provide service to 137,342 residential customers using the UNE platform and resell another 63,547 residential lines. Similarly, Bell Atlantic estimates that competitive LECs in New York serve at least 612,000 business customers over their own facilities. Competitive LECs serve an additional 14,713 business lines using the UNE platform and resell another 250,785 business lines. 15. Our action today clearly demonstrates that when a BOC takes the steps required to open its local markets to full competition, the company will be rewarded with section 271 authority to enter the long distance market. The market opening requirements of the 1996 Act demand substantial changes in the way the BOCs have historically done business, and opening the New York market to full local competition has not been an easy process for Bell Atlantic or the New York Commission. We commend their hard work in reaching this historic achievement. 16. Finally, we wish to emphasize that grant of this application may close this chapter of the proceeding, but it is not the end of the story. Bell Atlantic must continue to comply with the checklist requirements, and with the requirements of section 272 of the Act. Section 271(d)(6) provides specific tools that augment our preexisting enforcement authority, to be used if Bell Atlantic falls out of compliance with the conditions required for grant of its application. Most notably, section 271(d)(6) authorizes the Commission to suspend or revoke the authorization granted here. This is a powerful enforcement tool, which should create a strong incentive for Bell Atlantic to ensure that its performance does not diminish. We expect that Bell Atlantic will not risk facing the severe remedy of having its authority to market service suspended, but stress that we are prepared to use this remedy if Bell Atlantic's performance in implementing the checklist deteriorates. II. BACKGROUND A. Statutory Framework 17. In the 1996 Act, Congress conditioned BOC provision of in-region, interLATA service on compliance with certain provisions of section 271. Pursuant to section 271, BOCs must apply to this Commission for authorization to provide interLATA services originating in any in-region state. Congress has directed the Commission to issue a written determination on each application no later than 90 days after the application is filed. 18. To obtain authorization to provide in-region, interLATA services under section 271, the BOC must show that: (1) it satisfies the requirements of either section 271(c)(1)(A), known as "Track A" or 271(c)(1)(B), known as "Track B"; (2) it has "fully implemented the competitive checklist" or that the statements approved by the state under section 252 satisfy the competitive checklist contained in section 271(c)(2)(B); (3) the requested authorization will be carried out in accordance with the requirements of section 272; and (4) the BOC's entry into in- region, interLATA market is "consistent with the public interest, convenience, and necessity." The statute specifies that unless the Commission finds that these four criteria have been satisfied, the Commission "shall not approve" the requested authorization. 19. Section 271(d)(2)(A) requires the Commission to consult with the Attorney General before making any determination approving or denying a section 271 application. The Attorney General is entitled to evaluate the application "using any standard the Attorney General considers appropriate," and the Commission is required to "give substantial weight to the Attorney General's evaluation." Section 271(d)(2)(A) specifically provides, however, that "such evaluation shall not have any preclusive effect on any Commission decision." Thus, Congress clearly contemplated that, in some circumstances, the Commission could reach a different conclusion from the Department, even after giving "substantial weight" to the Department's views. 20. In addition, the Commission must consult with the relevant state commission to verify that the BOC has one or more state approved interconnection agreements with a facilities- based competitor, or a statement of generally available terms and conditions (SGAT), and that either the agreement(s) or general statement satisfy the "competitive checklist." In the Ameritech Michigan Order, the Commission determined that, because the Act does not prescribe any standard for Commission consideration of a state commission's verification under section 271(d)(2)(B), it has discretion in each section 271 proceeding to determine the amount of weight to accord to the state commission's verification. The Commission has held that, although it will consider carefully state determinations of fact that are supported by a detailed and extensive record, it is the Commission's role to determine whether the factual record supports the conclusion that particular requirements of section 271 have been met. In the instant proceeding, we accord the New York Commission's evaluation substantial weight, for the reasons set forth above. In particular, we note that the New York Commission has directed a rigorous collaborative process that has included: an extensive independent third-party test of Bell Atlantic's OSS interfaces, processes and procedures; active participation by New York Commission staff, Bell Atlantic, and competitive LECs in numerous technical conferences that helped to identify and resolve problems; and the development of a comprehensive performance monitoring and enforcement mechanism. Throughout these proceedings, the New York Commission has ensured that the process was open to participation by all interested parties and, as a result, received and reviewed a massive record of public comments. We thus place substantial weight on the New York Commission's conclusions, as they reflect its role not only as a driving force behind these proceedings, but also as an active participant in bringing local competition to the state's markets. B. History of this Application 21. On February 13, 1997, Bell Atlantic, filed a draft application under section 271, along with a Statement of Generally Applicable Terms and Conditions with the New York Commission. On July 8, 1997, after a number of technical conferences and collaborative meetings and technical and legal analyses, a New York Commission Administrative Law Judge concluded that Bell Atlantic had made a prima facie case regarding certain offerings, but had not met its burden of proof regarding commercial availability, procedure standardization, timeliness, and measuring parity. Subsequently, the New York Commission held additional collaborative sessions to work out technical details associated with development of a working Operations Support System (OSS). Specifically, these sessions resolved numerous OSS issues, including an agreement on business rules that would govern the development by competitors of systems to interface with those of Bell Atlantic. Following approval of the Bell Atlantic/NYNEX merger, Bell Atlantic filed a supplemental section 271 application with the New York Commission, which was followed by additional filings and technical conferences. After completion of this process, Bell Atlantic agreed to make additional commitments in connection with its application for section 271 approval. 22. On April 6, 1998, Bell Atlantic filed a Pre-Filing Statement with the New York Commission, which contained a number of commitments, including: 1) to provide combinations of elements (including UNE-P as a minimum service offering); 2) to engage a third-party to test Bell Atlantic's OSS; and 3) to establish a self-effectuating system to prevent backsliding. Pursuant to these commitments, Bell Atlantic obtained a comprehensive independent third-party test of its wholesale support systems and developed a plan to ensure adequate continuing wholesale performance. As described above, this test was conducted by KPMG Peat Marwick and Hewlett Packard under the supervision of the New York Commission. Together, the New York Commission and KPMG created an open testing environment in which they consulted with interested parties, issued draft plans and reports, and reported in detail on issues of serious concern. The problems identified through the test were addressed by Bell Atlantic through process improvements during the test period. The third-party test was completed with the release of KPMG's final report on August 6, 1999. As noted above, Bell Atlantic filed its application with this Commission on September 29, 1999. C. New York Commission and Department of Justice Evaluations 23. On October 18, 1999, the New York Commission submitted to this Commission its evaluation of Bell Atlantic's application. The New York Commission advised the Commission that, following two and half years of review, testing, and process improvements, Bell Atlantic-NY had met the checklist requirements of section 271(c). Specifically, New York stated that Bell Atlantic had met its obligation under section 271(c)(1)(A) by entering into more than 75 interconnection agreements approved by the New York Commission, and that competitive LECs are providing local exchange service in New York using their own facilities and those of Bell Atlantic. In addition, the New York Commission stated that the record developed in the New York proceeding establishes that Bell Atlantic has a legal obligation, under its interconnection agreements and state-approved tariffs, to provide the 14 items required under section 271's checklist, and that Bell Atlantic is meeting its legal obligation to provide those 14 items. 24. On November 1, 1999, the Department of Justice filed its evaluation. Consistent with its approach in past applications, the Department stated that it considers whether all three entry paths contemplated by the 1996 Act - facilities-based entry involving construction of new networks, the use of unbundled elements of the BOC's network, and resale of the BOC's services - are fully and irreversibly open to competitive entry to serve both business and residential customers. The Department of Justice found that "Bell Atlantic has completed most - but not all - of the actions needed to achieve a fully and irreversibly open market in New York." The Department concluded that it did not have substantial concerns about the ability of facilities-based carriers and firms that wish to resell Bell Atlantic's retail services to enter the local telecommunications markets in New York. It also concluded that Bell Atlantic has made "great progress in opening the market to competition through the use of unbundled network elements," but two major areas of deficiency-OSS and access to local loops - remain as important obstacles to local competition. The Department of Justice also concluded, however, that Bell Atlantic has not yet demonstrated that it can adequately provide access to unbundled local loops, either for traditional voice services or for digital subscriber line (DSL) technology used to provide a variety of advanced services. Moreover, the Department expressed concern that Bell Atlantic's systems for handling orders for the unbundled network platform rely on manual processes that are prone to error and delay. The Department expressly reserved judgment, however, on whether the facts in the record established compliance with the legal requirements of the competitive checklist or the Commission's rules. 25. The Department of Justice stated its belief that its assessment of the facts regarding Bell Atlantic's wholesale performance was substantially consistent with the New York's assessment. The Department of Justice noted that, to the extent there is a difference between its evaluation and that of the New York Commission, "it arises largely from the Department's conclusion that needed improvements should be achieved before Bell Atlantic is authorized to provide interLATA services in New York, rather than relying on post-271 approval mechanisms to attempt to ensure such improvements." 26. The Department urged us not to permit Bell Atlantic to offer interLATA services until "it demonstrates that it has solved the existing problems in its provision of access to unbundled network elements." It noted, however, that it "is possible that information from Reply Comments and ex parte submissions will provide additional support for Bell Atlantic's claims and justify a conclusion different from that reached by the Department on the basis of the current record." 27. The Department of Justice stated that this Commission could properly deny this application. As an alternative, the Department suggested the Commission might be able to approve the application subject to carefully crafted conditions "under which Bell Atlantic would be permitted to offer interLATA services only after taking specified steps and demonstrating that its performance has met appropriate requirements." The Department of Justice thus concluded that "the Commission may be able to approve Bell Atlantic's application at the culmination of these proceedings." 28. On November 8, 1999, the New York Commission, and 23 other parties, filed reply comments in this proceeding. Both Bell Atlantic and the New York Commission contended that the arguments raised in opposition are insufficient grounds for denying the application. III. ANALYTICAL FRAMEWORK A. Absence of Unbundling Rules 29. It is necessary to clarify, for the purpose of evaluating this application, which network elements we expect Bell Atlantic to demonstrate that it provides on an unbundled basis, pursuant to section 251(c)(3) and checklist item 2. In the Local Competition First Report and Order, the Commission established a list of seven UNEs which incumbent LECs were obliged to provide: (1) local loops; (2) network interface devices; (3) local and tandem switching; (4) interoffice transmission facilities; (5) signaling networks and call-related databases; (6) operations support systems; and (7) operator services and directory assistance. This obligation was codified in section 51.319 of the Commission's rules ("rule 319"). In January 1999, the Supreme Court vacated rule 319 and instructed the Commission to revise the standards under which the unbundling obligation is determined and to reevaluate the network elements subject to the unbundling requirement. 30. Although the former rule 319 was not in force at the time Bell Atlantic filed its application in this proceeding, Bell Atlantic has sought to demonstrate that it provides nondiscriminatory access to these network elements. Indeed, Bell Atlantic has stated that it believes it would be "reasonable" for the Commission to use the original seven network elements identified in former rule 319 in evaluating this application. In assessing Bell Atlantic's argument, we begin from the premise that compliance with the competitive checklist requires that Bell Atlantic provide nondiscriminatory access to network elements, as contemplated by, and in accordance with, the requirements of sections 251(c)(3) and 251(d)(2). We believe that using the network elements identified in former rule 319 as a standard in evaluating Bell Atlantic's application, during the interim period between its vacation by the Supreme Court and the effective date of the new rules, is a reasonable way to ensure that the application complies with the checklist requirements. We find it significant that no commenter has taken the position in this proceeding that Bell Atlantic should not be required to demonstrate that it provides these network elements. Accordingly, for the purposes of this application, we will evaluate whether Bell Atlantic provides nondiscriminatory access to the seven network elements identified under former rule 319. 31. We disagree with commenters that contend that Bell Atlantic must demonstrate, for the purposes of this application, compliance with the rules governing unbundled network elements recently established in the UNE Remand proceeding. These new rules, among other things, specify which network elements an incumbent LEC is obliged to unbundle, and establish several new obligations that were not present under the former rule 319. We recognize, however, that these new rules will not take effect until some time after release of this order. Therefore, we will not require Bell Atlantic to prove that it currently complies with rules that have yet to take effect. Moreover, we believe it would be inequitable to require Bell Atlantic to comply with these rules, particularly when no other incumbent LEC must comply before the effective date, just because Bell Atlantic has a section 271 application pending before the Commission. Of course, the Commission expects that Bell Atlantic will comply with the new UNE Remand rules once they take effect. B. Scope of Evidence in the Record 1. Procedural Framework 32. Section 271 proceedings are, at their core, adjudications that the Act requires the Commission to complete within ninety days of the application filing. The statute also requires us to consult with the Department of Justice and the relevant state commission in reviewing the application. 33. In the context of this statutory framework, the Commission has established procedural rules governing BOC section 271 applications. Among other things, these rules provide an opportunity for parties other than the Department of Justice and the relevant state commission to comment on section 271 applications. 34. Under our procedural rules governing BOC section 271 applications, we expect that a section 271 application, as originally filed, will include all of the factual evidence on which the applicant would have the Commission rely in making its findings. An applicant may not, at any time during the pendency of its application, supplement its application by submitting new factual evidence that is not directly responsive to arguments raised by parties commenting on its application. This includes the submission, on reply, of factual evidence gathered after the initial filing. In an effort to meet its burden of proof, however, a BOC may submit new factual information after the application is filed, if the sole purpose of that evidence is to rebut arguments or facts submitted by other commenters. The new evidence, however, must cover only the period placed in dispute by commenters and may, in no event, post-date the filing of the comments (i.e., day 20). In the event that the applicant submits new or post-dated evidence in replies or ex parte filings, we retain the discretion to start the 90-day review process anew or to accord such evidence no weight. 35. This precedent has served the Commission well, by deterring incomplete filings from the BOCs. In particular, the rule is designed to prevent applicants from presenting part of their initial prima facie showing for the first time in reply comments. The rule has enabled us properly to manage our own internal consideration of the application and ensures that commenters are not faced with a "moving target" in the BOC's section 271 application. We continue to believe, as a general matter, that it is highly disruptive to our processes to have a record that is constantly evolving. We emphasize, however, that our precedent makes clear that this rule is a discretionary one. 36. We do not expect that a BOC, in its initial application, will anticipate and address every foreseeable argument its opponents might make in their subsequent reply comments, but we have previously stated that a BOC must address in its initial application all facts that the BOC can reasonably anticipate will be at issue. Through state proceedings, BOCs should be able reasonably to identify and anticipate certain arguments and allegations that parties will make in their filings before the Commission. 37. In addition, the Commission has found that a BOC's promises of future performance to address particular concerns raised by commenters have no probative value in demonstrating its present compliance with the requirements of section 271. In order to gain in- region, interLATA entry, a BOC must support its application with actual evidence demonstrating its present compliance with the statutory conditions for entry, instead of prospective evidence that is contingent on future behavior. Thus, we must be able to make a determination based on the evidence in the record that a BOC has actually demonstrated compliance with the requirements of section 271. 2. Motions To Strike 38. On November 22, 1999, AT&T filed a motion to strike or to disregard portions of the reply submissions of Bell Atlantic and the New York Commission filed in this proceeding. AT&T argues that reply submissions of both Bell Atlantic and the New York Commission contain material that must be stricken or accorded no weight under the Commission's rules because they post-date Bell Atlantic's application and the due date for comments. In addition, AT&T argues that Bell Atlantic's reply submission contains numerous new promises of future performance. 39. We deny AT&T's motion because we do not rely, as a basis for our decision, on: (1) evidence submitted by Bell Atlantic after filing its application, unless such evidence both relates to events that occurred prior to the comment filing date (October 19, 1999) and is directly responsive to allegations in the record; (2) evidence submitted by the New York Commission that post-dates the comment due date; or (3) Bell Atlantic's promises of future compliance. 40. On December 17, 1999, Covad filed a motion to strike an ex parte submission filed by Bell Atlantic on December 10, 1999. We deny Covad's motion because we do not rely on Bell Atlantic's ex parte submission as a basis for our decision. 3. Ex Parte Submissions 41. Under the procedural rules governing section 271 applications, we strongly encourage parties to set forth their views comprehensively in their formal submissions (i.e., Brief in Support, oppositions, supporting comments, etc.), and not to rely on subsequent ex parte presentations. At the same time, the Commission expressly provided that parties may file ex partes. Our procedural Public Notice thus clearly contemplates that parties may file written ex partes, when appropriate, to clarify the record. We take this opportunity to clarify that like reply comments, ex partes must be directly responsive to arguments raised by parties commenting on the application. Such ex partes may, however, elaborate on, or provide additional explanation or detail in response to requests from Commission staff or in direct response to post-reply ex parte filings. 42. Nothing in our procedural rules or past precedent precludes the Commission and the staff from requesting clarification or an explanation about information or data contained in the filings specified above. Indeed, our procedural Public Notice expressly recognizes that the Commission may request additional information from the applicant, as the page limit for ex partes does not apply to written material filed in response to direct requests from Commission staff. It is critical to the agency's deliberative process that the Commission and staff fully understand the evidence and arguments presented in the BOC's section 271 application, arguments raised in opposition, and responses made by parties on reply. Accordingly, the Commission retains the discretion to request additional information from the applicant or other parties that elaborates on positions set forth in the original application, comments, or reply comments. We emphasize that we are not departing from our view that the applicant should set forth its position in a clear and concise manner in its formal filings. However, it is imperative that, as part of the Commission's deliberative process, we have the ability to engage in an ongoing dialogue with parties to ensure that we have a clear and accurate understanding of the information contained in all formal submissions. C. Framework for Analyzing Compliance with Statutory Requirements 43. In this section, we discuss two aspects of the framework for analyzing compliance with the statutory requirements of section 271. First, we discuss the legal standards we have enunciated in past orders for determining whether a BOC is meeting the statutory nondiscrimination requirements. Second, we discuss the evidentiary requirements of a BOC's section 271 application and, in particular, the types of showings we will find probative in deciding whether a BOC has met the statutory standards. 1. Legal Standard 44. In order to comply with the requirements of section 271's competitive checklist, a BOC must demonstrate that it has "fully implemented the competitive checklist in subsection (c)(2)(B)." In particular, the BOC must demonstrate that it is offering interconnection and access to network elements on a nondiscriminatory basis. Previous Commission orders addressing section 271 applications have elaborated on this statutory standard. First, for those functions the BOC provides to competing carriers that are analogous to the functions a BOC provides to itself in connection with its own retail service offerings, the BOC must provide access to competing carriers in "substantially the same time and manner" as it provides to itself. Thus, where a retail analogue exists, a BOC must provide access that is equal to (i.e., substantially the same as) the level of access that the BOC provides itself, its customers, or its affiliates, in terms of quality, accuracy, and timeliness. For those functions that have no retail analogue, the BOC must demonstrate that the access it provides to competing carriers would offer an efficient carrier a "meaningful opportunity to compete." As we stated in the Ameritech Michigan Order, there may be situations in which a BOC contends that, although equivalent access has not been achieved for an analogous function, the access that it provides is still nondiscriminatory within the meaning of the statute. 45. We do not view the "meaningful opportunity to compete" standard to be a weaker test than the "substantially the same time and manner" standard. Where the BOC provides functions to its competitors that it also provides for itself in connection with its retail service, its actual performance can be measured to determine whether it is providing access to its competitors in "substantially the same time and manner" as it does to itself. Where the BOC, however, does not provide a retail service that is similar to its wholesale service, its actual performance with respect to competitors cannot be measured against how it performs for itself because the BOC does not perform analogous activities for itself. In those situations, our examination of whether the quality of access provided to competitors offers competitors "a meaningful opportunity to compete" is intended to be a proxy for whether access is being provided in substantially the same time and manner and, thus, nondiscriminatory. 46. Finally, we note that a determination of whether the statutory standard is met is ultimately a judgment we must make based on our expertise in promoting competition in local markets and in telecommunications regulation generally. We have not established, nor do we believe it appropriate to establish, specific objective criteria for what constitutes "substantially the same time and manner" or a "meaningful opportunity to compete." We look at each application on a case-by-case basis and consider the totality of the circumstances, including the origin and quality of the information before us, to determine whether the nondiscrimination requirements of the Act are met. Whether this legal standard is met can only be decided based on an analysis of specific facts and circumstances. 2. Evidentiary Case 47. We previously have set forth the analytical framework that we use in assessing whether a BOC has demonstrated compliance with the statutory requirements of section 271. At the outset, we reemphasize that the BOC applicant retains at all times the ultimate burden of proof that its application satisfies all of the requirements of section 271, even if no party files comments challenging its compliance with a particular requirement. 48. The evidentiary standards governing our review of section 271 applications are intended to balance our need for reliable evidence against our recognition that, in such a complex endeavor as a section 271 proceeding, no finder of fact can expect proof to an absolute certainty. While we expect the BOC to demonstrate as thoroughly as possible that it satisfies each checklist item, the public interest standard, and the other statutory requirements, we reiterate that the BOC needs only to prove each element by "a preponderance of the evidence," which generally means "the greater weight of evidence, evidence which is more convincing that the evidence which is offered in opposition to it." 49. As we held in the Second BellSouth Louisiana Order, we first determine whether the BOC has made a prima facie case that it meets the requirements of a particular checklist item. The BOC must plead, with appropriate supporting evidence, facts which, if true, are sufficient to establish that the requirements of section 271 have been met. Once the BOC has made such a showing, opponents must produce evidence and arguments to show that the application does not satisfy the requirements of section 271, or risk a ruling in the BOC's favor. 50. When considering commenters' filings in opposition to the BOC's application, we look for evidence that the BOC's policies, procedures, or capabilities preclude it from satisfying the requirements of the checklist item. Mere unsupported evidence in opposition will not suffice. Although anecdotal evidence may be indicative of systemic failures, isolated incidents may not be sufficient for a commenter to overcome the BOC's prima facie case. Moreover, a BOC may overcome such anecdotal evidence by, for example, providing objective performance data that demonstrate that it satisfies the statutory nondiscrimination requirement. 51. We will look to the state to resolve factual disputes wherever possible. Indeed, we view the state's and the Department of Justice's role to be one similar to that of an "expert witness." Given the 90-day statutory deadline to reach a decision on a section 271 application, the Commission does not have the time or the resources to resolve the enormous number of factual disputes that inevitably arise from the technical details and data involved in such a complex endeavor. Accordingly, as discussed above, where the state has conducted an exhaustive and rigorous investigation into the BOC's compliance with the checklist, we may give evidence submitted by the state substantial weight in making our decision. Although we are statutorily required to accord substantial weight to the Department of Justice's evaluation, in appropriate circumstances, we may conclude that the evidence submitted by a state commission is more persuasive than that submitted by the Department of Justice, particularly if the state has conducted a rigorous analysis of the evidence. 52. To make a prima facie case that the BOC is meeting the requirements of a particular checklist item under section 271(c)(1)(A), the BOC must demonstrate that it is providing access or interconnection pursuant to the terms of that checklist item. In particular, a BOC must demonstrate that it has a concrete and specific legal obligation to furnish the item upon request pursuant to state-approved interconnection agreements that set forth prices and other terms and conditions for each checklist item, and that it is currently furnishing, or is ready to furnish, the checklist item in quantities that competitors may reasonably demand and at an acceptable level of quality. 53. The particular showing required to demonstrate compliance will vary depending on the individual checklist item and the circumstances of the application. We have given BOCs substantial leeway with respect to the evidence they present to satisfy the checklist. Although our orders have provided guidance on which types of evidence we find more persuasive, "we reiterate that we remain open to approving an application based on other types of evidence if a BOC can persuade us that such evidence demonstrates nondiscriminatory treatment and other aspects of the statutory requirements." In past orders we have encouraged BOCs to provide performance data in their section 271 applications to demonstrate that they are providing nondiscriminatory access to unbundled network elements to requesting carriers. We have concluded that the most probative evidence that a BOC is providing nondiscriminatory access is evidence of actual commercial usage. Performance measurements are an especially effective means of providing us with evidence of the quality and timeliness of the access provided by a BOC to requesting carriers. 54. A number of state commissions, including New York, have established a collaborative process through which they have developed, in conjunction with the incumbent and competing carriers, a set of measures, or metrics, for reporting of performance in various areas. Through such collaborative processes, New York has also adopted performance standards for certain functions, typically where there can be no comparable measure based on the incumbent LEC's retail performance. We strongly encourage this type of process, because it allows the technical details that determine how the metrics are defined and measured to be worked out with the participation of all concerned parties. We also strongly support the efforts of state commissions to build and oversee a process that ensures the development of local competition that Congress intended. An extensive and rigorous evaluation of the BOC's performance by the states provides greater certainty that barriers to competition have been eliminated and the local markets in a state are open to competition. 55. We caution, however, that adoption by a state of a particular performance standard pursuant to its state regulatory authority is not determinative of what is necessary to establish checklist compliance under section 271. We recognize that metric definitions and incumbent LEC operating systems will likely vary among states, and that individual states may set standards at a particular level that would not apply in other states and that may constitute more or less than the checklist requires. Therefore, it is unlikely that we will see uniform standards that measure precisely the same BOC conduct across states. At the same time, for functions for which there are no retail analogues, and for which performance benchmarks have been developed with the ongoing participation of affected competitors and the BOC, those standards may well reflect what competitors in the marketplace feel they need in order to have a meaningful opportunity to compete. 56. We emphasize that, because the Commission is statutorily required to determine checklist compliance, we must independently evaluate whether a BOC is fulfilling the nondiscrimination requirements of section 271. Nevertheless, in making our evaluation we will examine whether the state commission has adopted a retail analogue or a benchmark to measure BOC performance and then review the particular level of performance the state has required. If the state commission has made these determinations in the type of rigorous collaborative proceeding described above, we are much more likely to find that they are reasonable and appropriate measures of parity. Accordingly, we are inclined to rely on such standards and measurements in our own analysis but may reach a different conclusion where justified. 57. In the instant proceeding, for example, the New York Commission has determined, through a collaborative process with input from Bell Atlantic and competing carriers, that there are retail analogues for certain functions and performance benchmarks for others. We find this to be a reasonable basis for us to begin our analysis. Under the framework adopted by the New York Commission, Bell Atlantic determines whether any difference in its performance compared to its retail operations is statistically significant, and provides a figure indicating the degree of statistical significance. For measures where the New York Commission has set a performance benchmark, the New York Commission has required Bell Atlantic to provide the metrics for its performance to competing carriers, which can then be compared to the benchmark. 58. In this case, we conclude that to the extent there is no statistically significant difference between Bell Atlantic's provision of service to competitive LECs and its own retail customers, we need not look any further. Similarly, if there is no difference between the Bell Atlantic provision of service to competitive LECs and the performance benchmark, our analysis is done. 59. To the extent there is any statistically significant difference between Bell Atlantic's provision of service to competitive LECs and retail customers or an apparent difference between its provision of service to competitive carriers and the performance benchmarks set by the New York Commission, we will examine the evidence further to make a determination whether the statutory nondiscrimination requirements are met. Thus, we will examine the explanation that Bell Atlantic and other commenters provide about whether these differences provide an accurate depiction of the quality of Bell Atlantic's performance. For instance, we may examine the data on a more disaggregated level, in order to evaluate arguments made by Bell Atlantic that competitive LEC error, or differences in the composition of competitive LEC orders, or sudden changes in the quantity or timing of orders made by competitive LECs, are responsible for the apparent poor performance. We also may examine how many months a variation in performance has existed and what the trend has been in recent months. A steady improvement in performance over time may provide us with an indication that problems are being resolved. It may also provide us with evidence as to whether Bell Atlantic's systems are scaleable and can handle large volumes of orders for services. Finally, in some instances, we may find that statistically significant differences in measured performance may exist, but that such differences have little or no competitive significance in the marketplace. As such, we may deem such differences non-cognizable under the statutory standard. 60. The determination of whether a BOC's performance meets the statutory requirements necessarily is a contextual decision based on the totality of the circumstances and information before us. There may be multiple performance measures associated with a particular checklist item, and an apparent disparity in performance for one measure, by itself, may not provide a basis for finding noncompliance with the checklist. Other measures may tell a different story, and provide us with a more complete picture of the quality of service being provided. Thus, whether we are applying the "substantially same time and manner" standard or the "meaningful opportunity to compete" standard, we will examine whether the differences in the measured performance are large enough to be deemed discriminatory under the statute. IV. COMPLIANCE WITH SECTION 271 (C)(1)(A) A. Background 61. In order for the Commission to approve a BOC's application to provide in-region, interLATA services, a BOC must first demonstrate that it satisfies the requirements of either section 271(c)(1)(A) (Track A) or 271(c)(1)(B) (Track B). To qualify for Track A, a BOC must have interconnection agreements with one or more competing providers of "telephone exchange service . . . to residential and business subscribers." The Act states that "such telephone service may be offered . . . either exclusively over [the competitor's] own telephone exchange service facilities or predominantly over [the competitor's] own telephone exchange facilities in combination with the resale of the telecommunications services of another carrier." The Commission concluded in the Ameritech Michigan Order that, when a BOC relies upon more than one competing provider to satisfy section 271(c)(1)(A), each carrier need not provide service to both residential and business customers. B. Discussion 62. We conclude that Bell Atlantic demonstrates that it satisfies the requirements of Track A based on the interconnection agreements it has implemented with competing carriers in New York. Specifically, we find that AT&T, MCI WorldCom, and Cablevision Lightpath provide telephone exchange service either exclusively or predominantly over their own facilities to residential subscribers and to business subscribers. The New York Commission also concludes that Bell Atlantic has met the requirements of section 271(c)(1)(A). None of the commenting parties, including the competitors cited by Bell Atlantic in support of its showing, challenge Bell Atlantic's assertion in this regard. Thus, Bell Atlantic meets the requirements of section 271(c)(1)(A). V. COMPLIANCE WITH CHECKLIST A. Checklist Item 1 - Interconnection 1. Non-Pricing Aspects of Interconnection a. Background 63. Section 271(c)(2)(B)(i) of the Act requires a section 271 applicant to provide "[i]nterconnection in accordance with the requirements of sections 251(c)(2) and 252(d)(1)." Section 251(c)(2) imposes a duty on incumbent LECs "to provide, for the facilities and equipment of any requesting telecommunications carrier, interconnection with the local exchange carrier's network . . . for the transmission and routing of telephone exchange service and exchange access." In the Local Competition First Report and Order, the Commission concluded that interconnection referred "only to the physical linking of two networks for the mutual exchange of traffic." Section 251 contains three requirements for the provision of interconnection. First, an incumbent LEC must provide interconnection "at any technically feasible point within the carrier's network." Second, an incumbent LEC must provide interconnection that is "at least equal in quality to that provided by the local exchange carrier to itself." Finally, the incumbent LEC must provide interconnection "on rates, terms, and conditions that are just, reasonable, and nondiscriminatory, in accordance with the terms of the agreement and the requirements of [section 251] and section 252." 64. To implement the equal-in-quality requirement in section 251, the Commission's rules require an incumbent LEC to design and operate its interconnection facilities to meet "the same technical criteria and service standards" that are used for the interoffice trunks within the incumbent LEC's network. In the Local Competition First Report and Order, the Commission identified trunk group blockage and transmission standards as indicators of an incumbent LEC's technical criteria and service standards. In prior section 271 applications, the Commission concluded that disparities in trunk group blockage indicated a failure to provide interconnection to competing carriers equal-in-quality to the interconnection the BOC provided to its own retail operations. 65. In the Local Competition First Report and Order, the Commission concluded that the requirement to provide interconnection on terms and conditions that are "just, reasonable, and nondiscriminatory" means that an incumbent LEC must provide interconnection to a competitor in a manner no less efficient than the way in which the incumbent LEC provides the comparable function to its own retail operations. The Commission's rules interpret this obligation to include, among other things, the incumbent LEC's installation time for interconnection service and its provisioning of two-way trunking arrangements. Similarly, repair time for troubles affecting interconnection trunks is useful for determining whether a BOC provides interconnection service under "terms and conditions that are no less favorable than the terms and conditions" the BOC provides to its own retail operations. 66. Competing carriers may also choose any method of technically feasible interconnection at a particular point on the incumbent LEC's network. Incumbent LEC provision of interconnection trunking is one common means of interconnection. Technically feasible methods also include, but are not limited to, physical and virtual collocation and meet point arrangements. In the Advanced Services First Report and Order, the Commission revised its collocation rules to require incumbent LECs to include shared cage and cageless collocation arrangements as part of their physical collocation offerings. The provision of collocation is an essential prerequisite to demonstrating compliance with item 1 of the competitive checklist. To show compliance with its collocation obligations, a BOC must have processes and procedures in place to ensure that all applicable collocation arrangements are available on terms and conditions that are "just, reasonable, and nondiscriminatory" in accordance with section 251(c)(6) and our implementing rules. Data showing the quality of procedures for processing applications for collocation space, as well as the timeliness and efficiency of provisioning collocation space, helps the Commission evaluate a BOC's compliance with its collocation obligations. b. Discussion 67. We are persuaded, for the reasons discussed below, that Bell Atlantic demonstrates that, in New York, it provides equal-in-quality interconnection on terms and conditions that are just, reasonable, and nondiscriminatory in accordance with the requirements of section 251(c)(2) and 252(d)(1), as specified in section 271. We further find that Bell Atlantic meets its burden of proof that it designs its interconnection facilities to meet "the same technical criteria and service standards" that are used for the interoffice trunks within its own network, and that Bell Atlantic makes interconnection available at any technically feasible point. Finally, we find that Bell Atlantic demonstrates that it is providing collocation in New York in accordance with the Commission's rules. (i) Interconnection Trunking 68. Based on our review of the record, we are persuaded that Bell Atlantic provides competing carriers with interconnection trunking in New York that is equal-in-quality to the interconnection Bell Atlantic provides to its own retail operations, and on terms and conditions that are just, reasonable, and nondiscriminatory. Bell Atlantic makes interconnection available in New York through interconnection agreements and through a state approved tariff. Bell Atlantic receives orders for interconnection trunks through the Access Service Request (ASR) process, and accepts ASRs through an electronic application-to-application interface, its Internet Web Graphical User Interface (GUI), and manual orders. In addition, Bell Atlantic provides performance data to measure the quality of interconnection service provided to competing carriers. 69. In prior section 271 applications, we relied heavily on trunk group blockage data to evaluate a BOC's interconnection quality. Bell Atlantic's performance data show that, in the months leading up to its application, Bell Atlantic provided interconnection using the level of service that is received in its own network. Specifically, Bell Atlantic's performance data show that, for the three months immediately preceding its section 271 application, interconnection trunk groups provided to competing carriers experienced blockage less frequently than Bell Atlantic's own retail trunk groups. The comments of the New York Commission, Intermedia, and Nextlink corroborate Bell Atlantic's performance data, and further indicate that Bell Atlantic provides interconnection equal-in-quality to the interconnection provided to Bell Atlantic's own retail operations. As a final matter, we note that the failure of any commenter to raise trunk group blockage as an issue further supports our conclusion that Bell Atlantic adequately designs its interconnection facilities to ensure calls are completed. 70. We find that other aspects of Bell Atlantic's data further indicate that Bell Atlantic is providing nondiscriminatory interconnection trunking in New York. Bell Atlantic's performance data show that, for July and August 1999, Bell Atlantic rarely missed installation appointments for provisioning interconnection trunks for competitors. In fact, Bell Atlantic missed installation appointments for local exchange competitors less often than it did for interexchange carriers in July and August, and we note that Bell Atlantic's data show that Bell Atlantic provided comparable installation quality through September. 71. We have examined the issues pointed out by the Department of Justice, Teligent, e.spire, Allegiance, and others regarding Bell Atlantic's provisioning of new and large orders of interconnection trunks. These parties generally argue that requesting carriers have experienced unreasonable delays in Bell Atlantic provisioning of new and large orders of interconnection trunks. In its application, Bell Atlantic submitted performance data that showed a statistically significant difference between the provisioning of trunks for competitive LECs and for interexchange carriers as reflected in some performance measurements related to provisioning large orders of interconnection trunks. After further analysis and discussion with the Commission, Bell Atlantic identified significant errors in its New York Carrier-to-Carrier Performance Reports, and submitted revised data. In addition, Bell Atlantic submitted supplementary data to show its provisioning performance for interconnection trunks provided to both competitive LECs and interexchange carriers. Our review of Bell Atlantic's supplementary data shows that, although its provisioning performance has deteriorated since January 1999, Bell Atlantic's provisioning of interconnection trunks for competitive LECs is comparable to its performance for interexchange carriers, which indicates that Bell Atlantic is meeting its equal-in- quality obligations. We therefore conclude that, while the claims of e.spire and others may very well be true, evidence of such provisioning delays does not preclude a showing of compliance for section 271 purposes, so long as the equal-in-quality requirement is met. 72. We conclude that our decision that Bell Atlantic meets checklist item 1 rests upon its demonstration that trunk group blockage for competitors is lower than for Bell Atlantic's retail operations, Bell Atlantic's rate of missed installation appointments is lower for service to local competitors than for service to interexchange carriers, and there is no significant difference between its provisioning of interconnection trunks to local competitors and to interexchange carriers. For the benefit of future section 271 applications, and for purposes of evaluating Bell Atlantic's continued compliance with section 271(c)(2)(B)(I), we emphasize that our conclusion is based on a weighing of the various factors discussed in the foregoing paragraphs. A different combination of factors in another case might well lead us to conclude that, on the whole, competitive LECs do not receive equal-in-quality interconnection on just, reasonable, and nondiscriminatory terms and conditions. (ii) Collocation 73. Bell Atlantic has demonstrated that its collocation offering in New York satisfies the requirements of sections 271 and 251 of the Act. Bell Atlantic provides physical and virtual collocation through a state-approved tariff. In its application, Bell Atlantic indicates that shared, cageless, and adjacent collocation options are available in New York, and that it has taken other steps to implement the collocation requirements contained in the Advanced Services First Report and Order. In addition, Bell Atlantic demonstrates that it has deployed methods and procedures designed to ensure that its business units implement the Commission's collocation rules, including the designation of employees dedicated to providing collocation to competitive LECs, standard operating procedures related to collocation, and its CLEC HANDBOOK, which informs collocators of their rights and responsibilities. A number of commenters, including the New York Commission and several competitive LECs, agree with Bell Atlantic that its collocation offerings have been revised to reflect the requirements specified in the Advanced Services First Report and Order. 74. We disagree with the contentions of ALTS that the New York state tariff, and the New York Commission tariff review process, do not adequately ensure that Bell Atlantic's collocation offerings are consistent with section 251 and the Commission's rules. Specifically, ALTS contends that terms in the New York state tariff delay the provisioning of collocation space and impose restrictions on methods of interconnection and access to collocation. In addition, ALTS argues that the New York tariff does not clarify Bell Atlantic's allocation of collocation costs. After reviewing the record, we are persuaded by the New York Commission that Bell Atlantic is meeting its collocation obligations. Bell Atlantic revised its tariffed collocation offering to make it consistent with our Advanced Services First Report and Order. Bell Atlantic's collocation tariff underwent an active and thorough review at the state level. The New York Commission addressed the provisioning of collocation space and established standard provisioning intervals for caged, cageless, and virtual collocation. 75. Our review of Bell Atlantic's collocation performance data indicates that Bell Atlantic responds to applications for collocation space in a timely manner. Between May 1999 and August 1999, Bell Atlantic processed 667 requests for collocation space and almost always responded to such requests within the 8-day standard set by the New York Commission. Although we are concerned that Bell Atlantic's performance data shows recent failures to meet the 76-day provisioning interval established by the New York Commission for physical collocation, our finding of checklist compliance is predicated on Bell Atlantic's demonstration that 95% of the time it provisions collocation within the 76-day provisioning interval established by the New York Commission. Should these recent failures lead to a more widespread deterioration in provisioning collocation, however, enforcement action pursuant to section 271(d)(6) may be appropriate. (iii) Technically Feasible Points of Interconnection 76. We conclude that Bell Atlantic provides interconnection at all technically feasible points, as required by our rules, and therefore demonstrates checklist compliance. Bell Atlantic asserts that it makes interconnection available at all technically feasible points, including trunk- side at Bell Atlantic end offices and access tandems and line-side at Bell Atlantic end offices. Bell Atlantic demonstrates that it has an approved state tariff that spells out readily available points of interconnection, and provides a process for requesting interconnection at additional, technically-feasible points. We disagree with Sprint that its experience negotiating interconnection agreements with Bell Atlantic conclusively demonstrates that Bell Atlantic has violated its obligation to permit competing carriers to select interconnection points. Sprint's experience does not constitute evidence of systematic failures by Bell Atlantic to provide interconnection at all technically feasible points. Bell Atlantic points out that a state-approved process enables competitive LECs to obtain interconnection at technically feasible points not specified in the tariff, and the comments of the New York Commission support this statement. We agree with the New York Commission that the pending arbitration between Sprint and Bell Atlantic is the appropriate forum for addressing this issue. As a final matter, we conclude that Bell Atlantic has demonstrated that it provides two-way trunking in accordance with our rules, and no commenter presents credible information to show otherwise. 2. Pricing of Collocation a. Background 77. In order to comply with its collocation obligations, a BOC must make physical and virtual collocation arrangements available at rates that are "just, reasonable, and nondiscriminatory" in accordance with section 251(c)(6) of the Act and our rules implementing that section. Although the Commission's pricing rules were stayed by the U.S. Court of Appeals for the Eighth Circuit in 1996, pricing authority was restored by the Supreme Court on January 25, 1999. In reaching its decision, the Court acknowledged that section 201(b) "explicitly grants the FCC jurisdiction to make rules governing matters to which the 1996 Act applies." Furthermore, the Court determined that section 251(d) also provides evidence of an express jurisdictional grant by requiring that "the Commission [shall] complete all actions necessary to establish regulations to implement the requirements of this section." The Court also held that the pricing provisions implemented under the Commission's rulemaking authority do not inhibit the establishment of rates by the States. The Court concluded that the Commission has jurisdiction to design a pricing methodology to facilitate local competition under the 1996 Act, including pricing for interconnection and unbundled access, as "it is the States that will apply those standards and implement that methodology, determining the concrete result." b. Discussion 78. Based on the evidence in the record, we find that Bell Atlantic offers cageless physical collocation to those LECs that request it at just, reasonable, and nondiscriminatory prices, in compliance with checklist item 1. Commenters raised only two issues related to collocation prices, and, as discussed below, we find that these commenters misinterpreted Bell Atlantic's tariffs and their concerns are unfounded. Bell Atlantic asserts that its collocation prices are consistent with the Act and Commission rules. The New York Commission concludes that Bell Atlantic currently provides collocation under approved interconnection agreements and tariffs, consistent with FCC and New York Commission orders. We agree with the New York Commission that the issues raised by commenters with respect to checklist item 1 "do not preclude a finding that Bell Atlantic-NY is in compliance with this checklist item." The Department of Justice did not comment on Bell Atlantic's collocation prices. 79. We disagree with TRA's assertion that Bell Atlantic's collocation prices are discriminatory because they burden competing carriers with "unnecessary security measures and costs." These rates are not discriminatory because Bell Atlantic does not impose the costs of security measures. In Phase Three of its network elements rate case, the New York Commission held that Bell Atlantic may not recover any costs for cageless collocation security measures. Rather, it held that Bell Atlantic must bear such costs itself. Bell Atlantic later filed cageless security rates with the New York Commission, but these rates have not yet been approved and are not in effect. Despite the fact that competitors complained about the lack of set rates for cageless collocation security measures, the New York Commission did not impose temporary rates for cageless collocation security measures, holding instead that this cost and Bell Atlantic's associated cost justification will be considered in Phase Four of the New York Commission's unbundled network elements rate case. In its reply comments to Bell Atlantic's 271 application proceeding, the New York Commission noted that Bell Atlantic had a "placeholder" in its cageless collocation tariff for its security rate but that no rates are being imposed. We therefore find that TRA has misinterpreted Bell Atlantic's tariff and that its claim that Bell Atlantic's security rates are discriminatory is unfounded. 80. We also disagree with ALTS' claim that Bell Atlantic does not meet the Commission's requirements that it allocate its space preparation and related up-front costs among competing carriers on a pro-rata basis. In order to fulfill its obligation to provide nondiscriminatory access to interconnection, an incumbent LEC must "allocate space preparation, security measures, and other collocation charges on a pro-rated basis so the first collocator in a particular incumbent premises will not be responsible for the entire cost of site preparation." The New York Commission reviewed Bell Atlantic's interconnection tariff and rejected Bell Atlantic's initial proposal that it be allowed to charge the initial collocator the entire cost of space preparation. The New York Commission held that "it seems unreasonable to require the initial collocator to bear, up-front, the entire cost of protecting [Bell Atlantic] against the possibility that its costs may go unrecovered." The New York Commission further held that no reason existed to single out these costs for up-front recovery. The New York Commission instead estimated room construction costs and other up-front payments on a TELRIC basis and provided for their recovery through recurring charges. The New York Commission calculated on the basis of reasonable estimates of the likely number of users, thereby "obviating any possibility that the full cost would be imposed on the first [competing carrier]." Bell Atlantic has complied with this requirement in its tariff. Based on the record presented to us, we find that the New York Commission has set prices for a competing carriers' up-front site preparation costs at TELRIC- based costs, and ensured that the initial competitor to collocate will not bear the complete up- front collocation costs. Therefore, we conclude that this claim is without merit. B. Checklist Item 2 - Unbundled Network Elements 81. The nondiscriminatory provision of operations support systems (OSS) and the ability of competing carriers to combine unbundled network elements are integral aspects of the BOC's obligation to provide access to unbundled network elements as required by checklist item 2. In this section, we first outline section 271's nondiscrimination standard and our general approach to analyzing the adequacy of Bell Atlantic's OSS. We then briefly describe the critically important independent third-party testing conducted by KPMG and Hewlett Packard under the supervision of the New York Commission. Next, we describe briefly the systems, databases, and personnel on which Bell Atlantic relies in support of its claim that it provides access to OSS on a nondiscriminatory basis. We then address Bell Atlantic's change management process and the technical assistance that Bell Atlantic offers to competing carriers seeking to use its OSS. We also analyze Bell Atlantic's provision of access to the critical OSS functions of pre-ordering, ordering, provisioning, maintenance and repair, and billing. Finally, we analyze in this section whether Bell Atlantic provides access to unbundled network elements in a manner that allows competing carriers to combine such elements. 1. Operations Support Systems 82. As discussed below, we conclude that Bell Atlantic demonstrates that it provides requesting carriers nondiscriminatory access to OSS functions. Specifically, we find that Bell Atlantic provides a change management process and technical assistance that affords competing carriers a meaningful opportunity to compete. We also find that Bell Atlantic offers nondiscriminatory access to its pre-ordering, ordering, provisioning, maintenance and repair, and billing OSS functions. In reaching these conclusions, we acknowledge that we differ from the evaluation of the Department of Justice in certain material respects. Although we have accorded substantial weight to the Department's views as required by section 271, the statute prohibits us from giving the Department's views preclusive weight. With respect to access to OSS functions, we differ from the Department primarily in instances where we assess the totality of the evidence differently or where we have a greater amount of information available to inform our conclusions. a. Background 83. Incumbent LECs use a variety of systems, databases, and personnel (collectively referred to as OSS) to provide service to their customers. The Commission consistently has found that nondiscriminatory access to OSS is a prerequisite to the development of meaningful local competition. For example, new entrants must have access to the functions performed by the incumbent's OSS in order to formulate and place orders for network elements or resale services, to install service to their customers, to maintain and repair network facilities, and to bill customers. The Commission has determined that without nondiscriminatory access to the BOC's OSS, a competing carrier "will be severely disadvantaged, if not precluded altogether, from fairly competing" in the local exchange market. 84. Section 271 requires the Commission to determine whether a BOC offers nondiscriminatory access to OSS functions. Section 271(c)(2)(B)(ii) requires a BOC to provide "nondiscriminatory access to network elements in accordance with the requirements of sections 251(c)(3) and 252(d)(1)." The Commission has determined that access to OSS functions falls squarely within an incumbent LEC's duty under section 251(c)(3) to provide unbundled network elements under terms and conditions that are nondiscriminatory and just and reasonable, and its duty under section 251(c)(4) to offer resale services without imposing any limitations or conditions that are discriminatory or unreasonable. The Commission must therefore examine a BOC's OSS performance to evaluate compliance with section 271(c)(2)(B)(ii) and (xiv). In addition, the Commission has also concluded that the duty to provide nondiscriminatory access to OSS functions is embodied in other terms of the competitive checklist as well. Consistent with prior orders, we examine Bell Atlantic's OSS performance directly under checklist items 2 and 14, as well as other checklist terms. 85. As part of its statutory obligation to provide nondiscriminatory access to OSS functions, a BOC must provide access that sufficiently supports each of the three modes of competitive entry envisioned by the 1996 Act - competitor-owned facilities, unbundled network elements, and resale. For OSS functions that are analogous to those that a BOC provides to itself, its customers or its affiliates, the nondiscrimination standard requires the BOC to offer requesting carriers access that is equivalent in terms of quality, accuracy, and timeliness. The BOC must provide access that permits competing carriers to perform these functions in "substantially the same time and manner" as the BOC. The Commission has recognized in prior orders that there may be situations in which a BOC contends that, although equivalent access has not been achieved for an analogous function, the access that it provides is nonetheless nondiscriminatory within the meaning of the statute. 86. For OSS functions that have no retail analogue, the BOC must offer access "sufficient to allow an efficient competitor a meaningful opportunity to compete." In assessing whether the quality of access affords an efficient competitor a meaningful opportunity to compete, we will examine, in the first instance, whether specific performance standards exist for those functions. In particular, we will consider whether appropriate standards for measuring OSS performance have been adopted by the relevant state commission or agreed upon by the BOC in an interconnection agreement or during the implementation of such an agreement. If such performance standards exist, we will evaluate whether the BOC's performance is sufficient to allow an efficient competitor a meaningful opportunity to compete. 87. We analyze whether Bell Atlantic has met the nondiscrimination standard for each OSS function using the two-step approach outlined in prior orders. First, we determine "whether the BOC has deployed the necessary systems and personnel to provide sufficient access to each of the necessary OSS functions and whether the BOC is adequately assisting competing carriers to understand how to implement and use all of the OSS functions available to them." We next assess "whether the OSS functions that the BOC has deployed are operationally ready, as a practical matter." 88. Under the first inquiry, a BOC must demonstrate that it has developed sufficient electronic (for functions that the BOC accesses electronically) and manual interfaces to allow competing carriers equivalent access to all of the necessary OSS functions. For example, a BOC must provide competing carriers with the specifications necessary for carriers to design or modify their systems in a manner that will enable them to communicate with the BOC's systems and any relevant interfaces. In addition, a BOC must disclose to competing carriers any internal business rules and other formatting information necessary to ensure that a carrier's requests and orders are processed efficiently. Finally, a BOC must demonstrate that its OSS is designed to accommodate both current demand and projected demand for competing carriers' access to OSS functions. Although not a prerequisite, the Commission continues to encourage the use of industry standards as an appropriate means of meeting the needs of a competitive local exchange market. 89. Under the second inquiry, we examine performance measurements and other evidence of commercial readiness to ascertain whether the BOC's OSS is handling current demand and will be able to handle reasonably foreseeable demand volumes. The most probative evidence that OSS functions are operationally ready is actual commercial usage. Absent data on commercial usage, the Commission will consider the results of carrier-to-carrier testing, independent third-party testing, and internal testing in assessing the commercial readiness of a BOC's OSS. We reiterate, however, that the persuasiveness of a third-party review is dependent upon the qualifications, experience and independence of the third party and the conditions and scope of the review itself. b. Overview of OSS Operations 90. Bell Atlantic utilizes a number of systems and processes to support the entry of competing carriers into the local services market in New York. As an initial matter, a new entrant seeking to compete in the New York local services market must establish some form of connectivity with Bell Atlantic to submit service requests and receive responses. Bell Atlantic provides requesting carriers an application-to-application interface based on the Electronic Data Interchange (EDI) protocol for pre-ordering and ordering functions, as well as a Web-based Graphical User Interface (Web GUI or GUI) for pre-ordering, ordering and maintenance and repair functions. In addition, Bell Atlantic provides requesting carriers with training and reference guides for the use of each interface. A new entrant seeking to use the EDI interface must undergo a certification test with Bell Atlantic to verify that the carrier's operations support systems are capable of submitting valid service orders and receiving responses. 91. Before placing an actual order for service, a competing carrier can obtain pre- ordering information by sending a request over the Web GUI or EDI pre-ordering interface. Such pre-ordering information, which is often accessed while the customer is on the line, typically includes a customer's address and service history and the services and features available to that customer, as well as telephone numbers and delivery dates available from Bell Atlantic. Bell Atlantic returns the requested information over the same interface used by the carrier to submit the inquiry. The EDI interface enables competing carriers to populate an order form with information received from pre-ordering inquiries. 92. Using the information obtained in the pre-ordering process, the competing carrier submits an order for service using the EDI or Web GUI interface. An order sent by a competing carrier enters the Direct Customer Access System (DCAS) gateway system, which performs an initial check of the validity of the order. If the order is missing information or is determined not to be a valid transaction, Bell Atlantic will stop processing the order and send a Local Service Request Rejection (order rejection) notice to the carrier. An order that is not rejected will either flow automatically from DCAS to the Direct Order Entry (DOE) system or drop out for manual processing at a Telecom Industry Services Ordering Center (TISOC). At the TISOC, a Bell Atlantic representative will input the order into the Service Order Processor (SOP) directly. If the order flowed through to DOE, the order will pass through another series of checks and edits before it is passed to SOP for processing in the appropriate back end system. If the order does not pass the DOE screening, it is manually input into SOP by a Bell Atlantic representative. Once an order reaches SOP, it is mixed in and processed along with Bell Atlantic retail orders, and Bell Atlantic returns a Local Services Request Confirmation (order confirmation) to the carrier. The order confirmation provides, at minimum, the scheduled due date, service order identification, and account telephone number. At times, a carrier may need to "supplement" the order to reflect a subsequent change or to respond to an error message. 93. After an order is successfully entered into SOP, Bell Atlantic begins the process of provisioning the order, or activating the requested service or feature, which may involve assigning facilities, updating translations in a switch, and dispatching technicians. Specifically, an order flows from SOP to the Service Order Analysis and Control (SOAC) system. SOAC controls the progress of service orders through the provisioning process by distributing the service order to other necessary provisioning systems and then updating SOP. From SOAC, most orders flow automatically through the assignment systems, including the Loop Facility Assignment and Control System (LFACS), where the appropriate facilities are assigned or reserved for the order. After assignment, the next stage in the provisioning process for most orders is the loading of the translations into the switch, which is performed by the Recent Change Memory Administration Center (RCMAC). In addition, technicians at the central office perform any wiring work associated with the order. Orders that require work performed outside the central office are sent to the Work Force Administration (WFA) system for dispatch of a field technician. The Regional CLEC Coordination Center (RCCC) facilitates and coordinates the provisioning of wholesale orders. Competing carriers can monitor the provisioning process by viewing Bell Atlantic's regular posting of orders that are in jeopardy of missing an installation due date and by querying the order's status in SOP. Upon completion of the work involved in activating service, Bell Atlantic sends a notice of "work completion" to the carrier. In addition, after the order moves from SOP into Bell Atlantic's billing systems and is recorded as complete in the billing systems, Bell Atlantic sends a notice of "billing completion" to the carrier. 94. If a competing carrier's customer experiences service disruptions, the carrier can create and monitor trouble tickets, access trouble history for that line, and request a test of the customer's circuit by submitting inquiries over the Web GUI. A carrier's maintenance and repair inquiry is sent to the Repair Trouble Administration System (RETAS) gateway system, which routes requests to the appropriate back end systems and returns electronic responses. Most trouble reports are processed through the Loop Maintenance Operating System, handling overall maintenance, tracking and dispatch activities, and the StarMem system, which allows automatic feature updates to switches. To test for and analyze faults on a circuit, Bell Atlantic uses the Mechanized Loop Testing (MLT), Switched Access Remote Testing System (SARTS), and Delphi systems. Bell Atlantic's Regional CLEC Maintenance Center (RCMC) supports wholesale trouble reporting and repair issues. Bell Atlantic returns responses to trouble ticket inquiries over the same interface used by the carrier to submit the inquiry. 95. In order for competing carriers to bill their customers, Bell Atlantic provides carriers with usage billing information and a process for adjusting or correcting invalid or incorrect data. Bell Atlantic also provides requesting carriers documentation on its billing procedures, bill content and related interactions. Specifically, Bell Atlantic delivers a record of daily usage to competing carriers. Bell Atlantic also produces periodic bills (up to ten monthly) for wholesale carriers using the Customer Record Information System (CRIS), which provides billing for resale and unbundled loops, and the Customer Access Billing System (CABS), which provides billing for access services and other unbundled network elements. Competing carriers receive aggregated bills for the charges incurred by all their customers in a particular area, as well as charges for products and services ordered by the carrier itself. If a competing carrier believes that an individual usage item contains errors, it initiates a billing usage claim, and may be required to transmit the erroneous usage back to Bell Atlantic. Incorrect usage data may be either reprocessed or corrected with a billing adjustment. The competing carrier is responsible for billing the end user. c. Independent Third-Party Testing 96. The New York Commission retained KPMG to conduct an independent, third- party test of the readiness of Bell Atlantic's OSS, interfaces, documentation and processes. Over the course of fifteen months, KPMG evaluated 855 separate items relating to pre-ordering, ordering, provisioning, maintenance and repair, billing, and relationship management and infrastructure, by performing both transaction and operational tests. KPMG combined efforts with Hewlett Packard to accomplish the transaction-driven tests. In doing so, KPMG acted much like a "pseudo-competing carrier" operations department, working with Bell Atlantic business rules, creating and tracking orders, monitoring Bell Atlantic performance, logging trouble tickets, and evaluating carrier-to-carrier bills. At the same time, Hewlett Packard acted as a competing carrier information technology department, establishing electronic bonding with Bell Atlantic, translating back and forth between business and EDI rule formats, and resolving problems with missing orders and responses. By building and submitting transactions using Bell Atlantic's electronic interfaces with test accounts in central offices spread across New York, KPMG was able to live the experience of a competing carrier. In addition, KPMG used operational tests to evaluate the results of Bell Atlantic day-to-day operational management and change management processes to determine if they functioned in accordance with Bell Atlantic documentation and expectations. 97. KPMG's test was broad in scope. All stages of the relationship between Bell Atlantic and competing carriers were considered, from establishing the initial relationship, to performing daily operations, to maintaining the relationship. Resale, UNE-loops, UNE- platform, and combinations were all included in the test. In addition, both the application-to- application electronic data interchange (EDI) and the terminal-type web-based graphical user interface (GUI) were tested. KPMG performed pre-ordering, ordering, provisioning, maintenance and repair, billing, and relationship management and infrastructure tests to evaluate functional capabilities and determine whether competing carriers receive a level of service comparable to Bell Atlantic retail service. To fully test these systems, orders were submitted with known error conditions, canceled, and supplemented. Documentation was evaluated for usefulness, correctness, and completeness. KPMG also performed stress volume tests of Bell Atlantic systems and identified specific bottlenecks for wholesale customers. 98. In performing these tests, KPMG adopted a military-style test philosophy, or a mindset of "test until you pass." Thus, when situations arose where testing revealed that a Bell Atlantic process, document, or system did not meet expectations, Bell Atlantic would generally implement a fix and KPMG would retest the process, document, or system until satisfied. As a result, KPMG believes that competing carriers now have a "baseline set of working components" that a one-time diagnostic evaluation of Bell Atlantic's OSS would not have provided. 99. To the greatest extent possible, the KPMG test was both independent and blind. Neither KPMG nor Hewlett Packard had a reporting relationship to Bell Atlantic. Although it was virtually impossible for the KPMG transactions to be truly blind, KPMG instituted certain procedures to ensure that both KPMG and Hewlett Packard would not receive preferential treatment. For example, KPMG required that all documents provided to them were generally available to all competing carriers. The New York Commission monitored phone calls between KPMG and Hewlett Packard and Bell Atlantic, and competing carriers were invited to attend conference calls. In addition, KPMG made concurrent observations of the service quality delivered to other competing carriers during the course of its test. 100. The scope and depth of KPMG's review, and the conditions surrounding it, including KPMG's independence, military-style test philosophy, efforts to place themselves in the position of an actual market entrant, and efforts to maintain blindness when possible, lead us to treat the conclusions in the KPMG Final Report as persuasive evidence of Bell Atlantic's OSS readiness. As we have said before, the persuasiveness of a third-party review is dependent on the conditions and scope of the review. Because we recognize that various third-party tests may be adequate to demonstrate the operational readiness of a BOC's OSS, we emphasize that we do not foreclose the possibility that a third-party test designed differently than the KPMG review may also be persuasive. Nonetheless, were a third-party test less comprehensive, less independent, less blind, and, therefore, less useful in assessing the real world impact of a BOC's OSS on competing carriers, we would not necessarily find it persuasive and may accord it less weight than we do the KPMG Final Report. d. Change Management and Technical Assistance (i) Change Management 101. We conclude that Bell Atlantic demonstrates that it provides the documentation and support necessary to give competing carriers nondiscriminatory access to its OSS. Bell Atlantic makes this demonstration by showing that it has an adequate change management process in place in New York. The record also reflects that Bell Atlantic has adhered to its change management process over time. As a result, we find that Bell Atlantic provides access to its OSS in a manner that allows an efficient competitor a meaningful opportunity to compete. (a) Background 102. Competing carriers need information about and specifications for an incumbent's systems and interfaces in order to develop and modify their systems and procedures to access the incumbent's OSS functions. Thus, in the Ameritech Michigan Order, the Commission determined that in order to provide nondiscriminatory access to OSS, a BOC must first demonstrate that it "has deployed the necessary systems and personnel to provide sufficient access to each of the necessary OSS functions and . . . is adequately assisting competing carriers to understand how to implement and use all of the OSS functions available to them." By showing that it adequately assists competing carriers to use available OSS functions, a BOC provides evidence that it offers an efficient competitor a meaningful opportunity to compete. As part of this demonstration, the Commission will give substantial consideration to the existence of an adequate change management process and evidence that the BOC has adhered to this process over time. 103. The change management process refers to the methods and procedures that the BOC employs to communicate with competing carriers regarding the performance of and changes in the BOC's OSS system. Such changes may include operations updates to existing functions that impact competing carrier interface(s) upon a BOC's release of new interface software; technology changes that require competing carriers to meet new technical requirements upon a BOC's software release date; additional functionality changes that may be used at the competing carrier's option, on or after a BOC's release date for new interface software; and changes that may be mandated by regulatory authorities. Without a change management process in place, a BOC can impose substantial costs on competing carriers simply by making changes to its systems and interfaces without providing adequate testing opportunities and accurate and timely notice and documentation of the changes. As Allegiance suggests, change management problems can impair a competing carrier's ability to obtain nondiscriminatory access to UNEs, and hence a BOC's compliance with section 271(c)(2)(B)(ii). 104. Competing carriers have had a substantial role in the development of Bell Atlantic's change management process in New York. As part of a collaborative process dating back to October 1997 and conducted under the auspices of the New York Commission, Bell Atlantic and competing carriers developed a detailed process of managing changes to the Bell Atlantic systems and interfaces that affect competing carriers. This process resulted in the May 1998 document entitled "Telecom Industry Services-Change Management Process" (Change Agreement). Although there have been subsequent modifications to the Change Agreement, the basic process and timelines set out in this document are still applicable. 105. The Change Agreement sets forth detailed procedures for introducing changes in Bell Atlantic's systems and documentation. It divides all changes into five different categories and provides specific time lines and intervals for each category. Thus, the process is designed to accommodate emergency changes, regulatory changes, changes in industry standards, changes requested by Bell Atlantic, and changes requested by competing carriers. 106. Regardless of the type of change, the Change Agreement expressly provides for feedback from competing carriers on the proposed changes. In addition, the Change Agreement calls for Bell Atlantic and the competing carriers to develop jointly a schedule for the distribution of draft specifications or business rules, receipt of competing carrier comments on the documentation, and distribution of final documentation. Bell Atlantic has established a forum where representatives from Bell Atlantic and competing carriers meet-often more than once a month-to discuss upcoming system and interface changes as well as the change management procedures themselves. Moreover, in September 1999, representatives of Bell Atlantic and competing carriers began to prioritize changes based on merit, rather than the sponsor of the change. Thus, competing carriers had a substantial role in the development of methods and procedures for the change management process in New York and continue to have opportunities for meaningful input in the change management process today. 107. Bell Atlantic's basic change management process is memorialized and set forth in a single document, the Change Agreement. As a result, Bell Atlantic's change management process documentation is clearly organized and readily accessible to competing carriers. Competing carriers can readily access the Change Agreement on Bell Atlantic's Telecommunications Industry Services (TIS) web page. Modifications to this document are also available on the TIS web page. Moreover, in response to KPMG findings, Bell Atlantic has improved its procedures for competing carriers to cross-reference and track information regarding the change management process. Thus, Bell Atlantic now updates and maintains a database that tracks the progress of each specified change, reports changes systematically using change request numbers and uses these same numbers in communications with competing carriers to identify specific changes. 108. Bell Atlantic's change management process includes a method for dispute resolution that is separate and apart from any process that is set forth in interconnection agreements. As a result, competing carriers now have a forum specifically designed to address any change management disputes. In response to concerns raised by competing carriers, Bell Atlantic, in consultation with competing carriers and the New York Commission staff, established an escalation process for resolving change control disputes. This process allows competing carriers to appeal to upper level management at Bell Atlantic on change management issues and also allows competing carriers to raise these issues before the New York Commission staff. 109. Bell Atlantic's change management process provides for a stable testing environment. Competing carriers need access to a stable testing environment to certify that their OSS will be capable of interacting smoothly and effectively with Bell Atlantic's OSS, as modified. In addition, prior to issuing a new software release or upgrade, the BOC must provide a testing environment that mirrors the production environment in order for competing carriers to test the new release. If competing carriers are not given the opportunity to test new releases in a stable environment prior to implementation, they may be unable to process orders accurately and unable to provision new customer services without delays. KPMG originally found Bell Atlantic's testing environment "Not Satisfied," specifically noting that the testing environment "did not adequately mirror production capabilities." As the New York Commission suggests, this can result in competing carriers' transactions succeeding in the testing environment but failing in production. 110. In response to KPMG's initial finding, Bell Atlantic worked with New York Commission staff and competing carriers to establish a new testing environment and new testing procedures. Some of these changes were introduced in April 1999 as part of an interim Quality Assurance (QA) environment for carrier-to-carrier testing of new versions of OSS interfaces. KPMG reviewed the interim QA testing environment for pre-ordering and ordering and determined that the interim environment mirrored the production environment. At the same time, KPMG determined the availability of the testing environment under Bell Atlantic's interim procedures presented problems for competing carriers. As AT&T and MCI WorldCom note, the interim QA testing environment was only made available to competing carriers during business hours and for a maximum period of five business days. On September 20, 1999 Bell Atlantic introduced its permanent QA testing environment. Bell Atlantic represents that the permanent QA testing environment mirrors production and provides a physically separate environment for competing carrier testing. In addition, Bell Atlantic plans to maintain this testing environment for all but emergency changes for at least a month, including extended daily hours. Moreover, in order to ensure that competing carriers are not forced to test and cut over to a new industry standard release prematurely, Bell Atlantic maintains a pre-existing version after issuing a major new release rather than switching directly from one version to the next. Finally, Bell Atlantic, in response to a separate KPMG "Not Satisfied" finding, has introduced new procedures to certify that a competing carrier may move from the testing environment to the production environment. (b) Discussion 111. Based on the above record evidence, we conclude that Bell Atlantic demonstrates that it has a change management process in place in New York that provides an efficient competitor with a meaningful opportunity to compete. Specifically, we find that Bell Atlantic makes this showing with: (1) evidence of competing carrier input in the design and continued operation of the change management process; (2) the memorialization of the change management process in a basic document; (3) the availability of a separate forum for change management disputes; (4) and the availability of a stable testing environment that mirrors production. We note that even competing carriers have acknowledged in their comments that the processes in the Change Agreement are satisfactory as written. Because we recognize that various change management plans may be adequate to meet the needs of competing carriers, we emphasize that the individual factors described above are indicative, but not dispositive, of an adequate process. Although we will look for evidence of these same factors in evaluating a future applicant's change management process, we do not foreclose the possibility that a different plan may be sufficient. 112. We also find that the record demonstrates that Bell Atlantic has adhered to its change management process over time. Commenters, however, express concern that problems remain with respect to Bell Atlantic's ability to adhere to notification and documentation timelines in its Change Agreement and Bell Atlantic's ability to show that the permanent QA testing environment meets the needs of competing carriers. In addition, commenters allege that Bell Atlantic issues too many emergency changes and fails to consider competing carrier input in the change management process. (i) Notification and Documentation Timeliness 113. We conclude that Bell Atlantic provides competing carriers with change management notification and documentation for upcoming change releases in a manner sufficiently timely to allow an efficient competitor a meaningful opportunity to compete. As TRA suggests, the failure of a BOC to provide timely, complete, and accurate notice of alterations to its systems and processes hinders the ability of competitive providers to serve their customers adequately. Without timely notification and documentation, competing carriers are unable to modify their existing systems and procedures or develop new systems to maintain access to a BOC's OSS functions. As a preliminary matter, we find that the Change Agreement establishes reasonable intervals for the distribution of change management notification and documentation because they provide competing carriers with sufficient time to prepare for Bell Atlantic system changes. In addition, we commend Bell Atlantic and the New York Commission for developing metrics that report its compliance with these intervals. 114. We find that Bell Atlantic provides competing carriers with timely change management notification and documentation for changes made at the request of regulatory authorities (Type 2 changes), industry standard organizations (Type 3 changes), and competing carriers (Type 5 changes) in a manner sufficiently timely to allow an efficient competitor a meaningful opportunity to compete. For these types of changes, the data are extremely limited because they occur infrequently. Nonetheless, the data provided on these changes in both the Carrier-to-Carrier metrics and the KPMG Final Report demonstrate that Bell Atlantic has already established a pattern of compliance with the relevant notification and documentation intervals in its Change Agreement. 115. We also find that Bell Atlantic provides competing carriers with notification and documentation for Bell Atlantic-initiated changes (Type 4 changes) in a manner sufficiently timely to allow an efficient competitor a meaningful opportunity to compete. In its Final Report, KPMG found that Bell Atlantic was unable to meet documentation intervals set in the Change Agreement for Type 4 changes, and characterized this problem as "Not Satisfied." KPMG found that Bell Atlantic provided timely documentation in only three of nineteen instances for Type 4 changes from January to June 1999. During the same period, Bell Atlantic was able to provide timely notification of upcoming Type 4 changes in sixteen of twenty instances. Bell Atlantic contends, however, that it has now addressed the documentation timeliness problem identified by KPMG. With respect to initial notification timeliness, during the period from July to October 1999, the record shows that Bell Atlantic provided timely notification for eleven of twelve Type 4 changes. With respect to final documentation timeliness, during the period from August to October 1999, the record shows that Bell Atlantic provided timely documentation for eight of ten Type 4 changes. Thus, Bell Atlantic has demonstrated considerable improvement since the KPMG review. In particular, Bell Atlantic was able to provide both timely notification and documentation to competing carriers for two of two Type 4 changes that occurred in October 1999. We find that these improvements, coupled with the opportunities competing carriers have to participate in the prioritization of changes and the month long testing opportunities provided for Type 4 changes, indicate that an efficient competitor has a meaningful opportunity to compete. 116. In addition, we conclude that Bell Atlantic provides notification for emergency changes (Type 1 changes) in a manner sufficiently timely to allow an efficient competitor a meaningful opportunity to compete. Under the Change Agreement, timely emergency notification simply requires notification prior to implementation. As the KPMG Final Report suggests, timely emergency notification can range from several hours to several days advance notice. Although we understand advance notification is preferable for competing carriers, we also must acknowledge that given the nature of emergency changes, it will not always be possible for Bell Atlantic to notify competing carriers prior to implementation. Some commenters question Bell Atlantic's ability to provide competing carriers with timely notification of Type 1 emergency changes. MCI WorldCom, for instance, complains that the timeliness of Bell Atlantic's emergency notification fell considerably in September 1999, when Bell Atlantic was timely for only seven of twelve Type 1 changes. We note, however, that Bell Atlantic's Type 1 change notification was timely for twenty-five of twenty-six changes in July 1999 and six of six changes that occurred between October 1 and October 19, 1999. Because we believe that as a matter of course emergency changes will occur in situations where Bell Atlantic may be unable to notify competing carriers prior to implementation, we do not find that Bell Atlantic's September 1999 performance prevents us from concluding that Bell Atlantic provides emergency change notification to competing carriers in a manner sufficiently timely to allow an efficient competitor to compete. 117. Our conclusion that Bell Atlantic provides timely change management notification and documentation to competing carriers seeking to use its OSS differs from that reached by the Department of Justice. We reach this conclusion, however, by separately assessing the underlying issues associated with each of the Bell Atlantic change types identified in the Change Agreement. First, with respect to the limited number of changes made at the request of regulatory authorities, industry standard organizations, and competing carriers themselves, Bell Atlantic has established a pattern of general compliance with the notification and documentation intervals in its Change Agreement. Second, we find the recent improvement in Bell Atlantic's timely distribution of Type 4 notification and documentation demonstrates its ability to adhere to its change management process. Finally, while we acknowledge notification prior to implementation of an emergency change will not always be possible, we still find that Bell Atlantic provides sufficiently timely notification to competing carriers. 118. Although we reach the same conclusion as the New York Commission with respect to Bell Atlantic's change management notification and documentation timeliness, we do not rely on Bell Atlantic's willingness to have its future change management notification and documentation timeliness enforced through the Change Control Assurance Plan. In addition, we acknowledge that the timeliness of Bell Atlantic's performance falls short of the monthly standards for change management notification and documentation set out in the Carrier-to-Carrier metrics and used in the Change Control Assurance Plan. Nonetheless, when we view Bell Atlantic's overall performance over the course of recent months, we find that Bell Atlantic's notification and documentation timeliness is sufficient to allow an efficient competitor a meaningful opportunity to compete. We will, however, be prepared to take appropriate enforcement action if there is evidence of deteriorating performance in the future. Finally, although our conclusion is based on the specific categories of changes identified in the Bell Atlantic Change Agreement in place in New York, we do not foreclose the possibility that a different plan with a less disaggregated structure and different intervals for notification and documentation may also be sufficient. (ii) Testing Environment 119. We conclude that Bell Atlantic's permanent QA testing environment provides competing carriers with a stable environment and an adequate opportunity to test Bell Atlantic OSS changes prior to implementation. Specifically, we find the record demonstrates that Bell Atlantic's new testing environment adequately mirrors the production environment and offers the extended testing periods that competing carriers need for new entrant certification and new release testing. MCI WorldCom and AT&T note that as of the date of Bell Atlantic's application, no competing carriers had been given the opportunity to use the permanent QA testing environment and determine that it works in the manner Bell Atlantic represents in its application. We conclude there is sufficient evidence to demonstrate that Bell Atlantic's permanent QA testing environment provides a stable testing environment for competing carriers. 120. We base this conclusion on the experience of the competing carriers that used the permanent QA testing environment without difficulty for an October 16, 1999 software release. Thus, we find that the recent evidence from commercial usage suggests that Bell Atlantic's permanent QA environment works in the manner represented in its application. As the New York Commission attests, with only one minor exception, the results of the production run matched the results of the run in the permanent QA testing environment. The one exception, the absence of a billing telephone number for a directory listing, has been corrected. 121. Our conclusion is buttressed by the similarity between the interim and permanent QA testing environment and KPMG's finding that the interim testing environment adequately mirrored the production environment. Both environments mirror production and offer test decks of representative pre-ordering and ordering transactions. The basic processes for new release and new entrant testing distributed in April 1999 apply to both the interim and permanent environments. The only differences between the two environments are that the permanent QA testing environment is physically separate and expands the test period to one month, thus remedying the major problems identified by KPMG and competing carriers with the interim QA testing environment. 122. We find that the record demonstrates that Bell Atlantic's permanent QA testing environment provides competing carriers with a stable environment and adequate opportunity to test Bell Atlantic OSS changes prior to implementation. Although we reach the same conclusion as the New York Commission, we differ somewhat from that reached by the Department of Justice. The Department of Justice found that while it was hopeful that the permanent QA testing environment would meet competing carrier needs, the results of recent Bell Atlantic improvements did not appear in the record before them. Comments filed subsequent to the evaluation of the Department of Justice, however, demonstrate that the October 16, 1999 software release using the new QA testing environment was successful. As a result, we find that the record now demonstrates that Bell Atlantic provides a testing environment for OSS changes sufficient to enable an efficient competitor a meaningful opportunity to compete. (iii) Other Issues 123. AT&T and Sprint assert that Bell Atlantic improperly categorizes a substantial number of changes as Type 1 emergency changes in order to evade the longer notification requirements associated with other types of changes under the Change Agreement. We conclude these claims do not warrant a finding that Bell Atlantic fails to adhere to its change management procedures in a manner sufficient to provide an efficient competitor with a meaningful opportunity to compete. Type 1 emergency changes are specifically defined and provided for in the Change Agreement that was developed in a collaborative proceeding involving Bell Atlantic, competing carriers, and the New York Commission. Furthermore, as AT&T itself acknowledges, on June 30, 1999, Bell Atlantic and competing carriers began a series of workshops that resulted in a more narrow definition of Type 1 changes. This provides evidence of competing carriers' continuing opportunity to provide meaningful input in the change management process in New York. Since these workshops began, Bell Atlantic has reduced the number of Type 1 changes from twenty-six in July 1999 to ten in August, twelve in September and six in the first half of October. Because emergency changes are specifically provided for in the Change Agreement and Bell Atlantic's use of them has decreased in recent months, we find AT&T and Sprint's claims unpersuasive. 124. AT&T and MCI WorldCom allege that Bell Atlantic fails to give competing carriers opportunities to provide input on new releases as it is obligated to do under the Change Agreement. We find that the record simply does not support this claim. For instance, representatives of competing carriers and Bell Atlantic jointly prioritize upcoming changes. In addition, Bell Atlantic and competing carriers meet regularly to discuss upcoming changes and the change management process itself. As part of these meetings, Bell Atlantic and the competing carriers develop a detailed chart of competing carrier requests for action on specific change management issues, track the status of these problems, and note Bell Atlantic actions taken to address the problem. For example, when MCI WorldCom expressed a preference regarding how customer service record addresses be made available to competing carriers, Bell Atlantic agreed to add this functionality within the remaining weeks before the related change release. At the same time, Bell Atlantic devised a special software approach to defer implementation of this functionality for AT&T, the sole competing carrier that objected to this change. Although we would be concerned about the impact of a BOC disregarding input from competing carriers on change management issues, we do not believe the record indicates that this is a problem for carriers working with Bell Atlantic in New York. 125. We also conclude that problems with specific OSS changes described by MCI WorldCom, Allegiance, and Sprint do not warrant a conclusion that Bell Atlantic fails to adequately assist competing carriers seeking to use its OSS. Because Bell Atlantic must accommodate a variety of interests with any given change release, we reasonably expect some competing carriers to be less than satisfied with any given change. We do not, however, find that these complaints evidence a systemic problem. (ii) Technical Assistance and Help Desk Support 126. In the Ameritech Michigan Order, the Commission determined that in order to provide nondiscriminatory access to OSS, a BOC must first demonstrate that it "has deployed the necessary systems and personnel to provide sufficient access to each of the necessary OSS functions and . . . is adequately assisting competing carriers to understand how to implement and use all of the OSS functions available to them." By showing that it adequately assists competing carriers to use available OSS functions, a BOC provides evidence that it offers an efficient competitor a meaningful opportunity to compete. As part of this demonstration, the Commission will give substantial consideration to evidence showing that the BOC provides adequate technical assistance and help desk support to competing carriers seeking to use its OSS. 127. We conclude that Bell Atlantic demonstrates that it provides the technical assistance and help desk support necessary to give competing carriers nondiscriminatory access to its OSS. Bell Atlantic has produced a separate three volume handbook for resellers and purchasers of UNEs, both available on CD-ROM with word search capability. Documentation is updated for each release and also is made available on Bell Atlantic's web site. Thus, competing carriers have access to complete, up-to-date business rules and ordering codes. Bell Atlantic also conducts regular training courses for competing carriers in key areas. In addition, Bell Atlantic's "Systems Support Help Desk" provides a single point of contact for competing carrier reports of system outages and software defects and provides help to ensure that any problems are resolved as quickly as possible. We are further encouraged by Bell Atlantic's practice of evaluating the performance of its help desk call agents and, when necessary, replacing the tools available to them for analyzing information and resolving problems. Although KPMG reported confusion regarding contact lists and help desk numbers, we find that Bell Atlantic has since fixed this problem. Specifically, we note that in September 1999, Bell Atlantic posted on its web site a comprehensive and descriptive list of the different support features available to competing carriers, including the time of day these support functions are available. Accordingly, we find that Bell Atlantic provides efficient competitors a meaningful opportunity to compete by enabling them to understand how to implement and use all of the OSS functions available to them. Thus, we reject commenters' allegations that Bell Atlantic's technical assistance and help desk support is inadequate. e. Pre-Ordering 128. Based on the evidence in the record, we conclude that Bell Atlantic demonstrates that it provides nondiscriminatory access to OSS pre-ordering functions. Bell Atlantic offers requesting carriers an industry standard application-to-application pre-ordering interface that enables carriers to integrate pre-ordering and ordering functions. Through this and other pre- ordering interfaces, Bell Atlantic makes available to requesting carriers all the functionality that it provides to itself. Bell Atlantic also shows, through response times and interface availability performance data and third-party testing, that its pre-ordering interfaces and systems are operationally ready and capable of sustaining reasonably foreseeable demand volumes. (i) Background 129. The pre-ordering phase of OSS generally includes those activities that a carrier undertakes to gather and verify the information necessary to place an order. Given that pre- ordering represents the first exposure that a prospective customer has to a competing carrier, it is critical that inferior access to the incumbent's OSS does not render the carrier a less efficient or responsive service provider than the incumbent. Because most pre-ordering functions that support resale services, as well as many of the functions that support service through unbundled network elements, are analogous to the pre-ordering of a BOC's retail services, Bell Atlantic must demonstrate that it provides requesting carriers access that enables them to perform these functions in substantially the same time and manner as Bell Atlantic's retail operations. For those pre-ordering functions that lack a retail analogue, Bell Atlantic must provide access that affords an efficient competitor a meaningful opportunity to compete. (ii) Discussion 130. Application-to-Application Functionality. We find that Bell Atlantic offers requesting carriers access to an application-to-application interface for all pre-ordering functionality that Bell Atlantic provides to itself. In prior orders, the Commission has emphasized that providing pre-ordering functionality through an application-to-application interface is essential in enabling carriers to conduct real-time processing and to integrate pre-ordering and ordering functions in the same manner as the BOC. Bell Atlantic demonstrates through actual commercial usage and the results of third-party testing that it makes application-to-application functionality available for the pre-ordering functions that it provides to itself. 131. Bell Atlantic offers competing carriers pre-ordering OSS functionality through two electronic interfaces: a proprietary Web-based Graphical User Interface (Web GUI); and an application-to-application interface based on the industry standard EDI Issue 9 protocol. Bell Atlantic implemented EDI-9 in July 1998, along with the associated industry standard transaction formats. Requesting carriers have several options for connecting with the EDI interface, and Bell Atlantic documentation provides the specifications for and benefits of each option. Competing carriers therefore have access to complete, up-to-date business rules for pre-ordering functionality. As of the application filing date, approximately 100 carriers were using the Web GUI for pre-ordering, and three carriers were using the EDI interface. Furthermore, Bell Atlantic recently made available a second application-to-application pre-ordering interface, Common Object Request Broker Architecture (CORBA), which it was testing with one carrier when it filed its application. 132. Bell Atlantic represents that these interfaces allow competing carriers "to obtain the same information from the same underlying OSS as Bell Atlantic's own retail service representatives." Specifically, carriers are able to perform the following pre-ordering functions: (1) retrieve CSRs; (2) validate addresses; (3) select and reserve telephone numbers; (4) determine services and features available to a customer; (5) obtain due date availability; (6) access loop qualification information; and (7) view a customer's directory listing. Competing carriers also can check the status of pending orders. 133. With respect to actual commercial usage, Bell Atlantic demonstrates that competing carriers successfully have built and are commercially using application-to-application interfaces (EDI-9 and CORBA) to retrieve CSR information and validate addresses, two of the seven pre-ordering functions. MCI WorldCom, for example, implemented EDI access for parsed CSR retrieval on September 3, 1999, followed by address validation for migrating customers on November 1, 1999. Similarly, AT&T acknowledges that it has commercially deployed CORBA for the same two pre-ordering functions. In addition, CTC Communications, a reseller, successfully implemented EDI for parsed CSR retrieval in June 1999. 134. Along with commercial usage, we also base our conclusion on the demonstrated ability of the third-party testers to construct and extensively test the EDI interface for all pre- ordering functions. As part of the third-party testing, Hewlett Packard used documentation provided by Bell Atlantic to build an EDI interface capable of performing each pre-ordering function, including parsed CSR retrieval. KPMG then conducted a functional evaluation and volume and stress tests of the EDI interface, which verified Bell Atlantic's ability to provide the requisite pre-ordering functionality. Although MCI WorldCom alleges that KPMG's testing interface was not as robust as one required in an actual production environment, we find that KPMG's testing interface was able to handle numerous pre-order transactions and extensive scenarios, using common security and transport (i.e., File Transfer Protocol with Public Key Encryption). We therefore accord substantial weight to the demonstrated ability of the third- party testers in this case to build an application-to-application interface for all pre-ordering functions. 135. In this regard, we are not persuaded by commenters' claims that we should discount the ability of third-party testers to construct an EDI interface for all pre-ordering functions because the testers received favorable treatment from Bell Atlantic. The testing interface was constructed using publicly available Bell Atlantic documentation. Although KPMG acknowledges that at times it received better treatment from Bell Atlantic than that of an ordinary carrier, there is no evidence to suggest that such treatment skewed the test results. Indeed, the record shows that the New York Commission closely supervised the design and operation of the test. KPMG also specifically reviewed pre-order functionality experienced by actual carriers during its Live CLEC Functional Evaluation "in an effort to assess potential bias in the transaction tests." We find no evidence that the Live CLEC Functional Evaluation revealed that Bell Atlantic provided inferior documentation or technical support to competing carriers. 136. We further find that the fact that no carrier has chosen to access all seven pre- ordering functions using an application-to-application interface does not disprove Bell Atlantic's showing that it makes such functionality available. As we have previously stated, Bell Atlantic is not required to actually furnish a particular item to satisfy its obligations under the checklist; rather, it must show that it has a concrete and specific legal obligation to furnish the item upon request and is "presently ready" to furnish the item. The record in this case shows that factors internal to carriers have affected their decision not to develop and commercially deploy an application-to-application interface for all pre-ordering functions. For instance, carriers acknowledge that they place a higher priority on accessing certain functions (i.e., CSR retrieval and address validation) through an application-to-application interface than other functions that are not as critical to the carrier's business plan. Indeed, AT&T acknowledges that, with access to CSR retrieval and address validation, it can "ramp up commercial volumes using CORBA's present capabilities." It would therefore be inappropriate to penalize Bell Atlantic simply because carriers are not actively seeking to implement the remaining application-to-application functions at this time. In any event, we expect that the experience carriers gained in implementing parsed CSR retrieval and address validation will facilitate their efforts to deploy the remaining application-to-application functions. 137. Integration. We find that Bell Atlantic demonstrates that its application-to- application interfaces allow competing carriers to integrate pre-ordering information into Bell Atlantic's ordering interface and the carriers' back office systems, a finding that is fundamental to a BOC's showing of nondiscriminatory access to OSS. The Commission has explained previously that a BOC with integrated pre-ordering and ordering functions must provide competing carriers with access to the same capability. In this regard, the BOC must enable competing carriers to transfer pre-ordering information electronically to the BOC's ordering interface or to the carriers' own back office systems, which may require "parsing" pre-ordering information into identifiable fields. Without an integrated system, a competing carrier would be forced to re-enter pre-ordering information manually into an ordering interface, which leads to additional costs and delays, as well as a greater risk of error. This lack of integration would place competitors at a competitive disadvantage and significantly impact a carrier's ability to serve its customers in a timely and efficient manner. 138. Our finding that Bell Atlantic's pre-ordering and ordering interfaces are readily integratable is based on evidence of successful commercial integration and KPMG's findings. In terms of commercial usage, Bell Atlantic demonstrates that CTC Communications was able to develop an integrated EDI pre-ordering and ordering system for parsed CSR information. Similarly, we find that MCI WorldCom and AT&T have integrated parsed CSR retrieval and limited address validation functionality into their back office systems. This successful integration of two pre-ordering functions in a commercial setting is probative evidence that carriers are capable of integrating the remaining pre-ordering functions. This evidence is also consistent with KPMG's finding that Bell Atlantic's pre-ordering and ordering interfaces are integratable. Although KPMG did not build a back office system to automatically populate the pre-ordering data into the ordering interface, it did evaluate the compatibility of the pre-ordering and ordering field names and formats and found that carriers would be able to integrate the information into their back office systems. 139. We are not persuaded by commenters' claims that full integration is not presently possible because Bell Atlantic's pre-ordering and ordering field names and formats are not entirely uniform. Based on the record evidence of successful commercial integration, it does not appear that incompatible fields are significantly increasing carriers' costs or impeding their ability to integrate pre-ordering and ordering functionality. In fact, MCI WorldCom indicates that it resolved problems with field incompatibility for the two functions that it has integrated successfully. Of course, to the extent that Bell Atlantic becomes aware of any inconsistencies in field names or formats that would impede a carrier's ability to integrate pre-ordering and ordering functions, we expect that Bell Atlantic promptly will design and deploy a software correction or provide the necessary technical assistance to competing carriers in the interface integration. 140. Access to Loop Qualification Information. We find that Bell Atlantic demonstrates that it offers nondiscriminatory access to OSS pre-ordering functions associated with determining whether a loop is capable of supporting xDSL advanced technologies. As an initial matter, we recognize that the Commission's recently enunciated UNE Remand rules, which further defined an incumbent LEC's obligations regarding nondiscriminatory access to loop qualification information, are not in effect. We do not consider, therefore, whether Bell Atlantic complies with the requirements that resulted from that proceeding in the context of this section 271 application. Rather, for purposes of this application, in determining whether Bell Atlantic is providing nondiscriminatory access to its OSS in accordance with section 271(c)(2)(B)(ii) and (xiv), we evaluate only whether Bell Atlantic provides requesting carriers equivalent access to the loop qualification functionality that it provides to itself. 141. As the Department of Justice observes, "[a]ccess to pre-ordering information is particularly important in connection with DSL services because of the special loop requirements for such services." Whether a prospective customer can be provided a particular advanced service often depends upon the carrier having access to detailed information about available loops, including the actual loop length and the presence of bridged taps, load coils, and digital loop carrier equipment. As the Commission previously has explained, a BOC's duty to provide nondiscriminatory access to OSS extends beyond the interface component to encompass all of the processes and databases used by the BOC in providing service to itself and its customers. In the Advanced Services Order and NPRM, the Commission explained that "[i]f new entrants are to have a meaningful opportunity to compete, they must be able to determine during the pre- ordering process as quickly and efficiently as can the incumbent, whether or not a loop is capable of supporting xDSL-based services." A BOC therefore must provide requesting carriers nondiscriminatory access to the systems and processes for identifying loop characteristics that it provides to its retail representatives. 142. Bell Atlantic provides three avenues for competing carriers to obtain information regarding its loops. First, for a limited number of central offices, Bell Atlantic provides a mechanized loop qualification process that indicates a theoretical loop length and whether a loop is qualified for ADSL service. Bell Atlantic is currently surveying its entire loop inventory to identify loops that are ADSL-capable, and expects to have "93 percent of Bell Atlantic's central offices in New York with completed or pending collocation orders" pre-qualified by the end of 1999. Second, for central offices that are not included within the mechanized loop qualification database, Bell Atlantic will conduct a "Manual Loop Qualification" to provide carriers with the same information that is ordinarily available through the mechanized loop qualification process (i.e., theoretical loop length and ADSL capability). Third, in order to access more detailed information about the makeup of a particular loop, carriers can request a manual "Engineering Query" that can provide the physical loop length, the number and location of load coils, the length and location of bridged taps, the gauge of the wire at specific locations, and the locations of digital loop carrier equipment. Bell Atlantic states that almost all of this information must be obtained and verified using paper loop plant records, or "plats." 143. We find that these mechanized and manual processes enable requesting carriers to access loop qualification information in substantially the same time and manner as Bell Atlantic's retail operations. The record shows that competing carriers have access to the same database that Bell Atlantic makes available to its retail representatives, and therefore the same information for the same central offices. We disagree with commenters' claims that the mechanized process is discriminatory because, in populating the database, Bell Atlantic filtered its back office information in such a manner that it is useful only for Bell Atlantic's particular advanced services offering. Indeed, we find that competing carriers have access to the same underlying information that Bell Atlantic used to populate the mechanized loop qualification database. Although carriers seek real-time electronic access to other back office databases, we do not find convincing evidence on this record that the information that carriers seek in electronic form is currently contained in any existing Bell Atlantic database that carriers cannot already access. 144. Response Times. We find that Bell Atlantic demonstrates that it provides requesting carriers access to pre-ordering functionality in substantially the same time that it provides access to its retail operations. With respect to parsed CSR retrieval, which has no retail analogue, we conclude that Bell Atlantic provides access sufficient to allow an efficient competitor a meaningful opportunity to compete. 145. To compete effectively in the local exchange market, competing carriers must be able to perform pre-ordering functions and interact with their customers as quickly and efficiently as the incumbent. The Commission previously has determined that a slower, less efficient process would have a significant impact on a competing carrier's ability to compete. For example, competing carriers must be able to retrieve a prospective customer's service record and other pre-order information in substantially the same time that it takes a BOC's retail representative to access the same information. A slower process can lead to delay while a prospective customer is on the line, causing the customer to view the competing carrier as a less efficient competitor than the BOC. Such a delay would also increase a carrier's operating costs and impede its ability to engage in aggressive marketing campaigns. 146. Our finding that Bell Atlantic processes pre-order inquiries from competing carriers in substantially the same time that it takes to process analogous retail transactions is based on Bell Atlantic's performance data. Bell Atlantic reports pre-order response times according to a performance standard of "parity plus four seconds" established by the New York Commission based on a consensus reached in the Carrier-to-Carrier collaborative proceeding. Given the additional security measures and computer translations needed to process pre-order transactions from competing carriers, we find that the "parity plus four seconds" standard is a reasonable and appropriate measure of whether Bell Atlantic processes pre-order transactions for competing carriers in substantially the same time that it processes its own pre-order transactions. 147. Performance data from August through September 1999 show that Bell Atlantic responds to pre-order inquiries from competing carriers in substantially the same time that it responds to analogous pre-order inquiries from retail representatives. Where Bell Atlantic deviated from the parity standard, it did so by only a fraction of a second for some pre-order functions, and less than two seconds for all others. Although a few commenters claim that these disparities are significant, we disagree and find that the slight variations in response times are not likely to impair the ability of a competing carrier to negotiate a service order while a customer is on the line. We also find no evidence in the record that these slight deviations have impacted a competing carrier's ability to conduct an aggressive marketing campaign or to compete effectively in the local exchange market. We therefore do not find that the slight deviations warrant a finding that Bell Atlantic does not return pre-order transactions for competing carriers in substantially the same time that it does for itself. We are nonetheless prepared to take appropriate enforcement action should the deviations in response times become more commercially significant or widespread. 148. We reject commenters' assertions that Bell Atlantic's performance measurements do not accurately reflect pre-order response times experienced by carriers, given the measures that Bell Atlantic implemented prior to filing its application that capture pre-order response time more accurately. Specifically, as agreed to in the New York Commission's Carrier-to-Carrier collaborative proceeding, Bell Atlantic generates pre-order response time measurements using the EnView system (formerly called Sentinel). Instead of timing actual pre-order transactions, EnView simulates pre-ordering transactions for both competing carriers and Bell Atlantic's retail operations using "robots." These robots send periodic pre-order inquiries, at least ten transactions per hour for each transaction type, into Bell Atlantic's back office pre-ordering systems 24 hours a day, seven days a week. The response times reported in the metrics are monthly averages of the average daily transactions captured from 8:00 a.m. to 6:00 p.m., Monday through Friday. Prior to August, the EnView system reported response times only for Bell Atlantic's older Electronic Interface Format (EIF) interface. In August, at the request of the New York Commission staff, Bell Atlantic began separately measuring and reporting response times for the EDI interface and, for both interfaces, began measuring transaction time from receipt of the request at the Bell Atlantic firewall to return of the response through the Bell Atlantic firewall. 149. We find that the changes implemented in August significantly improved the accuracy of the EnView system as a measure of pre-order response time. Specifically, we find that the EnView system simulates pre-order transactions for all active pre-ordering interfaces; mirrors the type of transactions performed by Bell Atlantic retail representatives during retail service hours; and captures the entire time that the transaction passes through Bell Atlantic systems, including the firewall. Even though evidence of actual pre-order response time would also be useful for our analysis, we conclude that the EnView system is a suitable measure of the time that a carrier or retail representative's pre-order request traverses Bell Atlantic's systems. As more carriers access Bell Atlantic's pre-ordering systems through EDI and CORBA, however, we encourage the New York Commission to continue to work with Bell Atlantic and competing carriers to ensure that the EnView simulation system continues to accurately reflect Bell Atlantic's retail operations (in terms of variability of transactions and service hours) and capture response times properly. 150. We further find that, in addition to accommodating current demand, Bell Atlantic demonstrates that its pre-ordering systems and interfaces are scalable to handle reasonably foreseeable demand volumes. We base our conclusion on Bell Atlantic's current performance and KPMG's findings. We find that Bell Atlantic processed more than 1.3 million pre-ordering transactions from January through July 1999, with more than 200,000 processed in July alone. In addition, KPMG found that Bell Atlantic's pre-ordering interfaces and systems are capable of handling projected year-end 1999 volumes. KPMG also evaluated Bell Atlantic's network architecture and found that its systems have sufficient capacity to meet expected future usage volumes. 151. We also reject assertions by AT&T and MCI WorldCom that Bell Atlantic is not providing parsed CSR responses in competitive timeframes. As discussed above, parsed CSR functionality is necessary for carriers to integrate CSR data into their own back office systems. Because Bell Atlantic's retail representatives do not retrieve parsed CSRs, Bell Atlantic must provide access to parsed CSR functionality that affords an efficient competitor a meaningful opportunity to compete. 152. As an initial matter, we recognize that, for parsed CSR retrieval, unlike other pre- ordering transactions, Bell Atlantic must perform the additional step of parsing CSR information into identifiable fields prior to sending the information to the carrier. In light of this extra processing step, Bell Atlantic and competing carriers agreed in the Carrier-to-Carrier collaborative that the performance standard applicable to other pre-ordering response times should be modified for parsed CSR retrieval. Specifically, in late September, Bell Atlantic agreed to measure the timeliness of parsed CSR information according to a standard of "parity with retail unparsed CSR plus ten seconds," based on simulated transactions. Moreover, in the present proceeding, MCI WorldCom supports a similar ten-second standard for parsed CSR retrieval. Accordingly, we find that, for purposes of our analysis, a performance standard of parity with unparsed CSR retail response time plus ten seconds is a reasonable and appropriate measure of whether Bell Atlantic processes parsed CSR inquiries in a manner that allows an efficient carrier a meaningful opportunity to compete. 153. Performance data indicates that Bell Atlantic provides timely access to parsed CSRs. In response to commenters' claims regarding parsed CSR timeliness, Bell Atlantic submitted data on reply showing that in early October Bell Atlantic took, on average, 7.42 seconds to respond to parsed CSR inquiries. Although AT&T and MCI WorldCom assert that it takes much longer to receive parsed CSR responses, in view of the general and conclusory nature of their assertions, we have no confidence that the claimed longer response times are attributable to Bell Atlantic and not to delay in AT&T's or MCI WorldCom's own systems. Accordingly, we find these allegations insufficient to refute Bell Atlantic's performance data. We therefore conclude that the record evidence demonstrates that Bell Atlantic is processing parsed CSRs in a manner that affords competitors a meaningful opportunity to compete. 154. Interface Availability. We conclude that Bell Atlantic demonstrates that its interfaces are available in a manner that affords an efficient competitor a meaningful opportunity to compete. A stable, reliable pre-ordering interface is necessary for competing carriers to market their services and serve their customers as efficiently and at the same level of quality that Bell Atlantic provides to itself. The Commission previously has found that the unavailability of an interface could directly and negatively affect a carrier's interaction with its customers. 155. Bell Atlantic measures EDI interface availability 24 hours a day using the EnView emulation system. Based on the Carrier-to-Carrier collaborative proceeding, the New York Commission established a performance standard requiring that Bell Atlantic's interfaces be available at least 99.5 percent of their scheduled availability during prime-time hours, using simulated responses. As an initial matter, we find that the designation of prime time hours from 6:00 a.m. to 12:00 a.m., Monday through Saturday, appropriately captures critical hours in which competing carriers access the interfaces. Given the broad designation of prime time, we find the 99.5-percent standard a reasonable and appropriate measure of whether Bell Atlantic's interfaces are sufficiently available to afford an efficient competitor a meaningful opportunity to compete. Although competing carriers may also input pre-order transactions outside of these hours, we find it unlikely that they will have a customer on the line during those hours. For this reason, minor interface downtime during non-prime time hours is not as likely to deprive an efficient competitor of a meaningful opportunity to compete. We therefore find that Bell Atlantic's interface availability during non-prime time hours is a less important indicator of its ability to provide nondiscriminatory access to its OSS functions. 156. We base our conclusion that Bell Atlantic's interfaces are sufficiently available on performance data from July through September 1999 showing that Bell Atlantic's interfaces were generally available as scheduled. For prime time hours, the EDI interface was available 100 percent of its scheduled time in July and August 1999, and 99.94 percent in September. During non-prime time, the EDI interface was available 99.9 percent of its scheduled time in June and 100 percent in July and August. Although the availability dropped to 97.01 percent in September, because we place less emphasis on this metric, we do not consider unavailability for three percent of non-prime time hours to present a barrier to an efficient competitor's ability to meaningfully compete by completing transactions in a timely manner. 157. We also base our conclusion on KPMG's verification that Bell Atlantic's interfaces are consistently available during scheduled hours of operation. Despite noting some instances of connectivity interruption or system unavailable error messages, KPMG found that Bell Atlantic's EDI and Web GUI interfaces for pre-ordering and ordering were "consistently available." Furthermore, in its limited test of parsed CSR functionality, KPMG did not experience any outages or system unavailable errors. We also note that, following the KMPG test results, Bell Atlantic improved its File Transfer Protocol (FTP) process to resend files automatically and to alarm Bell Atlantic support staff if FTP transmissions are not successful. Given the evidence in the record, we reject claims by AT&T and MCI WorldCom that Bell Atlantic's interfaces are not available sufficiently to afford competitors a meaningful opportunity to compete. f. Ordering 158. In this section we address Bell Atlantic's ability to provide access to its OSS ordering functions to competing carriers. We conclude that Bell Atlantic demonstrates that it provides nondiscriminatory access to its ordering systems in accordance with the requirements of section 271. In addition, we find that Bell Atlantic shows that its systems will be able to meet reasonably foreseeable commercial volumes in the future. We note that the New York Commission also concludes that Bell Atlantic is able to satisfactorily process orders and that its ordering systems are scalable. We also conclude that Bell Atlantic satisfies its obligation to provide access to order status and jeopardy information, to the extent it is available, in a nondiscriminatory manner. Finally, we conclude that Bell Atlantic provides nondiscriminatory access to order completion notification. (i) Background 159. Bell Atlantic's interfaces provide competing carriers with electronic access for a full range of ordering functionality. Competing carriers may place service orders with Bell Atlantic over either an EDI interface or a Web GUI. As of the filing date of this application, six carriers were using EDI for ordering and three were in the certification process, which is a precursor to the use of EDI. In addition, over 100 competing carriers were using the Web GUI at the time of filing. Once an order is received, Bell Atlantic responds with either a "Local Service Request Confirmation" (order confirmation) notice or a "Local Service Request Rejection" (order rejection) notice. These notices are important because they provide information to a competing carrier about whether its order has been accepted, or whether it has been rejected and requires resubmission. 160. Bell Atlantic generates order confirmation and rejection notices as a result of either mechanized or manual processing of orders, and returns them electronically over the GUI or EDI interface regardless of how they were processed. Bell Atlantic's operations support systems generate a mechanized order confirmation or rejection notice automatically (i.e., without human intervention) if the order is able to "flow-through." For orders that do not flow-through, Bell Atlantic generates order confirmation and rejection notices after the order is manually processed by Bell Atlantic wholesale representatives. The Carrier-to-Carrier guidelines, which were established by the New York Commission in conjunction with Bell Atlantic and the competing carriers, require the return of 95 percent of mechanized order confirmation and rejection notices within two hours of submission to Bell Atlantic, and 95 percent of manually processed order confirmation and rejection notices under ten lines within 24 hours of submission. We find that this standard, developed as a result of a collaborative proceeding including Bell Atlantic and competing carriers, is generally a reasonable measure of whether Bell Atlantic processes orders in a manner that provides an efficient competing carrier with a meaningful opportunity to compete. As demonstrated below, Bell Atlantic generally meets these standards, and where Bell Atlantic has fallen short of the standards, the shortfall has not been significant. (ii) Discussion 161. As an initial matter, we find that, unlike prior section 271 orders where the Commission began its analysis of access to ordering functions with a discussion of order "flow- through rates," a number of factors present in this application weigh against doing so here. Specifically, in prior orders the Commission asserted that the "substantial disparity between the flow-through rates of the [applicant] and those of competing carriers, on its face, demonstrate[d] a lack of parity." To the extent that these prior statements could be read to suggest that flow- through rates standing alone are a conclusive measure of nondiscriminatory access to ordering functions, we now clarify that when presented with circumstances like those in the instant record it is unnecessary to focus on order flow-through rates to the same degree we have in past orders. As explained below, the record in this proceeding indicates that Bell Atlantic's provision of access to its ordering functions is substantially better than in any other prior application. When considered in the context of such performance, we find that it would be inappropriate to consider order flow-through rates as the sole indicia of parity. 162. The Commission has, in part, used order flow-through as a potential indicator of a wide range of problems that underlie a determination of whether a BOC provides nondiscriminatory access to its OSS. Where, as in this application, other evidence shows that such problems do not exist, however, it is unnecessary to center our analysis on flow-through rates. For example, in the Second BellSouth Louisiana order, the Commission expressly found that the low order flow-through in the record was indicative of deficiencies in a BOC's systems for which the Commission also had other independent record evidence, including: (1) the failure to provision orders in a timely manner, (2) the failure to provide competing carriers with complete, up-to-date, business rules and ordering codes; (3) the lack of integration between pre- ordering and ordering functions; and (4) the failure to provide order status notices electronically. We have also used flow-through rates as an indicator of a BOC's ability to process competing carriers' orders, at reasonably foreseeable commercial volumes, in a nondiscriminatory manner. Flow-through rates, therefore, are not so much an end in themselves, but rather are a tool used to indicate a wide range of possible deficiencies in a BOC's OSS that may deny an efficient competitor a meaningful opportunity to compete in the local market. 163. Unlike the BOC systems we examined in prior orders, none of the specific deficiencies that we have previously associated with low flow-through rates is present in Bell Atlantic's systems. As discussed above, Bell Atlantic provides virtually all order status notices electronically, provides complete, up-to-date, business rules and ordering codes, makes integrated pre-ordering and ordering interfaces available through EDI, and, as discussed below, provisions orders in a timely fashion. Moreover, as discussed more fully below, we find that Bell Atlantic scales its system as volumes increase, and demonstrates its ability to continue to do so at reasonably foreseeable volumes. As a result, in this application flow-through has significantly less value as an indicator of deficiencies of Bell Atlantic's OSS. Thus, a different analysis is warranted. Specifically, in light of the facts and circumstances of this application, we conclude that Bell Atlantic's overall ability to return timely order confirmation and rejection notices, accurately process manually handled orders, and scale its systems is more relevant and probative for analyzing Bell Atlantic's ability to provide access to its ordering functions than a simple flow-through analysis. We note that this approach is consistent with the New York Commission's view that Bell Atlantic's order flow-through is not the only indicator of Bell Atlantic's ability to process orders in a nondiscriminatory fashion or to meet significant increases in order volumes. (a) Unbundled Network Element Orders 164. We find that Bell Atlantic demonstrates that it is providing nondiscriminatory access to its OSS ordering functions for unbundled network elements (i.e., UNE-loop and UNE- platform). We note that Bell Atlantic supports its application with Carrier-to-Carrier performance data, which aggregates UNE-loop and UNE-platform data, and the New York Commission based its initial comments on this aggregated data. Although we analyze Bell Atlantic's provision of ordering access using primarily aggregated UNE data, we conclude that our analysis would yield the same results were we to examine disaggregated data. In recent months Bell Atlantic has met, or has come very close to meeting, the strict benchmark standards for on-time processing of unbundled network element orders established in the Carrier-to-Carrier proceeding. According to the New York Commission's own calculations, Bell Atlantic's performance in providing on- time order confirmation and rejection notices, whether manually processed or mechanized, was about 94 percent for August and September and has been trending upwards. Similarly, in recent months Bell Atlantic's average time for returning an order confirmation or rejection notice, whether manual or mechanized, was between six and eight hours and has also been improving. We note that even when orders are manually processed by Bell Atlantic, competing carriers are still receiving their order status notices electronically and, for nearly all of their orders, within twenty-four hours of placing the order. Notably, Bell Atlantic has improved its on-time performance despite the fact that monthly volumes of UNE orders have increased from over 8,600 orders in January to almost 70,000 orders in September. Accordingly, we find that Bell Atlantic's ability to process nearly all competing carrier UNE orders in under 24 hours, and a majority of such orders within two hours of submission, provides an efficient competing carrier with a meaningful opportunity to compete. Should Bell Atlantic's performance deteriorate, however, we will be prepared to take appropriate enforcement action. 165. We note that Bell Atlantic's ability to process such large order volumes in a timely fashion is in stark contrast to any BOC's performance the Commission has considered in previous section 271 proceedings. The record indicates that Bell Atlantic is able to process orders more quickly than other BOCs in prior section 271 proceedings. For example, in the Second BellSouth Louisiana Order the Commission noted that BellSouth only returned order confirmation notices, on average, over 18 to 19 hours after it received an order, and over 21 to 27 percent of such notices were returned beyond a 24 hour interval. In contrast, in recent months Bell Atlantic has returned order confirmation notices, on average, within about five to eight hours and, as discussed above, returns nearly all order confirmation and rejection notices within 24 hours. 166. Even considering Bell Atlantic's flow-through, however, we conclude that the Carrier-to-Carrier flow-through rate is not reflective of the actual flow-through capabilities of Bell Atlantic's systems. The record shows that Bell Atlantic's systems are capable of providing high levels of order flow-through, but are dependent, in part, on the performance of competing carriers to achieve high rates. We first examine commercial usage data because it is the most probative evidence that Bell Atlantic's ordering systems are operationally ready. To obtain the most accurate picture of a competing carrier's ability to access Bell Atlantic's ordering functions we look to the actual flow-through rates of individual carriers. Flow-through rates disaggregated by carrier show that the rates for competing carriers submitting UNE-platform orders in September range from about 1 to 83 percent. Similarly, the rates for carriers submitting UNE-loop orders range from about 1 to74 percent in September. Because all competing carriers interface with the same Bell Atlantic system, this wide range of results strongly implies that the competitors, rather than Bell Atlantic, are largely responsible for any "poor" UNE flow-through performance. For example, one such cause is competing carrier error. Bell Atlantic manually corrects certain types of errors in competing carrier orders, rather than rejecting such orders. The New York Commission found that over 30 percent of the orders that fail to flow-through are caused by such errors. In its evaluation, the New York Commission attributes the "bulk" of the competing carrier errors to typographical errors and notes that such errors should be eliminated with the implementation of integrated pre-order and order interfaces. 167. In prior orders the Commission has noted that a BOC is not accountable for flow- through problems that are attributable to competing carriers' errors. The Commission has previously rejected BOCs' claims that competing carrier "error" caused orders to be rejected or to fail to flow-through because we could not make a judgment regarding how many of the errors the BOC attributed to the competing carriers should have been assigned to the BOC for failure to provide clear business rules or integrated pre-ordering and ordering interfaces. We find that the record in this application demonstrates that Bell Atlantic's integration of its interfaces and timely and up-to-date business rules supports Bell Atlantic and the New York Commission's contention that such competing carrier errors are attributable to the actions of competing carriers. Based upon this evidence, we find that the bulk of these "errors" can be properly attributed to competing carriers that, for example, choose not to integrate their interfaces, do not adequately train and manage their employees, or do not invest in the necessary systems. 168. Second, KPMG's test also supports our conclusion that Bell Atlantic's systems are capable of achieving high rates of order flow-through. KPMG tested the ability of EDI and GUI orders to flow from competing carriers through the interface into the Bell Atlantic ordering system without human intervention. KPMG's test shows that Bell Atlantic's systems can achieve UNE-platform flow-through rates of over 99 percent and UNE-loop flow-through of over 85 percent for orders designed to flow-through. KPMG also found that over 99 percent of all UNE orders designed to flow-through did so at stress volume levels. Although higher than the rates reflecting commercial usage, we conclude that KPMG's test indicates that Bell Atlantic's systems are capable of achieving high levels of flow-through. 169. Although we recognize that the Department of Justice and commenters assert that the level of manual processing in Bell Atlantic's system suggests that Bell Atlantic's systems are not scalable, we believe that the totality of the evidence demonstrates Bell Atlantic's systems are scalable. In addition to showing its systems are handling current volumes of UNE orders in a nondiscriminatory manner, we find that Bell Atlantic demonstrates that its ordering systems will be able to handle reasonably foreseeable commercial volumes of such orders in a nondiscriminatory manner. We base our conclusion on Bell Atlantic's performance and the KPMG Final Report. As discussed above, Bell Atlantic has shown its ability to manually process orders in a timely and accurate fashion. As the New York Commission points out, Bell Atlantic has a track record of commercial performance that shows its ability to process orders in a timely fashion while demand increases. For example, as noted above, despite tremendous increases in monthly UNE order volumes from over 8,600 orders in January to almost 70,000 orders in September, Bell Atlantic has consistently increased its overall UNE on-time performance for the processing of order status notices. Moreover, as discussed above, actual carrier data and KPMG's test shows that Bell Atlantic's systems are capable of achieving high levels of UNE order flow-through. Thus, contrary to the Department of Justice's assertions, we conclude that the evidence discussed above supports a finding that Bell Atlantic's ordering systems will be able to handle reasonably foreseeable commercial volumes of competing carrier orders in a nondiscriminatory manner and, as such, provides competing carriers a meaningful opportunity to compete. Finally, we note that Bell Atlantic's recent commitment to implement improvements to its OSS demonstrates that Bell Atlantic will continue to scale its systems to accommodate the expected increase in competing carrier UNE-platform order volumes. 170. Moreover, Bell Atlantic has shown its commitment to maintain, and even improve upon, its current level of performance. Although not determinative of this issue, our confidence that Bell Atlantic's systems are scalable also stems, in part, from Bell Atlantic's commitment to working with competing carriers to increase their individual order flow-through performance and reduce the number of rejection notices they receive. For example, Bell Atlantic has committed to initiate monthly workshops to address order quality. At these workshops, Bell Atlantic will provide generic examples of orders that failed to meet flow-through criteria and suggested steps for improving orders. Bell Atlantic believes this will "serve to improve [competing carrier] order quality, reduce [order] rejects, and improve the overall flow-through rate." In addition, Bell Atlantic has committed to work with competing carriers on an individual basis to address their specific and unique order quality concerns. We are encouraged by these proposed refinements as they indicate an intention on the part of Bell Atlantic to further enhance the scalability of its OSS systems, thereby ensuring that it will continue to process orders in a timely and accurate manner. 171. We also come to a different conclusion than the Department of Justice and commenters with regard to Bell Atlantic's accuracy for manually processed orders. Although we recognize that manually processed orders are more prone to error than orders that are processed automatically, there is no reliable evidence that this is the case in the instant application or that Bell Atlantic's manual processing of orders injects a level of error that prevents efficient competitors a meaningful opportunity to compete. Bell Atlantic measures the accuracy of its manual processes in at least two ways: (1) accuracy of order confirmation notices (order confirmation accuracy); and (2) overall accuracy of competing carrier orders entered into its service order processor (service order accuracy). 172. Bell Atlantic's order confirmation accuracy metric is obtained by comparing certain fields in an order submitted by a competing carrier with the order confirmation notice issued by a Bell Atlantic representative. In recent months, Bell Atlantic's performance metrics range between 95 and 99 percent accuracy for UNE order confirmation notices. AT&T and the Department of Justice, however, claim that Bell Atlantic's order confirmation accuracy for loop orders is not accurately reflected in this metric. Specifically, the Department of Justice notes that during a July Technical Conference before the New York Commission, Bell Atlantic stated that its rate of returning accurate order confirmation notices for loop orders at the time was between 60 and 70 percent. AT&T alleges that data AT&T compiled shows that between June 21 and August 31 Bell Atlantic returned inaccurate order confirmation notices for more than 50 percent of hot cut loop orders. In the face of this discrepancy, we rely upon the New York Commission's conclusion that the disparity results in part from disagreement regarding the information that should be included in the order confirmation notices because the New York Commission has had greater opportunity to analyze this issue in the context of the collaborative process. Moreover, in its reply comments, Bell Atlantic states that subsequent improvements to its process for returning order confirmation notices caused it to reach levels of order confirmation accuracy for loop orders of more than 95 percent since the July Technical Conference. We are also satisfied that AT&T's claims have been largely remedied by the parties' agreement to include specific information in order confirmation notices for loop orders. Contrary to the Department of Justice, we therefore find that, based upon all the relevant record evidence, AT&T's claims do not warrant a finding that Bell Atlantic's order confirmation accuracy rate for loop orders is commercially significant. 173. The Department of Justice and commenters also assert that Bell Atlantic's "service order accuracy" metric shows that Bell Atlantic is unable to accurately process manual orders. This metric compares the order submitted by a competing carrier with the completed Bell Atlantic service order. The metric is compiled each business day by Bell Atlantic from an audit of a random sample of orders. Bell Atlantic contends the metric is flawed because it attributes to Bell Atlantic as errors all differences between the original competing carrier order and the order information entered in its service order processor. Thus, according to Bell Atlantic, this metric actually counts as Bell Atlantic errors those cases where Bell Atlantic has fixed an error in a competing carrier order. 174. In support of its contention that this metric is flawed, on reply, Bell Atlantic submitted an analysis of a random sample of orders. We are persuaded by Bell Atlantic's analysis that its service order accuracy metric is flawed and that its actual level of service order accuracy is significantly higher than reflected in its performance data. We believe that Bell Atlantic's position is further buttressed by its performance on the installation quality performance metrics, which measure, among other things, whether the services requested by the end user were accurately installed. These metrics show that Bell Atlantic has consistently provided service with very low levels of reported installation troubles, as compared to the service it provides its own customers. Given the totality of the evidence described above, including Bell Atlantic's analysis and its performance on the installation quality metrics, we find that Bell Atlantic's accuracy in processing manual orders is sufficient to provide competing carriers with a meaningful opportunity to compete. 175. Moreover, we do not share the Department of Justice's concern about the rate of competing carrier orders rejected by Bell Atlantic. Bell Atlantic has reported that on average it rejected between about 27 and 34 percent of the UNE orders that it received during June through September. Although the Department of Justice recognized that Bell Atlantic is not responsible for orders that are rejected because of competing carrier error, it expressed concern that some of the rejections may occur for reasons within Bell Atlantic's control. Ultimately, the Department of Justice concluded that it did not have sufficient information to determine the extent to which Bell Atlantic is, if at all, responsible for the level of rejected orders. We note, however, that in reaching its conclusion the Department of Justice did not discuss the evidence submitted by Bell Atlantic revealing that order rejections greatly vary on an individual carrier basis from 3 percent to greater than 70 percent. We agree with Bell Atlantic that this wide variation in the individual rates strongly implies that the care a competing carrier takes in submitting its orders makes a significant difference in the rate at which its orders are rejected. Accordingly, because we conclude the average rejection rate is overstated, we do not accord it as significant weight in this application as the other factors discussed in this section, such as Bell Atlantic's overall ability to return order confirmation and rejection notices, accurately process manually handled orders, and scale its systems. 176. We also conclude that AT&T and MCI's assertions that they have not received order confirmation or rejection notices for all of their orders are insufficient to rebut Bell Atlantic's evidence showing compliance with the requirements of this checklist item. Although we do not discount the importance of receiving an order confirmation or rejection notice for every order, the present record, including AT&T and MCI's claims, does not indicate that, to the extent any lapses exist, such failures are a systemic problem. Rather, they appear to be isolated problems attributable to either Bell Atlantic or the commenters. We note that we do not hold Bell Atlantic to a standard of perfection. If it were a systemic problem occurring for a significant number of orders, however, it would warrant a finding of noncompliance. 177. Thus after careful consideration of the evaluations of the Department of Justice and the New York Commission, as well as of the commenters, we find that the record demonstrates that Bell Atlantic provides nondiscriminatory access to its ordering functions for UNEs. Although our conclusion differs from that reached by the Department of Justice, we reach it by focusing, in part, on the timely return of order confirmation and rejection notices. Unlike the Department of Justice and various commenters, we place less importance on flow-through rates than in past orders because the deficiencies that we have previously associated with low flow- through rates are not present in Bell Atlantic's systems. Moreover, as explained above, we agree with the New York Commission that Bell Atlantic has shown that it is able to handle significant increases in order volumes and will be able to continue to do so at reasonably foreseeable order volumes. We also find that Bell Atlantic is able to manually process orders in an accurate manner. Finally, as noted above, the Department of Justice explicitly did not analyze Bell Atlantic's application under the competitive checklist of section 271(c)(2)(B) as we are required to do. Accordingly, we find that Bell Atlantic's overall ability to return timely order confirmation and rejection notices, accurately process manually handled orders, and scale its systems supports a finding that Bell Atlantic offers competing carriers a meaningful opportunity to compete. (b) Resale Ordering 178. We also find that Bell Atlantic demonstrates that it is providing nondiscriminatory access to its OSS ordering functions for resale services and, therefore, provides efficient competitors a meaningful opportunity to compete. As an initial matter, we note that there are virtually no objections from commenters to Bell Atlantic's provision of access to its ordering functions for resale services. Moreover, neither the Department of Justice nor the New York Commission found problems with Bell Atlantic's provision of access to its resale service ordering functions. 179. Although we recognize that the rate of flow-through of resale orders was an area of major concern in prior orders, as we explain above, it is of less concern here given the absence of the deficiencies that we have previously found to be associated with low order flow-through rates and Bell Atlantic's significantly better performance than seen in prior section 271 applications. Rather, we conclude that Bell Atlantic's overall ability, in light of the facts and circumstances of this application, to return timely confirmation and rejection notices accurately process manually handled orders, and process reasonably foreseeable commercial volumes in a nondiscriminatory manner is more relevant and probative for analyzing Bell Atlantic's provision of access to its ordering functions for resale services than a simple flow-through analysis. Thus, given these circumstances and evidence of other performance measures indicating that the access Bell Atlantic provides to its ordering functions offers efficient competitors a meaningful opportunity to compete, we place less emphasis on flow-through rates in this order than we have in prior orders. 180. In recent months Bell Atlantic has met, or has come very close to meeting, the strict benchmark standards set in the New York Carrier-to-Carrier proceeding. As discussed above, the Carrier-to-Carrier guidelines require the return of 95 percent of mechanized order confirmation and rejection notices within two hours of submission to Bell Atlantic, and 95 percent of manually processed order confirmation and rejection notices under ten lines within 24 hours of submission. Bell Atlantic has met, or has come close to meeting, these standards in recent months. According to the New York Commission's own calculations, this means that Bell Atlantic returned between 93 and 97 percent of all order confirmation and rejection notices on time for the months of June through September. We note that Bell Atlantic's average performance for returning an order confirmation or rejection notice, whether manual or mechanized, in recent months was between approximately four and seven hours. Bell Atlantic has achieved this reliable performance while resale order volumes have ranged from 14,000 orders to 23,000 orders monthly from January through September. Finally, we note that Bell Atlantic's ability to process relatively large volumes of orders in a timely and wholly electronic fashion is significantly better than the performance of the other BOCs in prior applications. Accordingly, we find that Bell Atlantic's ability to process nearly all competing carrier resale orders in under 24 hours, and nearly half of such orders within two hours of submission, provides a competing carrier with a meaningful opportunity to compete. Should Bell Atlantic's performance deteriorate, however, we will be prepared to take appropriate enforcement action. 181. Even considering Bell Atlantic's flow-through performance, however, we find that Bell Atlantic is providing an efficient competitor a meaningful opportunity to compete. As we concluded in our discussion of UNE ordering, the record shows that the average flow-through rate provided in the Carrier-to-Carrier reports do not reflect the actual flow-through capabilities of Bell Atlantic's systems. An examination of flow-through rates of individual competing carriers ordering resale services from Bell Atlantic show flow-through rates in September ranging from about one to 82 percent. Because all carriers ordering resale services from Bell Atlantic interface with the same Bell Atlantic systems, we conclude that this wide range of results for competitors strongly implies that competitors are likely more responsible for low average flow- through performance than Bell Atlantic. Moreover, the KPMG Final Report supports a finding that Bell Atlantic's systems are capable of high flow-through for resale orders, as KPMG found that over 99 percent of all resale orders designed to flow-through did so at normal and stress levels. 182. We also find that Bell Atlantic demonstrates that it is capable of providing nondiscriminatory access to its resale ordering functions at reasonably foreseeable volumes. Although, as mentioned above, Bell Atlantic processes significant volumes of resale orders, the record does not indicate an upward trend in those monthly volumes. We do not believe that the volumes of resale orders are likely to grow to the same degree as we expect volumes of UNE orders to increase in the foreseeable future. As the Department of Justice recognized, resale service in New York is principally used as a "transitional tool on the way to facilities based competition." Thus, because we do not expect monthly volumes of resale orders to increase substantially above the volumes that Bell Atlantic has shown it is currently capable of processing in a manner that provides competitors with a meaningful opportunity to compete, we are satisfied that Bell Atlantic will meet future demand for reasonably foreseeable volumes of resale orders. Moreover, we note that Bell Atlantic has shown its willingness and ability to accommodate the needs of its wholesale customers as their needs grow increasingly complex. Should our predictive judgment concerning future volumes of resale orders prove inaccurate, and should Bell Atlantic's performance in processing such orders deteriorate, we fully expect to take appropriate enforcement action. 183. Finally, as we concluded in our discussion of UNE ordering, we find that Bell Atlantic demonstrates adequate performance with respect to order accuracy and order rejection for resale services. First, the Carrier-to-Carrier data indicate that Bell Atlantic has consistently provided service with very low levels of reported installation troubles, as compared to the service it provides its own customers, and accurate order confirmation notices. Moreover, for the reasons discussed above with regard to UNE ordering, we disregard Bell Atlantic's low reported performance for service order accuracy. Second, we find that Bell Atlantic's overall rejection rate for resale orders more accurately reflects the particular capabilities of individual competing carriers, rather than deficiencies in Bell Atlantic's systems. The Carrier-to-Carrier rejection rates for resale orders in recent months indicate that, on average, Bell Atlantic rejects between about 23 and 31 percent of resale orders submitted by competing carriers. When examined on an individual carrier basis, however, rejection rates vary from three to 73 percent. As we concluded for UNE rejection rates, we find that this wide variation in individual rates strongly implies that the ability of a competing carrier to submit accurate orders significantly affects the rate at which its orders are rejected. Because we conclude the average rejection rate is overstated, we do not accord it as significant weight in this application as the other factors discussed in this section. Rather, it is Bell Atlantic's overall ability to return timely order confirmation and rejection notices, accurately process manually handled orders, and scale its systems that supports our finding that Bell Atlantic affords competitors a meaningful opportunity to compete. (c) Jeopardies 184. We conclude that Bell Atlantic makes order status and "jeopardy" information (i.e., notice that a service installation due date will be missed) available to competing carriers in a nondiscriminatory manner. Bell Atlantic explains that it makes this information available to competing carriers in several ways. First, it provides electronic access to jeopardy notices contained in Open Query System reports, which are generated three times daily from its Work Force Administration (WFA) system. The WFA system is updated by field technicians and reflects whether an order is pending, has been completed, or has been (or will be) missed. Competitors thus can retrieve this information and "determine whether there is a problem on a given order." Bell Atlantic also indicates that competing carriers may check on the status of an order in WFA or in the Service Order Processor (SOP) through the pre-ordering interfaces, or by calling one of Bell Atlantic's dispatch centers. Like their counterparts at competing carriers, Bell Atlantic's retail representatives also must take steps to determine whether there is any indication that an appointment will be missed, or has been missed. Specifically, Bell Atlantic states that its retail representatives may check the status of an order by querying the WFA system, by querying SOP, or by calling a dispatch center. 185. We conclude that the order status and jeopardy information system created by Bell Atlantic for wholesale orders is nondiscriminatory because it allows competing carriers to access order status and "jeopardy" information, to the extent that it is available, in substantially the same time and manner as Bell Atlantic's retail operations can access such information. We thus disagree with AT&T's suggestion that Bell Atlantic's inability to actively provide electronic jeopardy notices, instead of merely providing access to such information, reflects discriminatory access to its ordering functionality. We also disagree with NorthPoint's suggestion that Bell Atlantic must create a process for providing "notice before the due date that it is going to miss the due date." Although we recognize that a system designed to deliver jeopardy notification well in advance of missed appointments would lessen the impact of such misses, we reiterate that the standard sought in this instance is nondiscriminatory access to Bell Atlantic's OSS. Accordingly, we do not require Bell Atlantic to establish a system for creating and delivering jeopardy notifications to competing carriers that is superior to the system Bell Atlantic has for its own retail representatives or customers. 186. Although Bell Atlantic does not actually deliver jeopardy notices to competing carriers with respect to provisioning resale services, individual UNEs and UNE-P, we note that it has established a mechanism for actively providing such notices in connection with its hot cut process. Under the "due date minus two" procedure, Bell Atlantic is required to check for a competing carrier's dial tone two days before a hot cut due date and promptly to notify the carrier if there is a problem. The New York Commission recognizes that this "allows the [competitive LEC] the opportunity to notify its customer of potential delay and, if necessary, postpone the due date." We commend Bell Atlantic for developing this "due date minus two" jeopardy process, and find that it appears to be critical to the proper functioning of the hot cut process. (d) Completion Notices 187. We conclude that Bell Atlantic provides order completion notification in a manner that affords an efficient competitor a meaningful opportunity to compete. An order completion notice informs a competing carrier that Bell Atlantic completed the installation of the service requested by the particular order, which provides notice to the carrier that it has responsibility for the customer's care and may begin billing the customer for service. Until the competing carrier receives a completion notice, the carrier does not know that the customer is in service, and cannot begin billing the customer for service or addressing any maintenance problems experienced by the customer. Thus, untimely receipt of order completion notices directly impacts a competing carrier's ability to serve its customers at the same level of quality that Bell Atlantic provides to its retail customers. Accordingly, the Commission has instructed a section 271 applicant to demonstrate that it provides competing carriers with order completion notices in a timely and accurate manner. The BOC must minimize any delay between the actual installation of service and the competing carrier's receipt of an order completion notice. 188. We base our finding that Bell Atlantic provides sufficient order completion notification on Bell Atlantic's provision of both "billing completion" and "work completion" notices to competing carriers. Bell Atlantic sends billing completion notices when an order is recorded as completed in Bell Atlantic's billing systems. Specifically, after Bell Atlantic's Service Order Processor (SOP) passes order completion information to Bell Atlantic's billing systems (CRIS), the billing records are updated overnight and billing completion notices are sent the following day. In August 1999, Bell Atlantic began providing "work completion" notices (also referred to as a "provisioning completion" or "field completion" notice) to inform carriers of the completion of the work associated with an order. For orders requiring physical work, when the technician reports order completion to Bell Atlantic's Work Force Administration (WFA), it generates a completion in SOP, which automatically notifies the competing carrier of the work completion. For orders not requiring physical work, SOP is automatically updated during overnight processing and generates a work completion notice the following morning. Both types of completion notices are sent to the carrier over the same interface used to submit the order. 189. With respect to performance data, Bell Atlantic measures billing completion notification timeliness, or the time that elapses from when an order is recorded as completed in Bell Atlantic's billing systems until the time Bell Atlantic distributes a billing completion notice to the carrier. The New York Commission, based on the Carrier-to-Carrier collaborative proceeding, established a performance standard requiring Bell Atlantic to return 95 percent of billing completion notices by noon the day following order completion in its billing system. We find this standard to be a reasonable and appropriate measure of whether Bell Atlantic provides timely notification that a service order has been recorded as complete in Bell Atlantic's billing systems. For both resale and unbundled network elements, Bell Atlantic reports timely return of billing completion notices for 100 percent of carriers' orders from June through September 1999. In addition, KPMG verified that Bell Atlantic returned 99 percent of the billing completion notices for its test orders on time. KPMG also found that less than one percent of the 3,000 completion notices lacked complete information. In light of recent Bell Atlantic performance and KPMG's findings, we reject AT&T's allegation that Bell Atlantic does not deliver timely completion notices. 190. We note with approval that the New York Commission recently required Bell Atlantic to augment its reporting of the timeliness of billing completion notification by also reporting the timeliness of work completion notification. Specifically, Bell Atlantic must report work completion notification timeliness and the average time that elapses between work completion and billing completion, as well as the percentage of orders where this interval exceeds one and five days. For the timeliness of work completion notification, based on the Carrier-to- Carrier collaborative, the New York Commission established a performance standard requiring Bell Atlantic to deliver 95 percent of work completion notices by noon the day following completion of the work associated with the order. We find this standard a reasonable and appropriate measure of work completion notification timeliness. Although Bell Atlantic has not begun reporting these intervals, in this case we do not find that the lack of this performance data warrants a finding of noncompliance with this checklist item. Nonetheless, we expect that Bell Atlantic will promptly comply with the standard established by the New York Commission. 191. Based on the record evidence, we reject commenters' allegations that Bell Atlantic frequently fails to provide completion notices at all, and that the missing notices are not captured in the performance reporting. Although we do not discount the importance of receiving an order completion notice for every order, commenters do not demonstrate that the missing notices are attributable to Bell Atlantic's systems. Rather, based on the present record, we find that the failure to receive a notice may be attributable to either Bell Atlantic or the interfaces and systems of competing carriers. As such, we find that the commenters' allegations are insufficient to rebut Bell Atlantic's evidentiary showing. If in the future we find evidence of a systematic and widespread failure of Bell Atlantic to deliver completion notices to competing carriers, we are prepared to take appropriate enforcement action. 192. Furthermore, we are encouraged that Bell Atlantic will provide fielded complex completion notifications in April 2000. This functionality will enable competing carriers to detect and correct provisioning errors early. Although Bell Atlantic has yet to complete implementation of this functionality, AT&T admits that the decision to defer implementation until April 2000 came about by an August 1999 vote of Bell Atlantic and competing carriers in a change management collaborative meeting, with AT&T dissenting. Accordingly, we note that the delay in implementing this functionality is attributable in part to competing carriers. g. Provisioning 193. In this section we conclude that Bell Atlantic provisions competing LEC customers' orders for UNE-P and resale POTS in substantially the same time and manner as it is provisioning its own retail customers. Our conclusion is based on the totality of the evidence before us. First, we find that Bell Atlantic's systems are set up to provide parity of service for provisioning wholesale and retail orders. Second, we conclude that evidence from the Carrier to Carrier metrics shows that Bell Atlantic is missing fewer competitive LEC customer appointments and providing equal or better quality installations, compared to appointments for its own retail customers. Third, we consider evidence concerning Average Completed Intervals but conclude that, due to flaws in this data, as evidenced by the Gertner/Bamberger study and other evidence, such data should be accorded less weight. (i) Background 194. In the Ameritech Michigan Order, the Commission first addressed nondiscriminatory access to OSS provisioning functions in the context of a BOC's showing of compliance with checklist item 2. The Commission concluded that Average Installation Interval data are critical to determining whether a BOC provides equivalent access to OSS because such data are "direct evidence of whether [a BOC] takes the same time to complete installations for competing carriers as it does for [itself], which is integral to the concept of equivalent access." The Commission also recognized, however, that data showing average installation intervals, on its face, may erroneously suggest discriminatory conduct by a BOC because of underlying flaws in the manner in which the data is calculated. Such flaws may result in average installation intervals that appear to be longer for competing carriers than for a BOC, even though the BOC may be provisioning services for competing carriers in a nondiscriminatory manner. In the Ameritech Michigan Order, therefore, the Commission asked Ameritech to explain any underlying flaws in its average installation data by, for instance: (1) excluding transactions for customers that requested due dates beyond the first available due date; and (2) disaggregating by service types to account for the impact that different types of services may have on the average installation interval. At the same time, the Commission found that data on Missed Appointments (Due Dates Not Met) could be helpful "to explain any inconsistencies between the average installation intervals for [a BOC] and other carriers." The Commission explained that evidence that due dates are offered to a BOC's retail units and to competing carriers on a nondiscriminatory basis has probative value, although it found that Ameritech had not sufficiently explained its proposal for submitting such evidence for the Commission to determine whether it would be an adequate substitute for actual installation interval data. 195. In the OSS Performance Measures NPRM, the Commission tentatively concluded that the Average Completion Interval and Percentage of Due Dates Missed metrics are most probative in assessing whether an incumbent LEC processes and completes orders from competing carriers in the same time frame in which it processes and completes its own retail orders. The Commission tentatively concluded that both of these measurements are necessary to ensure that the incumbent LECs are not able to mask discrimination and, therefore, are necessary to provide a complete picture of an incumbent LEC's ability to complete orders for competing carriers in a nondiscriminatory manner. (ii) Discussion 196. For the reasons set forth below, we conclude that Bell Atlantic provisions UNE-P and resale orders to competitors in substantially the same time and manner that it provisions these orders to itself. To demonstrate parity in the provision of UNE-P and resale service orders, Bell Atlantic provides two performance measurements, the Average Completed Interval and Percentage of Missed Appointments, and the retail analogues for these measurements. 197. Provisioning Processes. Based on the evidence in the record, we conclude that Bell Atlantic demonstrates that it provides nondiscriminatory access to its provisioning processes. Specifically, we find that Bell Atlantic provides competitive LECs and its retail operations with equivalent access to information on available service installation dates. For non-dispatch orders, Bell Atlantic asserts that it makes available the same set of standard intervals for competing carriers and its retail representatives. A competitive LEC's customer representative can, for instance, offer a customer "same day" service for services such as Call Waiting, just as a Bell Atlantic retail representative can. For orders requiring dispatch of a Bell Atlantic service technician, competitive LECs have access to the same Smarts Clock system as do Bell Atlantic retail representatives. 198. Our conclusion is buttressed by KPMG's finding that overall, Bell Atlantic's provisioning processes for competing carriers are provided at parity with its retail operations. As part of its independent test of Bell Atlantic's OSS, KPMG conducted a thorough assessment of Bell Atlantic's provisioning systems. KPMG examined the performance of these systems in analyzing and routing orders, handling problems with orders, coordinating the work of different centers, loading translations into the switch for non-designed services (e.g., POTS, ISDN), and scheduling the work needed for dispatch and designed services. KPMG interviewed Bell Atlantic personnel, reviewed documentation, observed daily operations, and reviewed sample order files, in twelve centers involved in provisioning. KPMG concluded that Bell Atlantic satisfied all test criteria for the provisioning function. 199. We also find that Bell Atlantic provides requesting carriers with the same level of confidence as its own retail operations that the due date promised to customers will be the actual due date that the BOC assigns to the order when it is processed. Some commenters nevertheless argue that Bell Atlantic does not provide nondiscriminatory treatment in its provision of confirmed due dates. We acknowledge that there is evidence that some orders receive confirmed due dates later than was requested. For example, KPMG found that 9.7 percent of its test orders submitted through the EDI interface received confirmed due dates later than was requested. In addition, as discussed more fully below, evidence submitted by Bell Atlantic suggests that the average confirmed due date for UNE-P orders was later than the average requested due date by an average of 0.18 days, or 4.3 hours, for June-August 1999. We do not find, however, that this warrants a finding of checklist noncompliance. We find that the 4.3-hour average disparity between requested and confirmed due dates is not large enough to be competitively significant. We believe consumers are much more sensitive to whether their service is being installed on the arranged appointment date, as opposed to whether their appointment is set a little later after the originally requested time. We note that because 90 percent of KPMG's EDI UNE-P orders received confirmed due dates no later than requested, KPMG determined that it was satisfied that Bell Atlantic provisions confirmed due dates consistent with KPMG's requested due dates on its test orders. Thus, we agree with the New York Commission that Bell Atlantic provides competing carriers with confirmed service installation dates in a nondiscriminatory manner. 200. Due Dates Met. The record evidence also demonstrates that Bell Atlantic is meeting the service installation dates for competitive LEC customers at higher rates than for its own retail customers. The Percent Missed Appointment metric measures the percentage of confirmed appointments that Bell Atlantic has missed due to its own fault. Specifically, the data demonstrate that, over a four month period, Bell Atlantic has consistently met a higher percentage of installation appointments for competing carriers than for itself. 201. In addition, the evidence demonstrates that Bell Atlantic performs service installations for competitive LEC customers at a higher level of quality than for its own retail customers. The metrics "Percent Installation Troubles Reported Within 7 Days" and "Percent Installation Troubles Reported Within 30 Days" show the quality of Bell Atlantic's service installations by measuring customer troubles reported within 7 and 30 days, respectively. According to these metrics, a much smaller percentage of competitive LEC customers experiences difficulties after installation, than retail customers. 202. Average Completed Interval. In concluding that Bell Atlantic provisions resale and UNE-P orders for competing carriers on a nondiscriminatory basis, we accord little weight to data evidencing the average intervals in which resale and UNE-P installations are completed. The record contains performance data that, standing alone, shows that competing carriers experience longer average completed intervals than do Bell Atlantic retail customers. Although these disparities are statistically significant, we conclude that Bell Atlantic has presented sufficient evidence to demonstrate that the disparity between wholesale and retail average completed intervals is not the result of discriminatory conduct, but rather is the result of factors outside of its control and unrelated to the timeliness and quality of Bell Atlantic's provisioning of resale and UNE-P to competing carriers. As such, we agree with Bell Atlantic that the Average Completed Interval data is flawed and therefore, should be accorded little weight in our analysis here. 203. According to Bell Atlantic, the disparity between Average Completed Intervals for competitive LECs and Bell Atlantic is substantially caused by three factors unrelated to the timeliness of its service installations: (1) competitive LECs are choosing installation dates beyond the first installation date made available by Bell Atlantic's systems (the "W-coding" problem); (2) for non-dispatch orders, competitive LECs are ordering a relatively larger share of services and UNEs that have long standard intervals (the "order mix" problem); and (3) for dispatch orders, competitive LECs are ordering a relatively larger share of services in geographic areas that are served by busier garages and, as a result, reflect later available due dates (the "geographic mix" problem). In conjunction with its Average Completed Interval data, Bell Atlantic submits a study by Dr. Gertner and Dr. Bamberger (Gertner/Bamberger study) to support its claim that its Average Completed Interval data is flawed for these reasons. We note that although AT&T criticized some aspects of the Gertner/Bamberger study, no commenter disagrees with Bell Atlantic's assertions that its Average Completed Interval data is flawed. By submitting a study to substantiate its claims that the Average Completed Interval data is flawed, we note that Bell Atlantic's application is quite different from BellSouth's Louisiana II application. In that application, although BellSouth's data on its face consistently supported a general conclusion that BellSouth provided services to competing carriers' customers in twice the amount of time that it provided services to its retail customers, BellSouth offered no analysis or other evidence that purported to explain why these data might be flawed or to supplement BellSouth's showing on OSS provisioning. 204. First, we find that Bell Atlantic demonstrates that its average completed interval data for competing carriers reflects a disproportionate share of orders with installation dates beyond the first available date offered by Bell Atlantic (the "W-coding" problem). If competing carriers request later installation dates more often than Bell Atlantic, then installation intervals for those competing carriers will be, on average, longer than those for Bell Atlantic customers. Although Bell Atlantic relies upon competing carriers to "code" orders that include requests for longer-than-average provisioning intervals so that they can be excluded from the average completed interval measures, the Gertner/Bamberger study establishes that competing carriers "miscode" a significant percentage of non-dispatch orders, causing those requests to be erroneously included in the performance data. Although the Gertner/Bamberger study does not address dispatched orders, we agree with Bell Atlantic that it is likely that competing carriers similarly miscode dispatched orders for which an appointment date after the first available date is sought, which would result in longer average provisioning intervals. Furthermore, no commenter seriously challenges Bell Atlantic's claim that competing carriers frequently request installation dates beyond the first available date. Indeed, AT&T and MCI claim that they normally request longer intervals than the standard interval. 205. Second, we also find persuasive Bell Atlantic's argument that its average completed interval data for competing carriers' non-dispatch orders reflects a disproportionate share of order types with longer-than-average standard intervals (the "order mix" problem). The Gertner/Bamberger study shows that competing carriers order a relatively larger share of non- dispatch orders with longer-than-average standard intervals, which would result in longer average completed intervals. The study compared the average standard intervals for resale, UNE-P, and Bell Atlantic retail orders, for all orders and for orders within the standard interval (correctly "W" coded orders). The study found that for some months, the average standard interval was longer for wholesale customers than for retail customers. A difference in average standard intervals could cause the average completed intervals to be different, even if Bell Atlantic was provisioning orders in a nondiscriminatory fashion, and only properly coded orders were included in the Average Completed Interval metric. The observed difference in standard intervals supports the argument that there are differences in order mixes between wholesale and retail orders that will affect the average standard intervals and, therefore, the Average Completed Intervals for wholesale and retail orders. 206. With respect to dispatch orders, we are also persuaded by Bell Atlantic's argument that competing carriers experience longer completed intervals than its retail customers because the automatic appointment clock used to schedule available appointments offers longer average appointment intervals in some geographic areas than in others (the "geographic mix" problem). As a result, reported average completed intervals will vary depending upon where competitive carriers are ordering service. Average completed intervals for dispatch resale services and UNE-P would be longer if a high proportion of those competing carriers provide service to geographic areas with busy garages. 207. We disagree with the Department of Justice and AT&T that the gap between requested and completed intervals that Gertner and Bamberger's study found for wholesale UNE- P orders is evidence of discrimination. Specifically, the study found that the average requested interval was 1.39 days while the average completed interval was 1.57 days for orders in which competitors requested the standard interval over a three month period. Thus, the study finds a difference of 0.18 days longer in the provisioning intervals of wholesale orders. AT&T argues that this difference in the provisioning of UNE-P orders is likely to be statistically significant and, therefore, is evidence of discrimination. Both the Department of Justice and AT&T express concern about the even larger difference of 0.52 days, reported in August for UNE-P orders. 208. Gertner and Bamberger note, however, that "requested" due dates are not the same as "confirmed" due dates. Because Bell Atlantic is missing very few appointments, almost all orders should have completion dates that are the same as their confirmed due dates. Therefore the reported gap between requested and completed intervals is very likely caused by some orders being given later confirmed due dates than was requested. As discussed above, we do not believe that a delay of 0.18 days, or 4.3 hours, in the appointment date impairs the ability of a competing carrier to meaningfully compete. We therefore agree with Bell Atlantic that even though the difference may be statistically significant, it has no practical competitive significance. 209. In view of the conclusions of the Gertner/Bamberger study and other evidence submitted by Bell Atlantic that its average completed interval data for competing carriers is flawed, we find unpersuasive the claims of competing carriers that this data demonstrates that Bell Atlantic provisions resale services and UNE-P in a discriminatory manner. Although we continue to believe that average completed intervals can be probative in determining whether Bell Atlantic provisions resale services and UNE-P in a nondiscriminatory manner, where, as here, a BOC has made an adequate showing that the data on average completed intervals is flawed, we must consider other evidence in making our parity determination. Specifically, as described above, we find that Bell Atlantic provides competing carriers with equivalent access to its process for selecting service installation dates as well as its provisioning processes overall and with timely confirmed service installation dates. In addition, we find that Bell Atlantic consistently meets a higher percentage of installation appointments for competitors than for itself. Accordingly, based on the totality of the evidence submitted by Bell Atlantic, we conclude that Bell Atlantic demonstrates that it is provisioning resale services and UNE-P to competing carriers in substantially the same time and manner as for its retail operations. 210. Our conclusion is not undermined by KPMG's examination of Average Completed Interval data, which found an unexplained half day difference between the Average Completed Interval for its own test non-dispatch UNE-P orders and Bell Atlantic's own retail orders, and for which KPMG found it was Not Satisfied. Indeed, our own analysis of the average completed interval data for non-dispatch orders for the months of June-August 1999 for competing carriers and Bell Atlantic using the results of the Gertner/Bamberger study revealed an unexplained half day difference as well. Like the New York Commission, however, we do not believe that a half day difference in provisioning intervals is competitively significant. Rather, we find that given that there will always be some limited manual processing of competitors' orders, even where, as discussed below, such processing is considered "timely" as measured by performance metrics, such manual intervention will inevitably affect provisioning intervals. Under the circumstances of this application, where Bell Atlantic has shown that it is meeting the rest of the relevant provisioning performance metrics, we decline to find that Bell Atlantic is provisioning resale and UNE-P orders in a discriminatory fashion. h. Maintenance & Repair 211. We conclude that Bell Atlantic demonstrates that it provides nondiscriminatory access to maintenance and repair OSS functions. First, we find that Bell Atlantic has deployed the necessary interfaces, systems, and personnel to enable requesting carriers to access the same maintenance and repair functions that Bell Atlantic provides to itself. We then conclude that Bell Atlantic's systems allow carriers to access those functions in substantially the same time and manner as Bell Atlantic's retail operations. We further find that Bell Atlantic restores service to customers of competing carriers in substantially the same time and manner that it restores service to its own customers. Finally, we conclude that Bell Atlantic performs maintenance and repair work for customers of competing carriers at substantially the same level of quality that it provides to its own customers. (i) Background 212. As part of its obligation to provide nondiscriminatory access to OSS functions, Bell Atlantic must provide requesting carriers with nondiscriminatory access to its maintenance and repair systems. A competing carrier that provides service through resale or unbundled network elements remains dependent upon the incumbent LEC for maintenance and repair. Because Bell Atlantic performs analogous maintenance and repair functions for its retail operations, it must provide competing carriers access that enables them to perform maintenance and repair functions "in substantially the same time and manner" as Bell Atlantic. Equivalent access ensures that competing carriers can assist customers experiencing service disruptions using the same network information and diagnostic tools that are available to Bell Atlantic personnel. Without equivalent access, a competing carrier would be placed at a significant competitive disadvantage, as its customer would perceive a problem with Bell Atlantic's network as a problem with the competing carrier's own network. (ii) Discussion 213. Functionality. We conclude that Bell Atlantic offers maintenance and repair interfaces and systems that enable a requesting carrier to access all the same functions that are available to Bell Atlantic's retail representatives. Specifically, Bell Atlantic offers requesting carriers access to its maintenance and repair systems through a Web-based GUI electronic interface. Inquiries submitted over the Web GUI feed into the Repair Trouble Administration System (RETAS), which automatically directs the transaction to Bell Atlantic's back office maintenance and repair systems. The Web GUI enables carriers to perform the same functions that Bell Atlantic's retail operations perform, including: (i) conduct a mechanized loop test (for resale and the UNE platform but not for unbundled loops), (ii) create a trouble ticket, (iii) determine the status of a trouble ticket, (iv) modify a trouble ticket, (v) request cancellation of a trouble ticket, and (vi) request a trouble report history. The interface can be used for all local exchange services. Bell Atlantic also staffs a "Regional CLEC Maintenance Center" to support wholesale maintenance and repair services. 214. Commercial usage and extensive testing by KPMG show that Bell Atlantic provides requesting carriers with nondiscriminatory access to maintenance and repair functionality. Thus, we find that Bell Atlantic demonstrates that its maintenance and repair interface is operationally ready and capable of handling reasonably foreseeable demand levels. In terms of commercial usage, carriers perform more than 40,000 maintenance transactions per month. Furthermore, after evaluating Bell Atlantic's systems, performance, processes, documentation, network surveillance, work center operations and work coordination for the delivery of competing carriers' maintenance and repair services, KPMG verified the functionality of Bell Atlantic's maintenance and repair systems for competing carriers and found them at parity with Bell Atlantic's retail systems and processes. KPMG also verified that Bell Atlantic's retail systems were capable of handling 500 transactions per hour (or 4,000 in an eight-hour day). 215. We disagree with AT&T's assertion that Bell Atlantic must demonstrate that it provides an integratable, application-to-application interface for maintenance and repair. Bell Atlantic is obligated to provide maintenance and repair functionality in substantially the same time and manner that it provides the functionality to itself. Although the Commission has indicated that a BOC would afford carriers a more complete opportunity to compete by offering an integratable, application-to-application maintenance and repair interface, we also found that the lack of integration does not necessarily constitute discriminatory access, provided that the BOC otherwise demonstrates that it provides equivalent access to its maintenance and repair functions. Accordingly, although it presently does not offer an application-to-application interface, we find that Bell Atlantic satisfies its checklist obligation by demonstrating that it offers competitors substantially the same means of accessing maintenance and repair functions as Bell Atlantic's retail operations. 216. We also find that Bell Atlantic permits competing carriers to open trouble tickets immediately on recently-completed service orders. In light of an early exception noted by KPMG, Bell Atlantic implemented a function in RETAS in April that permits competing carriers to enter a trouble ticket immediately after completion of a service order. KPMG verified that the enhancement would resolve its concerns about a lag time in creating trouble tickets. As a result, Bell Atlantic claims that competing carriers can enter a trouble ticket electronically at an earlier point than its retail representatives. Although Covad asserts generally that it cannot open trouble tickets on new loops for 24 hours, we are unable to determine whether their allegation post-dates Bell Atlantic's system enhancement. In any event, we find that the record evidence does not support Covad's allegation. 217. Response Times. We further conclude that Bell Atlantic's maintenance and repair interface and systems process trouble inquiries from competing carriers in substantially the same time and manner as Bell Atlantic processes inquiries concerning its own retail customers. To compete effectively in the local exchange market, competing carriers must be able to diagnose and process customer trouble complaints with the same speed and accuracy that Bell Atlantic diagnoses and processes complaints from its retail customers. A slower process can lead to customer perception that the competing carrier is a less efficient service provider than the BOC. 218. We base our finding of nondiscriminatory OSS processing time on Bell Atlantic's performance data. Although it had previously reported maintenance and repair response times according to absolute benchmark standards, Bell Atlantic started reporting response times according to a performance standard of "parity plus four seconds" in its September Carrier-to- Carrier report. Given the additional security measures required for competing carriers' access to Bell Atlantic's maintenance and repair systems, we find that this "parity plus four seconds" standard is a reasonable and appropriate measure of whether Bell Atlantic processes maintenance and repair requests for competing carriers in substantially the same time that it processes those requests for its own retail operations. 219. Performance data from June through September 1999 indicates that Bell Atlantic met the parity standard each month for modifying trouble tickets, failed to meet the standard for creating trouble tickets, and had mixed results for canceling a trouble ticket and conducting a POTs test. With respect to conducting a POTS trouble test, which is the most common maintenance and repair function, Bell Atlantic processed requests from competing carriers faster than requests from its retail operations in June, July and September, with a slight deviation from the standard in August. For creating a trouble ticket, although Bell Atlantic deviated from the standard each month, we find that the deviations were slight and do not warrant a finding that Bell Atlantic fails to process requests to create trouble tickets in substantially the same time for competing carriers as it does for its retail operations. Likewise, Bell Atlantic did not consistently meet the standard for canceling trouble tickets, but failed by only a fraction of a second each time. Accordingly, in light of the slight deviations in response times and the lack of evidence that such deviations are impeding carriers' access to maintenance and repair OSS functions, we conclude that competing carriers are able to process maintenance and repair requests in substantially the same time as Bell Atlantic's retail operations. We are nonetheless prepared to take appropriate enforcement action should the deviations in response times become more commercially significant or widespread. 220. Time to Restore. We conclude that Bell Atlantic repairs trouble complaints for competing carriers in substantially the same time and manner that it repairs complaints from its own customers. The Commission has stressed that a BOC is obligated to repair trouble for a customer of a requesting carrier in substantially the same time that it takes to repair problems experienced by its own customers. For example, because a reliable telephone line may be crucial for a business customer to conduct its business, the Commission has emphasized the importance of timely resolution of trouble reports from a competing carrier's business customers. 221. We base our finding of nondiscriminatory restoration time on Bell Atlantic's performance data. From June through September 1999, for both resale and unbundled network elements, Bell Atlantic generally repaired trouble reported by customers of competing carriers faster than it repaired trouble reported by its own retail customers. In fact, during this period Bell Atlantic consistently cleared a higher percentage of trouble reports within 24 hours for competitors than for itself. In addition, customers of competing carriers were out of service for substantially the same amount of time that Bell Atlantic's retail customers were out of service. This level of performance is substantial evidence that Bell Atlantic responds to trouble reports and restores service in substantially the same time and manner for competing carriers as for itself. Although some commenters assert generally, without evidentiary support, that Bell Atlantic fails to address competitors' trouble tickets in a timely and efficient manner, they do not dispute the performance data submitted by Bell Atlantic and verified by the New York Commission. Given this, we find that the performance measurements provide compelling evidence that Bell Atlantic responds to competitors' trouble complaints in substantially the same time and manner that it responds to its own customers' complaints. 222. Quality of Work Performed. We also find that Bell Atlantic demonstrates that it performs maintenance and repair work for customers of competing carriers at the same level of quality that it performs repair work for its retail customers. In order to compete effectively in the local exchange market, competing carriers must be able to access maintenance and repair functions in a manner that enables them to provide service to their customers at a level of quality that matches the quality of service that Bell Atlantic provides its own customers. A competing carrier's customer may become dissatisfied if the customer experiences frequent service problems, especially repeated troubles. In determining the quality of maintenance and repair work performed by Bell Atlantic for competing carriers, we examine the rate of trouble reported by customers of competing carriers as compared with Bell Atlantic's own retail customers, as well as the rate of repeat reports of trouble. 223. Bell Atlantic's performance data reveals that customers of competing carriers reported a lower rate of network trouble than Bell Atlantic's retail customers. From June through September 1999, for both resale and unbundled network elements, the rate of loop trouble reported was lower for competing carriers than for Bell Atlantic's retail operations. Similarly, during the same period, the rate of central office trouble reported for carriers' resale customers was lower than, or equal to, Bell Atlantic's, and the rate for customers served through unbundled network elements was just slightly higher for competing carriers than for Bell Atlantic's retail operations. This level of performance, coupled with the lack of any conflicting data or claims of inferior maintenance in the record, indicates that Bell Atlantic is not discriminating against competing carriers in routine network maintenance and repair functions. 224. Similarly, performance data on the rate of repeat trouble reports indicates that Bell Atlantic repairs trouble for competitors at the same level of quality that it provides to itself, or better. Consistently from June through September 1999, for both resale and unbundled network elements, a lower percentage of competitors' customers reported repeat trouble within 30 days than Bell Atlantic's retail customers. Given the lack of conflicting data, we find that Bell Atlantic's performance on this measurement provides compelling evidence that the company is not discriminating in the quality of the repair work that it performs for competing carriers. 225. We further find that Bell Atlantic has implemented processes to safeguard against premature closing of trouble tickets. KPMG initially found that some Bell Atlantic technicians were closing out loop trouble tickets even if the customer was not back in service if they found no trouble at the specific dispatch location (e.g., the outside plant or the central office) without checking other locations. For these misdirected dispatch situations, carriers would need to open a second trouble ticket to resolve the problem. In response to KPMG's finding, Bell Atlantic implemented a new process under which Bell Atlantic's Regional CLEC Maintenance Center will open a second trouble ticket, either automatically (if the technician finds a problem on the line) or after it obtains the carrier's permission to issue a second ticket (if the technician finds no problem on the circuit). Although commenters allege that Bell Atlantic generally closes out trouble tickets without resolving the problem, we are unable to conclude, based on this record, that the process provided to competing carriers differs from Bell Atlantic retail operations or that Bell Atlantic is failing to adhere to the new procedures. Rather, the fact that competing carriers are reporting a lower rate of repeat trouble than Bell Atlantic's retail customers strongly signifies that Bell Atlantic is not closing out trouble tickets in a discriminatory manner. i. Billing 226. We find that Bell Atlantic provides nondiscriminatory access to its billing functions. Competing carriers need access to billing information to provide accurate and timely bills to their customers. Bell Atlantic is obligated to provide competing carriers with complete and accurate reports on the service usage of competing carriers' customers in substantially the same time and manner that Bell Atlantic provides such information to itself. To do so, Bell Atlantic provides competing carriers with billing information through Daily Usage Files (DUFs) and carrier bills. DUFs itemize daily usage records for competing carrier customers, while carrier bills serve as a monthly invoice that incorporates charges for all of the products and services provided to a competing carrier by Bell Atlantic. These are the same mechanisms that Bell Atlantic uses to provide billing information to its retail operations. 227. Like the New York Commission, we conclude that Bell Atlantic demonstrates that it provides nondiscriminatory access to its billing functions on the basis of the available Carrier-to- Carrier metrics and the KPMG Final Report. We find that the performance standards set by the New York Commission and developed in conjunction with Bell Atlantic and competing carriers are appropriate measures of Bell Atlantic's ability to provide competing carriers with DUFs and carrier bills in substantially the same time and manner that Bell Atlantic provides such information to itself. The Carrier-to-Carrier metrics indicate that, during the period from July to September 1999, Bell Atlantic's actual commercial performance consistently exceeds these standards. In addition, KPMG found Bell Atlantic's wholesale billing systems, processes, and operational support satisfactory. After testing seven bill types in eight billing cycles and making over 2,100 test calls to generate records, KPMG found that Bell Atlantic properly reported daily usage and applied correct rates and discounts to bill elements. 228. Although several commenters allege problems with Bell Atlantic's billing systems, we conclude that these allegations do not warrant a finding that Bell Atlantic fails to provide nondiscriminatory access to its billing functions. AT&T alleges that Bell Atlantic does not provide competing carriers with complete billing information on a consistent basis. The specific problems AT&T cites to support this argument, including difficulties with local usage file names and obtaining and processing local usage data, are not cited by any other commenter and are not supported by the Carrier-to-Carrier metrics or findings in the KPMG Final Report. Both CCA and Z-Tel argue that Bell Atlantic should alter its billing system to better meet their needs as competing carriers. Although we require a BOC to demonstrate that it is providing equivalent access to billing information, we do not mandate the use of a particular billing system. Accordingly, we reject CCA and Z-Tel's arguments. We also reject Adelphia, NALA, and TRA's allegations of double billing. Although we believe that evidence of a double billing problem demonstrates that a BOC is not providing nondiscriminatory access to its billing functions, we find that there is no evidence in the record to support these commenters' assertions. Similarly, we reject Z-Tel's allegation that Bell Atlantic refuses to provision service to residential customers that have outstanding balances on their Bell Atlantic retail accounts. Because Z-Tel offers no data to support this position and no other commenters raise this issue, we find that the record does not support Z-Tel's allegation. 2. Combinations of Unbundled Network Elements a. Background 229. In order to comply with the requirements of checklist item 2, a BOC must show that it is offering "nondiscriminatory access to network elements in accordance with the requirements of sections 251(c)(3)[.]" Section 251(c)(3) requires an incumbent LEC to "provide, to any requesting telecommunications carrier . . . nondiscriminatory access to network elements on an unbundled basis at any technically feasible point on rates, terms and conditions that are just, reasonable, and nondiscriminatory." Section 251(c)(3) of the Act also requires incumbent LECs to offer unbundled network elements to requesting carriers in a manner that allows them to combine them to provide a telecommunications service. 230. In the Ameritech Michigan Order, the Commission emphasized that the ability of requesting carriers to use unbundled network elements, as well as combinations of unbundled network elements, is integral to achieving Congress' objective of promoting competition in the local telecommunications markets. Using combinations of unbundled network elements provides a competitor with the incentive and ability to package and market services in ways that differ from the BOCs' existing service offerings in order to compete in the local telecommunications market. Moreover, combining the incumbent's unbundled network elements with their own facilities encourages facilities-based competition and allows competing providers to provide a wide array of competitive choices. Because the use of combinations of unbundled network elements is an important strategy for entry into the local telecommunications market, as well as an obligation under the requirements of section 271, we examine section 271 applications to determine whether competitive carriers are able to combine network elements as required by the Act and the Commission's regulations. b. Discussion 231. Based on the evidence in the record, we conclude that Bell Atlantic demonstrates that it provides to competitors combinations of network elements that are already preassembled in their network, as well as nondiscriminatory access to unbundled network elements, in a manner that allows competing carriers to combine those elements themselves. We base our conclusion on evidence of actual commercial usage and the results of KPMG's third party test. We note that the New York Commission concludes that Bell Atlantic has provided nondiscriminatory access to combinations of unbundled network elements. 232. The record indicates that Bell Atlantic, as required by the New York Commission, provides a variety of methods that allow competitive carriers to combine unbundled network elements with their own facilities. For example, in addition to the standard physical and virtual collocation arrangements, Bell Atlantic provides alternative collocation arrangements such as smaller physical collocation cages, shared collocation cages, and cageless collocation arrangements. The record also indicates that Bell Atlantic has provided eleven "Assembly Room" and "Assembly Point" arrangements which do not require conditioned space and take less time to implement than caged collocation arrangements. 233. The record also indicates that Bell Atlantic, as required by the New York Commission, provides access to preassembled combinations of network elements. For example, Bell Atlantic has provided to competitors more than 152,000 preassembled platforms of network elements, including the loop switch combination (UNE-P) out of certain central offices, as well as local switching elements in combination with other shared elements, such as shared transport, shared tandem switching, operator services, directory assistance, and SS7 signaling. In addition, Bell Atlantic provides Enhanced Extended Loops (EELs), a combination of loops and transport. All of these combinations are offered in accordance with the New York Commission's requirements. 234. We disagree with arguments that Bell Atlantic's collocation offerings are deficient. ALTS and several other carriers argue that BA's collocation arrangements involve delays that diminish the ability of the competitive LECs to provide the services they seek to offer. As discussed above, we conclude that Bell Atlantic's collocation offerings meet the Act's nondiscrimination requirements. 235. We are not persuaded by arguments that the restrictions Bell Atlantic places on the use of its loop-switch (UNE-P) and loop-transport (EEL) offerings warrant a finding of checklist noncompliance. Several parties argue that Bell Atlantic cannot limit the central offices from which the UNE-P is offered. They also assert that the sunset provision that allows Bell Atlantic's UNE-P offering to sunset 4-6 years is unlawful. With regard to Bell Atlantic's EEL offerings, several parties contend that Bell Atlantic also unlawfully restricts the availability of extended loops by refusing to allow competing LECs to use them to provide solely exchange access service. 236. In the wake of the Supreme Court's January 25, 1999 decision vacating the Commission's Rule 51.319 that identified the network elements incumbent LECs are required to provide on an unbundled basis, and prior to adoption of our order reinstating that rule, the incumbents' obligations with regard to offering unbundled network elements or combinations thereof has been unclear. Given this vacuum, we find it would be inequitable to penalize Bell Atlantic for complying with the rules established by the New York Commission. Although we have adopted new rules identifying the incumbent LECs' unbundling obligations, these rules are not in effect yet. Moreover, even under our new rules, the extent to which requesting carriers may place restrictions on their loop-transport combinations remains the subject of a further notice. We therefore find that the restrictions Bell Atlantic places on its loop-transport combinations and its UNE-P combinations do not warrant a finding of checklist noncompliance. Once our new rules identifying the unbundling obligations of network elements become effective, Bell Atlantic must fully comply with those rules. 3. Pricing of Network Elements a. Background 237. Checklist item 2 of section 271 states that a BOC must provide "nondiscriminatory access to network elements in accordance with sections 251(c)(3) and 252(d)(1)" of the Act. Section 251(c)(3) requires local incumbent LECs to provide "nondiscriminatory access to network elements on an unbundled basis at any technically feasible point on rates, terms, and conditions that are just, reasonable, and nondiscriminatory. . . ." Pursuant to section 252(d)(1), determinations by a state commission of just and reasonable rates for network elements shall be "based on the cost . . . of providing . . . the network element . . . and nondiscriminatory [ ] and may include a reasonable profit." Based on this statutory mandate, the Commission has determined that prices for interconnection and unbundled network elements (or UNEs) must be based on an incumbent LEC's forward-looking, long-run incremental costs for each network element. It adopted a pricing methodology that encompasses these concepts called TELRIC, or Total Element Long Run Incremental Cost. In order to prove compliance with these statutory provisions, a BOC must show that its prices for interconnection and unbundled network elements are based on forward-looking, long-run incremental costs. b. Discussion 238. Based on the evidence in the record, we conclude that Bell Atlantic demonstrates that its pricing of unbundled network elements complies with the requirements of checklist item 2. We agree with Bell Atlantic's assertion that it has worked with the New York Commission to establish prices for unbundled network elements and that these proceedings "have resulted in a full suite of TELRIC rates." Specifically, as discussed below, we agree with the New York Commission that Bell Atlantic's prices for switches and loops offered as unbundled network elements are priced pursuant to a forward-looking, long-run incremental cost methodology. The New York Commission further asserts that "prices conforming to the FCC's requirements are in effect for resale, interconnection, and unbundled network elements provided by Bell Atlantic- NY." The Department of Justice did not comment on Bell Atlantic's prices for unbundled network elements. We stress that we place great weight on the New York Commission's active review and modification of Bell Atlantic's proposed unbundled network element prices, its commitment to TELRIC-based rates, and its detailed supporting comments concerning its extensive, multi-phased network elements rate case, as discussed below. 239. Despite the fact that the Eighth Circuit stayed the Commission's pricing authority after the New York Commission had begun its network elements rate case, the New York Commission determined that it would proceed in the rate case on a TELRIC basis. In Phase One of its rate case, the New York Commission considered two different TELRIC-based cost models, one submitted by Bell Atlantic and another, the Hatfield model, submitted by AT&T and MCI. The New York Commission noted that Bell Atlantic objected to TELRIC "in principle" but that "the parties continued to rely on the TELRIC standard." The New York Commission held that it "need not evaluate the various costing methods on theoretical grounds" because The case was litigated on a TELRIC basis; all parties contemplate its being decided on that basis; TELRIC is certainly a reasonable approach to use, though just as certainly not the only one; and, as [Bell Atlantic] recognizes, as a practical matter there is no alternative other than the very unattractive one of temporary rates while a lengthy new case is litigated. 240. The New York Commission considered each of the cost elements to Bell Atlantic's TELRIC-based cost model. It approved, without modification, some of Bell Atlantic's proposed cost inputs, but substituted what it deemed "more reasonable inputs" to both Bell Atlantic's cost model and the Hatfield model. The New York Commission noted that, when it compared the modified results from the two cost models, the resulting costs converged and sometimes even crossed each other which, the New York Commission determined, defined a "sharply narrowed range of reasonable results that may be reached on the record here." The New York Commission determined that each cost model had its own advantages and disadvantages, and held that "in the absence of factors clearly tending one way or the other, prices will be set at the mid- point of that narrowed range." 241. Burden of Proof. We reject AT&T's assertion that Bell Atlantic has not provided sufficient detail in its section 271 application to demonstrate that its prices for unbundled network elements comply with the Act. In its section 271 application, Bell Atlantic asserts that the outcome of the New York Commission rate proceedings on network elements resulted in rates "fully consistent with this Commission's pricing rules, including the TELRIC methodology." While Bell Atlantic did not discuss in detail its pricing methodology in its section 271 application, it did provide sufficient documentation in its supporting affidavits and attachments for us to evaluate the pricing of each network element. Additionally, Bell Atlantic provided extensive records of the New York Commission's network elements rate case. 242. Switch Prices. We conclude that Bell Atlantic provides sufficient evidence to demonstrate that its switch costs are based on forward-looking, long-run incremental costs. We reject AT&T's allegation that Bell Atlantic's switching prices violate TELRIC principles because they fail to account for any cost savings from the steep switch discounts that an efficient carrier operating in the long run would unquestionably receive. AT&T previously raised this issue with the New York Commission, which considered AT&T's assertion and made significant modifications to Bell Atlantic's proposed switch prices. Using its TELRIC-based model, Bell Atlantic calculated an average total installed switch investment of $586 per line. This switch cost was significantly higher than those calculated by AT&T under the Hatfield model, which calculated a per-line switch investment of $125. The New York Commission held that the wide disparity between the two TELRIC models' inputs called both figures into question, and that the record before it suggested that neither figure was reliable. The New York Commission then conducted its own examination into switching costs, after which it estimated a per-line switch cost of $303, which it reduced to $192 to account for declining switch prices within the industry. The New York Commission contends that the resultant switch prices are TELRIC-based. Based on the evidence in the record, we find that the New York Commission has already considered AT&T's allegation that Bell Atlantic's proposed switch costs were too high and responded appropriately. Bell Atlantic may only recover $192 per switch per line, a significant reduction from its original proposal of $586 per line and an amount much closer to AT&T's estimation. We have no basis to disagree with the New York Commission that its calculation of switching costs is a "reasonable calculation of pertinent costs, arrived at by the New York Commission Staff's application of forward-looking TELRIC analysis." 243. We also disagree with AT&T's further assertions that: (1) the Commission has concluded in the context of the Universal Service Fund that TELRIC does not permit recovery of the cost of "augmented switches," which are existing switches with capacity upgrades, and Bell Atlantic's proposal to recover such costs here violates TELRIC; (2) the New York Commission admitted in its reply comments that it did not apply a TELRIC methodology to switch prices and set switch prices based on speculative claims, not facts; and (3) Bell Atlantic's switch rates are merely interim in nature, pending a new pricing rulemaking. 244. First, we note that in the Local Competition First Report and Order, the Commission held that, while TELRIC consists of "methodological principles" for setting prices, states retain flexibility to consider "local technological, environmental, regulatory, and economic conditions." In reviewing state pricing decisions in the context of section 271 applications, we will not reject an application because isolated factual findings by a commission might be different from what we might have found if we were arbitrating the matter under section 252(e)(5). Rather, we will reject the application only if basic TELRIC principles are violated or the state commission makes clear errors in factual findings on matters so substantial that the end result falls outside the range that the reasonable application of TELRIC principles would produce. 245. Here, in response to AT&T's allegations regarding switch discounts, the New York Commission asserts that it "appropriately exercised its power to take account of conditions in New York" when it determined switching costs pursuant to TELRIC. We agree with New York that it has appropriately exercised its flexibility to set prices within a range of TELRIC- based rates. We also agree with the New York Commission that its determination of allowable switch costs was the result of a complex analysis that does not lend itself to simple arithmetic correction through the adjustment of a single input. AT&T has presented no evidence to persuade us that New York did not conform to TELRIC principles simply because it failed to modify one input into its cost model. We are not persuaded by AT&T's assertion that in our Universal Service proceeding, we disallowed the cost recovery of "augmented switches," and that Bell Atlantic's recovery includes such cost recovery, which violates our rules. As we stated in the Universal Service Tenth Report and Order, that federal cost model "was developed for the purpose of determining federal universal service support, and it may not be appropriate to use nationwide values for other purposes, such as determining prices for unbundled network elements." We specifically cautioned parties from making any claims in any other proceedings based on the inputs adopted in the Universal Service Tenth Report and Order. 246. Second, contrary to AT&T's assertion, we see no admission in the record by the New York Commission that it did not use a TELRIC-based cost methodology for switch prices. We find no basis to disagree with the New York Commission's assertion that it calculated pertinent costs "arrived at by the NYPSC Staff's application of forward-looking TELRIC analysis." Moreover, we are not persuaded that Bell Atlantic's switching costs are based on speculation, simply because AT&T believes the New York Commission did not adequately reflect switching discounts. As discussed above, the New York Commission engaged in extensive fact- finding in its rate case, and specifically considered AT&T's assertions about switching discounts. As a result, Bell Atlantic's switching prices were greatly reduced, with a final result that is very close to AT&T's estimated switching prices, further undermining AT&T's claims that Bell Atlantic's switch prices are double or even triple what they should be. 247. Third, we see no reason to disagree with the New York Commission that Bell Atlantic's switch costs are not "interim" merely because they may be adjusted in the future to account for newly adduced evidence. The New York Commission held that, while it had initially been persuaded by Bell Atlantic that it did not receive large switch discounts from its vendors, AT&T later presented new evidence on such discounts, which the New York Commission will examine in its second network elements rate case. AT&T has presented no evidence that the New York Commission's "ongoing examination of the [switch discount] issue betokens a failure to set TELRIC-compliant rates," nor does it refute the New York Commission's claim that these rates may be refined in the future, "but they are no less TELRIC- compliant on that account." 248. Loops - Copper Feeder. We also reject AT&T's allegation that Bell Atlantic's unbundled element prices are not TELRIC-based because Bell Atlantic uses fiber in the feeder portion of its loop plant, which can be more expensive than copper in longer loop lengths. AT&T raised identical arguments before the New York Commission. There, AT&T asserted that copper feeder is cheaper than fiber for loops shorter than 9,000-12,000 feet, and that Bell Atlantic should not be allowed to recover the higher capital costs of fiber feeder. AT&T also asserted that Bell Atlantic installed all-fiber feeder in order to subsidize its own broadband network for the provision of future services, and that competitors should not be required to subsidize such costs. AT&T also asserts that loops that may be efficient for shorter loop lengths such as those in Manhattan may not be efficient in other parts of New York state. In response, the New York Commission notes that it analyzed the difference between fiber and copper feeder, but found that the higher cost of fiber feeder was "more than offset" by the lower provisioning and maintenance costs of fiber. Additionally, the New York Commission was not persuaded by assertions that Bell Atlantic had inflated its loop costs in order to subsidize its own broadband ventures. The New York Commission found that the economics of copper versus fiber depend "not only on loop length but on capacity." The New York Commission held that New York's population per square mile supports "the economies afforded by fiber's greater capacity . . . even where distances are short." AT&T also alleges that Bell Atlantic's prices for unbundled loops include the costs of terminating DLC circuits at the switch using antiquated terminations rather than the modern GR-303 technology used for the loop feeder. AT&T contends that Bell Atlantic's use of older DLC terminations does not reflect an efficient, forward- looking network and thus violates TELRIC principles. AT&T again raised an identical argument before the New York Commission. The New York Commission found no evidence to support AT&T's allegations regarding either fiber feeder or DLC terminations. The New York Commission also noted that, in the future, competitors may wish to purchase elements to provide enhanced services to their own customers, and that fiber may prove useful for these purposes. AT&T also asserts that the New York Commission improperly relied on a 1991 Bell Atlantic cost study that was never placed into the record of the New York Commission's rate case when it considered the costs of fiber feeder. The New York Commission responds that its reliance on the 1991 cost study was both limited and proper. 249. We find that AT&T has not presented sufficient evidence to prove that the New York Commission erred in its determination or that it neglected to consider any relevant facts relating to fiber feeder or DLC termination technology. We have no reason to disagree with the New York Commission's conclusion that Bell Atlantic's use of fiber and DLC termination technology in this case does not make its rates inconsistent with a TELRIC methodology. 250. Conditioning of xDSL-Capable Loops. We find that Bell Atlantic's interim rates for xDSL provisioning and conditioning, which are subject to refund or true-up when the New York Commission completes its xDSL cost study, are not a basis for rejecting the section 271 application. DSL describes a "family of transmission technologies that use specialized electronics at the customer's premises and at a telephone company's central office . . . to transmit high-speed data signals over copper cables." Bell Atlantic offers unbundled loops for use by competing carriers to provide Asymmetrical Digital Subscriber Line (ADSL) and High Bit-Rate Digital Subscriber Line (HDSL). Bell Atlantic offers "ADSL-qualified links" to loops of less than 18,000 feet, and "HDSL-qualified links" to loops of less than 12,000 feet. Bell Atlantic asserts, however, that "certain technical difficulties arise when ADSL or HDSL signals are transmitted over loops that exceed a certain length." Bell Atlantic asserts that, if a competitive carrier desires ADSL- or HDSL-level transmission over loops exceeding these lengths, loop "conditioning" may be required. Bell Atlantic's tariff regarding these services also includes a variety of "ancillary" charges, all but one of which are non-recurring charges. 251. Bell Atlantic's ancillary charges generally fall into one of two categories: 1) charges related to loop qualification, or 2) charges related to conditioning unqualified loops. In the first category of ancillary charges, Bell Atlantic operates a loop qualification database, which competitors must access to find necessary information about the loop they wish to use. Bell Atlantic imposes a "Mechanized Loop Qualification Charge" to recover the costs associated with the creation and maintenance of this database. If a loop is not included in the loop database, or if a competitive provider requires additional information about a loop, a manual loop qualification occurs, and additional charges may accrue. 252. In the second category of ancillary charges, Bell Atlantic charges competing carriers to remove load coils and bridge taps from its ADSL- and HDSL-qualified loops. Bell Atlantic asserts that load coils make loops generally unsuitable for xDSL transmission. Therefore, it charges these carriers to remove these load coils, as well as some bridge taps. Bell Atlantic asserts that, because the number of load coils on a loop depends on its length, its charge to remove load coils on loops longer than 18,000 feet is loop-length-sensitive. Bell Atlantic does not charge for the removal of load coils on loops of less than 18,000 feet. On loops of less than 18,000 feet, Bell Atlantic will not charge to remove bridge taps between 12,000 and 18,000 feet in order to accommodate xDSL technology. Bell Atlantic will remove these shorter bridge taps on its shorter loops, but will charge competing providers for this service. 253. Bell Atlantic asserts that its proposed rates for these ancillary services are "equal to their costs" and are forward-looking because they reflect the most efficient technology currently available for the services requested. Bell Atlantic also asserts that the charges for these ancillary services, most of which are non-recurring charges, are essentially determined as the product of an estimated worktime and a relevant labor rate. 254. In the Local Competition First Report and Order, the Commission found that, in some instances, incumbent LECs would be required to "take affirmative steps to condition existing loop facilities" to enable competitors to provide services not currently provided over the facilities, such as xDSL. The Commission stated that "such loop conditioning may involve removing load coils or bridge taps that interfere with the transmission of digital signals," and that the carrier requesting the loop conditioning would be required to "bear the cost of compensating the incumbent LECs for such conditioning." Pursuant to Commission rules, "nonrecurring charges . . . shall not permit an incumbent LEC to recover more than the total forward-looking economic cost of providing the applicable element." The costs incumbents impose on competitors for line conditioning, which are nonrecurring charges, must be in compliance with these pricing rules. 255. A number of carriers assert that Bell Atlantic does not demonstrate that its proposed prices for its xDSL-capable loops comport with TELRIC. These carriers assert that Bell Atlantic's xDSL loop provisioning policies are discriminatory, unjust, and unreasonable because they fail to give an efficient competitor a meaningful opportunity to compete. ALTS contends that Bell Atlantic's charge for loop qualification fails to comply with the TELRIC standard. 256. Bell Atlantic urges us to refrain from evaluating Bell Atlantic's xDSL charges because its xDSL rates, which are interim and subject to refund, are still being reviewed by the New York Commission, and "there is no warrant for additional review here." In its evaluation of Bell Atlantic's section 271 application, the New York Commission notes that it is currently considering the issue of permanent rates pertaining to recurring and nonrecurring charges related to xDSL-capable loops, including conditioning and database charges. Noting that commenters have asserted that such charges may be so high that they are prohibitive, the New York Commission stated that a separate, accelerated track is underway to address these issues in its network element rate proceeding. Additionally, the New York Commission asserts that, in the interim, both recurring and non-recurring xDSL charges proposed by Bell Atlantic are temporary and subject to refund or true-up. In its reply brief, the New York Commission states that, consistent with its commitment to TELRIC principles and "to setting prices that satisfy the requirements of the 1996 Act and the Commission, we can safely say that [xDSL] rates meeting those requirements will have been set before the end of the year." Bell Atlantic contends that any concerns regarding its xDSL rates "will be resolved by the New York Public Service Commission in accordance with TELRIC standards in less than two months." 257. We note that Bell Atlantic currently has interim rates in effect for its conditioning of xDSL-capable loops, pending completion by the New York Commission of its xDSL rate case. The Commission has not previously addressed the question of whether a section 271 applicant's reliance on interim rates should constitute grounds for rejection. 258. Although we recognize that interim rates create uncertainty, we are also aware that establishing permanent recurring and nonrecurring rates relating to unbundled network elements, resale, and transport and termination offerings is a complex and ongoing process. It was for that reason in the Local Competition First Report and Order that the Commission proposed interim proxy rates that states could use until they completed their permanent cost proceedings. We conclude that a BOC's application for in-region interLATA authority should not be rejected solely because permanent rates may not yet have been established for each and every element or nonrecurring cost of provisioning an element. We believe that this question should be addressed on a case-by-case basis. If the uncertainty caused by the use of interim rates can be minimized, then it may be appropriate, at least for the time being, to approve an application based on the interim rates contained in the relevant tariff. Uncertainty will be minimized if the interim rates are for a few isolated ancillary items, permanent rates that have been established are in compliance with our rules, and the state has made reasonable efforts to set interim rates in accordance with the Act and the Commission's rules. 259. We accept Bell Atlantic's proposal that we allow its interim rates until the New York Commission reviews its cost support and, if necessary, adjusts its rates to conform to a TELRIC-based cost methodology. The conditioning of xDSL loops is a relatively new issue, and because new issues are constantly arising, we believe that it is reasonable to allow a limited use of interim rates when reviewing a section 271 application where the state has not yet completed its permanent rate case for a new service. Additionally, the New York Commission, as discussed above, has a substantial track record of setting other applicable prices at TELRIC rates. Bell Atlantic's interim rates are subject to refund or true-up if the New York Commission determines that they exceed applicable TELRIC-based costs. Additionally, the Commission has clearly stated that incumbent LECs, if required to condition loops, may recover their costs of such conditioning. If any of these factors were absent, however, we would not be inclined to approve a section 271 application that contains interim rates because we would lack confidence that the permanent rates would be set in accordance with the Act. 260. Finally, although we would be willing, at this time, to grant a section 271 application with a limited number of interim rates where the confidence-building factors identified above are present, we emphasize that it is clearly preferable to analyze a section 271 application on the basis of rates derived from a permanent rate proceeding. At some point, states will have had sufficient time to complete these proceedings. We will, therefore, become more reluctant to continue approving section 271 applications containing interim rates. It would not be sound policy for interim rates to become a substitute for completing these significant proceedings. 261. In the instant case, Bell Atlantic is only charging for removal of load coils and bridge taps that impede xDSL service but are otherwise appropriate for providing voice-grade service. In these circumstances, the cost of removing load coils and bridge taps can only be done on a loop-by-loop basis and may be expensive. We are not in a position to judge whether Bell Atlantic's interim rates are too high until the New York Commission has completed its review. Given the limited scope of Bell Atlantic's interim rates, the refund mechanism and the New York Commission's track record in reviewing Bell Atlantic's rates, we find that Bell Atlantic's interim rates for xDSL-capable loops meet the checklist requirement at this time. We note, however, that any significant time delay in permanent rates could be a basis for finding noncompliance with section 271 requirements. 262. Glue Charges. We also reject Cable & Wireless' assertion that Bell Atlantic acts in a discriminatory fashion by imposing an additional "glue charge" on business customers when it sells them unbundled network elements. Cable & Wireless contends that this charge is unlawful and will hinder the development of broad-based local competition. The New York Commission has defined "glue charges" as "charges that competitors will pay Bell Atlantic (in some cases) to compensate it for combining together all of the network elements into the `platform.'" In its state UNE tariff revision with an effective date of February 15, 1999, Bell Atlantic proposed a "glue charge," which it stated would apply "to each Existing and New UNE Platform used to provide business POTS service." The New York Commission approved this glue charge. In a tariff revision that took effect September 24, 1999, however, Bell Atlantic removed the glue charges. As a general rule, we are skeptical of glue charges, and note with approval that these glue charges were removed from Bell Atlantic's tariff before Bell Atlantic filed its section 271 application. Thus, the issue of glue charges is moot, and we need not further consider it here. C. Checklist Item 3 - Poles, Ducts, Conduits, and Rights-of-Way 1. Background 263. Section 271(c)(2)(B)(iii) requires BOCs to provide "[n]ondiscriminatory access to the poles, ducts, conduits, and rights-of-way owned or controlled by the [BOC] at just and reasonable rates in accordance with the requirements of section 224." In the Local Competition First Report and Order, the Commission interpreted section 251(b)(4) as requiring nondiscriminatory access to LEC poles, ducts, conduits, and rights-of-way for competing providers of telecommunications services in accordance with the requirements of section 224. In addition, we interpreted the revised requirements of section 224 governing rates, terms, and conditions for telecommunications carriers' attachments to utility poles in the Pole Attachment Telecommunications Rate Order. Section 224(f)(1) states that "[a] utility shall provide a cable television system or any telecommunications carrier with nondiscriminatory access to any pole, duct, conduit, or right-of-way owned or controlled by it." Notwithstanding this requirement, section 224(f)(2) permits a utility providing electric service to deny access to its poles, ducts, conduits, and rights-of-way, on a nondiscriminatory basis, "where there is insufficient capacity and for reasons of safety, reliability and generally applicable engineering purposes." 264. Section 224 also contains two separate provisions governing the maximum rates that a utility may charge for "pole attachments." Section 224(b)(1) states that the Commission shall regulate the rates, terms, and conditions governing pole attachments to ensure that they are "just and reasonable." Notwithstanding this general grant of authority, section 224(c)(1) states that "[n]othing in [section 224] shall be construed to apply to, or to give the Commission jurisdiction with respect to the rates, terms, and conditions, or access to poles, ducts, conduits and rights-of-way as provided in [section 224(f)], for pole attachments in any case where such matters are regulated by a State." As of 1992, nineteen states, including New York, had certified to the Commission that they regulated the rates, terms, and conditions for pole attachments. 2. Discussion 265. Based on the evidence in the record, we find that Bell Atlantic demonstrates that it is providing nondiscriminatory access to its poles, ducts, conduits, and rights-of-way at just and reasonable rates, terms, and conditions in accordance with the requirements of section 224, and thus satisfies the requirements of checklist item 3. The New York Commission concludes that Bell Atlantic provides nondiscriminatory access to poles, ducts, conduits and rights-of-way in compliance with this checklist item. 266. Although ALTS argues that Bell Atlantic does not provide nondiscriminatory access to conduits, and rights-of-way within multiple tenant environments, Bell Atlantic responds that it does not control the conduits and rights-of-way within the multiple tenant environments cited by ALTS. Section 271(c)(2)(B)(iii) is limited to the requirements set forth in section 224 and thus does not require the incumbent LEC to provide access to wiring it does not control inside buildings. Given that ALTS does not cite specific instances where Bell Atlantic has denied access to any conduits or rights-of-way that it does own or control within multiple tenant environments, we do not find sufficient evidence in the record to refute Bell Atlantic's assertion. 267. RCN raises concerns regarding access to conduits and ducts provided by Bell Atlantic's wholly owned subsidiary Empire City Subway. RCN does not argue, however, that Empire City Subway is not providing competitive LECs with equivalent access to conduits, but instead argues that any delay in accessing conduits is more detrimental to competitors than to Bell Atlantic. Because RCN does not assert that Bell Atlantic is providing access to conduits in a discriminatory manner, we have no basis for finding noncompliance with this checklist item. We note that no other commenter challenges Bell Atlantic's compliance with this checklist item. D. Checklist Item 4-Unbundled Local Loops 1. Background 268. Section 271(c)(2)(B)(iv) of the Act, item 4 of the competitive checklist, requires that Bell Atlantic provide "[l]ocal loop transmission from the central office to the customer's premises, unbundled from local switching or other services." The Commission has defined the loop as "a transmission facility between a distribution frame, or its equivalent, in an incumbent LEC central office, and the network interface device at the customer premises." This definition includes different types of loops, including "two-wire and four-wire analog voice-grade loops, and two-wire and four-wire loops that are conditioned to transmit the digital signals needed to provide services such as ISDN, ADSL, HDSL, and DS1-level signals." 269. In order to establish that it is "providing" unbundled local loops in compliance with section 271(c)(2)(B)(iv), Bell Atlantic must demonstrate that it has a concrete and specific legal obligation to furnish loops and that it is currently doing so in the quantities that competitors reasonably demand and at an acceptable level of quality. Bell Atlantic must also demonstrate that it provides nondiscriminatory access to unbundled loops. In previous section 271 orders, the Commission has generally indicated that the ordering and provisioning of network elements has no retail analogue, and we therefore look to whether the BOC's performance offers an efficient competitor a meaningful opportunity to compete. 270. As the Commission stated in the Second BellSouth Louisiana Order, one way that a BOC can demonstrate compliance with checklist item 4 is to submit performance data evidencing the time interval for providing unbundled loops and whether due dates are met. As described in the discussion of checklist item 2, competing carriers must also have nondiscriminatory access to the various functions of Bell Atlantic's OSS in order to obtain unbundled loops in a timely and efficient manner. Thus, we look to performance data measuring whether competing carriers are informed of the status of their order and how responsive the BOC is in providing access to necessary support functions, including maintenance and repair. 271. Bell Atlantic must also provide access to any functionality of the loop requested by a competing carrier unless it is not technically feasible to condition the loop facility to support the particular functionality requested. In order to provide the requested loop functionality, such as the ability to deliver ISDN or xDSL services, the BOC may be required to take affirmative steps to condition existing loop facilities to enable competing carriers to provide services not currently provided over the facilities, with the competing carrier bearing the cost of such conditioning. The BOC must provide competitors with access to unbundled loops regardless of whether the BOC uses integrated digital loop carrier (IDLC) technology or similar remote concentration devices for the particular loop sought by the competitor. Again, the costs associated with providing access to such facilities may be recovered from competing carriers. 272. As part of allowing a competitor to combine its own facilities with an incumbent LEC's loops, a BOC must provide cross-connect facilities between an unbundled loop and a competing carrier's collocated equipment at prices consistent with section 252(d)(1) and on terms and conditions that are reasonable and nondiscriminatory under section 251(c)(3). Incumbent LECs must also provide access to unbundled network interface devices so that requesting carriers can connect their own loop facilities at that point. 2. Discussion 273. We conclude that Bell Atlantic demonstrates that it provides unbundled local loops in accordance with the requirements of section 271. As detailed below, Bell Atlantic demonstrates that it has a concrete and specific legal obligation to provide unbundled local loops to competing carriers in accordance with these requirements. In addition, Bell Atlantic provides sufficient evidence that it provides unbundled local loop transmission, for the provision of both traditional voice services and various advanced services, in a nondiscriminatory manner. 274. In reaching these conclusions, we acknowledge that we differ from the evaluation of the Department of Justice in certain material respects. Although we have accorded substantial weight to the Department's views as required by section 271, the statute prohibits us from giving the Department's views preclusive weight. With respect to Bell Atlantic's provision of unbundled loops, we reach conclusions that vary from those of the Department in instances where we assess the totality of the evidence differently or where we take an analytical approach distinct from that taken by the Department. 275. Bell Atlantic makes local loop transmission available on an unbundled basis in compliance with the 1996 Act through its NYPSC No. 916 Tariff and through various interconnection agreements. Specifically, Bell Atlantic provisions a full range of unbundled loops, including analog and digital 2-wire and 4-wire loops, that competing carriers can use to offer a full range of services such as ISDN, ADSL, HDSL, 1.544 Mbps digital (DS1) transmission, and 45 Mbps digital (DS3) transmission. Bell Atlantic provides access to stand- alone loops through cross-connects that run from the Bell Atlantic distribution frame to competing carriers' collocation space. 276. Bell Atlantic provisions these unbundled local loops to competing carriers in three distinct forms. First, when Bell Atlantic does not presently serve the customer on the lines in question, a competing carrier may obtain a "new" loop from Bell Atlantic. In this case, the customer would be provided service on the second line from a competitive carrier and not from Bell Atlantic, while retaining Bell Atlantic as the provider on the original line. Second, Bell Atlantic also provisions stand-alone loops to competing carriers through coordinated conversions of active loops to the carriers' collocation space. These coordinated loop cutovers, or "hot cuts," make it possible to transfer an active Bell Atlantic customer's service to a competing carrier. For both new loops and conversions of existing customers, when loops are provisioned on a stand- alone basis, the competing carrier obtains only the transmission facility between Bell Atlantic's central office and the customer's premises. Third, Bell Atlantic provisions loops as part of a platform of network elements. When Bell Atlantic provisions a loop as part of a platform, the competitor receives the local loop, shared transport, and switching capability. 277. Through September 1999, Bell Atlantic has provisioned to competing carriers 200,000 loops, including approximately 50,000 stand-alone loops and 150,000 loops provided as part of platforms of network elements. Nearly 150,000 of these loops, including approximately 15,000 stand-alone loops and 130,000 platform loops, were delivered to competing carriers during the period from May through September, 1999. Bell Atlantic represents that it can easily meet the current commercial demand for unbundled local loops and that it will, as needed, add personnel and resources to meet any further increases in commercial demand. Additionally, through September 1999, Bell Atlantic has provisioned to competing carriers more than 3,300 premium digital loops, which may be appropriate for the provision of advanced services, and approximately 1,100 xDSL-specific loops, which are specifically designed for the provision of advanced services. 278. To demonstrate that it provides unbundled loops in compliance with its checklist obligations, Bell Atlantic submitted performance data for various metrics relating to loop provisioning, including data on the length of provisioning intervals, missed appointment rates, "on-time" hot cut performance, and new loop and hot cut installation troubles. In addition, Bell Atlantic submitted performance data addressing both voice-grade loops and loops capable of transmitting the digital signals necessary to support high-speed data services. In view of the variety of these measures, we conclude that our analysis of this checklist item cannot focus on Bell Atlantic's performance with respect to any single metric or any single type of loop. Rather, we examine the performance data for all of the various loop metrics, as well as the factors surrounding those metrics, in order to obtain a comprehensive picture of whether Bell Atlantic is providing unbundled local loops in accordance with the requirements of checklist item 4. 279. As noted above, in the past we have evaluated whether a BOC is meeting its nondiscrimination obligation with respect to loops by examining whether loops are provided in a fashion that provides an efficient competitor a meaningful opportunity to compete. In this application, however, we note that the New York Commission adopted a retail analogue for new unbundled loops, and Bell Atlantic submitted accompanying data with which we can conduct a direct parity comparison. Because this retail analogue was developed as a result of the rigorous collaborative process described above, we find this means of comparison to be reasonable in this instance. We therefore conclude that Bell Atlantic must satisfy its duty of nondiscrimination by demonstrating that it provisions new unbundled local loops to competing carriers in substantially the same time and manner as it does to its retail customers. Because the New York Commission did not identify a retail analogue to the coordinated cutover of an active loop, i.e., a "hot cut," however, we will examine Bell Atlantic's provision of hot cuts in terms of whether its performance affords competitors a meaningful opportunity to compete. We also discuss separately Bell Atlantic's evidence regarding its performance with respect to xDSL loops, describing how we will consider such evidence in evaluating future applications filed under section 271. a. Provisioning of Unbundled Local Loops 280. We conclude that Bell Atlantic presented sufficient evidence to demonstrate that it provisions loops in the quantities that competitors reasonably demand, at an acceptable level of quality, and within a reasonable timeframe. With respect to unbundled loops provisioned both on a stand-alone basis and as part of a network platform, we find that Bell Atlantic demonstrates that it provides new unbundled local loops to competing carriers in substantially the same time and manner as it provides new loops to its retail customers. 281. Stand-Alone Loops. We find that Bell Atlantic demonstrates that it provides new stand-alone loops to competing carriers in a nondiscriminatory manner. Specifically, as discussed below, we conclude that Bell Atlantic's processes for offering and meeting confirmed appointment dates for installing new loops to competing carriers are substantially the same as the processes for offering and meeting Bell Atlantic retail appointments. Additionally, we find that the new, stand-alone loops Bell Atlantic provisions to competing carriers are of the same quality as the loops it provides to its retail customers. 282. First, we conclude that Bell Atlantic's systems afford competing carriers access to appointment dates that is equivalent to the access provided to Bell Atlantic representatives serving retail customers. Orders for new loops are referred to as "dispatch" orders because they require that a technician be dispatched to the customer's premises in order to complete the installation. With respect to these orders, competing carriers have access to the same "SMARTS" clock, which sets available dispatch loop appointments through an automated system, as do Bell Atlantic retail representatives. Accordingly, competing carriers and Bell Atlantic customer representatives have equivalent access to loop installation appointments. 283. We similarly conclude that Bell Atlantic's process for meeting confirmed appointment dates is nondiscriminatory. Specifically, we find that Bell Atlantic meets the confirmed due dates of the customers of competitive carriers in the same time and manner as it meets the confirmed due dates of its retail customers. Performance data indicate that Bell Atlantic is completing loop installations within the interval requested by competitors. Indeed, the Carrier-to-Carrier performance measures evidence consistently lower missed appointment rates for the customers of competing carriers than for Bell Atlantic customers. In June 1999, Bell Atlantic missed approximately 2 percent of new loop installation appointments for competing carriers and 9 percent of appointments for Bell Atlantic retail customers. In addition, for the period from July through September 1999, Bell Atlantic missed less than one percent of installation appointments for new loops provisioned to competing carriers. By contrast, during the same period, Bell Atlantic missed between 10 and 15 percent of new loop installation appointments for its retail customers. As these performance data demonstrate, Bell Atlantic provisions new loops to competing carriers on a more reliable basis than it does for its own customers. We find that this level of performance demonstrates that Bell Atlantic is provisioning new loops to competitors on a timely basis in accordance with the requirements of checklist item 4. 284. In addition, we conclude that Bell Atlantic is provisioning unbundled loops, both on a stand-alone basis and as part of a platform of network elements, to competing carriers at an acceptable level of quality. Bell Atlantic's performance data indicate that from June through September 1999, less than 2 percent of the new loops provisioned to competing carriers were the subject of a trouble report within 7 days of installation, whereas approximately 3 percent of Bell Atlantic retail customers reported loop troubles within the same period. Similarly, from June through September, competing carriers reported far less loop troubles within 30 days of installation than did Bell Atlantic retail customers. We find this to be substantial evidence that Bell Atlantic is provisioning new loops to competing carriers that are equivalent in quality to those it provisions to its retail customers. Furthermore, the record lacks evidence of conflicting data, nor do competing carriers raise serious disputes regarding the quality of the new voice-grade loops provisioned by Bell Atlantic. 285. In concluding that Bell Atlantic provides nondiscriminatory access to new unbundled loops, we note that, although data related to average installation intervals remain important in our framework for evaluating section 271 applications, in this instance Bell Atlantic provided information that convinces us that other factors more accurately reflect its compliance with this checklist item. Accordingly, under these facts, we accord little weight to data evidencing the average intervals in which loop installations are completed. The record contains performance data evidencing that, on average, competing carriers experience longer average loop installation intervals than do Bell Atlantic retail customers. These differences are statistically significant under the framework adopted by the New York Commission. As detailed below, however, we conclude that Bell Atlantic presented sufficient evidence to demonstrate that the disparity between wholesale and retail average installation intervals is not the result of discriminatory conduct, but rather is the result of factors outside of its control. 286. First, we find that Bell Atlantic demonstrates that competitive carriers frequently request later due dates than those offered by Bell Atlantic's automatic appointment clock. If competing carriers request later due dates for loop installations more often than Bell Atlantic customers, then installation intervals for those competing carriers will be, on average, longer than those for Bell Atlantic customers. Although Bell Atlantic relies upon competing carriers to specifically "code" orders that include requests for longer-than-average provisioning intervals so they can be excluded from the installation interval measures, a statistical study submitted by Bell Atlantic establishes that competing carriers "miscode" a significant percentage of non-dispatch orders, causing those requests to be erroneously included in the performance data. Although the statistical analysis does not address dispatched orders, such as orders for new unbundled loops, we agree with Bell Atlantic that it is likely that competing carriers similarly miscode dispatched orders for which an appointment date after the first available date is sought, which would result in longer requested and actual provisioning intervals. Indeed, AT&T states that it typically requests 5 days for non-dispatch orders with standard intervals of 2 days, and we find it likely that it similarly requests longer intervals for dispatch orders. Additionally, with the exception of AT&T, commenters have not taken serious issue with Bell Atlantic's provisioning of new, stand-alone unbundled loops. 287. We are also persuaded by Bell Atlantic's argument that competing carriers experience longer completion intervals than its retail customers because the automatic appointment clock used to schedule available appointments contains longer average appointment intervals in some geographic areas than in others. As a result, reported average installation intervals will vary depending upon where competitive carriers are ordering service. Average completion intervals for unbundled loops provisioned to competing carriers would be longer if a high proportion of those competing carriers provide service to geographic areas with busy service centers. This factor, however, is not accounted for in the performance data measuring average loop installation intervals. No commenter disputes that this factor affects average completion intervals, and we are persuaded by Bell Atlantic's arguments that it does. 288. In view of these factors, which are outside of Bell Atlantic's control and which can cause distortion to the average installation intervals, we find unpersuasive the claims of competing carriers that the average completion intervals on their face demonstrate that Bell Atlantic provisions new loops in a discriminatory manner, citing the Commission's previous statements that average installation intervals are a "critical measure of parity." Although we continue to believe that average installation intervals are important in determining whether loops are being provided in a nondiscriminatory manner, we look to other available data as well. Where, as here, the BOC makes a reasonable showing that the evidence on average installation intervals is distorted by other factors, it is reasonable to accord more weight to this other evidence and less weight to average installation intervals. Here, we find the missed rate of installation appointments to be the most accurate indicator of Bell Atlantic's ability to provision unbundled loops. In this regard, as discussed above, Bell Atlantic's performance in meeting loop installation appointments demonstrates that it is providing new loops to competing carriers within the intervals they are requesting. Accordingly, we conclude that Bell Atlantic demonstrates that it is providing new, stand-alone loops to competing carriers in a timely manner. 289. We similarly conclude that the same analysis is applicable to Bell Atlantic's provisioning of high capacity loops. As with standard, voice-grade loops, the average completion interval for the installation of DS1 loops ordered by competing carriers is longer than the completion interval experienced by Bell Atlantic retail customers. Bell Atlantic demonstrates, however, that it misses fewer appointments for installations of high capacity loops to competing carriers than it does for its retail customers. Further, although commenters allege that Bell Atlantic is unable to provision high capacity loops such as DS1s in a timely manner, none of these claims is documented with specific evidence or contained in a sworn affidavit. Accordingly, we conclude that Bell Atlantic is meeting its installation due dates for high capacity loops provided to competitors on a more reliable basis than it does for loops provided to its own customers and therefore establishes that it provisions these loops in accordance with its checklist obligations. 290. Loops Provisioned as Part of a Platform. We similarly find, based on the evidence in the record, that Bell Atlantic demonstrates that it is providing unbundled loops in combination with other network elements in a nondiscriminatory manner. As detailed above in our discussion of checklist item 2, Bell Atlantic establishes that it provisions platforms of network elements, including unbundled loops, within the intervals in which they are requested and that it misses fewer competing carriers' due dates for platforms of network elements than it does for its retail customers. Further, as discussed above, Bell Atlantic demonstrates that it provisions unbundled loops as part of platforms of network elements that are of substantially the same quality as the loops provisioned to its own customers. We therefore conclude that Bell Atlantic demonstrates that it is provisioning unbundled loops as part of platforms in a nondiscriminatory manner. b. Hot Cuts 291. We further conclude that Bell Atlantic demonstrates that it is provisioning unbundled loops through the use of coordinated conversions of active customers from Bell Atlantic to competing carriers, a process known as "hot cuts," in accordance with the requirements of checklist item 4. Because there is no retail equivalent to a hot cut, Bell Atlantic must demonstrate that it provides unbundled loops through hot cuts "in a manner that offers an efficient competitor a meaningful opportunity to compete." As detailed below, we conclude that Bell Atlantic demonstrates that it provisions hot cuts in sufficient quantities, at an acceptable level of quality, and with a minimum of service disruption, thereby offering competitors a meaningful opportunity to compete in the local exchange market. 292. On-Time Hot Cut Performance. Under the performance standards developed by the New York Commission, with input from Bell Atlantic and several competitive carriers, hot cut performance is measured according to the percent of coordinated conversions completed within a specified time window. The window, which establishes the time within which the entire hot cut must be completed, is a fixed period of time ranging from one hour to eight hours, depending upon the number of lines involved. For orders with fewer than ten lines, Bell Atlantic has one hour in which to complete the coordinated cutover and report the completion of the hot cut to the competing carrier. Because there is no retail analogue for a hot cut, the New York Commission adopted a benchmark performance metric to measure Bell Atlantic's on-time hot cut performance. In order to meet the New York Commission's adopted standard, Bell Atlantic must provision 95 percent of hot cuts within the window applicable to the particular order. The New York Commission also established a secondary on-time hot cut target of 90 percent for inclusion in the Performance Assurance Plan. 293. In its application, Bell Atlantic asserts that it completed 94 percent of hot cuts on- time in August and July 1999. The record also indicates that Bell Atlantic reported 94 percent on-time hot cut performance for September 1999. These figures, which are self-reported by Bell Atlantic, have been vigorously disputed by several competing carriers in the New York section 271 proceeding. In particular, AT&T submitted affidavits and its own performance data that challenged Bell Atlantic's on-time hot cut performance and raised serious concerns regarding the actual marketplace provisioning of hot cut loops. AT&T also argued that many of the hot cuts provisioned by Bell Atlantic resulted in non-functioning loops and extended service disruptions for its customers. 294. In response to these challenges to Bell Atlantic's data, the New York Commission conducted a reconciliation of the conflicting data. New York Commission staff reviewed all AT&T hot cut orders for both July and August. With respect to July, for which Bell Atlantic had reported 94 percent on-time performance, AT&T submitted data indicating that Bell Atlantic completed only 76 percent of its ordered hot cuts within the established window. After reviewing the disputed data and its supporting documentation, New York Commission staff concluded that Bell Atlantic had completed 88 percent of AT&T's orders on-time in July and 90.55 percent of AT&T's orders on-time in August. The staff then adjusted Bell Atlantic's self- reported performance to reflect the revised AT&T-specific data. The staff thus factored into the 94 percent July and August figures those AT&T orders that Bell Atlantic had reported as "on- time," but that staff determined through the reconciliation to have been provisioned outside the established window. This process resulted in the New York Commission staff's conclusion that Bell Atlantic's on-time hot cut performance for all competing carriers was 90.79 percent for July and 91.54 percent for August. 295. We find the most reliable evidence of Bell Atlantic's on-time hot cut performance for July and August 1999 to be the figures that resulted from the New York Commission staff's reconciliation of coordinated loop cutovers provisioned to AT&T. The staff did not conduct a review of non-AT&T orders during this period, however, and we therefore recognize that the staff's calculations of overall hot cut performance could, in fact, include missed or late hot cuts that were reported inaccurately as being on-time. Indeed, the Department of Justice notes that the New York Commission's estimate that 90.79 percent of all hot cuts in July were provisioned on-time would be accurate only if Bell Atlantic had reported every non-AT&T order correctly. With the exception of AT&T, however, no competing carrier submitted data directly challenging Bell Atlantic's self-reported performance. Rather, the allegations of competing carriers are conclusory and anecdotal, and none is included in a sworn affidavit. We therefore do not accord them a great deal of probative value and instead are persuaded by and give significant weight to the New York Commission staff's exhaustive review of Bell Atlantic's hot cut performance. While criticizing the New York Commission's conclusion that hot cuts are performed on-time roughly 90 percent of the time, the Department of Justice undertook no analysis to proffer an alternative figure in the record. 296. Although we could arrive at a different conclusion if presented with another set of facts, we find that the record in this proceeding provides a reasonable basis for us to conclude that, at a minimum, Bell Atlantic performed hot cuts within the prescribed time interval at least 88 percent of the time in July and 90 percent of the time in August, and Bell Atlantic's performance may have been closer to 90.79 percent and 91.54 percent in July and August, as the New York Commission found. There is also evidence in the record that Bell Atlantic performed hot cuts on-time 94 percent of the time in September 1999. Furthermore, Bell Atlantic provided this level of on-time performance each month in the face of increasing volumes. Moreover, in addition to maintaining this level of on-time performance, as detailed below, Bell Atlantic provisioned quality loops through hot cuts with a minimum of service disruption. We underscore, however, that the weight we accord to conflicting pieces of evidence here flows directly from our assessment of the probative value of each of those pieces of evidence. As such, we note that we could arrive at a different weighting if presented with another set of facts and circumstances. 297. The Department of Justice cites the failure to complete approximately 10 percent of hot cuts within the prescribed window as one of four problems that, collectively, evidence the need for Bell Atlantic to improve its hot cut performance. In addition to the level of on-time performance, the Department takes issue with Bell Atlantic's ability to return timely confirmations and rejections of hot cut orders, to return accurate order confirmations, and to ensure that customers' directory listings are not dropped during the provision of a hot cut. The Department of Justice, however, did not conclude that on-time hot cut performance of 90 percent, either alone or in combination with other factors, evidences Bell Atlantic's failure to comply with this checklist item. Although it found that the collective weight of these deficiencies imposes constraints upon competition, the Department did not specify in what manner and to what extent the New York local exchange market is affected adversely by these problems. Nor did the Department provide any indication as to what level of hot cut performance or what types of improvements Bell Atlantic should be required to demonstrate in order to satisfy section 271. 298. As discussed in our analyses of checklist items 2 and 7, we do not consider the factors identified by the Department of Justice, either alone or in combination, to have significant effects upon Bell Atlantic's overall hot cut loop performance. Thus, after careful consideration of the evaluations of the Department of Justice and the New York Commission, as well as the comments of competing carriers, we conclude that Bell Atlantic's demonstrated level of on-time hot cut performance is sufficient to offer efficient competitors a meaningful opportunity to compete. Although we recognize that this performance falls slightly short of the New York Commission's adopted standard, we make the independent judgment that on-time hot cut performance at a level of 90 percent or greater is sufficient to permit carriers to enter and compete in a meaningful way in the New York local exchange market. We conclude based upon the record before us that Bell Atlantic establishes that it attained this level of performance in August and September 1999. Furthermore, we are confident that the penalties attached to this performance measure in the New York Performance Assurance Plan are sufficient to ensure that Bell Atlantic maintains at least this 90 percent level of on-time performance, while also providing incentives to improve performance above this 90 percent level. We are prepared to take appropriate enforcement action in the event of a deterioration in Bell Atlantic's on-time performance below 90 percent. 299. Quality of Loops Provisioned Through Hot Cuts. We further conclude that Bell Atlantic demonstrates that it provisions hot cuts at a level of quality that offers competitors a meaningful opportunity to compete. The ability of a BOC to provision working, trouble-free loops through hot cuts is of critical importance in view of the substantial risk that a defective cut will result in end-user customers experiencing service disruptions that continue for more than a brief period. Upon review of the evidence in the record regarding hot cut installation quality, as well as service outages and disruptions, we conclude that Bell Atlantic provisions hot cuts to competitors in a manner sufficient to meet the requirements of the checklist. 300. Bell Atlantic submitted performance data that evidence extremely low rates of installation troubles reported on the lines provisioned through hot cuts. From July through September 1999, competitors reported installation troubles on less than two percent of the lines provisioned through hot cut loops. This level of performance is well below the two percent standard for hot cut installation troubles that was recently adopted by the New York Commission. 301. We find this evidence to be sufficient to overcome the claims of competing carriers that Bell Atlantic's hot cut provisioning results in a level of service disruptions that significantly affects their end-user customers and their ability to obtain and retain customers. Allegiance alleges that Bell Atlantic's hot cut provisioning results in outages for nearly 20 percent of its customers, although this claim is neither documented with specific facts nor contained in a sworn affidavit. AT&T makes the most serious challenge to the quality of Bell Atlantic's hot cut provisioning, asserting that between June 21 and August 31, 1999, Bell Atlantic provisioning errors placed nearly 12 percent of its customers out of service. 302. A comprehensive reconciliation of AT&T's outage data that was conducted by the New York Commission, however, largely refutes AT&T's allegations. The data reviewed by the New York Commission reveal that, in fact, less than 5 percent of the hot cuts that Bell Atlantic provisioned to AT&T between June 21 and August 31, 1999 resulted in end-user service outages as a result of a Bell Atlantic provisioning failure. The New York Commission further notes that many of the outages claimed by AT&T were not the result of Bell Atlantic failures and that many others had causes that could not be determined. Although the reconciliation demonstrates that approximately five percent of AT&T customers suffered service outages as a result of Bell Atlantic errors, we consider this to be sufficient for checklist compliance, particularly in view of the extremely low rates of installation troubles reported on the hot cut loops provisioned by Bell Atlantic. 303. Additionally, AT&T's reports of extended outages are called into question by Bell Atlantic's claims that AT&T fails to report installation troubles within a reasonable period of time. The New York Commission concluded that in many cases of service disruptions, "AT&T took longer to identify and report the problem to Bell Atlantic than Bell Atlantic took to fix it." In these circumstances, as the New York Commission notes, it is difficult to determine the cause for the duration of many service outages. Furthermore, performance data indicate that a percentage of Bell Atlantic's own customers suffer service disruptions at any given time. Based upon these factors, as well as the small percentage of AT&T service outages caused by Bell Atlantic and the lack of corroborating evidence of outages, we conclude that AT&T's claims of service disruptions are insufficient to overcome the performance data evidencing extremely low levels of installation troubles associated with the hot cut loops provisioned by Bell Atlantic. 304. Hot Cut Provisioning Process. We also dismiss claims by AT&T and other carriers that additional hot cut provisioning deficiencies, which are not reflected in performance data, impose significant costs and delays upon competing carriers and their customers, thereby impairing new entrants' ability to compete. After several parties in the New York proceeding challenged Bell Atlantic's hot cut performance and data, Bell Atlantic, the New York Commission, and several competing carriers collaborated to develop and adopt a standardized hot cut process that details operating methods and procedures to facilitate coordinated cutovers. In addition to identifying the steps to a hot cut, the procedure requires Bell Atlantic technicians to complete a checklist and report when each intermediate step has been completed. Although there are numerous steps in the hot cut process, the New York Commission and commenters identify four particular steps that have proven to be critical to on-time hot cut performance: the return of accurate order confirmations; the due date minus 2 days dial tone check; the due date minus one hour confirmation call from Bell Atlantic to the competing carrier; and the Bell Atlantic post-completion confirmation call. 305. Since the hot cut procedures have been in effect, competing carriers have continued to assert that Bell Atlantic fails to follow the agreed-to hot cut provisioning process. Compliance with the procedure's steps is currently not captured in any performance standard or measure, although competitors contend that Bell Atlantic's failure to comply with the process forces them to supplement and postpone many loop orders and to escalate problems throughout various levels of Bell Atlantic's wholesale organization, imposing costs and delays upon those carriers and their customers. AT&T asserts, for instance, that a high percentage of order confirmations received from Bell Atlantic are inaccurate, and that Bell Atlantic often fails to conduct the due date minus two days dial tone check and the due date minus one hour confirmation call. AT&T further states that it has devoted specific staff functions to escalating hot cut problems with Bell Atlantic and quantifies the resultant additional costs for each order. 306. By contrast, as Bell Atlantic argues, KPMG found that Bell Atlantic technicians followed the hot cut procedures 97 percent of the time. KPMG had previously taken exception with Bell Atlantic's ability to follow the established hot cut procedures, but, following a June 1999 two-week observation of hot cut provisioning, subsequently concluded that the problems had been resolved. Bell Atlantic indicates that it has undertaken extensive training of central office technicians and supervisors to ensure that the hot cut procedures are followed. As a result, the New York Commission confirms that hot cut checklists are completed by Bell Atlantic technicians for every order. 307. The Department of Justice notes that KPMG's observation of hot cut provisioning did not confirm whether Bell Atlantic performed any of the required steps prior to the due date, such as the due date minus two days dial tone check. Additionally, AT&T argues that Bell Atlantic's consistent failure to adhere to the hot cut procedures is evidenced by a letter from New York Commission staff in October 1999 stating that "[a]pplication of the due date minus 2 days check has not been rigorously adhered to at the operations level and it appears that technicians have been using different practices to effectuate coordination." Bell Atlantic responds that this statement refers to Bell Atlantic's practice of agreeing with competing carriers regarding the manner in which the dial tone check will be completed and is not an indication that Bell Atlantic is not following the hot cut procedures. Considering each of these factors, we conclude that the evidence weighs in favor of finding that Bell Atlantic adheres to the hot cut provisioning process. Bell Atlantic demonstrates, and KPMG and the New York Commission have confirmed, that the hot cut procedures are being followed, and we believe contrary allegations in the record are insufficient to refute this showing. Although we take seriously AT&T's claims regarding additional costs it incurs as a result of Bell Atlantic's hot cut provisioning failures, we nonetheless conclude that the record does not indicate that any alleged failure to comply with the procedures results in adverse hot cut provisioning that denies efficient competitors a meaningful opportunity to compete. Rather, Bell Atlantic's high rate of on-time hot cuts bolsters the evidence in the record that it is adhering to the hot cut procedures. 308. Additionally, although we concur with the Department of Justice's conclusion that the economic significance of competition through unbundled loops is greater than would be suggested by assessing the percentage of stand-alone unbundled loops currently being provisioned, we nonetheless conclude that Bell Atlantic demonstrates that it is capable of continuing its performance in provisioning quality hot cuts in a timely manner. In this regard, we further find that Bell Atlantic demonstrates that its ability to provision hot cuts is scalable such that the company can expand its capacity to perform hot cuts in response to increases in commercial demand. KPMG verified that Bell Atlantic's capacity to provision hot cuts is scalable, citing Bell Atlantic's intention to open a second service center for processing hot cut orders. Commenters argue that the hot cut provisioning problems and delays they are currently experiencing demonstrate that Bell Atlantic does not have the capacity to process increased commercial volumes. As discussed herein, however, we find that competing carriers' claims of provisioning deficiencies are insufficient to refute Bell Atlantic's demonstrated hot cut performance. Accordingly, we similarly find those claims to be insufficient to refute Bell Atlantic's showing that it is capable of expanding hot cut volumes to meet growing demand. Additionally, as discussed in our analysis of checklist item 2, we conclude that Bell Atlantic is providing nondiscriminatory access to its OSS ordering functions for unbundled network elements, including unbundled local loops, and is capable of processing large volumes of orders in a timely fashion. Thus, although we have accorded them substantial weight, we do not agree with the concerns raised by the Department of Justice regarding the effects of manual loop order processing upon Bell Atlantic's ability to process increased volumes of loop and hot cut orders. 309. Finally, we emphasize that although we consider Bell Atlantic's demonstrated on- time hot cut performance at rates at or above 90 percent, in combination with the evidence indicating that fewer than five percent of hot cuts resulted in service outages and that fewer than two percent of hot cut lines had reported installation troubles, to be sufficient to establish compliance with the competitive checklist, we view this as a minimally acceptable showing. We would thus have serious concerns if the level of performance in any one of these three measures were to decline and would be prepared, in that event, to take whatever enforcement action is warranted. We are especially concerned with hot cut performance because of the substantial risk that an untimely or defective cutover will result in an end-user customer's loss of service for more than a brief period, as well as the effect of such disruptions upon competitors. We also would be particularly concerned if there were any evidence that Bell Atlantic is competing in the marketplace in part by suggesting to consumers that there is a possibility of service disruptions when customers switch their service from Bell Atlantic to competing carriers. c. Maintenance and Repair of Unbundled Local Loops 310. We further conclude that Bell Atlantic demonstrates that it is providing maintenance and repair functions for unbundled local loops in substantially the same time and manner in which it provides those functions to its retail customers. Although Bell Atlantic does not perform some loop maintenance and repair functions for competitors as quickly as it performs them for Bell Atlantic retail customers, we do not consider these slight differences to be competitively significant. Rather, we find that Bell Atlantic provides nondiscriminatory maintenance and repair services for the unbundled loops it provides to competing carriers. 311. The New York Carrier-to-Carrier performance data demonstrate that Bell Atlantic performs maintenance and repair functions with respect to loops provisioned to competitors in substantially the same time and manner as it does with respect to loops provided to its retail customers. In July 1999, Bell Atlantic missed approximately 16 percent of loop repair appointments for competing carriers and 12 percent of repair appointments for its retail customers. In August, Bell Atlantic missed 14 percent of loop repair appointments for competitors and 10 percent for Bell Atlantic customers. Significantly, Bell Atlantic improved its performance substantially in September, missing approximately 12 percent of competitors' loop repair appointments and 11 percent of Bell Atlantic retail appointments. This demonstrates that Bell Atlantic is responding to competitors' trouble complaints in substantially the same manner as it responds to its own customers' complaints. 312. Additional data indicate that the average time to repair loops provisioned to competing carriers is comparable to the average time to repair loops provisioned to Bell Atlantic customers. In July, for instance, data demonstrate that repairs were made to loops provisioned to competitors in, on average, 28 hours and to loops provisioned to retail customers in, on average, 29 hours. Similarly, in August, repairs were made in an average of 26 hours for competitors and 25 hours for Bell Atlantic customers and in September, in 25 hours for competitors and 27 hours for Bell Atlantic customers. 313. We conclude that this level of performance demonstrates that Bell Atlantic is providing loop maintenance and repair functions in a nondiscriminatory manner. We do not consider the slight differences between the percentage of missed repair appointments to be indicative of discriminatory access to these functions, particularly in view of the improvements made by Bell Atlantic in September. Furthermore, data addressing the duration of loop maintenance and repair activities demonstrate that Bell Atlantic is repairing competitors' loop troubles in substantially the same time period as it is repairing its own customers' loops. We consider this to be persuasive evidence of nondiscriminatory access to loop maintenance and repair functions. 314. Furthermore, KPMG verified Bell Atlantic's performance in this regard through an extensive test of maintenance and repair services offered to both competing carriers and retail customers, as well as Bell Atlantic's ability to scale its maintenance and repair capabilities to meet future volumes and increased demand. Finally, Bell Atlantic demonstrates that it has addressed and resolved the situations in which it was not meeting performance standards governing maintenance and repair of unbundled loops. 315. Moreover, we do not find the concerns raised by commenters to be sufficient to overcome Bell Atlantic's evidence that it performs loop maintenance and repair functions in a nondiscriminatory manner. The few commenters that raise objections to Bell Atlantic's loop maintenance and repair performance do not raise specific allegations supported by documented facts. Rather, competing carriers claim generally that Bell Atlantic's performance of loop maintenance and repair functions are discriminatory. Accordingly, we find these allegations insufficient to rebut Bell Atlantic's showing that it provides access to loop maintenance and repair functions in a nondiscriminatory manner. d. xDSL-Capable Loops 316. Based upon its overall performance in providing unbundled access to local loops, we conclude that Bell Atlantic satisfies its obligations under item 4 of the competitive checklist. We note at the outset that our previous section 271 orders have not addressed the ordering or provisioning of xDSL-capable loops and that no previous applicant has made a separate showing on the provision of xDSL loops. Thus, although the obligation to provide access to unbundled loops capable of supporting xDSL technologies was adopted in 1996, we have not previously provided guidance to the BOCs as to the type and level of proof necessary in this area to establish compliance with section 271. 317. States are just now developing and adopting performance standards and measures for xDSL loop ordering and provisioning, and incumbent and competitive carriers themselves are in the process of defining the relevant criteria for adequate xDSL performance and developing operational provisioning procedures. The New York Commission did not begin to address xDSL- specific issues until August 1999. In response to early concerns raised by competing carriers in the New York section 271 proceeding regarding the timeliness and quality of Bell Atlantic's provisioning of xDSL loops, the New York Commission in August initiated a collaborative proceeding to address the issues raised by competitors. The collaborative proceeding is intended to focus on defining provisioning methods for xDSL loops to ensure the timely installation of functioning loops. In addition to conducting its xDSL collaborative proceeding, the New York Commission, in conjunction with Bell Atlantic and several competing carriers, is in the process of developing xDSL-specific performance standards and measures. The New York Commission expects to receive recommendations for xDSL-specific measures in December, in which case Bell Atlantic should begin officially reporting its performance to the New York Commission and competing carriers in January 2000. 318. Parties are thus actively working in New York to address issues associated with xDSL loops, and have already undertaken a number of process improvements. The New York xDSL collaborative is designed to improve communication among carriers and to develop agreed- upon common practices for xDSL loop provisioning. The New York Commission, for instance, instituted a process change to simplify xDSL central office cross-connections and is working to remedy customer contact problems that have led to a significant portion of installations in which Bell Atlantic cannot access the customers' premises. The collaborative proceeding is also addressing problems relating to the timing of loop installations by ensuring that carriers engage in close operational coordination so that loop installations are accurate and less likely to be the subject of trouble reports. 319. In addition, through the New York collaborative, Bell Atlantic and competing carriers have agreed to joint testing and provisioning procedures for xDSL loops. Provisioning xDSL loops to competitors involves processes that are more complex than those involved with the provision of a voice-grade loop. As a result, participants in the New York collaborative proceeding have agreed to a provisioning process for xDSL loops that involves collaborative testing between Bell Atlantic and the requesting carrier. The process, which has been in place since September 15, 1999, involves individual and joint testing of loops, sharing of test results, joint review of order status, and procedures for establishing a dialogue between Bell Atlantic and the requesting carrier on orders in jeopardy. These procedures ensure, for instance, that the parties test loops during the installation process and that competitors receive demarcation information at the time of installation. The New York Commission confirms that, where cooperative testing is conducted, xDSL loop installation problems are reduced. We are highly supportive of these initiatives and fully expect that the New York Commission will provide needed clarity in this area, both in terms of defining operational procedures and adopting performance standards. 320. In New York, competitors have been ordering xDSL-capable loops for a relatively limited period of time. According to Bell Atlantic, it provisioned 7 xDSL-specific loops in June, 56 xDSL-specific loops in July, 449 xDSL-specific loops in August, and 653 xDSL-specific loops in September. In addition, Bell Atlantic indicates that it provisioned more than 3,300 premium digital loops since January 1999, although not all of those loops have been used by competitors to provide xDSL services. Covad indicates that it submitted more than 2,300 orders for xDSL- capable loops in New York during the period from June through September 1999. Indeed, regardless of the data on which we rely, the record indicates that demand for xDSL-capable loops has grown dramatically in recent months. 321. Moreover, the xDSL-capable loops provisioned to competing carriers by Bell Atlantic to date represent only a small fraction of the entirety of unbundled loops provisioned in New York. Specifically, through September 1999, Bell Atlantic provisioned more than 50,000 unbundled, voice-grade loops in New York, compared to only 1,100 xDSL-specific loops. 322. This application presents unique factual circumstances with regard to xDSL loops in New York. Specifically, competitors have been ordering xDSL-capable loops in New York for a relatively short period of time; there has been a recent surge in demand; and xDSL-capable loops remain a small percentage of overall loop orders. Given these circumstances it is difficult to reach conclusive judgments about Bell Atlantic's provisioning performance for xDSL loops. We believe we could benefit from New York's input with regard to xDSL-capable loop provisioning but note that its review is still underway. In the absence of definitive state standards, we could look at Bell Atlantic's performance by examining whether the loops are delivered in a timely fashion and whether those loops actually are working. 323. In its application, Bell Atlantic submitted performance data that it asserts demonstrate that it provisions quality premium digital loops and xDSL-specific loops in a timely manner. Opponents of the application, however, heavily contest much of that data. The data submitted by Bell Atlantic indicate, for instance, that it missed between .70 percent and 4.60 percent of installation appointments for premium digital loops provisioned to competing carriers between January and September 1999. Bell Atlantic's data further indicate that it missed approximately 7 percent of xDSL-specific loop installation appointments for competitors in August 1999 and approximately 3 percent of xDSL-specific loop appointments in September 1999. By contrast, competitors contend that Bell Atlantic misses far more installation appointments. Covad, for instance, submits data indicating that between May and August, 1999, it received premium digital and xDSL-capable loops by the due date to which Bell Atlantic committed for only 29 percent of the loops it ordered. 324. Bell Atlantic also asserts that in August and September 1999, it provisioned xDSL loops in approximately 7 days, on average. Covad asserts that in its experience, the average interval for Bell Atlantic's installation of these loops has been approximately 40 days. Other competing advanced services providers argue that Bell Atlantic's performance data should be disregarded because the installation interval measure does not consider whether the loop installed by Bell Atlantic is functioning. 325. There are also sharp disparities in the record regarding the quality of Bell Atlantic's xDSL loop provisioning. Bell Atlantic reports, for instance, that during the first month since the September 15, 1999 implementation of joint installation and testing procedures, it received trouble reports on approximately three percent of the xDSL loops it installed. By contrast, Covad contends that only 39 percent of the loops it received in the first two weeks of the joint procedures were installed correctly. Similarly, NorthPoint argues that a substantial number of the xDSL loops provisioned by Bell Atlantic are defective or impaired. 326. The absence of a New York performance benchmark or Commission reconciliation of conflicting data claims makes it difficult for this Commission to decide between the competing statistics. A number of factors complicate our efforts to analyze the data. The record indicates, for instance, that Covad begins measuring its installation intervals on the date that it first sends an order for an xDSL loop to Bell Atlantic, whereas Bell Atlantic does not begin measuring the installation interval until it receives an error-free order from the requesting carrier. According to Bell Atlantic, twenty-five percent of Covad's orders have had two or more corrections associated with them, a result that could cause large disparities in installation intervals based solely upon the conflicting measurement techniques. With respect to the missed appointment data, Bell Atlantic contends, and competing carriers do not seriously dispute, that in many instances it is unable to gain access to the customers' premises to complete the installation and that many orders are cancelled by the customer when Bell Atlantic arrives to complete the installation. In such circumstances, Bell Atlantic does not score the appointment as having been missed, although it appears that at least some competing carriers do. We do not believe it appropriate to include legitimate "no access" situations in a measure of missed appointments. 327. We thus are faced with a situation in which competitors have been ordering xDSL- capable loops in New York for a relatively short period of time; there has been a recent surge in demand; and xDSL-capable loops remain a small percentage of loop orders. Although the ongoing New York proceeding is expected to resolve many key issues in the near future, the underlying performance data in this record are not reported in accordance with a common set of definitions and have not been validated by the New York Commission. Moreover, we have never before provided direction to the BOCs regarding the application of section 271 to the provision of xDSL loops. In light of these unique circumstances, we conclude that we should rely upon Bell Atlantic's overall showing of loop performance in evaluating whether Bell Atlantic has met its burden of demonstrating that it provides unbundled local loops in accordance with checklist item 4. 328. In reaching this conclusion, we take a different approach than the Department of Justice, which found that it could not conclude on the current record that Bell Atlantic demonstrates an acceptable level of performance in provisioning xDSL loops. Like this Commission, the Department had difficulty evaluating the evidence presented by Bell Atlantic in light of the contrary data submitted by competing carriers. The Department, however, concluded that the Commission should await completion of the New York Commission's ongoing xDSL collaborative proceeding and review Bell Atlantic's provisioning performance at that time. We have given substantial weight to the Department of Justice's views, but nonetheless, based upon our review of the record on loops as a whole, find that Bell Atlantic establishes that it provisions unbundled local loops at a level of performance sufficient for checklist compliance. 329. As detailed above, we conclude that Bell Atlantic's overall performance in providing access to unbundled local loops is sufficient to satisfy the competitive checklist. Bell Atlantic establishes that it meets the vast majority of installation appointments for standard and high-capacity voice grade loops and, in fact, misses fewer new loop installation appointments for competing carriers than it does for its retail customers. In addition, Bell Atlantic demonstrates that the loops it installs are of substantially the same quality as the loops it provides to its retail customers. Similarly, Bell Atlantic demonstrates that it provides coordinated cutovers of loops, i.e., hot cuts, to competing carriers within the prescribed time interval at least 90 percent of the time and that in no more than five percent of cases did the hot cut result in a service disruption. Finally, Bell Atlantic establishes that it provides loop maintenance and repair functions to competitors in substantially the same time and manner as it provides them to its retail customers. If xDSL services continue to grow rapidly, however, the aggregate loop results will be more heavily influenced by Bell Atlantic's performance in provisioning xDSL-specific loops. If the future aggregate performance declines from current levels, we will take appropriate enforcement action. 330. We choose to look at Bell Atlantic's overall performance due to the unique circumstances present in this application. Given our expectation that the unique circumstances present in this case will evolve over time or will otherwise not be present in future applications, we do not expect to rely solely on a BOC's overall loop performance in reaching a decision on this checklist item in future applications. Rather, we will find it most persuasive if future applicants under section 271, unlike this applicant, make a separate and comprehensive evidentiary showing with respect to the provision of xDSL-capable loops, either through proof of a fully operational separate advanced services affiliate as described below, which may also include appropriate performance measures, or through a showing of nondiscrimination in accordance with the guidance provided herein. Given our statutory obligation to encourage deployment of advanced services and the critical importance of the provisioning of xDSL loops to the development of the advanced service marketplace, we emphasize our intention to examine this issue closely in the future. 331. We believe that the creation of a separate affiliate for the provision of retail services may provide significant evidence that a BOC complies with the nondiscrimination requirements of the competitive checklist. A separate affiliate may be particularly appropriate for new offerings where it is difficult to demonstrate nondiscrimination through statistical evidence. In this case, we have further assurance that competing carriers in New York will have nondiscriminatory access to xDSL-capable loops in the future as a result of Bell Atlantic's commitment to establish a separate affiliate through which it will offer retail advanced services. 332. Providing advanced services through a separate affiliate would reduce the ability of a BOC to discriminate against competing carriers with respect to xDSL services. Significantly, under this structure, the BOC would be required to treat rival providers of advanced services the same way that it treats its own separate affiliate. Because the BOC's advanced services affiliate would use the same processes as competitors to conduct such activities as ordering loops, and pay an equivalent price for facilities and services, the creation of the affiliate should ensure a level playing field between the BOC and its advanced services competitors. We also believe that this structure would have the additional benefit of increasing the availability of and broadening the choices for advanced services for all Americans. A separate advanced services affiliate helps to attain the goal of encouraging entry into the provision of advanced services by numerous firms, in addition to the BOCs, while protecting against the risk that the BOCs could cripple these services in their infancy by discriminating against competing advanced services providers. 333. In the absence of a separate affiliate, a BOC seeking approval under section 271 in the future could demonstrate that it provides nondiscriminatory access to xDSL loops in accordance with checklist item four by establishing by a preponderance of the evidence that it provides xDSL-capable loops to competitors in a nondiscriminatory manner. If an applicant chose to make its case by submitting performance data, we would examine carefully the performance standards adopted by the relevant state commission. 334. In this regard, we emphasize our strong preference for a record that contains data measuring a BOC's performance pursuant to state-adopted standards that were developed with input from the relevant carriers and that include clearly-defined guidelines and methodology. The need for unambiguous performance standards and measures has been reinforced by the disputes in this record regarding, for instance, what performance is being measured and whether it is properly captured by particular measures. Accordingly, we encourage state commissions to adopt specific xDSL loop performance standards measuring, for instance, the average completion interval, the percent of installation appointments missed as a result of the BOC's provisioning error, the timeliness of order processing, the installation quality of xDSL loops provisioned, and the timeliness and quality of the BOC's xDSL maintenance and repair functions. We believe information on these dimensions of performance is critical to ensuring our joint federal and state commitment to the development of a vibrant advanced services marketplace. We also urge states to consider adoption of self-enforcing mechanisms to ensure compliance with any state-adopted standards. 335. Specifically, depending upon whether there is an appropriate retail analogue, we would expect a BOC to demonstrate, preferably through the use of state or third-party verified performance data, that it provides xDSL-capable loops to competitors either in substantially the same average interval in which it provides xDSL service to its retail customers or in an interval that offers competing carriers a meaningful opportunity to compete. The BOC would also be expected to establish, again through defined performance measures, that it meets substantially the same number of installation appointments for the customers of competing carriers that it meets for its retail customers or that the level of missed appointments is sufficiently low to offer competitors a meaningful opportunity to compete. Additionally, we would expect a showing that the quality of the loops provisioned to competing carriers is substantially the same as the quality of the lines used for the BOC's provision of retail advanced services or that the level of quality is sufficiently high to permit competitors to compete meaningfully. We would also look for evidence establishing that the BOC performs maintenance and repair functions for competitors' xDSL loops in substantially the same time and manner as it does for its retail lines. Finally, we would expect the BOC to demonstrate that it provides competing carriers with nondiscriminatory access to the pre-ordering and ordering OSS functions associated with the provision of xDSL loops, including access to loop qualification information and databases. In this regard, the BOC could make such a showing through evidence of either extensive commercial experience or third-party testing. 336. In conclusion, we reiterate that we do not expect the special circumstances that are present in this application to exist in future applications. Competitors are increasingly ordering xDSL loops, and, as the states begin to develop performance standards in this area, there will be a framework for future examination of performance data. Most importantly, in setting forth our views on the two avenues of proof that we would find persuasive in future applications, we have now provided direction to the BOCs regarding their obligation to provide xDSL-capable loops in accordance with the requirements of the competitive checklist. E. Checklist Item 5 -- Unbundled Local Transport 1. Background 337. Section 271(d)(2)(B)(v) of the competitive checklist requires a BOC to provide "[l]ocal transport from the trunk side of a wireline local exchange carrier switch unbundled from switching or other services." The Commission has required that BOCs provide both dedicated and shared transport to requesting carriers. Dedicated transport consists of BOC transmission facilities dedicated to a particular customer or carrier that provide telecommunications between wire centers owned by BOCs or requesting telecommunications carriers, or between switches owned by BOCs or requesting telecommunications carriers. Shared transport consists of transmission facilities shared by more than one carrier, including the BOC, between end office switches, between end office switches and tandem switches, and between tandem switches, in the BOC's network. 2. Discussion 338. Based on the evidence in the record, we conclude that Bell Atlantic provides both shared and dedicated transport in compliance with the requirements of this checklist item. The New York Commission also finds that Bell Atlantic is in compliance with this checklist item. 339. Bell Atlantic's August and September 1999 data concerning missed appointments for interoffice facilities show that its provisioning of transport to competitive LECs is nondiscriminatory. Moreover, none of the commenting parties challenge Bell Atlantic's showing concerning the provision of shared transport, except insofar as the commenters address OSS issues and matters concerning the provisioning of the UNE platform, which we address elsewhere. 340. We are not persuaded by the assertions of some commenters that Bell Atlantic fails to provide dedicated local transport in a timely manner. Bell Atlantic states that, with the exception of Choice One discussed below, these commenters have not ordered unbundled local transport from Bell Atlantic, but rather have requested special access services from Bell Atlantic's interexchange access tariffs. We cannot accept the assertion by a number of these parties that the provision of special access should be considered for purposes of determining checklist compliance in this proceeding. Although dedicated local transport and the interoffice portion of special access are generally provided over the same facilities, they differ in certain other respects. A number of these parties, however, assert that the checklist requirements focus on the provision of physical facilities, not the regulatory classifications that apply. We do not believe that checklist compliance is intended to encompass the provision of tariffed interstate access services simply because these services use some of the same physical facilities as a checklist item. We have never considered the provision of interstate access services in the context of checklist compliance before. The fact that competitive LECs can use interstate special access service in lieu of the EEL, a combination of unbundled loops and transport, and can convert special access service to EELs does not persuade us that we should alter our approach and consider the provision of special access for purposes of checklist compliance. This is especially true when Bell Atlantic is not required to demonstrate that it provides EELs for purposes of checklist compliance in this application because the application was filed before the effective date of the UNE Remand Order clearly establishing Bell Atlantic's federal obligation to provide EELs. 341. Nevertheless, to the extent that parties are experiencing delays in the provisioning of special access services ordered from Bell Atlantic's federal tariffs, we note that these issues are appropriately addressed in the Commission's section 208 complaint process. 342. In addition, we find that Bell Atlantic satisfactorily responds to Choice One's complaint that Bell Atlantic's provisioning interval for unbundled local transport reflects unacceptable delays. According to Bell Atlantic, Choice One failed to follow the recommended procedures and ordered entrance facilities after it ordered collocation. Bell Atlantic asserts that if Choice One had followed repeatedly suggested procedures and ordered collocation and entrance facilities simultaneously, both would have been ready at the same time. Based on the present record, this appears to be an isolated problem for which Bell Atlantic should not be held responsible. F. Checklist Item 6 - Unbundled Local Switching 1. Background 343. Section 271(c)(2)(B)(vi) of the 1996 Act requires a BOC to provide "[l]ocal switching unbundled from transport, local loop transmission, or other services." In the Second BellSouth Louisiana Order, the Commission required BellSouth to provide unbundled local switching that included line-side and trunk-side facilities, plus the features, functions, and capabilities of the switch. The features, functions, and capabilities of the switch include the basic switching function as well as the same basic capabilities that are available to the incumbent LEC's customers. Additionally, local switching includes all vertical features that the switch is capable of providing, as well as any technically feasible customized routing functions. 344. Moreover, in the Second BellSouth Louisiana Order, the Commission required BellSouth to permit competing carriers to purchase unbundled network elements, including unbundled switching, in a manner that permits a competing carrier to offer, and bill for, exchange access and the termination of local traffic. The Commission also stated that measuring daily customer usage for billing purposes requires essentially the same OSS functions for both competing carriers and incumbent LECs, and that a BOC must demonstrate that it is providing equivalent access to billing information. Therefore, the ability of a BOC to provide billing information necessary for a competitive LEC to bill for exchange access and termination of local traffic is an aspect of unbundled local switching. Thus, there is an overlap between the provision of unbundled local switching and the provision of the OSS billing function. 345. In the Second BellSouth Louisiana Order, the Commission stated that to comply with the requirements of unbundled local switching, a BOC must also make available trunk ports on a shared basis and routing tables resident in the BOC's switch, as necessary to provide access to shared transport functionality. The Commission also stated that a BOC may not limit the ability of competitors to use unbundled local switching to provide exchange access by requiring competing carriers to purchase a dedicated trunk from an interexchange carrier's point of presence to a dedicated trunk port on the local switch. 2. Discussion 346. Based on the evidence in the record, we conclude that Bell Atlantic demonstrates that it complies with checklist item 6. Specifically, Bell Atlantic demonstrates that it provides: (1) line-side and trunk side facilities; (2) basic switching functions; (3) vertical features; (4) customized routing; (5) shared trunk ports; (6) unbundled tandem switching; (7) usage information for billing exchange access, and (8) usage information for billing for reciprocal compensation. The New York Commission concludes that Bell Atlantic is in compliance with this checklist item. 347. We are not persuaded by Z-Tel that Bell Atlantic fails to meet the requirements for this checklist item. Z-Tel claims that Bell Atlantic has used the Network Design Request (NDR) process to delay implementation of Z-Tel's custom dialing plans in selected New York markets. We find that this claim does not warrant a conclusion that Bell Atlantic has failed to comply with this checklist item. We recognize that Z-Tel is better able to serve its customers if it is able to obtain a consistent level of service from Bell Atlantic statewide. We note, however, that the time frames for delivery of custom dialing plans are subject to negotiation between Bell Atlantic and competitive LECs under the terms of Bell Atlantic's interconnection agreements, and Z-Tel has not shown that Bell Atlantic's explanation for offering longer implementation time frames due to year 2000 system concerns is unreasonable. Moreover, Bell Atlantic states that, in the interim, it has offered, and Z-Tel is now pursuing, an option of obtaining a generic NDR instead of a customized NDR, until a conversion to a customized NDR can take place. Insofar as the commenters raise OSS issues and matters concerning the provisioning of the UNE platform, the primary vehicle used by competitive LECs to obtain unbundled local switching, we address these issues elsewhere. 348. We note that Z-Tel filed an ex parte letter on November 2, 1999, alleging that, after the comment date, Bell Atlantic ceased providing a vertical feature of the switch -- pre- programming speed dial capability. Bell Atlantic, in response, claims that this feature is designed to be initiated and controlled by the end user and, as such, it is not a feature that Bell Atlantic provides to its own retail users or to competing carriers. We find that, in view of the compelling evidence in the record that Bell Atlantic complies with this checklist item, Z-Tel's claim does not present a sufficient basis upon which to find that Bell Atlantic has fallen out of compliance in the course of the instant proceeding. If, however, future evidence reveals this to be the case, we will take appropriate enforcement action against Bell Atlantic. G. Checklist Item 7 1. 911 and E911 Access a. Background 349. Section 271(c)(2)(B)(vii) of the 1996 Act requires a BOC to provide "[n]ondiscriminatory access to - (I) 911 and E911 services." In the Ameritech Michigan Order, the Commission found that "section 271 requires a BOC to provide competitors access to its 911 and E911 services in the same manner that a BOC obtains such access, i.e., at parity." Specifically, the Commission found that a BOC "must maintain the 911 database entries for competing LECs with the same accuracy and reliability that it maintains the database entries for its own customers." For facilities-based carriers, the BOC must provide "unbundled access to [its] 911 database and 911 interconnection, including the provision of dedicated trunks from the requesting carrier's switching facilities to the 911 control office at parity with what [the BOC] provides to itself." b. Discussion 350. Based on the evidence submitted in the record, we conclude that Bell Atlantic demonstrates that it is providing nondiscriminatory access to 911/E911 services, and thus satisfies the requirements of checklist item (vii)(I). We note that no commenter disputes Bell Atlantic's compliance with this portion of checklist item 7, and the New York Commission concludes that Bell Atlantic is providing nondiscriminatory access to 911/E911. 2. Directory Assistance/Operator Services a. Background 351. Section 271(c)(2)(B)(vii)(II) and section 271(c)(2)(B)(vii)(III) require a BOC to provide nondiscriminatory access to "directory assistance services to allow the other carrier's customers to obtain telephone numbers" and "operator call completion services," respectively. Section 251(b)(3) of the 1996 Act imposes on each LEC "the duty to permit all [competing providers of telephone exchange service and telephone toll service] to have nondiscriminatory access to . . . operator services, directory assistance, and directory listing with no unreasonable dialing delays." The Commission implemented section 251(b)(3) in the Local Competition Second Report and Order. 352. We concluded in the Second BellSouth Louisiana Order that a BOC must be in compliance with the regulations implementing section 251(b)(3) to satisfy the requirements of sections 271(c)(2)(B)(vii)(II) and 271(c)(2)(B)(vii)(III). In the Local Competition Second Report and Order, the Commission held that the phrase "nondiscriminatory access to directory assistance and directory listings" means that "the customers of all telecommunications service providers should be able to access each LEC's directory assistance service and obtain a directory listing on a nondiscriminatory basis, notwithstanding : (1) the identity of a requesting customer's local telephone service provider; or (2) the identity of the telephone service provider for a customer whose directory listing is requested." The Commission concluded that nondiscriminatory access to the dialing patterns of 4-1-1 and 5-5-5-1-2-1-2 to access directory assistance were technically feasible, and would continue. The Commission specifically held that the phrase "nondiscriminatory access to operator services" means that ". . . a telephone service customer, regardless of the identity of his or her local telephone service provider, must be able to connect to a local operator by dialing `0,' or `0 plus' the desired telephone number." 353. Competing carriers may provide operator services and directory assistance by either reselling the BOC's services or by using their own personnel and facilities to provide these services. The Commission's rules require BOCs to permit competitive LECs wishing to resell the BOC's operator services and directory assistance to request the BOC to brand their calls. Competing carriers wishing to provide operator services or directory assistance using their own facilities and personnel must be able to obtain directory listings either by obtaining directory information on a "read only" or "per dip" basis from the BOC's directory assistance database, or by creating its own directory assistance database by obtaining the subscriber listing information in the BOC's database. b. Discussion 354. Based on the evidence in the record, we conclude that Bell Atlantic demonstrates that it provides directory assistance services in accordance with the requirements of checklist item 7. The New York Commission concludes that Bell Atlantic has satisfied this checklist item. 355. We are not persuaded by commenters' arguments that Bell Atlantic fails to comply with checklist item 7. AT&T submits studies to show that Bell Atlantic's systems drop more than 10 percent of the directory listings associated with unbundled loop orders from Bell Atlantic's directory assistance database. In response, Bell Atlantic asserts that AT&T's studies are flawed and do not properly reflect improvements Bell Atlantic has made to its systems. We find that Bell Atlantic has taken adequate measures to detect any dropped listings and restore them to the directory assistance database promptly. No other commenter raises this objection, suggesting the difficulty is of limited competitive consequence. In fact, several parties support Bell Atlantic's assertion of compliance with this checklist item. Accordingly, we conclude that these objections are not sufficient to conclude that Bell Atlantic has failed to comply with the requirements of checklist item 7. 356. In reaching this conclusion, we recognize that we differ somewhat from the Department of Justice. The Department of Justice, relying on evidence submitted by AT&T, however, did not have the benefit of Bell Atlantic's Reply, which we believe sufficiently rebuts AT&T's claims. Moreover, we note that the Department of Justice does not argue that Bell Atlantic fails to comply with checklist item 7 but rather simply cites Bell Atlantic's difficulties in this area as evidence that its hot cut performance needs improvement. H. Checklist Item 8 - White Pages Directory Listings 1. Background 357. Section 271(c)(2)(B)(viii) of the 1996 Act requires a BOC to provide "[w]hite pages directory listings for customers of the other carrier's telephone exchange service." Section 251(b)(3) of the 1996 Act obligates all LECs to permit competitive providers of telephone exchange service and telephone toll service to have nondiscriminatory access to directory listings. 358. In the Second BellSouth Louisiana Order, the Commission concluded that "consistent with the Commission's interpretation of `directory listing' as used in section 251(b)(3), the term `white pages' in section 271(c)(2)(B)(viii) refers to the local alphabetical directory that includes the residential and business listings of the customers of the local exchange provider." We further concluded, "the term `directory listing,' as used in this section, includes, at a minimum, the subscriber's name, address, telephone number, or any combination thereof." 359. In the Second BellSouth Louisiana Order, the Commission found that a BOC satisfies the requirements of checklist item 8 by demonstrating that it: (1) provided nondiscriminatory appearance and integration of white page directory listings to competitive LECs' customers; and (2) provided white page listings for competitors' customers with the same accuracy and reliability that it provides its own customers. 2. Discussion 360. Based on the evidence in the record, we find that Bell Atlantic satisfies the requirements of checklist item 8. Bell Atlantic demonstrates that it is providing white pages directory listings for customers of competitive LECs that are nondiscriminatory in appearance and integration, and have the same accuracy and reliability that Bell Atlantic provides for its own customers. The New York Commission concludes that Bell Atlantic complies with this checklist item. 361. We are not persuaded by AT&T and Choice One's assertions that Bell Atlantic fails to provide white pages directory listings in a nondiscriminatory manner. Although AT&T claims that Bell Atlantic's OSS consistently drop directory listing orders associated with UNE loop orders, AT&T provides no evidence of problems with the white pages directory listings themselves as a result. Moreover, although Choice One provides evidence of one dropped white pages directory listing, we do not find that this isolated incident is reflective of a systemic problem with Bell Atlantic's provisioning of their listings. I. Checklist Item 9 - Numbering Administration 1. Background 362. Section 271(c)(2)(B)(ix) of the 1996 Act requires a BOC to provide "nondiscriminatory access to telephone numbers for assignment to the other carrier's telephone exchange service customers," until "the date by which telecommunications numbering administration, guidelines, plan, or rules are established." The checklist mandates compliance with "such guidelines, plan, or rules" after they have been established. 363. Bell Atlantic does not assign telephone numbers to itself or competitive LECs. The Commission has designated NeuStar, Inc. ("NeuStar") as the North American Numbering Plan Administrator. NeuStar is responsible for assigning blocks of 10,000 telephone numbers (NXX Codes) to carriers within each area code, and for coordinating area code relief planning efforts with state commissions. Bell Atlantic must demonstrate that it adheres to industry numbering administration guidelines and Commission rules, including provisions requiring the accurate reporting of data to the code administrator. 2. Discussion 364. Based on the evidence in the record, we find that Bell Atlantic satisfies the requirements of checklist item 9. No commenters allege that Bell Atlantic has failed to meet the requirements for this checklist item. The New York Commission states that Bell Atlantic has demonstrated that it complies with the Commission's number assignment rules and Industry Numbering Committee Central Office Code Guidelines, and that it accurately reports data to the Central Office Code Administrator. J. Checklist Item 10 - Databases and Associated Signaling 1. Background 365. Section 271(c)(2)(B)(x) of the 1996 Act requires a BOC to provide "nondiscriminatory access to databases and associated signaling necessary for call routing and completion." In the Second BellSouth Louisiana Order, we required BellSouth to demonstrate that it provided requesting carriers with nondiscriminatory access to: "(1) signaling networks, including signaling links and signaling transfer points; (2) certain call-related databases necessary for call routing and completion, or in the alternative, a means of physical access to the signaling transfer point linked to the unbundled database; and (3) Service Management Systems (SMS);" and to design, create, test, and deploy Advanced Intelligent Network (AIN) based services at the SMS through a Service Creation Environment (SCE). 2. Discussion 366. Based on the evidence in the record, we find that Bell Atlantic satisfies the requirements of checklist item 10. No commenters allege that Bell Atlantic has failed to meet the requirements for this section. The New York Commission concludes that Bell Atlantic meets this checklist item. Although Z-Tel states "it is impossible to verify whether Bell Atlantic actually can provision AIN related services, because no carrier presently purchases these services from Bell Atlantic," we note that Bell Atlantic is not required to actually furnish a particular item in order to satisfy its obligations under the checklist. Rather, as we have previously stated, if no competitor is actually using a checklist item, a BOC must show that it has a concrete and specific legal obligation to furnish the item upon request and be "presently ready to furnish each item in the quantities that competitors may reasonably demand and at an acceptable level of quality." We find that Bell Atlantic has met this burden. K. Checklist Item 11 - Number Portability 1. Background 367. Section 271 (c)(2)(B)(xi) of the 1996 Act requires a BOC to be in compliance with the number portability regulations the Commission has adopted pursuant to section 251 of the 1996 Act. Section 251(b)(2) of the 1996 Act requires all LECs "to provide, to the extent technically feasible, number portability in accordance with requirements prescribed by the Commission." The 1996 Act defines number portability as "the ability of users of telecommunications services to retain, at the same location, existing telecommunications numbers without impairment of quality, reliability, or convenience when switching from one telecommunications carrier to another." The Commission has incorporated this definition into its rules. Moreover, to prevent the cost of number portability from thwarting local competition, Congress enacted section 251(e)(2), which requires that "[t]he cost of establishing telecommunications numbering administration arrangements and number portability shall be borne by all telecommunications carriers on a competitively neutral basis as determined by the Commission." 368. Pursuant to these statutory provisions, the Commission requires LECs to offer interim number portability "to the extent technically feasible." The Commission also requires LECs to gradually replace interim number portability with permanent number portability. The Commission has established guidelines for states to follow in mandating a competitively neutral cost-recovery mechanism for interim number portability, and created a competitively neutral cost-recovery mechanism for long-term number portability. 2. Discussion 369. Based on the evidence in the record, we conclude that Bell Atlantic complies with the requirements of checklist item 11. The New York Commission concludes that Bell Atlantic has satisfied this checklist item. 370. RCN states that Bell Atlantic will not provide number portability to customers with RCN-issued telephone numbers. For example, RCN asserts that, in the last few months, Bell Atlantic has refused to allow RCN's customers that switch to Bell Atlantic to keep a RCN- issued telephone number. Bell Atlantic denies this allegation. We do not find that RCN's unsupported assertions are indicative of a systemic failure in Bell Atlantic's provision of number portability. 371. Adelphia and AT&T allege that Bell Atlantic has problems coordinating number portability with loop cutovers. Specifically, Adelphia maintains that "Bell Atlantic frequently activates number portability prematurely," resulting in customers being unable to receive telephone calls. AT&T implies that Bell Atlantic's problems with hot cuts have "adversely affected" number portability. Like RCN's claim, we find both Adelphia and AT&T's claims to be unsupported, conclusory allegations that do not warrant a finding of noncompliance with this checklist item. L. Checklist Item 12 - Local Dialing Parity 1. Background 372. Section 271(c)(2)(B)(xii) requires a BOC to provide "[n]ondiscriminatory access to such services or information as are necessary to allow the requesting carrier to implement local dialing parity in accordance with the requirements of section 251(b)(3)." Section 251(b)(3) imposes upon all LECs "[t]he duty to provide dialing parity to competing providers of telephone exchange service and telephone toll service with no unreasonable dialing delays." Section 153(15) of the Act defines "dialing parity" to mean that: . . . a person that is not an affiliate of a local exchange carrier is able to provide telecommunications services in such a manner that customers have the ability to route automatically, without the use of any access code, their telecommunications to the telecommunications services provider of the customer's designation . . . 373. Customers of competing carriers must be able to dial the same number of digits the BOC's customers dial to complete a local telephone call. Moreover, customers of competing carriers must not otherwise suffer inferior quality service, such as unreasonable dialing delays, compared to the BOC's customers. 2. Discussion 374. Based on the evidence in the record, we find that Bell Atlantic demonstrates that it provides local dialing parity in accordance with the requirements of section 251(b)(3) and thus satisfies the requirements of this checklist item. No commenter challenges Bell Atlantic's assertion that it provides local dialing parity. The New York Commission concludes that Bell Atlantic meets the requirements of this checklist obligation. M. Checklist Item 13 -- Reciprocal Compensation. 1. Background 375. Section 271(c)(2)(B)(xiii) of the Act requires that a BOC's access and interconnection includes "[r]eciprocal compensation arrangements in accordance with the requirements of section 252(d)(2)." In turn, section 252(d)(2)(A) states that "a State commission shall not consider the terms and conditions for reciprocal compensation to be just and reasonable unless (i) such terms and conditions provide for the mutual and reciprocal recovery by each carrier of costs associated with the transport and termination on each carrier's network facilities of calls that originate on the network facilities of the other carrier; and (ii) such terms and conditions determine such costs on the basis of a reasonable approximation of the additional costs of terminating such calls." 2. Discussion. 376. Based on the evidence in the record, we conclude that Bell Atlantic demonstrates that its access and interconnection include reciprocal compensation arrangements in accordance with the requirements of section 252(d)(2), and thus, satisfies the requirements of checklist item 13. Bell Atlantic demonstrates that it (1) has reciprocal compensation arrangements in accordance with section 252(d)(2) in place, and (2) is making all required payments in a timely fashion. The New York Commission concludes that Bell Atlantic is in compliance with checklist item 13. 377. We are not persuaded by Global NAPs' claim that Bell Atlantic fails to meet this checklist item. Global NAPS argues that Bell Atlantic acts in an anticompetitive manner with respect to payments for traffic terminated by competitive LECs to ISPs by, among other things, failing to pay compensation in a timely manner under the parties' interconnection agreement, and disputing the amount of per-minute compensation payment which is owed pursuant to the NYPSC Reciprocal Compensation Order. Global NAPs also disputes Bell Atlantic's assertion that it is complying with the NYPSC Reciprocal Compensation Order requiring compensation for ISP- bound calls. In light of our prior ruling that "ISP-bound traffic is non-local interstate traffic" and that "the reciprocal compensation requirements of section 251(b)(5) of the Act . . . do[es] not govern inter-carrier compensation for this traffic," we conclude that Global NAPs' arguments are irrelevant to checklist item 13. We recognize that Bell Atlantic has an obligation to comply with New York Commission orders concerning inter-carrier compensation for ISP-bound traffic, pursuant to our Inter-Carrier Compensation for ISP-Bound Traffic Order and pending completion of our rulemaking on this issue. Inter-carrier compensation for ISP bound traffic, however, is not governed by section 251(b)(5), and, therefore, is not a checklist item. In addition, we deny e.spire's request that we condition any Bell Atlantic 271 authority on Bell Atlantic's promise to pay any reciprocal compensation amounts currently due. The statute requires Bell Atlantic to make reciprocal compensation in a timely manner and as stated above, we find that Bell Atlantic complies with this provision. N. Checklist Item 14 - Resale 1. Background 378. Section 271(c)(2)(B)(xiv) of the Act requires a BOC to make "telecommunications services . . . available for resale in accordance with the requirements of sections 251(c)(4) and 252(d)(3)." Section 251(c)(4)(A) requires incumbent LECs "to offer for resale at wholesale rates any telecommunications service that the carrier provides at retail to subscribers who are not telecommunications carriers." Section 251(c)(4)(B) prohibits "unreasonable or discriminatory conditions or limitations" on resale, with the exception that "a State commission may, consistent with regulations prescribed by the Commission under this section, prohibit a reseller that obtains at wholesale rates a telecommunications service that is available at retail only to a category of subscribers from offering such service to a different category of subscribers." Section 252(d)(3) sets forth the basis for determining "wholesale rates" as the "retail rates charged to subscribers for the telecommunications service requested, excluding the portion thereof attributable to any marketing, billing, collection, and other costs that will be avoided by the local exchange carrier." 379. In the Local Competition First Report and Order, the Commission promulgated several rules regarding the scope of the resale requirement and permissible restrictions on resale that a LEC may impose. Most significantly, resale restrictions are presumed to be unreasonable unless the LEC "proves to the state commission that the restriction is reasonable and non-discriminatory." 380. Finally, in accordance with section 271(c)(2)(B)(ii) and section 271(c)(2)(B)(xiv), a BOC must demonstrate that it provides nondiscriminatory access to operations support systems for the resale of its retail telecommunications services. 2. Discussion 381. Based on the evidence in the record, we conclude that Bell Atlantic demonstrates that it makes telecommunications services available for resale in accordance with sections 251(c)(4) and 252(d)(3) and thus, satisfies the requirements of checklist item 14. Bell Atlantic demonstrates that it: (1) offers for resale at wholesale rates any telecommunications service that the carrier provides at retail to subscribers who are not telecommunications carriers, and (2) offers such telecommunications services for resale without unreasonable or discriminatory conditions or limitations. Bell Atlantic demonstrates that it provides nondiscriminatory access to operations support systems for the resale of its retail telecommunications services. The New York Commission states that Bell Atlantic is in compliance with this checklist item. 382. We are unpersuaded by CCA's arguments that Bell Atlantic does not comply with checklist item 14 because the difference between Bell Atlantic's wholesale rates and retail rates is so narrow that it precludes a profit and hinders competition. CCA asserts that in New York, for example, one of Bell Atlantic's regional toll plans is priced below a reseller's cost to buy end- to-end wholesale switched access service. In addition, CCA contends that Bell Atlantic offers discriminatory pricing by offering resold services at an across the board discount off standard end user "tariff" prices even though each local product, service or vertical feature carries a different retail profit margin above its cost. CCA maintains that "the unitary discount forces the reseller to pay end-user retail profit margins instead of carrier based profit margins and that the margins above costs that Bell Atlantic collects from resellers should be no different than those collected from facilities-based carriers." CCA argues that because of this pricing structure, Bell Atlantic forces a reseller to overpay for products and services. 383. Section 252(d)(3) provides that "a State commission shall determine the wholesale rates on the basis of retail rates charged to subscribers for the telecommunications service requested, excluding the portion thereof attributable to any marketing, billing, collection, and other costs that will be avoided by the local exchange carrier." Bell Atlantic maintains that it provides services at wholesale discounts set by the New York Commission: 19.1 percent for lines with Bell Atlantic's Operator Services and Directory Assistance, and 21.7 percent for lines without these features. In addition, the New York Commission states that it set non-recurring charges for resellers in a manner consistent with the Commission's pricing regulations, and they are subject to further examination in a pending proceeding. CCA provides no evidence that the New York Commission failed to adhere to the statutory requirements in setting the wholesale rates with respect to marketing, billing, collection and other avoided costs. Furthermore, in the Local Competition First Report and Order, the Commission recognized "that a uniform rate is simple to apply, and avoids the need to allocate avoided costs among services." Although the Commission observed that avoided costs may, in fact, vary among services, it neither prohibited nor required use of a single, uniform discount rate for all of an incumbent LEC's services. Thus, we find that Bell Atlantic makes available telecommunications services at wholesale rates established by the New York Commission as required by the statute. 384. Termination liabilities. Bell Atlantic maintains that it "does not impose any unreasonable or discriminatory conditions or limitations on the resale of its telecommunications services." As discussed below, we are not persuaded by commenters that Bell Atlantic imposes unreasonable or discriminatory conditions or limitations on the resale of its services. Thus, we find sufficient evidence that Bell Atlantic is satisfying the requirement in checklist item 14 that it offers its telecommunications services for resale in accordance with section 251(c)(4)(B) of the Act. 385. Resellers may resell any of Bell Atlantic's CSAs to any customer that meets the terms and conditions of that particular arrangement, and customers may aggregate traffic from multiple customers to satisfy any volume requirement. In addition, if a customer elects to terminate its service with Bell Atlantic, it may be subject to termination liabilities to the extent it was part of the CSA agreed to by the customer. For example, Bell Atlantic explains that, if a customer terminates a five-year CSA for Centrex after two years, the termination liability will be the difference between what the customer would have paid under a two-year CSA and what the customer actually paid under the five-year CSA. According to Bell Atlantic, the Commission has previously recognized that these types of termination liabilities are both permissible and pro- competitive. 386. ALTS, e.spire/Net2000, and TRA argue that the termination liability provisions contained in Bell Atlantic's contracts are anti-competitive, unjust, unreasonable, excessive or unfair. Except for TRA, they contend that the Commission should adopt a "fresh look" requirement in which customers would be able to terminate long-term contracts without incurring any penalty before or upon any Bell Atlantic section 271 relief. The commenters contend that such a requirement is consistent with prior Commission decisions that adopted a "fresh look" policy because of changed circumstances, such as when a monopoly marketplace opens to competition, or where a regulatory area was subject to significantly altered circumstances. Bell Atlantic responds that its termination liabilities are pro-competitive, reasonable, and have not inhibited competing carriers from obtaining customers. 387. It appears that termination liability is not calculated in the same manner for all contracts. For example, Bell Atlantic's termination liability for Centrex customers is limited to the difference between what the customer would have paid under the shorter term and what the customer actually paid under the long-term contract. This method for calculating liability comports with the method that we recognized in the Expanded Interconnection Order. 388. e.spire/Net2000 contend that Bell Atlantic's termination liability constitutes a "take or pay" contract with respect to Flex Path T-1 service. Based on their understanding of Bell Atlantic's pricing structure for Flex Path T-1 services, e.spire/Net2000 maintain that customers are not entitled to any additional discounts based on the duration of the contract. Therefore, they assert that Flex Path T-1 service customers cannot realistically terminate their contract to move to a competitor since they will still be charged for the service. 389. In the BellSouth South Carolina section 271 proceeding, the Commission expressed concern with the application of termination liabilities to situations where a new entrant sought to assume an existing CSA contract. The Commission stated that "[b]ecause, depending, on the nature of these [termination] fees, their imposition creates additional costs for a CSA customer that seeks service from a reseller, they may have the effect of insulating portions of the market from competition through resale." Thus, under these circumstances, termination liability could constitute an unreasonable restriction on resale. 390. We do not have the same concerns here. Although the Commission has adopted "fresh look" requirements in prior proceedings, the Commission has not adopted such a policy for the CSAs at issue here, which are generally regulated by the states. The New York Commission has already addressed Bell Atlantic's policy of imposing termination charges specified in an original CSA, when a reseller wishes to resell the services covered by an existing CSA and the reseller accepts the terms and conditions set forth in the original contract. The New York Commission has held "that termination penalties may not be assessed in instances where the transaction involves an assignment of the customer's contract with Bell Atlantic-NY, and that Bell Atlantic-NY may not unreasonably bar such an assignment." Therefore, pursuant to the New York Commission CTC Order, the termination liabilities complained of here would not be triggered by an assignment of the contract. Rather, the termination liability is only triggered by a complete termination of the contract. Accordingly, the termination liabilities do not constitute a restriction on resale under checklist 14. Although termination liabilities that apply when a customer terminates a contact to take service from another provider could, in certain circumstances, be unreasonable or anticompetitive, they may not on their face put a carrier out of compliance with checklist item 14. Therefore, the absence of a "fresh look" requirement is not a basis for rejecting a section 271 application. In addition, as the New York Commission points out, parties may file a complaint about a termination liability provision at the New York Commission, or initiate a proceeding under section 253 of the 1996 Telecommunications Act. We find that the record does not support a finding that termination liability provisions contained in Bell Atlantic's CSAs constitute an unreasonable or discriminatory condition or limitation on the resale of its telecommunications services. 391. Several commenters also suggest that the Commission should impose a "fresh look" requirement in this proceeding on public interest grounds, that is, as part of our analysis under section 271(d)(3)(C). We note that a similar issue has been raised by KMC Telecom in a Petition for Declaratory Rulemaking, which is now pending before the Commission. We conclude that issues raised by parties in this proceeding relating to contract termination liability are more appropriately resolved in the context of that pending petition, and we thus decline to resolve the issue in this proceeding. 392. Resale of xDSL-based services. We are not persuaded by TRA's argument that Bell Atlantic is restricting resale in violation of section 251(c)(xiv) because it does not make volume and term offerings of xDSL-based services available for resale. According to TRA, in the BellSouth Louisiana Order the Commission stated that "any service sold to end users is a retail services and thus is subject to the wholesale discount requirement, even if it is already priced at a discount of another retail service." TRA contends that, by declining to make volume and term offerings of xDSL-based services available for resale, Bell Atlantic is creating a general exemption from the wholesale requirement. TRA further argues that in the Local Competition Order we stated that section 251(c)(4) "makes no exceptions for promotional or discounted offerings." Bell Atlantic responds that it is making all of its ADSL telecommunications services available for resale at the tariff rates pursuant to section 251(c)(4), and it is making its ADSL telecommunications service that it offers to its own end user customers available for resale pursuant to section 251(c)(4). Bell Atlantic argues that its "wholesale ADSL offering is not a retail service, and therefore is not subject to section 251(c)(4)'s requirement to provide retail services at an avoided cost discount." 393. We have recently addressed this issue in Deployment of Wireline Services Offering Advanced Telecommunications Capability. In that proceeding, we found that, although DSL services designed for and sold to residential and business end-users are subject to the discounted resale obligations of section 251(c)(4), where the incumbent LEC offers DSL services as an input component to ISPs who combine the DSL service with their own Internet service, the discount resale obligations of section 251(c)(4) do not apply. Therefore, we agree with Bell Atlantic that it is not required to provide an avoided-cost discount on its wholesale ADSL offering because it is not a retail service subject to the discount obligations of section 251(c)(4). 394. Other resale conditions and limitations. We are also not persuaded by NALA's argument that Bell Atlantic has imposed an unreasonable condition on resale because it does not provide a flat-rate local service option for resale in New York City. According to NALA, Bell Atlantic offers only message-rate service in New York City. Thus, NALA maintains that prepaid local providers must block all services that could result in per-call or per-minute charges, including toll, operator services, information services, directory assistance, and directory assistance call completion, and this constitutes an unreasonable limitation on the services which NALA members can resell. We find NALA does not make a persuasive argument. As Bell Atlantic points out, it does not offer a flat-rate telephone service in New York City at retail to subscribers who are not telecommunications carriers, and therefore Bell Atlantic is under no obligation to provide such services for resale under the statute. 395. In addition, we reject CCA's argument that Bell Atlantic violates checklist item 14 because Bell Atlantic's resale tariff is highly restrictive, bundles services and prices, only allows resale of tariffed end-user services that have been designated by Bell Atlantic's retail marketing department, and does not offer some vertical products for resale. CCA argues that these limitations make it virtually impossible for a reseller to differentiate its service offering, since Bell Atlantic already has defined the retail products and services. CCA further argues that, as a result of Bell Atlantic's actions, resellers cannot meet the needs of the local telephone consumer and compete with Bell Atlantic. 396. As stated above, Bell Atlantic demonstrates that it offers for resale at wholesale rates any telecommunications services that it offers at retail to subscribers and pursuant to the discounts set by the New York Commission. Therefore, we are not persuaded by CCA's argument that Bell Atlantic violates checklist item 14 because its resale tariff is highly restrictive, bundles services and prices, and only allows resale of tariffed end-user services that have been designated by Bell Atlantic's retail marketing department. 397. In addition, based on the evidence in the record, we are unable to conclude that Bell Atlantic fails to make some vertical products available to resellers in violation of checklist item 14. Sections 251(c)(4) and 252(d)(3) of the Act and the Commission's rules do not require Bell Atlantic to provide its retail customers with all of the vertical products that Bell Atlantic is capable of providing. This does not mean, however, that Bell Atlantic may limit the vertical products that it makes available to competitive LECs. In the Second BellSouth Louisiana Order, the Commission required Bell South to provide unbundled local switching that included line-side and truck-side facilities, plus the features, functions, and capabilities of the switch. The features, functions, and capabilities of the switch include the basic switching function as well as the same basic capabilities that are available to the incumbent LEC's customers. Additionally, local switching includes all vertical features that the switch is capable of providing, as well as any technically feasible customized routing functions. Bell Atlantic provides local and tandem switching unbundled from loops and other network components. Unbundled local switching is available as a line-side or a trunk-side port (shared and dedicated) and includes all of the vertical features available to Bell Atlantic's retail customers on a line-by- line basis. Bell Atlantic states that it is prepared upon request to provide competitive LECs with access to other features resident in its switches that Bell Atlantic does not offer its retail customers. In situations where a competitive LEC seeks to resell vertical products that Bell Atlantic does not offer at retail to its subscribers, we find that Bell Atlantic complies with the resale obligations contained in checklist item 14 by providing competitive LECs with access to unbundled switching. We clarify that under these circumstances, the avoided cost discount under section 251(c)(4) does not apply because Bell Atlantic is not offering the vertical products at retail to its customers. 398. We also reject the claim of Destek, that Bell Atlantic Network Integration (BANI) and Bell Atlantic Digital Services (BADS) have associated themselves with state owned and operated universities in joint ventures through exclusive and anticompetitive, special contract interconnection agreements in New Hampshire. We find that Destek's argument does not pertain to Bell Atlantic's resale practices in New York and thus is not relevant to a determination of whether it meets checklist item 14 in this proceeding. Moreover, although Destek alleges that Bell Atlantic employs the same practices through out its service territories, it presents no evidence to support this claim with respect to Bell Atlantic's resale practices in New York. 399. Similarly, Ntegrity's argument that Bell Atlantic engages in anticompetitive practices in Pennsylvania, Maryland, and New Jersey is not relevant to a determination of whether Bell Atlantic meets checklist item 14 in New York. 3. Provisioning 400. Provisioning. Based on the evidence in the record, we find that Bell Atlantic satisfies the provisioning requirements of checklist item 14. As discussed supra Section V.B, Bell Atlantic is provisioning competitive LECs' orders for resale in substantially the same time and manner as for its retail customers. We are not persuaded by various claims that Bell Atlantic fails to provision resale services in a nondiscriminatory manner. Commenters assert that occasionally Bell Atlantic: (a) continues to bill its former customers following their switch to a competing provider, resulting in the customer being double billed; (b) fails to activate toll blocks on competitors' orders; (c) misses appointments to connect service to new customers; (d) changes the phone number preassigned to a reseller's customer without any notification; and (e) does not process the requests by resale customers to change their Presubscribed Interexchange Carrier ("PIC") seamlessly, as it does for the requests of Bell Atlantic's own retail customers. Although we do not discount the effect of such occasional incidents on affected customers, the present record does not indicate that these are systemic problems. Were these widespread problems, they would appear to warrant a finding of noncompliance. We conclude, however, that these problems are insufficient to overcome Bell Atlantic's showing that it is in compliance with the provisioning requirements of this checklist item. VI. SECTION 272 COMPLIANCE A. Background 401. Section 271(d)(3)(B) requires that the Commission shall not approve a BOC's application to provide interLATA services unless the BOC demonstrates that the "requested authorization will be carried out in accordance with the requirements of section 272." The Commission set standards for compliance with section 272 in the Accounting Safeguards Order and the Non-Accounting Safeguards Order. Together, these safeguards discourage and facilitate the detection of improper cost allocation and cross-subsidization between the BOC and its section 272 affiliate. In addition, these safeguards ensure that BOCs do not discriminate in favor of their section 272 affiliates. 402. As we stated in the Ameritech Michigan Order, compliance with section 272 is "of crucial importance" because the structural, transactional, and nondiscrimination safeguards of section 272 seek to ensure that BOCs compete on a level playing field. The Commission's findings regarding section 272 compliance constitute independent grounds for denying an application. Past and present behavior of the BOC applicant provides "the best indicator of whether [the applicant] will carry out the requested authorization in compliance with section 272." B. Discussion 403. Based on the record, we conclude that Bell Atlantic has demonstrated that it will comply with the requirements of section 272. We note that neither the New York Commission nor the Department of Justice addressed Bell Atlantic's showing of section 272 compliance. We address each section 272 requirement below. 1. Structural, Transactional, and Accounting Requirements of Section 272 404. Section 272(a) - Separate Affiliate. Section 272(a) requires BOCs and their local exchange carrier affiliates that are subject to section 251(c) to provide certain competitive services through structurally separate affiliates. For the reasons described in the section below, we conclude that Bell Atlantic demonstrates that it will operate in accordance with section 272(a). 405. Bell Atlantic has established three section 272 affiliates to provide in-region interLATA services upon gaining section 271 approval: Bell Atlantic Communications, Inc. (BACI), NYNEX Long Distance (NLD), and Bell Atlantic Global Networks, Inc. (BAGNI). Each affiliate is a wholly-owned subsidiary of Bell Atlantic Corporation, and each is incorporated in Delaware. Bell Atlantic plans to offer interLATA services to residential consumers through BACI, and to serve business customers through NLD. Both BACI and NLD will conduct business under the trade name "Bell Atlantic Long Distance." One affiliate, BAGNI, will build a telecommunications network and serve BACI and NLD. Bell Atlantic demonstrates that each affiliate has implemented internal control mechanisms to prevent, as well as detect and correct, any noncompliance with section 272. 406. Section 272(b)(1) - Operate Independently. Based on the evidence in the record, Bell Atlantic has demonstrated that it will comply with section 272(b)(1), which requires a section 272 affiliate to "operate independently from the Bell operating company." The Commission has interpreted the "operate independently" requirement to impose four important restrictions on the ownership and operations of a BOC and its section 272 affiliate: (1) no joint ownership of switching and transmission facilities; (2) no joint ownership of the land and buildings on which switching and transmission facilities are located; (3) no provision by the BOC (or other non- section 272 affiliate) of operation, installation, and maintenance services (OI&M) with respect to the section 272 affiliate's facilities; and (4) no provision of OI&M by the section 272 affiliate with respect to the BOC's facilities. 407. We disagree with AT&T's contentions that the disclosures Bell Atlantic makes on the Internet pursuant to section 272(b)(5) reveal the provisioning of proscribed OI&M services by a Bell Atlantic BOC to a section 272 affiliate. Bell Atlantic explains that the services noted by AT&T were construction services that do not involve installation or servicing telecommunications equipment. Our review of Bell Atlantic's Internet postings, its cost allocation manual (CAM), and its independent auditor's reports support Bell Atlantic's explanation. The Internet disclosures referenced by AT&T refer to certain types of employees and the rates at which such employees were billed to Bell Atlantic's section 272 affiliates. Reading this information in context, it is clear that the employees referenced in the Internet disclosures are not telecommunications technicians and engineers performing OI&M services. 408. Section 272(b)(2) - Books, Records, and Accounts. Based on the evidence in the record, Bell Atlantic demonstrates that it will comply with the requirement that its section 272 affiliates "shall maintain books, records, and accounts in a manner prescribed by the Commission which shall be separate from the books, records, and accounts maintained by the [BOCs]." We note that no party challenges Bell Atlantic's showing. 409. Section 272(b)(3) - Separate Officers, Directors, and Employees. Based on the evidence in the record, Bell Atlantic has demonstrated that it will comply with the "separate officers, directors, and employees" requirement of section 272(b)(3). We note that no party challenges Bell Atlantic's showing. 410. Section 272(b)(4) - Credit Arrangements. Based on the evidence in the record, Bell Atlantic has demonstrated that it will comply with section 272(b)(4), which prevents a section 272 affiliate from obtaining "credit under any arrangement that would permit a creditor, upon default, to have recourse to the assets of [any Bell Atlantic BOC]." We note that no party challenges Bell Atlantic's showing. 411. Section 272(b)(5) - Affiliate Transactions. Based on our review of its application, we conclude that Bell Atlantic demonstrates that it will comply with the public disclosure requirements of section 272(b)(5) for transactions between its BOCs and its section 272 affiliates. Section 272(b)(5) requires that a section 272 affiliate conduct all transactions with its affiliated BOCs on an arm's length basis. In addition, the statute requires section 272 affiliates to reduce all such transactions to writing and make them available for public inspection. Consistent with the Commission's Accounting Safeguards Order, Bell Atlantic must ensure that all transactions between its section 272 affiliates (i.e., BACI, NLD, and BAGNI) and any affiliated BOC are posted on the company's Internet homepage within 10 days of the transaction. To ensure that all affiliate transactions occur at arm's length, Bell Atlantic must abide by the Commission's affiliate transactions rules. The Commission evaluates the sufficiency of a BOC's Internet disclosures by referring to its ARMIS filings, its cost allocation manuals, and the CAM audit workpapers. 412. AT&T argues that Bell Atlantic failed to post all transactions between its BOCs and its section 272 affiliates on the Internet, and that Bell Atlantic fails to provide sufficient detail of such transactions. Although we are concerned about the issues raised by AT&T, Bell Atlantic persuades us that it will comply with section 272(b)(5)'s public disclosure requirement. To the extent that AT&T's comments and our review of the record revealed minor discrepancies between Bell Atlantic's Internet postings and its regular accounting submissions, we find that Bell Atlantic has submitted satisfactory evidence to explain the inconsistencies. As Bell Atlantic points out, a variety of circumstances may result in minor differences between ARMIS and CAM disclosures and the section 272(b)(5) Internet postings. Furthermore, we find that the value of the posting discrepancies is small, totaling less than the amount of the discrepancies at issue in the Second BellSouth Louisiana Order. Given these factors, we conclude that these isolated instances are not sufficient to show systemic flaws in Bell Atlantic's ability to comply with section 272(b)(5). Finally, we note that Bell Atlantic's Internet postings will undergo a thorough and systematic review in the section 272(d) biennial audit, which will ensure that any failures to post are identified in time for appropriate remedial action. 413. We likewise reject AT&T's assertion that Bell Atlantic's Internet postings do not contain sufficient detail to show that Bell Atlantic will comply with section 272(b)(5). As required by the Commission's section 272(b)(5) rules, Bell Atlantic discloses "the number and type of personnel assigned to the project, the level of expertise of such personnel, any special equipment used to provide the service, and the length of time required to complete the transaction." Although we are concerned that some descriptions of affiliate transactions may contain ambiguous descriptions of services, we are persuaded that, on balance, Bell Atlantic's descriptions are sufficiently detailed to facilitate the purchasing decisions of unaffiliated third parties. In addition, we find that Bell Atlantic has implemented the internal controls and processes needed to identify and correct potential problem areas with its Internet disclosures. We note that the section 272(d) biennial audit will ensure that Bell Atlantic continues to provide adequate descriptions of its posted transactions because inadequate descriptions will be identified by the Federal-State audit team, and disclosed in the subsequent audit report. 414. Based on the record evidence, we conclude that Bell Atlantic demonstrates that it will comply with the affiliate transactions rules, which is necessary to ensure that all transactions between a BOC and its section 272 affiliate occur at arm's length. We note that no party challenges Bell Atlantic's showing that it values transactions between its BOCs and its section 272 affiliates in accordance with our affiliate transactions rules. 415. Section 272(c)(2) - Accounting Principles. Based on the evidence in the record, Bell Atlantic demonstrates that its BOCs account for all transactions with its section 272 affiliates in accordance with the accounting principles designated or approved by the Commission. In the Accounting Safeguards Order, we concluded that complying with the Part 32 affiliate transactions rules satisfies the accounting requirements of section 272(c), which pertain to the BOC's "dealings" with its separate affiliate. We note that no party challenges Bell Atlantic's showing. 416. Section 272(d) - Biennial Audit. Based on the evidence in the record, we conclude that Bell Atlantic demonstrates that it will comply with section 272(d), which requires an independent audit of a BOC's compliance with section 272 after receiving interLATA authorization. Because the audit process involves a thorough and systematic evaluation into a BOC's compliance with section 272 and its affiliate relationships, we expect that the section 272(d) biennial audit will address the concerns raised by AARP, Closecall, and others for stringent post-entry oversight of section 272 compliance. 2. Nondiscrimination Safeguards of Section 272 417. Section 272(c)(1) - Nondiscrimination Safeguards. Based on the evidence in the record, we conclude that Bell Atlantic demonstrates it will comply with section 272(c)(1), which prohibits a BOC from discriminating in favor of its section 272 affiliate in the "provision or procurement of goods, services, facilities, and information, or in the establishment of standards." The Commission's nondiscrimination safeguards require a BOC to, among other things, "provide to unaffiliated entities the same goods, services, facilities, and information that it provides to its section 272 affiliate at the same rates, terms, and conditions." Although we agree with AT&T, CERB, and others regarding the broad nature of the nondiscrimination safeguards, we reject their contentions that Bell Atlantic fails to demonstrate compliance with the section 272(c)(1) nondiscrimination safeguards. As we noted with respect to section 272(b)(5) above, Bell Atlantic posts information about transactions between the BOC and its section 272 affiliates, and thereby provides unaffiliated entities with notice of opportunities to obtain the same goods, services, and facilities at the same rates, terms, and conditions available to the section 272 affiliate. We reject AT&T's assertion that Bell Atlantic failed to show compliance with section 272(c)(1) because Bell Atlantic failed to provide unaffiliated third parties equal opportunities to lease real estate space. Bell Atlantic persuades us that, with respect to the leases for real estate raised by AT&T, it regularly advertises its real estate listings, and thereby provides unaffiliated third parties with opportunities to lease space provided to its section 272 affiliates. 418. Section 272(e) - Fulfillment of Certain Requests. Based on the evidence in the record, Bell Atlantic demonstrates that it will comply with section 272(e), which requires Bell Atlantic to fulfill requests for, among other things, telephone exchange and exchange access services from unaffiliated entities within the same time period Bell Atlantic fulfills such requests for its own retail operations. In addition, section 272(e) also provides that a BOC "shall not provide any facilities, services, or information concerning its provision of exchange access to the [section 272 affiliate] unless such facilities, services, or information are made available to other providers of interLATA services in that market on the same terms and conditions." Finally, section 272(e) places certain accounting and nondiscrimination requirements on BOCs with respect to exchange access and facilities or services provided to its interLATA affiliate. We note that no party challenges Bell Atlantic's showing that it will comply with section 272(e). 3. Joint Marketing Requirements of Section 272 419. Section 272(g)(1) - Affiliate Sales of Telephone Exchange Access Services. Based on the evidence in the record, we conclude that Bell Atlantic has demonstrated that it will comply with the joint marketing provisions of section 272(g)(1). We reject as inconsistent with Commission precedent AT&T's contention that Bell Atlantic must submit proposed marketing scripts in order to demonstrate compliance with section 272(g). Although Bell Atlantic makes no assertions regarding the plans of one section 272 affiliate, BAGNI, to market or sell Bell Atlantic telephone exchange services, we conclude that BAGNI's evidence of a corporate compliance program and BAGNI's assertions that it plans to provide service only to BACI and NLD adequately persuade us that Bell Atlantic will operate in accordance with section 272(g)(1) for BAGNI. 420. We decline to adopt the suggestion of Excel to impose conditions on Bell Atlantic that would limit the ability of its section 272 affiliates to resell Bell Atlantic's local exchange service. Specifically, Excel requests that the Commission require Bell Atlantic either to forego the use of total service resale or to provide a greater discount for total service resale packages provided to competing carriers in New York. As we recently noted in the Non-Accounting Safeguards proceeding, section 272 does not prohibit a section 272 affiliate from providing both local exchange and interLATA services. We conclude that the need for restrictions on the ability of Bell Atlantic's section 272 affiliate to provide local service is unnecessary at this time, and that the existing section 272 safeguards adequately address the concerns raised by Excel. 421. Section 272(g)(2) - Bell Operating Company Sales of Affiliate Services. We conclude that Bell Atlantic demonstrates that it will comply with section 272(g)(2), which prevents a BOC from marketing or selling within its region any interLATA service provided by a section 272 affiliate absent authorization obtained pursuant to section 271(d). We note that no party challenges Bell Atlantic's assertions or provides evidence to rebut Bell Atlantic's showing. VII. PUBLIC INTEREST ANALYSIS A. Overview 422. In addition to determining whether a BOC satisfies the competitive checklist and will comply with section 272, Congress directed the Commission to assess whether the requested authorization would be consistent with the public interest, convenience, and necessity. We conclude that approval of this application is consistent with the public interest. In reaching this determination, we find that compliance with the competitive checklist is, itself, a strong indicator that long distance entry is consistent with the public interest. This approach reflects the Commission's many years of experience with the consumer benefits which flow from competition in telecommunications markets. 423. Nonetheless, the public interest analysis is an independent element of the statutory checklist and, under normal canons of statutory construction, requires an independent determination. Thus, we view the public interest requirement as an opportunity to review the circumstances presented by the application to ensure that no other relevant factors exist that would frustrate the congressional intent that markets be open, as required by the competitive checklist, and that entry will therefore serve the public interest as Congress expected. Among other things, we may review the local and long distance markets to ensure that there are not unusual circumstances that would make entry contrary to the public interest under the particular circumstances of this application. Another factor that could be relevant to our analysis is whether we have sufficient assurance that markets will remain open after grant of the application. While no one factor is dispositive in this analysis, our overriding goal is to ensure that nothing undermines our conclusion, based on our analysis of checklist compliance, that markets are open to competition. As discussed below, we conclude that the public interest would be met by grant of this application. 424. Finally, we note that a strong public interest showing can not overcome a failure to demonstrate compliance with one or more checklist items. The Commission is specifically barred from "limit[ing] . . . the terms used in the competitive checklist," or forbearing from requiring compliance with all statutory conditions under section 271. B. Competition in Local Exchange and Long Distance Markets 425. As set forth below, we conclude that approval of this application is consistent with promoting competition in the local and long distance telecommunications markets. 1. Impact on Local Competition 426. Consistent with our extensive review of the competitive checklist, which embodies the critical elements of market entry under the Act, we find that barriers to competitive entry in the local market have been removed and the local exchange market today is open to competition. We disagree with commenters' arguments that the public interest would be disserved by granting Bell Atlantic's application because the local market in New York has not yet truly been opened to competition. Commenters cite an array of evidence which, they argue, demonstrates that the local telecommunications market is not open and that competition has not sufficiently taken hold in New York. For example, commenters point to: the low percentage of total access lines served by competitive LECs; the concentration of competition in New York City and other urban areas; minimal competition for residential services; modest facilities-based investment; and prices for local exchange service at the maximum permissible levels under the price caps. 427. Congress specifically declined to adopt a market share or other similar test for BOC entry into long distance, and we have no intention of establishing one here. Moreover, pursuant to section 271(c)(2)(B), the Act provides for long distance entry even where there is no facilities-based competition satisfying section 271(c)(1)(A). This underscores Congress' desire to condition approval solely on whether the applicant has opened the door for local entry through full checklist compliance, not on whether competing LECs actually take advantage of the opportunity to enter the market. Although evidence of the type cited by commenters could result from checklist non-compliance or continuing barriers to entry in some circumstances, we have not found this to be the case here. Indeed, commenters do not link these market facts to any sin of omission or commission by Bell Atlantic. We have found nothing in the record to indicate, for example, that the limited competition outside of Manhattan is attributable to a refusal to provide collocation requests outside of Manhattan, or the provision of inferior OSS to competitive carriers upstate. Moreover, while competition for residential end users has proceeded less rapidly than competition for high-volume business end users, we have found that Bell Atlantic has satisfied its statutory obligations and made competitive entry possible in this market sector. Accordingly, we conclude that these indicators do not undermine Bell Atlantic's showing that it has complied with the competitive checklist. 2. Impact on Long Distance Competition 428. We find that the record confirms our view that BOC entry into the long distance market will benefit consumers and competition if the relevant local exchange market is open to competition consistent with the competitive checklist. As a general matter, we believe that additional competition in telecommunications markets will enhance the public interest. Absent checklist compliance, grant of section 271 authority could potentially harm the long distance market because the BOC would have a unique ability to introduce vertical service packages (i.e., long distance and other telecommunications services bundled with local exchange service). This is not the case here - we find that the local market is open and determine that reasonable assurances exist that the market will remain open. We will not require Bell Atlantic to make a substantial additional showing that its participation in the long distance market will produce public interest benefits. We thus decline to address directly the comments and economic studies submitted by Bell Atlantic and by parties opposing Bell Atlantic's application, which seek to demonstrate alternately that Bell Atlantic's entry will have a positive, or a negative, impact on competition in the long distance market. C. Assurance of Future Compliance 429. As set forth below, we find that the performance monitoring and enforcement mechanisms in place in New York, in combination with other factors, provide strong assurance that the local market will remain open after Bell Atlantic receives section 271 authorization. The Commission previously has explained that one factor it may consider as part of its public interest analysis is whether a BOC would continue to satisfy the requirements of section 271 after entering the long distance market. The standard of review employed by the Department of Justice in evaluating Bell Atlantic's application - whether the local market is fully and irreversibly open - also supports this approach. Although the Commission strongly encourages state performance monitoring and post-entry enforcement, we have never required BOC applicants to demonstrate that they are subject to such mechanisms as a condition of section 271 approval. The Commission has, however, stated that the fact that a BOC will be subject to performance monitoring and enforcement mechanisms would constitute probative evidence that the BOC will continue to meet its section 271 obligations and that its entry would be consistent with the public interest. 430. We also believe that it is important to evaluate the benefits of these reporting and enforcement mechanisms in the context of other regulatory and legal processes that provide additional positive incentives to Bell Atlantic. It is not necessary that the state mechanisms alone provide full protection against potential anti-competitive behavior by the incumbent. Most significantly, we recognize that the Commission's enforcement authority under section 271(d)(6) already provides incentives for Bell Atlantic to ensure continuing compliance with its section 271 obligations. We also recognize that Bell Atlantic may be subject to payment of liquidated damages through many of its individual interconnection agreements with competitive carriers. Furthermore, Bell Atlantic risks liability through antitrust and other private causes of action if it performs in an unlawfully discriminatory manner. 1. Summary of Performance Reporting and Enforcement Mechanisms 431. The New York Commission has ordered Bell Atlantic to report performance data, on a monthly basis, pursuant to a series of 152 measurements or metrics. These measurements were developed through the "Carrier-to-Carrier Service Quality" proceeding before the New York Commission, and cover Bell Atlantic's performance on key functions essential to an open, competitive local market: pre-ordering, ordering, provisioning, maintenance and repair, network performance (interconnection trunks), collocation, billing and operator services. Associated with most of these measurements are standards - either benchmarks or retail analogs - also developed through the Carrier-to-Carrier proceeding. 432. The New York Commission also has required Bell Atlantic to submit to a comprehensive performance enforcement mechanism upon receiving authorization to provide interLATA services under section 271. The Amended Performance Assurance Plan ("APAP"), along with the Amended Change Control Assurance Plan ("ACCAP") (collectively, the "enforcement mechanism" or the "enforcement plan"), establish an automatic process under which affected competitors receive bill credits in the event Bell Atlantic fails to satisfy pre-determined performance standards on a set of 122 performance measures - essentially a sub-set of the Carrier-to-Carrier reporting metrics. The procedures and requirements of the Plan are described generally in Bell Atlantic's application and in detail in submissions made to the New York Commission. 2. Key Elements of the Enforcement Plan 433. Where, as here, a BOC relies on performance monitoring and enforcement mechanisms to provide assurance that it will continue to maintain market-opening performance after receiving section 271 authorization, we will review the mechanisms involved to ensure that they are likely to perform as promised. While the details of such mechanisms developed at the state level may vary widely, we believe that we should examine certain key aspects of these plans to determine whether they fall within a zone of reasonableness, and are likely to provide incentives that are sufficient to foster post-entry checklist compliance. In this instance, we believe that the enforcement mechanisms developed in New York will be effective in practice. We base this predictive judgment on the fact that the plan has the following important characteristics: ? potential liability that provides a meaningful and significant incentive to comply with the designated performance standards; ? clearly-articulated, pre-determined measures and standards, which encompass a comprehensive range of carrier-to-carrier performance; ? a reasonable structure that is designed to detect and sanction poor performance when it occurs; ? a self-executing mechanism that does not leave the door open unreasonably to litigation and appeal; ? and reasonable assurances that the reported data is accurate. 434. Parties to this proceeding have identified numerous criticisms relating to the structure and methodologies of these monitoring and enforcement mechanisms, and suggest a long list of possible improvements. None of these criticisms, however, are sufficient to cause us to conclude that the plan will fail to foster post-entry compliance with the checklist requirements. We address each of the major challenges to these plans briefly below. 435. Total Liability At Risk. We conclude that the total of $269 million in potential bill credits placed at risk, on an annual basis, under all components of the performance plans represents a meaningful incentive for Bell Atlantic to maintain a high level of performance. We thus disagree with commenters who suggest that $269 million is insufficient and fails to provide adequate assurance of Bell Atlantic's compliance in the future. Most fundamentally, we disagree with a basic assumption made by several commenters: that liability under the Plan must be sufficient, standing alone, to completely counterbalance Bell Atlantic's incentive to discriminate. The performance plans adopted by the New York Commission do not represent the only means of ensuring that Bell Atlantic continues to provide nondiscriminatory service to competing carriers. In addition to the $269 million at stake under this Plan, as noted above, Bell Atlantic faces other consequences if it fails to sustain a high level of service to competing carriers, including: federal enforcement action pursuant to section 271(d)(6); liquidated damages under 32 interconnection agreements; and remedies associated with antitrust and other legal actions. 436. Nonetheless, we recognize that the level of potential liability under a performance enforcement plan matters, as a plan with relatively low potential liability would be unlikely to provide meaningful incentives to maintain service quality levels. We believe it is useful to compare the maximum liability level to Bell Atlantic's net revenues derived from local exchange service - after all, it is primarily its local service profits that Bell Atlantic would have a theoretical incentive to "protect" by discriminating against competing local carriers. A "Net Return" figure developed using ARMIS data, which represents total operating revenue less operating expenses and operating taxes, is a reasonable approximation of total profits derived from local exchange service. In 1998, Bell Atlantic reported a Net Return of $743 million in New York: $269 million would represent 36% of this amount. On the basis of this comparison, we conclude that $269 million represents a substantial percentage of Bell Atlantic's profits, and agree with the New York Commission that "the dollars at risk in the [APAP] are substantial and should deter [Bell Atlantic's] incentive to provide discriminatory service." 437. We disagree with commenters who suggest that, because the Plan is divided into multiple sub-categories with the overall liability divided into corresponding "sub-caps," Bell Atlantic will never face sizable penalties. We agree that it is important to assess whether liability under an enforcement mechanism such as the APAP would actually accrue at meaningful and significant levels when performance standards are missed. Indeed, an overall liability amount would be meaningless if there is no likelihood that payments would approach this amount, even in instances of widespread performance failure. We do not believe, however, that the Plan suffers from this flaw. The New York Commission has sought to place sizable penalties on the most critical performance areas, thereby ensuring that Bell Atlantic will incur fixed, certain sanctions if its performance slips in these critical areas. In addition, the New York Commission has retained the authority to re-allocate money within the sub-categories, thereby, in its own words, "dramatically increasing [Bell Atlantic's] incentives to maintain or improve service in particular areas." 438. Performance Measurements and Standards. Each performance metric developed through the Carrier-to-Carrier proceeding in New York has a clearly-articulated definition, or "business rule," which sets forth the manner in which the data is to be collected by Bell Atlantic, lists any relevant exclusions, and states the applicable performance standards. The clarity provided by these business rules will help to ensure that the reporting mechanism provides a "benchmark against which new entrants and regulators can measure performance over time to detect and correct any degradation of service rendered to new entrants." While commenters raise concerns about the details of a handful of specific metrics, we note that many of these issues are currently being considered in the ongoing Carrier-to-Carrier proceeding in New York. We applaud the role played by the New York Commission in providing a forum for ongoing modification and improvement of the performance metrics. This is an important feature because it ensures that the Plan can evolve to reflect changes in the telecommunications industry and in the New York market. 439. We also believe that the scope of performance covered by the Carrier-to-Carrier metrics is sufficiently comprehensive, and that the New York Commission reasonably selected key competition-affecting metrics from this list for inclusion in the enforcement plan. We disagree with commenters who suggest that additional metrics must be added to the plan in order to ensure its effectiveness, and note that the New York Commission has considered and rejected similar arguments. Moreover, we note that the New York Commission has indicated that it will consider adding new metrics, if necessary, in the future. Indeed, in light of the ongoing development of xDSL-related measurements related to xDSL-capable loops in New York, we are not concerned that the APAP does not contain such measurements at present. The New York Commission has stated that it expects to adopt measurements addressing xDSL- capable loops once their development is complete. Accordingly, we expect Bell Atlantic to work with the New York Commission in developing performance measurements for xDSL- capable loops, and to incorporate these measurements into its "Carrier-to-Carrier" reports and the APAP. 440. Structural Elements of the Plan. We believe that the structural elements of the Plan appear reasonably designed to detect and sanction poor performance when it occurs. The APAP and the ACCAP set forth, in great detail, the processes by which Bell Atlantic's performance is measured and evaluated, the method for determining compliance and non- compliance with respect to individual metrics, and the manner in which noncompliance with individual metrics will translate into bill credits. Commenters have set forth a long list of specific criticisms, arguing that the Plan: unduly forgives discriminatory conduct; fails to deter targeted discrimination directed against individual competing carriers; excessively aggregates performance data and combines metrics, thereby masking unsatisfactory results; and does not include penalties that escalate with the severity of the performance shortfall. These criticisms, however, do not undermine our overall confidence that the Plan will detect and sanction poor performance when it occurs. We also find it significant that the New York Commission considered and rejected most of these arguments. 441. Self-executing mechanism. We conclude that the performance monitoring and enforcement mechanisms are reasonably self-executing. We recognize, however, that several commenters, as well as the Department of Justice, expressed considerable concern that the "exceptions" or "waiver" process built into the Plan could effectively destroy the self-executing aspect of the plan and open the door to extensive delay and litigation. We agree that a waiver process, if not narrowly limited to a discrete set of circumstances and subject to time constraints, could have such an impact. In this instance, however, we conclude that the waiver process is designed so as to alleviate the concerns noted above. First, the three grounds on which Bell Atlantic may seek a waiver review appear to be reasonable and - with one exception - are defined narrowly under the Plan. The New York Commission has explained that it will consider waiver requests only in "limited, extraordinary circumstances." Second, the New York Commission placed time limits on the resolution of waiver requests, which will help to ensure that the Plan functions in a timely and predictable manner. 442. Data Validation and Audit Procedures. We note with approval that the performance data used in the enforcement mechanism in New York appears to be subject to regular scrutiny. The New York Commission has independently replicated Bell Atlantic's performance reports from raw data submitted by Bell Atlantic, in order to identify and investigate any discrepancies, and will continue to do so for the next six months, and possibly longer. The New York Commission also will perform an annual review of Bell Atlantic's data and performance measures. These review and monitoring mechanisms provide reasonable assurance that the data will be reported in a consistent and reliable manner. 443. Accounting Requirements. Consistent with our accounting rules with respect to antitrust damages and certain other penalties paid by carriers, we conclude that Bell Atlantic should not be permitted to reflect any portion of market adjustments as expenses under the revenue requirement for interstate services of the Bell Atlantic incumbent LEC. Such accounting treatment ensures that ratepayers do not bear, in the form of increased rates, the cost of market adjustments under the APAP and ACCAP in the event Bell Atlantic fails to provide adequate service quality to competitive LECs. We agree with CPI that any other approach would seriously undermine the incentives meant to be created by the Plan. We note that the New York Commission has adopted a similar approach at the state level. D. Other Arguments 444. We recognize that commenters raise several other concerns which, they contend, support a finding that grant of this application is not in the public interest. These arguments do not convince us that grant of this application would be inconsistent with the public interest. Several commenters offer specific allegations that Bell Atlantic has engaged in anti-competitive behavior. We have previously stated that we will not withhold section 271 authorization on the basis of isolated instances of allegedly unfair dealing or discrimination under the Act. In this instance, we do not find that the various incidents cited by commenters constitute a pattern of discriminatory conduct that undermines our confidence that Bell Atlantic's local market is open to competition and will remain so after Bell Atlantic receives interLATA authority. In addition, the City of New York argues that Bell Atlantic's exemption from payment of City franchise fees gives the company an unfair competitive advantage, and thus asks the Commission to require Bell Atlantic to submit to a City franchise arrangement, as a condition of section 271 approval. We conclude that this franchise arrangement is a matter for initial determination between the City of New York and Bell Atlantic and, therefore, we decline to address this issue in the context of this Order. 445. Finally, AT&T asserts that Bell Atlantic's provision of National Directory Assistance (NDA) service violates section 272 and "appears to violate" section 271(a). We note that the Common Carrier Bureau adopted an order finding that Bell Atlantic's provision of NDA service falls within the exception for incidental, interLATA services under section 271(g)(4). As such, Bell Atlantic may provide this service without prior Commission authorization pursuant to section 271. In addition, the Bureau forbore from applying the separate affiliate requirements of section 272, with the exception of the nondiscrimination requirements of section 272(c)(1), to Bell Atlantic's provision of NDA service. Although it is not clear from the record whether Bell Atlantic was in compliance with the requirements of section 271(g)(4) at the time it filed its section 271 application with the Commission, we find that a temporary period of noncompliance does not warrant a finding that granting this application would not be in the public interest. We note that the Commission released an order (U S WEST Forbearance Order), which placed the BOCs on notice that their NDA services could be considered in-region, interLATA services, on September 27th, only two days before Bell Atlantic filed its 271 application. Moreover, since the issuance of the U S WEST Forbearance Order, we find that Bell Atlantic has taken prompt action to restructure its NDA service offering to comply with the Act. Given the particular circumstances present in the instant application, therefore, we find that AT&T's assertions do not provide a sufficient basis for rejecting Bell Atlantic's application. VIII. SECTION 271(D)(6) ENFORCEMENT AUTHORITY 446. Through section 271, Congress withheld from the BOCs, including Bell Atlantic, authority to provide in-region interLATA service until they satisfy various conditions related to competition in local markets. In this manner, Congress sought to create incentives for BOCs to cooperate with competitors and to accelerate acts facilitating the development of local competition. Those incentives may diminish with respect to a given state once a BOC receives authorization to provide interLATA service in that state. The record in this proceeding, for example, evidences considerable concern regarding so-called "backsliding" by Bell Atlantic once it obtains section 271 approval and begins providing in-region interLATA service in New York. Swift and effective post-approval enforcement of section 271's requirements thus is essential to achieve Congress's goal of maintaining conditions conducive to achieving durable competition in local markets. We describe below the post-entry enforcement framework that will govern now that Bell Atlantic has received authorization to provide interLATA service in New York. 447. The Commission's Section 271(d)(6)(A) Powers. Congress included provisions in section 271 to ensure that a BOC continues to comply with the statutory requirements after the Commission approves an application to provide in-region interLATA service. Section 271(d)(6)(A) discusses several actions the Commission is authorized to take should it determine that a BOC "has ceased to meet any of the conditions required for such approval." After "notice and an opportunity for hearing," the Commission "may": (i) issue an order to such company to correct the deficiency; (ii) impose a penalty on such company pursuant to title V; or (iii) suspend or revoke such approval. As the Commission previously has determined, these substantial powers augment the agency's pre-existing enforcement powers, including its authority under sections 206-209 of the Communications Act. 448. Suspension of Approval to Provide InterLATA Service. Section 271(d)(6)(A)(iii) authorizes the Commission to suspend approval to provide interLATA service in the event we determine that a BOC has ceased to meet any of the conditions required for approval. This critically important power underscores Congress's concern that BOCs continue to comply with the statute post-entry. Given this evident congressional concern, we will not hesitate to use this power - and employ it quickly - in appropriate circumstances. 449. We take this opportunity to elaborate on how we intend to implement the "suspension" power under section 271(d)(6)(A)(iii). Specifically, we envision issuing an order similar in effect to the "stand-still" order the Commission issued recently in another context involving section 271. Such a stand-still order would not only prohibit a non-compliant BOC from enrolling additional subscribers for interLATA service, but also could prohibit the BOC from all marketing and promotion of interLATA service. This status would continue until the record is clear that the specified deficiency has been corrected for a sufficient length of time and the stand- still order is dissolved. Such an action involving Bell Atlantic in New York would thus freeze Bell Atlantic's interLATA subscriber base as of the date of the order. 450. Swift action in this area will further Congress's goal to ensure that markets remain open post-entry. Section 271(d)(6)(A) authorizes the Commission to suspend interLATA approval "after notice and an opportunity for hearing." The Commission previously has determined that this language does not require formal, trial-type evidentiary proceedings before an administrative law judge. Section 271(d)(6)(A) does not contain the requisite "on the record after opportunity for an agency hearing" language which triggers trial-type evidentiary hearings under sections 553 and 554 of the Administrative Procedure Act (APA). Nor is there any reason to believe that Congress intended section 271(d)(6) to require trial-type hearings independently of the APA. We thus conclude that generally we may exercise the suspension power of section 271(d)(6)(A)(iii) without holding time-consuming formal, trial-type evidentiary hearings. Rather, we envision expeditious paper proceedings. 451. With respect to this application, any diminution in performance below levels deemed sufficient in this order may expose Bell Atlantic to possible enforcement action under section 271(d)(6), including suspension of authorization to provide service. For instance, our finding of checklist compliance with respect to collocation is predicated on Bell Atlantic's demonstration that it provisions collocation within the 76-day provisioning interval established by the New York Commission 95 percent of the time. We are prepared to institute suspension proceedings in the event of a decrease in this on-time provisioning rate that we believe demonstrates that Bell Atlantic is no longer in compliance with that checklist item. Although we do not attempt to catalogue here all possible ways in which Bell Atlantic may come out of compliance, we emphasize that we view suspension as a potential remedy in any instance where other disincentives have failed to deter decreased performance by Bell Atlantic. 452. Complaints. In addition to FCC-initiated enforcement actions (such as forfeitures, suspensions, and revocations), Congress provided for the expeditious review of complaints concerning failure by a BOC to meet the conditions required for section 271 approval. Such complaints may include requests for damages. The Commission will consider and resolve those complaints alleging violations of section 271 as well as the Commission's rules and orders implementing the statute. Complaints involving a BOC's alleged noncompliance with specific commitments the BOC may have made to a state commission, or specific performance monitoring and enforcement mechanisms imposed by a state commission, should be directed to that state commission rather than the FCC. 453. Conclusion. As these statutory provisions demonstrate, obtaining section 271 authorization is not the end of the road for Bell Atlantic in New York. Congress deemed satisfaction of section 271's requirements at a single moment in time insufficient to ensure continuing competition in local markets. In order to ensure that conditions conducive to local competition in New York are not ephemeral, the statute mandates that Bell Atlantic continue to meet "the conditions required for . . . approval" of its application. Working in concert with the New York Commission, we intend to monitor closely Bell Atlantic's post-entry compliance and to enforce vigorously the provisions of section 271 using the various enforcement tools Congress provided us in the Communications Act. We require that Bell Atlantic provide us with the monthly Carrier-to-Carrier performance data reports that it provides to the New York Commission for at least one year from the date of the release of this order, so that we can review Bell Atlantic's performance to ensure continued compliance with the statutory requirements. IX. CONCLUSION 454. For the reasons discussed above, we grant Bell Atlantic's application for authorization under section 271 of the Act to provide in-region, interLATA services in the state of New York. X. ORDERING CLAUSES 455. Accordingly, IT IS ORDERED that, pursuant to sections 4(i), 4(j), and 271 of the Communications Act of 1934, as amended, 47 U.S.C.  154(i), 154(j), 271, Bell Atlantic New York's application to provide in-region interLATA service in the State of New York filed on September 29, 1999, IS GRANTED. 456. IT IS FURTHER ORDERED that the motion to strike filed by AT&T Corp. on November 22, 1999, IS DENIED. 457. IT IS FURTHER ORDERED that the motion to strike filed by Covad Communications Company on December 17, 1999, IS DENIED. 458. IT IS FURTHER ORDERED that this Order SHALL BECOME EFFECTIVE January 3, 2000. FEDERAL COMMUNICATIONS COMMISSION Magalie Roman Salas Secretary APPENDIX A: LIST OF COMMENTERS Commenter Abbreviation 1. @Link Networks (@Link) 2. American Association of Retired Persons (AARP) 3. AT&T Corporation (AT&T) 4. Adelphia Business Solutions (Adelphia) 5. Allegiance Telecom, Inc. (Allegiance) 6. Alliance for Public Technology (APT) 7. American Council of the Blind, American (ACB) Foundation for the Blind, National Association of the Deaf, Telecommunications for the Deaf, Inc., and World Institute on Disability 8. Association for Local Telecommunications Services (ALTS) 9. Cable and Wireless, Inc. (C&W USA) 10. Cablevision Lightpath, Inc. (Lightpath) 11. Choice One Communications, Inc. (Choice One) 12. City of New York 13. Closecall America, Inc. (Closecall) 14. Coalition to Ensure Responsible Billing (CERB) 15. Competition Policy Institute (CPI) 16. Competitive Telecommunications Association (CompTel) 17. Consortium for School Networking (CoSN) 18. Consumer Federation of America (CFA) 19. CoreComm Limited and CoreComm New York, Inc. (CoreComm) 20. Covad Communications Company (Covad) 21. Destek Networking Group, Inc. (Destek) 22. DSL.net, Inc. (DSL.net) 23. E.Spire Communications, Inc. & (E.Spire) Net 2000 Communications Services, Inc. 24. Excel Communications, Inc. (Excel) 25. Focal Communications Corporation of New York (Focal-NY) 26. General Services Administration (GSA) 27. Global NAPS, Inc. 28. ICG Telecom Group, Inc. (ICG) 29. Intermedia Communications, Inc. (Intermedia) 30. Keefe, Barbara, MainePOINT Project Director University of Maine System 31. Keep America Connected et. al 32. KMC Telecom, Inc. (KMC) 33. League of United Latin American Citizens, Brent Wilkes (LULAC) 34. MCI WorldCom, Inc. (MCI) 35. National ALEC Association (NALA) 36. National Association of Partners in Education (NAPE) 37. National Black Chamber of Commerce (NBCC) 38. National Consumers League (NCL) 39. National Small Business United (NSBU) 40. Nextlink New York, Inc. (NEXTLINK) 41. Network Access Solutions (NAS) 42. New England Conference of Public Utilities Commissioners (NECPUC) 43. NorthPoint Communications, Inc. (NorthPoint) 44. Ntegrity Telecontent Services, Inc. (Ntegrity) 45. Omnipoint Communications, Inc. (Omnipoint) 46. Organization of Chinese Americans, Inc. (OCA) 47. Organizations Concerned about Rural Education (OCRE) 48. Prism Communication Services, Inc. (Prism) 49. RCN Telecom Services, Inc. (RCN) 50. Rhythms Netconnections, Inc. (Rhythms) 51. Santo, Virginia 52. Sprint Communications Company, L.P. (Sprint) 53. State of New York Attorney General, Eliot Spitzer 54. New York Public Service Commission (NYPSC) 55. Telecommunications Resellers Association (TRA) 56. Teligent, Inc. (Teligent) 57. United Seniors Health Cooperative (USHC) 58. Z-Tel Communications, Inc. (Z-Tel) REPLY COMMENTS Commenter Abbreviation 1. AT&T Corporation (AT&T) 2. Allegiance Telecom, Inc. (Allegiance) 3. Association for Local Telecommunications Services (ALTS) 4. BellSouth 5. Communications Workers of America (CWA) 6. Competition Policy Institute (CPI) 7. Conversent Communications, LLC (Conversent) 8. Covad Communications Company (Covad) 9. DSL.net, Inc. (DSL.net) 10. Focal Communications Corporation of New York (Focal) 11. Keep America Connected et. al 12. Level 3 Communications, LLC (Level 3) 13. MCI WorldCom, Inc. (MCI) 14. MediaOne Group, Inc. (MediaOne) 15. National Association of Partners in Education (NAPG) 16. National Council on the Aging (NCOA) 17. National Education Association of New York (NEA/NY) 18. Network Access Solutions (NAS) 19. New York Public Service Commission (NYPSC) 20. NorthPoint Communications, Inc. (NorthPoint) 21. OmniPoint Communications, Inc. (OmniPoint) 22. Prism Communication Services, Inc. (Prism) 23. RCN Telecom Services, Inc. (RCN) 24. Rhythms Netconnections, Inc. (Rhythms) 25. State of New York Attorney General, Eliot Spitzer 26. Teligent, Inc. (Teligent) 27. U S WEST Communications, Inc. (US WEST) APPENDIX B: STATISTICAL METHODOLOGY 1. In this appendix, we discuss the statistical methodology and test statistics that Bell Atlantic employed in its application. We find that the modified z-test that Bell Atlantic uses for measurements with large sample sizes is an appropriate test. We also find that the tests that Bell Atlantic uses for measurements with small sample sizes, the binomial and t-tests, and the permutation tests, are also appropriate tests. We note that, in so concluding, we do not preclude the use of other statistical tests that have been developed in collaborative proceedings in other states. Finally, we discuss how we will use the z-scores provided in the Carrier to Carrier reports to determine if a difference in performance is statistically significant. We conclude that a 95 percent confidence level is the appropriate threshold to use for a determination of statistical significance. 2. When making a parity comparison, statistical analysis is a useful tool to take into account random variation in the metrics. We note that random variation is inherent in the incumbent LEC's process of providing interconnection and access to unbundled network elements. Our concern is primarily that the process that the incumbent LEC employs be nondiscriminatory. Thus, the incumbent LEC could have a provisioning process that is identical in its ability to provide the same function to retail customers and to competitive LECs, but because of random factors outside the control of the BOC, the average completed interval could vary for retail customers and competitive LECs from month to month, such that for one particular month, the metric for competitors would show a longer average interval than would the metric for Bell Atlantic's retail customers. Thus, metric results showing weaker performance to competitors could be due to random variation in the measures, even though the process is inherently nondiscriminatory. Therefore, the use of statistical analysis to take into account random variation in the metrics is desirable. 3. Statistical tests can be used as a tool in determining whether a difference in the measured values of two metrics means that the metrics probably measure two different processes, or instead that the two measurements are likely to have been produced by the same process. This can be done using traditional hypothesis testing. Hypothesis testing involves testing to determine which of two hypotheses, usually called the null and the alternative hypotheses, is likely to be correct. Usually this means devising a statistical test to determine whether the null hypothesis can be rejected, given the data available. If the data is not consistent with the null hypothesis, then we reject the null hypothesis, and accept the alternative hypothesis. The null hypothesis here would be the hypothesis that the two processes are the same, so that the measurements reflect different observations taken from the same (or identically performing) processes. The alternative hypothesis asserts that the two processes are different. 4. In Second BellSouth Louisiana Order, we encouraged BOCs to submit data allowing us to determine if any detected differences in performance are caused by random variation in the data. In its application, Bell Atlantic has presented us with performance data, as well as a statistical test and its corresponding test statistic (called z-scores) that can be used to determine whether a detected difference between the wholesale and retail metrics is statistically significant. Bell Atlantic has been required to utilize this statistical methodology in reporting its performance to New York as part of the Carrier-to-Carrier proceeding. 5. The statistical test that is used depends on the kind of metric being tested, and the number of observations or "sample size" for that metric. The Carrier to Carrier guidelines specify that there are two kinds of metrics, "measured" and "counted." Measured metrics are averages or means of observations (for example, Average Completed Interval). Proportionate (counted) metrics measure the proportion or percentage of a group of observations that meet some criterion (for example, Percentage of Appointments Missed). 6. The statistical tests used by Bell Atlantic were initially proposed by Local Competition Users Group (LCUG), a group of competitive LECs. The test LCUG advocated for large sample sizes is commonly known as the "modified z-test", which uses the "modified z statistic." The modified z-test uses only the incumbent LEC's standard deviation, and not the competitive LECs' standard deviation, in calculating the z statistic. It is a variation of the standard textbook z-test, which uses the standard deviations for both the incumbent LEC's and competitive LECs' observations. In its application Bell Atlantic presents us with z-scores, which are the test statistic used to perform the z-test. 7. The modified z-test for a difference in means between two populations, assuming the means are normally distributed, used for measured metrics, is: z = (mC-mI) / (sI * SQRT [1/NC+1/NI]) where mC?=?competitive LEC sample mean, mI?= incumbent LEC sample mean, sI = incumbent LEC's standard deviation, NC = number of competitive LEC observations, and NI = number of incumbent LEC observations. z is the test statistic ("z-score") that results from this calculation. 8. The modified z-test for a difference in proportions between two populations, used for proportionate metrics, is: z = (PC-PI) / SQRT [PI(1-PI) (1/NC+1/NI)] where PC = competitive LEC sample proportion, PI = incumbent LEC sample proportion, NC = number of competitive LEC observations, NI = number of incumbent LEC observations, and z is the resulting z-score. 9. The z-test involves comparing the z-score for a particular metric with a critical value (call it zC) to determine if we can reject the (null) hypothesis that the same process generated the Bell Atlantic and competing carrier means. The critical value zC is chosen based on a particular desired confidence level (call the confidence level C). If the z-score is less than this critical value (z?. See KPMG Final Report RMI1 VII-9 (Test R1-7); New York Commission Comments at 56. KPMG Final Report at RMI1 VII-9 (Test R1-7); Bell Atlantic Miller/Jordan Decl. at para. 101; New York Commission Comments at 56. Draft specifications, for instance, are shared by electronic mail with approximately 290 individual competing carriers that participate in the change management process. Bell Atlantic Miller/Jordan Decl. at para. 98. New York Commission Comments at 62. New York Commission Comments at 62. A stable testing environment means that no changes by the BOC are permitted after the testing period commences. See generally U S WEST Sept. 27 Letter; NY Attorney General Comments at 17 (describing the importance of testing opportunities for competing carriers). See generally Department of Justice Evaluation at 35 ("testing is necessary to prevent major service disruptions when Bell Atlantic makes changes in its side of the interface"). KPMG Final Report P1-2 at IV-17 (Test P1-2); New York Commission Comments at 59. New York Commission Comments at 59. New York Commission Comments at 60; Department of Justice Evaluation at 36. The test procedures developed provide for the availability of a test environment that mirrors production, a baseline validation test deck (a compilation of transactions designed to test whether a new release produces expected results) with test account data so competing carriers can test transactions of their choice, and protocols for identifying and resolving issues during testing. Both the baseline validation test deck and a progression test deck are made available to competing carriers on the Bell Atlantic TIS web page. New York Commission Comments at 60. Id. KPMG Exception Closure Report 21 at 3 (as referenced in KPMG Final Report at POP1 IV-18 (Test P1-2)). Id. AT&T Crafton/Connolly Aff. at para. 232; MCI WorldCom Lichtenberg/Sivori Decl. at para. 148 (interim QA test environment allotted only 30 hours over a 5-day period for competing carrier testing and a maximum of 3 hours of technical support). Commenters also claim that the interim QA testing environment was inadequate because orders submitted in production that had previously proved successful in testing were rejected and that Bell Atlantic failed to provide sufficient resources for competing carriers to conduct thorough carrier-to-carrier testing. Allegiance Comments at 8-9; AT&T Crafton/Connolly Aff. at para. 234; MCI WorldCom Lichtenberg/Sivori Decl. at para. 148. Bell Atlantic Miller/Jordan Decl. at para. 106; Department of Justice Evaluation at 36; AT&T Crafton/Connolly Aff. at para. 235. Bell Atlantic Miller/Jordan Decl. at para. 106. Bell Atlantic Miller/Jordan Decl. Attach. G at 89-91 (describing versioning under the Change Agreement). Certification testing is a process conducted jointly by Bell Atlantic and competing carriers to determine whether or not a competing carrier's OSS are capable of submitting valid service orders and receiving responses using Bell Atlantic's EDI interface. KPMG Final Report at POP1 IV-3. KPMG determined that Bell Atlantic failed to offer a repeatable process for planning and coordinating certification testing activities and that Bell Atlantic lacked clearly defined entrance and exit criteria designed to certify that a competing carrier can move from the testing environment to the production environment. KPMG Final Report at POP1 IV-17 (Test P1-1); see also KPMG Exception Report 22 (as referenced in KPMG Final Report at POP1 IV-17 (Test P1-1)). Based on KPMG's findings, industry comment, and competing carrier input, Bell Atlantic issued new procedures in May 1999. KPMG reviewed and validated these procedures. New York Commission Comments at 61-62; KPMG Exception Closure Report 22 (as referenced in KPMG Final Report at POP1 IV-17 (Test P1-1)). MCI WorldCom Comments at 19; MCI WorldCom Lichtenberg/Sivori Decl. at para. 127; see also AT&T Crafton/Connolly Aff. at para. 195. TRA Comments at 11 n.38. See also MCI WorldCom Comments at 20-21; NY Attorney General Comments at 17. Under the Change Agreement, Bell Atlantic must provide competing carriers initial notification of most upcoming changes at least 66 days prior to the implementation of the change. For these changes, Bell Atlantic must also distribute final documentation describing the change in detail 45 days prior to implementation. For emergency changes, however, the Change Agreement only requires that Bell Atlantic notify competing carriers at any time prior to implementation. For regulatory changes, notification and documentation intervals may be set by the New York Commission or other regulatory authority. Changes in industry standards may also proceed on a different schedule. See generally Bell Atlantic Miller/Jordan Decl. Attach. G at 15-20. See Order Adopting Inter-Carrier Service Quality Guidelines, Case 97-C-0139 (NYPSC Feb. 16, 1999) (Bell Atlantic Application, App. E, Tab 61) (NYPSC Guidelines Order); Order Establishing Permanent Rule, Case 97-C-0139 (NYPSC Jun. 30, 1999) (Bell Atlantic Application, App. E, Tab 83) (NYPSC Permanent Rule Order). See generally KPMG Final Report at RMI1 VII-10 (Table VII-1.9); Bell Atlantic Dowell/Canny Decl. Attach. D at 84-85, 96-97 (metrics PO-4-01, PO-4-02, PO-4-03); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 1-2 (metrics PO-4-01, PO-4-O2, and PO-4-03). Type 4 changes are those that Bell Atlantic seeks to implement on its own accord, rather than at the request of regulatory authorities, industry standard organizations, or competing carriers themselves. See generally Bell Atlantic Miller/Jordan Decl. Attach. G at 6. KPMG Final Report at RMI1 VII-8. KPMG Final Report at RMI1 VII-10. Id. Bell Atlantic Miller/Jordan Decl. at para. 102. Bell Atlantic asserts that the deficiencies identified by KPMG resulted from Bell Atlantic missing several dates for the distribution of documentation in February 1999, and excluding updates to RETAS documentation from the change management process. According to Bell Atlantic, it now includes RETAS documentation in the change management process. Bell Atlantic Miller/Jordan Decl. at para. 102; New York Commission Comments at 57. In addition, we note that billing changes also are now a part of the change management process in New York. Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 81. Because the sample sizes in any given month for Type 4 changes are so small, we prefer to review Bell Atlantic performance over the course of several recent months rather than in any one individual month. This also provides us with a better comparison to the data provided in the KPMG Final Report. KPMG Final Report at RMI1 VII-10 (Table VII-1.9); Bell Atlantic Dowell/Canny Decl. Attach. D at 84, 96; Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 1 (observations listed for metric PO-4-01). In response to commenters' claims regarding untimely change notification and documentation, Bell Atlantic submitted data showing its Type 4 notification was timely in two of two Type 4 changes that occurred in October through October 19, 1999. Letter from Penny Rubin, Managing Attorney, New York Commission, to Magalie Roman Salas, Secretary, Federal Communications Commission, CC Docket No. 96-98 Attach. at 1 (filed Nov. 30, 1999) (New York Commission Nov. 30 Ex Parte Letter); Letter from Dolores A. May, Director, Federal Regulatory, to Magalie Roman Salas, Secretary, Federal Communications Commission, CC Docket No. 99-295 (filed Dec. 14, 1999) at 1 (Bell Atlantic Dec. 24 Ex Parte Letter). The timeliness of Bell Atlantic documentation for Type 4 changes is still listed as under development in July 1999. Bell Atlantic Dowell/Canny Decl. Attach. D at 85, 97; Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 2. In response to commenters' claims regarding untimely change notification and documentation, Bell Atlantic submitted data showing its Type 4 change documentation was timely for two of two changes that occurred in October through October 19, 1999. New York Commission Nov. 30 Ex Parte Letter, Attach. at 2; Bell Atlantic Dec. 14 Ex Parte Letter at 1. In response to commenters' claims regarding untimely change notification and documentation, Bell Atlantic submitted data showing its Type 4 change documentation was timely for two of two changes that occurred in October through October 19, 1999. New York Commission Nov. 30 Ex Parte Letter, Attach. at 2; Bell Atlantic Dec. 14 Ex Parte Letter at 1. Bell Atlantic Miller/Jordan Decl. at paras. 100, 106; MCI WorldCom Lichtenberg/Sivori Decl. at para. 135. Bell Atlantic Miller/Jordan Decl. Attach. G at 19-20. See, e.g, Bell Atlantic Dowell/Canny Decl. Attach. D at 84, 96 (listing the standard for timely notification of emergency changes as "Notification before implementation"). KPMG Final Report at RMI1 VII-10 (Table VII-1.9). AT&T Comments at 32-33; MCI WorldCom Comments at 23; MCI WorldCom Lichtenberg/Sivori Decl. at paras. 62-63. Documentation timeliness for Type 1 changes is not reported in the Carrier-to-Carrier metrics, because it is not applicable. Bell Atlantic Dowell/Canny Decl. Attach. B at 12 n.5. MCI WorldCom Reply at 12; Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 1. Bell Atlantic Dowell/Canny Decl. Attach. D at 84 (metric PO-4-01). In response to commenters' claims regarding untimely change notification and documentation, Bell Atlantic submitted data showing that its Type 1 change notification was timely for six of six changes that occurred between October 1-19, 1999. New York Commission Nov. 30 Ex Parte Letter Attach. at 1; Bell Atlantic Dec. 14 Ex Parte Letter (listing October 1-19, 1999 observations for metric PO-4-01). In addition, not all emergency releases result in system changes, thus limiting the inconvenience imposed on competing carriers by Type 1 changes. See, e.g., Bell Atlantic Nov. 24 Ex Parte Letter at 4-5 (describing September 1999 Type 1 changes). Moreover, we expect Bell Atlantic's new practice to notify by pager key individuals at competing carriers when an emergency change occurs, and to conduct a conference call whenever there is an immediate software change, will minimize the impact of Type 1 change notification difficulties on competing carriers. Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at paras. 69-71; MCI WorldCom Dec. 14 Ex Parte Letter at 10. Department of Justice Evaluation at 34 (expressing concern that Bell Atlantic has not yet demonstrated that it is able to provide competing carriers with "relatively stable and predictable documentation"). See also AT&T Comments at 28; MCI WorldCom Comments at 20-21; NY Attorney General Comments at 17; Sprint Comments at 22; TRA Comments at 11; AT&T Crafton/Connolly Aff. at paras. 119-121, 124-132; MCI WorldCom Lichtenberg/Sivori Decl. at paras. 57-60. New York Commission Comments at 57. The standard adopted by the New York Commission for both the Carrier-to-Carrier metrics and the Change Control Assurance Plan is 95 percent change management notification and documentation sent on time with no delays greater than 8 days. Bell Atlantic Dowell/Canny Decl. Attach. B at 11 (listing metric PO-4 performance standard); Bell Atlantic Dowell/Canny Decl. Attach. C, Ex. 2 (Appendix A to Amended Change Control Assurance Plan). MCI WorldCom Comments at 24-25; AT&T Crafton/Connolly Aff. at para. 235; AT&T Reply at 25; MCI WorldCom Reply at 11-12; AT&T Crafton/Connolly Reply Aff. at para. 24 n.14. In response to commenters' claims regarding lack of evidence that the permanent QA testing environment actually works as represented in the Bell Atlantic Application, the New York Commission submitted information regarding successful competing carrier use of the permanent QA testing environment for the October 16, 1999 software release. New York Commission Reply at 19. Id. at 19 n.2. Id. See generally KPMG Exception Closure Report 21 (as referenced in KPMG Final Report at POP IV-18 (Test P1-2)) (evaluating and finding generally satisfactory improved interim QA testing environment). Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 93. A test deck is a compilation of transactions designed to test whether a new release produces expected results. New York Commission Comments at 60. See Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 93. Id. New York Commission Comments at 59-60. Department of Justice Evaluation at 36. In response to commenters' claims regarding lack of evidence that the permanent QA testing environment actually works as represented in the Bell Atlantic Application, the New York Commission submitted information regarding successful competing carrier use of the permanent QA testing environment for the October 16, 1999 software release. New York Commission Reply at 19. In addition, unlike the Department of Justice, we consider the similarity of the interim QA testing environment to the permanent QA testing environment. See Sprint Comments at 20-21; AT&T Crafton/Connolly Aff. at para. 199. Bell Atlantic Miller/Jordan Decl. Attach. G at 7-8, 19-20, 40-45, 80-88. AT&T Crafton/Connolly Aff. at para. 201. See generally AT&T Crafton/Connolly Aff., Attach. 8. Based on feedback from competing carriers, Bell Atlantic also agreed to add a pager notification system to ensure that key individuals at competing carriers receive notice of emergency changes as soon as possible. Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 69. Bell Atlantic Dowell/Canny Decl. Attach. D at 84, 96; Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 1. In response to commenters' claims regarding Type 1 change frequency, Bell Atlantic submitted data showing that only six Type 1 changes occurred in October through October 19, 1999. New York Commission Nov. 30 Ex Parte Letter Attach. at 1; Bell Atlantic Dec. 14 Ex Parte Letter at 1 (observations listed for metric PO-4-01). MCI WorldCom Comments at 20-21; AT&T Crafton/Connolly Aff. at paras. 207-212. Bell Atlantic Miller/Jordan Decl. at para. 100; MCI WorldCom Lichtenberg/Sivori Decl. at para. 135. Further, as described above, Bell Atlantic and competing carriers participated in a series of workshops to come up with a more narrow definition of Type 1 emergency changes. AT&T Crafton/Connolly Aff. at para. 201. See generally AT&T Crafton/Connolly Aff. Attach. 8. MCI WorldCom Comments at 19; MCI WorldCom Lichtenberg/Sivori Decl. at para. 127; Bell Atlantic Miller/Jordan Decl. Attach. G at 4. Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 86. Id. Allegiance Comments at 8-9; Sprint Comments at 18-20 (describing difficulties with Bell Atlantic's decision to skip LSOG 3); MCI WorldCom Reply Comments at 15-17, 20 (alleging problems with implementation of the GUI III interface, parsed CSR); MCI WorldCom Dec. 14 Ex Parte Letter at 11 (criticizing Bell Atlantic change management notification proposal involving closing trouble tickets without root cause analysis). See, e.g., Bell Atlantic Reply at 39 n.43 (noting that Sprint complaints regarding LSOG 3 must be viewed in light of a general consensus reached by competing carriers in the change management process). Ameritech Michigan Order, 12 FCC Rcd at 20616; Second BellSouth Louisiana Order, 13 FCC Rcd at 20654. Second BellSouth Louisiana Order, 13 FCC Rcd at 20655 (citing Ameritech Michigan Order, 12 FCC Rcd at 20619; Local Competition First Report and Order, 11 FCC Rcd at 15660; Local Competition Second Reconsideration Order, 11 FCC Rcd at 19742). Demonstration of adequate technical assistance and help desk support is also part of the BOC's "obligation `to provide competing carriers with the specifications necessary to instruct competing carriers on how to modify or design their systems in a manner that will enable them to communicate with the BOC's legacy systems and any interfaces utilized by the BOC for such access.'" BellSouth South Carolina Order, 13 FCC Rcd at 628; Ameritech Michigan Order, 12 FCC Rcd at 20617. Volume I provides basic information competing carriers need to know about doing business with Bell Atlantic, Volume II addresses the interfaces available to competing carriers for obtaining access to Bell Atlantic's OSS and provides information on how to obtain the technical specifications for them, and Volume III provides business rules for ordering Bell Atlantic products. Bell Atlantic Miller/Jordan Decl. at paras. 87-89. Bell Atlantic Miller/Jordan Decl. at 87-88. Moreover, Bell Atlantic has addressed many of the problems with its business rules and EDI specification documentation identified during the KPMG review, resulting in more accurate documentation for competing carriers seeking to access Bell Atlantic's OSS. See generally KPMG Final Report at POP9 IV-227-228 (Tests P9- 12, P9-14, P9-17-23). Bell Atlantic Miller/Jordan Decl. at para. 92. Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 97. Id. The KPMG Final Report found that Bell Atlantic documents for competing carriers failed to provide useful contact lists and help desk numbers. KPMG characterized this problem as "Not Satisfied." KPMG Final Report at POP9 IV-220 (Test P9-16). Bell Atlantic Miller/Jordan/Zanfini Reply Decl. Attach. U (listing from web site of help desk and assistance information for competing carriers). Adelphia Comments at 3 (alleging difficulties reaching the appropriate contact person at Bell Atlantic when problems arise that require technical assistance); AT&T Comments at 29 (alleging problems with help desk errors); MCI WorldCom Comments at 23-24 (citing KPMG Final Report); TRA Comments at 12-13 (citing KPMG Final Report); Z-Tel Comments at 14-16 (alleging inadequate wholesale account support); AT&T Reply at 26 (citing KPMG Final Report); MCI Dec. 14 Ex Parte Letter at 12 (criticizing Bell Atlantic help desk attendants). BellSouth South Carolina Order, 13 FCC Rcd at 589; see also Second BellSouth Louisiana Order, 13 FCC Rcd at 20660 (referring to "pre-ordering and ordering" collectively as "the exchange of information between telecommunications carriers about current or proposed customer products and services or unbundled network elements or some combination thereof."). Pre-ordering consists of several functions and, in prior orders, the Commission has identified the following five functions: (1) customer service record (CSR) information; (2) address validation; (3) telephone number information; (4) due date information; and (5) services and feature information. Second BellSouth Louisiana Order, 13 FCC Rcd at 20660; First BellSouth Louisiana Order, 13 FCC Rcd at 6274; BellSouth South Carolina Order, 13 FCC Rcd at 619. See Second BellSouth Louisiana Order, 13 FCC Rcd at 20669. BellSouth South Carolina Order, 13 FCC Rcd at 619; see also Second BellSouth Louisiana Order, 13 FCC Rcd at 20655; Ameritech Michigan Order, 12 FCC Rcd at 20618-19. See Second BellSouth Louisiana Order, 13 FCC Rcd at 20661-67; BellSouth South Carolina Order, 13 FCC Rcd at 623-29 (concluding that failure to deploy an application-to-application interface denies competing carriers equivalent access to pre-ordering OSS functions). Moreover, the Commission also found that, without access to an application-to-application interface, a competing carrier would be unable to develop its own customized interface that its staff could use nationwide, and would be required to train its staff on a BOC's proprietary system. See Second BellSouth Louisiana Order, 13 FCC Rcd at 20662 n.291; BellSouth South Carolina Order, 13 FCC Rcd at 624-25. Bell Atlantic describes the Web GUI as "a graphical interface that a [competing carrier] can access from a personal computer via a dedicated/private line or a secure dial-up line, using either Netscape Communicator 4.0 or higher, or Microsoft IE Version 4.0 or higher." Bell Atlantic Miller/Jordan Decl. at para. 23. Although Z-Tel complains that the Secure ID system for carrier access to the Web GUI is inefficient and costly, Bell Atlantic recently eliminated the need for Secure IDs by enabling carriers to access the Web GUI via the Internet using a URL address and password. See Z-Tel Comments at 16-17; Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 55. Bell Atlantic states that it provided Z-Tel with passwords on September 20, 1999. Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 55. Bell Atlantic Miller/Jordan Decl. at para. 21. Bell Atlantic implemented the transaction formats specified in Local Service Ordering Guidelines (LSOG) version 3 (address validation, appointment scheduling, feature/service availability and telephone number reservation/selection), and worked with MCI WorldCom to develop EDI specifications and business rules for additional functionality (CSR retrieval, loop qualification information, directory listing information, and service order inquiry and installation status). Bell Atlantic Miller/Jordan Decl. at para. 21. Carriers' options for connecting with Bell Atlantic's EDI interface are: direct connection (dial-up or dedicated); Value Added Networks (VANs); public network (Internet) connectivity; and Interactive Agent connectivity using Secure Socket Layer 3 (SSL3) technology. Bell Atlantic Miller/Jordan Decl. at para. 27. Bell Atlantic Application at 37. Bell Atlantic Miller/Jordan Decl. at para. 20 (indicating CORBA testing in progress with AT&T); Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 23 (stating that CORBA is available to any requesting carrier). AT&T claims that CORBA is superior to EDI in that it "provides faster transmission responses to queries, and it is a more flexible standard that permits fine-tuning to improve data transmission." AT&T Crafton/Connolly Aff. at para. 86. Bell Atlantic Application at 37 n.36. Bell Atlantic's back office pre-ordering systems include: LiveWire (formerly PREMIS) for address validation and telephone number selection and reservation; Work Force Administration (WFA) for service installation status; Customer Record Information System (CRIS) or Carrier Access Billing System (CABS) for customer service records; Direct Order Entry system (DOE) for service and feature availability; SOP for due date availability and service order inquiry; Automated Telephone Listing and Address System (ATLAS) for directory listing information; and PHOENIX for ISDN and ADSL loop qualification. See Bell Atlantic Miller/Jordan Decl. Attach. B. In August 1999, Bell Atlantic began replacing the PREMIS system with LiveWire, which, among other things, enhances Bell Atlantic's address validation capabilities. CSRs depict the end user's account with Bell Atlantic, including billing name and address, billing and working telephone numbers, a list of services provided to the end user, and the end user's presubscribed interexchange carrier and local presubscribed interexchange carrier. Bell Atlantic Miller/Jordan Decl. at para. 17. Bell Atlantic implemented "parsed" CSR functionality in May 1999. Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 18. With parsed CSRs, pre-order customer information is separated into identifiable fields (e.g., street number, street name) can automatically populate an order form. See MCI WorldCom Comments at 27 n.36; MCI WorldCom Reply at 17. This function allows competing carriers to select a telephone number from up to five available numbers. The selected number is then removed from the pool of available numbers and, if the carrier subsequently submits an order, assigned to the carrier. Letter from Dee May, Director, Federal Regulatory Affairs, Bell Atlantic, to Magalie Roman Salas, Secretary, Federal Communications Commission, CC Docket No. 99-295, at 3-4 (filed Nov. 24, 1999) (Bell Atlantic Nov. 24 Ex Parte Letter) (indicating that Bell Atlantic retail representatives obtain a telephone number using the same process and that, with the implementation of LiveWire, residential numbers are removed from the pool for three months and business numbers for twelve months). Bell Atlantic Application at 37 n.36. We note that the seven pre-ordering functions that Bell Atlantic provides to itself go beyond the five functions previously identified by the Commission. See supra n. 371. We do not consider the Web GUI's functionality in this section because Bell Atlantic does not represent that the Web GUI is an application-to-application interface. We note, however, that the Web GUI provides an economically efficient pre-ordering interface for low-volume carriers and new entrants. See Ameritech Michigan Order, 12 FCC Rcd at 20661; see also AT&T Crafton/Connolly Aff. at para. 73; Department of Justice Evaluation at 34 n.92; New York Commission Comments at 37; Z-Tel Comments at 16 (noting the Web GUI's suitability for use by small carriers). KPMG conducted a comprehensive functional evaluation and verified that the Web GUI pre-ordering interface enables carriers to perform the seven pre-ordering functions. See KPMG Final Report at POP2 IV-20-41. Bell Atlantic Miller/Jordan Decl. at para. 22. We do not rely on Bell Atlantic's unsubstantiated claims that carriers are also using the EDI pre-ordering interface for telephone number reservation and selection and due date availability. See Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 14. MCI WorldCom Comments at 27, 31; MCI WorldCom Lichtenberg/Sivori Reply Decl. at paras. 10, 13. Although MCI WorldCom recently discovered that its parsed CSR functionality does not cover all order types, it does not assert that it is incapable of adding such functionality or that the exclusion of ISDN orders will impede its ability to compete in the local services market. AT&T Comments at 26; AT&T Crafton/Connolly Aff. at para. 87 (indicating that AT&T deployed CORBA for commercial production for address validation in September 1999, and for parsed CSR retrieval during the first week of October 1999). See Bell Atlantic Miller/Jordan Decl. Attach. A at 2 (Donnellan Affidavit). See KPMG Final Report at Executive Summary II-3; Bell Atlantic Application App. C, Tab 654, Hewlett Packard Consulting, "CTTG Project Final Report," Final Version (Apr. 20, 1999) (HP CTTG Final Report). See KPMG Final Report at POP5 IV-75-137 (EDI Functional Evaluation and Normal Volume Test); POP6 IV138-149 (EDI Stress Test); see also New York Commission Comments at 37-38. In particular, KPMG tested the following pre-order functions: address validation; telephone number selection and reservation; directory listing inquiry; service scheduling and due date availability; feature and service availability; customer service record retrieval; carrier access billing retrieval; installation status request; loop qualification and reservation channel facility inquiry; and service order inquiry. KPMG Final Report at POP5 IV-77-78. KPMG also retrieved a limited number of parsed CSRs, and confirmed Bell Atlantic's ability to provide parsed CSR functionality. KPMG Final Report at POP5 IV-135. MCI WorldCom Comments at 28. For instance, MCI WorldCom claims that KPMG did not attempt to design the transport and security necessary for the interface in actual production. Id. See New York Commission Comments at 33-34, 38; KPMG Final Report at POP5 IV-102 (Table IV-5.10) (indicating that KPMG sent 3,400 transactions over the pre-ordering interfaces during its functional evaluation, and more than 23,000 during the volume tests). See MCI WorldCom Comments at 28 (claiming that, because Bell Atlantic "showed favoritism" to the testers, KPMG's ability to construct an EDI interface for all pre-ordering functions does not demonstrate that Bell Atlantic provides the documentation and support necessary for other carriers to build all functionality for use in a production environment). See KPMG Final Report at Executive Summary II-3; HP CTTG Final Report, Overview  1.4 at 3. See KPMG Final Report at Executive Summary II-8 ("For the most part we believe that the quality of service we received during the test was comparable to that generally received by CLECs. However, on several occasions we believe that we received better treatment than a normal CLEC. For example, BA-NY resources assigned to handle many of our problem escalations were very senior BA-NY resources."). Rather, to the extent that Bell Atlantic incorporated the testers' suggestions for enhancing its documentation, we find that competing carriers benefited significantly from the third-party testers' construction and testing of the interface. See Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 12 (indicating that Bell Atlantic incorporated Hewlett Packard's suggestions into its EDI documentation). See New York Commission Comments at 31-34. KPMG Final Report at POP3 IV-42 (noting that the Live CLEC Functional Evaluation "allowed for an element of blind testing and tracking performance in a `real world' environment."). See KPMG Final Report at POP3 IV-42-64. See Ameritech Michigan Order, 12 FCC Rcd at 20601-02, 20614 (explaining that a BOC's duty to "provide" a checklist item where no competitor is actually using the item requires that it demonstrate that it makes the item available as both a legal and practical matter); id. at 20618 (recognizing that a BOC need not ensure that competing carriers are currently using every OSS function as long as the BOC can demonstrate that the lack of use is a result of carriers' business decisions). MCI WorldCom, for example, claims that retrieving parsed CSRs is the most important pre-ordering function, and that lack of application-to-application access to service and feature information is "not nearly as problematic" and "has not proven to be a commercial necessity." MCI WorldCom Lichtenberg/Sivori Reply Decl. at para. 6. See also MCI WorldCom Lichtenberg/Sivori Decl. at para. 69. AT&T Crafton/Connolly Aff. at para. 88. MCI WorldCom further notes that its deployment schedule has been affected by a self-imposed "Y2K moratorium" on software changes that began on October 1, 1999, although it was able to secure an exception to implement EDI address validation on November 1, 1999. MCI WorldCom Lichtenberg/Sivori Decl. at para. 96. Nevertheless, MCI WorldCom implies that application-to-application access to telephone number selection, due date availability, and address validation for new customers could be implemented as early as the first quarter of 2000, and the other pre-ordering functions later that year. Id.; MCI WorldCom Reply at 20-21. See also Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 16 (indicating that MCI WorldCom has completed EDI testing for telephone number reservation and selection, due date availability and directory listing information). See New York Commission Comments at 48. See Second BellSouth Louisiana Order, 13 FCC Rcd at 20661-67; First BellSouth Louisiana Order, 13 FCC Rcd at 6275-79; BellSouth South Carolina Order, 13 FCC Rcd at 602, 620-29. See BellSouth South Carolina Order, 13 FCC Rcd at 620. See Second BellSouth Louisiana Order, 13 FCC Rcd at 20661, 20666, 20676-77; First BellSouth Louisiana Order, 13 FCC Rcd at 6276-77; BellSouth South Carolina Order, 13 FCC Rcd at 602, 623-24, 629 (finding that, in addition to increased costs and delays, manual retyping of information can contribute to a high error rate); see also AT&T Comments at 26; AT&T Crafton/Connolly Aff. at paras. 70, 73, 81 (noting that, absent integration, a carrier would incur substantial costs, delays, and risks of error by entering data twice - once into Bell Atlantic's OSS and again into the carrier's own systems); MCI WorldCom Comments at 26; MCI WorldCom Lichtenberg/Sivori Decl. at paras. 9-10, 21 (claiming that manual re-entry of pre-ordering information hinders a carrier's ability to reach commercial volumes of orders). BellSouth South Carolina Order, 13 FCC Rcd at 623. See Bell Atlantic Miller/Jordan Decl. at para. 22. Bell Atlantic submitted the testimony of Michael H. Donnellan, Vice President of Operations for CTC Communications, describing CTC's development of an EDI pre- ordering interface through which "Bell Atlantic data is seamlessly inserted into CTC systems." Bell Atlantic Miller/Jordan Decl. Attach. A at 3. Specifically, Donnellan asserts that "the information requested through a CSR flows in a file from Bell Atlantic's pre-order systems into CTC's information systems," where it is "reviewed on line and then an EDI order is created." Id. Donnellan also cites "Bell Atlantic's demonstrated effort" in assisting CTC through the development and testing stages. Id. We expect that Bell Atlantic will provide all necessary documentation and technical assistance to other carriers that seek to integrate pre-ordering and ordering functions. See, e.g., Letter from Lori Wright, Senior Manager, Regulatory Affairs, MCI WorldCom, to Magalie Roman Salas, Secretary, Federal Communications Commission, CC Docket No. 99-295 (filed Nov. 24, 1999) (MCI WorldCom Nov. 24 Ex Parte Letter) (indicating that MCI WorldCom has successfully integrated parsed CSR retrieval and address validation using EDI); AT&T Crafton/Connolly Reply Aff. at para. 32 (indicating that AT&T has successfully integrated parsed CSR retrieval and address validation using CORBA). See supra at para. 136 (discussing carriers' internal business decisions to delay deployment of other application-to-application functionality, some of which MCI WorldCom has completed testing). For example, KPMG stated: For [competing carriers] attempting to integrate the EDI pre-order and order processes, efficiencies can be achieved by automating the population of order input fields with information returned in the pre-order response forms. [Bell Atlantic] has published a guide that identifies the transport format of an EDI transaction. [Bell Atlantic] has also published business rules documents that specify how [competing carriers'] pre-orders and orders should be structured. KPMG Final Report at POP5 IV-76 (footnotes omitted). As KPMG reported: [A] limited number of integrated pre-order/order transactions were conducted. In these cases, the information returned in the pre-order response was manually copied, without modifications, into the Local Service Request (LSR). This test was conducted to highlight any inconsistencies in field name and format between pre-order and order forms. KPMG Final Report at POP5 IV-79; see also id. at POP5 IV-90 (identifying the integrated pre-order/order scenarios tested). KPMG identified certain field name and format inconsistencies, but found that the problems could be addressed by building a logical interface between pre-order responses and orders. Id. at POP5 IV-119- 121; POP5 IV-128-130 (Table IV-5.20). See AT&T Comments at 13, 22, 26 (claiming that CORBA cannot be "fully" integrated with the EDI ordering interface); AT&T Crafton/Connolly Aff. at paras. 82-83, 88, 91 n.51 (claiming that inconsistencies in the data elements for pre-ordering and ordering preclude full integration). Without uniformity, the pre-ordering data cannot automatically populate an order form but instead must be translated into the proper field characteristics for ordering. See AT&T Crafton/Connolly Aff. at paras. 79-85. AT&T nonetheless admits that it has not yet tested whether it can integrate the remaining pre-ordering functions using CORBA. See AT&T Crafton/Connolly Reply Aff. at para. 32. See MCI WorldCom Nov. 24 Ex Parte Letter ("MCI WorldCom has resolved the problems with the differences in the pre-order and order field sizes for the two functions (CSR and address validation) that currently are up-and-running."). We note that Bell Atlantic plans to minimize inconsistencies in fields and formats and simplify the use of pre-ordering and ordering interfaces with the rollout of LiveWire, the implementation of LSOG 4 in February 2000 and in ongoing collaborative discussions with competing carriers "which will result in still further commonality in mid-2000." Bell Atlantic Nov. 24 Ex Parte Letter at 5. Aside from access to loop qualification information and due date information, which is discussed in Section V.B.1.f below, commenters do not dispute that the functionality provided by Bell Atlantic for the other pre- ordering functions is nondiscriminatory. Because characteristics of a loop, such as its length and the presence of various impediments to digital transmission, can hinder certain advanced services technologies, carriers often seek to "pre-qualify" a loop by accessing basic loop makeup information that will assist carriers in ascertaining whether the loop, either with or without the removal of the impediments, can support a particular advanced service. See Covad Conley/Poulicakos Decl. at para. 39; Rhythms Geis/Williams Aff. at paras. 13, 38-39, 49-51; see also Deployment of Wireline Services Offering Advanced Telecommunications Capability, et al., CC Docket Nos. 98-147 et al., Memorandum Opinion and Order and Notice of Proposed Rulemaking, 13 FCC Rcd 24012, 24037 (1998) (Advanced Services Order and NPRM), recon. pending. We note that, after the effective date of the UNE Remand rules, Bell Atlantic and all other incumbent LECs must comply with these rules, and future section 271 applicants must demonstrate compliance with the new requirements. Department of Justice Evaluation at 25. See Ameritech Michigan Order, 12 FCC Rcd at 20616; see also id., 12 FCC Rcd at 20615 (considering "all of the automated and manual processes that a BOC has undertaken to provide access to OSS functions."). Advanced Services Order and NPRM, 13 FCC Rcd at 24038. The Commission explained that "[a]n incumbent LEC does not meet the nondiscrimination requirement if it has the capability electronically to identify xDSL-capable loops, either on an individual basis or for an entire central office, while competing providers are relegated to a slower and more cumbersome process to obtain that information." Id. As these statements demonstrate, there can be no doubt that Bell Atlantic and other BOCs have had sufficient notice that their section 271 obligation to provide nondiscriminatory access to OSS extends to loop qualification information. Specifically, the mechanized loop qualification database identifies unloaded copper loops that are 18,000 feet or less in length, all of which were designed with less than 6,000 feet of bridged taps. See Bell Atlantic Application at 21; Bell Atlantic - New York's Joint Affidavit in Support of Proposed Rates for ADSL-Qualified, HDSL-Qualified, and Digital-Designed Links, Case 98-C-1357 (Sept. 13, 1999) at para. 24; Letter to Magalie Roman Salas, Secretary, FCC, to Dee May, Director, Federal Regulatory Affairs, Bell Atlantic, CC Docket No. 99- 295 (filed Nov. 22, 1999) (Bell Atlantic Nov. 22 Ex Parte Letter). In contrast to competing carriers, Bell Atlantic's retail representatives can "prequalify" a loop only through the mechanized loop qualification process. If a customer's line is not shown as qualified for ADSL service through the mechanized database, Bell Atlantic's sales representatives will not sell ADSL services to that customer. Bell Atlantic Miller/Jordan Decl. at para. 17; Bell Atlantic Lacouture/Troy Reply Decl. at para. 99. Bell Atlantic Application at 21; Bell Atlantic Lacouture/Troy Decl. at para. 84. According to Bell Atlantic, central offices with collocation represent 90 percent of the company's access lines in New York. Bell Atlantic Lacouture/Troy Decl. at para. 84. Bell Atlantic populates the mechanized loop qualification database for a particular central office by conducting a mechanized loop test of a sample of the loops in each terminal served by that office and determining whether the individual loop is served by copper or by fiber technology. See Bell Atlantic - New York's Joint Affidavit in Support of Proposed Rates for ADSL-Qualified, HDSL-Qualified, and Digital-Designed Links, Case 98-C-1357 (Sept. 13, 1999) at para. 23. Specifically, the Manual Loop Qualification process provides the total metallic loop length, the presence of load coils and digital loop carrier equipment and the capability of the loop to support ADSL. See Bell Atlantic Application at 21; Bell Atlantic Lacouture/Troy Decl. at para. 85; Bell Atlantic - New York's Joint Affidavit in Support of Proposed Rates for ADSL-Qualified, HDSL-Qualified, and Digital-Designed Links, Case 98-C-1357 (Sept. 13, 1999) at para. 29. See Bell Atlantic Application at 21; Bell Atlantic Lacouture/Troy Reply Decl. at para. 102; Bell Atlantic Nov. 22 Ex Parte Letter at 2. Bell Atlantic Lacouture/Troy Reply Decl. at para. 102. Given the mechanized and manual processes described above, we differ with the Department of Justice's belief that the record is not sufficiently developed to conclude that Bell Atlantic is providing nondiscriminatory access to loop qualification information. See Department of Justice Evaluation at 26. Bell Atlantic Lacouture/Troy Decl. at para. 85. Although Bell Atlantic is still in the process of surveying loops, the company claims that, as the loop information is gathered, it is made available simultaneously to competitors and its retail operations. We therefore disagree with carriers that argue that the mere fact that the mechanized loop qualification tool is not yet available in every central office renders it discriminatory. See CompTel Comments at 26; CoreComm Comments at 7; Covad Comments at 28; Northpoint Comments at 6, 8-9; Rhythms Comments at 14-20. See Covad Comments at 28-29; MCI WorldCom Comments at 34-35; MCI WorldCom Kinard Decl. at paras. 7-11; Network Access Comments at 9-10; New York State Attorney General's Comments at 16; Northpoint Comments at 7, 11-12; Rhythms Comments at 15-17; Sprint Comments at 11-14. MCI WorldCom, for example, claims that the mechanized loop qualification tool fails to provide carriers with loop length for loops over 18,000 feet, the length of the loop without bridged taps, the location and number of bridged taps, the loop wire gauge, spectrum management information, and the presence of load coils, digital loop carriers, repeaters, Digital Added Main Lines and pair gain devices, which could be used to assess the loop's compatibility with xDSL services other than ADSL. MCI WorldCom Comments at 35. Although commenters note that manual loop qualification processes (the Manual Loop Qualification and the Engineering Query) are time consuming and costly, they do not dispute that the manual processes provide access to all the loop makeup information that they need to make an independent assessment about a loop's suitability for a particular advanced service. See CompTel Comments at 27; Covad Comments at 29; Covad Conley/Poulicakos Aff. at para. 48; MCI WorldCom Comments at 32-36; Network Access Comments at 9-10; NorthPoint Comments at 7; Prism Comments at 21; Rhythms Comments at 15. We recognize that, pursuant to its tariff investigation, the New York Commission is in the process of reviewing the costs, as well as terms and conditions, of the access to loop makeup information that Bell Atlantic provides to competing carriers. See infra Section V.B.3. See CompTel Comments at 26-27; Covad Reply at 14-15; MCI WorldCom Comments at 35 n.48; MCI WorldCom Kinard Decl. at para. 15 n.18; Northpoint at 5, 11-12; Rhythms at 17-20; Rhythms Geis/Williams Aff. at paras. 36-37, 43. Specifically, commenters seek access to the Loop Facility Assignment and Control System (LFACS), which inventories, maintains and assigns outside plant local loop facilities, and the Trunk Inventory Record Keeping System (TIRKS), which inventories, maintains and assigns facilities for interoffice transmission, trunking and other special services. Bell Atlantic Miller/Jordan Decl. at para. 64. In response to commenters' assertions, Bell Atlantic claims that it "does not itself use or maintain" loop makeup information in a mechanized database, and that competing carriers seek "information that is not mechanized in [Bell Atlantic's] systems." Bell Atlantic Reply at 15; Bell Atlantic Lacouture/Troy Reply Decl. at para. 102. See also Bell Atlantic Nov. 22 Ex Parte Letter at 3 (representing that LFACs does not contain loop makeup information "[i]n well over 90 percent of the cases."). We find no conflicting evidence on the present record. See BellSouth South Carolina Order, 13 FCC Rcd at 625, 634-36 (expressing concern that significantly greater time is required for competitors to access and review pre-ordering information); Ameritech Michigan Order, 12 FCC Rcd at 20616 (finding that limits on the processing of information between an interface and legacy systems that prevent a competitor from performing a transaction in substantially the same time and manner as the BOC would be discriminatory). BellSouth South Carolina Order, 13 FCC Rcd at 636. See BellSouth South Carolina Order, 13 FCC Rcd at 588; see also AT&T Crafton/Connolly Aff. at para. 85 n.47 ("AT&T representatives perform the CSR retrieval while the customer is on the line."). See BellSouth South Carolina Order, 13 FCC Rcd at 636. We also note that KPMG reported response times for pre-order transactions, but given the significant improvement in the recent commercial usage data, we place less weight on KPMG's response times. See KPMG Final Report at POP5 IV-131, 136. Response time is the time that elapses between the submission of a query and the receipt of a response by the requesting carrier. See KPMG Final Report at POP8 IV-166; see also Performance Measurements NPRM, 13 FCC Rcd at 12837 (discussing the average interval for providing access to pre-ordering information). See Bell Atlantic Dowell/Canny Decl. Attach. B at 5-7; New York Commission Comments at 38-39. Most pre-order transactions, except for retrieval of parsed CSRs, have a retail analogue and are subject to a performance standard of "parity plus four seconds." We discuss the response times for parsed CSRs below. See infra paras. 151-53. The four-second differential accounts for additional security requirements and computer translations that Bell Atlantic systems undertake to provide access to competing carriers. See Bell Atlantic Dowell/Canny Decl. at para. 23, Attach. B at 6; New York Commission Comments at 38-39. Although Bell Atlantic reported pre-order response times in June and July that met the "parity plus four seconds" standard for all pre-order functions reported, we rely on data starting in August because, as discussed below, Bell Atlantic made changes in the way that it calculates response times in August that more accurately capture response times experienced by competing carriers. For EDI unparsed CSR retrieval, Bell Atlantic failed to meet the standard by .95 of a second in August and 1.52 seconds in September. For EDI due date availability, Bell Atlantic met the standard each month. For EDI address validation, Bell Atlantic met the standard in August and deviated by 1.87 seconds in September. For EDI product and service availability, Bell Atlantic met the standard in August and deviated by .16 of a second in September. See Bell Atlantic Dowell/Canny Decl. Attach. D at 96 (metrics PO-1-01; PO-1-02; PO-1-03; PO-1-04 for August 1999); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 1 (metrics PO-1-01; PO-1-02; PO-1-03; PO-1-04 for September 1999). See AT&T Crafton/Connolly Aff. at para. 85 n.47; AT&T Crafton/Connolly Reply Aff. at paras. 37-40. See AT&T Comments at 48; AT&T Crafton/Connolly Aff. at para. 78 n.44, 85 n.47; AT&T Crafton/Connolly Reply Aff. at para. 36; MCI WorldCom Kinard Decl. at paras. 7-8. See Ameritech Michigan Order, 12 FCC Rcd at 20656 (requiring Commission satisfaction that performance measures submitted by the BOC actually measure performance in a manner that shows whether the BOC provides nondiscriminatory access to OSS functions). EnView was initially developed to monitor the internal TISOC systems response and availability times. See KPMG Final Report at POP8 IV-164. Bell Atlantic describes EnView as a "performance evaluation software tool that measures and records the actual response time of transactions through emulation by logging into applications and executing individual transactions." Bell Atlantic Dowell/Canny Decl. Attach. B at 6. In response to AT&T's criticism of the EnView system, Bell Atlantic notes that AT&T agreed in its interconnection agreement with Bell Atlantic to use the EnView system to measure pre-ordering response times. Bell Atlantic Dowell/Canny Reply Decl. at para. 12. The EnView system consists of two emulation programs, or "robots," one operating out of Manchester, New Hampshire and the other out of Andover, Massachusetts. The robots run pre-defined scripts requesting information as if the information were being requested from a competing carrier (which would be processed through the DCAS system) or from a Bell Atlantic retail representative (which would flow directly to back office systems). See KPMG Final Report at POP8 IV-164-165 (describing EnView system). Bell Atlantic Dowell/Canny Decl. Attach. B at 5. See Bell Atlantic Dowell/Canny Decl. at para. 24; Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 21; New York Commission Comments at 39; see also NYPSC Permanent Rule Order, App. at 3-4 (ordering Bell Atlantic to measure separately response times for each type of interface, and to begin reporting EDI interface response times immediately). The New York Commission agrees that Bell Atlantic's August data more accurately capture pre-order response time because Bell Atlantic started measuring the EDI interface and implemented other changes. The New York Commission also notes that additional refinements to the EnView pre-order measurement system are currently being considered in the Carrier-to-Carrier proceeding. New York Commission Comments at 39. Although Bell Atlantic does not yet report CORBA pre-order response times, in light of the nascency of that interface and Bell Atlantic's reporting of the alternative EDI-9 interface, failure to report CORBA performance data does not preclude a finding that Bell Atlantic is meeting its pre-order OSS checklist requirements. See New York Commission Comments at 40. Bell Atlantic Application at 38. Furthermore, in response to commenters' claims that the pre-ordering interfaces are deficient, Bell Atlantic notes that the interfaces handled more than 283,000 pre-order transactions in September. Bell Atlantic Reply at 32; Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 5. See KPMG Final Report at POP5 IV-102 (showing daily pre-order submission volume of 3,400 for the EDI functional evaluation; 10,500 for the EDI normal volume tests; and 13,200 for the EDI peak volume test); see also id., at POP6 IV-139, 145 (showing submission of 15,269 pre-order requests in a 4-hour period during EDI stress test). During the stress test, KPMG found that Bell Atlantic's pre-order systems were able to maintain operability at levels up to 119 percent above the baseline established for peak volume testing, which represents a 50-percent increase over normal daily volume. KPMG Final Report at POP6 IV-149. See KPMG Final Report at POP13 IV-300-314 (scalability review of interfaces and architecture). AT&T Crafton/Connolly Reply Aff. at paras. 37-38; MCI WorldCom Comments at 29; MCI WorldCom Reply at 19. See Bell Atlantic Nov. 24 Ex Parte Letter at 2. See Bell Atlantic Dowell/Canny Reply Decl. at para. 13. The New York Commission recently adopted "parity with retail unparsed CSR plus tens seconds" as a performance standard for parsed CSR retrieval. NYPSC Additional Guidelines Order at 15; see also New York Commission Reply at 17. Although this standard was not formally adopted by the New York Commission until November 5, 1999, given that Bell Atlantic committed to the standard in collaborative meetings in late September and that we find the measure to be reasonable, we do not believe that we are precluded from independently relying on this standard for purposes of our analysis. See MCI WorldCom Comments at 29; MCI WorldCom Reply at 18 (indicating that MCI WorldCom can presently operate in a competitive market if Bell Atlantic meets a 10-second standard for parsed CSR retrieval). Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 21, Attach. A (listing daily average parsed CSR response time for October 5 through October 14, 1999). MCI WorldCom asserts generally that it takes between 10 to 15 seconds during the day (9:00 a.m. to 5:00 p.m.) and 20 to 40 seconds in the evening (6:00 p.m. to 9:00 p.m.) to receive responses for its parsed CSR inquiries. MCI WorldCom Reply at 19. See also MCI WorldCom Comments at 29; MCI WorldCom Lichtenberg/Sivori Decl. at para. 62 (claiming that it experiences intervals of between 15 and 20 seconds for parsed CSRs). AT&T claims that "response times on CORBA have been as long as 45 seconds in some instances," but notes that "CORBA has been in commercial production for too short a time for AT&T to provide comprehensive data." AT&T Crafton/Connolly Reply Aff. at para. 38. See Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 21 (noting that Bell Atlantic has no ability to measure what happens on MCI WorldCom's side of the firewall, and that MCI WorldCom personnel have informed Bell Atlantic that they have experienced problems on MCI WorldCom's side of the firewall). In this section we evaluate the availability of Bell Atlantic's interfaces for all functionality, including the EDI, Web GUI and CORBA for pre-ordering, ordering, and maintenance and repair functions. See New York Commission Comments at 41 (concluding that Bell Atlantic is providing satisfactory interface availability). With respect to its back office pre-ordering systems, Bell Atlantic states that it periodically takes these systems out of service for routine maintenance, during which time they are equally unavailable to competing carriers as well as Bell Atlantic's retail representatives. We find no evidence on the record that Bell Atlantic discriminates in the availability of its back office pre-ordering systems. See BellSouth South Carolina Order, 12 FCC Rcd at 637-38. Bell Atlantic Dowell/Canny Decl. at paras. 25-27, Attach. A at 8-9. See Bell Atlantic Dowell/Canny Decl. Attach. B at 8. We are further encouraged by, but our decision does not rely on, the New York Commission's recent modifications to the methodology used to calculate interface availability. See NYPSC Additional Guidelines Order, at 15-16 (reporting that Bell Atlantic will include actual outages reported by carriers as well as outages captured by the EnView simulations, change the EnView system to send transactions on average every six minutes rather than fifteen, and make available for inspection by carriers its logs of carrier-reported outages). We also note that Bell Atlantic performs necessary maintenance on the interfaces during non-prime time. Bell Atlantic Miller/Jordan Decl. at para. 26. We note that the New York Commission did not establish a performance standard for non-prime time. See Bell Atlantic Dowell/Canny Decl. Attach. B at 8-9. Because Bell Atlantic began reporting availability for the EDI interface in July, we do not rely on earlier data in this section. Bell Atlantic Dowell/Canny Decl. Attach. D at 73, 84, 96 (metric PO-2-02 for June, July and August 1999); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 1 (metric PO-2-02 for September 1999). In June, Bell Atantic reported interface availability only for the EIF interface. Bell Atlantic Dowell/Canny Decl. Attach. D at 73, 84, 96 (metric PO-2-03 for July and August 1999). Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 1 (metric PO-2-03 for September 1999). See KPMG Final Report at POP5 IV-106, 110-111 (noting some "sporadic and not routinely experienced" disconnections of the EDI ordering interface). During its functional evaluation of the Web GUI, KPMG did not experience any outages or down time for pre-ordering capability, although it did experience some temporary outages for ordering capability. See KPMG Final Report at POP2 IV-34; POP2 IV-37. KPMG Final Report at POP5 IV-135. Bell Atlantic Miller/Jordan Decl. at para. 28. Although commenters report periodic interface outages, they fail to assert that the reported outages are not captured in the relevant performance measurements. For example, MCI WorldCom states that it has experienced "periodic failures" of the EDI pre-ordering interface. MCI WorldCom Comments at 28; MCI WorldCom Lichtenberg/Sivori Decl. at paras. 61, 139-40; MCI WorldCom Reply at 19-20 (indicating that the EDI pre- ordering interface was down 11 times from September 3 through October 19); MCI WorldCom Lichtenberg/Sivori Reply Decl. at para. 10, Attach. 1. In addition, AT&T claims that since it began using CORBA for commercial production in October, the interface has failed on a number of occasions. When CORBA was down, AT&T used the Web GUI to conduct pre-order transactions. AT&T Crafton/Connolly Reply Aff. at paras. 34, 89-94. Ordering functions for DSL capable loops are addressed in the DSL discussion of Checklist Item 4, infra, at section V.D.2.c. New York Commission Comments at 16 (concluding that Bell Atlantic has demonstrated its ability to "satisfactorily process orders" and that its "automated and manual processes are scalable."). See Bell Atlantic Application at 40. KPMG Final Report at POP5 IV-111 (Test P5-8) ("BA-NY system or representative provides required order transaction functionality"). Bell Atlantic Miller/Jordan Decl. at para. 35. Of the six competing carriers using EDI for ordering functions, multiple carriers are using it to order UNEs and resale services. See id.; Letter from Dee May, Director, Federal Regulatory Affairs, Bell Atlantic, to Magalie Roman Salas, Secretary, Federal Communications Commission, CC Docket 99-295 (filed December 17, 1999) (listing carriers using EDI for UNE and resale service ordering). Bell Atlantic Miller/Jordan Decl. at para. 35. Bell Atlantic Miller/Jordan Decl. at para. 34; New York Commission Comments at 41. An order is confirmed when it is accepted into Bell Atlantic's Service Order Processor and rejected when it contains certain kinds of errors. Bell Atlantic Miller/Jordan Decl. at para. 41. Second BellSouth Louisiana Order, 13 FCC Rcd at 20678, 20680; see Bell Atlantic Application at 40. Bell Atlantic Miller/Jordan Decl. at paras. 38-43. Bell Atlantic will accept resale and UNE POTS orders only over either EDI or the Web GUI. Bell Atlantic Dowell/Canny Decl. Attach. B at 17, 20. In contrast, for non- POTS UNE orders and interconnection trunk orders, Bell Atlantic will accept facsimile and mail orders in addition to accepting orders over EDI or the Web GUI. Id. Bell Atlantic Miller/Jordan Decl. at para. 41; see Bell Atlantic Miller/Jordan Decl. Attach. D at 1. A competing carrier's orders "flow-through" if they are transmitted electronically through the gateway and accepted into Bell Atlantic's back office ordering systems without manual intervention. Second BellSouth Louisiana Order, 13 FCC Rcd at 20671; see also Bell Atlantic Dowell/Canny Decl. Attach. B at 79 (defining "mechanized flow- through" as "[o]rders received electronically through the ordering interface (DCAS) and requiring no manual intervention to be entered into the SOP"). Although under this definition a "rejected" order does not "flow- through," some commenters in this proceeding refer generally to orders that are mechanically processed by Bell Atlantic's systems without human intervention as "flowing-through." Bell Atlantic has designed its system to flow-through certain order types. Bell Atlantic Miller/Jordan Decl. at para. 38. Order types that are not designed to flow-through will drop out of Bell Atlantic's systems for manual processing. Bell Atlantic Miller/Jordan Decl. at para. 42. Moreover, for orders containing certain types of errors, such as mis-typed address information, Bell Atlantic has designed the system to direct the order for manual correction by Bell Atlantic representatives, rather than rejecting the order. Bell Atlantic Miller/Jordan Decl. at para. 41. Bell Atlantic Dowell/Canny Decl. Attach. B at 17, 21. These standards apply only for UNE and resale POTS orders under ten lines and certain "pre-qualified" complex orders under ten lines. Bell Atlantic Dowell/Canny Decl. Attach. B at 17, 21. The New York Commission established a 48 hour standard for manually processed resale and UNE special services orders under 10 lines, and a 72 hour standard for all manually processed resale and UNE orders of greater than or equal to ten lines. Bell Atlantic Dowell/Canny Decl. Attach. B at 17, 21. We do not analyze Bell Atlantic's performance for such orders because the relative volumes of orders in these categories are too low to make a meaningful judgment. In prior orders the Commission concluded that ordering functions for unbundled network elements have no retail analogue. Ameritech Michigan Order, 12 FCC Rcd at 20619. In contrast, the Commission has previously found that resale ordering functions have a retail analogue and, as such, BOCs must provide resale ordering functions to competing carriers in substantially the same time and manner as the incumbent performs that function for itself. Ameritech Michigan Order, 12 FCC Rcd at 20616. In this application, the New York Commission has established benchmark standards to measure Bell Atlantic's ability to provide order status notices to competitors in a timely fashion, as it concluded that there are no retail analogues for ordering in Bell Atlantic's system. New York Commission Comments at 42. These benchmarks apply to both UNEs and resale. Bell Atlantic Dowell/Canny Decl. Attach. B at 17, 21. We find that the New York Commission's benchmarks, which were established in a collaborative proceeding, provide a reasonable means of comparison for purposes of the instant proceeding. See, e.g., Second BellSouth Louisiana Order, 13 FCC Rcd at 20670-71; First BellSouth Louisiana Order, 13 FCC Rcd at 6263; BellSouth South Carolina Order, 13 FCC Rcd at 599. Second BellSouth Louisiana Order, 13 FCC Rcd at 20670; First BellSouth Louisiana Order, 13 FCC Rcd at 6263; BellSouth South Carolina Order, 13 FCC Rcd at 599. Second BellSouth Louisiana Order, 13 FCC Rcd at 20670-71; First BellSouth Louisiana Order, 13 FCC Rcd at 6263; BellSouth South Carolina Order, 13 FCC Rcd at 599. Commenters argue that Bell Atlantic's flow-through rates are insufficient and therefore fail to satisfy section 271. AT&T Comments at 16-17; Choice One Comments at 11; MCI WorldCom Comments at 10; NY Attorney General Comments at 12-13; NorthPoint Comments at 14; see Covad Comments at 29-30. Because we conclude that, under the facts of this application, we need not focus on flow-through rates, we find that such arguments are not dispositive of our analysis. Second BellSouth Louisiana Order, 13 FCC Rcd at 20671; First BellSouth Louisiana Order, 13 FCC Rcd at 6259-70, 77; BellSouth South Carolina Order, 13 FCC Rcd at 597-611. Second BellSouth Louisiana Order, 13 FCC Rcd at 20671; see Performance Measurements NPRM, 13 FCC Rcd at 12850 (flow-through rate "serves as a yardstick to evaluate whether an incumbent LEC's OSS is capable of handling reasonably foreseeable commercial volumes of orders"). See discussion supra paras 160. See discussion supra paras. 127, 131. See also Bell Atlantic Miller/Jordan Decl. at paras. 87-91. See discussion supra paras. 137-39. See discussion infra paras. 173-210; see also paras. 287-88, 292-98. New York Commission Reply at 11. We note that the New York Commission focused its analysis of Bell Atlantic's ordering functions on on-time order processing. New York Commission Comments at 44; New York Commission Reply at 11. Bell Atlantic Dowell/Canny Decl. Attach. D at 78, 90, 102 (metrics OR-1 and OR-2); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 7; New York Commission Comments at 43. In contrast, the Department of Justice submitted data disaggregated by UNE-loop and UNE-platform for the first time in its evaluation. Department of Justice Evaluation at 15 n.28 (noting that Bell Atlantic provided Department of Justice with "supplemental data disaggregating its UNE-L and UNE-P performance" after filing its section 271 application and that "[t]o the Department's knowledge, these data have not been provided to the Commission, the NYPSC or the CLEC community for review.") On reply and in subsequent Ex Partes, the New York Commission submitted analyses of the Carrier-to-Carrier data in aggregated and disaggregated form. New York Commission Reply at 13; id. at Exh. 1; Letter from Penny Rubin, Managing Attorney, New York Department of Public Service, to Magalie Roman Salas, Secretary, Federal Communications Commission, CC Docket 99-295 (filed December 7, 1999) (New York Commission Dec. 7 Ex Parte Letter) (resale data and aggregated UNE data); Letter from Penny Rubin, Managing Attorney, New York Department of Public Service, to Magalie Roman Salas, Secretary, Federal Communications Commission, CC Docket 99-295 (filed Dec. 2, 1999) (New York Dec. 2 Ex Parte Letter) (disaggregated UNE-loop and UNE-platform data). For example, as the New York Commission has shown, Bell Atlantic is able to provide order confirmation and rejection notices for UNE-loop and UNE-platform in a manner that provides efficient competitors a meaningful opportunity to compete even when disaggregated data is considered. New York Commission Reply Evaluation, Exh. 1 (showing, for example, that in September Bell Atlantic delivered order confirmation and rejection notices on time 89% of the time for UNE-loop and 94% for UNE-platform). In contrast, the Department of Justice concluded that Bell Atlantic has not met its obligation to provide order confirmation and rejection notices in a timely manner for UNE-loops and UNE-platform. Department of Justice Evaluation at 15, 31-32. After careful consideration of the Department of Justice's evaluation we conclude, however, that the evidence demonstrates that Bell Atlantic is providing nondiscriminatory access to its ordering functions for both UNE-loops (including hot cuts) and UNE-platform. In addition to the reasons discussed more fully in this section, we note that our conclusions are based, in part, upon September performance data submitted by both Bell Atlantic and the New York Commission that the Department of Justice did not discuss in its evaluation. See, e.g., Department of Justice Evaluation at 16 & n.29, 31-32 & n.86. Thus, although we recognize that there may be circumstances in which we find it appropriate to examine disaggregated data in the context of analyzing the ordering access a BOC provides to competing carriers, those circumstances do not present themselves in this application. In June, July, August and September respectively, Bell Atlantic returned 98, 97, 99,and 89 percent, of mechanized order confirmation notices within two hours, and 80, 80, 88, and 89 percent of manually processed order confirmation notices within 24 hours. Bell Atlantic Dowell/Canny Decl. Attach. D at 78, 90, 102 (metrics OR-1-02 and 1-04); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 7 (same); Letter from Dee May, Director, Federal Regulatory Group; Bell Atlantic, to Magalie Salas, Secretary, Federal Communications Commission, CC Docket No. 99-295, Attach. at 2 (filed Dec. 17, 1999) (Bell Atlantic Dec. 17, 1999 Ex Parte Letter correcting September data). For those same months, respectively, Bell Atlantic returned 86, 87, 94 and 93 percent of mechanized order rejection notices within two hours, and 71, 71, 83 and 91 percent of manually processed order rejection notices within 24 hours. Bell Atlantic Dowell/Canny Decl. Attach. D at 78, 90, 102 (metrics OR-2-02, 2- 04); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 7 (same); Bell Atlantic December 17, 1999 Ex Parte Letter correcting September data, Attach. at 2. On December 17, 1999, Bell Atlantic filed an ex parte letter correcting data for September that it filed as part of the Carrier-to-Carrier reports, which Bell Atlantic submitted on reply. Bell Atlantic explains that it did not properly conform to a change, first instituted in September, in the way the New York Commission required Bell Atlantic to classify certain orders (affecting metrics OR-1-01 through 1-04 and OR-2-01 through 2-04). Id. at 1. This reclassification caused Bell Atlantic's performance in September to show an anomalous dip that does not reflect a change in Bell Atlantic's actual performance when compared to prior months. Letter from Dee May, Director, Federal Regulatory Group; Bell Atlantic, to Magalie Salas, Secretary, Federal Communications Commission, CC Docket No. 99-295 at 1-2 (filed December 17, 1999) (Letter explaining September correction). New York Commission Reply, Exh. 1, page 1; New York Commission Reply at 11; New York Commission Dec. 7 Ex Parte Letter (88, 88, 94, and 94 percent of all UNE orders received confirmation or rejection notices on time during June, July, August and September, respectively). AT&T also asserts that Bell Atlantic's performance in providing "order acknowledgments" for orders placed over the EDI interface declined in September. AT&T Crafton/Connolly Aff. at para. 257. We note, however, that we have never required the provision of acknowledgements for the purposes of satisfying section 271. On average, for June, July, August, and September Bell Atlantic returned order confirmation notices in 8.48, 8.84, 6.16, and 6.46 hours, respectively, and order rejection notices in 16.28, 12.63, 8.12, and 6.20 hours, respectively. These averages were calculated by Commission staff from the Carrier-to-Carrier data provided by Bell Atlantic. Bell Atlantic Dowell/Canny Decl. Attach. D at 78, 90, 102 (metrics OR-1 and OR-2); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 7 (same); Bell Atlantic Dec. 17, 1999 Ex Parte Letter correcting September data, Attach at 2. New York Commission Dec. 7 Ex Parte Letter (Total UNE order volumes: January (8,612), February (15,442), March (19,796), April (39,427), May (45,136), June (72,121), July (58,575), August (64,350), September (69,791)). For example, in the Ameritech Michigan Order the Commission observed, over the course of the first four months of 1997, that Ameritech received almost 20,000 resale orders over its EDI interface for the state of Michigan. 12 FCC Rcd at 20629-30 (Ameritech represented that it received 19,671 orders over EDI and accepted, and processed, 17,789 of those orders). In the BellSouth South Carolina Order we noted that BellSouth received, on a region-wide basis for one month, 6,715 resale orders through its EDI interface. 13 FCC Rcd at 596. In contrast, in the month of September in the state of New York alone Bell Atlantic processed almost 20,000 resale orders, over half of which were received over EDI, and 70,000 UNE orders, almost 50,000 of which were received over EDI. Letter from Dee May, Director, Federal Regulatory Affairs, Bell Atlantic, to Magalie Roman Salas, Secretary, Federal Communications Commission, CC Docket 99-295 at 2 (filed November 17, 1999) (listing volumes by individual carriers over EDI and GUI interfaces); see New York Commission Dec. 7 Ex Parte Letter. Virtually all of the orders not received over EDI are received over the GUI. Second BellSouth Louisiana Order, 13 FCC Rcd at 20681, para. 122 & n.420. Bell Atlantic's performance is also significantly better than the BOC performance described in other section 271 orders. We note that this is the first time the Commission has done a full analysis of UNE ordering in a section 271 order. We conclude, however, that our precedent regarding resale ordering is generally applicable to UNE ordering. For example, in the Ameritech Michigan Order, between 14 and 45 percent of order confirmation notices were not returned to competing carriers within three days and, based upon monthly averages, it took as many as six days to return rejection notices to competing carriers. Ameritech Michigan Order, 12 FCC Rcd at 20643. Evidence in the record in the South Carolina Order indicated that carriers did not receive 38 to 90 percent of their order confirmation notices in 24 hours, and for one carrier it took on average took up to 7 days from submission to receive such notices. BellSouth South Carolina Order, 13 FCC Rcd at 608. The evidence in the First BellSouth Louisiana Order showed that BellSouth only returned between 20 to 62 percent of competing carrier orders confirmation notices within 24 hours, and for one carrier it took an average of 3.5 workdays to receive an order confirmation. First BellSouth Louisiana Order, 13 FCC Rcd at 6268-69. In the Second BellSouth Louisiana Order, for electronically submitted orders for resale residential service, BellSouth returned a reject notice on average somewhere between 2 and 8 days after it received an order, depending on the month. Second BellSouth Louisiana Order, 13 FCC Rcd at 20679-80. Further, over 37 percent of such notices were returned beyond a 24 hour interval. Second BellSouth Louisiana Order, 13 FCC Rcd at 20679-80. For manually submitted orders for resale residential service, the average reject notice inteval was 1.61 days, and over 63 percent of such notices were returned beyond a 24 hour interval. Second BellSouth Louisiana Order, 13 FCC Rcd at 20679-80. We also note that, unlike Bell Atlantic, which returns rejection and order confirmation notices over electronic interfaces, in prior applications BellSouth returned some notices by facsimile. First BellSouth Louisiana Order, 13 FCC Rcd at 6262; BellSouth South Carolina Order, 13 FCC Rcd at 598-99. Electronic notifications are superior to faxed notifications because they are quicker and do not require competing carriers to manually reenter information from the notice into their OSS. Bell Atlantic has asserted that retail flow-through is a "misnomer" for its systems. Bell Atlantic Miller/Jordan Decl. at para. 57; Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 37. Specifically, Bell Atlantic claims it is a misnomer because "every retail order must be typed by a BA-NY representative in order to enter it into the service order processor." Bell Atlantic Miller/Jordan Decl. at para. 57; Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 37. Moreover, the New York Commission has agreed that there is not a retail analogue for Bell Atlantic's systems. New York Commission Comments at 42 ("Since there is no retail analogue in Bell Atlantic-NY's retail system, ordering metrics are `absolute standard' metrics."). In the alternative, Bell Atlantic argues on Reply that an evaluation of all its October retail orders shows that 61.5% of its retail orders "flow-through." Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 39; see also id. Attach. E. Because this number is derived from an evaluation of data from the entire month of October, and therefore post- dates the comment filing date, we accord it no weight. Given New York and Bell Atlantic's conclusion that a retail analogue does not exist, and in absence of a credible retail analogue in the record, we find that for purposes of this application Bell Atlantic must demonstrate that the access it provides to its ordering functions offers an efficient carrier a meaningful opportunity to compete. The Carrier-to-Carrier reports indicate that overall UNE orders flowed-through 59.28 percent and 62.81 percent of the time for August and September, 1999, respectively. Bell Atlantic Dowell/Canny Decl. Attach. D at 102 (metric OR-5-01); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 7 (same). Second BellSouth Louisiana Order, 13 FCC Rcd at 20655; BellSouth South Carolina Order, 13 FCC Rcd at 593; Ameritech Michigan Order, 12 FCC Rcd at 20618. Letter from Dee May, Director, Federal Regulatory Affairs, Bell Atlantic, to Magalie Roman Salas, Secretary, Federal Communications Commission, CC Docket 99-295, Attach 1 (filed November 19, 1999) (Bell Atlantic Nov. 19 Ex Parte Letter) (listing volumes and flow-through rates by individual carriers for UNE-platform and UNE- loop). Indeed, we note that those carriers submitting among the largest volumes of orders have achieved high flow-through rates. Bell Atlantic Nov. 19 Ex Parte Letter. Bell Atlantic Miller/Jordan Decl. at para. 60; Bell Atlantic Miller/Jordan Reply Decl. at para. 36. The Commission has recognized in previous orders that there are limited circumstances in which manual intervention is appropriate. BellSouth South Carolina Order, 13 FCC Rcd at 599, 107. We find that manually correcting and processing orders containing errors instead of rejecting them is one such circumstance. Bell Atlantic notes that if it were to reject such orders instead of correcting them, its flow-through rates would be much higher than currently reported. Bell Atlantic Miller/Jordan Reply Decl. at para. 36 (projected 75% flow-through for UNEs). New York Commission Comments at 46; see also Bell Atlantic Application at 43 n.37. New York Commission Comments at 46 n.2. The New York Commission also concludes that flow-through suffers as competing carriers enter the market, and hire and train new employees. New York Commission Reply at 13. Second BellSouth Louisiana Order, 13 FCC Rcd at 20674.First BellSouth Louisiana Order, 13 FCC Rcd at 6263. See BellSouth South Carolina Order, 13 FCC Rcd at 601-02. KPMG Final Report POP7 IV-150. Bell Atlantic Miller/Jordan Decl. at para. 61 (citing KPMG Report POP7 IV-160 (Test P7-2)). Certain types of orders are not designed to flow-through, such as complex orders that require manual handling. KPMG Final Report POP IV-160 (Test P7-2); see also New York Commission Reply at 12; Bell Atlantic Reply at 16. The New York Commission noted that although the KPMG's results show a higher level of flow-through performance than Bell Atlantic's metrics, the difference "was anticipated and is easily explained." New York Commission Reply at 12. As the New York Commission explained, flow-through in the real commercial environment "is affected by such factors as ordering errors, pending orders, features not intended to flow-through, and the market entry learning curve; and one therefore would expect it to be lower." New York Commission Reply at 12. Department of Justice Evaluation at 32; AT&T Comments at 20; MCI WorldCom Comments at 16; see Choice One Comments at 11. See New York Commission Reply at 11-12. See, e.g., New York Commission Dec. 7 Ex Parte Letter (Total UNE order volumes: January (8,612), February (15,442), March (19,796), April (39,427), May (45,136), June (72,121), July (58,575), August (64,350), September (69,791)). KPMG Final Report POP IV-160 (Test P7-2); see also New York Commission Reply at 12; Bell Atlantic Reply at 16. Specifically, Bell Atlantic proposed a series of enhancements to further reduce the manual processing of UNE-platform orders. In its proposal, Bell Atlantic presented the New York Commission with a three-phase plan to increase the percentage of electronically processed UNE-platform orders. Bell Atlantic Dowell/Canny Reply Decl. Attach. B at 36. As we have stated previously, "the development of OSS functions is not a static process, and we encourage and expect [a BOC] to make improvements to its operations support systems, even after it has filed a section 271 application." Ameritech Michigan Order, 12 FCC Rcd at 20624. We recognize, of course, that there is a fundamental difference between making improvements to OSS access that, at the time of the filing of the application, meets the nondiscriminatory requirement, and taking or proposing post-filing remedial measures to try to bring the OSS access into compliance during the pendency of the application. Id. We find that Bell Atlantic's proposed improvements are the former, not the latter. Bell Atlantic Dowell/Canny Reply Decl. Attach. B at 39 (Affidavit submitted on behalf of Bell Atlantic to New York Commission on October 8, 1999). Bell Atlantic Dowell/Canny Reply Decl. Attach. B at 39 (Affidavit submitted on behalf of Bell Atlantic to New York Commission on October 8, 1999) Bell Atlantic Dowell/Canny Reply Decl. Attach. B at 39 (Affidavit submitted on behalf of Bell Atlantic to New York Commission on October 8, 1999) Bell Atlantic Dowell/Canny Reply Decl. Attach. B at 39 (Affidavit submitted on behalf of Bell Atlantic to New York Commission on October 8, 1999) Department of Justice Evaluation at 31-32; AT&T Comments at 19; NorthPoint Comments at 13. Bell Atlantic Dowell/Canny Decl. at para. 53. Bell Atlantic reported order confirmation accuracy of 99.54, July 97.97, August 98.39, and September 95.08 percent for June, July August, and September, respectively. Bell Atlantic Dowell/Canny Decl. Attach. D at 79, 91, 102 (metric OR 6-03); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 7 (same). AT&T makes extensive claims regarding LSRC accuracy with respect to UNE loop orders. AT&T Meek Aff. at paras. 35-41. The Department of Justice similarly raises concerns in this regard, arguing that the high level of inaccurate confirmations returned by Bell Atlantic imposes significant costs and delays upon competing carriers. Department of Justice Evaluation at 17. Department of Justice Evaluation at 16; see Bell Atlantic Application, App. C, Vol. 59, Tab. 890 at 3956. AT&T Meek Aff. at paras. 95. New York Commission Comments at 81 n.3. Bell Atlantic Reply at 8; Bell Atlantic Dowell/Canny Reply Decl. at para. 35. See New York Commission Comments at 81 n.3. Bell Atlantic Dowell/Canny Decl. Attach. D at 102 (metric OR 6-01) (August (59.28%)); Bell Atlantic Reply Dowell/Canny Decl. Attach. C at 7 (metric OR 6-01) (September (41.52%)). Bell Atlantic Dowell/Canny Decl. at para. 37. Bell Atlantic Dowell/Canny Decl. at para. 37. Members of Bell Atlantic's "Quality Management Team" examine the selected orders and compare twelve specified field identifiers in the service orders with corresponding information in the orders placed by competing carriers. Id. Bell Atlantic then reports the percent of orders that match completely. Id. Bell Atlantic also reports the percent of the fields with errors (i.e., "percent opportunities"). Id. Bell Atlantic's performance for the percent opportunities metric has been significantly better than for order accuracy. For example, Bell Atlantic reported performance in August and September of 93.18 percent and 90.58 percent, respectively. Bell Atlantic Dowell/Canny Decl. Attach. D at 102 (metric OR 6-02); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 7 (same). We find that this performance further supports Bell Atlantic's assertion that it manually processes orders accurately. Bell Atlantic Dowell/Canny Reply Decl. at para 38. Bell Atlantic notes that some orders are handled manually because there is an error in the order submitted by the competing carrier. Id. When this happens, the Bell Atlantic wholesale representative corrects the error manually. Id. Bell Atlantic claims that the resulting differences between the original order and the corrected order are attributed to Bell Atlantic as errors merely because the order submitted by the competing carrier and the order entered by Bell Atlantic into its service order processor do not match. Id. Moreover, Bell Atlantic asserts that the process of translating competing carrier orders into the service order processor may result in "a literal mis-match of information" between the order submitted by the competing carrier and the order entered in the service order processor even when the end result is that the order is provisioned as requested. Id. For example, as Bell Atlantic explains, a single package ordering code on an order placed by a competing carrier may require translation into multiple ordering codes for entry into Bell Atlantic's service order processor. Id. Bell Atlantic further claims that the low rate is due, in part, to the fact that members of its "Quality Management Team" who compile the data have not yet "mastered the intricacies of the order process and, as such, are not properly attributing errors." Id. at para. 39. Bell Atlantic began reporting data for this metric in August 1999. Bell Atlantic Dowell/Canny Decl. at para. 53. Bell Atlantic Dowell/Canny Decl. at para 38. Bell Atlantic Dowell/Canny Decl. at para 38; id. Attach. G. The analysis consisted of a random sample of August orders identified as containing errors. Bell Atlantic Reply Dowell/Canny Aff., para 40-41; Letter from Dee May, Director, Federal Regulatory Affairs, Bell Atlantic, to Magalie Roman Salas, Secretary, Federal Communications Commission, CC Docket 99-295 at 1 (filed December 13, 1999). Each order was listed with the original error identified. Bell Atlantic Dowell/Canny Decl. at para 38; id. Attach. G. Based upon this analysis, Bell Atlantic should have received a score of 87 percent for service order accuracy. Bell Atlantic Dowell/Canny Decl. at para. 41. Bell Atlantic Dowell/Canny Decl. at para 74 (Installation quality metrics are "additional measures of service order accuracy, since an end user will report a trouble if a service is not installed accurately."). See, e.g., Bell Atlantic Dowell/Canny Decl. Attach. D at 80, 92, 104 (metrics PR-6-01, 6-02, 6-03); Bell Atlantic Reply Dowell/Canny Decl. Attach. C at 9 (same); but see AT&T Crafton/Connolly Aff. at para. 112 (speculating that "errors alone could result in provisioning inaccuracies . . . for more than 15 percent of all CLEC UNE orders"). Department of Justice Evaluation at 30. See also AT&T Crafton/Connolly Aff. at paras. 103-04 (Bell Atlantic's rejection rates are "commercially unreasonable"). For June, July, August, and September , respectively, Bell Atlantic rejected 28.69, 34.01, 33.65, and 32.14 percent of competing carrier orders. Bell Atlantic Dowell/Canny Decl. Attach D at 79, 91, 102 (metric OR-3-01); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 7 (same). Department of Justice Evaluation at 30 ("Many of these orders are undoubtedly rejected because of errors committed by [competing carriers], for which Bell Atlantic should not be held responsible."). Department of Justice Evaluation at 30. Bell Atlantic Miller/Jordan Decl. at para. 42; Bell Atlantic Miller/Jordan Reply Decl. at para. 33; id. Attach. C at 7-12 (showing monthly rejection rates and order volumes by carrier for June through August 1999). We note that many carriers placing among the highest order volumes have been able to achieve rejection rates well below the average rate reported by Bell Atlantic in the Carrier-to-Carrier metrics. Bell Atlantic Miller/Jordan Reply Decl. Attach. C at 7-12. This is in contrast to the circumstances in prior section 271 applications where we concluded that a BOC had not shown that order rejections were attributable to competing carrier error. For example, in the BellSouth South Carolina Order we concluded that BellSouth had not shown that the level of order rejections for carriers using the EDI interface was attributable to competing carrier error, in part, because every competing carrier attempting to use the interface was experiencing high order rejection rates, Bell South was not providing competing carriers with adequate business rules, and BellSouth failed to provide integrated pre-ordering and ordering interfaces. BellSouth South Carolina Order, 13 FCC Rcd at 600-01. None of these factors is present in this application. Bell Atlantic Miller/Jordan Reply Decl. at para. 33. Both AT&T and Z-Tel assert that Bell Atlantic issues spurious rejection notices. AT&T Crafton/Connolly Decl. Attach. 18; Z-Tel Comments at 19. Bell Atlantic asserts that "the vast majority" of the rejections were not spurious, but resulted from the submission of incorrect orders. Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 47. No other commenters have raised this issue. Moreover, neither AT&T nor Z-Tel has provided specific evidentiary support for their claims. As such, we are unable to find, based upon these claims, that Bell Atlantic has failed to comply with the requirements of this checklist item. AT&T Connolly/Crafton Aff. at para. 258 (asserting that AT&T did not receive order confirmation or rejection notices for 1% of its orders in August and 9% in September); MCI WorldCom Lichtenberg/Savori Reply Decl. at para. 19 (contending that MCI WorldCom did not receive order confirmations for 28 and 374 orders in August and September, respectively). No other commenters have raised this issue. There is no evidence in the record that shows, or even indicates, that Bell Atlantic's systems and interfaces, and not the competing carriers', are responsible for the failure of competing carriers to receive order confirmations. Department of Justice Evaluation at 29-30 (expressing concern that Bell Atlantic's "heavy reliance" on manual processing increases competing carrier costs and creates significant risk of customer-affecting service problems when order volumes increase); AT&T Comments at 15-22; Choice One Comments at 11; MCI WorldCom Comments at 9-19; NY Attorney General Comments at 11-13; NorthPoint Comments at 13-16; see Covad Comments at 29-30. The Department of Justice also asserts that manual processing of orders increases the costs of competing carriers, and that such cost "may impair the competitive vitality of competing carriers." Department of Justice Evaluation at 32. We conclude, however, that the record does not support a finding that such an impairment would occur. Although AT&T has asserted specific costs associated with various potential Bell Atlantic failures, we are unable to conclude that such costs are accurate and that an efficient competitor would be subjected to them. AT&T Crafton/Connolly Aff Attach. 2. Although significant costs associated with a BOC's manual processing of competing carrier orders might prevent an efficient competitor a meaningful opportunity to compete, the evidence in the record does not support such a conlusion. Although we have previously analyzed resale flow-through performance under the "substantially same time and manner" standard, we are unable to do so in this application. See, e.g., First BellSouth Louisiana Order, 13 FCC Rcd at 6259 (finding that BellSouth "failed to demonstrate that it is offering competing carriers the ability to order services for resale on a nondiscriminatory basis, i.e., within substantially the same time and manner as the BOC provides the service to itself"). As discussed above, unlike other BOCs that provided retail flow-through data in prior applications, Bell Atlantic has asserted that retail flow-through is a "misnomer" for its systems. Bell Atlantic Miller/Jordan Decl. at para. 57; Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 37. Moreover, the New York Commission has agreed that there is not a retail analogue in Bell Atlantic's systems. New York Commission Comments at 42 ("Since there is no retail analogue in Bell Atlantic-NY's retail system, ordering metrics are `absolute standard' metrics."). Thus, given New York and Bell Atlantic's conclusion that a retail analogue does not exist, and in absence of a credible retail analogue in the record, we find that for purposes of this application Bell Atlantic must demonstrate that the access it provides to its ordering functions offers an efficient carrier a meaningful opportunity to compete. Department of Justice Evaluation at 12 ("While Bell Atlantic's wholesale performance to resellers has not been perfect, the Department does not believe that there are performance deficiencies that are significantly impeding entry by resellers."); New York Commission Comments at 16 (concluding that Bell Atlantic has demonstrated its ability to "satisfactorily process orders" and that its "automated and manual processes are scalable."). We also note that although we have previously recognized the continuing need for all three of the competitive modes of entry, we also stated that we "continue to believe, however, that the ability of unbundled network elements, including various combinations of unbundled network elements, is integral to achieving Congress' objective of promoting rapid competition . . .." See also UNE Remand Order at para. 5. Bell Atlantic Dowell/Canny Decl. Attach B at 17, 21. Bell Atlantic Dowell/Canny Decl. Attach D at 74, 86, 98 (metrics OR-1 and OR-2); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 3 (same); Bell Atlantic Dec. 17, 1999 Ex Parte Letter correcting September data, Attach at 2. In June, July, August, and September, Bell Atlantic returned 98, 97, 99, and 99 percent, respectively, of mechanized order confirmation notices within two hours and 94, 93, 95, and 85 percent, respectively, of manually processed order confirmation notices within 24 hours. Bell Atlantic Dowell/Canny Decl. Attach D at 74, 86, 98 (metrics OR-1-02 and 1-04); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 3 (same); Bell Atlantic Dec. 17, 1999 Ex Parte Letter correcting September data, Attach at 2. Moreover, for mechanized rejection notices for those same months, Bell Atlantic returned 98, 98, 100, and 99 percent within two hours, respectively, and 96, 92, 93, and 91 percent, respectively, of manual rejection notices within 24 hours. Bell Atlantic Dowell/Canny Decl. Attach D at 74, 86, 98 (metrics OR-2-02 and 2-04); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 3 (same); Bell Atlantic Dec. 17, 1999 Ex Parte Letter correcting September data, Attach at 2. Although the September data in this footnote was also affected by the correction described in the UNE section above its effect was only marginal. New York Commission Dec. 7 Ex Parte Letter (Bell Atlantic returned 97, 95, 97 and 93 percent of its order confirmation and rejection notices on time for June, July, August, and September, respectively). On average for June, July, August, and September Bell Atlantic returned order confirmation notices in 5.27, 6.53, 6.27, and 7.25 hours, respectively, and order rejection notices in 4.20, 5.98, 5.31, and 6.25 hours, respectively. These averages were calculated by Commission staff from the Carrier-to-Carrier data provided by Bell Atlantic. Bell Atlantic Dowell/Canny Decl. Attach. D at 74, 86, 98 (metrics OR-1 and OR-2); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 3 (same); Bell Atlantic Dec. 17, 1999 Ex Parte Letter correcting September data. New York Commission Dec. 7 Ex Parte Letter (stating resale service volumes as follows: January (14,206), February (14,457), March (21,833), April (20,974), May (20,702), June (17,787), July (16,885), August (17,549), September (22,856)). See supra para. 165. The Carrier-to-Carrier reports indicate that on average resale orders flowed-through 53.77, 54.02, 45.97, and 51.60 percent of the time for June, July, August, and September, 1999, respectively. Bell Atlantic Dowell/Canny Decl. Attach. D at 74, 86, 98 (metric OR-5-01); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 3 (same). Bell Atlantic Nov. 23 Ex Parte Letter (listing volumes and flow-through rates by individual carriers for resellers). We note that factors under competing carrier control, such as failing to integrate pre-ordering and ordering interfaces, adequately train and manage their employees, or invest in the necessary systems, will have significant impacts on competing carrier flow-through rates. KPMG Final Report at POP7 IV-150 (Test P7-1). See Department of Justice Evaluation at 11-12; New York Commission Dec. 7 Ex Parte Letter (listing total volumes by month for resale services). Department of Justice Evaluation at 11 (quoting Bell Atlantic Application Taylor Decl. at para. 43). The Department of Justice further described the resale entry strategy as follows: [s]pecifically resale allows CLECs-especially those that serve the more lucrative business market-to build a customer base with minimal investment while they construct their own network facilities. Resale allows those CLECs that cannot justify the cost of investing in their own network facilities, such as those serving the less lucrative residential market, the ability to offer local exchange service as part of a bundled package of telecommunications services that "one-stop shopping" customers demand. Thus, although resale alone is not likely to be a major avenue for competitive entry, particularly for serving the residential market, the number of resale line in service continues to grow in New York. Department of Justice Evaluation at 11-12. See, e.g., Letter from Dee May, Director, Federal Regulatory Affairs, Bell Atlantic, to Magalie Roman Salas, Secretary, Federal Communications Commission, CC Docket 99-295, Attach 1 (filed December 7, 1999) ("To handle the increase in complex [resale] orders, Bell Atlantic is modifying its staffing needs to meet the new work requirements."). Bell Atlantic Dowell/Canny Decl. Attach. D at 75, 87, 99 (metric OR-6-03); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 3 (same). For June, July, August, and September, respectively, Bell Atlantic's order confirmation accuracy performance was 95.10, 91.04, 95.11, and 96.30 percent. Bell Atlantic Dowell/Canny Decl. Attach. D at 74, 86, 98 (metric OR- 6-03); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 3 (same). See supra paras. 173-74. For August and September, Bell Atlantic reported service order accuracy performance of 70.37 and 56.90 percent, respectively. Bell Atlantic Dowell/Canny Decl. Attach. D at 98 (metric OR-6-01); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 3 (same). Bell Atlantic began reporting this metric in August. Specifically, on average Bell Atlantic rejected 30.59, 30.43, 29.39, and 23.50 percent of competing carrier orders in June, July, August, and September. Bell Atlantic Dowell/Canny Decl. Attach. D at 74, 86, 98 (metric OR-3-01); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 3 (same). Bell Atlantic Miller/Jordan Aff. at para. 42; Bell Atlantic Miller/Jordan Reply Decl at para. 33; id. Attach. C at 1-7 (showing monthly rejection rates and order volumes by carrier for June through August 1999). See Bell Atlantic Application at 44; Bell Atlantic Miller/Jordan Decl. at para. 67; see also Letter from Robert W. Quinn, Director, Federal Government Affairs, AT&T, to Magalie Roman Salas, Secretary, Federal Communications Commission, CC Docket No. 99-295 at 57 (filed Dec. 15, 1999) (AT&T Dec. 15 Ex Parte Letter). See Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 50; Bell Atlantic Nov. 24 Ex Parte Letter at 6-7. Bell Atlantic Application at 44. See Bell Atlantic's Miller/Jordan Decl. at para. 18; Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 50. See Bell Atlantic Nov. 24 Ex Parte Letter at 6. Bell Atlantic also states that its dispatch centers receive Open Query System reports on a daily basis and, based on information contained in these reports, call customers to reschedule appointments when an appointment has been missed. See id. Because competing carriers also have access to these reports, they would be able to reschedule missed appointments in the same manner. In particular, we find that the regular access to Open Query System reports, in addition to real-time access to order status information through SOP and WFA, allows competing LECs access to obtain information about pending orders in substantially the same time as Bell Atlantic's retail operations. See AT&T Comments at 22; AT&T Reply at 28; AT&T Crafton/Connolly Aff. at paras. 152-158; see also AT&T Dec. 15 Ex Parte Letter at 57. See NorthPoint Comments at 16-17; see also Prism Comments at 12; Z-Tel Comments at 15. See New York Commission Comments at 88; Bell Atlantic Application at 70; Bell Atlantic Reply at 10. See New York Commission Comments at 88. The Commission has indicated in prior section 271 orders that a BOC should provide order completion notification in substantially the same time and manner as it provides such information to its retail operations. See First BellSouth Louisiana Order, 13 FCC Rcd at 6264-65; BellSouth South Carolina Order, 13 FCC Rcd at 603. See also Second BellSouth Louisiana Order, 13 FCC Rcd at 20685-86 (instructing BOCs to provide competing carriers with order completion notices "in a timely and accurate manner."). In this case, however, Bell Atlantic represents that it does not provide any completion notification to its own retail representatives, and the New York Commission similarly concluded that order completion notification lacks a retail analogue. See Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 52 (explaining that if a retail representative "has some need to check on a particular feature, he or she would pull up the customer's CSR or the service order."); New York Commission Comments at 42 (indicating that ordering metrics have no retail analogue). Given the New York Commission and Bell Atlantic's conclusions that a retail analogue does not exist, and in absence of a credible retail analogue in the record, we find for purposes of this application that Bell Atlantic must demonstrate that it provides completion notification sufficient to allow an efficient competitor a meaningful opportunity to compete. See Second BellSouth Louisiana Order, 13 FCC Rcd at 20685; BellSouth South Carolina Order, 13 FCC Rcd at 615; Ameritech Michigan Order, 12 FCC Rcd at 20650 n.512. See also Performance Measurements NPRM, 13 FCC Rcd at 12847. Second BellSouth Louisiana Order, 13 FCC Rcd at 20685-86. First BellSouth Louisiana Order, 13 FCC Rcd at 6265 (indicating that "order status notices have a direct impact on a new entrant's ability to serve its customers, because they allow competing carriers to monitor the status of their resale orders and to track the orders both for their customers and their own records."). Second BellSouth Louisiana Order, 13 FCC Rcd at 20686. See also First BellSouth Louisiana Order, 13 FCC Rcd at 6265; BellSouth South Carolina Order, 13 FCC Rcd at 615. Second BellSouth Louisiana Order, 13 FCC Rcd at 20685-86; BellSouth South Carolina Order, 13 FCC Rcd at 615. See Bell Atlantic Miller/Jordan Decl. at para. 50; Bell Atlantic Dowell/Canny Decl. at para. 46 ("For every order completed in the Billing system, a completion notice has been sent."). Bell Atlantic Dowell/Canny Decl. at para. 46. Bell Atlantic Dowell/Canny Decl. at para. 48; Bell Atlantic Miller/Jordan Decl. at para. 51; see also New York Commission Comments at 49; NYPSC Additional Guidelines Order at 16 (noting that in Carrier Working Group meetings during August and September Bell Atlantic offered to notify competing carriers when the work completion has been entered into its service order processing system). Bell Atlantic Dowell/Canny Decl. at para. 48. Id. at para. 47; Bell Atlantic Miller/Jordan Decl. at paras. 50, 51. Although Z-Tel complains that it does not receive affirmative notification from Bell Atlantic over the Web GUI interface, we find that this functionality is available using the EDI interface. See Z-Tel Comments at 16, 19-20. Bell Atlantic Dowell/Canny Decl. at para. 46; Attach. B. at 26-27 (describing metrics OR-4-01, OR-4-02, OR-4-03). Bell Atlantic Dowell/Canny Decl. at para. 47; Bell Atlantic Miller/Jordan Decl. at para. 50; see also NYPSC Guidelines Order at 2 (adopting, after input from Bell Atlantic and competing carriers in the Carrier-to-Carrier collaborative, a general performance standard of 95 percent as a reasonable and achievable level that will permit competing carriers to enter the local exchange market). Bell Atlantic Dowell/Canny Decl. Attach. D at 74, 79, 86, 91, 98, 102 (metric OR-4-02 for June, July, and August 1999); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 3, 7 (metric OR-4-02 for September 1999). KMPG Final Report at POP5 IV-114-15 (excluding approximately ten percent of orders where KPMG did not receive a completion notice due to a problem occurring primarily in January 1999 that Bell Atlantic later resolved). See New York Commission Comments at 49. KPMG Final Report at POP5 IV-116. See also New York Commission Comments at 49. See AT&T Crafton/Connolly Aff. at para. 260 (claiming that AT&T received only 79 percent of billing completion notices on time for AT&T orders that were eligible to receive such notices in September); AT&T Crafton/Connolly Reply Aff. at para. 83; AT&T Pfau/Kalb Reply Decl. at para. 56 (indicating that AT&T received only 72 percent of billing completion notices on time for eligible October orders). AT&T does not demonstrate that the delay is attributable to Bell Atlantic's systems. See NYPSC Additional Guidelines Order at 16-17; Performance Measurements NPRM, 13 FCC Rcd at 12845, 12847 (tentatively concluding that incumbent LECs must measure the average completion notice interval, or "the amount of time it takes an incumbent LEC to send a competing carrier notice that work on an order has been completed" by "subtracting the date and time that it completed the work from the date and time a valid completion notice leaves its OSS interface."). See also NYPSC Permanent Rule Order App. at 21-22; NYPSC Guidelines Order, App. 3 at 1 (directing parties in the Carrier-to-Carrier collaborative to consider measuring the time of completion of the physical work). NYPSC Additional Guidelines Order at 17. Id. We note that Bell Atlantic's pre-ordering interfaces enable carriers to view a pending order's installation status to determine whether the physical work on an order has been completed. See infra at Section V.B.1.c. See also Bell Atlantic Dowell/Canny Decl. at para. 48; Bell Atlantic Miller/Jordan Decl. at para. 21. Moreover, Bell Atlantic notifies competing carriers by phone when hot cut and trunk orders are completed. We therefore do not consider AT&T's and MCI WorldCom's allegations that Bell Atlantic does not deliver timely work completion notices particularly probative to approval of this application. See AT&T Crafton/Connolly Aff. at para. 259 (claiming that AT&T received only 66 percent of work completion notices on time for AT&T orders that were eligible to receive such notices in September); MCI WorldCom Kinard Decl. at paras. 16-17 (indicating that notification of provisioning completions "still takes too long."). See AT&T Crafton/Connolly Aff. at paras. 259, 260 (claiming that AT&T did not receive a work completion notice for 23 percent, nor a billing completion notice for 17 percent, of eligible September orders); AT&T Pfau/Kalb Reply Decl. at paras. 55, 56 (indicating that AT&T did not receive work completion notices for 19 percent of orders submitted in the first half of October and failed to receive billing completion notices for 24 percent of such orders); MCI WorldCom Kinard Decl. at para. 18 (claiming that MCI WorldCom failed to receive billing completion notices, but speculating that the addition of provisioning completion notices may improve the situation); MCI WorldCom Reply at 9-11; MCI WorldCom Lichtenberg/Sivori Reply Decl. at 9-12 (indicating that MCI WorldCom failed to receive completion notices for a number of August, September and October orders). MCI WorldCom admits that for half of the August and September orders that are missing billing completions, it did receive a work completion notice. MCI WorldCom Lichtenberg/Sivori Reply Decl. at 10. Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 52 (indicating that Bell Atlantic "is prepared to implement this functionality in April"). A fielded complex completion notification takes information about a completed order and assigns it to specific fields. AT&T Comments at 22. AT&T states that it can use fielded completion notices to confirm that Bell Atlantic provisioned the order accurately and that the customer received the correct services and features. AT&T Crafton/Connolly Aff. at paras. 159, 162. AT&T Crafton/Connolly Aff. at para. 165 n.87. According to AT&T, carriers agreed to the postponement because of concerns "about the effects of the implementation on the Y2K moratorium." Id. We discuss loop provisioning below. See infra Section V.D.2.a. The Gertner/Bamberger study was submitted to us by Bell Atlantic. It examines the reasons for the differences in the observed Average Completed Intervals for competing carriers orders as compared to orders for Bell Atlantic's retail customers. For a discussion of the study, see infra paras. 203-210. Ameritech Michigan Order, 12 FCC Rcd at 20612-58. We will use "Average Installation Interval," "Average Completed Interval," and "Average Completion Interval" interchangeably for purposes of this discussion. Ameritech Michigan Order, 12 FCC Rcd at 20633-34.. Id. at 20632-33. Id. at 20633. Id.. Id. The Commission also stated that data on the percentage of installations completed within a certain number of days may be useful, even though such data could mask discriminatory conduct. See id. at 20631-32. Performance Measurements NPRM, 12 FCC Rcd 12842-43. The Average Completion Interval compares the average length of time it takes an incumbent LEC to complete orders for competing carriers with the average length of time it takes to complete comparable incumbent LEC retail orders. The Percentage of Due Dates Missed seeks to determine whether the agreed-upon due dates for order completion are equally reliable for orders placed by competing carriers and orders placed by an incumbent LEC's end user customers. Id. Id. at 12844. Bell Atlantic also provides other performance measurements, including Percent Completed within "X" Days, Percent Missed Appointments, Average Delay Days, and Percent Installation Troubles reported within "X" Days. Non-dispatch refers to orders for which no field work was needed for provisioning by a Bell Atlantic technician. Dispatch orders require a technician to be dispatched in order to fulfill the order. Bell Atlantic Dowell/Canny Decl. at para. 59; see also Performance Measurements NPRM, 12 FCC Rcd 12841 n.71. Bell Atlantic Dowell/Canny Decl. at para. 63. Standard intervals are the minimum number of days that Bell Atlantic offers for the provision of service for orders not requiring dispatch. They vary according to the type of products and services being ordered. For example, the product Remote Call Forwarding has a standard interval of two days, while Call Waiting can receive same day service (if ordered before 3:00), and Caller ID has a standard interval of four days. Therefore, if a customer orders Caller ID, Bell Atlantic says that the earliest it can provision the customer is four days later. Bell Atlantic Dowell/Canny Decl. at para. 63 & Attach. B, App. L at 143. Bell Atlantic Dowell/Canny Dec. at para. 63. Smarts Clock is a calendar of available appointment dates for orders requiring dispatch. On the calendar a red mark indicates that Bell Atlantic has reached its capacity for that day; a yellow mark indicates that Bell Atlantic is close to reaching capacity, but is still accepting due date requests; a green mark indicates that Bell Atlantic has sufficient capacity that the carrier's due date request for that day will likely be accepted. Bell Atlantic Dowell/Canny Decl. at para. 63; Bell Atlantic Dowell/Canny Reply Decl. at para 53. The only test criterion to receive a "Satisfied with Qualifications" concerned the assignment of skilled personnel to the Regional CLEC Coordination Center (RCCC). It received this qualification because "Bell Atlantic did not replicate the retail processes at the RCCC. However, KPMG determined that equal functionality existed." KPMG Final Report at POP11 IV-282 to IV-284. According to KPMG, "[t]he focus of the evaluation [was] on the activities downstream from order entry through service activation. The objective of this test [was] to evaluate the degree to which the provisioning environment supporting wholesale orders is on parity with provisioning for Bell Atlantic New York retail orders." KPMG Final Report at POP11 IV-258. KPMG Final Report at POP11 IV-258 to IV-269. KPMG Final Report at POP11 IV-284. BellSouth First Louisiana Order, 13 FCC Rcd at 6280-81 (concluding that BOCs must provide equivalent access to due dates); see also BellSouth Second Louisiana Order, 13 FCC Rcd at 20667; BellSouth South Carolina Order, 13 FCC Rcd at 629-30; Ameritech Michigan Order, 12 FCC Rcd at 20639-41. AT&T Crafton/Connolly Aff. at paras. 74-5; Covad Conley/Poulicakos Decl. at para. 24; MCI WorldCom Lichtenberg/Sivori Decl. at para. 68; CoreComm Comments at 13-14; Prism Comments at 9 n.16. Both AT&T and MCI WorldCom claim that they normally request longer intervals than the standard interval because of the problem of getting the due date they request. MCI WorldCom Lichtenberg/Sivori Decl. at para. 68; AT&T Pfau/Kalb Aff. at para. 143. KPMG also found that 2.4 percent of its test orders received confirmed due dates earlier than requested. KPMG Final Report at POP5 IV-113 & Table IV-5.16. Bell Atlantic provides a study that examined the reasons why Average Completed Intervals for competing carriers might be longer. This study demonstrates that the average completed interval is longer than the average requested interval, for UNE-P orders. Bell Atlantic Gertner/Bamberger Decl. at Table 2. As we discuss below, we assume that the confirmed due dates are the same as the completed dates. Bell Atlantic Gertner/Bamberger Reply Decl. at 1 n.1. As the Commission has stated before, we would be concerned if we saw that confirmed due dates were set significantly later than was requested. See BellSouth Second Louisiana Order, 13 FCC Rcd at 20667; Ameritech Michigan Order, 12 FCC Rcd at 20639-41; BellSouth First Louisiana Order, 13 FCC Rcd at 6280-81; BellSouth South Carolina Order, 13 FCC Rcd at 629-30. KPMG was "Satisfied" with orders submitted through the GUI, and "Satisfied with Qualifications" for orders submitted through the EDI interface. KPMG Final Report at POP2 IV-38-9, POP5 IV-113. No reason for the qualification designation for EDI orders was given, although KPMG indicated in its comments that the 88 percent of orders having confirmed due dates the same as the due date requested was a key factor in its analysis. KPMG Final Report at POP5 IV-113. The New York Commission states that "[t]he record before [them] does not suggest that [competing LECs] have been having problems receiving intervals for platform orders as requested or within the standard intervals set forth in the Carrier-to-Carrier guidelines. MCI WorldCom acknowledged that because it requested longer intervals for certain UNE-P products, [Bell Atlantic's] overall average interval offered and completed metrics may be longer than they otherwise would be. Moreover, [Bell Atlantic's] good missed appointment performance demonstrates that it is meeting requested intervals." New York Commission Comments at 69 n.1. For example, in September Bell Atlantic missed appointments for 0.03 percent of competing carriers' non- dispatch UNE-P orders, versus 0.79 percent of its own corresponding retail orders. For dispatch orders, it missed 8.9 percent of competing carriers' appointments and 12.1 percent of its own retail appointments. The four month average (June through September) missed appointment rate for resale non-dispatch orders is 0.04 percent for competing carriers, versus 0.70 percent for Bell Atlantic customers; and for resale dispatch orders it is 7.26 percent for competing carriers versus 10.32 percent for its own retail customers. For UNE platform non-dispatch orders it is 0.04 percent for competing carriers versus 0.70 percent for its retail customers; and for dispatch orders it is 6.85 percent for competing carriers versus 10.32 percent for its retail customers. Bell Atlantic Dowell/Canny Decl. Attach. D; Bell Atlantic Dowell/Canny Reply Decl. Attach. C. For example, for resale POTS orders, in September only 0.74 percent of competitive LEC customers reported difficulties within the first seven days of installation, compared to 3.15 percent of Bell Atlantic customers. Bell Atlantic Dowell/Canny Decl. Attach. D; Bell Atlantic Dowell/Canny Reply Decl. Attach. C. For June through September, resale POTS orders, dispatch and non-dispatch, business and residential, generally showed a monthly difference of a half day to a full day longer to fulfill for competitive LEC customers, and the monthly differences were usually statistically significant, with the exception of July for residential dispatch orders, for which the difference was not statistically significant. The four month average (June-September) difference for resale POTS orders is 1.18 days for dispatch business, 0.80 days for dispatch residential, 0.51 days for non-dispatch business, and 0.87 days for non-dispatch residential. Bell Atlantic Dowell/Canny Decl. Attach. D; Bell Atlantic Dowell/Canny Reply Decl. Attach. C. The difference in times was greater for UNE platform orders, for the same time period, and were always statistically significant. Competitive LEC UNE platform non-dispatch orders took from 0.8 to 2.0 days longer for June through August, averaging more than four months (June- September 1999) 2.43 days for competing carrier orders versus 1.09 days for Bell Atlantic orders, for a difference of 1.34 days. Meanwhile, UNE platform dispatch orders took from 2.6 to 3.6 days longer, averaging over the four months 6.49 days for competing carriers orders versus 3.26 days for Bell Atlantic orders, for a difference of 3.23 days. Bell Atlantic Dowell/Canny Decl. Attach. D; Bell Atlantic Dowell/Canny Reply Decl. Attach. C. The Carrier to Carrier report also contains data about how many orders were completed within "X" number of days for Bell Atlantic and competitive LEC customers, with metrics provided for "X" ranging from one to six days (the "Percent Completed within 'X' Days" metrics). Bell Atlantic Dowell/Canny Decl. at para. 61. These metrics paint a similar picture to the average completed intervals data, of competitive LEC orders having longer completion times than Bell Atlantic retail orders. The differences for this measure for UNE platform orders were statistically significant, for the months of June through September. Another interval metric, which measures the time it takes for Bell Atlantic to provide service to customers, is average delay days for missed appointments. This metric, which measures how long it takes to complete service to a customer if the appointment has been missed, generally shows large and statistically significant differences in performance in favor of Bell Atlantic retail customers, for both UNE and resale orders. For example, the average delay days for UNE platform orders for September for Bell Atlantic retail customers was 4.76 days, while for competitive LEC customers it was 6.66 days. Although Carrier to Carrier metrics are intended to exclude orders placed by competitive LECs that request due dates later than they are offered, this is not happening due to a coding error on the part of competing carriers. For example, if the requested due date (by the competitive LEC or by a retail customer) is later than the offered due date, then the order is supposed to be coded with an "X". If the customer accepts the offered due date, then the order should be coded with a "W." All orders coded with an "X" are excluded from the interval metrics. However, if a competitive LEC fails to mark orders that request later due dates with an "X", they will be counted in the metrics, and are likely to increase the reported completion intervals because of their longer intervals. Bell Atlantic has found that in some categories large numbers of competitive LEC orders are incorrectly coded as "W." Bell Atlantic Dowell/Canny Decl. at paras. 65-66. We note that in March 2000, Bell Atlantic's systems will begin to automatically code orders requesting later due dates with an "X," thus eliminating this bias to the data. Bell Atlantic Dowell/Canny Decl. at para. 67. Bell Atlantic Dowell/Canny Decl. at para. 62. For non-dispatch orders, the offered intervals a competitive LEC may choose depend on the service order. As described above, both Bell Atlantic representatives and competitive LECs are given the same list of standard intervals. The standard interval varies by service requested. So, for example, if a customer (competitive LEC or Bell Atlantic retail) asks for Call Waiting on an existing line, Bell Atlantic offers same day service if the order is placed before 3:00 pm. If the customer wants Caller ID, the standard interval offered is 4 days. Therefore if a large proportion of competitive LEC customers order Caller ID, while most Bell Atlantic retail customers are only ordering Call Waiting, completion intervals will be much longer for competitive LEC customers than for Bell Atlantic retail customers. Bell Atlantic Dowell/Canny Decl. at paras. 64-65; Bell Atlantic Dowell/Canny Reply Decl. at para. 53. For installations of service requiring dispatch of a Bell Atlantic service technician, Bell Atlantic argues that the average completed interval data for competitive LECs is skewed because it includes a larger share of orders in areas that are served by busier garages and, as a result, reflect later due dates available from Smarts Clock. Bell Atlantic argues that the dates received from Smarts Clock can vary by garage location, since busier garages tend to offer later dates. Therefore, geographic location of the customer can affect the completion intervals for dispatch orders. Bell Atlantic Dowell/Canny Reply Decl. at para. 53. Bell Atlantic Gertner/Bamberger Decl. AT&T Pfau/Kalb Aff. at paras. 140-50. Second BellSouth Louisiana Order, 13 FCC Rcd at 20683. Bell Atlantic Application, App. A, Dowell/Canny Decl. at para. 66. See Bell Atlantic Dowell/Canny Decl. at para. 66; Bell Atlantic Gertner/Bamberger Reply at paras. 3-4 & Table 1. The Gertner/Bamberger study used a randomly chosen sample of "W" coded non-dispatch 1-5 line resale POTS and UNE platform orders to examine the impact of incorrect "W" coding on the completion intervals for non-dispatch orders. The study examined 300 orders for June, 800 for July, and 800 for August. Bell Atlantic Gertner/Bamberger Decl. at 1 n.2. For each order in the sample, the study compared the requested interval with the standard interval appropriate to that order based on the service requested, to determine if the order was improperly coded as "W." The study then examined the impact of the improperly coded orders on the average requested interval. In addition the study compared the average requested intervals with the average completed intervals, to see if, on average, Bell Atlantic was filling the orders within the time requested. Bell Atlantic Gertner/Bamberger Decl. at paras. 7-12 & n.2. See Bell Atlantic Bamberger/Gertner Decl. at para. 12. We note that the findings of the Gertner/Bamberger study are applicable to the Average Completed Interval data for dispatch orders, even though the Gertner/Bamberger study examined only non-dispatch orders for resale services and UNE-P. Just as the differences between wholesale and retail Average Completed Interval times for non-dispatch orders are likely to be inflated by these factors, so will dispatch orders, and average completed intervals for other types of dispatch orders, such as UNE loops. We note that other metrics, such as Percent Completed in "X" Days, and Average Delay Days, will also be affected in a similar manner by the factors identified in the study. MCI WorldCom says it sets a default due date of four days for migrations, and seven days for new orders for UNE platform orders. MCI WorldCom Lichtenberg/Sivori Declaration at para. 68. AT&T states that it requests five day intervals for UNE platform orders, even if the standard interval is only two days. AT&T Pfau/Kalb Aff. at para. 143. For example, in August the average standard interval for UNE-P orders that were within the standard interval was 1.84 days, while the average standard interval for retail orders was only 1.22 days, a difference of 0.62 days. Bell Atlantic Gertner/Bamberger Reply Decl. at paras. 5-6 & Table 2. Gertner and Bamberger also point out that customer-caused delays in completing orders that missed the due date can also lengthen the Average Completed Interval for wholesale orders. They analyzed the data looking for orders more than three days late, which they considered to be "outliers." They found that for August customer delays increased the Average Completed Intervals for platform and resale orders. Meanwhile there was little or no impact on June or July's intervals. Bell Atlantic Gertner/Bamberger Reply Decl. at paras. 7-9 & Table 3. Bell Atlantic Dowell/Canny Rep. Decl. at para. 53. Department of Justice Evaluation at 33 n.89; AT&T Pfau/Kalb Aff. at para. 143. Bell Atlantic Gertner/Bamberger Decl. at paras. 12-14 & Table 2. For resale orders within the standard interval, Gertner and Bamberger found that the average completed interval of 0.99 days was less than the average requested interval of 1.09 days. Bell Atlantic Gertner/Bamberger Decl. at Table 2. Gertner and Bamberger conclude that Bell Atlantic generally met the Standard Intervals if competitive LECs request service within the Standard Interval. Bell Atlantic Gertner/Bamberger Decl. at paras. 12-14. AT&T Pfau/Kalb Aff. at paras. 140-43. The average completed interval for UNE-P orders requesting the standard interval was 2.36 days, while the average requested interval was 1.84 days, for a difference of 0.52 days. Bell Atlantic Gertner/Bamberger Decl. at Table 4; Department of Justice Evaluation at 33 n.89; AT&T Pfau/Kalb Aff. at para. 143. Bell Atlantic Gertner/Bamberger Reply Decl. at 1 n.1. Only 0.03 percent in September according to the Carrier to Carrier metrics. Bell Atlantic Dowell/Canny Reply Decl. Attach. C. Bell Atlantic Gertner/Bamberger Reply Decl. at paras. 10-11; Bell Atlantic Dowell/Canny Reply Decl. at para. 54. We also note that the New York Commission reports that competing LECs have not been having difficulty getting the intervals they request. New York Commission Comments at 69 n.1. We said in the Ameritech Michigan Order that information about missed appointments can explain inconsistencies in the Average Completed Intervals. See Ameritech Michigan Order, 12 FCC Rcd 20633. KPMG did some analysis of the data for January for non-dispatch Average Completed Intervals, and after accounting for geography, number of lines, type of order, and date of completion, still found an unexplained difference of 0.56 days. It found a similar difference in the closely related Average Offered Interval metric. KPMG Final Report at POP8 IV-193 to IV-194. KPMG determined that with respect to its analysis of the metrics, it was "Not Satisfied," because of these detected differences. KPMG Final Report at POP8 IV-202. Bell Atlantic argues that the KPMG analysis did not fully account for the impact of differing order types, because KPMG's correction for "order types" only took into account whether orders were "N" (new), "T" (to another address), and "C" (change existing features), and not the various services ordered, with their differing standard intervals. Bell Atlantic Dowell/Canny Reply Decl. at para. 51. The adjusted differences were calculated as follows. The Bell Atlantic retail Average Completed Interval was taken from the Carrier to Carrier metrics. To obtain the competing carrier's adjusted intervals for June, July and August, the study's reported Average Completed Interval for only orders within the standard interval (which corrects for the "X" coding problem) were used (top line of Table 4 in the Gertner/Bamberger Decl.), and then adjusted for the order mix problem by taking the difference between the wholesale and retail average standard intervals provided in Gertner/Bamberger's Reply (right column of Table 2). We found that the adjusted differences in Average Completed Intervals for non-dispatch UNE-P orders is 0.43 days for June, 0.36 days for July, and 0.67 days for August. These differences should all be statistically significant, with z-scores less than -7. The differences for resale are more difficult to determine, because the Carrier to Carrier data is broken down by business and residential, while the study aggregates the two together. However, the Carrier to Carrier data for business and residential can be combined to yield aggregate results. If this is done, and the competing carrier data is then adjusted for the factors discussed above, the differences come out to less than a third of a day for both business and residential orders for July and August, and competing LECs have shorter intervals for June. For the details of our analysis, see infra Appendix C. In future applications, we expect applicants to correct their Average Completed Interval data for factors outside the BOC's control, as the Commission recommended in Ameritech Michigan Order and as we have done here using data from the study. Ameritech Michigan Order, 12 FCC Rcd at 20633. The New York Commission states "the remaining unexplained difference of a half day does not warrant a conclusion that Bell Atlantic is offering discriminatory service." New York Commission Comments at 50. The Carrier-to-Carrier guidelines require the return of 95 percent of mechanized order confirmation and rejection notices within two hours of submission to Bell Atlantic, and 95 percent of manually processed order confirmation and rejection notices for orders under ten lines within 24 hours of submission. Bell Atlantic Dowell/Canny Decl. Attach. B at paras. 17, 20. See New York Commission Comments at 53 (finding that competing carriers have nondiscriminatory access to Bell Atlantic's maintenance and repair systems). Maintenance and repair issues specific to unbundled local loops are discussed in checklist item 4 below. See Second BellSouth Louisiana Order, 13 FCC Rcd at 20692; Ameritech Michigan Order, 12 FCC Rcd at 20613, 20660-61. Second BellSouth Louisiana Order, 13 FCC Rcd at 20692-93. Id. at 20692. See id. See id. at 20693; BellSouth South Carolina Order, 13 FCC Rcd at 593-94; Ameritech Michigan Order, 12 FCC Rcd at 20617. The Commission has previously indicated that, without electronic access for competing carriers, the BOC's ability to correct trouble reports while on line with the customer would be a "crucial competitive advantage." Second BellSouth Louisiana Order, 13 FCC Rcd at 20698. See Bell Atlantic Miller/Jordan Decl. at para. 68. In the past, Bell Atlantic also offered carriers access to an Electronic Interface Format (EIF) application-to-application interface, and one carrier presently is using that interface to access maintenance and repair functions. The main RETAS application is a routing tool that accepts trouble administration messages, routes requests to the appropriate back end systems and returns electronic responses. KPMG Final Report at M&R1 V-7. The New York Commission describes RETAS as a "web-based interactive system that allows a [competing carrier], upon receiving a report of trouble from a customer, to test the line and, if appropriate, arrange for a Bell Atlantic- NY technician to repair the problem," as well as to monitor progress on the trouble report and learn when the problem was corrected. New York Commission Comments at 50-51. Bell Atlantic's back office maintenance and repair systems include: StarMEM for memory feature fixes; Work Force Administrator (WFA) for processing special services trouble tickets and trouble history inquiries; Loop Maintenance Operating System (LMOS) for processing POTS trouble tickets and trouble history; Mechanized Loop Test (MLT) for conducting a POTS mechanized loop test; and Switched Access Remote Test System (SARTS) for conducting a special services test. See Bell Atlantic Miller/Jordan Decl. Attach. E. Bell Atlantic submits that competing carriers have more automatic functionality than Bell Atlantic's retail representatives. For example, in conducting a mechanized loop test, a Bell Atlantic retail representative must assess the circuit type, geographic region and destination, and manually submit the test to the proper back end system, whereas RETAS automatically sends a competing carrier's test to the proper system. Similarly, a Bell Atlantic representative must interpret the highly technical test results, but the system automatically analyzes the test results and issues a recommendation for competing carriers. Bell Atlantic Miller/Jordan Decl. at para. 72. Bell Atlantic Application at 45 n.40; Bell Atlantic Miller/Jordan Decl. at para. 68. In response to a KPMG finding that competing carriers did not have the same access as Bell Atlantic's retail representatives to extended trouble history for a given line, Bell Atlantic added that functionality to RETAS in June 1999. Bell Atlantic Miller/Jordan Decl. at para. 72. Since June, competing carriers can access the three most recently reported trouble tickets on any given line. Id. Although the Web GUI can be used to report trouble associated with unbundled loops, carriers can also submit unbundled loop trouble tickets manually. Bell Atlantic Miller/Jordan Decl. at para. 75. We reject as unsupported by the record evidence Prism's mere assertion that it must manually submit trouble tickets because RETAS cannot be used for unbundled network elements. See Prism Comments at 13. See Bell Atlantic Application at 45; Bell Atlantic Miller/Jordan Decl. at 74 (indicating 47,000 transactions in July). See KPMG Final Report at M&R1 V-13-23 (RETAS functional and parity evaluation); M&R5 V-75-77 (parity evaluation). Although Bell Atlantic submitted average volume per month on a region-wide rather than state-wide basis, KPMG determined that Bell Atlantic could handle approximately 500 transactions per hour with acceptable response time performance. See KPMG Final Report at M&R2 V-36-37, 38-43. See also KPMG Final Report at M&R3 V-47-55 (scalability review of system infrastructure, gateways and resources). AT&T Comments at 26-27; AT&T Crafton/Connolly Aff. at paras. 169-71. Although one carrier is accessing maintenance and repair functions through the application-to-application EIF interface, we find that Bell Atlantic does not make that interface available generally to any requesting carrier, and therefore do not rely on it for purposes of our analysis. Second BellSouth Louisiana Order, 13 FCC Rcd at 20695-96. Id. In conjunction with AT&T and MCI WorldCom, Bell Atlantic is developing an application-to-application interface for local service maintenance and repair functions that employs electronic bonding. Bell Atlantic Miller/Jordan Decl. at para. 73. See also Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 56 (expressing willingness to work with other interested carriers in developing electronic bonding). Aside from one function (mechanized loop testing for local POTS, which Bell Atlantic is in the process of implementing), Bell Atlantic represents that there are no application-to-application industry standards for local services maintenance and repair. Bell Atlantic Application at 45; Bell Atlantic Miller/Jordan Decl. at para. 73; Bell Atlantic Reply at 36; Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 56. Without citing any specific standard, AT&T asserts generally that industry standards for reporting maintenance and repair troubles using electronic bonding have been in effect since 1992 and that Bell Atlantic is required to implement them pursuant to its commitments in the Bell Atlantic-NYNEX merger proceeding. AT&T Crafton/Connolly Aff. at para. 171 n.90. Without reference to any specific standard, the record is insufficient for us to verify AT&T's claim. Moreover, AT&T does not represent that the unspecified 1992 industry standard is for local exchange services. New York Commission Comments at 51; Bell Atlantic Reply at 37 n.41. We note that RCN complains that Bell Atlantic does not permit competing carriers to submit a single trouble ticket when a loop-transport combination experiences service disruption. See RCN Comments at 2, 9-10. We do not find that this practice warrants a finding that Bell Atlantic fails to comply with this checklist item. New York Commission Comments at 51; Bell Atlantic Reply at 37 n.41; Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 59. The new functionality enables RETAS to check SOP to validate the presence of recently- completed service order. See KPMG Final Report M&R5 V75-76. Bell Atlantic Miller/Jordan/Zanfini Reply Decl. at para. 59. Covad Comments at 31-32. Covad claims that it is unable to open a trouble ticket for at least 24 hours after the due date because neither the Regional CLEC Coordination Center nor the Regional CLEC Maintenance Center will take responsibility for an improperly provisioned loop. See New York Commission Comments at 53. Response time, or the number of seconds from the issuance of a query to the receipt of a response by the requesting carrier, is measured for competing carriers using actual response times reported by the RETAS gateway and for Bell Atlantic retail using actual response times reported by its Caseworker retail trouble report system. See Bell Atlantic Dowell/Canny Decl. Attach. B at 50. The New York Commission formerly required Bell Atlantic to report maintenance and repair response times using absolute standards derived from the KPMG test results. See New York Commission Comments at 52-53; NYPSC Permanent Rule Order App. at 49 (recommending the temporary use of KPMG response times as the performance standards while Bell Atlantic investigates response times experienced by KPMG, competing carriers and its retail operations). In July and August 1999, with the exception of one measurement, Bell Atlantic failed to meet these absolute standards either for itself or for competing carriers. See Bell Atlantic Dowell/Canny Decl. Attach. D at 85, 97 (metrics MR-1-01, MR-1-03, MR- 1-04, MR-1-06 for July and August 1999). Upon further review, the New York Commission found that the KPMG-based absolute standards did not measure each transaction processing step and were not "representative of real world" experience. New York Commission Comments at 52-53. Accordingly, based on a consensus reached by Bell Atlantic and competing carriers in the Carrier-to-Carrier collaborative, the New York Commission adopted a modified performance standard of "parity plus not more than four seconds." NYPSC Additional Guidelines Order at 10-11. Under this modified standard, Bell Atlantic will report maintenance and repair OSS response times according to the same performance standard that applies to its reporting of pre-ordering OSS response times. In light of Bell Atlantic's retail operations, we agree that the parity standard is a more appropriate measure of maintenance and repair response time than the absolute benchmarks. See supra para. 146; KPMG Final Report at M&R1 V-7-8 (describing the layers of security for RETAS to limit unauthorized use and to preserve data confidentiality). Bell Atlantic Dowell/Canny Decl. Attach. D at 73, 85, 97 (metrics MR-1-01, MR-1-03, MR-1-04 for June, July, and August 1999); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 2 (metrics MR-1-01, MR-1-03, MR- 1-04 for September 1999). Bell Atlantic does not submit statistical analyses for response times, therefore we review any deviation from the performance standard. Bell Atlantic Dowell/Canny Decl. Attach. D at 73, 85, 97 (metric MR-1-06 for June, July, August 1999); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 2 (metric MR-1-06 for September 1999). Although, using the "parity plus four seconds" standard, Bell Atlantic processed test requests 24.32 seconds faster for its retail operations in August (82.40 seconds for retail compared with 110.72 seconds for competing carriers), Bell Atlantic achieved parity in September (83.63 seconds for retail; 83.17 seconds for competing carriers). Bell Atlantic deviated from the standard by 3.84 seconds in June, 5.38 seconds in July, 8.05 seconds in August, and 7.69 seconds in September. Bell Atlantic Dowell/Canny Decl. Attach. D at 73, 85, 97 (metric MR-1- 01 for June, July, and August 1999); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 2 (metric MR-1-01 for September 1999). We therefore reject AT&T's contention that these response times are "far longer" than Bell Atlantic's retail operations. AT&T Crafton/Connolly Aff. at para. 172. Although it met the standard in June and August, Bell Atlantic deviated from the standard by .96 of a second for July and .34 of a second for September. Bell Atlantic Dowell/Canny Decl. Attach. D at 73, 85, 97 (metric MR- 1-04 for June, July, and August 1999); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 2 (metric MR-1-04 for September 1999). Second BellSouth Louisiana Order, 13 FCC Rcd at 20693. Id. Bell Atlantic submits performance measurements that calculate the "mean time to repair," or average duration from receipt of a trouble report through its clearance. Bell Atlantic Dowell/Canny Decl. Attach. B at 57- 59. See also Performance Measurements NPRM, 13 FCC Rcd at 12854 (discussing measurement of the average time to restore). For resale, Bell Atlantic took less time to repair reported loop and central office trouble from its competitors' customers than its own retail customers in each month in June through September 1999. See Bell Atlantic Dowell/Canny Decl. Attach. D at 77, 89, 101 (metrics MR-4-01, MR-4-02, MR-4-03 for June, July, and August 1999); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 6 (metrics MR-4-01, MR-4-02, MR-4-03 for September 1999). Similarly, for the mean time to repair unbundled network elements, Bell Atlantic performed better for its competitors' customers than for its own retail customers in June, July, and September 1999. See Bell Atlantic Dowell/Canny Decl. Attach. D at 82, 94 (metric MR-4-01 for June, July, and August 1999); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 11 (metric MR-4-01 for September 1999). Although Bell Atlantic's performance deviated slightly for the mean time to repair loops in August (26.22 hours for competing carriers versus 25.32 hours for Bell Atlantic retail), given that the difference is slight and did not cause a statistically significant difference in the total mean time to repair, we find that Bell Atlantic repaired unbundled network element troubles in substantially the same time for itself and for competing carriers. With respect to special services, Bell Atlantic met the standard each month from June through September 1999, for both resale and unbundled network elements. See Bell Atlantic Dowell/Canny Decl. Attach. D at 77, 82, 89, 94, 101, 106 (metric MR-4-01 for June, July, and August 1999); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 6, 11 (metric MR-4-01 for September 1999). For both resale and unbundled network elements, Bell Atlantic cleared a higher percentage of trouble reports within 24 hours for competing carriers than for itself in each month from June through September 1999. See Bell Atlantic Dowell/Canny Decl. Attach. D at 77, 82, 89, 94, 101, 106 (metric MR-4-04 for June, July, and August 1999 for POTS and Special Services); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 6, 11 (metric MR-4-04 for September 1999 for POTS and Special Services). For resale POTS services, from June through September 1999, a smaller percentage of competing carriers' customers were out of service at the 4-hour, 12-hour and 24-hour measured intervals than Bell Atlantic's retail customers. See Bell Atlantic Dowell/Canny Decl. Attach. D at 77, 89, 101, (metrics MR-4-06, MR-4-07, MR-4-08 for June, July, and August 1999); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 6 (metrics MR-4-06, MR- 4-07, MR-4-08 for September 1999). For POTS service through unbundled network elements, the results were more varied. From June through September, although a smaller percentage of competing carriers' customers were out of service after 4 hours and after 24 hours compared with Bell Atlantic's retail customers, a higher percentage were out of service at the 12-hour interval. See Bell Atlantic Dowell/Canny Decl. Attach. D at 82, 94, 106, (metrics MR-4-06, MR-4-07, MR-4-08 for June, July, and August 1999); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 11 (metrics MR-4-06, MR-4-07, MR-4-08 for September 1999). Considering the performance data for the 4-hour, 12-hour and 24-hour intervals collectively, we do not consider the slight deviations in percent of troubles out of service at the 12-hour interval indicative that Bell Atlantic takes longer to repair trouble for customers of its competitors than for its own retail customers. Similarly, with respect to specials, a statistically significant percent of Bell Atlantic's competitors' resale customers were out of service after four hours, but not after 24 hours. See Bell Atlantic Dowell/Canny Decl. Attach. D at 77, 82, 89, 94, 101, 106, (metrics MR-4-06, MR-4-08 for June, July, and August 1999); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 6, 11 (metrics MR-4-06, MR-4-08 for September 1999). See Covad Conley/Poulicakos Decl. at para. 10; Prism Comments at 4, 13. See Second BellSouth Louisiana Order, 13 FCC Rcd at 20694. See Bell Atlantic Dowell/Canny Decl. Attach. B at 53, 60. In prior orders the Commission specifically instructed BOCs to provide performance data showing repeat trouble reports. Second BellSouth Louisiana Order, 13 FCC Rcd at 20694 (using the repeat trouble report rate as an indicator of a BOC's performance in the initial resolution of trouble reports); Ameritech Michigan Order, 12 FCC Rcd at 20657. See also Performance Measurements NPRM, 13 FCC Rcd at 12854 (indicating that the percentage of access lines that receive trouble tickets in a thirty-day period is indicative of the quality of network components supplied by the incumbent LEC, and the frequency of repeat troubles in a thirty-day period reflects the quality of the incumbent LEC's initial resolution of troubles). See Bell Atlantic Dowell/Canny Decl. Attach. D at 77, 82, 89, 94, 101, 106 (metric MR-2-02 for June, July and August 1999); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 6, 11 (metric MR-2-02 for September 1999). For specials, although the rate of trouble reported was higher for for competing carriers' resale customers than for Bell Atlantic each month, we do not consider the disparities indicative that Bell Atlantic overall is providing competing carriers with access to resale services at a level of quality inferior to its own. See Bell Atlantic Dowell/Canny Decl. Attach. D at 77, 82, 89, 94, 101, 106 (metric MR-2-03 for June, July, and August 1999); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 6, 11 (metric MR-2-03 for September 1999). With respect to the rate for central office trouble reported, the June rate for competing carriers (0.19 percent) exceeded Bell Atlantic's retail rate (0.16 percent) only slightly, followed by similar performance in July, August and September. Bell Atlantic Dowell/Canny Decl. Attach. D at 82, 94, 106 (metric MR-2-03 for June, July, and August 1999); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 11 (metric MR-2-03 for September 1999). We do not find these disparities dispositive of inferior quality of access provided by Bell Atlantic. See Bell Atlantic Dowell/Canny Decl. Attach. D at 77, 82, 89, 94, 101, 106 (metric MR-5-01 for June, July, and August 1999 for POTS and Special Services); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 6, 11 (metric MR-5-01 for September 1999 for POTS and Special Services). KPMG Final Report at M&R5 V-76-77. See New York Commission Comments at 52; TRA Comments at 11 n.37 (noting KPMG's findings). See AT&T Crafton/Connolly Aff. at para. 177; Prism Comments at 13-14; Covad Conley/Poulicakos Decl. at paras. 86-87 (contending that Bell Atlantic's technicians often improperly close trouble tickets). See New York Commission Comments at 52 (noting Bell Atlantic's claim that it also took longer to clear trouble tickets when its own technicians were dispatched in error). Second BellSouth Louisiana Order, 13 FCC Rcd at 20698. Id. Bell Atlantic Dowell/Canny Decl. at para. 102. Bell Atlantic Dowell/Canny Decl. at para. 102. Bell Atlantic Application at 46; Bell Atlantic Miller/Jordan Decl. at paras. 80-81. See New York Commission Comments at 53-54. Specifically, the standard adopted by the New York Commission for the Carrier-to-Carrier metrics requires that Bell Atlantic transmit 95 percent of its DUFs for resale and UNEs to competing carriers within four business days after creation and send 98 percent of its carrier bills to competing carriers within ten business days of the bill date. Bell Atlantic Dowell/Canny Decl. Attach. B at 66, 70 (Carrier-to-Carrier Guidelines listing performance standards); NYPSC Guidelines Order App. 2 at 5 (describing the development of billing performance standards). Bell Atlantic Dowell/Canny Decl. Attach. D at 85, 97; Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 2 (listing Bell Atlantic performance for metric BI-1-02 in July, August, and September 1999 as 98.78, 99.60, and 99.59 percent, respectively; listing Bell Atlantic performance for metric BI-2-01 in July, August, and September 1999 performance as 99.84, 99.54, and 98.71 percent, respectively). The New York Commission has yet to adopt a standard for billing accuracy. Bell Atlantic Dowell/Canny Decl. Attach. B at 71; New York Commission Comments at 54. Nonetheless, we note that Bell Atlantic's billing accuracy performance, measured as the dollars adjusted for billing errors out of the total dollars billed, is comparable with Bell Atlantic retail in recent months. Bell Atlantic Dowell/Canny Decl. Attach. D at 85,97; Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 2 (listing Bell Atlantic retail/competing carrier performance for metric BI-3-01 in July, August, and September 1999 as 98.67/96.66, 98.17/98.33, and 98.23/99.14 percent, respectively); see also Bell Atlantic Dowell/Canny Decl. Attach. B at 71 (describing the measurement of metric BI-3-01). New York Commission Comments at 53-54 (noting that 81 percent of 287 test points were satisfied and 19 percent were satisfied after exceptions were resolved). See generally KPMG Final Report at BLG IV-1-126. KPMG Final Report at Executive Summary II-10. AT&T Comments at 27; AT&T Crafton/Connolly Aff. at paras. 178-187; AT&T Crafton/Connolly Reply Aff. at paras. 100-102; AT&T Dec. 15 Ex Parte Letter at 58-61. See generally Bell Atlantic Nov. 24 Ex Parte Letter at 3-4 (refuting AT&T allegations regarding usage for originating toll free calls, provision of classification codes for UNE records, and provision of billing records for operator-assisted, collect, third-party, and directory assistance calls). CCA Comments at 6-7 (arguing that reseller accounts should be moved to Bell Atlantic's wholesale billing systems); Z-Tel Comments at 22 (arguing that a "read-only" CD-ROM format is inadequate). Second BellSouth Louisiana Order, 13 FCC Rcd at 20723. NALA Comments at 4; TRA Comments at 15-16 (alleging problems with service orders that are provisioned but not accounted for in Bell Atlantic's filing system, resulting in double billing of customers by Bell Atlantic and competing carriers); Adelphia Livengood Decl. at para. 18. See Ameritech Michigan Order, 12 FCC Rcd at 20651. Z-Tel Comments at 22. 47 U.S.C.  271(c)(1)(B)(ii). 47 U.S.C.  251(c)(3). Id. Ameritech Michigan Order, 12 FCC Rcd 20543, 20718-19; BellSouth South Carolina Order, 13 FCC Rcd at 646. BellSouth South Carolina Order, 13 FCC Rcd at 646; see also Local Competition First Report and Order, 11 FCC Rcd at 15666-68. See Lacouture/Troy Decl. at paras. 117-25. Pursuant to NY P.S.C. 914 Tariff, Bell Atlantic offers standard physical and virtual collocation arrangements as well as a variety of alternative collocation arrangements that competing carriers can use to combine individual network elements. Pursuant to NY P.S.C. 916 Tariff, Bell Atlantic provides access to preassembled combinations of network elements. Through August 1999, Bell Atlantic had provided over 152,000 network element platforms in service. Bell Atlantic Lacouture/Troy Decl. at para. 122. KPMG has verified that Bell Atlantic can process more than 570,130 platform orders a year. Id. (citing KPMG Final report at Appendix C (App. C, Tab 916)). See Bell Atlantic Lacouture/Troy Decl. at para. 115 (stating that "the New York Public Service Commission has agreed that [Bell Atlantic] is providing [competing carriers] with `every technically feasible method available today for competitive LECs to access network elements combinations to provide service."). Bell Atlantic Lacouture/Troy Decl. at para. 118; NY P.S.C. 914 Tariff. Bell Atlantic Application at 26; Lacouture/Troy Decl. at para. 118 (citing NY P.S.C. 914 Tariff). Bell Atlantic's Assembly Rooms are rooms within Bell Atlantic's central offices where competitive carriers can combine loops and switching ports, and Assembly Points are cabinets adjacent to Bell Atlantic's central offices where competitive carriers can combine loops and switching ports. Id. Bell Atlantic Application at 24; Lacouture/Troy Decl. at paras. 122-24. Id. at 125. Id. at paras. 115, 122, 125. TRA Comments at 21; ALTS Comments at 11. See, e.g., ALTS Comments at 49-64; DSL.net Comments. at 7-8. See discussion of checklist item 1 above. See, e.g., Sprint Comments at 16-17; TRA Comments at 19; AT&T Comments at 49-50; AT&T Reply at 44; CompTel Dec. 10 Ex Parte Letter. Bell Atlantic does not provide the full loop-switch platform for business services in New York City wire center in which there are two or more competing carriers already collocated and tariffed to provide local service. See Pre-Filing Statement of Bell Atlantic New York at 9, Case 97-C-0271 (PSC filed Apr. 6, 1998). Bell Atlantic's residence and business platform offerings have duration periods of either 4 or 6 years, depending on whether the central office is located in Zone 1 or Zone 2. See Pre-Filing Statement of Bell Atlantic New York at 9-10, Case 97-C-0271 (PSC filed Apr. 6, 1998). See, e.g., AT&T Comments at 50-51; TRA Comments at 19-20; RCN Comments at 6-8. The Supreme Court also reinstated the Commission's Rule 51.315 (b) (prohibiting incumbents from separating preassembled combinations of network elements) which, along with rules 51315(c)-(f) (requiring incumbents' to combine non-preassembled combinations of elements for requesting carriers), had been overturned by the Eighth Circuit. AT&T Corp v. Iowa Utilities Bd., 119 S.Ct. 721 (1999). In light of the reasoning set forth in that decision, the Commission has asked the Eighth Circuit to reinstate rules 51.315(c)-(f). That matter is still pending. See Implementation of the Local Competition Provisions of the Telecommunications Act of 1996, CC Docket No. 96-98, Third Report and Order and Fourth Further Notice of Proposed Rulemaking, FCC 99-238 (rel. Nov. 5, 1999) (Third Report and Order and Fourth FNPRM). In the Fourth FNPRM, we stated that it is not clear that the 1996 Act permits any restrictions to be placed on the use of unbundled network elements. We concluded, however, that under existing law, a requesting carrier is entitled to obtain existing combinations of loop and transport between the end user and the incumbent LEC's serving wire center on an unrestricted basis at unbundled network element prices. Third Report and Order and Fourth FNPRM at para. 484. In a Supplemental Order, we modified those conclusions with respect to the use of unbundled network elements to provide exchange access services. Implementation of the Local Competition Provisions of the Telecommunications Act of 1996, CC Docket No. 96-98 (adopted Nov. 24, 1999) (Supplemental Order). Specifically, we stated that in order to preserve the issue in the Fourth FNPRM as we intended, we would "allow incumbent LECs to constrain the use of combinations of unbundled loops and transport network elements as a substitute for special access service subject to the requirements of [the Supplemental Order"]. Id. at para 2. We also concluded that this constraint does not apply if an interexchange carrier uses combinations of unbundled loop and transport network elements to provide a significant amount of local exchange service, in addition to exchange access service, to a particular customer. Id. at para. 5. We note that Bell Atlantic states that it will comply with the Commission's unbundling rules once they become effective. Bell Atlantic Application at 25. 47 U.S.C.  271(B)(ii). 47 U.S.C.  251(c)(3). 47 U.S.C.  252(d)(1). Local Competition First Report and Order, 11 FCC Rcd at 15845. Id. at 15844-46. See Bell Atlantic Application at 66; NYPSC Collocation Order at 7; NYPSC Interconnection Tariff at 5.1.17(A)(B) and 10.5.1(A)(B); NYPSC Tariff No. 916 (Bell Atlantic Application App. H, Tab 3) (NYPSC UNE Tariff) at 5.12.9.5; Opinion and Order Concerning Methods for Network Element Recombination, Case Nos. 98-C- 0690 and 95-C-0657 (NYPSC Nov. 23, 1998 (Bell Atlantic Application App. D, Vol. 6, Tab 121) (NYPSC UNE Recombination Order); Opinion and Order Setting Rates for First Group of Network Elements, Case Nos. 95-C- 0657, 94-C-0095, 91-C-1174 (NYPSC April 1, 1997) (Bell Atlantc Application App. G, Vol. 1, Tab 9) (NYPSC Phase I Order); New York Commission Comments at 152-62; New York Commission Reply at 49-50. Bell Atlantic Application at 65-66. New York Commission Comments at 162; see also New York Commission Reply at 42. NYPSC Phase I Order at 4. Id. at 14. Id. Id. at 13. In the New York Commission rate case, Bell Atlantic filed under the name of "New York Telephone d/b/a/ Bell Atlantic-New York." See, e.g., NYPSC Phase 3 Order at 1. NYPSC Phase 1 Order at 14. Id. at 48-64. Id. at 99. Id. at 120. We note that Phase Four of the New York Commission's network elements rate case has not been completed, and several important network element issues remain outstanding. New York Commission Comments at 154-55. AT&T Comments at 54. Bell Atlantic Application at 66. See, e.g., NYPSC Phase I Order; NYPSC Phase 3 Order; Bell Atlantic Pre-Filing Statement; Bell Atlantic- New York Joint Affidavit in Support of Proposed Rates for ADSL-Qualified, HDSL-Qualified, and Digital- Designed Links, Case 98-C-1357 (NYPSC Sept. 13, 1999) (Rhythms Comments, Attach. EHG-RW-3) (Bell Atlantic Affidavit in Support of DSL Links); NYPSC Collocation Order. NYPSC Phase I Order at 84. AT&T Comments at 60. NYPSC Phase I Order at 83-84. Id. at 83-84. Id. at 84. Id. at 84-85; see also Order Denying Motion to Reopen Phase I and Instituting New Proceeding (NYPSC Sept. 30, 1998) (Bell Atlantic Application App. G, Vol. 1, Tab 18) (NYPSC Order Denying Motion to Reopen Phase I). New York Commission Reply at 47-48. Id. at 48. AT&T Comments at 60. Letter from Mark C. Rosenblum, Vice President-Law, AT&T, to Magalie Roman Salas, Secretary, Federal Communications Commission (filed Nov. 23, 1999) (AT&T Nov. 23 Ex Parte Letter) at 6. AT&T Comments at 62-63. Local Competition First Report and Order, 11 FCC Rcd at 15812. Id. at 15559. New York Commission Reply at 46. See id. at 48. See AT&T Comments at 60. In re Federal-State Joint Board on Universal Service, CC Docket No. 96-45, Tenth Report and Order, FCC 99-304 (rel. Nov. 2, 1999) (Universal Service Tenth Report and Order) at para. 32. Id. New York Commission Reply at 48. AT&T Comments at 61; see also AT&T Nov. 23 Ex Parte Letter at 4-5. New York Commission Reply at 47-48. NYPSC Phase I Order at 85, n.1; see also New York Commission Reply at 47-48; NYPSC Order Denying Motion to Reopen Phase I. New York Commission Reply at 47. AT&T Comments at 58-60. New York Commission Reply at 45-46. NYPSC Phase I Order at 70. Id. Id. AT&T Nov. 23 Ex Parte Letter at 4-5. NYPSC Phase I Order at 83-84. Id. New York Commission Reply at 45-46. Id. at 46 and n.4. AT&T Clarke/Petzinger Aff. at paras. 5-24. Id. at paras. 5-24; see also AT&T Nov. 23 Ex Parte Letter at 4. NYPSC Phase I Order at 71-72. Id. at 83-84. Id. AT&T Nov. 23 Ex Parte Letter at 5. New York Commission Reply at 46 n.2. We note, however, that in other states it may be acceptable, and even preferable, to assume the use of copper in certain parts of a LEC's network. Bell Atlantic Affidavit in Support of DSL Links at 4. A small "x" before the letters "DSL" signifies the use of the term as a generic transmission technology. See infra Section V.D. Bell Atlantic-New York's Joint Affidavit in Support of Proposed Rates for ADSL-Qualified, HDSL- Qualified, and Digital-Designed Links, Case 98-C-1357 (Sept. 13, 1999) at 4-5. Id. at 6. Id. at 6. Id. Id. Id. at 8. Id. at 8-9. Id. at 6. Bell Atlantic states that it would be willing to recover these charges through a non-recurring, loop based charge. Id. Id. at 9-10. In addition to a manual loop qualification charge, Bell Atlantic may impose an engineering query charge, an engineering work order charge, and a pair swap charge. Id. at 10-13. A load coil is an inductor that is connected into a loop in order to improve its voice transmission characteristics. Id. at 14. Bridge taps are a branching of a copper loop that permit the appearance of the loop at a number of alternative servicing terminal locations, which give the telephone company greater flexibility in reassigning a telephone number to a different address without rearranging existing facilities. Id. at 14-16. Id. at 14. Id. at 16. Additional charges may accrue when a competitive provider orders a two-wire digital link that is longer than 18,000 feet. Id. NYPSC UNE Tariff at 5.5.1.1(D)(2)(b). Id. Bell Atlantic-New York's Joint Affidavit in Support of Proposed Rates for ADSL-Qualified, HDSL- Qualified, and Digital-Designed Links, Case 98-C-1357 (Sept. 13, 1999) at 16. Id. Id. at 17. Local Competition First Report and Order, 11 FCC Rcd at 15692. Id. Id. 47 C.F.R.  51.509(e). ALTS Comments at 36-37; CoreComm Comments at 6; Covad Comments at 6; Intermedia Comments at 8; MCI WorldCom Comments at 21. ALTS Comments at 36-37. ALTS Comments at 36-37; CoreComm Comments at 6; Covad Comments at 6; Intermedia Comments at 8; MCI WorldCom Comments at 21. ALTS Comments at 36-37. Id. at 36. Bell Atlantic Reply at 53-55. New York Commission Comments at 79-80. Id. Id. New York Commission Reply at 49. Bell Atlantic Lacouture/Troy Reply Decl. at para. 195. New York Commission Reply at 49. Local Competition First Report and Order, 11 FCC Rcd at 15812. We note that the New York Commission has committed to review Bell Atlantic's cost studies in support of its DSL prices and to conform such prices to TELRIC before the end of 1999. New York Commission Reply at 49-50. We note that New York Commission is taking reasonable steps to complete its permanent rate-setting proceeding within a short time-frame, and the New York Commission and Bell Atlantic have both committed to the use of forward-looking economic costs for determining unbundled network elements rates. NYPSC Collocation Order at 7; Bell Atlantic Reply at 55. Local Competition First Report and Order, 11 FCC Rcd at 15692. Cable & Wireless Comments at 6. Bell Atlantic Pre-Filing Statement at 1. NYPSC UNE Tariff at 5.12.8.5. Bell Atlantic Pre-Filing Statement, Attach. Letter from John F. O'Mara, Chairman, New York State Public Service Commission, to Hon. Maureen O. Helmer, Deputy Chairman, New York State Public Service Commission (filed April 6, 1998) at 4. NYPSC UNE Tariff at 5.12.9.5. 47 U.S.C.  271(c)(2)(B)(iii). As originally enacted, section 224 was intended to address obstacles that cable operators encountered in obtaining access to poles, ducts, conduits, or rights-of-way owned or controlled by utilities. The 1996 Act amended section 224 in several important respects to ensure that telecommunications carriers as well as cable operators have access to poles, ducts, conduits, or rights-of-way owned or controlled by utility companies, including LECs. Second BellSouth Louisiana Order, 13 FCC Rcd at 20706, n.574. Local Competition First Report and Order, 11 FCC Rcd at 16073. Implementation of Section 703(e) of the Telecommunications Act of 1996, Amendment of the Commission's Rules and Policies Governing Pole Attachments, CS Docket No. 97-151, 13 FCC Rcd 6777 (1998) (Pole Attachment Telecommunications Rate Order). 47 U.S.C.  224(f)(1). Section 224(a)(1) defines "utility" to include any entity, including a LEC, that controls, "poles, ducts, conduits, or rights-of-way used, in whole or in part, for any wire communications." 47 U.S.C.  224(a)(1). 47 U.S.C.  224(f)(2). In the Local Competition First Report and Order, the Commission concluded that, although the statutory exception enunciated in section 224(f)(2) appears to be limited to utilities providing electrical service, LECs should also be permitted to deny access to their poles, ducts, conduits, and rights-of-way, because of insufficient capacity and for reasons of safety, reliability and generally applicable engineering purposes, provided the assessment of such factors is done in a nondiscriminatory manner. Local Competition First Report and Order, 11 FCC Rcd at 16080-81. Section 224(a)(4) defines "pole attachment" as "any attachment by a cable television system or provider of telecommunications service to a pole, duct, conduit, or right-of-way owned or controlled by a utility." 47 U.S.C.  224(a)(4). 47 U.S.C.  224(b)(1). See States That Have Certified That They Regulate Pole Attachments, Public Notice, 7 FCC Rcd 1498 (1992). The 1996 Act extended the Commission's authority to include not just rates, terms, and conditions, but also the authority to regulate nondiscriminatory access to poles, ducts, conduits, and rights-of-way. Local Competition First Report and Order, 11 FCC Rcd at 16104; 47 U.S.C.  224(f). Absent state regulation of terms and conditions of nondiscriminatory attachment access, the Commission retains jurisdiction. Local Competition First Report and Order, 11 FCC Rcd at 16104; 47 U.S.C.  224(c)(1). Bell Atlantic Application at 26-27; Bell Atlantic Lacouture/Troy Decl. at paras. 128-139. New York Commission Comments at 70-75. See also Intermedia Comments at 6 (stating that in Intermedia's experience, Bell Atlantic has complied with the requirements of this checklist item). ALTS Comments at 48-49. RCN raises similar issues regarding house and riser cables under checklist items 2 and 4. RCN Comments at 3-5. Bell Atlantic Lacouture/Troy Reply Decl. at para.144. Letter from Patrick J. Donovan, Swidler Berlin Shereff Friedman, LLP, Counsel for RCN, to Magalie Roman Salas, Secretary, Federal Communications Commission, CC Docket No. 99-295 (Filed November 3, 1999) (RCN Ex Parte Letter). RCN claims that access to conduits and ducts requires 90 to 120 days and these delays are especially burdensome to competitive LECs with more limited infrastructure than Bell Atlantic. See also RCN Reply at 4-5. 47 U.S.C.  271(c)(2)(B)(iv). Local Competition First Report and Order, 11 FCC Rcd at 15691. Id. Second BellSouth Louisiana Order, 13 FCC Rcd at 20637. Id. at 20712-13. Ameritech Michigan Order, 12 FCC Rcd at 10619. Second BellSouth Louisiana Order, 13 FCC Rcd at 20713. Id.; Ameritech Michigan Order, 12 FCC Rcd at 20614. Second BellSouth Louisiana Order, 13 FCC Rcd at 20713; Local Competition First Report and Order, 11 FCC Rcd at 15691. IDLC technology permits a carrier to aggregate and multiplex loop traffic at a remote concentration point and to deliver that multiplexed traffic directly into the switch without first demultiplexing the individual loops. Local Competition First Report and Order, 11 FCC Rcd at 15692. Local Competition First Report and Order, 11 FCC Rcd at 15692-93. Second BellSouth Louisiana Order, 13 FCC Rcd at 20713. Id. at 15693. The network interface device is a cross-connect device used to connect the loop facilities to inside wiring. See id. See supra Section II.A. See Bell Atlantic Lacouture/Troy Decl. at para. 64. Id. Id. See id. at para. 66. See id. at para. 66; Bell Atlantic Lacouture/Troy Reply Decl. at para. 34. Bell Atlantic Lacouture/Troy Decl. at para. 66; Bell Atlantic Lacouture/Troy Repy Decl. at para. 34. Bell Atlantic Lacouture/Troy Decl. at para. 67. Id. at para. 78; Bell Atlantic Lacouture/Troy Reply Decl. at para. 73. Bell Atlantic Lacouture/Troy Decl. at para. 81 & Attach. K; Bell Atlantic Lacouture/Troy Reply Decl. at para. 73. Ameritech Michigan Order, 12 FCC Rcd at 20619. In particular, Bell Atlantic provides data regarding its performance in provisioning second lines and other new loops to its retail customers to its retail customers. Second BellSouth Louisiana Order, 13 FCC Rcd at 20655; Local Competition First Report and Order, 11 FCC Rcd at 15763-64. Ameritech Michigan Order, 12 FCC Rcd at 20619. Bell Atlantic Dowell/Canny Decl. at para. 59. Id. at para. 63; see supra Section V.B.1.g. Bell Atlantic Lacouture/Troy Decl. at para. 76. In June, Bell Atlantic missed 1.96 percent of installation appointments for competing carriers and 9.02 percent of appointments for Bell Atlantic customers. Bell Atlantic Dowell/Canny Decl. Attach. D at 90 (metric PR- 4-04 - Loop New for June 1999). In July, Bell Atlantic missed .33 percent of dispatched new loop installations for competing carriers and in August, .12 percent. Bell Atlantic Dowell/Canny Decl. Attach., D at 92, 104 (metric PR-4-04 - Loop New for July and August 1999). Similarly, Bell Atlantic missed .41 percent of loop installation appointments for competing carriers in September. Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 9 (metric PR-4-04 - Loop New for September 1999). Bell Atlantic missed 10.69 percent of retail loop installation appointments in July and 9.41 percent of appointments in August. Bell Atlantic Dowell/Canny Decl. Attach. D at 92, 104 (metric PR-4-04 - Loop New for July and August 1999). Finally, Bell Atlantic missed 12.14 percent of retail loop installation appointments in September. Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 9 (metric PR-4-04 - Loop New for September 1999). Installation quality performance data measure both new, stand-alone loops and loops provisioned as part of a platform. Accordingly, the only types of loops provisioned by Bell Atlantic that are not included in these reports are those provisioned as hot cuts. See Bell Atlantic Dowell/Canny Reply Decl. Attach. B at 47. In June, competing carriers reported troubles within 7 days for 1.28 percent of the loops installed by Bell Atlantic, and retail customers reported troubles with 2.85 percent of installed loops. Bell Atlantic Dowell/Canny Decl. Attach. D at 80 (metric PR-6-02 - Loop for June 1999). July data indicate that 1.65 percent of loops installed for competing carriers received trouble reports, and 2.90 percent of Bell Atlantic retail loops had reported troubles. Id. at 92 (metric PR-6-02 - Loop for July 1999). In August, competing carriers reported troubles within 7 days for 1.57 percent of the loops installed by Bell Atlantic, and retail customers reported troubles with 2.92 percent of installed loops. Id. at 104 (metric PR-6-02 - Loop for August 1999). In September, 1.06 percent of loops provisioned to competitors had troubles reported within 7 days of installation, while 3.15 percent of Bell Atlantic retail customers reported loop troubles within 7 days. Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 9 (metric PR-6-02 - Loop for September 1999). In June, competing carriers reported troubles within 30 days for 3.31 percent of the loops installed by Bell Atlantic, and retail customers reported troubles with 4.85 percent of installed loops. Bell Atlantic Dowell/Canny Decl. Attach. D at 80 (metric PR-6-01 - Loop for June 1999). July data indicate that 4.05 percent of loops installed for competing carriers received trouble reports within 30 days and 5.22 percent of Bell Atlantic retail loops had reported troubles. Id. at 92 (metric PR-6-01 - Loop for July 1999). In August, competing carriers reported troubles within 30 days for 3.50 percent of the loops installed by Bell Atlantic, and retail customers reported troubles with 5.02 percent of installed loops. Id. at 104 (metric PR-6-01 - Loop for August 1999). In September, 2.65 percent of loops provisioned to competitors had troubles reported within 30 days of installation, while 5.74 percent of Bell Atlantic retail customers reported loop troubles within 30 days. Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 9 (metric PR-6-01 - Loop for September 1999). We note that Prism alleges that Bell Atlantic often fails to provision functioning unbundled loops. Prism Comments at 9-11. Although we have considered these claims, Prism has not asserted that any installation problems it has experienced are not reflected or captured in the relevant performance measures. Moreover, we find Prism's general allegations to be insufficient to overcome the substantial evidence in the record of the quality of new, unbundled loops provisioned by Bell Atlantic. As discussed in our analysis of checklist item 2, we also rely heavily upon KPMG's comprehensive evaluation of Bell Atlantic's provisioning systems for both wholesale and resale services. KPMG examined the degree to which Bell Atlantic's provisioning environment for wholesale orders is "on parity" with provisioning for Bell Atlantic retail customers and concluded that Bell Atlantic had satisfied each of its testing criteria. See generally KPMG Final Report at POP11 IV-258-84 (Provisioning Parity Process Evaluation). See also supra Section V.B.1.g. Bell Atlantic's data measure the "average completed interval," which is the average number of business days between the order application date and the work completion date. Bell Atlantic Dowell/Canny Decl. Attach. B at 35. For purposes of this discussion, we use the terms "average completed interval" and "average installation interval" interchangeably. With respect to customers of competing carriers, the average competed interval in June 1999 for loops with one to five lines was 6.55 days, while the average completion interval for Bell Atlantic retail customers was 3.27 days. Bell Atlantic Dowell/Canny Decl. Attach. D at 80 (metric PR-2-03 - Loop for June 1999). In July 1999, the average installation interval for loop orders of one to five lines was 5.39 days for competing carriers and 3.08 days for Bell Atlantic customers. Id. at 92 (metric PR-2-03 - Loop for July 1999). In September 1999, the average installation interval for customers of competing carriers for loop orders of one to five lines was 5.88 days, and the Bell Atlantic retail average interval was 3.52 days. Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 9 (metric PR-2-03 - Loop for September 1999). The data further reveal similar trends for loop orders involving more than five lines, although the number of such loops ordered by competitors has consistently been very small. See id. (metric PR-2-04 and 2-05 - Loop for June, July, August, and September 1999). Bell Atlantic Dowell/Canny Decl. Attach. B at 39. See Bell Atlantic Application at 17; Bell Atlantic Bamberger/Gertner Decl. See Bell Atlantic Bamberger/Gertner Decl. at para. 12. AT&T Pfau/Kalb Aff. at para. 143. We note, however, that AT&T states that it does so because it lacks confidence in Bell Atlantic's ability to complete orders on-time. Id. We note that Prism alleges a low rate of successful loop installations performed by Bell Atlantic, although it does not dispute directly Bell Atlantic's data. See Prism Comments at 10-11. Although we take seriously Prism's claims, we nonetheless find them to be insufficient to overcome the record evidence that Bell Atlantic provisions quality unbundled loops in a nondiscriminatory manner. In addition, although it mentions the disparity between Bell Atlantic's loop provisioning intervals, the Department of Justice does not address the provisioning of new unbundled loops in its evaluation. See Department of Justice Evaluation at 19 n.42. Bell Atlantic Dowell/Canny Reply Decl. at para. 53. Bell Atlantic also contends that, generally, average provisioning intervals are longer for competing carriers because those carriers order proportionately more products with longer standard provisioning intervals than Bell Atlantic customers. Bell Atlantic Bamberger/Gertner Decl. at para. 12. We note first that Bell Atlantic makes no specific reference to this claim with respect to loop orders, and we are therefore unable to determine if such a claim would be applicable to those orders. We are unpersuaded, however, that this "order mix" argument is applicable to stand-alone new loop orders because the feature mixes that Bell Atlantic alleges result in longer provisioning intervals do not come into play when Bell Atlantic provisions a stand-alone loop. A competing carrier, for instance, would not order any feature such as Caller ID or Call Answering from Bell Atlantic when it provides service over an unbundled loop that is cross-connected to its own switch, as is the case with a stand-alone loop, for such features are provided through the competitive carrier's switch and not the loop. See Bell Atlantic Dowell/Canny Decl. at para. 64; Local Competition First Report and Order, 11 FCC Rcd at 15706. AT&T Comments, Exhibit K, para. 134. See Prism Comments at 7-10. In the Ameritech Michigan Order, for example, the Commission stated that the BOC "is free to use data on due dates not met to explain any inconsistencies between the average installation intervals for itself and other carriers. For example, if a particular competing carrier consistently requests a standard, longer interval for completion of all of its orders, rather than the first available installation date, such data may explain that any differences in the average installation intervals between [the BOC] and the other carrier are not due to discriminatory conduct on the part of [the BOC]." Ameritech Michigan Order, 12 FCC Rcd at 20633. The average completed interval for competing carriers in July was 15.00 days, and the interval for Bell Atlantic customers was 11.34 days. See Bell Atlantic Dowell/Canny Decl. Attach. D at 93 (metric PR-2-07 - DS1 for July 1999). For August, DS1 loops were provisioned to competing carriers in, on average, 24.13 days and to Bell Atlantic customers in 8.07 days. Id. at 105 (metric PR-2-07 - DS1 for August 1999). In June, Bell Atlantic missed 2.94 percent of installation appointments for high capacity services delivered to competing carriers and 3.71 percent of appointments for its retail customers. Bell Atlantic Dowell/Canny Decl. Attach. D at 81 (metric PR-4-01 - Total for June 1999). In July, Bell Atlantic missed 22.22 percent of installation appointments for high capacity loops delivered to competing carriers and 5.44 percent of appointments for its retail customers. Id. at 93 (metric PR-4-01 - Total for July 1999). In August, however, Bell Atlantic's performance towards competitors improved substantially, and it missed 15.79 percent of appointments for competing carriers and 18.03 percent of installations for its own customers. Id. at 105 (metric PR-4-01 - Total for August 1999). In September, Bell Atlantic missed only 4 percent of installation appointments for high capacity loops provided to competing carriers and 18.58 percent of appointments for installations to its retail customers. Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 10 (metric PR-4-01 - Total for September 1999). Allegiance indicates that 46 percent of the DS1 loops it ordered from Bell Atlantic were delivered after the confirmed due date. Allegiance Comments at 12. See also Omnipoint Comments at 10; Focal Comments at 5-6. See Ameritech Michigan Order, 12 FCC Rcd at 20569 ("[W]e will attach greater weight to comments and pleadings supported by a sworn statement than we will to an unsupported contrary pleading."). A hot cut entails manually disconnecting the customer's loop in the Bell Atlantic central office and reconnecting the loop at the competing carrier's collocation space. It also involves coordinated switch software changes at both Bell Atlantic's switch and the competing carrier's switch and the implementation of local number portability. The customer is taken out of service while the hot cut is in progress, thereby making the cut "hot," although if the cut is successful, the service disruption will last no more than five minutes. Bell Atlantic Lacouture/Troy Decl. at para. 69. Ensuring that a hot cut is provisioned correctly with coordination between Bell Atlantic and the competing carrier is therefore critical because problems with the cutover could result in an extended service disruption for the customer. Second BellSouth Louisiana Order, 13 FCC Rcd at 20714. Bell Atlantic Dowell/Canny Decl. at para. 73. Id. Bell Atlantic Application, App. C, Order Establishing Final Rule, C2C Record, Tab 83. New York Commission Comments at 82. Id. Bell Atlantic Dowell/Canny Decl. Attach. D at 80, 92, 103 (metric PR-4-06 - Hot Cut for July and August 1999). Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 9 (metric PR-4-06 - Hot Cut for September 1999). See New York Commission Comments at 83; Bell Atlantic Application, App. C, Vol. 61, Tab 941. In the face of these and other challenges to its data, Bell Atlantic was forced to withdraw all of the hot cut data it had submitted prior to June 18, 1999. See Bell Atlantic Application, App. C, Vol. 51, Tab 789 (Letter from Randal Milch, Associate General Counsel, Bell Atlantic-State Regulatory North, to Andrew Klein, Assistant Counsel, New York Public Service Commission, June 18, 1999). New York Commission Comments at 85. A portion of AT&T's Motion to Strike is directed to the New York Commission's submission with its reply comments of the results of its data reconciliation for August 1999. See AT&T Motion to Strike at 4. Specifically, AT&T argues that our rules prohibit us from relying on the material submitted by the New York Commission because it post-dates this application's comment period. The New York Commission's hot cut reconciliation, however, responds directly to arguments made in the comments filed by AT&T regarding Bell Atlantic's on-time hot cut performance. Additionally, the reconciliation addresses data for the month of August 1999, which is prior to the filing of Bell Atlantic's application. Accordingly, as discussed supra Section III, we deny AT&T's Motion to Strike with respect to the New York Commission's August hot cut reconciliation. See New York Commission Rubino Reply Aff. at para. 6. AT&T Meek Aff. at para. 118. New York Commission Rubino Reply Aff. at paras. 9-11. Id. at para. 10. New York Commission Rubino Reply Aff. at paras. 9-10. The staff's reconciliation is ongoing, although its conclusions regarding September performance are not yet complete. Department of Justice Evaluation at 18-19 & n.41. See AT&T Comments at 39; AT&T Meek Aff. at paras. 132-35. Department of Justice Evaluation at 18-19 & n.41. See AT&T Comments at 39; AT&T Meek Aff. at paras. 132-35. See Allegiance Comments at 11 (hot cut process caused hot cut failures attributable to Bell Atlantic to drop from more than 70 percent to 20 percent in recent months); ChoiceOne Comments at 4 (Bell Atlantic failed to provision properly 21 of 43 loop orders). In addition, Bell Atlantic on reply addresses the specific allegations made by Allegiance and ChoiceOne regarding its hot cut performance. Specifically, Bell Atlantic states that between June 21 and August 31, 1999, it completed 91.3 percent of Allegiance's hot cut orders within the prescribed window and 95.40 percent of ChoiceOne's orders within the prescribed window. Bell Atlantic Lacouture/Troy Reply Decl. at paras. 41, 42. See Ameritech Michigan Order, 12 FCC Rcd at 20569 ("[W]e will attach greater weight to comments and pleadings supported by an affidavit or sworn statement than we will to an unsupported contrary pleading."). We also find that this level of on-time performance would not be significantly affected if it were to capture hot cuts that are delayed as a result of Bell Atlantic provisioning deficiencies, as commenters argue it should. See Department of Justice Evaluation at 19 (citing AT&T Meek Aff. at paras. 127-30). Under the on-time performance standard, a hot cut that is not completed at the initially-scheduled time, but rather is completed in a subsequently- rescheduled time, is considered "on-time," even where a Bell Atlantic provisioning error causes the rescheduling. See id. at 19. The Department of Justice determined that this aspect of the metric causes the on-time performance measure to "overstate" Bell Atlantic's hot cut performance. Id. at 19. KPMG, however, found that the majority of rescheduled hot cuts are attributable to competing carriers, and Bell Atlantic argues that it causes only 11 percent of delayed or postponed hot cuts. Bell Atlantic Application at 19 & Lacouture/Troy Decl. at para. 73. Commenters allege that the percentage of hot cut delays attributable to Bell Atlantic is much higher. See Allegiance Comments at 11; AT&T Comments at 38. AT&T argues that KPMG acknowledged in the New York proceeding that 40 percent of supplements were attributable to Bell Atlantic. AT&T Meek Aff. at para. 102 (citing New York Technical Conference Transcript at 3936-37). As Bell Atlantic responds, however, this statement predated the final report, which represents KPMG's comprehensive analysis of Bell Atlantic's performance in New York. We therefore rely upon the KPMG final report, which found that approximately 11 percent of postponed orders were attributable to Bell Atlantic, and conclude that the failure of the on-time performance measure to include hot cut delays attributable to Bell Atlantic does not overstate overall performance. See KPMG Final Report at POP12 IV- 294-95 (Table IV-12.6: POP12, P12-3). We find that the number of hot cut delays not included in the metric and attributable to Bell Atlantic is sufficiently small that it would not effect a change in Bell Atlantic's on-time hot cut performance. We also note that, although commenters argue that "early" cuts, i.e., those made prior to the Frame Due Time, are not reflected in the On-Time Hot Cut Performance Measure, a review of the Carrier-to-Carrier performance standards indicates that early cuts are, in fact, reported as missed hot cuts. See Bell Atlantic Dowell/Canny Decl. Attach. B at 43. See also New York Commission Reply at 27. Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 9 (metric PR-4-06 - Hot Cut for September 1999). See Bell Atlantic Dowell/Canny Decl. Attach. D at 80, 92, 104; Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 9. Moreover, even reviewing the data in a light most favorable to the opponents of the application indicates that in only one month was performance slightly below 90 percent, namely 88 percent. Department of Justice Evaluation at 18. Id. at 15-16, 19. Id. at 20. See New York Commission Reply at 28. We note that the Department of Justice recognized that deviation from a New York Commission performance standard should not be dispositive in a determination of checklist compliance. Department of Justice Evaluation at 20. Under the New York Performance Assurance Plan, the Percent On-Time Performance Measure is considered to be a "Critical Measure," requiring the payment of $787,037 for every month that Bell Atlantic fails to meet the 90 percent on-time performance standard and a smaller portion of that amount if Bell Atlantic's performance is between 90 and 95 percent for more than two consecutive months. Bell Atlantic Dowell/Canny Decl. Attach. C, App. B at 1. In addition, recent amendments to the Plan placed an additional $24 million per year at risk for poor on-time hot cut performance. Bell Atlantic Dowell/Canny Decl. Attach. C, App. H at 2. See CPI Reply at 7. Indeed, KPMG recognized during its test of hot cut provisioning that hot cut failures have the potential to affect customers detrimentally, causing service disruptions ranging from hours to days. KPMG Final Report at POP3, POPIV-60P3-33, Table IV-3.33. Installation troubles for hot cut loops are reported in terms of the number of lines, not hot cuts, that are the subject of trouble reports. See Bell Atlantic Dowell/Canny Decl. Attach. B at 47. For July 1999, Bell Atlantic reports that it received trouble reports within seven days of installation on .34 percent of the lines provisioned through hot cut loops. Bell Atlantic Dowell/Canny Decl. Attach. D at 92 (metric PR-6-02 - Hot Cut Loop for July 1999). In August, it received 1.26 percent of troubles reported within seven days. Id. at 103 (metric PR-6-02 - Hot Cut Loop for August 1999). September data reveal that .51 percent of lines provisioned by Bell Atlantic through hot cuts received trouble reports within seven days of the cutover. Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 9 (metric PR-6-02 - Hot Cut Loop for September 1999). Beginning September 1999, the New York Commission adopted a standard of 2 percent for the Percent Hot Cut Installation Troubles Reported within 7 Days Measure. See Bell Atlantic Dowell/Canny Reply Decl. Attach. D at 9 (metric PR-6-02 - Hot Cut Loop for September 1999). Allegiance Reply at 3. AT&T Meek Aff. at para. 86. Specifically, AT&T alleges that Bell Atlantic's failures caused service disruptions to 170 out of 1438 customers. Id. AT&T also contends that 61 percent of these service outages endured for more than twenty-four hours. Id. at para. 87. Although the Carrier-to-Carrier performance measures do not address them directly, service disruptions or outages can occur in two situations. First, an early cut occurs when a customer's loop is moved to a competitor's collocation space prior to the Frame Due Time and the switch translations are removed from the Bell Atlantic switch prior to such time. AT&T Meek Aff. at para. 84. In that case, the customer would lose service because the competing carrier is unaware that the customer's line is being cut and does not take the steps necessary to port the customer's telephone number. Id. Such an occurrence would be scored as a "miss" under the Percent On-Time Hot Cut Performance Measure and would also result in an outage. A second type of outage involves a defective cut, in which the customer would lose service because of a failure that occurs during the cutover. Id. at para. 85. In this case, if the cutover occurred during the one hour window, the hot cut would be scored as having been on-time, although the customer suffered a disruption of service. In this regard, AT&T raises several arguments with respect to the allegedly misleading nature of the "On- Time Hot Cut Performance" metric. See AT&T Meek Aff. at paras. 112-17. AT&T argues, for instance, that Bell Atlantic is able to manipulate its reported on-time hot cut performance data because it can score as "on-time" hot cuts that result in outages. AT&T Meek Aff. at para. 112. Although this is the case, such an occurrence would nonetheless be reflected in the "Percent Installation Troubles" metric for hot cuts. AT&T further argues that it is inappropriate to score an outage as a "trouble" and not as a "provisioning problem." Id. at para. 114. The "Percent Installation Troubles" measure, however, is a provisioning metric that measures provisioning quality and therefore appropriately captures installation troubles that are not reflected in the on-time measure. A portion of AT&T's Motion to Strike is directed to the New York Commission's submission with its reply comments of the results of its data reconciliation of AT&T's claims of outages. See AT&T Motion to Strike at 4. Specifically, AT&T argues that our rules prohibit us from relying upon the material submitted by the New York Commission because it post-dates this application's comment period. The New York Commission's reconciliation, however, responds directly to arguments made in the comments filed by AT&T regarding outages caused by Bell Atlantic's hot cut provisioning failures. Additionally, the reconciliation addresses data for the period from June 21 through August 1999, which is prior to the filing of Bell Atlantic's application. Accordingly, as discussed supra Section III, we deny AT&T's Motion to Strike with respect to the New York Commission's outage data reconciliation. New York Commission Rubino Reply Aff. at para. 13 & Ex. 5. The New York Commission staff's reconciliation demonstrates that approximately 4.5 percent of AT&T's customers suffered outages between June 21 and August 31, 1999 as a result of a Bell Atlantic provisioning error. Id. Id., Ex. 5. The New York staff's reconciliation indicates that, of the 167 alleged outages reviewed, 66 were attributable to Bell Atlantic provisioning errors, 75 were not attributable to Bell Atlantic, and 26 had causes that could not be determined. Id. In this regard, we note that the Department of Justice did not raise the issue of service disruptions in its evaluation. New York Commission Reply at 29-30 (citing NYDPS Staff Analysis of AT&T Reported Service Outages- June 21-August 31, 1999, Ex. 5). The New York staff also observed that, unlike other carriers, AT&T does not perform mechanized loop tests when it accepts a hot cut. Rather, AT&T attempts to call the customer and, in the absence of a completed call, waits until the customer calls AT&T. Id. New York Commission Reply at 29. We also note, although we do not rely upon them as a basis for our decision, that recently-adopted performance measures in New York will monitor the percentage of defective, early, and late hot cuts, as well as the duration of customer service disruptions. See NYPSC Additional Guidelines Order at 28-29. See e.g., Bell Atlantic Dowell/Canny Decl. Attach. D at 94 (metric MR-2-02 - Loop for July 1999). In July, for instance, Bell Atlantic reported loop troubles on 1.56 percent of its network. Id. We also note that in each month from June through September, the network trouble report rate for loops was higher for Bell Atlantic's network than for those of competing carriers. See Bell Atlantic Dowell/Canny Decl. Attach. D at 82, 94, 106 (metric MR-2-02 - Loop for June, July, and August 1999); Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 10 (metric MR-2-02 - Loop for September 1999). See Bell Atlantic Application at 18; New York Commission Comments at 83. For a description of the entire hot cut provisioning process, see Bell Atlantic Lacouture/Troy Decl. at para. 70. New York Commission Comments at 84. Id. at 83. See also AT&T Comments at 34; AT&T Meek Aff. at paras. 25-29. The process and tracking checklist were adopted in New York on June 21, 1999. Bell Atlantic Application, App. C, Vol. 61, Tab 941 at 17. See AT&T Comments at 34; ALTS Comments at 29-30; Choice One Comments at 5; Allegiance Comments at 11. We note, however, that Bell Atlantic has agreed that, upon a grant of interLATA relief, it will include in the on-time hot cut performance measure whether it has completed the due date minus 2 days dial tone check. See New York Commission Comments at 88. See AT&T Comments at 34; AT&T Meek Aff. at paras. 49, 51, 90-91; Choice One Comments at 4. AT&T Comments at 35-36; AT&T Meek Aff. at paras. 35-40, 95-98. As discussed in our analysis of checklist item 2, we find that AT&T's claims of LSRC inaccuracy are largely overstated. See supra Section V.B.1.f.(ii).(a). AT&T Comments at 35; AT&T Meek Aff at paras. 46-52. AT&T Comments at 37; AT&T Mulligan Aff. at para. 38. Bell Atlantic Application at 19; Bell Atlantic Lacouture/Troy Decl. at para. 73. KPMG Final Report at POP3, IV-60-62 (Test Cross Reference P3-22). KPMG opened an "Exception" regarding compliance with the hot cut procedures. See New York Commission Comments at 89. The Exception was closed following a two week test during which KPMG observed technicians performing the due date hot cut procedures. KPMG Final Report at POP3, IV-60-62 (Test Cross Reference P3-24). See also Bell Atlantic Application at 19; Bell Atlantic Lacouture/Troy Decl. at para. 73. Bell AtlanticLacouture/Troy Reply Decl. at para. 71. New York Commission Comments at 88. Department of Justice Evaluation at 18 n.40. The Department also notes that KPMG did not test whether the hot cut resulted in a working loop. Id. With regard to this argument, we refer to our previous discussion and finding that Bell Atlantic demonstrates that it provides hot cut loops at an acceptable level of quality and with a minimum of service disruption. AT&T Meek Aff. Attach. 6 at 3 (Letter from Peter McGowan, Associate Counsel, New York PSC, to Randal Milch, Associate General Counsel, Bell Atlantic, and Bob Mulvee, Associated General Counsel, AT&T, dated October 12, 1999). Bell Atlantic Lacouture/Troy Reply Decl. at para. 60. See AT&T Mulligan Aff. at para. 38. We similarly reject AT&T's argument that Bell Atlantic is not able to perform accurate migrations of loops that are served over IDLC facilities. See AT&T Meek Aff. at paras. 132-35. Rather, we accord significant weight to KPMG's finding that the methods and procedures adopted by Bell Atlantic permit effective migrations of these loops. After reaching this conclusion, KPMG closed the exception regarding Bell Atlantic's performance in providing cutovers of ILDC loops. See KPMG Exception No. 44. See also KPMG Final Report at POP3 IV-61-62 (Test Cross Reference P3-24); New York Commission Comments at 91-92. Department of Justice Evaluation at 21. As the Department noted, customers served by unbundled local loops tend to be heavy telecommunications users and, therefore, also tend to be extremely profitable customers for both Bell Atlantic and competing carriers. See AT&T Mulligan Aff. at paras. 6-7; Department of Justice Evaluation at 21. KPMG Final Report at  IV.L.3.1, Table IV12.6, P12-4. KPMG stated that it "confirmed that BA-NY as stated taken actions [sic] to address [increased volumes] of LNP Hot Cut orders. Specifically, BA-NY is opening a second RCCC to handled coordinated orders within Bell Atlantic North, including New York. This new RCCC currently has a staff of 20 coordinators and an ultimate staffing goal of 128 non-management personnel." Id. We also note, however, that we expect Bell Atlantic to expand its manual hot cut capacity further as it experiences increases in demand. AT&T Comments at 37; AT&T Mulligan Aff. at para. 38; Department of Justice Evaluation at 21. See also supra Section V.B.1.f.(ii).(a). See infra Section VII. Bell Atlantic Lacouture/Troy Decl. at para. 87. Bell Atlantic Dowell/Canny Decl. Attach. D (metric MR-3-01 - Loop for July 1999). In July, Bell Atlantic missed 16.57 percent of loop repair appointments for competitors and 12.28 appointments for its own customers. Id. Bell Atlantic Dowell/Canny Decl. Attach. D (metric MR-3-01 - Loop for August 1999). In August, Bell Atlantic missed 14.00 percent of loop repair appointments for competing carriers and 10.47 percent of appointments for repairs to its own customers' loops. Id. Bell Atlantic Dowell/Canny Decl. Attach. C, Ex. C at 6 (metric MR-3-01 - Loop for September 1999). September data demonstrate that Bell Atlantic missed 12.27 percent of repair appointments for competitors and 11.23 percent of appointments for its own customers. Id. Bell Atlantic Dowell/Canny Decl. Attach. D (metric MR-4-02 - Loop Trouble for July 1999). In July, loop repairs were completed in, on average, 28.33 hours for competitors and 29.60 hours for Bell Atlantic customers. Id. Bell Atlantic Dowell/Canny Decl. Attach. D (metric MR-4-02 - Loop Trouble for August 1999). Specifically, in August, loop repairs were completed in, on average, 26.22 hours for competitors and 25.32 hours for Bell Atlantic customers. Id. Bell Atlantic Dowell/Canny Reply Decl. Attach. C at 6 (metric MR-4-02 - Loop Trouble for August 1999). In September, loop repairs were completed in, on average, 25.08 hours for competitors and 27.12 hours for Bell Atlantic customers. Id. KPMG Final Report at M&R1 V-13-23 (RETAS functional and parity evaluation) & M&R5 V-75-77 (parity evaluation). Bell Atlantic Lacouture/Troy Decl. at para. 89. Omnipoint Comments at 11; Prism Comments at 13. With xDSL technology, two modems are attached to the local loop: one at the subscriber's premises and one at the telephone company's central office. The use of xDSL modems allows transmission of data over the copper loops at vastly higher speeds than can be achieved with analog data transmission. An ordinary voice channel in the United States, for instance, generally allows transmission of digital information at the rate of up to 56,000 bits per second. By contrast, xDSL services permits data to be transmitted to the end user at up to several million bits per second, depending upon loop length, loop design, and the technology deployed. Advanced Services Order and NPRM, 13 FCC Rcd at 24026-27. Local Competition First Report and Order, 11 FCC Rcd at 15691. New York Commission Comments at 92-93. Id. at 94-95. New York Commission Reply at 34. Id. Id. Id. at 31-32. New York Commission Comments at 94. Id. at 94; Bell Atlantic Lacouture/Troy Reply Decl. at para 97. New York Commission Comments at 94; New York Commission Reply at 35. Bell Atlantic Lacouture/Troy Decl. at para. 81; Department of Justice Ex. 8 at 2. Bell Atlantic Lacouture/Troy Decl. at paras. 77-78; Department of Justice Ex. 8 at 2. Bell Atlantic provides two types of loops over which competitors may provide advanced services: premium digital loops and loops that are specifically intended for use in the provision of xDSL services. Bell Atlantic Lacouture/Troy Decl. at paras. 77, 80. Premium digital loops are used for the provision of Bell Atlantic's retail ISDN services and, on occasion, can be utilized for the provision of xDSL services. We are unable to determine from the record what portion of Bell Atlantic's premium digital loops has been used by competitors for the provision of advanced services. Covad Cutcher/McChesney/Clancy Decl. at para. 37. Bell Atlantic Lacouture/Troy Decl. at paras. 66, 81; Bell Atlantic Lacouture/Troy Reply Decl. at para. 34; Department of Justice Ex. 8 at 2. Bell Atlantic missed 4.60 percent of digital loop installation appointments in January 1999, and then demonstrated significantly improved performance through July 1999. See Bell Atlantic Lacouture/Troy Decl. at para. 79 & Attach. J. In August 1999, Bell Atlantic again missed 4.00 percent of installation appointments for premium digital loops provisioned to competing carriers. Id. Bell Atlantic Lacouture/Troy Reply Decl. Attach. F. Id. Covad Conley/Poulicakos Decl. at para. 28. Bell Atlantic Lacouture/Troy Decl. Attach. K; Bell Atlantic Lacouture/Troy Reply Decl. Attach. F. Covad Conley/Poulicakos Decl. at para. 28. NorthPoint Comments at 18-19; NAS Comments at 8. Bell Atlantic Lacouture/Troy Reply Decl. at para. 82. Specifically, Bell Atlantic states that it received 21 repair orders on the 824 xDSL loops it installed between September 15 and October 15, 1999. Id. Covad Cutcher/McChesney/Clancy Decl. at para. 62. NorthPoint Comments at 18. Bell Atlantic Lacouture/Troy Reply Decl. at para. 85. Id. This is similarly the case with respect to the timely return of Firm Order Commitments (FOCs). Although Covad claims that from June through August 1999, Bell Atlantic was, on average, two days late in providing it with FOCs for xDSL orders, Covad begins measuring the FOC interval the first time it submits an order, whereas Bell Atlantic calculates the interval from the time it receives an error-free order. See Covad Cutcher/McChesney/Clancy Decl. at para. 34. We believe that it would be appropriate to measure FOC intervals from the time a valid order is placed. Bell Atlantic Lacouture/Troy Reply Decl. at para. 86. Department of Justice Evaluation at 27-28. Id. at 28. See supra paras. 283, 284, 291-302. Future applicants, for instance, may have the benefit of clearly-defined performance standards and verified performance data with respect to xDSL-capable loop provisioning. In addition, future applicants will have a clear picture of the evidentiary showing we would expect for a showing of checklist compliance with respect to xDSL- capable loops. The principal section of the 1996 Act concerning advanced telecommunications services is Section 706, Pub. L. 104-104, Title VII,  706, Feb. 8, 1996, 110 Stat. 153, reproduced in the notes under 47 U.S.C.  157. Pursuant to the Local Competition First Report and Order, a BOC must offer access to loops capable of transmitting the digital signals necessary to provide the full range of xDSL-based services. Local Competition First Report and Order, 11 FCC Rcd at 15692-93. Separate affiliates can also be utilized to demonstrate checklist compliance for conventional services. Specifically, Bell Atlantic on December 10, 1999 committed to establish a separate advanced services affiliate that will be distinct from its local exchange company and will operate largely in accordance with the structural, transactional, and nondiscrimination requirements of sections 272. See Letter from Thomas J. Tauke, Senior Vice President - Government Relations, Bell Atlantic, to The Honorable William E. Kennard, CC Docket No. 99-295, Attach. 1 at 1-2 (filed December 10, 1999). Under Bell Atlantic's commitment to establish this affiliate, which conforms to the conditions to our approval of the SBC/Ameritech merger, Bell Atlantic will transfer to the separate affiliate specified advanced services equipment on an exclusive basis during a limited grace period, to end on July 1, 2000. After February 1, 2000, all new advanced services equipment will be purchased and owned by the separate affiliate. After July 1, 2000, the responsibility of providing retail advanced services would rest with the separate affiliate. The particular activities in which the separate affiliate and the incumbent LEC may engage are set forth in the SBC/Ameritech merger conditions. See In re Ameritech Corp. and SBC Communications Inc. For Consent to Transfer Control of Corporations Holding Commission Licenses and Lines Pursuant to Section 214 and 310(d) of the Communications Act and Parts 5, 22, 24, 25, 63, 90, 95 and 101 of the Commission's Rules, CC Docket No. 98-141, Memorandum Opinion and Order, 14 FCC Rcd 14712, 14859-67, 14969-99 (1999). We view it as critical that a BOC provide all forms of advanced services through a separate affiliate, and not just ADSL, so the affiliate would need to obtain stand-alone loops from the BOC in order to provide all varieties of advanced services. As discussed supra in Section III, where a retail analogue exists, a BOC must provide access to competing carriers in "substantially the same time and manner" as it provides access to itself. Ameritech Michigan Order, 12 FCC Rcd at 20118-19. If there is no appropriate retail analogue, the BOC must demonstrate that the access it provides to competing carriers would afford an efficient carrier a "meaningful opportunity to compete." Id. 47 U.S.C.  271(c)(2)(B)(v). Second BellSouth Louisiana Order, 13 FCC Rcd at 20719. Id. A BOC has the following obligations with respect to dedicated transport: (a) provide unbundled access to dedicated transmission facilities between BOC central offices or between such offices and serving wire centers (SWCs), SWCs and interexchange carriers points of presence (POPs), tandem switches and SWCs, end offices or tandems of the BOC, and the wire centers of BOCs and requesting carriers; (b) provide all technically feasible transmission capabilities such as DS1, DS3, and Optical Carrier levels (e.g., OC-3/12/48/96) that the competing carrier could use to provide telecommunications; (c) not limit the facilities to which dedicated interoffice transport facilities are connected, provided such interconnections are technically feasible, or restrict the use of unbundled transport facilities; and (d) to the extent technically feasible, provide requesting carriers with access to digital cross-connect system functionality in the same manner that the BOC offers such capabilities to interexchange carriers that purchase transport services. Id. at 20719. Id. at 20719 n.650. The Commission also found that a BOC has the following obligations with respect to shared transport: (a) provide shared transport in a way that enables the traffic of requesting carriers to be carried on the same transport facilities that a BOC uses for its own traffic; (b) provide shared transport transmission facilities between end office switches, between its end office and tandem switches, and between tandem switches in its network; (c) permit requesting carriers that purchase unbundled shared transport and unbundled switching to use the same routing table that is resident in the BOC's switch; and (d) permit requesting carriers to use shared (or dedicated) transport as an unbundled element to carry originating access traffic from, and terminating traffic to, customers to whom the requesting carrier is also providing local exchange service. Id. at 20762, n.652. Bell Atlantic Lacouture /Troy Decl. at para. 106; NY PSC 916 Tariff  5.3 (Appendix H, Tab 3 of Bell Atlantic's 271 Application). New York Commission Comments at 100-04. See also Intermedia Comments at 8-9. Bell Atlantic's August 1999 data shows a missed appointment rate of 12.03 percent for interoffice facilities provided to competitive LECs and a missed appointment rate of 18.03 percent for Bell Atlantic retail special services. Bell Atlantic Dowell/Canny Decl. Attach. G at 15 (metric PR-4-01). In September 1999, Bell Atlantic had a missed appointment rate of 18.75 percent for interoffice facilities provided to competitive LECs and a missed appointment rate of 18.58 percent for Bell Atlantic retail special services. Bell Atlantic Comments Dowell/Canny Reply Decl. Attach. C at 10. The New York Commission uses a retail analogue to measure parity New York Commission Comments at 103. See supra Section V.B. See Allegiance Comments at 12; Choice One Comments at 9; Focal Comments at 3-6; OmniPoint Comments at 7-8,12-13; Teligent Comments at 16. Bell Atlantic Lacouture/Troy Reply Decl. at para. 114. See, e.g., Letter from Jonathan Askin, Vice President - Law, The Association for Local Telecommunications Services, Carol Ann Bischoff, Executive Vice President and General Counsel, Competitive Telecommunications Association, James Falvey, Vice President, Regulatory Affairs, e.spire Communications, Inc., Richard J. Metzger, Vice President, Regulatory and Public Policy, Focal Communications Corporation, Douglas G. Bonner, Arent Fox Kintner Plotkin & Kahn, PLLC, Counsel to Omnipoint Communications, Inc., and David S. Turetsky, Senior Vice President, Law and Regulatory, Teligent, Inc. to Magalie Roman Salas, Secretary, Federal Communications Commission, CC Docket No. 99-295 (filed December 16, 1999) (ALTS Dec. 16 Ex Parte Letter). For example, local transport is provided between BOC and/or competitive LEC wire centers or switches while in the case of special access at least one end of the transmission facility is located at a customer premise. Letter from Dee May, Director, Regulatory Affairs, Bell Atlantic to Claudia Pabo, Common Carrier Bureau, FCC, CC Docket No. 99-295 (filed Nov. 19, 1999). These parties do not challenge the assertion that special access is a service offering while unbundled transport is not, although they argue that this should not remove it from consideration in the context of checklist compliance. We note that a number of checklist items in addition to unbundled transport have interstate access tariff analogs, including the local loop and local switching. Our reasoning here applies equally to the consideration of the local loop component of special access in the context of the unbundled local loop checklist requirement. For the reasons addressed in this section, we also conclude that there is no need to consider the provision of special access in the context of the public interest requirement. See, supra, Section V.B.2. The fact that Bell Atlantic provides EELs pursuant to state requirements is not dispositive of section 271 checklist obligations. Bell Atlantic Lacouture/Troy Reply Decl. at para. 115. Id. In reaching this conclusion, we note that Choice One is the only competitive LEC which reports experiencing this problem with the provisioning of dedicated transport in this proceeding. 47 U.S.C.  271(c)(2)(B)(vi); see also Second BellSouth Louisiana Order, 13 FCC Rcd at 20722. Second BellSouth Louisiana Order, 13 FCC Rcd at 20722-24. Id. at 20722. Id. at 20722-23. Id. at 20723, 20733-34. Id. at 20723 (citing the Ameritech Michigan Order, 12 FCC Rcd at 20619, 20717-18). Id. at 20723. Id. at 20723. Id. at 20723 (citing the Ameritech Michigan Order, 12 FCC Rcd at 20705). Id. at 20723 (citing the Ameritech Michigan Order, 12 FCC Rcd at 20714-15). Bell Atlantic provides unbundled local switching under its tariffs and approved interconnection agreements. Bell Atlantic Application at 22, n.25; Bell Atlantic Lacouture/Troy Decl. at para. 90. See also Bell Atlantic Application at 23 (citing KPMG Report); Bell Atlantic Lacouture/Troy Decl. at paras. 90, 91, 95, 105; Letter from Joseph J. Mulieri, Director, Government Relations, Bell Atlantic, to Magalie Roman Salas, Secretary, Federal Communications Commission, CC Docket No. 99-295 (filed November 18, 1999). Line-side facilities include, but are not limited to, the connection between a loop termination at a main distribution frame, and a switch line card. Trunk-side facilities include, but are not limited to, the connection between trunk termination at a trunk-side cross-connect panel and a switch trunk card. Second BellSouth Louisiana Order, 13 FCC Rcd at 20724 nn.679-680. The basic switching function includes, but is not limited to: connecting lines to lines, lines to trunks, trunks to lines, trunks to trunks, as well as the same basic capabilities that are available to the BOC's customers, such as a telephone number, directory listing, dial tone, signaling, and access to 911, operator services, and directory assistance. Second BellSouth Louisiana Order, 13 FCC Rcd at 20726 n.690. Second BellSouth Louisiana Order at 13 FCC Rcd at 20726. Vertical features provide end-users with various services such as custom calling, call waiting, call forwarding, caller ID and Centrex. Id. at 20726. An incumbent LEC must provide customized routing as part of the local switching element, unless it can prove to the state commission that customized routing in a particular switch is not technically feasible. Second BellSouth Louisiana Order at 13 FCC Rcd at 20728 n. 705. Customized routing permits requesting carriers to designate the particular outgoing trunks associated with unbundled switching provided by the incumbent, which will carry certain classes of traffic originating from requesting carriers' customers. See id. at 20728-29. Customized routing is also referred to as selective routing. Id. at 20728 n.704. Local Competition Third Reconsideration Order, 12 FCC Rcd at 12475-79; Ameritech Michigan Order, 12 FCC Rcd at 20716-17; see also Second BellSouth Louisiana Order, 13 FCC Rcd at 20732. The requirement to provide unbundled tandem switching includes: (i) trunk-connect facilities, including but not limited to the connection between trunk termination at a cross-connect panel and a switch trunk card; (ii) the base switching function of connecting trunks to trunks; and, (iii) the functions that are centralized in tandem switches (as distinguished from separate end-office switches), including but not limited to call recording, the routing of calls to operator services, and signaling conversion features. Second BellSouth Louisiana Order, 13 FCC Rcd at 20733 n.732. See id. at 20733-35. See id. at 20735-37. New York Commission Comments at 110. See also ALTS Comments at 14-15; Intermedia Comments at 9. Z-Tel Comments at 10-13. See id. at 13. Bell Atlantic Reply at 27-28. Bell Atlantic Lacouture/Troy Decl. at para. 91. See supra Section V.B. We note that none of Bell Atlantic's metrics applies expressly to the provisioning of unbundled local switching separate from provisioning this element as part of the UNE platform. Letter from Michael B. Hazzard, Lawler, Metzger & Milkman, LLC, Counsel to Z-Tel, to Magalie Roman Salas, Secretary, Federal Communications Commission, CC Docket No. 99-295 (filed Nov. 2, 1999) (Z-Tel Nov. 2, 1999 Ex Parte Letter). Letter from Dee May, Director, Federal Regulatory Affairs, Bell Atlantic, to Sanford Williams, Policy & Program Planning Division, Common Carrier Bureau, Federal Communications Commission, CC Docket No. 99- 295 at 1 (filed November 23, 1999). 47 U.S.C.  271(c)(2)(B)(vii). Ameritech Michigan Order, 12 FCC Rcd at 20679. Id. Id. Bell Atlantic Lacouture/Troy Decl. paras 161-170. See KPMG Final Report at  VIIE.3.1, Table VII5.5, R5.2-1-7 (App. C, Tab 916) (verifying that Bell Atlantic's process for E911 access for competitive LECs using Bell Atlantic's switches are satisfactory); KPMG Final Report at  VII5.4, R5.1-4 (App. C, Tab 916) (verifying Bell Atlantic's ability to provide E911 functionality). New York Commission Comments at 116. See also ALTS Comments at 15-16 (asserting that Bell Atlantic has provided nondiscriminatory access to 911/E911); Intermedia Comments at 9-10 (stating that in Intermedia's experience, Bell Atlantic complies with this element of checklist item 7). 47 U.S.C.  271(c)(2)(B)(vii)(II), (III). 47 U.S.C.  251(b)(3). 47 C.F.R.  51.217; Implementation of the Local Competition Provisions of the Telecommunications Act of 1996, CC Docket 96-91, Second Report and Order and Memorandum Opinion and Order, 11 FCC Rcd 19392 (1996) (Local Competition Second Report and Order) aff'd in part and vacated in part sub nom, People of the State of California v. FCC, 124 F.3d 934 (8th Cir. 1997), overruled in part, AT&T Corp. v. Iowa Utils Bd., 119 S. Ct. 721 (1999); Provision of Directory Listings Information under the Telecommunications Act of 1934, as amended, CC Docket No. 99-273, FCC 99-227, Notice of Proposed Rulemaking (rel. Sep. 9, 1999). While both section 251(b)(3) and 271(c)(2)(B)(vii)(II) refer to nondiscriminatory access to "directory assistance," section 251(b)(3) refers to nondiscriminatory access to "operator services" while section 271(c)(2)(B)(vii)(III) refers to nondiscriminatory access to "operator call completion services." 47 U.S.C. 251(b)(3), 271(c)(2)(B)(vii)(III). The term "operator call completion services" is not defined in the Act, nor has the Commission previously defined the term. However, for section 251(b)(3) purposes, the term "operator services" was defined as meaning "any automatic or live assistance to a consumer to arrange for billing or completion, or both, of a telephone call." Local Competition Second Report and Order, 11 FCC Rcd at 19448. In the same order the Commission concluded that busy line verification, emergency interrupt, and operator-assisted directory assistance are forms of "operator services," because they assist customers in arranging for the billing or completion (or both) of a telephone call. Id. at 19449. All of these services may be needed or used to place a call. For example, if a customer tries to direct dial a telephone number and constantly receives a busy signal, the customer may contact the operator to attempt to complete the call. Since billing is a necessary part of call completion, and busy line verification, emergency interrupt, and operator-assisted directory assistance can all be used when an operator completes a call, we concluded in the Second BellSouth Louisiana Order that for checklist compliance purposes "operator call completion services" is a subset of or equivalent to "operator services." Second BellSouth Louisiana Order, 13 FCC Rcd 20740 n.763. As a result, we use the nondiscriminatory standards established for operator services to determine whether nondiscriminatory access is provided. 47 C.F.R.  51.217(c)(3); Local Competition Second Report and Order, 11 FCC Rcd at 19456, 19457. The Local Competition Second Report and Order's interpretation of section 251(b)(3) is limited "to access to each LEC's directory assistance service." Id. at 19456. However, section 271(c)(2)(B)(vii) is not limited to the LEC's systems but requires "Nondiscriminatory access to . . . directory assistance to allow the other carrier's customers to obtain telephone numbers." 47 U.S.C.  271(c)(2)(B)(vii). Combined with the Commission's conclusion that incumbent LECs must unbundle the facilities and functionalities providing operator services and directory assistance from resold services and other unbundled network elements to the extent technically feasible, Local Competition First Report and Order, 11 FCC Rcd at 15772-73, section 271(c)(2)(B)(vii)'s requirement should be understood to require the BOCs to provide nondiscriminatory access to the directory assistance service provider selected by the customer's local service provider, regardless of whether the competitor provides such services itself; selects the BOC to provide such services; or chooses a third party to provide such services. Provision of Directory Listings Information under the Telecommunications Act of 1934, as amended, CC Docket No. 99-273, Notice of Proposed Rulemaking (rel. Sep. 9, 1999). Local Competition Second Report and Order, 11 FCC Rcd at 19464. Id. at 19449, 19450 47 C.F.R.  51.217(d); Local Competition Second Report and Order, 11 FCC Rcd at 19455, 19463. For example, when customers call the operator or calls for directory assistance, they typically hear a message such as "Thank you for using XYZ Telephone Company." Competing carriers may use the BOC's brand, request the BOC to brand the call with the competitive carrier's name or request that the BOC not brand the call at all. 47 C.F.R.  51.217(d). 47 C.F.R.  51.217(c)(3)(ii); Local Competition Second Report and Order, 11 FCC Rcd at 19460-61. Bell Atlantic Lacouture/Troy Decl. 172-192. See KPMG Final Report at  VII5.5, R5.2-1-7 (App. C, Tab 916) (concluding that Bell Atlantic's processes for Directory Assistance are satisfactory). New York Commission Comments at 116-117. See also ALTS Comments at 15-16; Intermedia Comments at 9-10. AT&T Comments at 41-44. See also Choice One Comments at 7-8 (citing a single customer whose directory listing was dropped from the database); Department of Justice Evaluation at 19-20 (citing AT&T's studies). Bell Atlantic Lacouture/Troy Reply Decl. at para. 152. According to Bell Atlantic, AT&T's studies are misleading in that they address an unrepresentative subset of the total competitive LEC local service orders that are added to Bell Atlantic's directory listing database on a monthly basis. Bell Atlantic asserts that AT&T's studies do not address directory listings which are established for competitive LEC resale or UNE-Platform orders. According to Bell Atlantic, these types of orders account for nearly 80 percent of all competitive LEC orders and enjoy a 0 percent error rate. Id. at para. 154. Bell Atlantic further argues that "[f]ully 60 percent of the orders AT&T claimed were missing were, in fact, in the database by the end of the second business day in accordance with the Carrier to Carrier Guidelines." Id. at para. 155. For example, beginning in September 1999, Bell Atlantic increased the personnel dedicated to monitoring and correcting database entries. Bell Atlantic Lacouture/Troy Reply Decl. at para. 157. See ALTS Comments at 15-16; Intermedia Comments at 9-10; New York Commission Comments at 116- 17. The Department of Justice stated that evidence in the record subsequent to KPMG's review of Bell Atlantic's process improvement plan "suggests that the process changes have not provided a sufficient solution" to the problem of dropped directory listings associated with provisioning of hot cuts. Department of Justice Evaluation at 19-20. 47 U.S.C.  271(c)(2)(B)(viii). 47 U.S.C.  251(b)(3). Second BellSouth Louisiana Order, 13 FCC Rcd at 20748. Id. at 20748. We note that in the Second BellSouth Louisiana Order, we stated that the definition of "directory listing" was synonymous with the definition of "subscriber list information." Id. at 20747 (citing the Local Competition Second Report and Order, 11 FCC Rcd at 19458-59). However, the Commission's decision in a recent proceeding obviates this comparison, and supports the definition of directory listing delineated above. See Implementation of the Telecommunications Carriers' Use of Customer Proprietary Network Information and Other Customer Information, CC Docket No. 96-115, Third Report and Order; Implementation of the Local Competition Provisions of the Telecommunications Act of 1996, CC Docket No. 96-98, Second Order on Reconsideration; Provision of Directory Listing Information under the Telecommunications Act of 1934, as amended, CC Docket No. 99-273, FCC 99-227, Notice of Proposed Rulemaking, para. 160 (rel. Sept. 9, 1999). Second BellSouth Louisiana Order, 13 FCC Rcd at 20747-48. Bell Atlantic provides competitive LECs with basic white page directory listings under interconnection agreements and tariffs. Bell Atlantic Lacouture/Troy Decl. at para. 194. Additional white page listings and other white page listing services are provided under tariff on the same terms and conditions as those provided to Bell Atlantic customers. Id. See also Bell Atlantic App. C, Tab 535 (KPMG Closure Report for Exception 56, 7/22/99). Bell Atlantic Application at 30; Bell Atlantic Lacouture/Troy Decl. at para. 195. See Second BellSouth Louisiana Order, 13 FCC Rcd at 20748-49; see also New York PSC Comments at 122-23; ALTS Comments at 17- 18; Intermedia Comments at 10. Bell Atlantic Application at 30; Bell Atlantic Lacouture/Troy Decl. at para. 196. See Second BellSouth Louisiana Order, 13 FCC Rcd at 20749-50. New York Commission Comments at 123. AT&T Comments at 41-44; Choice One Comments at 7-8. See supra Section V.B. AT&T Comments at 41-44; AT&T Callahan/Connolly Aff. KPMG similarly found little evidence of actual dropped listings in its discussion of problems with Bell Atlantic's OSS for white pages directory listings orders. Bell Atlantic Application App. C, Tab 535 (KPMG Closure Report for Exception 56, 7/22/99). See also Bell Atlantic Application at 31; Bell Atlantic Lacouture/Troy Decl. at para. 210. Choice One Comments at 7-8. 47 U.S.C.  271(c)(2)(B)(ix). Id. In the Matter of Request of Lockheed Martin Corporation and Warburg, Pincus & Co. for Review of the Transfer of the Lockheed Martin Communications Industry Services Business, CC Docket No. 92-237, Order, FCC 99-346 (rel. Nov. 17, 1999). See generally 47 C.F.R. Part 52 - Numbering, Subpart B - Administration,  52.7- 52.19. See Administration of the North American Numbering Plan, Report and Order, 11 FCC Rcd 2588, 2615; NANP Third Report and Order, 12 FCC Rcd at 23042-46; see also Bell Atlantic Lacouture/Troy Decl. at para. 211. See Second Louisiana Order, 13 FCC Rcd at 20752. See also Bell Atlantic Lacouture/Troy Decl. at para. 213. See, e.g., Central Office Code (NXX) Assignment Guidelines (INC 95-0407-008) (revised August 1999). Bell Atlantic Lacouture/Troy Decl. at para. 213. See also Bell Atlantic Application App. C, Tab 535 (KPMG Closure Report for Exception 46, 7/22/99). New York Commission Comments at 127. Bell Atlantic's compliance is also supported by KPMG's findings. Bell Atlantic Lacouture/Troy Decl. at para. 216. 47 U.S.C.  271(c)(2)(B)(x). Second BellSouth Louisiana Order, 13 FCC Rcd at 20753. Id. at 20755-56. Bell Atlantic Application at 31-33; Bell Atlantic Lacouture/Troy Decl. at paras. 218-247. Bell Atlantic provides access to signaling, call-related databases, SCE, and the SMS databases under interconnection agreements and tariffs. Bell Atlantic Lacouture/Troy Decl. at paras. 220, 226, 229, 232, 234, 238, 242, 245. New York Commission Comments at 133. Z-Tel Comments at 23. Ameritech Michigan Order, 12 FCC Rcd at 20602; BellSouth South Carolina Order, 13 FCC Rcd at 582. See also Bell Atlantic Lacouture/Troy Decl. at paras. 243-247. 47 U.S.C.  271(c)(2)(B)(xi). 47 U.S.C.  251(b)(2). 47 U.S.C.  153(30). 47 C.F.R.  52.21(k). Second BellSouth Louisiana Order, 13 FCC Rcd at 20757 (citing 47 U.S.C.  251(e)(2) and In the Matter of Telephone Number Portability, Third Report and Order, 13 FCC Rcd 11701, 11702-11704, para. 4 & nn.4, 7, 9, 12 (1998) (Third Number Portability Order)). See also In the Matter of Telephone Number Portability, Fourth Memorandum Opinion and Order on Reconsideration, CC Docket No. 95-116, RM 8535 at paras. 1, 6-9 (June 23, 1999)(Fourth Number Portability Order). Fourth Number Portability Order at para. 10 (citing In re Telephone Number Portability, First Report and Order and Further Notice of Proposed Rulemaking, 11 FCC Rcd 8352, 8409-12 (1996)(First Number Portability Order). See also 47 U.S.C.  251(b)(2). See 47 C.F.R. 52.23(b)-(f); Second BellSouth Louisiana Order, 13 FCC Rcd at 20758; First Number Portability Order, 11 FCC Rcd at 8355-56, 8399-8404; Third Number Portability Order, 13 FCC Rcd at 11708-12. See 47 C.F.R.  52.29; Second BellSouth Louisiana Order, 13 FCC Rcd at 20758; First Number Portability Order, 11 FCC Rcd at 8417-24. See 47 C.F.R.  52.32-52.33; Second BellSouth Louisiana Order, 13 FCC Rcd at 20578; Third Number Portability Order, 13 FCC Rcd at 11706-07; Fourth Number Portability Order at para. 9; see generally Fourth Number Portability Order. Bell Atlantic Application at 33-34; Bell Atlantic Lacouture/Troy Decl. at paras. 248-256. Bell Atlantic provides number portability to requesting carriers under interconnection agreements and tariffs. Bell Atlantic Lacouture/Troy Decl. at paras. 249, 255. New York Commission Comments at 136. RCN Comments at 10-11. Id. at 10. Bell Atlantic Reply Lacouture/Troy Decl. at paras. 117-118. Adelphia Livengood Aff. at para.17. See id. See AT&T Meek Aff. at paras. 42-43. See supra Section V.D (for further discussion regarding Bell Atlantic's hot cut provisioning). Based on the Commission's view that section 251(b)(3) does not limit the duty to provide dialing parity to any particular form of dialing parity (i.e., international, interstate, intrastate, or local), the Commission in August 1996 adopted rules to implement broad guidelines and minimum nationwide standards for dialing parity. Local Competition Second Report and Order, 11 FCC Rcd 19392 at 19407; Interconnection Between Local Exchange Carriers and Commercial Mobile Radio Service Providers, CC Docket No. 9-98, Further Order On Reconsideration, FCC 99-170 (rel. July 19, 1999). 47 U.S.C.  251(b)(3). Id. at  153(15). 47 C.F.R  51.205, 51.207. See 47 C.F.R.  51.207 (requiring same number of digits to be dialed); Local Competition Second Report and Order, 11 FCC Rcd at 19400, 19403. Bell Atlantic Lacouture/Troy Decl. at paras. 257-261, Att. A. New York Commission Comments at 137-139. See also ALTS Comments at 21 (Bell Atlantic appears to provide local dialing parity); Intermedia Comments at 12-13 (Bell Atlantic appears to be providing local dialing parity throughout New York). 47 U.S.C.  271(c)(2)(B)(xiii). Id.  252(d)(2)(A). Bell Atlantic Lacouture/Troy Decl. at para. 262. Bell Atlantic provides reciprocal compensation to competing carriers for the termination of local calls from Bell Atlantic customers under approved interconnection agreements and tariffs. (See, e.g., AT&T Interconnection Agreement). During the first seven months of 1999, Bell Atlantic exchanged an average of 2.5 billion minutes of traffic each month with 27 local wireline carriers in New York. During this period, Bell Atlantic paid more than 98.4 million dollars to competitive LECs as reciprocal compensation. Bell Atlantic asserts that it is paying reciprocal compensation payments consistent with the New York Commission's order, which governs Internet Service Provider (ISP) bound traffic. See Opinion and Order Concerning Reciprocal Compensation, Case 99-C- 0529 (NYPSC Aug. 26, 1999) (NYPSC Reciprocal Compensation Order). See also Bell Atlantic Brief at 34; Bell Atlantic Lacouture/Troy Decl. at paras. 262-264; Bell Atlantic Lacouture/Troy Reply Decl. at paras. 119-124. With regard to the second requirement, we note that section 271(c)(2)(A)(i) requires a showing that a BOC "is providing access and interconnection pursuant to one or more agreements . . . or . . . is generally offering access and interconnection pursuant to [an SGAT]." 47 U.S.C.  271(c)(2)(A)(i) (emphasis added). New York Commission Comments at 144. Global NAPS Comments at 2-4. Id. at 2. Implementation of the Local Competition Provisions in the Telecommunications Act of 1996: Inter-Carrier Compensation for ISP-Bound Traffic, Declaratory Rulemaking and Notice of Proposed Rulemaking, 14 FCC Rcd 3689 at n.87 (1999) (Inter-Carrier Compensation for ISP-Bound Traffic Order). Inter-Carrier Compensation for ISP-Bound Traffic Order, 14 FCC Rcd at 3707-3710. See Letter and Attachment from Ross A. Buntrock, Attorney, Kelley Drye & Warren, LLP, Counsel for e.spire, to Magalie Roman Salas, Secretary, Federal Communications Commission, CC Docket No. 99-295 (filed Nov. 3, 1999) (e.spire Nov. 3 Ex Parte Letter). 47 U.S.C.  271(c)(2)(B)(xiv). 47 U.S.C.  251(c)(4)(A). 47 U.S.C.  251(c)(4)(B). 47 U.S.C.  252(d)(3). See, e.g., 47 C.F.R.  51.613-51.617. The Eighth Circuit acknowledged the Commission's authority to promulgate such rules, and specifically upheld the sections of the Commission's rules concerning resale of promotions and discounts in Iowa Utilities Board. Iowa Utils. Bd. v. FCC, 120 F.3d at 818-19, aff'd in part and remanded on other grounds, AT&T v. Iowa Utils. Bd., 119 S.Ct. 721 (1999). 47 C.F.R.  51.613(b). Bell Atlantic Lacouture/Troy Decl. at para. 265. The telecommunications services that Bell Atlantic provides at retail to subscribers that are not telecommunications carriers are available at the following discount levels ordered by the New York Commission: 19.1 percent for lines with Bell Atlantic's Operator Services and Directory Assistance, and 21.7 percent for lines without these features. Bell Atlantic Application at 35, Bell Atlantic Lacouture/Troy Decl. at para. 265, Opinion and Order Determining Wholesale Discount, Case No. 95-C-0657 (NYPSC Nov. 27, 1996) (Bell Atlantic Application App.G, Tab 7) (NYSPSC Wholesale Discount Order). Bell Atlantic's retail telecommunications services are available for resale under interconnection agreements and its tariffs. Bell Atlantic Application at 35, Bell Atlantic Lacouture/Troy Decl. at para. 266. Through July 1999, Bell Atlantic has provided 314,000 resold lines to more than 65 competing carrier including more that 250,000 business lines and more than 63,000 residential lines. Also, as of September 1999, Bell Atlantic has provided 319,000 resold lines to more than 65 competing carriers. In addition, forty companies resell more than 100 lines, 22 companies resell more than 1,000 lines, and five companies resell more than 10,000 lines. Of the 522 Bell Atlantic wire centers in New York, 90 percent have at least one resold listing, and 64 have at least 10. Bell Atlantic Application at 35, Bell Atlantic Reply at 30, Bell Atlantic Lacouture/Troy Decl. at para. 267, Bell Atlantic Lacouture/Troy Reply Decl. at para. 159, Bell Atlantic Taylor Decl. at para. 42. Bell Atlantic's customer-specific arrangements (CSAs), grandfathered services and promotional offerings in effect more than 90 days are also provided at wholesale discounts set by the New York Commission. Bell Atlantic Application at 35, Bell Atlantic Lacouture/Troy Del. at para. 268. Customer-specific arrangements or contract service arrangements are contractual agreements made between a carrier and a specific customer, tailored to that customer's individual needs. The customer is typically a high-volume user of telecommunications services. Contract service arrangements may include volume and term arrangements, special service arrangements, customized telecommunications service arrangements and master service agreements. e.Spire/Net2000 Comments at 7 (citing Order Granting Petition, Case No. 98-C-426 (NYPSC Sept. 14, 1998) (NYPSC CTC Order)). We note that promotional offerings for a period greater than 90 days must be offered for resale at wholesale rates pursuant to section 251(c)(4)(A). Local Competition First Report and Order, 11 FCC Rcd at 15970. If Bell Atlantic's promotional offering is limited to 90 days or less, the CLEC may elect to have Bell Atlantic apply the wholesale discount to the retail price of telecommunications services included in the promotional offering, or pay the promotional offering price. Bell Atlantic Lacouture/Troy Decl. at para. 268. We note that there is a presumption that promotional offerings for a period of 90 days or less need not be offered at a discount to resellers. Local Competition First Report and Order, 11 FCC Rcd at 15970. New York Commission Comments at 150-151. CCA Comments at 2, 4-5. See also ALTS Reply at 15. CCA Comments at 4. CCA Comments at 6. Id. Id. 47 U.S.C.  252(d)(3). Bell Atlantic Lacouture/Troy Decl. at para. 265. See also NYPSC Wholesale Discount Order at 79. New York Commission Comments at 148 (citing Second Network Proceeding, Case 98-C-1357). Local Competition First Report and Order, 11 FCC Rcd at 15957-58. Id. Bell Atlantic Lacouture/Troy Decl. at para. 266. Bell Atlantic Application at 36; Bell Atlantic Lacouture/Troy Decl. at para. 268-270. Bell Atlantic Lacouture/Troy Decl. at para. 270. Bell Atlantic Brief at 36 (citing Expanded Interconnection First Reconsideration Order, 8 FCC Rcd at 7341). See also BellSouth South Carolina Order, 13 FCC Rcd at 539. ALTS Comments at 65-67; e.spire/Net2000 Comments at 3-10; TRA Comments at 23-28; ALTS Reply at 14; Letter from Ross A. Buntrock, Attorney, Kelley Drye & Warren, LLP, Counsel for ALTS, to Magalie Salas, Secretary, Federal Communications Commission, CC Docket No. 99-295 (filed Nov. 3, 1999) (ALTS Nov. 3 Ex Parte Letter); Letter from Ross A. Buntrock, Attorney, Kelley Drye & Warren, LLP, Counsel for e.spire, to Magalie Salas, Secretary, Federal Communications Commission, CC Docket No. 99-295 (filed Nov. 3, 1999) (e.spire Nov. 3 Ex Parte Letter); Letter from Ross A. Buntrock, Attorney, Kelley Drye & Warren, LLP, Counsel for Net.2000, to Magalie Salas, Secretary, Federal Communications Commission, CC Docket No. 99-295 (filed Nov. 3, 1999) (Net.2000 Nov. 3 Ex Parte Letter); Letter from Ross A. Buntrock, Attorney, Kelley Drye & Warren, Counsel for Net2000, to Claudia Pabo, Federal Communications Commission, CC Docket No. 99-295 (filed Dec. 3, 1999) (Net.2000 Dec. 3 Ex Parte Letter). ALTS Comments at 87-88; e.spire/Net2000 at 8-9. Bell Atlantic Reply Application at 30-31; Bell Atlantic Lacouture/Troy Reply Decl. at paras. 163-167. Bell Atlantic Application at 36; Bell Atlantic Reply at 30-31; Bell Atlantic Lacouture/Troy Reply Decl. at para. 166. Expanded Interconnection Order, 7 FCC Rcd at 7464-7465 n.466. e.spire/Net2000 Comments at 5; Net.2000 Dec. 3 Ex Parte Letter. According to Net.2000, Flexpath is essentially a T-1 line service for those customers who utilize PBX systems. Net2000 Dec. 3 Ex Parte Letter at 5. e.spire/Net2000, Net.2000 Dec. 3 Ex Parte Letter at 5. BellSouth South Carolina Order, 13 FCC Rcd at 539, 662. BellSouth South Carolina Order, 13 FCC Rcd at 662. See, e.g., Expanded Interconnection Order, 7 FCC Rcd at 7463-64; Interexchange Marketplace Order, 6 FCC Rcd 5880, 5906 (1991); Amendment of the Commission's Rules Relative to Allocation of the 849-851/894- 896 MHz Bands, 6 FCC Rcd 4582, 4583-84 (1991). See also 47 U.S.C.  253. New York Commission Reply at 40 (citing NYPSC CTC Order). We note that the New York Commission expressed concern about Bell Atlantic's use of termination liability, but did not find that Bell Atlantic's past actions constituted a violation of 47 U.S.C.  251(b)(1) or section 251(c)(4). New York Commission Reply at 40 (citing Order Denying Motion to Compel and for Sanctions and Clarifying the Order Granting Petition Complaint of CTC Communications Inc., Case No. 98-C-426 (NYPSC Feb. 1, 1999) (NYPSC Order Denying Motions and Clarification Order)). See New York Commission Reply at 41. Thus, we need not evaluate Bell Atlantic's liability provisions for termination of Flex Path T-1 service contracts in this proceeding because the termination liability does not on its fact constitute a violation of checklist item 14. New York Commission Reply at 41. 47 U.S.C.  253. See KMC Comments at 13; see also Allegiance Comments at 17; Allegiance Reply at 7-8. e.spire/Net.2000 Comments at 9 n.12 (citing In re Establishment of Rules to Prohibit the Imposition of Unjust, Onerous Termination Penalties on Customers Choosing to Partake of the Benefits of Local Exchange Telecommunications Competition, Petition for Declaratory Ruling, CC Docket No. 99-142 (filed Apr. 26, 1999) (requesting that the Commission declare unlawful termination penalties imposed by ILECs, to prohibit enforcement of ILEC termination penalties, and to require the removal of ILEC termination penalties from state tariffs until more competition develops)). TRA Comments at 27 (citing Bell South Louisiana Order, 13 FCC Rcd at 6245). TRA Comments at 27; see also Local Competition First Report and Order, 11 FCC Rcd at 15499. Bell Atlantic Lacouture/Troy Joint Decl. at para. 169. Bell Atlantic Reply at 31; Bell Atlantic Lacouture/Troy Reply Decl. at para. 170 (citing Federal Communications Commission Adopts Rules Applicable to the Sale of High-Speed Internet Services, News Release, CC Docket No. 98-147 (rel. Nov. 2, 1999)). Deployment of Wireline Services Offering Advanced Telecommunications Capability, CC Docket No. 98- 147, Second Report and Order, FCC 99-330 (rel. Nov. 9, 1999) (Advanced Services Second Report and Order). Advanced Services Second Report and Order at para. 19. NALA Comments at 2. See also ALTS Reply at 15-16. NALA Comments at 3. Id. at 2-3. Bell Atlantic Reply at 31, Bell Atlantic Lacouture/Troy Reply Decl. at para. 172. Similarly, Bell Atlantic states that if it offers flat-rate local telecommunications service in the future, Bell Atlantic will make it available for resale. Bell Atlantic Lacouture/Troy Reply Decl. at para. 172. Vertical features provide end-users with various services such as custom calling, call waiting, call forwarding, caller ID and Centrex. Second BellSouth Louisiana Order, 13 FCC Rcd at 20726. CCA Comments at 5-6. See also ALTS Reply at 14. CCA Comments at 5. Id. at 5-6. See also supra Section V.F. Second BellSouth Louisiana Order, 13 FCC Rcd at 20722-24. Second BellSouth Louisiana Order, 13 FCC Rcd at 20722. Second BellSouth Louisiana Order, 13 FCC Rcd at 20722-23. Bell Atlantic Lacouture/Troy Decl. at para. 90. Id. Id. Destek Comments at 2-3, App. A. According to Destek, the interconnection agreements provides for the deployment of Asynchronous Transfer Mode (ATM) Cell Relay services that allow Bell Atlantic and the state owned institutions to provide information, data, real-time voice and video conferencing, voice communications, internet access, local and wide area networking and telecommunications services initially to K-12 schools, states and federal agencies, nonprofit organizations and ultimately to businesses throughout the service area. Destek Comments at 3. Ntegrity Comments at 1-3. Bell Atlantic provides competitive LECs with retail telecommunications services available for resale under interconnection agreements and tariffs. Bell Atlantic Lacouture/Troy Decl. at paras. 84, 266. See also Bell Atlantic Application at 35-36; Bell Atlantic Lacouture/Troy Decl. At paras. 84-88, 265-279. Bell Atlantic's compliance is also supported by KPMG's findings. Bell Atlantic Application at 35; Bell Atlantic Lacouture/Troy Decl. at paras. 84, 267 Adelphia Comments, Aff. at para. 18; TRA Comments at 16-17. NALA Comments at 5. Id. Id. RCN Reply at 9. See also NALA Comments at 4; TRA Comments at 15. 47 U.S.C.  271(d)(3)(B). See Implementation of the Accounting Safeguards Under the Telecommunications Act of 1996, CC Docket No. 96-150, Report and Order, 11 FCC Rcd 17539 (1996) (Accounting Safeguards Order), recon. pending. Implementation of the Non-Accounting Safeguards of Sections 271 and 272 of the Communications Act of 1934, as amended, CC Docket No. 96-149, First Report and Order and Further Notice of Proposed Rulemaking, 11 FCC Rcd 21905 (1996) (Non-Accounting Safeguards Order), petition for review pending sub nom. SBC Communications v. FCC, No. 97-1118 (filed D.C. Cir. Mar. 6, 1997) (held in abeyance May 7, 1997), First Order on Reconsideration, 12 FCC Rcd 2297 (1997) (First Order on Reconsideration), Second Order on Reconsideration, 12 FCC Rcd 8653 (1997) (Second Order on Reconsideration), aff'd sub nom. Bell Atlantic Telephone Companies v. FCC, 131 F.3d 1044 (D.C. Cir. 1997), Third Order on Reconsideration, FCC 99-242 (rel. Oct. 4, 1999) (Third Order on Reconsideration). Non-Accounting Safeguards Order, 11 FCC Rcd at 21914; Accounting Safeguards Order, 11 FCC Rcd at 17550; Ameritech Michigan Order, 12 FCC Rcd at 20725. Non-Accounting Safeguards Order, 11 FCC Rcd at 21914; Ameritech Michigan Order, 12 FCC Rcd at 20725. Ameritech Michigan Order, 12 FCC Rcd at 20725; see AT&T Comments at 64; ALTS Comments at 69; CERB Comments at 5-6; CloseCall Comments at 8. Second BellSouth Louisiana Order, 13 FCC Rcd at 20785-86. Id. Section 272(a) states that a BOC may not provide certain services except through one or more affiliates that meet the requirements of section 272(b). See 47 U.S.C.  272(a)(1)(B). Bell Atlantic Application App. A, Vol. 1, Tab 5, Declaration of Maureen C. Breen at paras. 1-3 (Bell Atlantic Breen Decl.); Bell Atlantic Application App. A, Vol. 1, Tab 6, Declaration of Stewart Verge at paras. 2-3 (Bell Atlantic Verge Decl.); Bell Atlantic Application App. A, Vol. 1, Tab 7, Declaration of Susan C. Browning at paras. 4-6 (Bell Atlantic Browning Decl.). For an organizational chart, see Bell Atlantic Browning Decl. Attach. P at 12 (showing Bell Atlantic section 272 affiliates, operating telephone companies, and service organizations). Bell Atlantic Breen Decl. at para. 4, Attach. A (submitting articles of incorporation for BACI and NLD); Bell Atlantic Verge Decl. at para. 4, Attach. A (submitting articles of incorporation for BAGNI). Bell Atlantic Browning Decl. Attach. E & P. Bell Atlantic Application at 54 (citing Bell Atlantic Browning Decl. at paras. 30-34; Bell Atlantic Breen Decl. at paras. 18-24; Bell Atlantic Verge Decl. at paras. 20-26). Among its internal control mechanisms are a corporate compliance program, corporate-wide supervision of affiliate relationships, and periodic employee training. See Bell Atlantic Browning Decl. Attach. E. 47 U.S.C.  272(b)(1); see also Non-Accounting Safeguards Order, 11 FCC Rcd at 21981-87; Second BellSouth Louisiana Order, 13 FCC Rcd at 20787-88; see Bell Atlantic Application at 49-50, 54-55 (describing internal control structure); Bell Atlantic Browning Decl. at para. 8(b)-8(c); Bell Atlantic Breen Decl. at paras. 11 (stating that BACI and NLD own neither domestic telecommunications facilities nor related land and buildings), 13 (stating that BACI and NLD do not jointly own switching and transmission facilities or related land and buildings); Bell Atlantic Verge Decl. at para. 10 (stating that BAGNI will operate, install, and maintain its own network either directly or by contracting with unaffiliated third parties). 47 C.F.R.  53.203(a)-203(c); see Non-Accounting Safeguards Order, 11 FCC Rcd at 21981-82; see also Second BellSouth Louisiana Order, 13 FCC Rcd at 20787. AT&T Comments at 67-68; AT&T Kargoll Aff. at paras. 24-26 (submitting Bell Atlantic Internet disclosures). Bell Atlantic Reply at 43-44. See Letter from Gerald Asch, Director, Federal Regulatory Affairs, Bell Atlantic Corp., to Anthony Dale, Attorney, Federal Communications Commission, CC Docket No. 99-295 (filed Oct. 19, 1999) (Bell Atlantic Oct.19 Ex Parte Letter). See AT&T Kargoll Aff. Attach. 2; Bell Atlantic Reply Decl. at paras. 5-7. 47 U.S.C.  272(b)(2); 47 C.F.R. 53.203(b); Accounting Safeguards Order, 11 FCC Rcd at 17617-18; Second BellSouth Louisiana Order, 13 FCC Rcd at 20786-89; see Bell Atlantic Breen Decl. at para. 6 & Attach. E (submitting corporate accounting policy); Bell Atlantic Verge Decl. at para. 6 & Attach. D. 47 U.S.C.  272(b)(3); 47 C.F.R.  53.203(c); Ameritech Michigan Order, 12 FCC Rcd at 20730-31; Second BellSouth Louisiana Order, 13 FCC Rcd at 20789-90; see Bell Atlantic Browning Decl. at paras. 3(a), 3(b) (stating that Bell Atlantic compared payroll registers of the section 272 affiliates to the records for the operating telephone companies); Bell Atlantic Breen Decl. at para. 5, Attach. B (presenting list of corporate directors), C (presenting list of corporate officers); Bell Atlantic Verge Decl. at para. 5, Attach. B & C. 47 U.S.C.  272(b)(4); 47 C.F.R.  53.203(d); Non-Accounting Safeguards Order at paras. 189-90; see Bell Atlantic Application at 50; Bell Atlantic Browning Decl. at para. 11; Bell Atlantic Breen Decl. at paras. 7-8, Attach. F (submitting support agreement between holding company and nonregulated lending affiliate), G (submitting promissory note for BACI), H (submitting promissory note for NLD); Bell Atlantic Verge Decl. at 7, Attach. E (submitting promissory note for BAGNI). The Commission has rejected section 271 applications in part because BOCs failed to disclose fully all transactions in a manner consistent with section 272(b)(5) and the Commission's rules. See Ameritech Michigan Order, 12 FCC Rcd at 20734-37; Second BellSouth Louisiana Order, 13 FCC Rcd 20791-92. 47 U.S.C.  272(b)(5); 47 C.F.R.  53.203(e). Section 272(b)(5) states that the section 272 affiliate "shall conduct all transactions with the [BOC] of which it is an affiliate on an arm's length basis with any such transactions reduced to writing and available for public inspection." 47 U.S.C.  272(b)(5) (emphasis added). Accounting Safeguards Order, 11 FCC Rcd at 17593-94; Ameritech Michigan Order, 12 FCC Rcd at 20734- 37; Second BellSouth Louisiana Order, 13 FCC Rcd at 20790-95. 47 C.F.R.  32.27; Accounting Safeguards Order, 11 FCC Rcd at 17582-17; see Second BellSouth Louisiana Order, 13 FCC Rcd at 20790-95. The Commission's affiliate transactions rules require BOCs to report transactions between regulated and nonregulated affiliates, and to value the cost of affiliate transactions in accordance with a hierarchy of valuation techniques. Bell Atlantic Browning Decl. Attach. L; see Second BellSouth Louisiana Order, 13 FCC Rcd at 20791-92. In their Automated Reporting Management Information System ("ARMIS") reports, the BOCs provide summary information about their transactions with nonregulated affiliates. See ARMIS 43-02 USOA Report, Tables I-2, B- 4. In their CAMs, the BOCs disclose the nature, terms, and frequency of their anticipated affiliate transactions. See 47 C.F.R.  64.903; see also Bell Atlantic Corp., COST ALLOCATION MANUAL  V (Dec. 1998). Pursuant to the Commission's Part 64 rules, all the BOCs receive annual audits of their ARMIS data conducted by an independent auditor. 47 C.F.R.  64.904. In addition, the Commission regularly reviews the CAMs and the audit materials related to the independent audits, which show the actual amount of affiliate transactions that occurred in the audited period. AT&T Kargoll Aff. at paras. 32-51; AT&T Nov. 8 Ex Parte Letter at 1-4. See AT&T Comments at 69-70; AT&T Reply at 47-48. But see Bell Atlantic Reply at 44. See Bell Atlantic Breen Decl. at para. 14 (citing and www.callbell.com/regreqs2/index.htm), Attach. I; Bell Atlantic Verge Decl. at paras 14-15 (citing ). The working papers of Bell Atlantic's independent auditors show that, in 1998, two Bell Atlantic BOCs provided approximately $96,000 worth of data services and $37,790 in voice messaging services to BACI; approximately $69,000 in property management services to BAGNI; and approximately $18,000 in real estate services to NLD. See, e.g., Bell Atlantic Corp., COST ALLOCATION MANUAL at App. V-1 (Dec. 1998) (identifying services provided by a Bell Atlantic BOC to its section 272 affiliates). See Bell Atlantic Oct. 19 Ex Parte Letter. See Bell Atlantic Browning Decl. Attach. L (explaining potential differences in dollar values of posted transactions); Browning Reply Decl. at 8-12, 14; see also Bell Atlantic Oct. 19 Ex Parte Letter. But seeAT&T Reply at 47-48 (criticizing Bell Atlantic's explanations); Letter from Robert W. Quinn, Director - Federal Government Affairs, AT&T Corp., to Magalie Roman Salas, Secretary, Federal Communications Commission (filed Nov. 8, 1999) (AT&T Nov. 8 Ex Parte Letter). The total value of the discrepancies between Bell Atlantic's Internet disclosures and its other accounting information amounts to approximately $220,000. When compared to the total volume of affiliate transactions for all three affiliates combined, the discrepancies amount to less than one percent of the total dollar value. By comparison, in the Second BellSouth Louisiana Order, we found approximately $610,000 worth of discrepancies between the BOC's Internet postings and its ARMIS data, which amounted to 7.3 percent of the total dollar value of transactions. Second BellSouth Louisiana Order, 13 FCC Rcd at 20792 n.1046. In the Second BellSouth Louisiana Order, BellSouth failed to provide explanations regarding its discrepancies, while Bell Atlantic presented explanations in the instant proceeding. See Bell Atlantic October 19 Ex Parte Letter at 1-2 & Attach. AT&T Kargoll Aff. at paras. 34-40. Second BellSouth Louisiana Order, 13 FCC Rcd at 20793-94; see Accounting Safeguards Order, 11 FCC Rcd at 17593-94; Ameritech Michigan Order, 12 FCC Rcd at 20735. According to its Internet postings, its CAM, and its ARMIS data, Bell Atlantic did not transfer any assets from a BOC to its section 272 affiliates in 1998. See, e.g., BACI Technical Services Agreement - New York , which is located on BACI's Internet site at: . See Bell Atlantic Breen Decl. Attach. I; Bell Atlantic Verge Decl. Attach. F; see also Bell Atlantic Browning Reply Decl. at paras. 8-12, 16. See 47 U.S.C.  272(d) (requiring a joint Federal-State audit of section 272 compliance conducted by an independent auditor). Second BellSouth Louisiana Order, 13 FCC Rcd at 20794-95; Accounting Safeguards Order, 11 FCC Rcd at 17592; 47 C.F.R.  32.27; see Bell Atlantic Browning Decl. at paras. 22-25 & Attach. K-S (presenting various corporate policies and standard operating procedures pertaining to affiliate transactions compliance); Bell Atlantic Breen Decl. at paras. 14-17; Bell Atlantic Verge Decl. at paras. 14-19. 47 U.S.C.  272(c)(2); see Bell Atlantic Browning Decl. at paras. 22-26 & Attach. K (submitting reports of independent auditors), P (presenting employee training materials related to affiliate transaction compliance). 47 C.F.R.  32.27; Accounting Safeguards Order, 11 FCC Rcd at 17586-87; Second BellSouth Louisiana Order, 13 FCC Rcd at 20785-86. 47 U.S.C.  272(d); see 47 C.F.R.  53.209-213; see also Second BellSouth Louisiana Order, 13 FCC Rcd at 20794; see Bell Atlantic Application at 52; Bell Atlantic Browning Decl. at para. 27 & Attach. P at 36-40 (describing internal controls related to the biennial audit). AARP Comments at 1; Closecall Comments at 8 (raising concerns about affiliate structure); ALTS Comments at 72; see also AT&T Reply at 47 (arguing that Bell Atlantic cannot evade its section 272 obligations by chaining transactions through its affiliates); AT&T Nov. 8 Ex Parte Letter at 2 (addressing risk of chain transactions). 47 U.S.C.  272(c)(1); Non-Accounting Safeguards Order, 11 FCC Rcd at 21997-17; Second BellSouth Louisiana Order, 13 FCC Rcd 20796-800. The Commission found that the nondiscrimination safeguards extend to any good, service, facility, or information that a BOC provides to its section 272 affiliate, including administrative services and other non-telecommunications goods and services. Non-Accounting Safeguards Order, 11 FCC Rcd at 22003-04. Non-Accounting Safeguards Order, 11 FCC Rcd at 22000-01. ALTS Comments at 69-72; AT&T at 71-73; CERB at 2, 10; Letter from Kristine DeBry, Swidler Berlin Shereff Friedman, LLP, Counsel for CERB, to Magalie Roman Salas, Secretary, Federal Communications Commission, CC Docket 99-295 (filed Nov. 8, 1999) (CERB Nov. 8 Ex Parte Letter). AT&T Comments at 71-72. Bell Atlantic Reply at 45; see AT&T Comments at 71-73. 47 U.S.C.  272(e)(1); Non-Accounting Safeguards Order, 11 FCC Rcd at 22018-22; Second BellSouth Louisiana Order, 13 FCC Rcd at 20800-01; see Bell Atlantic Application at 52-53; Bell Atlantic Browning Decl. at para. 17(d) (citing Bell Atlantic FCC Tariff No. 1, Bell Atlantic FCC Tariff No. 11, NYPSC Tariff No. 918, NYPSC Tariff No. 900). Bell Atlantic demonstrates that it will provide accurate data regarding actual service intervals so that unaffiliated parties can evaluate the performance Bell Atlantic provides itself and its affiliates and compare such performance to the service quality Bell Atlantic provides to competing carriers. Bell Atlantic Browning Decl. at para. 17(e), Attach J; see id. at para. 18(a) (showing data that can be used to evaluate whether Bell Atlantic meets its nondiscrimination obligations). Bell Atlantic likewise addresses the accounting requirements of section 272(e) in its application. See Bell Atlantic Browning Decl. at para. 19(a) (addressing accounting for amounts charged for access to telephone exchange and exchange access); Bell Atlantic Breen Decl. at paras. 14, 16; Bell Atlantic Verge Decl. at paras. 17-18. 47 U.S.C.  272(e)(2). 47 U.S.C.  272(e)(3), (e)(4); Second BellSouth Louisiana Order, 13 FCC Rcd at 20802-03; see Bell Atlantic Application at 53; Bell Atlantic Application at 53; Bell Atlantic Browning Decl. at paras. 19(a), 20; Bell Atlantic Breen Decl. at paras. 14-16; Bell Atlantic Verge Decl. at paras. 17-18. Bell Atlantic Application at 54; Bell Atlantic Browning Decl. at para. 21, Attachment P, 21, 27 (submitting portions of employee training materials); see also Bell Atlantic Verge Decl. at paras. 20-26 (describing corporate compliance program); Bell Atlantic Breen Decl. at para. 15; Bell Atlantic Reply at 46-47. AT&T Comments at 73-77; AT&T Reply at 48-49; BellSouth South Carolina Order, 13 FCC Rcd at 668. Bell Atlantic Verge Decl. at paras. 21-26. See id. at para. 2. Excel Comments at 6-13. Id. at 7. Non-Accounting Safeguards Third Order on Reconsideration, FCC 99-242, paras. 22-24; see also Non- Accounting Safeguards Order, 11 FCC Rcd at 22055-56. 47 U.S.C.  272(g)(2); Second BellSouth Louisiana Order, 13 FCC Rcd at 20804; Bell Atlantic Application at 54; Bell Atlantic Browning Decl. at para. 21 & Attach. P at 21, 27 (submitting portions of employee training materials); see also Verge Decl. at paras. 20-26 (describing corporate compliance program); Bell Atlantic Breen Decl. at para. 15; Bell Atlantic Reply at 46-47. 47 U.S.C.  271(d)(3)(C). The Commission has offered direction for the benefit of section 271 applicants relating to the meaning and scope of the public interest inquiry. See generally Second BellSouth Louisiana Order, 13 FCC Rcd at 20805-08; Ameritech Michigan Order, 12 FCC Rcd at 20741-51. In addition, Congress specifically rejected an amendment that would have stipulated that full implementation of the checklist necessarily satisfies the public interest criterion. See Ameritech Michigan Order, 12 FCC Rcd at 20747; see also 141 Cong. Rec. S7971, S8043 (Jun. 8, 1995). See Second BellSouth Louisiana Order, 13 FCC Rcd at 20805-06 (the public interest analysis may include consideration of "whether approval . . . will foster competition in all relevant telecommunications markets"). 47 U.S.C.  271(d)(4). 47 U.S.C.  160(d). See, e.g., AT&T Comments at 78-84, 94-100; MCI WorldCom Comments at 43-45; CPI Comments at 5-19. See AT&T Comments at 79-80; AT&T Kelley Aff. at paras. 2-3, 14-33; AT&T Hubbard/Lehr Aff. at para 54 and Attach. 13; CPI Comments at 10-16; KMC Comments at 11; MCI WorldCom Comments at 44; NY Attorney General Comments at 8. While Bell Atlantic has offered evidence that it has lost large numbers of access lines to competitors, we recognize that competition may be slender as a percentage of access lines controlled by Bell Atlantic, particularly outside of urban areas. See AT&T Comments at 79-80; KMC Comments at 10; MCI WorldCom Comments at 44; MCI WorldCom Beard/Mayo Decl. at paras. 35-41. See ALTS Comments at 68; CPI Comments at 3-5, 10-20; CPI Reply at 2-3; KMC Comments at 11; NY Attorney General Comments at 7-9; TRA Comments at 28-29. See AT&T Kelley Decl. at paras. 24-32; MCI WorldCom Comments at 44; MCI WorldCom Beard/Mayo Decl. at para. 37; see also Department of Justice Evaluation at 10 ("[g]iven the extent of facilities-based entry in metropolitan New York and other cities in upstate New York, we have no substantial concerns about the ability of facilities-based carriers to enter the market"). See AT&T Comments at 80-81; AT&T Bernheim/Ordover/Willig Aff. at paras. 35-36; AT&T Hubbard/Lehr Aff. at paras. 57-64. This is consistent with the Commission's approach in prior section 271 orders. See Ameritech Michigan Order, 12 FCC Rcd at 20585; see also BellSouth Reply at 19. For similar reasons, we decline to require Bell Atlantic to demonstrate, as urged by CPI, that all end users in New York have a "realistic choice" between facilities-based local carriers. See CPI Comments at 10-20. See generally AT&T Hubbard/Lehr Aff. at paras. 28-136; AT&T Bernheim/Ordover/Willig Aff. at paras. 99-171; AT&T Selwyn Aff. at paras. 4-35; MCI WorldCom Beard/Mayo Decl., Attach. 3; Bell Atlantic Taylor Decl. at paras. 1-78; Bell Atlantic MacAvoy Decl. at paras. 1-122. See Second BellSouth Louisiana Order, 13 FCC Rcd at 20806; see Ameritech Michigan Order, 12 FCC Rcd at 20747. See Department of Justice Evaluation at 7, Schwartz Aff. at paras. 149-192. These mechanisms are generally administered by state commissions and derive from authority the states have under state law or under the federal Act. As such, these mechanisms can serve as critical complements to the Commission's authority to preserve checklist compliance pursuant to section 271(d)(6). Moreover, in this instance, we find that the extensive collaborative process by which these mechanisms were developed and modified in New York has, itself, helped to bring Bell Atlantic into checklist compliance. See Second BellSouth Louisiana Order, 13 FCC Rcd at 20806. See infra Section VIII. See Bell Atlantic Application at 71; Bell Atlantic Dowell/Canny Decl. at paras. 8, 125, and Attach. A; AT&T Comments at 94 (recognizing that 32 of Bell Atlantic's 85 interconnection agreements contain liquidated damages provisions). See Bell Atlantic Application at 71 (recognizing that competitive carriers could seek "private remedies under generally applicable statutes, including the treble-damages remedy of the federal antitrust laws"). See NYPSC Guidelines Order; see also NYPSC Permanent Rule Order. The New York Commission explained that, wherever possible, it established "parity" standards (a performance level which is the same for competitors as it is for Bell Atlantic's retail operations). See NYPSC Guidelines Order at 2. For wholesale functions that do not have retail analogues, the New York Commission established absolute standards, usually a fixed percentage or a fixed period of time. Id. Although the enforcement plans were formally adopted by the New York Commission on November 3, 1999, see Order Adopting the Amended Performance Assurance Plan and Amended Change Control Plan, Case Nos. 97-C-0271 and 99-C-0949 at 32 (NYPSC Nov. 3, 1999) (Bell Atlantic Dowell/Canny Reply Decl., Att. A) (NYPSC Enforcement Plan Order), we disagree with commenters who suggest that, consistent with our policy of requiring that applications be final when filed, we may not consider these plans in our public interest analysis. See, e.g., CoreComm Comments at 10; Sprint Comments at 24; AT&T Motion to Strike at 7; see also NY Attorney General Comments at 36. These plans were developed through a 16 month process in New York and were submitted to the New York Commission for adoption on September 24, 1999. We take administrative notice of the fact that the plans were adopted virtually unchanged by the New York Commission. See AT&T Reply Comments at 38. What is critical to our analysis is that the plans were described in detail in Bell Atlantic's initial application, and have been subject to extensive comment in this proceeding. Because this aspect of our public interest inquiry necessarily is forward-looking and requires a predictive judgment, this is a situation where it is appropriate to consider commitments made by the applicant to be subject to a framework in the future. Accordingly, this is different from our checklist analysis in which we assess present or past compliance by an applicant. See Bell Atlantic Application at 67-71; Bell Atlantic Dowell/Canny Decl., App. A, Vol. 3, Attach. C (Petition for Approval of the Amended Performance Assurance Plan and Amended Change Control Assurance Plan for Bell Atlantic-New York, NYPSC Cases 97-C-0271 and 99-C-0949 (Sept. 24, 1999)); see also NYPSC Enforcement Plan Order at 3-6; New York Commission Comments at 164-172. As is clear from our discussion of the checklist requirements, we do not base our decision that the checklist has been satisfied on the existence of the New York performance plans. We thus approach our analysis of the New York performance monitoring and enforcement mechanisms from a different angle than the Department of Justice. While we conclude that the checklist has been met, and assess the predicted impact of these monitoring and enforcement mechanisms on Bell Atlantic's ability to maintain compliance with section 271, the Department of Justice has assessed whether these mechanisms will be sufficient to "ensure the rapid completion of necessary market-opening measures." Department of Justice Evaluation at 37 (emphasis added), and Schwartz Aff. at paras. 137-140. Our examination of the New York performance monitoring and enforcement mechanisms is solely for the purpose of determining whether the risk of post-approval non-compliance is sufficiently great that approval of its section 271 application would not be in the public interest. Our analysis has no bearing on the separate question of how the Commission would view and respond to any particular conduct by Bell Atlantic in the federal enforcement context. Several parties also urge us to adopt, in the context of this section 271 application, automatic "federal" remedies, in addition to those developed in New York. See Allegiance Comments at 14-17; ALTS Comments at 79; AT&T Reply at 39; Comptel Comments at 47-57; e.spire/Net2000 Comments at 24-25; MCI WorldCom Reply at 30; MediaOne Reply at 17. As discussed more fully below, see infra Section VIII, we fully intend to enforce the provisions of section 271 using the enforcement tools set forth in the Communications Act. See NYPSC Enforcement Plan Order at 2, 17; Bell Atlantic Application at 69. We reach this number by adding the following components: $75 million (MOE); $75 million (MOE "doubling" provisions); $75 million (Critical Measures); $34 million (Special Provisions); and $10 million (ACCAP). See AT&T Comments at 87-88; ALTS Comments at 79; ChoiceOne Comments at 12; CoreComm Comments at 10-11; CPI Comments at 22-23; Focal Comments at 8; KMC Comments at 12-13; MCI WorldCom Comments at 39-40; NY Attorney General Comments at 30-32. Several parties also argue that any cap or total limit on liability unnecessarily weakens an enforcement mechanism. See, e.g., ALTS Reply at 26-27; e.spire/Net2000 Comments at 23; Intermedia Comments at 15. See MCI WorldCom Comments at 39-40; MCI WorldCom Ford/Jackson Decl. at 14 (arguing that the APAP must entail liability "equal to or greater than the benefits that BA-NY would receive over time from providing such poor performance," which MCI WorldCom claims would exceed $600 million per year); NY Attorney General Comments at 31 ("in order to effectively deter certain conduct, sanctions should be much larger than the cost to comply," which it calculates at $495 million per year); Cable & Wireless Comments at 16. MCI WorldCom submits a detailed economic study, in which it seeks to calculate with precision the hypothetical benefits Bell Atlantic would derive from certain levels of discrimination, with the purpose of identifying a corresponding "optimal" penalty amount. The New York Commission concluded that a similar study submitted by MCI WorldCom in New York was "flawed" (NYPSC Enforcement Plan Order at 18) and, in this proceeding, Bell Atlantic challenges MCI WorldCom's assumptions and methodology. See Bell Atlantic Duncan Reply Decl., Attach. A. Because we do not find it necessary to determine the "optimal" penalty amount for a stand-alone enforcement mechanism, we will not specifically address the details of MCI WorldCom's study, the "flaws" identified by the New York Commission, or Bell Atlantic's counterarguments. See MCI WorldCom Ford/Jackson Decl. at paras 22, 49 (suggesting that local service profits represents a meaningful frame of reference in this analysis); see also CPI Comments at 22-23; NY Attorney General Comments at 30-31. While we are using net local revenue as a reference point or yardstick for comparison purposes, we do not suggest that local revenues constitute the only relevant figure. We recognize that Bell Atlantic may also derive benefits in other markets (such as long distance) from retaining local market share. See New York Commission Reply, Ex. 7 at 2, n.1. To arrive at a total "Net Return" figure that reflects both interstate and intrastate portions of revenue derived from local exchange service, we combined line 1915 (the interstate "Net Return" line) with a computed net intrastate return number (total intrastate operating revenues and other operating income, less operating expenses, nonoperating items and all taxes). See ARMIS 43-01 Annual Summary Report, Table 1, Cost and Revenue Table (1998). NYPSC Enforcement Plan Order at 18, 32. The New York Commission, in its Evaluation, also expressed its "confiden[ce] that Bell Atlantic-NY, once having earned section 271 approval, has the proper incentive to continue to meet its commitments." New York Commission Comments at 172. See AT&T Comments at 89; Cable & Wireless Comments at 16; CoreComm Comments at 11; MCI WorldCom Comments at 40-42; Sprint Comments at 26. We also disagree with Sprint and find that the amount at stake under the ACCAP ($10 million, plus up to $15 million in penalties "unused" by the APAP) provides reasonably sizable incentive for Bell Atlantic to adhere to change management procedures developed in New York. See Sprint Comments at 31. New York Commission Comments at 166; see also NYPSC Enforcement Plan Order at 32 (commenting that this reallocation power "allows the Commission flexibility to ensure that potential loopholes may be closed rapidly and pointedly"). Second BellSouth Louisiana Order, 13 FCC Rcd at 20806. See AT&T Comments at 47-48; Choice One Comments at 5; AT&T Pfau/Kalb Decl. at paras 53-62; AT&T Reply at 31; CPI Reply at 13. The New York Commission has explained that questions have arisen about certain performance measurements, and that several of these are currently under further review in the Carrier-to-Carrier proceeding. See New York Commission Comments at 7 n.2 and 46 n.1; NYPSC Enforcement Plan Order at 25-26, 30 and 39 n.4. See also Department of Justice Evaluation at 6 (recognizing that the New York Commission "is continuing its efforts to refine [certain] performance measures"). The New York Commission adopted interim guidelines for inter-carrier service quality on March 16, 1998 and, in conjunction with a collaborative process involving working groups and subject-area sub-groups, has reviewed and modified these guidelines on an ongoing basis since that time. See NYPSC Guidelines Order at 1-2; NYPSC Permanent Rule Order at 1-4; NYPSC Additional Guidelines Order at 1-2. Moreover, the New York Commission has stated that it "fully expect[s] that metrics will continue to be developed and refined." See New York Commission Reply at 4. The New York Commission concluded that the reporting requirements "are comprehensive and will help fulfill our goal of achieving expeditiously an open, competitive local exchange market." NYPSC Permanent Rule Order at 3. See NYPSC Enforcement Plan Order at 14-15. In particular, we applaud the New York Commission and Bell Atlantic for addressing the very important issue of change management by designing metrics that measure Bell Atlantic's compliance with its change management processes and give the company incentives to satisfy performance standards in this area. AT&T Comments at 91; MCI WorldCom Comments at 43; AT&T Kalb/Pfau Aff. at paras. 205-206 (arguing that every Carrier-to-Carrier metric must have a penalty attached); see also Focal Comments at 7 (the mechanisms fail to address metrics relating to special access services); Sprint Comments at 30-31 (additional metrics should be added to the change control plan). See NYPSC Enforcement Plan Order at 14-15; see also New York Commission Comments at 165. See NYPSC Enforcement Plan Order at 15 (explaining that "[o]nce the [Performance Assurance Plan] is in effect, market conditions will be examined to determine whether metrics should be added or deleted"). The New York Commission also may add metrics to the ACCAP. See Bell Atlantic Dowell/Canny Decl., Attach. C, Ex. 2 at 3 (Amended Change Control Assurance Plan, September 1999). See ALTS Comments at 37-38; AT&T Reply at 31; @link Comments at 6; Covad Comments at 33-34. See New York Commission Comments at 94-95 ("[r]ecommendations to the NYPSC are expected in December for the adoption of DSL-specific metrics to ensure that [DSL services] can be separately monitored to ensure provisioning at a commercially reasonable level of quality and timeliness"); see also New York Commission Reply Comments, Ex. 7 at 4, n.2. See Bell Atlantic Dowell/Canny Decl. at paras. 122-157 and Attach. C (Petition for Approval of Amended Performance Assurance Plan); see also New York Commission Comments, Appendix 1. See ALTS Comments at 78 (arguing that the "forgiveness" provision of the Plan would allow Bell Atlantic to "hide discriminatory practices"); AT&T Comments at 92-93; Intermedia Comments at 16; KMC Comments at 12; AT&T Kalb/Pfau Aff. at para. 214. See MCI WorldCom Comments at 40; Intermedia Comments at 16; AT&T Pfau/Kalb Decl. at para 209. See ALTS Comments at 78 (suggesting that aggregating measures together would result in "offset[ting] poor performance in one performance category with good performance in another category"); AT&T Comments at 92; KMC Comments at 12; MCI WorldCom Comments at 41-42. See MCI WorldCom Comments at 42; MCI WorldCom Ford/Jackson Aff. at para. 67; AT&T Kalb/Pfau Aff. at para. 217. See NYPSC Enforcement Plan Order at 12-14; see also New York Commission Reply, Ex. 7 at 3-6. See NYPSC Enforcement Plan Order at 11-12. We also note with approval that the APAP "will be enforceable as a New York Commission order," and that failure by Bell Atlantic to comply with the terms of these mechanisms could subject the company to penalties in the amount of $100,000 per day. See New York Commission Comments at 165, n.1. Complaints alleging that Bell Atlantic is not complying with these state- crafted mechanisms thus would be directed to the New York Commission rather than the FCC. See Department of Justice Evaluation at 39-40; Sprint Comments at 30; NY Attorney General Comments at 33-34; e.spire/Net2000 Comments at 23. The Plan allows Bell Atlantic to seek a waiver on grounds of "unusual" or "inappropriate" CLEC behavior, listing a handful of examples. We find this category to be vague, and note that it could be used to challenge a very wide range of data. We note, however, that the New York Commission has stated that "waiver relief is intended for limited, extraordinary circumstances," see NYPSC Enforcement Plan Order at 24, and thus we expect that this exception will not be applied expansively. NYPSC Enforcement Plan Order at 24. In its order adopting the APAP, the New York Commission explained that "resolution of a waiver exception request must occur prior to the scheduled payment period." NYPSC Enforcement Plan Order at 24. We understand this to mean that waiver petitions will be resolved expeditiously, such that bill credits due for poor performance in a given month will never be "stayed" by a waiver petition. This interpretation is consistent with the sample waiver processing timeline contained in Bell Atlantic's petition requesting NYPSC approval of the APAP. See Bell Atlantic 271 Application, Attach. C, Ex. 1 (Amended Performance Assurance Plan, Appendix D at 5) (showing a hypothetical waiver petition being resolved before bill credits for a given month are due). See New York Commission Comments at 12, 169 n.1. See id; Bell Atlantic Dowell/Canny Decl., Attach. C, Ex. 1 at 17-19. Bell Atlantic has also committed to implement a Quality Assurance Program (more accurately, an "Accuracy Assurance Program") under which it will document and verify its data in an open, reviewable manner and provide an internal mechanism for investigating and resolving CLEC disputes about the accuracy of reported data. See id. at 15. MCI WorldCom has commented that this replication commitment is "extremely valuable in enabling CLECs to ensure that metrics are being reported as intended . . . after long distance entry by [Bell Atlantic]." MCI WorldCom Kinard Decl. at 3. AT&T, however, argues that this replication is incomplete. See AT&T Pfau/Kalb Aff. at para. 184. See Accounting for Judgments and Other Costs Associated with Litigation, 12 FCC Rcd 5112 (1997); 47 C.F.R.  32.7370(d). As a general matter, a carrier's operating expenses recovered through its rates must be legitimate costs of providing adequate service to ratepayers. See, e.g., West Ohio Gas Co. V. PUC, 294 U.S. 63, 74 (1935); Mountain States Tel. and Tel. Co. v. FCC, 939 F.2d 1035, 1044 (D.C. Cir. 1991). Under the SBC/Ameritech merger, the Commission held that bill credits "shall not be reflected in the revenue requirement of an SBC/Ameritech incumbent LEC." See Applications of Ameritech Corp. and SBC Communications Inc., CC Docket No. 98-141, Memorandum Opinion and Order, Appendix C at para. 34 (rel. Oct. 8, 1999). See CPI Comments at 24. NYPSC Enforcement Plan Order at 31 ("[Bell Atlantic] will be specifically prohibited from recovering revenue losses attributable to the remedial performance credits given in connection with the [penalty plans]"). For example, several commenters suggest that Bell Atlantic has engaged in unfair and dilatory tactics in interconnection negotiations. See ICG Comments at 2-7; Ntegrity Comments at 2; Z-Tel Comments at 22. See also Global NAPS Comments at 2-5 (asserting that Bell Atlantic's conduct in resolving ongoing disputes concerning inter-carrier compensation for ISP-bound calls is anticompetitive); but see Complaint of Bell Atlantic- Delaware, et al. v. Global NAPs, Inc., File No. E-99-22, Memorandum Opinion and Order, FCC 99-381(rel. Dec. 2, 1999) (concluding that challenged sections of a Global NAPs tariff in Massachusetts are unlawful, based on the fact that the Massachusetts Department of Telecommunications and Energy has yet to resolve whether and how the parties' existing interconnection agreement provides for inter-carrier compensation for ISP-bound traffic). See Ameritech Michigan Order, 12 FCC Rcd at 20749. We emphasize that grant of this application does not reflect any conclusion that Bell Atlantic's conduct in the individual instances cited by commenters is nondiscriminatory and complies with the company's obligations under the Communications Act. See City of New York Comments at 2-4. AT&T Comments at 65-67. See Petition of Bell Atlantic for Forbearance from Section 272 Requirements in Connection with National Directory Assistance Services, Memorandum Opinion and Order, CC Docket No. 97-172, DA 99-2990 (rel. Dec. 22, 1999). This determination does not remove the possibility of future enforcement action to the extent that Bell Atlantic may have failed to comply with the Act. Petition of U S WEST Communications, Inc. for a Declaratory Ruling Regarding the Provision of National Directory Assistance, CC Docket No. 97-172, Memorandum Opinion & Order, FCC 99-133 (rel. Sept. 27, 1999), recon. pending (U S WEST Forbearance Order). U S WEST v. FCC, 177 F.3d 1057, 1060 (D.C. Cir. 1999). As the Department of Justice has observed, section 271 serves a critical market-opening role by "ensuring the BOC has powerful incentives (i.e., the ability to enter the long distance market) to cooperate to open its markets." Department of Justice Evaluation at 38. See, e.g., ALTS Comments at 74-79; AT&T Comments at 81-94; Cable & Wireless Comments at 12-14; CPI Comments at 20-23; CompTel Comments at 27-34; MCI WorldCom Comments at 36-37; Sprint Comments at 23-31; NY Attorney General Comments at 27-36. See also Department of Justice Evaluation at 36-40. Of course, this statutory framework would apply whenever a BOC receives section 271 authorization for a particular state. 47 U.S.C.  271(d)(6)(A). Specifically, the Commission may impose monetary forfeitures pursuant to Title V by issuing a written notice of apparent liability for forfeiture and providing the subject an opportunity to respond in writing. 47 U.S.C.  503(b)(4). 47 U.S.C.  271(d)(6)(A). Implementation of the Non-Accounting Safeguards of Sections 271 and 272 of the Communications Act of 1934, as amended, CC Docket No. 96-149, First Report and Order and Further Notice of Proposed Rulemaking, 11 FCC Rcd 21905, 22066 (1996) (Non-Accounting Safeguards Order). See Bell Atlantic Application at 71 ("Any anticompetitive conduct is unthinkable in light of this Commission's powers under section 271(d)(6)(A). That provision allows the Commission to enforce the requirements of section 271 with penalties, up to and including possible revocation of long distance authority."); see also Bell Atlantic Reply at 60. See AT&T Corp. v. Ameritech Corp., File No. E-98-41, Memorandum Opinion and Order, 13 FCC Rcd 14508 (1998) (Ameritech Stand-Still Order) (stand-still order issued pursuant to 47 U.S.C.  154(i) temporarily preventing Ameritech from enrolling additional customers in, and marketing and promoting, a "teaming" arrangement with Qwest Corporation pending a decision concerning the lawfulness of the program); see also United States v. Southwestern Cable Co., 392 U.S. 157 (1968) (affirming Commission's authority to impose a stand-still order pursuant to 47 U.S.C.  154(i)). Service to existing interLATA subscribers would not be interrupted. See Ameritech Standstill Order. See Non-Accounting Safeguards Order, 11 FCC Rcd at 22077. 5 U.S.C.  553 and 554 (emphasis added). See AT&T v. FCC, 572 F.2d 17, 22-23 (2nd Cir.) (where statute does not require hearing "on the record," APA does not require trial-type evidentiary hearing), cert. denied, 439 U.S. 875 (1978); United States v. Florida East Coast Railway Co., 410 U.S. 224, 234-38 (1973). For example, the 90-day deadline in section 271(d)(6)(B) for resolving complaints concerning failures by a BOC to meet conditions required for approval suggests that Congress did not intend to afford BOCs trial-type hearings in all post-approval enforcement proceedings. See Non-Accounting Safeguards Order, 11 FCC Rcd at 22077. 47 U.S.C.  271(d)(6)(B ); 47 C.F.R.  1.736; Implementation of the Telecommunications Act of 1996, Amendment of Rules Governing Procedures to Be Followed When Formal Complaints Are Filed Against Common Carriers, CC Docket No. 96-238, Report and Order, 12 FCC Rcd 22497, 22610-12 (1997). Non-Accounting Safeguards Order, 11 FCC Rcd at 22066. See supra para 441. Statistical testing can be used, but is not necessary, for metrics using benchmarks. It would be unreasonable to expect a particular performance metric to always show ex post equal or better performance for service to a requesting carrier, compared to that provided to the incumbent LEC's customers. Such a requirement, if implemented, would demand that the incumbent LEC provide ex ante superior service to a requesting carrier, in order to ensure that random variation does not cause performance to the requesting carrier to drop accidently below the level needed for a determination of parity. Other methods of testing are possible, such as the use of Bayesian estimation techniques. We will not discuss those methods here. See John Neter, William Wasserman, and G.A. Whitmore, Applied Statistics at ch. 27-28 (4th ed., 1993). Researchers usually call the hypothesis they are trying to prove the alternative hypothesis. The null hypothesis is the hypothesis which they are trying to determine whether to reject. Ramakant Khazanie, Statistics in a World of Applications 495 (4th ed., 1997). Devising a statistical test usually involves creating a test statistic and then comparing it to some critical value. See Khazanie, supra n.4 at ch. 9. Statisticians would say that the observations are a sample taken from the population. The population is the theoretical set of values obtained if an infinite number of observations were taken of the underlying process. Therefore the population mean is the theoretical mean produced by the process, while the sample mean is the measured mean. Khazanie, supra n.4 at 5-6; Neter, Wasserman, and Whitmore, supra n.3 at 235-36, 248-49; Alexander Mood, Franklin Graybill and Duane Boes, Introduction to the Theory of Statistics 219-31 (3rd ed., 1974). Second BellSouth Louisiana Order, 13 FCC Rcd at 20659 and n.274. Bell Atlantic Dowell/Canny Decl. at para. 112, and Attach. B, App. K. Bell Atlantic Dowell/Canny Decl. Attach. B, App. K. The use of different formulas for statistical testing for measured and proportionate (counted) metrics is recommended in statistical textbooks. See Khazanie, supra n.4 at 538-48; Neter, Wasserman, and Whitmore, supra n.3 at ch. 14. To be consistent with textbook usage, we will refer to "counted" metrics as "proportionate." Any metric measuring average times is a measured metric. The sample mean, also called the average or the arithmetic mean, is defined as the sum of the observations, divided by the number of observations. Mathematically it is m = ä Xi / N, where Xi are the observations, and N is the number of observations. Khazanie, supra n.4 at 77- 79, 234-35; Neter, Wasserman, and Whitmore, supra n.3 at 71, 248-49. Proportionate metrics are generally said to have a binomial distribution. Neter, Wasserman, and Whitmore, supra n.3 at 363-65. Competitive LECs have suggested that there is a third kind of metric involved called rates. Rates are measures that involve the division of two numbers (for example, the trouble rate). Letter from Robert Quinn, Director-Federal Government Affairs, AT&T, to Magalie Roman Salas, Secretary, Federal Communications Commission, CC Docket No. 99-295 Attach. at 13 (Local Competition Users Group, Statistical Tests for Local Service Parity, version 1.0) (filed December 17, 1999) (LCUG Statistical Tests for Local Service Parity). In theory rates can exceed 1, unlike proportions. For example, more than one trouble could be reported for each line, so the trouble rate (which is the number of troubles divided by the number of lines) could be greater than one. Rates are classified as proportionate (counted) metrics by Bell Atlantic, and there are no special formulas for rates used in this application, so we will not discuss formulas for rates here. The z and t tests to be used for measured and proportionate variables were agreed upon by the Carrier-to- Carrier Group. They were proposed by LCUG and agreed to by Bell Atlantic. LCUG Statistical Tests for Local Service Parity; Bell Atlantic Dowell/Canny Decl. at para. 112, and Attach. B, App. K. Bell Atlantic Dowell/Canny Decl. Attach. B, App. K. It is also sometimes known as the "LCUG modified z- test." The only known published discussion of the modified z-test is in Cavell Brownie, Dennis D. Boos, and Jacqueline Hughes-Oliver, "Modifying the t and ANOVA F Tests When Treatment Is Expected to Increase Variability Relative to Controls," Biometrics 46, 259-66 (1990). AT&T Pfau/Kalb Aff., Attach. 2, "AT&T's Responses to FCC's Questions Dated April 12, 1999", at 3. Bell Atlantic Dowell/Canny Decl. Attach. B, App. K. The standard deviation is the square root of the variance. The sample variance is the sum of the squares of the differences between the mean and the observations, divided by the number of observations minus one. For measured metrics the sample variance is: s2?=?ä?(m-Xi)2?/?(N-1), where s2 is the sample variance, m is the sample mean, Xi are the observations, and N is the number of observations. For proportionate metrics the sample variance is: s2 = N * P * (1-P), where P is the proportion. Khazanie, supra n.4 at 257; Neter, Wasserman, and Whitmore, supra n.3 at 82-83, 363-71; Mood, Graybill, and Boes, supra n.7 at 229. Assuming the means have identical but unknown variances, and the sample size is large, the standard z-test for a difference in means between two populations, stated in terms of competitive LEC and incumbent LEC means, is z?= (mC?mI) / (sp * SQRT [1/NC+1/NI]), where mC = competitive LEC sample mean, mI = incumbent LEC sample mean, sp = pooled standard deviation (either uses the observations of both populations, or combines the standard deviations of the two populations), NC = number of competitive LEC observations, and NI = number of incumbent LEC observations. The test statistic is normally distributed, so the critical value is obtained from the standard normal distribution. If the sample size is small and the populations are normal, then the standard test is a t-test, using the same test statistic, but the test statistic has a t distribution with NI+NC-2 degrees of freedom. Khazanie, supra n.4 at 540-41, 563; Neter, Wasserman, and Whitmore, supra n.3 at 397-402. The normal and standard normal distributions, t-statistics, and critical values are discussed below at infra para. 9 and n.17, 26, 31. If the variances are assumed to be unknown and different, and the sample size is large, then the standard z-test uses the test statistic z?=?(mC?mI)?/?SQRT?[sC2/NC+sI2/NI]), where sC = competitive LECs' standard deviation and sI = incumbent LEC's standard deviation. The test statistic is normally distributed, and the standard normal distribution is used to determine the critical value. Khazanie, supra n.4 at 538-40, 563. If the sample size is small and the populations are normal, however, then the problem is known as the Behrens-Fisher problem (or the Behrens problem or the Fisher-Behrens problem), which is considerably more complicated to solve. Hamparsum Bozdogan and Donald E. Ramirez, "An Adjusted Likelihood-Ratio Approach to the Behrens-Fisher Problem," Communications in Statistics: Theory and Methods, 15 (8) at 2405 (1986); Brownie, Boos, and Hughes-Oliver, supra n.14 at 259-60. One solution is to use the Aspin-Welch test, using the same test statistic as for the large sample size test, but here the test statistic has a t distribution, with a complicated calculation of the degrees of freedom. Acheson J. Duncan, Quality Control and Industrial Statistics 616-617 (5th ed., 1986); William H. Beyer, CRC Standard Mathematical Tables 525 (26th ed., 1987). Bell Atlantic Dowell/Canny Decl. Attach. B, App. K. Bell Atlantic calls this a t-test. The t-test uses Student's t distribution to determine the critical value. For large sample sizes, this is approximately equivalent to doing a z- test, because for sample sizes of greater than 30 observations, the t distribution has approximately the same distribution as the standard normal distribution. With a t-test, the critical value varies according to the degrees of freedom, which depend on the sample size. Since Bell Atlantic is using a fixed critical value, it is effectively using a z-test. The formula for calculating the test statistic is effectively the same for both kinds of tests. Khazanie, supra n.4 at 410-413, 521; Neter, Wasserman, and Whitmore, supra n.3 at 913; see infra n.31. The formula for the test statistic can be more simply described as the difference in means divided by the standard error (Bell Atlantic calls the standard error the "sampling error" in the Carrier to Carrier metric reports), or (mC-mI) / S.E. Neter, Wasserman, and Whitmore, supra n.3 at 266, 290-91; LCUG Statistical Tests for Local Service Parity at 7- 8. Bell Atlantic Dowell/Canny Decl. Attach. B, App. K. We note that the standard z-test for a difference in proportions is z = (PC-PI) / SQRT [Pp(1-Pp) (1/NC+1/NI)], where PC = competitive LEC sample proportion, PI = incumbent LEC sample proportion, PP = pooled sample proportion, NC=number of competitive LEC observations, and NI=number of incumbent LEC observations. Khazanie, supra n.4 at 546-47, 563; Neter, Wasserman, and Whitmore, supra n.3 at 408-12. As noted above, the critical value for a z-test is taken from tables based on the standard normal distribution. See supra n.16. In New York the tests have been set up so that z-scores that indicate worse performance for competing carriers are negative. Thus if the critical value is -1.645, only z-scores that are less, such as -2 or -3, would yield statistically significant results. For example, if the confidence level C is 95 percent (0.95), then ? is 5 percent (0.05). While falsely rejecting the null hypothesis (i.e., falsely finding that the BOC's processes of serving retail and competitors' customers are different) is called a Type I error, falsely accepting the null hypothesis when it is not true (i.e., falsely finding that the BOC's processes are identical) is called a Type II error. The probabilities of a Type I error and a Type II error are commonly referred to by the Greek letters ? (alpha) and ? (beta), respectively. Khazanie, supra n.4 at 498; Neter, Wasserman, and Whitmore, supra n.3 at 319-20. Usually statisticians choose one hypothesis to be the null hypothesis because falsely rejecting it (Type I error) is considered more serious than falsely accepting it (Type II error), so controlling ? is more important than controlling ?. Khazanie, supra n.4 at 499, 506; Neter, Wasserman, and Whitmore, supra n.3 at 320; Mood, Graybill, and Boes, supra n.7 at 411. Statistical tests virtually never determine anything with certainty. There is always a certain probability of being wrong and choosing the incorrect hypothesis. Statistical tests are devised to minimize this probability of being wrong, i.e., to keep the probabilities of Type I and Type II errors at a minimum. The rationale for using a one-tailed test is described below. See infra para. 18. This means that, if a 95 percent confidence level is used for a statistical test, when the null hypothesis is true, 95 percent of the time we will correctly choose the null hypothesis. Meanwhile there will be a 5 percent chance that a statistical test will show a statistically significant difference. This is caused by random variation in the data. One way to interpret this is that out of every 100 measurements, on average five should show statistically significant differences, even with identical processes serving retail and competing LECs' customers. A normal distribution is sometimes referred to as a Gaussian distribution. It is often described as having a "bell-shaped" curve. A standard normal distribution is a normal distribution that has been transformed such that its mean is zero and standard deviation is one. Khazanie, supra n.4 at 281, 294-96. The Central Limit Theorem is a powerful theorem in statistics. It says that under most circumstances, the distribution of the mean will approach a normal distribution for a large enough sample size, even if the distribution of the population from which the mean is drawn is not normal. Khazanie, supra n.4 at 344-45; Neter, Wasserman, and Whitmore, supra n.3 at 267-68; Mood, Graybill, and Boes, supra n.7 at 233-36. Textbooks are vague about what the minimum sample size should be to use a large sample test like the z-test on measured metrics, but 30 is often cited as appropriate. Textbooks generally agree, however, that at 30 observations and greater, the t-test can be replaced by the z-test, for distributions that are approximately normal. See, e.g., Khazanie, supra n.4 at 413, 521, 539; Neter, Wasserman, and Whitmore, supra n.3 at 913; Duncan, supra n.16 at 150. See supra n.17. Doubts about whether a sample size of 30 is sufficient for measured metrics have been raised by AT&T in other proceedings. AT&T Pfau/Kalb Aff. Attach. 3, at 4-5. We note that KPMG used 100 as the threshold for using permutation testing in their test analysis of Bell Atlantic's metrics. KPMG Final Report at POP8 IV-176-77. The parties in this proceeding have agreed to use 30 as the minimum sample size for use of a z-test, and there is insufficient evidence in the record for us to reject this choice. Bell Atlantic Dowell/Canny Decl. Attach. B, App. K. The minimum sample size needed before a z-test should be used generally depends on the distribution of the underlying observations, and, in particular, how skewed it is. Neter, Wasserman, and Whitmore, supra n.3 at 296. Khazanie, supra n.4 at 262-64; Neter, Wasserman, and Whitmore, supra n.3 at 368-69. Bell Atlantic Dowell/Canny Decl. Attach. B, App. K. KPMG used a hypergeometric test (also known as Fisher's Exact Test) for its analysis when the number of observations is less than 10,000, for comparing two proportions. KPMG Final Report at POP8 IV-177. For a discussion of the binomial and hypergeometric distributions and tests, which are similar, see Khazanie, supra n.4 at 246-64; Neter, Wasserman, and Whitmore, supra n.3 at ch. 7. The t-test is similar to a z-test. Unlike a z-test, it is used for small sample sizes, when the population is assumed to be normal, and the variance is not known. The t-test uses the same formula for the test statistic as the z-test (see supra para. 7), but instead of obtaining the appropriate critical value from a table of the standard normal distribution, the critical value has to be taken from the tables for the t distribution, taking into account the appropriate degrees of freedom (i.e., number of observations). Note that the t-test yields about the same result as a z-test for sample sizes of 30 or more. Since z-tests are easier to do, they are usually used for large sample sizes. See supra n.17; Khazanie, supra n.4 at 410-413, 521; Neter, Wasserman, and Whitmore, supra n.3 at 335-36, 402- 03, 913. Bell Atlantic says it will initially use a t-test until it is able to run a permutation test in "an automated fashion." Bell Atlantic Dowell/Canny Decl. Attach. B, App. K. In other words, it assumes: H0: ?I = ?C and ?I2=?C2, and HA: ?I ? ?C or ?I2??C2, where ? is the population (theoretical) mean for the incumbent I and competitive LEC C, ?2 is the variance, and H0 and HA are the null and alternative hypotheses, respectively. Brownie, Boos, and Hughes-Oliver, supra n.14 at 260; LCUG Statistical Tests for Local Service Parity at 8-9. There are no standard textbook z-tests for these hypotheses. There are standard z- tests for a test of difference of means which assume that the incumbent and competitive LEC variances are always the same, or that the variances are always different. See Khazanie, supra n.4 at 563; Neter, Wasserman, and Whitmore, supra n.3 at 538-42, 563; supra n.16. In a standard z-test, if the competitive LECs' standard deviation rises, so will the standard error (the denominator in the z statistic), causing the z statistic to fall, even if the difference in the means stays constant. This will not happen with the modified z, since its standard error does not directly depend on the competitive LECs' standard deviation. If the competitive LEC variance (and standard deviation) is large, then the competitive LEC means mC will be much more variable. Since the standard error for the modified z does not depend on the competitive LEC standard deviation, unlike the standard z, the modified z will be more likely to find that a difference in means is statistically significant. AT&T Pfau/Kalb Aff., Attach. 2, "AT&T's Responses to FCC's Questions Dated April 12, 1999" at 3-4. Bell Atlantic Dowell/Canny Decl. at para. 112, and Attach. B, App. K, and Attach. C, Ex. 1, App. D; AT&T Pfau/Kalb Aff. at para. 54. Statisticians define the power of a test as its ability to correctly determine when the alternative hypothesis is true, while keeping fixed the probability of falsely rejecting the null hypothesis, for every possible alternative hypothesis (or Power=1-? while ? is fixed, for all HA). A more powerful test has a lower ?, for the same ? and HA. Neter, Wasserman, and Whitmore, supra n.3 at 339-47; Mood, Graybill, and Boes, supra n.7 at 406-11; William H. Greene, Econometric Analysis 156-57 (3rd ed., 1997). Therefore, these tests are more powerful if they are better able to detect differences in means when the processes serving retail customers and competitors are truly different, while maintaining the same probability of falsely finding a difference when the processes are, in fact, the same. The modified z has been shown to be a more powerful test than a standard z under the hypotheses outlined above (supra n.33), using power curves. Brownie, Boos, and Hughes-Oliver, supra n.14 at 261-63. Permutation tests are classified as a bootstrap method. Bootstrap methods involve repeated resampling of the original data to generate the statistical results of interest. A.C. Davison and D.V. Hinkley, Bootstrap Methods and Their Applications at chs. 1, 4 (1997); H. Scheffe, The Analysis of Variance 313-18 (1959). Duncan, supra n.16 at 608, 973-75. See infra para. 17. The Carrier-to-Carrier metrics are set up in such a way that negative scores indicate that competitive LECs are receiving worse performance than Bell Atlantic customers, while positive scores indicate the opposite. See Bell Atlantic Dowell/Canny Decl. Attach. C, App. D at 1. The plan also provides for performance scores of -1, which represent a confidence level of 79 percent. The adjustment used in the plan of erasing a -1 if followed by zeros in two following months effectively raises the confidence level to 90 percent for -1's that are not erased. Bell Atlantic Dowell/Canny Decl. at paras. 128-29, and Attach. C, App. E at 1. Note that a "larger" negative score is actually closer to zero, so -1 is larger than -2. Bell Atlantic Dowell/Canny Decl. Attach. B, App. K. For metrics with observations excluded from their measurement, the number of observations excluded should also be reported, to improve our ability to determine how accurately the metric measures the universe of orders or customers. A difference in metric values that is statistically significant, however, does not necessarily mean that the BOC's service is discriminatory. We will examine the totality of the evidence before making a determination whether the BOC is providing parity. Khazanie, supra n.4 at 506; Neter, Wasserman, and Whitmore, supra n.3 at 298. We note that Bell Atlantic argues that the 95 percent confidence level is appropriate. Bell Atlantic Dowell/Canny Decl. Attach. B, App. K; Bell Atlantic Duncan Reply at para. 36-38. Type I and Type II errors are described above. See supra para. 9. AT&T argues that choosing a critical value to balance the probabilities of Type I and Type II errors is desirable, because it balances the interests of BOC and competitive LECs by setting equal the chances of falsely finding discrimination and of falsely missing discrimination. While acknowledging that the critical value to achieve this balancing ("balancing critical value") will depend on the number of BOC and competitive LEC observations, they argue that using a fixed critical value based on an 85 percent confidence level is a reasonable approximation of the balancing critical value, given typical competitive LEC sample sizes. AT&T Pfau/Kalb Aff. at paras. 88-93 and n.97 and Attach. 2 at 27-30. This would mean using a critical value for the z-test of 1.04. AT&T's proposal to balance the Type I and Type II error probabilities does appear to have the attractive feature that the interests of the incumbent LEC and the competitive LECs are given equal weight, so that the probabilities of falsely concluding the incumbent LEC may be discriminating and of missing existing discrimination are balanced (so ?=?). Such an approach could be used in future section 271 applications. We would be more likely to accept use of such an approach if the state commission and parties have agreed on its use, particularly since there are details that need to be worked out before it is used. For example, the relevant alternative hypothesis must be agreed upon. We note that the New York Commission has not accepted AT&T's proposal. Bell Atlantic argues that AT&T's proposal is not standard and is difficult to implement. Bell Atlantic Duncan Reply at paras. 36-38. The alternative is to use a two-tailed test to determine whether an incumbent LEC's performance to competitive LECs is either inferior or superior to the performance that it provides itself. Our analysis does not take into account whether superior performance is being provided. We are unable to determine how much superior performance in one metric or for one month could offset inferior performance in another metric or for another month. Bell Atlantic Dowell/Canny Decl. Attach. B, App. K. The use of passing scores in some months to offset negative scores in other months is used in the Performance Assurance Plan to lower the probability of Bell Atlantic making payments under parity. Bell Atlantic Dowell/Canny Decl. at paras. 128-29. See supra n.41. This is one reasonable method of reducing the probability of a Type I error. Bell Atlantic Dowell/Canny Decl. Attach. C, Ex. 1 at 4 and App. C. See supra Section III.C.2. Bell Atlantic Dowell/Canny Decl. at paras. 62, 65-66; Bell Atlantic Gertner/Bamberger Decl. at paras. 7-12. The Gertner/Bamberger study provides us with no information about the impact of the factors they discuss on dispatch orders, so we are unable to make the same adjustments for Average Completed Intervals for dispatch orders. Sources are Carrier to Carrier metrics, Bell Atlantic Dowell/Canny Decl. Attach. D; Bell Atlantic Gertner/Bamberger Decl. at Table 4; Bell Atlantic Gertner/Bamberger Reply Decl. at Table 2. The Bell Atlantic retail numbers used for comparison with the study data for CLECs were taken from the carrier to carrier metrics. The bottom row includes an adjustment to the CLEC average completed interval to take into account the different lengths of the average standard intervals (listed in the third row). The calculation of the CLEC intervals in the bottom row involved taking the study's estimate of the interval for only properly coded orders from Table 4 (2.36 days in August) and adding the difference in average standard intervals between retail and CLEC orders caused by the different order mixes, taken from Table 2 of the Reply (1.84?1.22=0.62 days in August), to get the revised CLEC interval (2.36?0.62=1.74). The column "Diff" contains the differences between Bell Atlantic and CLEC intervals. Results that appear to be statistically significant are marked with an asterisk. See infra n.8. Statistical significance is determined by calculating a z-score, which is the difference in the means divided by the standard error (called "sampling error" by Bell Atlantic), and then examining whether the z-score is less than - 1.645. In order to determine whether our estimated differences in Average Completed Intervals are statistically significant, the standard error must be recalculated. The standard error used here differs from the value published in the Carrier to Carrier report because the number of CLEC orders in that report was used in its calculation, and that number was inflated because of the number of miscoded orders included in it. The standard error is: SE = sI SQRT [1/NC+1/NI], where sI is the standard deviation for Bell Atlantic, NC is the number of CLEC observations, and NI is the number of observations for Bell Atlantic. For our calculations sI and NI are the same as in the Carrier to Carrier report. We adjust the published NC to remove miscoded orders from the count. This was done using the percentage of orders that were miscoded, which was provided in the right column of Table 1, in Bell Atlantic Gertner/Bamberger Reply. So, for example, in August for UNE-P there were 25,270 Bell Atlantic orders (NI) and Bell Atlantic's standard deviation was 2.35 (sI). There were 10,642 CLEC orders, of which 45.9 percent were miscoded, leaving 5,757 orders correctly coded (NC). The result is a standard error of 0.034. The calculated standard error for July was 0.034, and for June was 0.043. The resulting z-scores are -10.1, -10.7 and -19.5, for June, July, and August, all of which are statistically significant. Sources are Carrier to Carrier metrics, Bell Atlantic Dowell/Canny Decl. Attach. D; Bell Atlantic Gertner/Bamberger Decl. at Table 4; Bell Atlantic Gertner/Bamberger Reply Decl. at Table 2. The Bell Atlantic retail numbers were aggregated from the Carrier to Carrier metric data on business and residential orders, to allow comparison with the study's numbers for CLECs. For the calculations of the adjusted CLEC numbers. See supra n. 4. Results that appear to be statistically significant are marked with an asterisk. See infra n.8. Both retail and CLEC data are aggregated for both business and residential orders. The measured difference of 0.03 days is not likely to be statistically significantly different from zero. See infra n.8. Calculations of statistical significance were made using the same formulas as in n.4, except that calculating an aggregate standard error was more difficult for resale orders because the standard deviations are provided only in disaggregated form, for business and residential orders. The business and residential numbers of observations were added to yield the total numbers of observations for Bell Atlantic (NI) and CLECs (NC). The aggregate Bell Atlantic standard deviation was approximated by taking the weighted average of the business and residential standard deviations, weighted by the number of observations. This should yield a standard deviation close to the true standard deviation for the pooled set of observations, if the means for business and residential customers are close together. The means are close for August (1.07 for business versus 1.06 for residential) and for July (0.99 for business versus 1.01 for residential), and the calculated standard error for August is 0.049, and for July is 0.044. The means are not close for June (1.25 for business versus 0.94 for residential), but the Average Completed Intervals show that competing carriers received better service than retail customers in June. The calculated z- scores are -3.9 for August, and -6.3 for July, both of which are statistically significant. If we use July or August's standard errors, it is apparent the June difference of +0.03 days is not statistically significantly different from zero. Federal Communications Commission FCC 99-404 2 Federal Communications Commission FCC 99-404