1. Key Success Factors
Provision was required to address the many developing influences in clinical work in primary care (general practice) in which international standard definitions are emerging. Examples are the protocols for drug formulation, for disease classification, and for clinical procedures. While commitments were not made to specific standards, opportunity was provided to ease the adoption of a particular standard at a future point in time.
The requirement for functionality was split between those described as essential, highly desirable, and desirable. In this way suppliers could provide software which met the essential requirements and be given a basis for addressing further requirements in future releases. Similarly users were given the opportunity to operate a relatively simple and basic system with the option of further features in future releases in line with the maturity of operation of a computer based system.
1.2 Competition among software suppliers
The availability of a choice of software products provides a good environment for competition among suppliers. Each product conforms to the essential requirements, meets some of the desirable requirements, and may also be distinguished in terms of special additional capabilities. The meeting of a common base of requirements will ensure innovation is carried out in additional features relating to functionality, usability etc.
Special attention to ease of use is intended to ensure continued use of the system rather than reversion to a manual system through frustration with a poor user interface. The preferred outcome is that which ensures that medical as well as administrative personnel access the system on an ongoing basis.
3.4 Extensibility and Maintainability
A good design structure is sought. This allows easy enhancement and extension of the software in further releases. Good code structure and good technical documentation ensures ease of maintenance. This is of particular concern as many users are remote from the suppliers location. This also facilitates maintenance services by local software organisations on a product developed by another organisation.
3.5 Data interface
The ability to transmit data to other systems and receive data from other systems meets the potential for a greater degree of integration in health informatics.
The existence of a good user support service is emphasised. In the absence of external communications capability in many of the computers in use, a local support service is preferred. This is expected to change with the advent of widespread use of external networking of systems.
3.7 Data integrity
The data stored is confidential. Means of ensuring that it is not corrupted, that it is capable of being restored if it is corrupted or lost, and the ability to selectively limit access are important factors in ensuring that the integrity and confidentiality of information is maintained.
The availability of the software on a range of standard computing platforms provides for increased opportunity for flexibility in the use of the software in the future.
4. The Certification Scheme
This is a certification scheme for all software products for practice management for general practitioners in Ireland. All certified products are allowed to be promoted through the regional health boards. All systems purchased which include a certified software product for practice management are partly funded by the local regional health board. Each product which is evaluated and accepted as in conformance with the essential requirements is certified. An evaluation report is also produced which provides additional information identifying the extent to which desirable requirements are met and elaborating, wher useful, on how essential requirements are met.
The scheme consists of documents which are used to support the certification process. The primary document is that which sets down the requirements of the software product. This document was established in conjunction with and agreement by the sponsoring organisations. The requirements are defined in accordance with the ISO/IEC 9126 software product quality model  which identifies quality characteristics and sub-characteristics.
A set of documents is available to the evaluator giving guidelines on the methods to be employed in the evaluation of the product. These documents conform with the SCOPE method  of product evaluation as developed in the ESPRIT project of the same name and published in 1993. This method has now become the basis for the emerging international standard being developed by a working group within a joint technical committee of ISO and IEC. The first parts of this standard ISO/IEC 14598  are due for publication in 1998.
The software supplier requests that it’s product is certified. Documents are available to guide the supplier and to identify the respective roles of the evaluator and the software supplier during the course of the evaluation.
5. The Certification Process
The key steps in the process are as follows:
• STEP 1: Supplier
Completion of agreement with the evaluator to carry out evaluation; specific reference to product name and release number being evaluated. Both supplier and evaluator agree to meet the requirements of the process in aspects such as the submission of product parts, execution of the evaluation, production of certificate and evaluation report, protection of confidentiality and copyright of supplier, return or destruction of product parts.
• STEP 2: Supplier
Submit the stated product parts for the product name and release number specified in the agreement, and as further elaborated in discussion with the evaluator.
• STEP 3: Evaluator
Carry out evaluation on product parts submitted by supplier: use the pre-defined requirements established in the current version of the requirements document in the scheme; use the evaluation method documented in the scheme, supported by selected supporting evaluation modules as appropriate to the product parts submitted.
• STEP 4: Evaluator
Publish certificate to confirm that the product meets the essential requirements; show minor qualifications if applicable; submit to supplier, and advise regional health boards of awarding of certification.
• STEP 5: Evaluator
Publish evaluation report stating extent to which all requirements were met, and elaborating on how met if applicable; submit to supplier; submit copy of report to regional health boards if specifically requested by supplier.
6. Experience to Date
6.1 The products
Of the 9 products which have been evaluated, all met the essential requirements and received certification. In the course of evaluation some improvements were leveraged in the products prior to completion of evaluation and market release. The evaluation report showed up varied notable limitations in most products. Some products are experiencing significantly greater market success than others.
6.2 The Users
There is a significant increase in awareness of the benefits of use of information systems for both clinical and administrative purposes among doctors. The take up rate of systems is meeting expectations of the sponsors.
6.3 Health Informatics
Some of these products have been used in pilot projects in aspects of the integrated health informatics plan, such as telematics and clinical test laboratory results.
6.4 The Scheme
The scheme was established in late 1994 and reflected version 1 requirements. These requirements were set at a level which encouraged a significant initial level of take-up by doctors. A document within the scheme provided a statement of direction. This document was intended to provide a bulletin board for suggestions for the next releases, and to provide an influence to suppliers in the future enhancement of their products.
This scheme provided a comprehensive and strategic support to the successful introduction of computer systems for practice management for general practitioners; it especially included:
• a set of essential requirements, through which certification was achieved, and led to greater acceptance of a product by a purchaser, and related health board funding
• a further set of desirable requirements, against which an evaluation report was produced, and led to a means of distinguishing products’ capability in the market place
• a scheme with a special version number, with a determination to raise the capability of products through an updating of the requirements in a new version, produced over a reasonably spaced timeframe
• a means of co-ordination with emerging developments in other areas of health informatics with which these systems will interface.
The continued exploitation of the opportunity provided by the scheme will ensure it’s continued success. This is especially relevant in the opportunity to update the scheme; providing the prospect of increased efficiency in practice management, and increased integration of health informatics.