ISO/IEC 15504 Assessment of the Assigned Procedure

 

Background:

This project requires us to perform a ISO/IEC 15504 Software Process

Assessment of a given process belonging to Software Service Center (SSC). This process consists of several interactions between the contractor (Municipality) and the service center is modeled in Appendix -I and described in Appendix -II.

Internal ISO/IEC 15504 Analysis of the Procedure

A process purpose is achieved in an organization through the activities, tasks and practices being carried out to produce work products that demonstrate whether the specific purpose is achieved. ISO/IEC 15504 provides five categories to classify a process based on these performed tasks, activities and practices, and the characteristics of the work products produced. These categories: CUS, ENG, SUP, MAN and ORG, are discussed below in the light of the assigned process.

 
Process Categories Category Description Category Process Validity
Customer-Supplier Process Category (CUS) This category consists of process that directly impact the customer.  There  are  five processes belonging to this process category.

  • Acquire software

  • Manage customer needs

  • Supply software

  • Operate software

  • Provide Customer Service

 Yes
Yes

Yes
No
No
Engineering Process Category (ENG)

This category consists of process that directly specify, implement  or  maintain  a system and software product and its user documentation.

  • Develop system requirements and design

  • Develop software requirements

  • Implement software design

  • Integrate & test software

  • Integrate & test system

  • Maintain system & software

 No

No
Yes
No
Yes
Yes
Support Process Category (SUP) This category consists of process that may be employed by any of the other processes (including  other  supporting processes) at various points in the software lifecycle.

  • Develop documentation

  • Perform configuration management

  • Perform quality assurance

  • Perform work product verification

  • Perform work product validation

  • Perform joint reviews

  • Perform audits

  • Perform problem resolution

 No
No

No
Yes
Yes
Yes
No
No
Management Process Category (MAN) Process that contain practices of a generic nature that may be used by anyone who manages any sort of project or process within a software lifecycle

  • Manage the project

  • Manage quality

  • Manage risks

  • Manage subcontractors

 Yes
Yes
No
No
Organization Process Category (ORG) Process that establish the business goals of the company and    develop    processes, product and resource assets that, when used by the projects in the organization will help the organization achieve its goals.

  • Engineer the business

  • Define the processes

  • Improve the processes

  • Provide skilled human resources

  • Provide software engineering infrastructure
Yes
No
No
Yes
No

Table -1         ISO/IEC 15504 Process Categories

In Table - 1, we evaluated each process category as it applies to the process under assessment. For this evaluation, we used the information extracted from the process text and answers to the questions asked in Appendix III, to judge on the validity of each category process. The table indicates that SSC puts the most emphasis on the Customer-Supplier category, followed by the Customer-Supplier category.
The ISO/IEC 15504 Software Process Assessment model has six well defined process capability levels describing the evolutionary plateaus towards achieving a mature software process. Each of these levels is described in terms of its mail process characteristics, and the attributes used for capability measurements (Zahran, 1998). For our assessment, we will decompose these capability levels in sufficient enough detail so that we can apply them to our process. Then, we will be in the position to make evaluations, assessments and improvement recommendations on this process.
Figure - 1   ISO 15504 capability levels for their validity to the SSC processes

The characteristics of each level are listed in Table -2 for their validity to the SSC process.

ISO 15504 Level Brief Level Description Validity
Level 0 : Incomplete There's general failure to attain the purpose of the process. There are no easily identifiable work products or outputs of the product._ N/A
Level 1 : Performed The purpose of the process is generally achieved though it may not be planned. There are identifiable work products for the process and these testify to the achievements of the process. Yes
Level 2 : Planned and Tracked The process delivers work products of acceptable quality within defined time scales and resources. Performance according to specifies procedures and planned and tracked. Work products conform to special standards and requirements. Yes
Level 3 : Established The process is performed and managed using a defined process based upon good software engineering principals. Individual implementations of the process use approved, tailored versions of standard, documented processes. The resources necessary to establish the process definition are also in place. Largely
Level 4 : Predictable The defined process is performed consistently in practice within defined control limits, to achieve its goals. Detailed measures of performance are collected and analyzed. Performance is objectively managed. The quality of work products is quantitatively known No
Level 5 : Optimizing Performance of the process is optimized to meet current and future business needs, and the process achieves repeatability in meeting its defined business goals. Quantitative process effectiveness and efficiency goals for performance are established, based on the business goals of the organization. Continuous process monitoring against these goals is enabled by obtaining quantitative feedback and improvement is achieved by analysis of the results. No

Table - 2    ISO/IEC 15504 Process Capability Levels

Process Evaluation and Assessment

Each key process area is described in terms of the key attributes that contribute to satisfying its goals. The key attributes describe the infrastructure and activities that contribute most to the effective implementation and institutionalization of the key process areas.

Except for Level 0, each capability level is decomposed into attributes that indicate the areas an organization should focus on to improve its software process. A process attribute represents a measurable characteristic of any process as defined above. The rating scale defined in Table-3 is used to describe the levels of achievement of the capability of the process attributes (Zahran, 1998). Our findings are listed in Table - 4. Table - 4 shows the capability maturity rating of the SSC to be somewhere between level 2 and level 3. Level 4 and 5 do not apply.

 
Rating Name Description
N
P
L
F		 Not achieved
Partially achieved
Largely achieved
Fully achieved 		There's no evidence of achievement of the defined attribute.
There's some achievement of the defined attribute.
There's significant achievement of the defined attribute. 
There's full achievement of the defined attribute

Table-3      ISO/IEC 15504 Process attribute rating scale
ISO 15504 Level Level Attributes and Process Attributes Validity
Level 1: Performed The extent to which the execution of the process follows the practice defined in the process definition. Such practices are initiated and followed using identifiable input work products to produce identifiable output work products that are adequate to satisfy the purpose.
  • Process Performance .............................................................................




Fully
Level 2: Planned and Tracked The extent to which the execution of the process is managed to produce work products within stated time resources requirements. The extent to which the execution of the process is managed to produce work products. Such work products are documented and controlled to meet their functional and non-functional requirements.
  • Process Performance .............................................................................
  • Performance management .......................................................................
  • Work product management .....................................................................





Largely
Fully
Largely
Level 3: Established

The extent to which the execution of the process uses a process definition based upon a standard process, that enables the process to contribute to the defined business goals of the organization.
The extent to which the execution of the process uses suitable skilled human resources and process infrastructure effectively to contribute to the defined business goals of the organization.

  • Process Performance  .................................................................................
  • Performance management  ..........................................................................
  • Work product management  .........................................................................
  • Process definition and tailoring  ....................................................................
  • Process resource  .......................................................................................




Fully
Fully
Largely
Largely
Fully
Level 4: Predictable

The extent to which the execution of the process is supported by goals and measures that are used to ensure that implementation of the process contributes to the achievement of the goals.
The extent to which the execution of the process is controlled through the collection and analysis of measure to control and correct, where necessary, the performance of the process. The objective is to reliably achieve the defined process goals.

  • Process Performance .................................................................................
  • Performance management............................................................................
  • Work product management...........................................................................
  • Process definition and tailoring .....................................................................
  • Process resource ........................................................................................
  • Process measurement..................................................................................  
  • Process control ...........................................................................................




Largely
Partially
Partially
Partially
Not
Not
Not
Level 5: Optimizing

The extent to which changes to the definition, management and performance of are controlled better to achieve the business goals.
The extent to which changes to the process are identified and implemented to ensure continuous improvement in the fulfillment of the defined business goals of the organization.

  • Process Performance .................................................................................
  • Performance management ...........................................................................
  • Work product management ..........................................................................
  • Process definition and tailoring .....................................................................
  • Process resource ........................................................................................
  • Process measurement..................................................................................
  • Process control ...........................................................................................
  • Process change ...........................................................................................
  • Continuous improvement................................................................................



Partially
Not
Not
Not
Partially
Not
Not
Partially
Partially

Table - 4 ISO/IEC 15504 level attributes

Process Improvements

The ISO/IEC 15504 model can be used for process improvements. In Table - 5 below, we comment on the recommendations for improvements in the process.

Recommendations
Features Application to the procedures
Performance Management
  • Establish plans and procedures, perform the work, tracking it, and take corrective
  • actions as necessary
  • SSC should have policy statements to emphasize the connection between
  • organizational commitment and the process.
  • An organizational structure is absent in SSC. It is needed to support the key process areas for example, the quality assurance teams
  • Access to special skills, adequate funding, and access to tools must be provided. A lack of these will result in poor work performance
Work Product Management
  • SSC should provide product servicing after the product is delivered
  • SSC should track of the ongoing customer needs throughout the operational life
  • of the software.
  • Establish a mechanism that allows the customer to easily determine the status of
  • their product progress.
Process Definition and Tailoring
  • Take corrective actions when project targets are not achieved
  • Introduce Quality Assurance (QA) team to enforce the documented rules in the
  • areas of product quality, performance management, etc.
  • Take corrective actions to better reuse the skills and knowledge gained from
  • previous projects and contracts.
Process Change
  • Support effective interaction between the management and the individuals
  • Identify and recruit individuals with the required skills and competencies.
  • Formal and informal training of the new staff is recommended, also ongoing training to ensure that all individuals have the required skills to do their work.
  • Do regular process audits/checks to identify weak areas and improve on them
Process Control
  • Establish traceability between software requirements and software design.
  • Establish procedures to communicate software requirements or software plan changes to all effected parties.

Table - 5 ISO/IEC 15504 process improvement recommendations

Conclusion

An ISO/IEC 15504 evaluation and assessment was performed on the assigned process of a software service center. An in depth analysis of the process based upon ISO/IEC 15504 capability level attributes revealed that the process has partial adoption to the standard, at present it lye somewhere between levels 2 and 3. This indicates that SSC uses a process definition based upon a standard process to employ effectively the suitable skilled human resources and process infrastructure to contribute to the defined business goals of the organization. We also performed an analysis on process categories and found that SSC shows good attributes in Customer-Supplier Process Category (CUS), followed by Support Process Category (SUP). To improve this process we recommended redesign in the areas of Performance management, Work product management, Process definition and tailoring, Process Change and Process Control.

APPENDIX - II

"After the commercial department has obtained an OK both from the Municipality and from the management in Engineering, Max contacts the municipality and obtains the tapes and the list of additional requirements, that is, the request for personalized data treatment. Max then notifies Jupiter about the project start, and sends the tapes to Dorian. Dorian converts the data set in a format readable and processable by Engineering's internal servers with the help of Moll.

In the meantime, Max has notified Jupiter that the Municipality required a personalization: in particular, the Municipality requires that on-line normalization be carried out by externally, by the Municipality itself. Jupiter is the only one that has the skills for it, so he starts working.

When the archives are ready, Dorian sends them to Bruce, and then notifies Jupiter of the task completion. Bruce runs the batch procedures for normalization and upload, and notifies Jupiter when the task is complete.

When Jupiter completes the personalization, he notifies Bruce and waits for him to finish. Then it converts the archives for exporting them to the Municipality, and delivers them to Max with the Municipality, which is in charge of actually delivering the data.

The Municipality performs the on-line normalization, and then returns the data to Max.

Max then notifies Jupiter that the data are ready, and simultaneously delivers the data to Bruce who (thanks to Jupiter's personalization) can load the data and start the automatic filtering procedure.

When Bruce finishes, he notifies Jupiter, and passes the data to Frances. Frances performs the on-line filtering and when she finishes she notifies Jupiter and handles the archives to Bruce.

Bruce unloads the data from the conversion program and loads them into the main software: THEBIT. He informs Jupiter about it and then starts the data calculation procedure in THEBIT. When the calculation is over. Bruce presents the results to who in turn notifies both Max and the commercial interface that the product is ready and an invoice must be sent to the Municipality."

APPENDIX - III

Here's a list of questions ^t should be asked from the Software Service Center before proceeding with the CMM assessment.

1. Is the work done in a team environment with well defined projects? If so, is there a project manager? Are the employee roles defined?

Yes, work is done in a team environment, teams are formed (members are assigned) on a per project basis dependent on the nature of the project. The project manager assigns specific roles to the team members, these roles stay throughout the project life-time. Project leader is fully responsible for the project. S/he is in charge of the allotted budget, completion by the due date, etc.

2. Are the company rules and regulation, code of behaviour, company vision, etc. documented?

Yes, there's an official documentation of these rules and regulations for the company. These encompass employee rights, benefits, salaries, roles and functions. There's also a code of employee behavior, on and off the job. Oaths of confidentiality, employee contracts, etc. are signed and used in cases of need. Company vision and goals are also well defined and documented and are put in places where easily visible. Copies of all such documents are made available to the employees.

3. Are all the daily procedures documented? Is it routinely practiced?

There are company rules in place to require procedure documentation, but it's rarely practiced and enforced. In fact, the common practice is not to document, or if it is done, there's nothing to enforce its official availability to the rest of the team or other company employees.

4. Is there a quality assurance body to enforce the company rules and regulations?

No, there's no official body with the assignment of this purpose. It's pretty much left to the project leader or the department manager to deal with. Firstly, it is considered the project leader's responsibility to produce the required product within the given restraints and specifications. It's also his duty to resolve any outstanding issues or conflicts within the team. The matters go up in the company-hierarchy, starting with the manager, if they're not resolved by the project leader or if they start to affect the project, company or other projects.

5. Is there procedures in place to test and verify the finished products for the pre-set goals?

This also falls under the responsibility of first the project leader and then the manager. Upper management gets involved when things start to really go wrong. The ultimate responsibility lies on the shoulders of the project leaders who can be sacked in extreme cases.

6. How are the contracts given to the customers? Is the customer-client relationship based on common understanding?

Majority of contract are won based on the results of bidding. Bids are entered for contracts and if won then the contract is allotted. Some business is also generated from companies that SSC has signed yearly contracts with.

7. Is there effective actions taken when the project's performance deviates from the software goals?

Only when things start to really go wrong, then the upper management gets involved. The ultimate responsibility lies on the shoulders of the project leaders who can be sacked in extreme cases.

8. Is there small checks or modular testing during the progress of the software procedure to see whether the software is on track?

Yes, there are such checks. At the beginning of each project, a project schedule is made. Frequent inquiries are made by the manager. Appropriate actions are taken if project is lagging behind the projected dates of task completion.

9. Is there formal or informal training for the staff to keep up-to-date with the new skills and technology?

No. there no official arrangements for such training. It all depends on the department manager. If it is felt that a specific skill is needed then a training course will be approved.

10. How do the team members or members of different teams interact with each other? On average how often does this contact take place and by what means?

Different teams are in touch with each other by whatever means are felt necessary. By phone, email or in person. Usually persons in different groups working closely with each other will contact each other several times a day as felt needed. Members of a specific group are located in the same office space, so they work with each other. Weekly meetings are also held to stay up to date with each member's work or check project progress.

11. How are the conflicts between team members, or teams resolved? Is there a quality assurance committee responsible for this?

No, there's no official body with the assignment of this purpose. It's pretty much left to the project leader or the department manager to deal with. Firstly, it is considered the project leader's responsibility to produce the required product within the given restraints and specifications. It's also his/her duty to resolve any outstanding issues or conflicts within the team. The matters go up in the company-hierarchy, starting with the manager, if they're not resolved by the project leader or if they start to affect the project, company or other projects.

12. Is the knowledge gained from previous projects taken into use for the future or present projects?

Except for the documentation as required by the customer, usually the daily procedures are not documented. However, teams are formed based on their past experiences, skills, learning objectives, etc. A very friendly, helpful environment is encouraged, therefore it's always easy to ask questions or ask some one (with the right experience and skills) for help.

13. What procedures were in place in case of crisis, such as a delay in the completion date of a project?

It's the manager's duty to resolve any outstanding issues or conflicts within the team. The matters go up in the company-hierarchy, starting with the manager, if they're not resolved by the project leader or if they start to affect the project, company or other projects. Project schedule issues are dealt by the upper management who pay particular attention to project schedule coherence.

14. What mechanisms were set up for feedback on company products from the customers or the company employees?

Lines of employee feedback are encouraged by all channels, whether through project leader or management. Ideas or criticism of value are asked to be documented in the form of official proposals for consideration by the upper management. Considering the small size of the company, employee feedback is working satisfactorily in its present form.

15. Was constant fire-fighting a standard routine? If yes, what approach was used?

Well, there was not constant fire-fighting, but there were definite incidents that required constant fire-fighting. In such incidents, the employee roles would be set aside and whatever means available were applied to put out the fire. This approach worked effectively in almost all cases, in others the damage would be suffered by the company.

16.What kind of a relationship is maintained with the customer? And how are the outstanding issues resolved?

A friendly environment is maintained with the customer with open channels of dialogue. Customer needs are respected while confining to the original contract conditions signed upon at the beginning of the company-client relationship.

17.What form of a relationship with the customer is maintained after the completion of the project (after the product is delivered)?

The relationship usually ends with the ending the contract which happens once the product is delivered satisfactorily to the customer. This means, the product is installed and the customer staff is trained in its use. After that, customer is own their own to maintain it. Except rare cases, no customer support is dedicated to a specific product. Any product modification requests result in a new contract.

Bibliography

Zahran, Sami. Software Process Improvement. Addison Wesley Longman. Essex. England: 1998.


Home | Resume | Transcript | Documents | My Life | Resourcess
E-Books | Code | Greeting Cards | Galleries | My University | IEEE

© 2002 Upal.ca  All rights reserved.