Comprehensive students' evaluation system

COMPREHENSIVE AND EFFECTIVE EVALUATION SYSTEM IN EDUCATIONAL INSTITUTIONS

 Preamble: Every institution carries out internal examination/class test in order to evaluate how far the student has understood the portions covered by the teacher/faculty. On completion of the class test, each faculty collates the data pertaining to his/her subject, identifies the failed students, in order to provide additional coaching/counseling. Additionally, this evaluation also provides actions to be taken on the students’ performance before final examination in order to ensure that the “failed” students get thro’ the same.

Process: A simple process given below, using the collated data by the faculty makes a comprehensive system to measure the performance of the students in relation to the overall class performance and to evaluate the performance of the individual faculty

Performance evaluation of the students

o   Compile the marks scored in the class test student wise- subject wise in MS-Excel sheet

o   Grade the marks under the following groups

o   > 80%- Very Good

o   60-79%- Good

o   50-59%- Satisfactory

o   < 50%- Poor

·        Calculate the average score

·        Assign the Grades A, B, and C as given below

·        Take the following decisions for students, based on the above

o   If the individual score is > the class average score- GRADE A

CONGRATULATE/MOTIVATE

o   If the individual score is < than the class average score, but >  minus 5% of the class average score- GRADE B- MOTIVATE the student

o   If the individual score is < the minus 5%, but > than minus 10% of the class average score- GRADE C-  COUNSEL the students,  give special coaching classes and watch the next exam score

·        Performance evaluation of the faculty

·        Calculate the number of Grades A, B, and C

·        Take the following decisions for the Faculty, based on the above

o   If the % of Grade A students is more than 70%, the Faculty is to be CONGRATULATED

o   If the % of Grade A Students is between 50 to 69%, the Faculty is to counseled for improvement

o   If the % of Grade A Students is less than 50%, the Faculty is to be counseled for improvement and identified for further training

Inference:

·        The above system identifies the following requirements in respect of faculty

o   Changes in lesson plans

o   Changes in teaching methodology, use of audio-visual teaching aids, etc.,

o   Identification of faculty training requirements, much more effectively- looking from the perspective of the students

Measurement of effectiveness:

o   For Students- measure the Grades for each cadet for all the subjects to get an overview of the academics

o   For Faculty- combine the number of Grade A Students and the feedback from the students to get an overview of the competence

FOCUS ON DELIVERY TO CUSTOMER- A SIMPLE FLOW

An organization can be viewed a system of processes and activities linked to deliver.

Delivery is one of the key characteristics of an organization.

Delivery is defined as an activity to satisfy (meet) a requirement

An organization links its activities to deliver a product or service either internally or external to it

Delivery is the sum total of the efforts of people, carrying out processes with the resources provided by a leadership which ensures that both the internal and external requirements are met.

Since resources of any nature- be it money, people, equipment, and knowledge are limited, the various elements of delivery are to be optimized

Optimization is dynamic in nature and needs continual review

Hence any delivery process works efficiently and effectively under controlled conditions

One of the controlled conditions is a system approach

Since system approach is standardized, ISO 9001 provides a platform to conceptualize and implement this with ease and consistently

The basic elements of a system approach in an organization to be deployed to ensure delivery are

Leadership, Process concerns, Customer concerns, Quality concerns, HR concerns, Infrastructure concerns, Outsourcing concerns, etc.,

While everyone agrees that the Leadership is the core element, the rest of the elements are bound and linked to ensure consistency

MOST COMMON NCR's

Document related

·         Uncontrolled documents in the manufacturing floor- including illegibility, scribbling and mutilations

·         Procedure does not reflect the practice and vice versa.

• Vendor base not being consistently updated with current revisions of drawings and documentation.
• Current and Obsolete drawings being used simultaneously
• Approved vendor list not updated regularly.
• Critical process instructions not available at the place of use.
• Instructions displayed in workplace found not under document control.
• Unauthorised and undated hand-written changes made to procedure
• Inconsistency in the revision status of same document at different places of use

 Product related

• No clear identification of good and non conforming material.
• Incorrect storage of materials.
• Shelf life products not assessed periodically
•  Over aged product in storage

·         Management related

• Internal quality audits not carried out at specified intervals.
• Actions minuted in the management review committee meetings  not followed up for
• completion.
• Product non-conformances at in-process stage not selected for corrective actions.
• Internal quality auditors carrying out audits without formal training

·        Records related

• Quality record  not legible
• Improper indexing /storing of records

Solutions to the above lie in the following

• Design simple and operable management systems- adopt KISS- Keep It Simple and Short
• Bring in all the documents under management system purview- never have any procedure/template outside the system
• Assign process owners and make them accountable for the process performance
• Make the system user-friendly and comprehensive
• Create, store and retrieve documents and record using soft copies and avoid hard copies as much s possible
• Bring in a culture of adhering to systems by periodical training to users
•  Provide leadership in system maintenance

·        

• 
  

IA Catchwords– Series 9

Observe

Observe is an important catchword in Auditing. In fact, an experienced Auditor uses this effectively, in identifying non-compliances and incompatibilities during Auditing. Non-compliance occurs, when there is an observed gap between documents and practices and incompatibility occurs, when the auditee provides a process explanation which is different from the observed ones. In both the cases “observations” by the auditee is critical.

Given below are 10 commandments for the Auditors

1. Observe the process operations in action- conversion of inputs to outputs
2. Observe the deployed process controls for correctness and effectiveness
3. Observe the records of process monitoring for correctness and completeness- including control charts and process cards
4. Observe the workplace arrangement
5. Observe the workplace safety
6. Observe the process document dynamics- how often the process operations have been revised/improved
7. Observe the body language of the auditee to verify the process knowledge
8. Observe the comfort and confidence in the auditee’s answers
9. Observe the handling and retrieval of records maintained
10. Observe the storage & handling of process rejects

Hope you would be able to practice the above. Please share your experiences. Good Luck

QMS Data Management- Series 2

Collation of data is done using a data matrix. The next step is prioritizing the parameters towards process improvements. Prioritization is based on the criticality of parameters/processes, expectations of customers, stakeholders, etc., The next step would be to carry out data analysis.

Data analysis consists of using various statistical tools. Statistical methods help to understand processes, to bring them under control, and then to improve them.  If one is running a high-volume product process, Control Charts help in Pareto Chart and Cause-and-Effect chart are the most useful tools for others. When we are dealing with simple individual processes, the above can be applied effectively. However, in an organization where many processes are involved, it is better to deal with them at a global level, since there are multiple process owners.

Approach

A sensible and effective approach to carry out Data Analysis would be by using a Data Analysis Team (DAT) drawn for the specific purpose. The following steps are recommended.

·         As the Team Leader, Management Representative (MR) takes the initiative in forming this DAT. He selects the team members having good analytical skills and process knowledge in his/her area of work.

·         MR has to ensure that at least one member is represented by each process/function identified for improvement.

·         Brainstorming sessions would make the job easier for the MR.

·         The first step is to study the existing system, and identify potential causes of root causes and prepare a Ready Reckoner

·         The various heads under which these Root Causes are identified could be

o   Men

o   Machine

o   Measurement

o   Method/System

o   Materials

o   Documentation

·         One must realize that the various potential root causes fall under 80/20 rule- meaning that common causes are 80% and only 20% are special to an organization.

·         Some examples of potential root causes under each head are

o   Men- lack/less of communication skills, lack/less of commitment/attitude of personnel, lack of product knowledge, lack of product knowledge

o   Machine- improper machine/tools/dies/jigs setting, inadequate/improper machine maintenance, inadequate machine/equipment performance

o   Measurement- inadequate test equipment, improper recording, inadequate calibration

o   Method/system- inadequate monitoring/controlling of processes, inadequacy in operational planning, inadequate supplier control

o   Materials- improper choice of parts/components, material mix-up, alternative materials used

o   Documentation- inadequate process documentation, inadequate recording, non-availability of controlled documentation

·         Deployment of the ready-reckoner in completing the data analysis would be discussed in the next article. Meanwhile, initiate and go ahead with forming DAT. Any help, do not hesitate to contact. Good luck

Training Videos for Safety in workplace- OHSAS 18001

Unsafe Acts of employees

Safe Working with Forklifts

Safety Skills

Unsafe Acts_compilation

IA Catchwords– Series 8

Not explained

Process realization activity is to be documented as per the requirements, needs of the process and the competency of the personnel. Process explanations are to be documented to ensure that the process is carried out to meet the requirements. If it is observed that the absence of such explanations had caused or potential to cause non-conformities, then it is necessary to provide explanations either in the form of documentation or visual displays.

Not trained

Training of personnel is critical to any activity. Training is to be planned both for meeting the existing process/product requirements and enhancing the competency of operating personnel. If it can be concluded that lack of training had resulted in process/product/service non-conformities, then it is important that training is identified as a means of closure of the non-conformity. in fact, "not trained" is one of the common causes of analysis, if proper RCA (Root Cause Analysis) is carried out. This is true in not only in product realization or maintenance or marketing. For example, if a company has identified that a lower hit-rate is an effect and the cause is due to inadequate training of marketing/application personnel in understanding clients requirements, then the same is to be addressed.  

QMS Data Management- Series 1

Data management includes the following

·         Identification of opportunities for data identification

·         Prioritization/classification of data in each of those opportunities

·         Identification of data to be managed

·         Planning for data management

·         Systematic arrangement for data collection

·         Data collation

·         Analysis

·         Identification and implementation of corrective and preventive actions

QMS clause No. 8, identifies Measurement, Analysis and Improvement and focuses on measurement of QMS and calls for determination of applicable methods, including statistical techniques. Data planning however starts from Cl. No. 4.1, when there is a need to monitor, measure, and analyze QMS processes. In addition, clause number 8.2.1, 8.2.2, 8.2.3, and 8.2.4 also need to be managed.

Records form a major chunk of data management. Records requirement are specified and needs to be addressed. Records provide evidence that the realization processes meet the QMS requirements. Hence, management of data from those records forms the basis of the improvements. Records could be in the form of sheer information and measurable data. Such measureable data management is what is needed in the Clause No. 8.

Data measurement

Records form the basis for data identification.

Below given are some of the areas of data measurement:

·         Process related- 5.6, 6.2.2, 6.3, 7.2, 7.3, 7.4.1, 7.4.2,7.4.3, 7.5.1,7.5.2, 7.5.3, 7.5.4,7.5.5, 7.6, 8.2.1, 8.2,2,8.2.3

·         Product/service related- 7.2, 7.3,7.4.3,7.5.1, 7.5.3, 8.2.4,

The first step is to segregate information and monitorable and measurable data from the records.

Data management includes systematic collection of data from the processes and product/service realization

Data collection is done in such a manner, that they can be collated and analysed

Designing of various formats/templates/checklists/control charts to be used for data collection play an important role. Frequency of collection and method of recording data is planned to provide adequate information on the effectiveness of the control of the process/product/service.

A poorly designed format/template can defeat the purpose of data measurement, without providing required information. Once the data is collected, a collation matrix would help in consolidating them and help in prioritizing for subsequent analysis.

A collation Matrix would typically contain data on various parameters of each process being monitored, frequency of data collection, and the source of data collected.

Data analysis would be discussed in the next article. Meanwhile, initiate data collection and collation. Any help, do not hesitate to contact. Good luck

IA Catchwords- Series 7

No records maintained

Records are the results of operations. Records demonstrate results and compliance achieved QMS standard identifies 20 records to be maintained. They provide evidence of successful completion of the QMS processes. If there is no evidence of records being maintained as per QMS and if it is observed that there is a need for records maintenance for any of the processes even when not specified, then it is essential to ensure the same.

No specific methodology defined

QMS defines process of value addition to achieve specific planned results. In order to achieve consistent results, processes/activities need to be controlled. One of the methods of controlling is to define a methodology. In the absence of the same, results are likely to be misleading and one cannot be sure for the achievement of the specific targets, specifically in the implementation of monitoring, measurement, analysis and improvement- one of the examples of specific methodologies is - measurement of customer satisfaction. QMS also specifies use of statistical techniques as one of the methodologies

Not documented

Documentation is the heart of the QMS. The Standard calls for documenting a quality policy, quality objectives, a quality manual, 6 mandatory procedures , 21 records and procedures for effective planning, operation and control of the processes. These documents are intended to address the various requirements of the QMS. While there can be no two opinions on an auditor " looking for" the mandatory documents and records, the identification of "documents and records determined to be necessary to ensure the effective planning, operation an control of the processes" is crucial for the organization. Even when a document exists for controlling the processes, if it is not brought under the purview of the QMS and controlled, it is not acceptable and considered to be "not documented". The document shall adequately describe the activities of the process and in such a manner "operable" by the process owner and the team.

Not effective

This statement is "all-pervading" and an "effective tool" for an auditor. An auditor shall verify the

· documents, records, processes- such as planning, operation, monitoring, controlling, etc.,

· activities- such as data collection, collation, reviewing, recording, authorising, communicating, , etc.,

· operating personnel-for understanding, applying the skills, and knowledge, performing on the machines, etc.,

· maintaining the machines, etc.,

Measurement of effectiveness is done under 3 major categories- results, time, and effort. If it is observed that there is a lack of focus and improvement in any of the above, then it can concluded that they are "not effective". For example, even if parameters of a process are recorded routinely and correctly over a period of time to meet the output requirements, it can be considered "not effective", if the same has not yielded any opportunity to improve the process.

Not effectively monitored

Monitoring is an important activity to verify if the process/activity is being carried out as per the defined /documented requirement. Monitoring is the first step towards process improvement. Monitoring is needed to demonstrate conformity to the product requirements, conformity of the QMS and conformity to the processes. Effective monitoring must lead to a definite and defined process improvement. Monitoring shall be followed by collection of data, collation of data and subsequent analysis. These shall include use of various statistical techniques. Records maintained at various processes shall form the basis of verification of effective monitoring.

Not established

QMS standard uses various standard nomenclatures such as - defined, determined, established, etc., Establishing involves, setting up or creating. It could be a procedure, a record, or a process. Establishing includes the setting up of the process, understanding by the operating personnel and stabilization/standardisation.The process is said to be "not established", even if any one of the criteria given above is not met with.

Not evidenced

Evidencing includes Documents, records, and the Practices and these shall be carried out as per the defined standards, and practices. Evidencing is the proof of completion of an activity. "Not evidenced" shall include absence, incompleteness, or uncontrolled, unless specified so. Evidencing can be done either physically or by electronic media. For example, Video-conferencing record is acceptable as an evidence of a review.

No records maintained

Records are the results of operations. Records demonstrate results and compliance achieved QMS standard identifies 20 records to be maintained. They provide evidence of successful completion of the QMS processes. If there is no evidence of records being maintained as per QMS and if it is observed that there is a need for records maintenance for any of the processes even when not specified, then it is essential to ensure the same.

No specific methodology defined

QMS defines process of value addition to achieve specific planned results. In order to achieve consistent results, processes/activities need to be controlled. One of the methods of controlling is to define a methodology. In the absence of the same, results are likely to be misleading and one cannot be sure for the achievement of the specific targets, specifically in the implementation of monitoring, measurement, analysis and improvement- one of the examples of specific methodologies is - measurement of customer satisfaction.QMS also specifies use of statistical techniques as one of the methodologies

Not documented

Documentation is the heart of the QMS. The Standard calls for documenting a quality policy, quality objectives, a quality manual, 6 mandatory procedures , 21 records and procedures for effective planning, operation and control of the processes. These documents are intended to address the various requirements of the QMS. While there can be no two opinions on an auditor " looking for" the mandatory documents and records, the identification of "documents and records determined to be necessary to ensure the effective planning, operation an control of the processes" is crucial for the organization. Even when a document exists for controlling the processes, if it is not brought under the purview of the QMS and controlled, it is not acceptable and considered to be "not documented". The document shall adequately describe the activities of the process and in such a manner "operable" by the process owner and the team.