Committment to Assessment

CHAPTER 4: CRITERION THREE

Criterion Three: Commitment to Academic Assessment:

At Labette Community College (LCC), assessment is a means of measuring and evaluating student learning. It leads to improvement in teaching and learning and is used to improve curriculum at the institution. LCC has clearly stated general education outcomes. Course level outcomes and competencies are used to assess the overall effectiveness of the college’s instructional mission. LCC incorporates outcome assessment as part of the college’s strategic planning process. Outcomes assessment is not used for instructor evaluation, but is part of program review.

To ensure that LCC is fulfilling its stated academic mission and core values, the goals of the assessment are as follows:

Improving the teaching and learning process in each course and program.

Increasing accountability to those whose interests are served by Labette Community College.

Linking instructional outcomes and competencies with the college’s educational support services.

History of Assessment at LCC since 1994:

Labette Community College’s formal process for assessment of academic outcomes began with the approval of The Plan to Assess Student Learning by NCA in 1994-1995. With this plan in place, a course level assessment system was instituted according to the timeline stated in that plan.

As early as fall 1994, faculty began work on course outcomes. By fall 1996, all LCC syllabi contained student-learning outcomes and competencies. Faculty identified satisfactory outcome levels and evaluation systems capable of measuring outcomes. Beginning spring 1997, fulltime faculty began filing outcomes reports with their associate dean. These reports included number of students in the course and percentage of students reaching a satisfactory outcome. LCC supported this paradigm shift by providing a day during inservice week to work on outcomes and assessment reporting. This workday continued until Fall 2000 when most faculty members were familiar with the process and the time was better used for other activities. Outcomes reporting became part of the department budget process for academic year 1997-1998. This process provided faculty with a means of linking budget request for resources to student learning outcomes.

During fall 1998, the assessment process was extended to adjunct faculty, who began filing assessment plans for each of their courses. In spring 1999, Kansas community colleges and regents’ universities began meetings in Wichita, Kansas, to establish common core outcomes and competencies in general education transfer courses. The LCC math, English and biology departments participated in this first round of meetings. These meetings have continued to the present time, and LCC has been fully represented at each meeting. LCC curriculum, outcomes and competencies have been adjusted, where necessary, to meet the core standards established at these meetings.

As the understanding of the need and value of outcomes reporting became clearer, processes and forms changed to standardize data and to give it more meaning. The Instructional Outcomes and Assessment Committee began the arduous task of reviewing The Plan to Assess Student Learning submitted to NCA in 1995. Major tasks were to check LCC’s progress in student assessment and to bring the plan and processes up-to-date. In addition, the committee began work on the assessment of student learning in general education. Members of this committee, in conjunction with other members of the LCC staff, the Dean of Instruction, and the College President began work on the Institutional Improvement Plan to support the State Core Indicators (2000-2001) as part of Kansas Senate Bill 345.

Assessment of general education began through review of syllabi of general education courses from LCC and other institutions. The Instructional Outcomes and Assessment Committee met several times with general education faculty for input. Several modifications were made to the document until it was finalized in Spring 2003. During the spring 2002 semester, the Institutional Outcomes and Assessment Committee began reviewing exit assessment tests and models to determine appropriate means for LCC to to evaluate success of it’s general education courses Participation in the State Core Competencies meetings and the submission of Outcomes Assessment reports continued throughout this time. In Spring 2002, vocational students took Work Keys assessment tests. They were tested in reading, listening, and applied math.

The Instructional Outcomes and Assessment Committee recommended adoption of the Collegiate Assessment of Academic Proficiency (CAAP) to use as an exit exam for General Education assessment For academic graduates. The committee also recommended broader use of COMPASS tests for assessment purposes beginning in spring 2003.

CAAP exit assessment tests for writing, science reasoning, critical thinking, math, and reading were given in the spring 2003 semester to students graduating with academic concentrations. LCC students’ scores were consistent with national averages on the CAAP. Students graduating with vocational/technical concentrations took the WorkKeys exams for applied math, reading, and listening. LCC’s mean scores for WorkKeys on a scale of 3 to 7 were 5.7 for reading, 5.4 for Applied Math, and 3.6 for listening.

During fall 2003 and spring 2004, the Instructional Outcomes & Assessment Committee, in conjunction with the humanities, social sciences, and computer science departments, developed a Humanities Exit Assessment Exam, adopted the use of the College Base Social Science Exam for social studies, social science, and history, and developed a Computer Skills Assessment Exam to be implemented with the CAAP Test and the Work Keys Test during the exit assessment testing in April 2004. Members of the committee attended a statewide conference on assessment in April 2004. Work continued on compiling and completing the documents concerning the LCC assessment.

For presentation at the fall Inservice 2003, the Instructional Outcomes and Assessment Committee created a comprehensive document detailing all components of course level assessment. In addition, an assessment flow chart was devised to illustrate how all elements of instructional assessment are integrated at LCC. Gradually, a document describing instructional assessment was completed and became part of the Faculty Handbook and the Report on Student Assessment at LCC. This document is a complement to the assessment flow chart and details instructional assessment at an institution wide level, course level assessment, and assessment through exit testing of graduates.

Overview of Academic Assessment at LCC:

Assessment of student learning has many components that can stand independently but also mesh with one another to comprise a clearer and more complete picture.

The academic assessment process at LCC uses the following direct and indirect performance indicators for each goal:

1. Improving the teaching and learning process in each course and program

1A. Course Level Assessment

1A-1 Faculty submit course level assessment reports and improvement plans each semester

1A-2. Dean of Instruction produces executive summaries of course level assessment reports and improvement plans each semester

1B. Program Reviews

1B-1 Review all programs on a three-year cycle

1B-2 Generate summary reports with strengths, weaknesses, recommendations, and improvement plans

1C. Exit Assessment Testing For Degree Seeking Students

Place Flow Chart Here

Labeled as Appendix 4-A

Place Outcomes Report

1 Vocational

1 Academic

-1 Compare exit test results with internal assessment results

1C-2 Compare exit test results from year-to-year

1C-3 Compare exit test results to national norms

1C-4 Compare entry to exit results (COMPASS to CAAP/Work Keys)

1D. Instructional Committees

1D-1 Curriculum and Instruction Committee

1D-2 Distance Education Technology Committee

1D-3 Enrollment Management/Recruitment Committee

1D-4 Instructional Outcomes Assessment Committee

1D-5 Library Committee

1D-6 Strategic Planning

1D-7 Advisory Committees: Internal and External

2. Increasing accountability to those whose interests are served by Labette Community College

2A. Student Satisfaction Inventory

2A-1 Noel-Levitz survey data pertaining to student satisfaction with faculty and instruction

2B. External Advisory Committees

2B-1 Advisory Committee surveys and recommendations

2C. Graduate Surveys

2C-1 Student satisfaction survey of general education program

2C-2 Vocational Graduate surveys

2C-3 Graduate employer surveys

2C-4 KVED Reports

2D. Transfer student data at 4 year Institutions

2D-1 Compare results of LCC transfer students to in-house Regent university students

3. Linking instructional outcomes and competencies with the College educational support services

3A. Grant Writing

3A-1 Student support grants

3A-2 Faculty development grants

3A-3 Technology grants

3B. Resource Management

3B-1 Budget process

3B-2 Faculty development

3B-3 Instructional technology procurement and maintenance

3C. Instructional Technology Resources

3C-1 Computer support

3C-2 Technology plans

3C-3 Computer labs

3C-4 Classroom instructional technology

3D. Student Success Center

3D-1 Testing services

3D-2 Tutorial services

3D-3 Computing services

3D-4 Services for students with disabilities

3D-5 GED and ABE

3D-6 Operation Excel

3E. Instructional Media Center

3E-1 Faculty and staff technology training

3E-2 Faculty support services

3E-3 Software evaluation

3E-4 Innovative concept dissemination

3E-5 Audio-visual support

3F. LCC Library

3F-1 Printed materials

3F-2 Video media

3F-3 Computer/Web databases

3F-4 Research Assistance

Place Exit Assessment Of General Education Outcomes For Students Completing Associate Degrees

Annual Report

Label Appendix 4-C

Surveys of graduates and external advisory committees for some programs and departments provide data to be considered in curricular planning and the formation of outcomes and competencies.

(Example of Graduate Survey Used By Respiratory Care Program Appendix 4-D.

4. The program review process provides data and curriculum revision suggestions to the departments within the college. Programs are reviewed on a three-year cycle. The Program Review Committee identifies strengths and weaknesses and creates an action plan, which is updated annually. Program reviews are approved by the LCC Board of Trustees.

The college obtains data from the Kansas regents’ universities concerning performance of LCC transfer students and from the Noel-Levitz Student Satisfaction Inventory. The college feeds the data back through the Vice President of Academic Affairs, the Dean of Instruction, and the instructional departments and programs for use in the process of developing curriculum and course level outcomes and competencies. This process is detailed in the Flow Chart of Institutional Assessment of Instruction at Labette Community College.

The Instructional Outcomes & Assessment Committee continues work on revising data collection in order to provide meaningful feedback to improve instruction.

CORE COMPONENT 3A:

The organization provides evidence of student learning and teaching effectiveness that demonstrates it is fulfilling its educational mission.

Labette Community College clearly differentiates its learning goals for its students by identifying the expected learning outcomes for each course, department, and program.

Faculty at LCC define student learning outcomes and competencies in a master syllabus for each course on campus. Academic departments establish the outcomes and competencies for their courses offered by that department, and adjunct instructors use those outcomes and competencies in the courses they teach for the department.

In addition to defined outcomes in courses and departments, the Allied Health programs also meet state and nationally defined competencies and track the students for each class in the program. All other vocational programs have defined program competencies.

Place Example of one program competency on next page Appendix 4-E

A list of general education outcomes has been developed by the Instructional Outcomes Committee with significant input and participation by the full time faculty of LCC. The college faculty identify which of the general education competencies are addressed in each general education course taught by the college. Place Copy of this on next page Appendix 4-F

All full-time and adjunct faculty members are required to maintain course level assessment measures based upon the outcomes and competencies defined in the master syllabus for the particular courses being taught. Faculty members compile the results of course assessment and formulate improvement plans for the next time the course will be taught. These improvement plans for teaching and learning are reported to the Dean of Instruction at the end of each semester. The Dean compiles an executive summary of each year’s Outcome/Assessment Reports and Improvement Plans submitted by faculty.

LCC regularly evaluates fulltime and adjunct faculty for teaching effectiveness. The evaluation is comprised of a Class Visitation by the Dean of Instruction or appropriate Director, a student evaluation, and a conference. Faculty provides the evaluator with course materials prior to the classroom visit. A reflective statement, written by the faculty member, regarding teaching and learning is part of the process. Evaluation schedule for full time faculty occurs one per fall and spring semester for instructors in their first and second year of instruction. Instructors in the third or fourth year are evaluated at least once per academic year prior to February 15^th. Once achieving tenure, instructors are evaluated at least once every three years. Each adjunct faculty member undergoes a complete evaluation including classroom visitation, student evaluation, and self-evaluation for the first two semesters teaching for LCC. During the third and fourth semesters, student evaluations will be required. The remaining portions of the evaluation will be completed at the discretion of the Dean of Instruction. The fifth semester will start a rotation in which adjunct faculty are evaluated every third year.

LCC uses a variety of institution-wide exit assessment tests (CAAP, College Base, Work Keys, Humanities Exit Assessment, Computer Skills Exit Assessment) to assess learning that has taken place for students graduating from LCC with academic and vocational degrees. Delivery methods of tests are appropriate to areas of study being assessed.

LCC uses COMPASS English Math and Reading exams, along with ACT and SAT scores, to assess the knowledge base of first-time enrollees and place these students into appropriate math, English and reading/writing intensive courses.

Program reviews of vocational and academic departments track grade distributions in courses on a three-year rotating cycle. Allied Health and other vocational programs use registry exams and other instruments, such as portfolio reviews, to assess student knowledge and learning as they exit the program.

LCC requires elements in each course syllabus summarizing the methods of instruction and evaluation used in each course taught on campus. Instructors are free to design their own methods for the courses they teach. Methods used include but are not limited to lecture, discussion, presentations by students, student papers, projects, portfolios, instructor generated exams, standardized exams, computer assisted instruction, and lab experiments.

Assessment results are included in reports filed with the KBOR. Executive summaries of instructor outcomes/assessment plans are communicated to the instructional divisions and are available in the LCC Library. Results of exit assessment testing are shared with the instructional divisions and are available in the LCC library.

Instructional committees on campus at LCC all have strong faculty representation. The Instructional Outcomes & Assessment Committee has produced a course-level assessment plan with input from academic departments. In this report, full-time faculty in each department have defined outcomes and competencies for their courses. The Instructional Outcomes Committee (with regular and extensive input from LCC faculty at regular monthly instructional meetings and at fall and spring in-services) has also produced a document defining the general education outcomes. The Curriculum and Instruction Committee and the Instructional Outcomes & Assessment Committee have developed the Master Syllabus to insure uniform outcomes and competencies in every LCC course. Faculty members attend the State Core Competency meetings and integrate statewide outcomes into course curriculum and syllabi.

The Instructional Outcomes & Assessment Committee has been meeting at various times since the original plan was written in 1994-1995. The course level assessment system was implemented in 1999.

Strengths:

Faculty are involved in developing and modifying outcomes identified for each course and department.

Faculty are involved in devising evaluation methods to determine achievement of outcomes at the course level.

Vocational programs have program competencies that define learning goals.

Assessment of student learning provides evidence at the course, program and institutional level.

Assessment of student learning uses multiple direct and indirect measures of student learning.

Institutional assessment testing uses various methods of delivery and questions at different cognitive levels.

Results obtained through assessment of student learning are available to interested parties, including students themselves.

LCC integrates into its external data into its assessment of student learning (e.g. graduation rates, passage rates on licensure exams, placement rates, transfer rates).).

LCC tracks all assessment data and files reports with the Kansas Board of Regents (KBOR).

Faculty and administration routinely review the effectiveness and uses of the organization’s plan to assess student learning.

Qualified faculty members determine curricular content and strategies for instruction.

Supporting Evidence:

Master Syllabus Form
Adjunct Faculty Handbook
Bloom’s Taxonomy
Course Syllabi
Nursing Department Competencies
Respiratory Care Competencies
Radiography Program Competencies
Graphic Design Competencies
Other Vocational Care Competencies
General Education Outcomes at LCC
Summary Report of where LCC courses fit in the General Education Outcomes
Membership and Definitions of Committees which work on General Education Outcomes
Membership and Definition of Committees which work on Assessment
Assessment Report Form
Assessment Report Forms of Various Departments
Executive Summary Form
Samples of Executive Summaries
Summary Results 1999-2004
Policies on Exit Assessment Testing
Statement on Humanities and Computer Skills Exams
Exit Assessment results 2003 and 2004
Comparison of Exit Assessment Results 2003 and 2004
COMPASS Testing Policies for math, English/Writing and Reading
Math English/Writing, Reading Placement Test data/results
Nursing COMPASS Reading and Listening tests for ESL students
Program Review Documents, synopsis, and policies
Sample grade distributions from past program reviews
Table of each program and the test taken, who takes the test, portfolio reviews, etc.
Registry exam results 1000-2004
Reports to Vocational Advisory Committees
Survey of Instructional Methodologies used
Survey of Assessment Measures used
Sample Questions and Cognitive Level Information from CAAP, Work Keys, College Base, Humanities, Computer, and Registry Exams
Copies of Assessment Reports to KBOR
Copies of Assessment results and Executive Summaries in Library
Assessment Structure Flow Chart
Course Level Assessment Document
Samples of Statewide Outcomes in General Education Courses
LCC Assessment History Documents
Executive Summaries by Associate Deans
Two or three generations of Outcomes/Assessment Report Forms
Outcomes and Assessment Committee records, minutes, etc.
Evaluation procedure for full time and adjunct faculty.

Concerns:

Sharing of assessment results with stakeholders.
Validity of LCC generated exit exams.
Complicated and time intensive process required to test for humanities outcomes involving coordination of several areas of the college.

Recommendations:

Post assessment results on the LCC website and make available to the Student Government Association (SGA).

Present a formal report on Assessment to the Board of Trustees.

Collect data from LCC generated exit exams for three years to establish baseline.

Analyze validity of LCC generated exit exams.

Ask Computer Support to review process by which students are downloaded into Blackboard for the purpose of taking the Humanities Exit Examination.

CORE COMPONENT 3B:

The organization values and supports effective teaching.

Faculty who are teaching transfer courses at Labette Community College are required to have a Master’s degree and at least 18 graduate hours in the field or a closely related field. The Dean of Instruction and the Vice President of Academic Affairs make determinations about which fields are closely related.

Faculty who teach non-transfer courses, such as professional/ technical courses, developmental, or College preparation courses, continuing education, and work courses must have credentials, special training, experience, creative production, or other accomplishments or distinctions appropriate to the discipline.

Exceptions to the credentials requirement may be made if: the instructor has a three- to five-year plan to complete the appropriate credentials on file with the Dean of Instruction. The instructor must also demonstrate significant progress toward accomplishing the plan and completing the appropriate credentials by submitting to the Dean yearly documentation of completed coursework. Exceptions may also be made if the instructor has a Master’s Degree or higher with ten years or more of teaching and/or work experience in the discipline.

Faculty for online courses offered by LCC must meet criteria recommended by the Distance Education Committee and published in the policy and procedure manual. Prior to approval to teach an online course, the faculty member must have completed at least a one credit hour course on online teaching. In addition, sufficient access to a computer with reliable Internet connection and software that is adequate for the facilitation of online instruction must be demonstrated. LCC values instructors who seek out online teaching opportunities and will compensate for training courses. New online instructors are assigned a mentor through the Distance Education Committee during the course development process in order to facilitate the course development and ensure the quality of the online course offering. The Distance Education Committee must review any newly developed online course. The committee will make a recommendation regarding approval of the course to the Curriculum and Instruction Committee. The Distance Education Committee and the Dean of Instruction must approve courses that LCC purchases from proprietary sources. Because of the demanding nature of online courses, online instructors are paid 1.5 times the regular overload rate when calculating compensation; 1.5 times the credit hour(s) taught when calculating compensation, and 1.5 times the credit hour(s) taught when calculating faculty load.

The College has a professional development fund designated in the Master Agreement with the faculty. This fund is administered through the Vice President of Academic Affairs, who determines the allocation and use of the funds for individual professional development projects. Faculty are encouraged to use these funds to stay up-to-date in their instruction and personal career development. Each full-time faculty member is allotted a specific amount of professional development funds, one half of which is allowed for credit course tuition, and one half of which is allowed toward other professional development activities or professional fees. A Professional Development Application Form has been developed for requesting the use of the allocated funds. Faculty requesting funds in addition to the allotted amount will have their approvals delayed until it can be determined that other faculty members will not be using their allocations.

In addition to professional development activities off campus, several professional development activities are held on campus each semester. To encourage faculty to learn how to use the latest technology in their classroom presentations, LCC has a Center for Innovative Instruction at which faculty can learn how to use technology in the classroom and how to develop online courses. Another example is the Gribben Lecture held by the English Department each year in October. For each of these lectures/workshops, a nationally known speaker on different methods of teaching Writing is brought to the campus. These workshops/lectures are not only attended by the faculty of the English Department, but are also attended by faculty from many high schools, community colleges, and 4-year institutions around the four-state area. In addition, several professional development activities have been brought to the campus by way of online or distance learning technology.

In an effort to recognize exemplary performance by faculty, LCC presents a Distinguished Faculty Award on an annual basis for fulltime and adjunct faculty. To be eligible for the award, the faculty member must have completed a minimum of four years teaching service for LCC. Previous winners of the award are ineligible for nomination for a 3-year interim after selection. Nominations for fulltime faculty can be made by full time students or full time employees. Adjunct faculty nominations can be made by full time employees, adjunct faculty, or students. Nominees for the award complete a Distinguished Faculty Award data sheet, and students from a course are asked to fill out an Inventory for Distinguished Faculty Award form. The data are given to a committee and a recipient recommended based on the following criteria for full time faculty: Teaching Effectiveness, Student Support Efforts, College Service/Leadership, Professional Development, and Community Service. For adjunct faculty, the criteria are as follows: Student Support Efforts, College Service, Professional Development, and Community Service. The awards are presented before their peers. For full time faculty, the presentation generally occurs in the spring semester before graduation. For adjunct faculty, the presentation occurs at the annual General Session for Faculty held each fall prior to semester start. Awardees are presented with a plaque, and funds are made available to attend the National Institute for Staff and Organization Development (INSOD) Conference in Austin, Texas, or a comparable conference in the individual’s discipline.

Strengths:

§ LCC assures qualified instructors are hired in academic and vocational programs.

§ LCC supports professional development designed to facilitate teaching suited to various learning environments.

§ LCC recognizes effective teaching, and provides support to improve teaching effectiveness and student learning.

Supporting Evidence:

§ Policy Procedure Manual

§ Master Agreement

§ Faculty Handbook

§ Instructors’ Credentials

§ Program Review Documents

§ Curriculum and Instruction Committee minutes and documents

§ Outcomes/Assessment Reports and Executive Summaries

§ Professional Development activities and Expenditure Reports

§ List of conferences, workshops, seminars, training.

§ DOI, Title III, Carl Perkins records of funds spent for professional development

§ Full-time and Adjunct Faculty evaluation processes, procedures and forms (Faculty Handbook)

§ Distinguished faculty awards for full-time and adjunct faculty—processes and forms (Faculty Handbook)

§ In-service activities and speakers

§ Online Instruction—policies and processes

§ Computer Service training seminars

§ Support activities by Dean and Vice Presidents.

§ Media Services documents

§ On campus technology acquisitions in classrooms

§ Use of grants for Instructional Technology

§ Instructional budgets

Concerns:

· Ability to continue support for online and computer assisted on ground instruction.

· Finding enough qualified adjunct instructors to teach all needed courses.

· Enough funds for professional development for faculty.

· Low number of full time faculty accepting nomination for Distinguished Faculty Award.

Recommendations:

· Continue to provide technical support for faculty and students.

· Provide technical support for faculty and students outside of normal workday.

· Hang projection units from ceiling to promote ease of use and increase safety.

· Improve recruiting efforts for adjunct faculty.

· Improve adjunct faculty pay.

· Revisit allocation of professional development funds.

· Analyze Distinguished Faculty Award process. Consider changing incentive.

CORE COMPONENT 3C:

The organization creates effective learning environments.

Central to effective learning is appropriate advisement. The college has devoted resourses to enhance this vital area. An advising system was developed to enhance student success by helping students enroll in the courses appropriate for their goals and performance level. The new advisement system was piloted in Fall 01 with more than 100 participants. An Advising Handbook was written and distributed to all advisors. In Spring 2002 (is this date correct), a Director of Admissions/Advising Coordinator was added greatly strengthening the advisement process.

All advisors must go through a training session. Advisors must know which classes are required in the program or major of each student as well as how to interpret the results of the COMPASS enrollment assessments so the students can be enrolled properly. LCC uses the advising process to ensure that students do not enroll in courses for which they do not have the appropriate skills.

Students who score low on the COMPASS tests must take developmental courses to bring up their skills before enrolling in courses that are more difficult. LCC has developmental courses in several areas. Following is a list of math, English classes and reading courses, and the scores, which place students in these courses:

§ Developmental math courses include the following:

o Foundations of Math (33 and below on pre-algebra)

o Beginning Algebra (33 and below on algebra; 34 and above on pre-algebra)

o Intermediate Algebra (34-56 on algebra)

Students who score 20 or above on the COMPASS or who have an ACT score of 20 or a SAT score of 600 may take College Algebra.

§ Developmental English/Writing Courses include the following:

o Basic Writing I (0-51)

o Basic Writing II (52-69)

Students who score 70 and above on the COMPASS or who have an ACT score of 18 or a Verbal Score of 470 on the SAT may take English Composition I. Students who score 49-51 or 52 to 69 on the COMPASS are in the decision zone and may write an essay to try to go up one level to the next class (i.e. from Basic Writing I to Basic Writing II or from Basic Writing II to Composition I). These essays are graded by English Department Full-Time Faculty.

§ Developmental Reading Courses include the following:

o Basic Reading I (0 to 54 or an ACT reading score of 1o or below)

o Basic Reading II (55-74 or an ACT reading score between 11 and 17)

Students who score 75 or above on the COMPASS Reading Test or who have 18 or more on the ACT score are not required to take a reading course.

Students who are placed in Basic Reading I can enroll in the following classes:

§ Piano Classes (Commercial Music and Music)

§ Private Lessons (Commercial Music and Music)

§ All Ensembles (Commercial Music and Music)

§ Lighting Technology

§ Sound Reinforcement

§ Explorations in Music lessons

§ Fundamentals of Music

§ Ceramics I & II

§ Photography I & II

§ Design I & II

§ Printmaking