Design

Both CMA1 and CMA2 were designed by members of the Student Learning Assessment Team, led by the three math teachers on the team. Both assignments were carefully tested and revised with the help of other team members, and the ensuing dialogue about priorities and presentation was instructive to all participants. The scoring and interpretation of results clarified issues in the math curriculum and revealed possible actions that the College could take to improve General Education efforts. Further, the project precipitated a major community forum on the CMA theme-the distribution of wealth in Connecticut. Members of the Assessment Team feel the course-embedded projects of the past two years have laid the ground for the development of a genuine culture of inquiry among faculty at the College. The following characteristics serve as guidelines for further General Education assessments:

• Educational for faculty as well as students
• Real application of discipline tools authenticity
• Interdisciplinary shared responsibility, faculty conversations
• Voluntary participation by teachers
• Responsive to faculty critiques
• Embedded in regular courses, not extraneous
• Inclusive of students at all levels and across curriculum
• Specific to our students, programs, concerns
• Practical, informing instruction and policy
• Interesting, engaging
• Productive of spin-off into community activity

Embedding

The assignments were embedded by volunteer teachers in the following courses:

Scoring

The math cohort on the Student Learning Assessment Team spent much time and effort on the preparations for scoring. Each of the three members prepared sample answers representing each level of proficiency and collaborated on training potential scorers the week before the fall scoring session. Further adjustments and logistical plans emerged from that training session, and on October 23, Team members and guests scored 75 samples of CMA1. CMA2 was shorter and the preparation for scoring was eased by experience from the fall session. On April 9, Team members scored 110 samples.

For both scoring sessions:

• Samples were randomly selected but evenly distributed across all participating sections.
• Student ID numbers were replaced with key codes that identified a given student’s academic history but not the student’s identity.
• Scorers specialized in one category of problem and were normed within scoring groups for reliability. A math teacher supervised each group.
• Scores were assigned by analytic categories first, and then a math teacher assigned an overall score in place of a holistic score.
• The paragraph questions were treated separately from the mathematical ones. In the case of CMA2, a special rubric measured
  responses according to the following categories:
  1. interpretation of data,
  2. response to question,
  3. clarity.
• Scorers devoted the last half hour to assessing the problems and lessons of the project and of the scoring session.
  For more details, see Meeting Notes of 10/23/02 and 4/9/03.

Responses from Participating Students

For CMA1, students were asked to identify the degree of difficulty of each question. Their answers were sporadic, but the incline of difficulty that the test designers envisioned was corroborated by the ratings that students offered. For a rating of the test overall, the average was 2.53 on a 4 point scale, midway between “moderate” and “difficult.” The highest difficulty rating ( 4 “impossible” ) was assigned by some students to the three items that students, in fact, performed most poorly on, showing consistency between students’ experience and results.

In both CMA1 and CMA2, students were asked how well they liked the assignment. In both cases, negative and positive responses were evenly balanced, with a large middle group not responding at all. In both cases, the neutral and no-response group constituted over half of the samples. Comments ranged from hating the assignment or feeling that it had little or nothing to do with the course, to neutral responses, to notes that the project was useful because it stimulated thought about poverty and hometowns.

Responses from Participating Teachers

After both CMA1 and CMA2, participating teachers were asked to complete an online evaluation of the project, answering questions about compatibility with syllabi, student reactions, format, use of time, and reflections on the uses of math in their fields. The survey can be found at http://www.ccc.commnet.edu/cgi-bin/survey/surveyH.pl?survey_name=cma_feedback

Teachers’ responses to the evaluation of CMA1 pointed us toward simplifying directions for students and seeking ways of eliciting student interest in reading the article and doing the math problems rather than leaving this up to individual teachers. We drastically simplified the assignment for CMA2 and advertised a forum on the issues discussed in the article. Even so, teachers’ evaluations after CMA2 continued to identify student confusion and disaffection as problems.

At the same time, teachers’ comments on their own interest and involvement were positive, indicating that faculty are willing to cooperate in efforts to improve the math skills of students, and that the CMA is generally compatible with course syllabi, useful to students (in the opinions of the teachers), and worth the teachers’ time.

Consequences

• On April 10, the College sponsored a public forum on “The Web of Poverty” in which seven panelists from local organizations discussed the causes and effects of poverty in the Hartford area. Students who had read the article on which the CMA was based were encouraged to attend, and although the forum was held in the evening, at a time when student participation is difficult to muster, the event was packed. The forum was a direct outgrowth of the conversation on poverty in Hartford which was initiated by the Common Math Assignment. While students’ answers to the question of how they liked the assignment indicated resistance to the topic and the questions, the hundreds of students who attended the forum were highly engaged. Many of them had participated in the CMA and therefore had an enriched context for understanding the speakers and asking questions.
• The three math teachers on the Assessment Team who labored diligently to design both CMAs and scoring systems had to respond constantly to the questions and comments of those on the Team who were not mathematicians. This dialogue was full of discoveries for both groups. Both the difficulties and the possibilities of math across the curriculum are clearer now, and the conversations continue.
• Before the project was complete, the math department was already making plans for some curricular clarifications and mentoring of adjunct math teachers, based on the incoming results and emerging problems of definition. The CMA assignments have provided new lenses for looking at math syllabi and pedagogy.
• The Student Learning Assessment Team has learned how to adapt the embedded assignment model to varying situations. The CMA maintained many of the characteristics that made the Common Writing Assignment successful in 2001-2002, but it also bent to respond to the needs for more controlled responses, more precise scoring, and more quantitative analysis of results. The experiment with a rubric for grading the paragraph in CMA2 provided a pilot for next year’s work on Critical Thinking assessment. The CMA year has provided evidence that embedded assignments in different areas can build on each other’s strengths and lessons.
• A report on the findings and recommendations is being made available to interested groups at the College, and the Academic Division will discuss institutional uses of the results for the improvement of General Education in quantitative skills.