Participants:
Winchester Brown, Evelyn Farbman, Janet Frost, Kathy Herron, Lynn Marino, Asomgyee Pamoja, Jennifer Pelletier, Jose Ricardo Rivera, Peggy Schuyler, Kurt Simonds, Lisa Tessier, Eleanor Vendetti. Contributing Guests: Mary Ann Affleck, Nancy Caddigan, Renee dePastino, Ray Hughes, Camilla Thompson.
Consultant:
Barbara Wright

Tuesday, May 19
Morning Interpreting and reporting Critical Thinking Assessment
Afternoon Experimental scoring of critical thinking artifacts

In the morning the team looked at charts and tables showing the results of our critical thinking projects so far: Critical Thinking Day, which included Casino Critico and the Patriot Act Debate; the fall and spring iterations of the course-embedded assignment; and the focus groups that emerged from the spring assignment. We studied the quantitative charts that report on the embedded assignment and the qualitative charts that report on the questionnaires and interviews associated with both the assignment and the other activities. We discussed interpretations and recommendations. In summary, we noted:

1. Quantitative scores are unacceptably low.
2. They echo the pattern we’ve seen in the writing and math assessments—that students score better on ascertaining information than on making meaning from it.
3. Qualitative data indicate that students may need help with some critical thinking processes and with making connections between critical thinking and work in their classes.
4. Nonetheless, students are favorably disposed toward critical thinking.

These four general findings outline both a need and an opportunity to raise consciousness of critical thinking throughout the College curriculum. Dean Affleck suggested three methods for supporting such a cross-curricular spread for all General Education goals:

1. At a Professional Development Day on General Education, go through each of the General Education goals and brainstorm the curricular locations of instruction for each one. The day’s program would include information on developing assignments that would produce artifacts where accomplishment within each area is visible. The goals and the scoring rubrics would be promulgated widely.
2. When asking faculty for course outlines each semester, the Dean would append a form to be attached to each outline. The form would ask teachers to indicate which course assignments address particular General Education goals. In a separate section, the form would ask a similar question for the goals of key degree programs which are served by the course.
3. In the request for faculty commitments to Additional Responsibilities, the Dean would identify curricular mapping as a high priority, and would designate a work group to study the material generated by step 2 above. This group would create a grid showing where each goal is addressed throughout the College, clarifying threads and reiterations.

In the afternoon the workshop resumed with a brief business meeting. Among the issues discussed were:

• Making the General Education goals and rubrics ubiquitous. Jen will work on a webpage that’s easily accessible from the CCC homepage. Evelyn and Jen will work on the creation of a folder that would show the General Education goals and rubrics. The folder would be given to each student and teacher.
• Simplifying the Three Stones project. Evelyn will follow through on agreements to embed a single writing assignment within courses at three stages of college progress. The assignment will serve as a written interview about general education goals and will be submitted electronically to a Three Stones webfolder. Our new qualitative assessment software will be available to simplify analysis of the interview results. Writing proficiency will be assessed by the usual holistic scoring teams.

The team then turned to the examination of critical thinking artifacts submitted by 5 colleagues teaching the following classes: Math 046, Nursing 111, and English 101. The norming session raised several issues:

1. Teachers need more guidance about how to tailor assignments for artifact purposes.
2. If we want more comparability among artifacts, we may need to suggest discussion points for introducing the rubric to students.
3. For more comparability, we could provide model assignments.
4. If we can live with less comparability, we will get more naturalistic results.
5. We may want to specify that the assignment must elicit evidence of at least three items in the rubric.
6. The steps of critical thinking may be clearer in artifacts that are presented in paragraph form than in those that show numeric processes
7. The written artifact scores clumped toward the middle, while the numeric ones, where the answers were clearer, were high or low. This conformed with findings from the CWA and the CMA, as well as with the difference between CTA Accidents and Lakes.

Next, Dr. Wright led a review of what we have learned from the Critical Thinking year. The table below summarizes our comments.

Resulting ideas for the coming year include:

1. Selective embedding, tailoring assignments to classes.
2. Distinguishing between embedded and common assignments. Embedded requires wide variability. Common has led us toward narrow comparability.
3. Providing only a template for embedding, as we did with CWA, rather than the whole assignment, as we did with CMA and CTA.
4. Clearer instructions for artifact collection, possibly templates.
5. Develop a kickoff activity to raise interest and provide fun.
6. Collect data over longer periods, using data from these first 4 years as baselines (need for deep IR engagement in this process).
7. Developing a reporting schedule with IR for the above.
8. Pursue reporting gaps—placement scores and follow-up problems elsewhere.

From this, we turned to next year’s assessment of the College’s fourth General Education goal: Develop a Global Perspective. To provide our own global perspective on the goal, we uncovered a table full of globular objects and considered their attributes. Then we distributed reading packets containing relevant models and supporting information gathered by Dr. Wright. These constituted homework for the Thursday session.

The afternoon closed with a presentation of gifts to Evelyn, who will leave the team in June. Lynn, who was leaving at the end of the day, had engineered an amazing technicolor kaleidoscope embedding several assessment metaphors in visual and kinesthetic profusion. The kaleidoscope will be available to team members who need to shift their perspectives from time to time.

Tuesday, May 19
Morning Cracking open the Global Perspective door—scope and problems Afternoon Stepping through—rubrics and methods for the year ahead

In the morning, Dr. Wright led the team in gathering ideas about cultural awareness within our own environment at Capital. A spirited, wide-ranging discussion ensued, and the whiteboard filled with notes falling into two categories:

1. Observations
• Graduation essays included many favorable comments about interactions with people from diverse backgrounds. The College contains wide diversity but we may not be using it in ways that encourage global awareness.
• We all have our selective blindnesses; we consider the rules that we follow obvious and natural until we find ourselves outside our own groups, and then we may either question those rules or apply them inappropriately to other groups. Examples included incidents involving public manners, sports outcomes, personal distance, time precision, urgency levels in waiting situations, academic integrity.
• In terms of international awareness, we need to note that the diversity we see among ourselves is obscured by a transnational corporate image of American superness. This is conveyed around the globe by policies and by communications media, leading to serious misunderstandings, both abroad and among new immigrants, about American culture.
• Raising these issues beyond the level of multicultural food fairs has sometimes proven volatile, since cultural awareness is delicate and people’s sensitivities are hard to see before they’re stepped on. We passed around the metaphor of the hot potato.
2. Questions
   • What do we want to do with our Global Perspectives assessment project—to test or teach our students? Do we want students to celebrate/appreciate other perspectives? Become aware, analyze, evaluate them?
   • Do we want to build civic responsibility? Help students see the parallels between different cultural behaviors? Develop students’ intercultural competencies?
   • How do we combine and distinguish among intellectual and affective skills?

Dr. Wright directed us toward the models provided in our reading packet and presented some new ones. Controversy arose about the adaptation of a religious school’s model for our secular purposes, and the group felt how fast potatoes can get hot. This was a cautionary experience that could inform our assessment design: we must be ready for our efforts to make some steamy potatoes roll suddenly out of ovens.

Before lunch, we discussed embeddable questions in a list proposed by Dr. Wright modeling how to tailor a general inquiry to specific classes. These questions could be adapted across the curriculum as artifact assignment models and/or as mini-embedded questions to apply within a common template.

After a brief lunch, the team and some guests divided up for a simulation game in which groups were trained separately to enact two conflicting communication styles and then engage with the members of the other group. After the resulting giddy swirl, the groups separately listed their observations/judgments of the other group and their reflections on their own cultural patterns. The game dramatized in a safe way the conflicts inherent in raising awareness of cultural difference.

With little time remaining, we discussed rubric content and assessment methods.

1. Rubric
   We looked at Evelyn’s draft of a rubric based on the simplest of the models, using three categories: knowledge, skills, and attitudes. We struggled with ways of making the rubric work in a circular format and tried to fit in the competencies we most care about. We concluded that Evelyn should send out a more complete draft for the team to consider after the workshop.
2. Methods
   We agreed to employ multiple assessment methods including:
   • Global Pavilion in the fall. This will offer lively low-intensity activities for endearing global thinking to students and staff: a birthplace map at the entrance for participants to mark, an attitude survey, some hypothetical problems and simulation games, geography and history bees and/or bubble quizzes*, a Jeopardy-style food game, an online questionnaire*, an art project including materials and tools, music. These may be followed by a movie.
   • Questionnaire for use in classes (online where possible)*, followed by focus groups on hypothetical situations and “aha” moments. Transcripts of taped discussions*.
   • Selective embedding of a global perspective template. This will include assignment models and mentoring for teachers who are interested in tailoring a question for their courses. Where possible, responses will be submitted electronically.*
   • Clarified artifact solicitation. This will be accompanied by the rubric and support for assignment design. Participating teachers must indicate what parts of the rubric the assignment will illustrate.
   * All starred items may be scanned by the NUD*IST software for qualitative analysis.

Team members volunteered to work on pieces:

The workshop closed with a presentation of gifts (to Barbara Wright from Evelyn and to Lisa Tessier’s baby Anna from the Humanities Department) and ice cream cake.

Suggested schedule:
Early June -Rubric draft to all team members for comments
Mid June -Goals and rubric folder to printer?
End June-All CT reports complete and posted on website.
Early September -New leadership. Plan for Professional Day announcements.
Sept 24 -Prof Day introduction of Global Perspectives goal and methods.
Oct 1 -Workshop with Dr. Wright.
Mid Oct -Selective embedding pilot under way in team members’ classes.
Oct 19 or 26 -Global Pavilion.
Early Nov -Follow-up focus groups.
Mid Nov -Scoring of selective embedding and artifacts.
Early December -Preliminary quantitative results, decisions about spring activities.
Mid Jan 2005 -Convocation announcements of wider activities based on pilots.
Spring -Informed reiterations of the methods piloted in the fall.
Spring -Scoring sessions for artifacts coming in for other Gen Ed goals.
May workshop-Reports on year past, on Gen Ed baselines and projects.Sustainable plan for the Long Slow Distance.