Participants: Winchester Brown, Evelyn Farbman, Janet Frost, Terry DeVito, Kathy Herron, Lynn Marino, Asomgyee Pamoja, Jennifer Pelletier, Jose Ricardo Rivera, Peggy Schuyler, Kurt Simonds, Peter Wursthorn, Carmen Yiamouyiannis.

Contributing Guests: Mary Ann Affleck, Nancy Caddigan, Patton Duncan. John Mohammadi. Consultant: Barbara Wright

Tuesday, May 20
Morning Interpreting and reporting Common Math Assignment data
Afternoon Selecting critical thinking elements for assessment at CCC

In the morning, the team looked at charts and tables showing the results of CMA1 and CMA2 data and decided on what to present in the report of the year's assessment project. We discussed interpretive language to accompany each chart and revised a draft of reflections and recommendations that summarize the outcomes. We agreed on five recommendations relating to the following: 1) use of baseline data to pilot instructional interventions in key math classes, 2) staffing and training, 3) use of Institutional Research resources to answer key questions, 4) changing college policy to hasten student fulfillment of math requirements, and 5) suggestions for math across the curriculum. Evelyn will prepare the report and, after checking with math faculty, will distribute it according to the system establish for last year's Common Writing Assignment (executive summary to President, full report to dean, website posting, etc.)

During the lunch break, team members went to the auditorium to film a reprise of the assessment trip to OZ (originally presented at Professional Day in February). Media services will prepare a CD and videotape of the tale in the expectation that it can be included in the AAHE poster session at the Assessment Conference in Seattle, where we will be represented by Terry, Peggy, and Kathy.

In the afternoon, Dr. Wright led the team in the search of a definition of critical thinking for our local needs, and in fitting an assessment to that definition. She started by eliciting a list of our favorite characteristics of previous assessments (CWA & CMA) so that those can guide the development of the new one. These included:

• Educational for faculty as well as students
• Real application of discipline tools—authenticity
• Interdisciplinary— shared responsibility, faculty conversations
• Included students at all levels
• Useful, informing instruction and policy
• Interesting, engaging
• Embedded in regular courses
• Produced (in the case of CMA) spin-off into community activity
• Focus on process as well as product
• Scored by locally designed rubrics that reflect our own priorities
• Limited to a few traits—less is more; can't do everything at once

We noted that CWA and CMA differed in a few aspects:

These observations clarified several practical questions:

1. How generic / specific do we want to be this time?
2. Where and when should students complete the work?
3. Should we work with a common theme?
4. How much guidance should we offer through scaffolding and prompts?

Discussion shifted to the elements of critical thinking that we want to measure, eliciting the following points:

• Definitions of Critical Thinking often mark dialogue between:
  • Deductive & inductive reasoning
  • Ability to draw inferences from information
  • Ability to stay open to new information
• Definitions also involve degrees of thinking about thinking—unconscious or deliberate metacognitive activity. How important is it for a critical thinker to be conscious of thought processes?
• Need to find the middle ground between the chaos of uncertainty and the self-absorption of opinion
• Facione's triad: information, application, disposition.

We closed the afternoon working in groups to scan and report on key readings in the substantial packet of materials that included: definitions of critical thinking, cognitive taxonomies and dialogues, teaching and assessment methods, and scoring procedures. (Packet and notes available on request).

Wednesday, May 21
Morning Consolidation of CT focus, key categories. Method experimentation. Afternoon Decisions on method, schedule, division of responsibilities

In the morning, Dr. Wright led discussion based on the previous day's ideas and the night's reflections. Several team members anchored comments in the language of CCC General Education goal 3, and others suggested potential rubrics. Discussion veered back and forth between rubric categories (necessarily theoretical) and methods (fundamentally practical). Controversies emerged about the following issues:

• Whether hard sciences could use the same assignment as other disciplines
• Whether a common theme would fuel community conversation or inhibit embedding
• Whether problem solving was central to critical thinking
• The role of ambiguity and reflection in critical thinking
• Standardization vs discipline-based variability in the assignment

Some controversies were resolved by reference to our CCC General Education goals, others by practical considerations, and others were left unresolved.

Working with suggestions contributed by Peter and Evelyn, we developed the following rubric-like template for the critical thinking process and hence for the common assignment:

1. GIVEN: a situation (text, case study, graphic, table, TV clip, etc) presenting information. Students should demonstrate accurate perception of the information, define a question, select key pieces of information in response to the question, find their way through the ambiguities.
2. WHEREAS: analysis and structuring of information. Students should find the appropriate correlations, models, analogies, equations, patterns and connections in order to structure an argument using the selected information.
3. THEREFORE: conclusion. Students should reach a decision, a judgment, or a statement of meaning based on the argument.
4. AND. . . : evaluation of process. Students should do some of the following: reflect on their reasoning and how confident they feel in their conclusion, show signs of understanding how they reached their conclusion, identify further information they might need to develop the argument, identify ways in which their conclusion might indicate further questions, assess their engagement with the assignment.

Dr. Wright suggested that we try out this template using (for step #1) articles she supplied from the Washington Post, and that we do so from four disciplinary perspectives: humanities, social sciences, natural sciences/ math, and professional programs. We divided into groups, not necessarily by our usual affiliations, and drafted assignments over lunch.

In the afternoon, Carmen presented a different kind of assignment in which students get a list of six characteristics of critical thinkers and then are asked to display those characteristics in a brief essay about one of six topics. With this model in mind, the groups reported on the assignments they had designed from the four perspectives. They differed in some details but each was able to use the four steps productively. Most started with a question that was raised but not answered by the given article : Are the news media reliable in reporting on the Iraq war? How likely are we to see democracy emerge in Iraq? What can we know about the condition of lakes in Connecticut? Can SARS be contained? From that point, the differences were primarily in language, with possibilities listed below:

• Step 1— describe observations, select key details relevant to the question,
• Step 2— make connections, list 3 points pro and 3 points con, develop a thesis, develop an hypothesis
• Step 3— develop the argument, make a presentation, make a judgment or conclusion, test the hypothesis and draw a conclusion, explain an answer to the question,
• Step 4— evaluate your argument, describe how confident you are in it, identify missing information or remaining questions, suggest larger implications of the argument.

Throughout this exercise, the key problem was how much scaffolding the assignment should provide. If the assignment outlines the process explicitly, it may measure students' ability to follow directions rather than their ability to do their own critical thinking. But if the instructions aren't clear, students may not understand what we're looking for. And the more open-ended the assignment is, the more dependent it becomes on writing skills. This balance remained undefined, but we agreed to develop a list of our own characteristics of critical thinkers (like the one Carmen presented) followed by two topics that teachers can choose between: one using a text-based article and the other a graphic and tabular representation of information. Both will ask students to follow a single set of instructions based on our template. The template will be simplified into accessible language that clarifies what we're looking for but doesn't lead the students in rigid steps. Teams will work over the summer on selecting the two sources of information and on developing the characteristics, assignment, and scoring rubric.

The following further logistical decisions emerged:

• Teachers will present the assignment during 2 classes, for a half an hour each time. In the first class, they'll introduce the information source, discuss the characteristics of critical thinking, and explain the importance of critical thinking within their field and in the relevant course. In the second class (a week later, to give students—including those who were absent the first day—time to read the material), the teachers will give the full assignment and students will complete it in class.
• The sources of information (article, tables, etc) should be short and in accessible language. They should be on topics that would be relevant to as many fields as possible. They should be of general interest but avoid emotional triggers.

Finally, Evelyn presented a tentative schedule of events for the year:

Late September: Casino Critico. A pizza party at which students earn points toward pizza slices by participating in critical thinking games: number sequences, analogies, figuring out magic tricks, throwing balls into category holes, calculating cars in traffic jam or marbles in bottle, etc. Observers take notes on student responses at each booth, participants complete brief questions on their scorecards at each booth, and participants complete an exit questionnaire about critical thinking before getting pizza. Occasional drawings for King Critico, etc. Use the event as PR for the later debates, as opportunity to bring students' ideas into rubric design, as groundwork for later embedded assignment.

October, November Public debates on civil liberties issues: Security vs liberty (Homeland Security focus); freedom of speech vs incitement (rap lyric focus). Students submit notes focusing on critical thinking.

October, November A few courses pilot the embedded assignment and we pilot the scoring rubric. Find and fix problems.

November, December Recruit colleagues for the embedded assignment. Explain and respond to their concerns. Prepare materials and have project ready to run by return in January.

January, February Teachers assign the questions and collect samples.

March We score, sort & analyze data, interpret results

April We prepare report and recommendations. We distribute them at public lunch discussion. Participating teachers distribute reports to their students.

The team discussed and agreed to the schedule and divided up responsibilities for summer work (* indicates instigator):

• Choosing text-based article, preferably with pictures & graphs: Terry*, Kurt, Jose, Ellie.
• Choosing graphic-based information source: Janet*, Peter, Carmen, Wink
• Listing (based on our agreements) CT characteristics, clarifying assignment, developing rubric: Evelyn*, Terry, Kathy, Lynn
• Planning Casino Critico: Peggy*, Ellie, Jose, Lynn, Evelyn (especially for finances). Help from Sharon Rackozy via Janet.
• Designing the IT report that grows out of CMA: Lynn*

We collected home e-mails, and Evelyn will use the list to negotiate a time for our first September meeting. Valedictions and silly gifts ensued, with hopes for relaxing summers.