Home / BLOG


We cannot solve our problems with the same thinking we used when we created them-Albert Einstein

Let’s resolve to work for better thinking in 2018.

Amid the noise and clutter of the holidays, the confusion of international and domestic politics, the flash of new tech tools and toys, and the static of trivial tweets, nothing calms, focuses, sorts, stores, sheds, shields, and serves us better than good thinking.

It will take strong thinkers to navigate through the problems and opportunities facing us today and in the future. We give ourselves the best opportunity for success in 2018 when:

  • we can differentiate between unfounded opinions and well-reasoned points of view
  • we can explain our perspectives based on sound reasons and good evidence
  • we maintain a positive thinking mindset that prizes truth-seeking and fair-mindedness.

“We cannot solve our problems with the same thinking we used when we created them.” — Albert Einstein

Strength in thinking skills and mindset is a global educational goal. Democracies flourish, professions serve, businesses excel and student achieve their career goals when people make well-reasoned, reflective and purposeful judgments about what to believe or what to do in all aspects of their lives. 

Join Insight Assessment, the worldwide leader in the measurement of reasoning skills and thinking mindset, in making 2018 the “Year for Better Thinking.”  Let’s all do whatever we can to encourage and to celebrate well-reasoned decision making and effective problem solving in education, business, and government, and in our personal and civic lives. 

Call out and praise good thinking. Don’t settle for myopic, biased or careless thinking. Listen carefully. Evaluate and understand the sources of information. Consciously think about the quality of your decision making. Cultivate positive habits of mind. Practice your skills of analysis, evaluation, inference, inductive and deductive reasoning. Be thoughtful.

Critical Thinking: What It Is and Why It Counts 

Follow our blog, Thinking INSIGHT to explore ways we can all develop and benefit from stronger thinking in the classroom and workplace.

Contact us to learn more about our thinking and reasoning assessment solutions.

INSIGHT Decision Making Requires Skills and Mindset

Business success depends on building a more diverse, more productive workforce.

Today HR departments need to be sensitive to unconscious biases to be sure that candidate selection is not skewed. The challenge is how. Standard screening processes have proven unreliable.

Musicians are auditioned behind a screen so that they will be judged purely by the quality of their music. It's not about who they are--it's about what they can do (the quality of their musicianship).

Obviously businesses can't audition candidates behind a screen, so objective ways to determine the potential of candidates must be based on more relevant metrics.

What could be more effective--and useful--than a "blind hiring" process that starts with an objective assessment of the cognitive skills and motivational attributes that predict professional success.

Metrics that focus on the strengths and weaknesses of individual candidates, not their resume or demographic data, are powerful tools for your decision-making.  

Incorporating INSIGHT Business Professional into your hiring process enables you to identify and target highly skilled individuals with the professional mindset and problem-solving skills to achieve company goals. INSIGHT is calibrated to assess and report on 15 thinking skills and attributes such as problem analysis, evaluation of options, anticipating risk, commitment, motivation, tolerance and focus that predict workplace success. Reports provide numerical scores and recommended performance analysis.

Businesses need objective actionable data delivered in a cost-effective way. 

True employee assessment doesn’t have to be hard. Easy online administration through our industry leading app based technology and results sent to your inbox within minutes will take the stress out of employee assessment. Anonymous results make decision making more objective.

It’s easy to get started. Contact us today to discuss how our INSIGHT assessments can give you the objective data you need to build a workforce focused on high quality decision making.

Critical thinking definition from the APA Delphi Expert Consensus

Yes, critical thinking can and should be taught.  (See our Resources section for teaching and learning tools)

The idea that somehow critical thinking is unteachable has been disproven by decades of research.  But there are always some who initially believe otherwise.  There is no question  that each of us is born with a greater or lesser natural capacity.  As with anything, some of us can sing better or swim further than others, but that does not mean that singing or swimming cannot be taught.  It only means that one person’s highest potential is different than another’s.

Those who claim that humans cannot develop their critical thinking skills and habits of mind because they are “either born with critical thinking or not” are wrong.  The objective data disconfirm that hypothesis.  In fact, there is ample research, going back decades and across multiple academic subjects and professional disciplines, documenting that critical thinking can be learned, taught, and measured.

Is there a formula for success?

  • Teaching FOR critical thinking is not the same as teaching ABOUT critical thinking. 
  • Teaching FOR critical thinking can be achieved at every educational level and with every academic or professional subject matter. 

The three things to be sure to do are:

  1. Consistently engage the learner’s critical thinking skills by asking questions that evoke analyses, interpretations, evaluations, explanations, and inferences AND following up by demanding thoughtful and fair-minded explanation of the reasons for those judgments. 
  2. Create and sustain an environment that fosters all the positive critical thinking habits of mind, truth-seeking, inquisitiveness, open-mindedness, maturity of judgment, etc.
  3. Practice thinking about thinking, that means being reflective about how and why each important conclusion was reached, each assumption was made, and each alternative was evaluated.

The academic subject matter or the set of problems and decisions that the professional field must grapple with become the topics to which individuals and teams apply their critical thinking.  The goal is always to enable each person to maximize their natural potential. The educator’s challenge is: “Given the raw material, how much can we achieve?”

Teaching FOR thinking is like coaching a championship

Consider, as an analogy, what it takes to succeed at the highest levels of competitive athletics.  Native strength, size, agility, and coordination are very important.  So are a knowledge of the game and thousands of hours practicing and playing the game competitively.  And so is good coaching that addresses not only the physical moves, but the mental preparation and the in-game tactics and adjustments, and too, the overall game-plan (strategy) that will best position the competitor to prevail against a given opponent. Think of tennis, for example.  Now ramp it up several notches because all these pieces are essential for success at the highest levels in team sports.  To become a championship team we need every athlete on the team to be applying native talents and their critical thinking (well-trained and practiced so it can be done in real time) to the specific problems and decisions that must be made within the context of that sporting competition.  

What about critical thinking in health care, business, governmental leadership or military success? The same elements are essential (critical thinking, native intelligence, content knowledge, and experience with the problems and decisions that constitute that concerns at that level/ in that professional field.     

When we say that the data show that we can develop strengths in critical thinking in people, we are saying something akin to what a coach says when looking at a promising athlete and saying that it will be possible to coach the person, over time, into a champion.   To do this the coach (athletic or critical thinking) must work on the skills and the mindset dimensions, which go hand in hand.  But the process is not quick, nor easy.  It is a matter of exercising (not just talking about) the skills in multiple and progressively more challenging situations, and of the constant formation of the mind, inculcating and reinforcing mental disciplines including the discipline to think first, before reacting or relying on a decision heuristic unreflectively. 

Learn more

For more on critical thinking in educational settings download “Talking Critical Thinking.” For a fuller explanation of the concept of critical thinking and its relationship to the science of human decision making, download “Critical Thinking: What It Is and Why It Counts”. 

Insight Assessment provides industry leading validated objective critical thinking assessment solutions structured for educational and professional purposes. Contact us today to discuss your assessment goals.

Bulldog Artedelgatto

This is a guest post by Peter A Facione, Measured Reasons LLC.

Posing as a Midwestern housewife and using thousands of other fake accounts on Twitter and facebook, lying Russian meddlers reached 126 million Americans during the 2016 election. 

These days all of us need to defend ourselves against blizzards of false information.  We cannot permit our democracy to be buried under our own gullibility. After all, if we didn’t believe the fake news messages, we would not have re-tweeted them. Right?

So, step number one on the road to civic health is for all of us to acknowledge our problem: When we see something in the news that supports our point of view, we “like” it and, without taking sufficient precautions, we pass it along to our friends like a bad cold.

Our problem is double edged. We are drawn to agree with messages that support our point of view, and we prefer to shrug off information that disagrees with our point of view.  How easy it is now for us to sympathize with poor old Galileo. Remember when he said, “But friends and esteemed leaders, there is evidence showing that the Earth is not the center of the Universe.”  Which, of course, caused his friends to double over in laughter at his foolish denial of the “common sense” of their day; and which lead the powers that were to threaten, “Retract what you said or go to Jail.  …  Your choice.”     

Human nature has not changed much in the centuries since, at least not in our response to messages that contradict our current version of “common sense,” or that threaten to undercut the orthodoxy of the powers that be.  We do not like to be wrong!  We do not like it when people disagree with our point of view!  And we hate it when the facts get in the way of our ambitions. 

But none of us has the time or the money (except maybe the idle super rich) to check out every story we see in the media and every message that our friends retweet or post to us.  Life is complex, we are busy. What is a person to do?  

Apply the “Credibility Test”: If the source is not credible, it probably is fake news.

By using our natural critical thinking skills, every one of us can apply the Credibility Test. It is easy.  We only need to ask a few simple Yes-No questions. Every “No” answer is a warning that the source is pedaling fake news.   

1.  Is the source an expert on the topic? 

  • If not, be skeptical. When Neil deGrasse Tyson talks about the acceleration of the expansion of our Universe, I trust what he is saying.  Expertise counts! If my neighbor, a former LA Top-40 radio DJ were to tell me the same thing, I would not take his word for it.  I like the guy, but he is no astrophysicist.  Science is still science, even if people who harbor their contrary “heartfelt beliefs” do not like what the genuine experts have to say.

2.  Is the source relying on firsthand experience? 

  • If not, be skeptical. I trust what Chris Collingsworth says about professional football because I know that he played the game successfully as a professional for years.  But I do not trust what my brother Roger says about football at any level.  Sorry, Rog, you never played a single down.

 3.  Is the source speaking on the right topic?  

  • If the person is out of their element, be skeptical.  Don’t go to Chris Collingsworth for astrophysics information, and don’t go ask Neil deGrasse Tyson for advice about play calling in football.   Experts are experts only when they are on topic.  Which means, don’t buy a car because you like the movie star who is sitting behind the wheel in the TV commercial.

 4.  Is the source’s knowledge up-to-date?  

  • If not, be skeptical. A friend of mine told me that it was safer to stand still than to run during a lightning storm. I asked where the person heard that. She said back in elementary school.  But, when we looked it up, the latest science does not support that advice.  Turns out that the positrons in the Earth are drawn up toward the negatively charged electrons moving down from the storm.  The electrons jump to the closest Earth-bound thing where they can reach the positrons. Boom! Lightning! Don’t climb on top of the roof in a lightning storm or you might become the best target for the bolt, whether you are running around up there or standing still.

5.  Can the source explain the basis for their claim?  

  • If the person cannot, be skeptical.  As a rule, only trust those who can explain the reasons why their opinion is right. Don’t trust people whose only answer is “Trust me, I know.” One reason why people who had to work hard in school became some of the best teachers is because they know from personal experience how to help people learn.  One of the reasons that the greatest athletes often make terrible coaches is because they did not have to be students of the game to experience their success.  And, so, they often cannot explain the how and the why of what they want their players to do.   

 6.  Is the source truthful? 

  • If the person is a known liar, be skeptical. Seems obvious, right? Yet, we catch people lying to us about this or that, and yet we still believe them the next time they tell us something.  A person has a history of violating contracts, and yet we believe them when they tell us that they will keep their promise to us.  A person has a history of throwing others under the bus, and yet we think that we are the one person to whom they will remain loyal.  Who’s the idiot in that scenario?  The circus huckster, or us?  As politicians know, it can be tempting to tell one audience one thing and another audience another thing. Especially if we expect that neither audience is going to know what we said to the other.  Don’t trust people who will tell any lie to gain power, to be loved, to avoid rejection, to close a deal, to get their way.

 7.  Is the source unbiased? 

  • If the person is biased, be skeptical. Biases are lies we tell ourselves about other groups of people.  Like that all 70-year-old White guys are Republicans, or that all Christians are charitable. Be skeptical of opinions based on stereotypes. Just because it was a White guy who opened fire on concertgoers in Las Vegas, or blew up the Federal Building in Oklahoma, or killed nine people in a movie theater in Colorado, or attacked first graders at Sandy Hook Elementary, or gunned down 26 church goers in Texas does not mean all White guys are terrorists. 

8.  Is the source free of conflicts of interest?

  • If the person is compromising your interests to advance their own, be very skeptical. Learn to ask, “What’s in it for you?” It is one thing for a person accused of a crime to plead not guilty, it is another for us to believe him. That’s why we have the trial. How many famous people have we seen one day denying wrongdoing only to learn soon after that those tweets and press releases were just propaganda?  I expect my real estate agent, my lawyer, my banker, my mentor, and my friends to give me advice that is in my interest, not theirs. After all, they are supposed to be helping me, not using me to promote their own agendas. If your “news” broadcasts are meant to expand your ratings, gain higher revenue for commercial time, and benefit your stockholders, then you are a fake news entertainment outlet, fun to watch but foolish to believe.

9.  Is the source speaking freely? 

  • If the person is under duress, be skeptical.We have all seen the ISIS videos of captives being forced to “confess” while someone holds a knife to their neck. We all know to be skeptical of whatever that prisoner says.  The person is saying what they are forced to be saying.  Same if the person speaking is drugged, or if their loved one is being held for ransom, or if they are emotionally distraught, or deranged by starvation or disease.  Although it is a less extreme situation, we need to include under this heading the case of a person who is constrained by a legal agreement.  If you signed a non-disclosure agreement with your former employer, your contractual obligations may prevent you from providing us with all you know about a topic.

10.   Is the source mentally stable? 

  • If not, be skeptical. Sadly, we all have been drawn in by stories told to us by people whom we love but who are, unfortunately, mentally unstable. We want to help the person, we may even be torn by guilt if we do not take action.  But, when we do so, we discover that the facts are not as they were presented, and that the real story is not as we first heard it.  Was the person deliberately trying to manipulate us, or did the person really believe what they were saying?  Who’s to know.  But, as a rule of thumb, if you know your relative is a nut case, be skeptical.

If there is one theme running through these ten guidelines, it is this: Be skeptical. Apply your critical thinking skills to test the credibility of the source of the news you see or hear.

Instead of gulping fake news like a trained seal swallowing a mackerel, we all can demand full explanations, reasons, and evidence. We all can be suspicious of messages that come to us from people with something to gain by deceiving us.

And, finally, as a favor to one another, and to strengthen our democracy, we all can check the credibility of the source before we pass along something we heard at work or saw on our social media feed or the Internet.

About the author:  Peter A. Facione, PhD, is recognized internationally as an expert in critical thinking. In this essay he is speaking on that topic, based on decades of experience teaching and researching it. He can explain the basis for each of the ten recommendations in more detail, if asked.  He is speaking truthfully, he is not biased about critical thinking or the many benefits it offers to everyone. Although he holds the copyright, he offers this essay for free distribution to anyone who wishes to download a copy.  When he wrote this essay, he was not under duress and he was of sound mind. 

To download a pdf version of this blog post

Contenido relacionado

multiple devices

Many critical thinking assessment customers find that the move from traditional paper testing to electronic testing leads to greater value to students, faculty and administration.   

Insight Assessment clients can select the testing method which is most suitable for their specific test environments and needs.  Clients who do not have access to technology or who require unique controlled, proctored testing, can choose traditional paper.   The majority of our clients prefer the flexibility of anytime, anywhere testing use our industry leading app-based online option.

There are many advantages to choosing electronic testing. Insight Assessment flexible electronic testing options provide:

  • Immediate reporting advantages for electronic testing. Electronic testing makes scored results immediately available to you on your dashboard, and spreadsheets and group reports can be downloaded at your convenience; There’s no need to worry about testing data being lost in the mail.
  • Unlimited generation of group reports using our reporting system capability. Using the group indicator to identify each program allows you to prepare program level reports using our online report generator, adding to the value of your current plan. You can easily support reporting demands for overall and department level data.
  • The ability to customize your demographic profile so that you can easily record the school/program and up to ten additional data points for later analysis.
  • Identify and remove false tests that typically lower group mean scores. The electronic system records time on test and also the percent of items completed   
  • Individual reports: At your option, electronic testing will provide a personal report to each student and allow them to see it and email it to themselves at completion of the assessment.
  • Testing is always available 24/7/365 without the need to order materials in advance.  Our testing platform is now delivered through a secure app-based interface. The mobile app can be downloaded at no cost to most  iOS, Apple, and Google devices. Our technology and system support staff is available (at no additional cost) to work with your IT personnel to prepare for electronic administration. Other clients typically have students sign into labs and use provided login information to complete the assessment at specified time frames.
  • Expanded multilingual assessment options allow more accurate assessment of reasoning skills in ESL applicants or students. Insight Assessment provides assessments, interface and reports in a wide arrange of languages.
  • Expanded access to wider array of assessments, including those that incorporate improvements in the scale score reporting including the addition of Numeracy (quantitative reasoning).
  • Maintaining reporting of OVERALL scores in relationship to national percentiles Insight Assessment maintains the careful calibration of our test instruments, so national percentiles for all versions are exactly comparable, and OVERALL scores (and mean scores) are also directly comparable, allowing clients to move to electronic testing. and not lose data. 

The majority of Insight Assessment clients, large and small, test electronically.  Whether to initiate an assessment project in paper or online is often an early question in discussions with potential clients.  If you are currently considering critical thinking assessments or a change from paper to electronic administration, contact us today. Our testing specialists will help you determine if electronic testing or paper testing is the best match for your specific test environments and needs.

How to choose a thinking assessment

How to optimize your thinking data before the assessment starts

Insight Assessment Logo

Insight Assessment is committed to helping our clients achieve their assessment goals. We provide support starting with assessment selection and assessment program design through the reporting and interpretation of results.  

Fall is a busy time for our clients. They are deep into planning and implementing fall critical thinking assessment programs—or wrapping up spring assessment projects, mining their data. Researchers are writing up their results for publication, grants or doctoral proposals. Some clients are looking to optimize their ongoing assessment data management. Project leaders are using data to target the improvement of skills needing improvement. Insight Assessment is there to assist them with excellent service staff and resources. Our customers can receive the maximum benefit from their investment in assessing reasoning skills and thinking mindset. You can start with some of the free resources on our website listed below.

10 resources that make our client’s assessment process easier and more productive

 Getting new assessment projects started:

Maximizing impact of Assessment Data:

Most important for our customers:

  • Comprehensive Test Manuals (Available to clients only): for current information on test administration options, scales, score interpretation, instrument validity, norms and comparison percentiles.
  • Our dedicated customer service staff who are ready to help clients achieve their assessment goals.

Our website includes many additional resources on critical thinking, optimizing assessment design and improving reasoning. Follow our blog, Thinking INSIGHT or RSS feed to receive updates.

 If you’re not already an Insight Assessment client, contact us today to learn how we can jumpstart your assessment program.


Insight Assessment Thinking & Learning Resources

Are you interested in the importance of improving critical thinking skills and mindset? Then you’ll want to save this list.

Here are eight of Insight Assessment’s most used resources for teachers, trainers and others who are involved in developing learning outcomes assessment projects.

Feel free to download and use these teaching and training tools in your work to promote improved thinking in students and adults:

  1. Critical Thinking: What it is and Why it Counts PDF: The most recent version of Dr. Peter Facione’s essay, explores the meaning and importance of critical thinking in all aspects of life and work.
  2. Characteristics of Strong Critical Thinkers: Based on the APA Expert Consensus Delphi Report description of strong critical thinkers.
  3. Cultivating a Critical Thinking Mindset PDF: Essay suggests specific practices people can do to develop strong critical thinking habits of mind
  4. Ten Positive Examples of Critical Thinking:  10 opportunities in daily life to engage problems and decisions using strong thinking skills and mindset
  5. 18 Ways Strong Thinking Skills are Applied in Business
  6. Effective Techniques for Building Reasoning Skills: ways to engage students & trainees in successful skills development and to reinforce a positive thinking mindset
  7. Holistic Critical Thinking Scoring Rubric: A rating measure that can be used to assess observable critical thinking demonstrated by presentations, essays, projects etc
  8. Sample Critical Thinking Questions: Examples that illustrate the types of situations which might appear on a generic adult level reasoning skills test

Be sure to check out the rest of the free Resources on the Insight Assessment website. We make these teaching, training and learning tools available as part of our commitment to supporting the measurement and improvement of good thinking worldwide.

Keep in touch!

  • Follow our blog, Thinking INSIGHT, which explores ways we can all develop and benefit from stronger thinking.
  • You can keep up with the latest Insight Assessment posts about measuring critical thinking skills, mindset, leadership, and other related constructs using our RSS feed, or on social media.
Better decision-making creates better results

Identify and hire the best talent

You need to minimize the costly training time, reduce the risk of the new hire washing out and assure that new addition brings stability or improvement to team morale.

You work hard to hire the best available talent, but how do you know those new employees will become productive quickly?  How do you adapt their existing professional skills to new business needs?

Take the guesswork out of qualifying candidates

When you need to hire and then train your new employees to do the job, there’s a tool that will take the guesswork out of qualifying candidates.  INSIGHT assessment tools provide skills and mindset metrics to help you and your professionals maximize business performance by improving the quality of staff decision-making and reasoning throughout your organization. Get the reliable individual and group metrics you need to:

The right tools give you the data you need to accomplish your goals

An objective description of an individual's comparative strength in reasoning skills is a valuable aid in determining a person's capacity to benefit from educational training or to succeed in their job. INSIGHT starts with high quality, specialized thinking assessment tools for business, health care, government and education uses. Our solutions are designed take the hassle out of employee assessment:

  • Insight Assessment can work with you to identify the key metrics that impact the objectives challenging your staff responsibilities.
  • We provide full support for administration, setting up and managing your program.
  • Our highly flexible multilingual interface offers global 24/7 access for your team.
  • Reports provide comprehensive analysis of individual and group metrics for the key values you need.

In this tighter job market, you have more competition for the best talent. It's harder to hire the highly skilled and motivated candidates who are ready to succeed—and more important to prepare them to thrive in your culture.

Insight Assessment delivers the results and the data required to focus on implementing improvements. Contact us to discuss your specific needs.
Insight Assessment critical thinking tests available in Arabic, Chinese-Simplified, Chinese-Traditional, Dutch, English, Farsi, Finnish, French, Greek, Hebrew, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Norwegian, Portuguese, Russian, Spanish, Thai, Turkish & Vietnamese translations

Good thinking skills are a key ingredient to understanding ourselves and others in today's changing interconnected world.

We need to think about what we all have in common. The importance of developing open and strong thinking skills applies not just to us as individuals, but as a community, region, country and across the entire world. 

Good thinking is our strongest defense against ignorance and intolerance.

When strong thinking skills are applied with a fair and honest mindset, individuals and teams have the best chance to agree on solutions to world, national, local and our personal problems.

Good thinking skills are universal. In every society reaching an optimal problem solution requires people to:  

  • identify and analyze the problem, identifying key elements
  • assess the accuracy of facts and information sources
  • evaluate options and explain the strength and weakness of alternatives
  • draw conclusions in ambiguous contexts to determine the solution with the most likelihood of success
  • anticipate outcomes and see logical consequences in precise contexts
  • interpret and evaluate information presented in numerical formats

A strong thinking mindset is equally necessary for fair minded decision making. People who are motivated to make good decisions demonstrate:

  • a focused, organized and systematic approach to problem-solving,
  • openmindedness and tolerance, respecting diversity in perspectives, opinions,
  • thoughtfulness in decision-making with a commitment to use reasoning and fact-finding to resolve difficulties,
  • adaptablity to change,
  • self motivation and resourcefulness,
  • intellectual curiosity, following reasons & evidence wherever they may lead.

If we want to solve complex problems and propose the changes that will move our world forward, the development and application of good thinking skills and mindset attributes will be the foundation.  

Insight Assessment is passionate about the impact of growing, measuring and promoting good thinking worldwide. We are proud to count within our customers institutions and businesses who are committed to improving the quality of thinking in adults and children. Our multilingual online assessments and resources are used by education, business, healthcare, government and legal institutions to develop better curriculum, to focus training on improving thinking, and to identify the strong thinkers who will move their initiatives forward. If you want us to help bring these values to your workplace, give us a call.

Measurement tools

Many people believe that writing samples provide a good test of critical thinking.  But Insight Assessment does not use writing samples in your testing instruments.  Why is that?

We asked Dr. Peter Facione, senior researcher and author at Insight Assessment to respond:

Measurement science has developed objective methods for evaluating cognitive skills and habits of mind that are more precise, more valid, and more reliable than writing samples.

Writing prompts that invite a person to give reasons and evidence to support their analyses, inferences, explanations, evaluations and interpretations can be useful exercises for developing stronger critical thinking.  That said, using writing sample is a less than ideal method of assessing critical thinking because of inherent issues like invalidity, a lack of precision, insufficient variance, unreliability, and misrepresentation.

First, and most obviously, ratings of writing samples introduce a validity threat when evaluators conflate writing craft and a facility with rhetorical devices with critical thinking skills. (This is apart from the common human tendency to favor writing samples that present views with which the reader may agree vs. those that argue for positions with which the reader disagrees.) 

  • There are other threats to validity as well.  For example, all the ways that humans use critical thinking that do not include writing are automatically eliminated if the only manifestation evaluated is what the person is able to express in written form.  This limitation severely restricts the range of possible applications and manifestations of our critical thinking skills. 
  • Additionally, using writing samples typically does not provide the opportunity to test specific aspects of critical thinking such as its application in different reasoning contexts, empirical, comparative, ideological, and quantitative.  Nor does using writing typically permit the more detailed scrutiny of the writer’s ability to resist locking-in prematurely to a given alternative, to recognize and avoid common reasoning errors, and to overcome the tendency to misapply cognitive heuristics. 

A second area of concern is that writing based assessments do not spread out scores widely (lack variance) and those scores they do yield are imprecise.  Typically written work is scored using four or five categories in the way that a professor might assign letter grades to an essay.  While there is a rank ordering to the grades, the intervals between the grades are not necessarily uniform.  In measurement science we prefer smoothly uniform intervals between scores, which is why a large range of numerical scores, e.g. between 65 and 100, offers more precision and more variance.  The more precision and the more variance a valid and reliable measurement tool permits, the better the tool.

  • The reliability problems in the evaluation of writing samples are well documented.  That is why so much training is needed for the evaluators. In our experience even well-trained human graders using rubrics can disagree about what score to apply to written work.  We know too that human graders have difficulty reliably evaluating the writer’s critical thinking when the writer uses humor, irony, hyperbole, invective, or sarcasm. Computer can be programmed to be as reliable as trained human raters, but that is actually a rather low threshold.  Currently available computer grading algorithms are incapable of understanding what is written. The computers are not considering the quality of the critical thinking process used, instead the machines are looking only for syntactical markers, like sentence length, the frequency of the use of specific words, and grammatical construction.  

But, perhaps the most important consideration in favor of finding a better method than using writing samples comes from the very nature of what a writing sample represents.  Inevitably a writing sample is a person’s a reconstruction of their own thinking, but not the thinking itself.  Writers do not record and report their thought process as an entirely unedited steam of consciousness.  Writing samples are fabrications offered in most cases to make one appear to have been more thoughtful and more reasonable than was actually the case. 

  • Thanks to Dr. Peter Facione, a Senior Researcher at Insight Assessment and principal at Measured Reasons LLC, a Los Angeles based research and consulting firm supporting excellence in strategic thinking and leadership decision making.  Dr. Facione is the developer and author of the California Critical Thinking Skills Test family of measurement tools; his latest book is Think Critically, 2016, Facione & Gittens.

How are Insight Assessment test instruments validated?

What is the best way to assess critical thinking?

Good thinking is in demand. Download Critical Thinking Insight from your app store today:

Contact Us

* Required fields

Insight Assessment will not share your data with anyone. Click here to view our privacy statement.

[ Hide ]

Contact Us

* Required fields

Insight Assessment will not share your data with anyone. Click here to view our privacy statement.