Programme for International Student Assessment (PISA)
Random

Click random to get a fresh chapter.

Scientific literacy according to PISA

 

The components of the PISA scientific literacy framework

 

The PISA scientific literacy domain refers to an individuals’:
… scientific knowledge and use of that knowledge to identify questions, acquire new knowledge, to explain scientific phenomena and to draw evidence-based conclusions about science related issues; their understanding of the characteristic features of science as a form of human knowledge and enquiry; their awareness of how science and technology shape our material, intellectual and cultural environments; and their willingness to engage with science-related issues, and with the ideas of science, as a reflective citizen.

The definition includes knowledge of science, which refers to the knowledge of the natural world across the major fields of physics, chemistry, biological science, Earth and space science, and science-based technology, and knowledge about science, which refers to the knowledge of the means (scientific enquiry) and the goals (scientific explanations) of science.

The PISA scientific literacy assessment items required students to identify scientifically oriented issues, explain phenomena scientifically, and use scientific evidence. These three competencies were chosen because of their importance to the practice of science and their connection to key cognitive abilities such as inductive and deductive reasoning, systems-based thinking, critical decision-making, transformation of information (e.g. creating tables or graphs out of raw data), and thinking in terms of models and use of science.

Using scientific evidence
- Interpreting scientific evidence and making and communicating conclusions
- Identifying the assumptions, evidence and reasoning behind conclusions
- Reflecting on the societal implications of science and technological developments

Explaining phenomena scientifically
- Applying knowledge of science in a given situation
- Describing or interpreting phenomena scientifically and predicting changes
- Identifying appropriate descriptions, explanations, and predictions

Identifying scientific issues
- Recognising issues that are possible to investigate scientifically
- Identifying keywords to search for scientific information
- Recognising the key features of a scientific investigation

Scientific issues must lend themselves to answers based on scientific evidence. The competency identifying scientific issues includes recognising questions that it would be possible to investigate scientifically in a given situation and identifying keywords to search for scientific information on a given topic. It also involves recognising key features of a scientific investigation; for example, what things should be compared, what variables should be changed or controlled, what additional information is needed, or what action should be taken so that relevant data can be collected.
Identifying scientific issues requires students to possess knowledge about science itself, and may also draw on students´ knowledge of science.
Students demonstrate explaining phenomena scientifically by applying appropriate knowledge of science in a given situation. The competency includes describing or interpreting phenomena and predicting changes, and may involve recognising or identifying appropriate descriptions, explanations, and predictions.
The competency using scientific evidence requires students to make sense of scientific findings as evidence for claims or conclusions. The required response can involve knowledge about science or knowledge of science or both. Students should be able to assess scientific information and produce arguments based on scientific evidence. The competency may also involve: selecting from alternative conclusions in relation to evidence, giving reasons for or against a given conclusion in terms of the process by which the conclusion was derived from the data provided, and identifying the assumptions made in reaching a conclusion. Reflecting on the societal implications of scientific or technological developments is another perspective of this competency.

Scientific knowledge
As noted previously, scientific knowledge refers to both knowledge of science (knowledge about the natural world) and knowledge about science itself.

Knowledge of science
Clearly only a sample of students´ knowledge of science could be assessed in any one PISA assessment, and the focus of the assessment is the extent to which students are able to apply their knowledge in contexts of relevance to their lives. The assessed knowledge was selected from the major fields of physics, chemistry, biology, Earth and space science, and technology according to the following criteria. Items had to be:
- relevant to real-life situations scientific knowledge differs in the degree to which it is useful to the life of individuals;
- representative of important scientific concepts and thus have enduring utility; and
- appropriate to the developmental level of 15-year-old students.

The four content areas defined within knowledge of science
The four areas represent knowledge required for understanding the natural world and for making sense of experiences in personal, social and global contexts. For this reason the PISA framework uses the term “systems” instead of “sciences” in the descriptors of the content areas. The intention is to convey the idea that citizens have to understand concepts from the physical and life sciences, Earth and space science, and technology in different contexts.

Physical systems
- Structure of matter (e.g. particle models, bonds)
- Properties of matter (e.g. changes of state, thermal and electrical conductivity)
- Chemical changes of matter (e.g. reactions, energy transfer, acids/bases)
- Motions and forces (e.g. velocity, friction)
- Energy and its transformation (e.g. conservation, dissipation, chemical reactions)
- Interactions of energy and matter (e.g. light and radio waves, sound and seismic waves)

Living systems
- Cells (e.g. structures and functions, DNA, plant and animal)
- Humans (e.g. health, nutrition, subsystems [i.e. digestion, respiration, circulation, excretion, and their relationship], disease, reproduction)
- Populations (e.g. species, evolution, biodiversity, genetic variation)
- Ecosystems (e.g. food chains, matter and energy flow)
- Biosphere (e.g. ecosystem services, sustainability)

Earth and space systems
- Structures of Earth systems (e.g. lithosphere, atmosphere, hydrosphere)
- Energy in Earth systems (e.g. sources, global climate)
- Change in Earth systems (e.g. plate tectonics, geochemical cycles, constructive and destructive forces)
- Earth’s history (e.g. fossils, origin and evolution)
- Earth in space (e.g. gravity, solar systems)

Technology systems
- Role of science-based technology (e.g. solve problems, help humans meet needs and wants, design and conduct investigations)
- Relationships between science and technology (e.g. technologies contribute to scientific advancement)
- Concepts (e.g. optimisation, trade-offs, cost, risk, benefit)
- Important principles (e.g. criteria, constraints, innovation, invention, problem solving)

The general meanings of the two categories.

Scientific enquiry
- Origin (e.g. curiosity, scientific questions)
- Purpose (e.g. to produce evidence that helps answer scientific questions, current ideas/ models/theories guide enquiries)
- Experiments (e.g. different questions suggest different scientific investigations, design)
- Data (e.g. quantitative [measurements], qualitative [observations])
- Measurement (e.g. inherent uncertainty, replicability, variation, accuracy/precision in equipment and procedures)
- Characteristics of results (e.g. empirical, tentative, testable, falsifiable, self-correcting)

Scientific explanations
- Types (e.g. hypothesis, theory, model, scientific law)
- Formation (e.g. existing knowledge and new evidence, creativity and imagination, logic)
- Rules (e.g. logically consistent, based on evidence, based on historical and current knowledge)
- Outcomes (e.g. new knowledge, new methods, new technologies, new investigations)

The PISA Scientific Literacy Scale

PISA categories of knowledge about science

There were two knowledge domains defined in scientific literacy – knowledge about science and knowledge of science.

Contexts for the PISA scientific literacy assessment

 

- Support for scientific enquiry

Students show they can:
- Acknowledge the importance of considering different scientific perspectives and arguments.
- Support the use of factual information and rational explanations.
- Express the need for logical and careful processes in drawing conclusions.

Measures include: questions on support for scientific enquiry (that were integrated into the assessment of science performance); general value of science; personal value of science.

Student beliefs about learning science

Students believe they can:
- Handle scientific tasks effectively.
- Overcome difficulties to solve scientific problems.
- Demonstrate strong scientific abilities.

Measures include: questions on self-efficacy in science; self-concept in science.

Interest, engagement and motivation in science

Students show that they:
- Indicate curiosity in science and science-related issues and endeavours.
- Demonstrate willingness to acquire additional scientific knowledge and skills, using a variety of resources and methods.
- Demonstrate willingness to seek information and have an ongoing interest in science, including consideration of science related careers.

Measures include: questions on interest in learning science topics (that were integrated into the assessment of science performance); general interest in science; importance of learning science; enjoyment of science; instrumental motivation to learn science and future-oriented science motivation.

Responsibility towards resources and environments

Students show they can:
- Show a sense of personal responsibility for maintaining a sustainable environment.
- Demonstrate awareness of the environmental consequences of individual actions.
- Demonstrate willingness to take action to maintain natural resources.

Measures include: questions on responsibility for sustainable development; awareness of environmental issues; level of concern for environmental issues; optimism for the evolution of selected environmental issues.

Support for scientific enquiry

General value of science
PISA collected information on students´ perception of the general value of science. A strong general value of science relates to students valuing the contribution of science and technology, for understanding the natural and constructed world, and for the improvement of natural, technological and social conditions of life. Students were asked to indicate their level of agreement
(strongly agree; agree; disagree; and strongly disagree) with the following statements:
- Advances in science and technology usually improve people´s living conditions.
- Science is important for helping us to understand the natural world.
- Advances in science and technology usually help improve the country.
- Science is valuable to society.
- Advances in science and technology usually bring social benefits.

These statements were used to create an index on the general value of science.

Personal value of science
The majority of Australian students perceived science as generally important. The personal value of science index considered whether science was important in a student´s own life and affected their behaviour. The personal value of science related to students´ value of science and the scientific advancement of understanding the world for their own sake, and the usefulness of science and scientific inquiry at an individual level.

Five items were used to measure perceptions of the personal value of science. Students were asked to indicate the extent of their agreement (strongly agree; agree; disagree; and strongly disagree) with the following statements:
- Some concepts in science help me see how I relate to other people.
- I will use science in many ways when I am an adult.
- Science is very relevant to me.
- I find that science helps me to understand the things around me.
- When I leave school there will be many opportunities for me to use science.

Self-efficacy in science

 

Students´ feelings of confidence about a specific problem are important to an individual´s capacity to solve that problem. Eight items measuring the student´s confidence to perform science related tasks were used to assess self-efficacy in science for PISA 2006. These statements cover important themes identified in the scientific literacy framework. Students were asked how confidently they could:
- Recognise the science question that underlies a newspaper report on a health issue.
- Explain why earthquakes occur more frequently in some areas than in others.
- Describe the role of antibiotics in the treatment of disease.
- Identify the science question associated with the disposal of garbage.
- Predict how changes to an environment will affect the survival of certain species.
- Discuss how new evidence can lead you to change your understanding about the possibility of life on Mars.
- Interpret the scientific information provided on the labelling of food items.
- Identify the better of two explanations for the formation of acid rain.

 

Students were asked to respond using a scale ranging from “could do easily”, “could do with some effort” to “would struggle on my own” to “couldn’t do it”. Figure above shows the proportion of students in Australia who thought they could do it easily, or with some effort, and the proportion that would struggle or not be able to do it, along with the average scores for students in each category. On most items students showed reasonable self-efficacy.

- General interest in learning science

Eight items were used to measure how interested students are in learning science as a subject.

Students were asked to indicate their level of interest on a range of science topics on a four-point scale (high interest, medium interest, low interest and no interest). The science topics were:
- physics,
- chemistry,
- astronomy,
- geology,
- biology of plants,
- human biology,
- ways scientists design experiments, and
- what is required for scientific explanations.

- Enjoyment of science

Students were asked to think about their attitudes to science and learning science and indicate their agreement (strongly agree; agree; disagree; and strongly disagree) with the following statements:
- I generally have fun when I am learning science topics.
- I like reading about science.
- I am happy doing science problems.
- I enjoy acquiring new knowledge in science.
- I am interested in learning about science.

- Concern for environmental issues

Students were asked to indicate their perceptions of a range of environmental issues by marking one of four categories: this is a serious concern for me personally as well as others; this is a serious concern for other people in my country but not me personally; this is a serious concern for people in other countries; and this is not a serious concern to anyone. The environmental issues were:
- Air pollution.
- Energy shortages.
- Extinction of plants and animals.
- Clearing of forests for other land use.
- Water shortages.
- Nuclear waste.