The website was created by Métisz | Contract
Huszti, A., Revákné Markóczi, I. & Máth, J. (2016). Investigating the relationship between problem-solving and metacognition among university students entering university education. Hungarian Educational Research Journal, 6(1), 68-80, DOI: 10.14413/HERJ.2016.01.05.
Investigating the relationship between problem-solving and metacognition among university students entering university education
Anett Huszti[1], Ibolya Revákné Markóczi[2] & János Máth[3]
Understanding and solution of science problems, advanced-level mastering of science knowledge and methods cannot take place without this ability. However, research has suggested that in recent years students starting their BSc in science do not possess the necessary level of this skill (Radnóti, 2010). Finding the reasons is a multi-faceted and complex task. This study describes an investigation whose purpose was to explore the interrelationships between problem-solving and metacognition. The investigation involving 156 students entering BSc education in biology at the University of Debrecen was carried out in autumn 2014. Problem-solving was measured using a series of tasks on biology and general, everyday life topics while metacognition related to problem-solving was assessed using the MCAI (Metacognitive Assessment Inventory) standard devised by Cooper and Urena (2009). To establish correlations, students were grouped into clusters based on their results in the two tests, which allowed us to perform a more detailed and more thorough analysis. The study presents the results for the various clusters and the conclusions that could be drawn from them.
Keywords: freshmen, metacognition, problem solving, MCAI
The results of research into problem-solving and the present situation of science education reveal the fact that teachers do not make proper use of the potential of the development of problem-solving in their classes.  Students’ unsatisfying science knowledge and attitude, however, is the result of several shortcomings. Finding the reasons is a multi-faceted and complex task.  But the problem does exist, as suggested by the results of several investigations. 
International surveys have been measuring students’ applicable knowledge since the 1970s. The first investigations related to sciences were the surveys IEA (International Association for the Evaluation of Educational Achievement) in 1983 followed by TIMSS (Third International and Science Study) in 1995. TIMSS focused on the level of development of the ability of problem-solving thinking as well as the application of acquired knowledge. The results clearly showed that 18-year-old Hungarian students leaving secondary school education cannot keep pace with the other students from other participating countries given the fact that they finished 18th among the 21 participating countries. In the late 1990s OECD (Organisation for Economic Cooperation and Development) launched the triennial international monitoring PISA (Programme for International Student Assessment) survey that measures 15-year-old students’ ability to apply their knowledge to real-life situations, in other words, the extent to which they possess the knowledge and skills that will equip them for full participation in society, further study, or finding employment (Engler, 2013).
The low level of the science knowledge of Hungarian students entering higher education and application of that knowledge has been attested by a body of research in Hungary (Tóth & Radnóti, 2009; Radnóti, 2010a, 2010b; Revákné & Radnóti, 2011).  However, diagnosis alone is not sufficient, further investigations are necessary to uncover the causes of these results. The research to be described below, which investigated the factors affecting the science problem-solving performance of students who have just started higher education, is such an investigative analysis. Our research focused on examining and analysing the relationship between problem-solving and the awareness of problem-solving.
The role of metacognition in problem-solving
Research into the role of metacognition in problem-solving has reported that the level of metacognition is a good predictor for students’ performance in problem-solving (Lester, 1994; Desoete, Roeyers & Buysse, 2001; Veenman, 2005). According to Goos, Galbraith and Renshaw (2000) the main connection points of metacognition and problem-solving are the collection, selection, and sorting of relevant information necessary for the understanding and representation of the problem, the conscious planning of problem-solving as well as the conscious monitoring of the various steps of the solution process.
Artz and Armour (1992) have demonstrated that the causes of students’ failure in problem-solving are their inability to monitor their mental processes during the problem-solving process and their lack of knowledge the various stages of problem-solving. This is the reason why the process of problem-solving needs to be taught explicitly. This is particularly true for problem-solving in science subjects where the success or failure of performing an experiment often depends on whether the students are familiar with the various steps of experimenting analogous to the logical course of problem-solving and their interrelationship.
In relation to the development of declarative metacognition Lin (2001) concluded that, in terms of convictions about knowledge and knowledge of subject matter, it is implicit methods that are the most fruitful. According to Péntek (2000), implicit methods involve contextual diversification and structuring of content, in other words, the relation of as many diverse situations as possible (tasks tied to various subject contents as well as in diverse situations) in order to decide and recognise whether the solution to the given problem is correct or not, how and in what diverse ways we can think about a problem, and whether we understand the task or not, etc. Lin (2001) also notes that we cannot make much progress in the field of development by collecting metacognitive phrases pertaining to the way a problem is solved (e.g., I read the task, I understand it, consider whether it is correct or not, etc.) and making students consciously cram and apply them. In fact, it can lead to a slowing down of the problem-solving process and a decrease in performance.
Tóth (2007), too, highlights the central role of metacognition in problem-solving. He defines problem-solving, as applied thinking, as a complex cognitive process in which critical thinking, guidance of the restructuring of already existing knowledge structures, and creative thinking, in charge of gaining new knowledge, are equally important. Among the cognitive components of critical thinking he emphasises analysis, evaluation, and the search for correlations. He identifies three cognitive components of creative thinking: synthesis, elaboration, and recognition of relations, including metacognitive knowledge. In his interpretation metacognitive knowledge covers all those higher-level checking and monitoring processes that are used during problem-solving and decision making. These elements are the following: choice of the strategy necessary for problem-solving, the ways of analysing and evaluating data, and comparison of the ways a solution can be worked out.
It is also still a bone of contention whether to perform development of metacognition through subject-related content or independently from it.
According to the theory of „conceptual shift” development of cognitive abilities takes place while mastering some subject-related knowledge at school, which can then be transferred to other fields of science and can also be well applied in solving problems of everyday life (Carey, 1985). This is true for the development of metacognition, too. The Hungarian educational system today prefers acquisition of knowledge within the framework of lessons, which is why we will have to keep this option in mind during development. The common experimental nature of science subjects makes it possible for problem-solving strategies acquired in biology, physics, and chemistry lessons to be used in the process of solving problems in other sciences as well. Thus development of metacognition linked to subject-related content allows an internal transfer among science subjects and an outward-pointing external transfer in solving problems of everyday life.
The results of the research presented here demonstrate that the success of problem-solving is influenced by how conscious learners are during the solution process, whether they are aware of what they are doing, why and how they are doing it, and whether they can monitor the solution process. This makes the raison d'etre of implicit and explicit methods that develop awareness justified in lessons.
Measuring awareness of problem-solving
In his monography on metacognition published in 2007, Csíkos summarised the issues belonging to evaluation of metacognition and concluded that at present there is no uniformly accepted appropriate method or measuring device to evaluate metacognition. In this work he mentions that previous studies have identified three stages of measuring components of procedural metacognitive knowledge: (1) before, (2) during, and (3) after solution of the task. Among the methods of pretests he lists use of questionnaires and oral interviews, the think-aloud technique and observation in the „on-line” stage, and finally the use of questionnaires and interviews as posttests.
Most of the investigations relating to the awareness of science problem-solving have been performed /carried out within the framework of the subjects of physics and chemistry and the measuring devices used there were similar to the measurement techniques described in Csíkos’s study (2007) (Meijer et al., 2006; Anderson & Nashon, 2007). In terms of the purpose of measurement these techniques belonged to one of two large groups: (1) measurement of various components of metacognition during problem-solving (declarative, procedural, and conditional knowledge, planning, monitoring, and evaluation), (2) measurements related to the consciousness of the various components of the problem-solving process.
An example of measuring the various components of metacognition on-line is the complex measuring system MSA (Metacognitive Skills and Knowledge Assessment) developed by Desoete, Roeyers and Buysse (2001), in which students compared their predictions about their own performance in the field of problem-solving in mathematics with  the actual results of the solutions. During the measurement, both oral interviews and written tests were employed and seven parameters (declarative, procedural, and conditional knowledge, and the skills of prediction, planning, follow-up, and evaluation) within the two main metacognitive components (knowledge and skills) were studied. During measurement of declarative knowledge students were asked to choose the five most difficult and the five least difficult tasks just by looking at them, before actually solving them. Then they solved all the tasks. During the evaluation the students’ predictions were compared with the solutions and were given scores accordingly. The researchers followed the logic of declarative knowledge measurement in the case of procedural metacognition, too, asking students to explain how they solved the tasks whereas during measurement of conditional knowledge students had to answer the question why they perceived the tasks as easy or difficult to answer. Examination of the reliability of the MSA method (Cronbach α for the seven parameters: 0,60 – 0,87) revealed medium reliability, while examination of its validity (correlation between first and last measurements: r = 0,81, p < 0,005) showed correct validity.
When measuring consciousness during the solution of problems in physics, Meijer et al. (2006) studied not only the specific components of metacognition (planning, monitoring, and evaluation) but also the concrete extent of awareness that accompanies the various steps of the problem-solving process (orientation, elaboration, and implementation). The think-aloud method was used for measurement, which had medium reliability.       
Measuring awareness of science problem-solving can be done using questionnaires, too. This method is preferred if a great number of students (over 100) are examined at any given time. However, their use is questionable on several counts. One problem is that students may manipulate in judging the truth content of the statements, over- or underestimating themselves. Another problem is that statements about procedural metaknowledge can often be regarded as meta-knowledge about metaknowledge. Forming the actual picture is also made difficult by students’ inability to understand the content of the statement or to treat the generality of the statements properly (Csíkos, 2007). However, by taking account of correcting potential sources of error the questionnaire format can be improved and turned into a reliable and valid measurement instrument.
Among the questionnaires mention should be made of the Metacognitive Activities Inventory (MCAI) by Cooper and Urena (2009), designed to measure awareness of chemistry problem-solving. It has an internal structure that enables detailed exploration of the awareness of the problem-solving process. These structural elements can easily be compared with the details of performance of the actual problem-solving and can be used to interpret them, which is why we chose to use this questionnaire in our study to measure awareness of science problem-solving.
The study
Purpose of the study
The investigation was part of our research carried out between 2012 and 2015 to study how awareness of problem-solving affects science BSc students’ performance in problem-solving. To measure awareness we looked for a measuring instrument which would enable us to explore the metacognitive features of as many elements of the problem-solving process as possible. The MCAI (Metacognitive Activities Inventory) questionnaire designed by Cooper and Urena (2009), which was originally designed to measure the metacognitive features of chemistry problem-solving, turned out to be such a device.
The purposes of the investigation described in the study are: (1) to analyse the performance of the students in science problem solving tasks and (2) to explore the relationship between students’ performance in the actual problem-solving task and awareness.
The main questions of the study: (1) What is the students’ performance like in the test measuring the process of problem-solving? (2) What is the extent of students’ awareness related to problem-solving based on the MCAI questionnaire? (3) What correlation can be found in the examined sample between the process of problem-solving and the results achieved on the metacognitive questionnaire?
Sample and methodology
The study was conducted at the University of Debrecen in September 2014 and involved a total of 55 biology students (BSc level) who had just entered higher education. Our first-year university students write an aptitude test every year when they start their university studies to assess knowledge of content and skills necessary for progress. This aptitude test was completed in 2012 with a series of problem-solving tasks as well as with an MCAI questionnaire to measure awareness of the problem-solving method.
The students needed to judge statements in the MCAI questionnaire as pertaining to themselves based on a five-point Likert scale. The questionnaire, designed by Cooper and Urena (2009), was originally used to measure metacognitive regulatory skills pertaining to first and last year chemistry students’ chemistry problem-solving. When compiling the content of the questionnaire, the main goal was to include items related to activities and skills necessary for successful problem-solving. The items, originally 53, were collected by an expert panel consisting of professors, students, and psychologists and were pre-tested. The main test contained the final questionnaire consisting of 29 (21 positive and 8 inverse) items that students needed to judge using a 5-point Likert scale. The scores given to the items demonstrated how often the students used the particular activity during problem-solving. 
Cooper and Urena (2009) subjected the final version to structural analysis during which they looked for internal structures and dimensions. However, these dimensions could not be demonstrated during the investigation conducted using factor analysis, which the authors interpreted as the high interdependency of (the elements of) metacognitive regulatory skills and the difficulty involved in their separation from one another.
This measuring instrument also provided the students’ professors with important information which they could use to conclude how the students perceived their own problem-solving activities and skills. The measurement results made the approach of professors to students more conscious as, in view of the deficiencies, they changed their teaching methods to make problem-solving and learning more efficient.
Simultaneous completion of the questionnaire by the students, following proper instructions, took 20 minutes in our study.
In later parts of our study we wished to analyse what kind of relationship there is between the outcomes of problem-solving and awareness of the problem-solving process. To this end, we compiled a series of problem-solving tasks consisting of 6 tasks, which were analysed using the activities in the MCAI questionnaire, thus the two outcomes were comparable.  The first two of the six tasks were calculation exercises requiring biological knowledge. These were followed by two tasks that used well-known examples of medical science and finally by tasks related to solving problems from everyday life. Every singly task contained similar questions asking for the existence of the components of Polya’s cognitive model:
a) What is the main problem of the experiment?
b) What is the problem of the experiment?
c) Preliminary information (knowledge, correlations, formulae, equations, algorithms)  necessary to solve the question. Demonstrate what you can, using drawings or graphs.
d) Make a plan to solve the problem of the question. (Describe the way you would solve the problem. If you have more than one plan for solution, describe them each separately.)
e) Carry out your plan(s) of solution, solve the task, explain and interpret the solution. (Explain why you chose the given solution.)
f) Check the solution. Has the main problem of the experiment been answered? If not, how would you think go on? 
Evaluation of the solutions took place according to the following points of view:
1. The problem
1.1. Understanding the problem:   yes: 1   no : 0
1.2. Representation of the problem (can describe the understood problem, can illustrate it using drawings and graphs): yes: 1 no: 0
1.3. Collecting all information relevant to the solution (even if one piece of information is missing, the solution cannot be accepted): complete: 1, incomplete or something missing: 0
1.4. Demonstrating the relationship between pieces of relevant information: yes: 1 no: 0
2. Plan of solution and implementation
2.1. Formulating a hypothesis:  yes 1;  no: 0
2.2. Description of plan of solution accompanied by reasoning: yes, there is one, and the reason explains the chosen plan: 1; the reasoning does not explain choice of plan or there is no reason or plan, or there is neither: 0
2.3. Shift between different plans of solution: more than one plan of solution with reasoning: 1;  not more than one plan of solution: 0
2.4. Implementation of solution: the solution is correct and shows a relationship with the plan of solution: 1;  the solution is not correct: 0
3. Evaluation
3.1. Reasoning behind the solution: yes, and it is correct: 1; yes, but not correct or   there is none: 0
3.2. Comparison of the solution with the problem: yes, and it is correct: 1; yes, but it is not correct, or there is none: 0
3.3. Restructuring of the problem-solving process for a better solution. At the end of the solution, the learner realizes that there are other solutions, or their solution proves to be incorrect, finding out, they restart the process: 1; no other perspective: 0
To analyse the results the SPSS 17.00 programme was used. To analyse the problem-solving tasks and the MCAI questionnaire, ANOVA (The one-way Analysis of Variance) and descriptive statistics, while for the analysis of correlations correlational study was employed. To enable a deeper understanding of correlations between problem-solving skillfulness and the level of awareness affecting it, cluster analysis, similar to pretesting, was used (Revákné, Máth, Huszti & Pollner, 2013).
Results and discussion
First of all reliability of the problem-solving tasks and the MCAI questionnaire was determined. Cronbach-α was 0.83 for the problem-solving tasks and 0.81 for MCAI (the latter verifying the values 0.6--0.87 in Cooper and Urena’s (2009) measurements). 
To answer the first question of the study (What is the performance of the students like in the test measuring the problem-solving process?), we analysed students’ performance on the problem-solving tasks. When evaluating the tasks, we employed the scoring method described above in each task. We found the means of the components of the process as pertaining to the sample studied (Figure 1) and the values of significance between them. The mean of the total score of the students studied on the problem-solving tasks was 29, which represented 44% of the tasks solved - a result below the average.
The students involved in the study had the least difficulty with understanding and representing the problem and with formulating hypotheses. At the same time, collecting the preliminary information necessary for the solution was difficult for them as was finding potential correlations between them. In lack of these the course of the solution may become problematic, relevant information and knowledge that could lead to successful solutions may be missing. This is attested by the fact that in the studied sample making a plan of solution had a significantly lower mean compared with hypothesis formulation, and the level of the stages following planning was really low. Naturally, the weakness of the latter stages may have other reasons as well, one being that education does not place due emphasis on developing these skills. The most conspicuous problem is that the students in the sample cannot see the correlations and cannot think flexibly or creatively to find new paths and alternatives after failure to solve a problem. This means that our students are not experienced in solving problems since education fails to pay due attention to developing problem-solving skills (or if it does, it is ineffective). There was a significant difference between the means of the various dimensions (F(10)= 609,233, p = 0,000). Exceptions to these were the differences between the means for hypothesis-making and problem representation (p = 0,515) and those between the means for finding relevant information and reasoning for the solution (p= 0,140). All in all we can say that the first third of the problem-solving process (understanding and representation of the problem, formulating hypotheses) does not cause any problems for the students studied, however, they are poor at performing later stages (planning, implementation, evaluation) of it that are very important from the aspect of finding a solution. 
In later stages of the study we tried to find out about the extent of students’ awareness of problem-solving using the MCAI questionnaire. The measurement in our sample had been  preceded by a pretest in 2012, which allowed us to make changes to the questionnaire to make it more effective (it is necessary to explain the purpose and structure of the questionnaire to the students in advance, explain the meaning of the statements and make students practise correct judgement of the statements) (Revákné, Máth, Huszti and  Pollner, 2013). Cooper and Urena (2009) performed factor analysis to find out whether distinct blocks of statements comprising the components of the problem-solving process can be identified in the questionnaire. However, neither they nor we were able to demonstrate presence of these, a fact that can be explained by the interdependency of (the elements of) metacognitive regulatory skills and the difficulty involved in their separation from one another. Thus the performance achieved on the MCAI questionnaire can be described in terms of the total score achieved, but this result in itself is not informative from the aspect of our research. It gains meaning only in comparison with performance achieved  on the problem-solving tasks. The total mean score of the students involved in the MCAI questionnaire sample was 97, 86% of the maximally achievable 113 points. Based on this, the students in question show good awareness of the problem-solving process. Like in the pretest, the question still remains: how real is this value is and what is the extent of students’ under- and overestimation?. We get a better chance of answering that question by examining the correlation between the results in the problem-solving process and the results of the MCVAI questionnaire and assuming that it really does exist.
Thus, in order to answer the third important question of the study (What correlation can be found in the examined sample between the process of problem-solving and the results achieved on the metacognitive questionnaire?), we performed a correlation analysis between the various dimensions measured in the problem-solving tasks and the various statements of the MCAI questionnaire.
The following significant correlations were found between the 11 dimensions of the problem-solving tasks and the 29 statements of the MCAI questionnaire (Table 1).
Table 1.  Significant correlations between the various dimensions of the problem-solving tasks and the statements of the MCAI questionnaire
Dimensions of the problem-solving tasks
Statement of the MCAI questionnaire
R (Pearson’s correlation coefficient)
Understanding the problem
I read the statement of a problem carefully to fully understand it.
0, 265*
I determine what the goal is.
I try to determine the way and form in which the results will be evaluated.
I find relations between concepts, quantities, and factors before attempting a solution.
Representation of the problem
I identify the goal of a problem.
Once I have understood the problem, I try to gain further knowledge and experience to secure successful solution of the problem.
I find relations between concepts, quantities, and factors before attempting a solution.
I do not read every single unnecessary detail before starting to solve the problem
- 0,230**
Hypothesis making
I identify the goal of the problem.
Once I have understood the problem, I try to gain further knowledge and experience to secure successful solution of the problem.
I do not read every single unnecessary detail before starting to solve the problem.
Making a plan of solution
It makes me feel good to find the mistakes and be able to correct them.
I write down everything during the solution process.
I find relations between concepts, quantities, and factors before attempting a solution.
I do not read every single unnecessary detail before starting to solve the problem.
- 0,240**
Execution of the solution
It makes me feel good to find the mistakes and be able to correct them.
I make sure that my solution actually answers the question.
I do not reflect on how it feels to find the answer.
- 0,187*
Reasoning for the solution
It makes me feel good to find the mistakes and be able to correct them.
If I think I can solve a problem, I do not spend time gaining new knowledge.
- 0,182*
Restructuring of the solution---
I write down everything during the solution process.
I check that my result is an answer to the problem.
I always think about whether what I am doing at any time is necessary and good for the solution of the problem.
I always analyse every step of the solution process.
I find the starting point through detailed analysis of the problem.
One I have solved a problem, I do not spend time looking for new knowledge.
- 0,196*
If I think I can solve a problem, I do not spend time looking for new knowledge.
- 0,189*
Table 1. shows that there is a significant correlation, albeit not strong, between the total score of the components of the problem-solving tasks achieved by the students in the sample and those of the students on the MCAI questionnaire. This means that the MCAI questionnaire predicts, although weakly, performance in the problem-solving tasks. It remains to be seen how much this correlation could be improved if students were more experienced in completing questionnaires on metacognition and on the conscious analysis of the metacognitive processes of problem-solving. One thing is certain: in the studied sample, it is the dimensions of the problem-solving tasks with higher means (understanding, representation of the problem, hypothesis formulation, making a plan of solution) that show a higher correlation with the metacognitive activities on the MCAI questionnaire, while in the weaker dimensions (collecting relevant information, finding relations between pieces of information, finding relations between plans of solution, etc.) no correlation was found between the two variables. There is one exception to the latter group, namely restructuring of the solution, in other word, finding another path to the solution after one has failed. The statements provided by the students on the MCAI questionnaire suggest that this is a met cognitively complex, multi-step activity, the possession of which could indeed improve the success rate of problem-solving.
As a result of the „two-step” cluster analysis two clusters were identified in the MCAI questionnaire, those of the underestimators (78 students perceived the activities represented by their statements as medium difficult or even weaker and the other consisting of the overestimators (76 students, mostly giving the items 4 or 5 points).  There was a marked difference in items 1 („I read the statement of a problem carefully to fully understand it”) and 5 („Once a result is obtained, I check to see that it agrees with what I expected”) as well as the  statements with a negative content 23-29 („One I have solved a problem, I do not spend time looking for new knowledge”, „If I think I can solve a problem, I do not spend time looking for new knowledge”;  „I do not reflect on how it feels to find the answer”;  „If I do not know exactly how to solve a problem, I immediately try to guess the answer”;  „I do not read all unnecessary details before starting to solve the problem”), („I spend little time on solving problems especially if I know I cannot do them”),  „”If a problem takes several attempts and I cannot get it right, I get someone to do it for me and I try to memorise the procedure”). In these cases the over-estimators almost entirely rejected or failed to use these activities to solve the problems, thus giving themselves remarkably lower scores compared with the previous statements. We examined the correlation between the scores on the problem-solving tasks and those on the questionnaire on metacognition. We found a negative correlation (r = - 0.334; p < 0.01) in cluster two (over-estimators), that is, students who overestimated themselves on the questionnaire on metacognition performed poorly in carrying out the problem-solving tasks.
We can conclude that, in the studied sample, the MCAI questionnaire was a weak predictor of the performance achieved on the problem-solving tasks. The reasons for this may be: 1) The activities described in the MCAI questionnaire are not specific to the components of the problem-solving process. 2) Students are not experienced in performing and judging similar metacognitive activities. 3) Students are not realistic when judging the statements on the MCAI questionnaire. 4) In contrast to the measurements of Cooper and Urena (2009), the MCAI questionnaire is not a perfect predictor of performance on the problem-solving tasks.
The background of our study was provided by the poor biology knowledge and problem-solving of students entering science BSc-level education measured in the past couple of years. Since science problem-solving is an indispensable element of learning and researching science subjects, it is imperative that it should be raised to the required level in the students concerned. However, to be able to do this, it needs to be clarified what reasons lie behind the poor performance and also what we can do to improve these results. Of course, the success of problem-solving is affected by several factors. In our study we undertook to find correlations between problem-solving and its awareness assuming that the success of problem-solving is greatly affected by the awareness and metacognition of problem-solving.  To this end, we compiled a problem-solving task series consisting of 6 items which we then analysed for the presence and level of the components of the problem-solving process. To measure awareness of problem-solving we used the MCAI questionnaire, a standardised measuring instrument of metacognition designed by Cooper and Urena (2009). The results show that the studied students are indeed weak in terms of their performance on the results of the problem-solving process and are also inexperienced in the field of awareness of the problem-solving process.  Using the instrument measuring metacognition, we were able to find only a weak correlation between the success and awareness of problem-solving. In this way our hypothesis was only partly proved. We are convinced, however, that this weak correlation is not necessarily the fault of the measuring instrument. It can be due to students’ inexperience in metacognitive activities and in judging similar questionnaires. To prove this assumption, further studies will be necessary. It is also necessary to find out what factors are responsible for the unsatisfying level of students’ problem-solving skills.
The research was supported partly by the Hungarian Scientific Research Fund. Grant No. OTKA K-105262.
Anderson, D. & Nashon, M. (2007). Predators of knowledge construction: Interpreting students’ metacognition in an amusement park physics program. Science Education, 91(2), 298–320.
Artz, A. F. & Armour-Thomas, E. (1992). Development of a cognitive-metacognitive framework for protocol analysis of mathematical problem solving in small groups. Cognition and Instruction, 9. 137–175.
Carey, S. (1985): On the origin of casual understanding. In. Sperber, D. Premack, D. és Premack A. J. (szerk.). Casual cognition. Clarendon Press, Oxford, 268–302.
Cooper, M. & Urena, S. (2009). Design and validation of an instrument to assess metacognitive skillfulness in chemistry problem solving. Journal of Chemical Education, 86(2), 240–245.
Csíkos Cs. (2007). Metakogníció. A tudásra vonatkozó tudás pedagógiája. [Metacognition. Pedagogy of knowledge about knowledge] Budapest: Mûszaki Kiadó.
Desoete, A., Roeyers, H. & Buysse. A. (2001). Metacognition and mathematical problem solving in grade 3. Journal of Learning Disabilities, 34, 435–449.
Engler Ágnes (2013). Az élethosszig tartó tanulás gender-szempontú megközelítése. [Life-long learning relate to gender] Iskolakultura, 23(2), 3-12.
Goos, M., Galbraith, P. & Renshaw P. (2000). A money problem: A source of insight into problem solving action. International Journal of Mathematics Teaching and Learning, 80, 128–151.
Lin, X. D. (2001) . Designing metacognitive activities. Educational Technology Research and Development, 49, 23–40.
Lester, F. K. (1994).  Musings about mathematical problem solving research: 1970–1994. Journal for Research in Mathematics Education, 25(6), 660–675.
Meijer, J., Veenman, J. & Hout, W. B. (2006). Metacognitive activities in text-studying and problem solving: Development of a taxonomy. Educational Research and Evaluation, 12, 209–237.
Péntek I. (2000). Tudatos és implicit metakognitív folyamatok a problémamegoldásban. [Awareness and implicit metacognitive procedures in problem solving. ] Erdélyi Pszichológiai Szemle, 1(2), 85–97.
Radnóti K. (2010a): Elsõéves fizika BSc-s és mérnökhallgatók fizikatudása. [Knowledge of  bachelor freshmen in science engineering and physics.] A fizika tanítása, 18(1),  8–16.
Radnóti K. (2010b). Elsõéves hallgatók kémiatudása. [Knowledge of freshmen in chemistry.] A kémia tanítása, 18(1), 13–24.
Revákné Markóczi I. & Radnóti K. (2011): A felsõoktatásba belépõ hallgatók biológiatudása egy felmérés tükrében. [A survey relate to knowledge of freshmen starters in biology.] A biológia tanítása, 19(2), 3–13.
Revákné Markóczi I., Máth J., Huszti A., & Pollner K. (2013). A természettudományos problémamemgoldás metakogníciójának mérése a felsõoktatásban. [Assessment of metacognition in science problem solving at high level.] Magyar Pedagógia, 113 (4), 221-241.
Tóth P. (2007): Gondolkodásfejlesztés informatika órán [Development of thinking in informatics lesson.. Iskolakultúra, 17(6-7),  47-65.
Tóth Z. & Radnóti K. (2009). Elsõéves BSc-hallgatók sikeressége egy meghatározó reagenssel kapcsolatos számítási feladat megoldásában. [Successfulness of bachelor freshmen in solving of a computational task relate to determining reagent. ] Középiskolai kémiai lapok, 36(5),  375–390.
Veenman, M. V. J. (2005). The assessment of metacognitive skills: What can be learned from multimethod designs? In. Artelt, C. és Moschner, B. (szerk.): Lernstrategien und Metakognition: Implikationen für Forschung und Praxis Berlin: Waxmann. 75–97.
[1] Anett Huszti, University of Debrecen, Doctoral Programme in Educational Studies, Debrecen (Hungary),
[2] Ibolya Revákné Markóczi, University of Debrecen, Faculty of Science and Technology, Debrecen (Hungary),
[3] János Máth, University of Debrecen, Faculty of Arts and Humanities, Debrecen (Hungary),