In economic terms, Europe can survive as a “high wages/high productivity” region only with a work force having sufficient command over skills and competences in a way to tackle the future challenges of modern societies. This does not seem to be the case, according to recently published results of the OECD-PIAAC survey (OECD 2013a, b).
I. Who and what has been tested?
“The Survey of Adult Skills, a product of the Programme for the International Assessment of Adult Competences or PIAAC, assesses the proficiency of adults in literacy, numeracy and problem solving in technology-rich environments. These skills are ´key information processing competencies´ that are relevant to adults in many social contexts and work situations, and necessary for fully integrating and participating in the labour market, education and training, and social and civic life” (OECD 2013a, Chapter 2, Box 2.1.).
In 2011/2012, around 166.000 adults aged 16-65 were surveyed in 24 countries, starting with a background questionnaire which took between 30 and 45 minutes and continuing with a cognitive assessment either on a laptop or by completing a paper version. Solving these test examples took the participants on average 50 minutes. The sample size ranged from 4.469 (Sweden) to 27.285 (Canada) but was around 6.000 for most of the countries. For the interpretation of the PIAAC results it is important to take into account the fact that participating adults were born between 1947 and 1996, i. e., the former entered primary school in the early 1950s and the latter in the early 2000s – obviously under very different circumstances regarding schools and educational matters in general. “In each of the three domains assessed, proficiency is considered as a continuum of ability involving the mastery of information-processing tasks of increasing complexity. The results are represented on a 500-point scale. At each point on the scale, an individual with a proficiency score of a particular value has a 67% chance of successfully completing test items located at that point. This individual will also be able to complete more difficult items … with a lower probability of success and easier items … with a greater chance of success.” (OECD 2013a, p. 60). In the literacy example below, a person with a proficiency score of 240 has a chance of 67% to answer correctly. There are 5 proficiency levels for literacy and numeracy (and 3 for problem solving) ranging from the lowest performance, “Below level 1, score range lower than 176 points”, to “Level 1, 176-225 score range”, “Level 2, 226-275 points”, “Level 3, 276-325 points”, “Level 4, 326-375 score range”, and finally to the highest “Level 5, higher than 376 points”.
II. Examples for Literacy (Level 2) and Numeracy (Level 3)
In the assessment part of the survey, respondents have to solve test examples in the area of literacy, numeracy and problem solving. To give an idea of the degree of difficulty of the problems, two examples are shown below for illustrative purpose: In the literacy example (on level 2), the test-taker has to find a phone number to contact the organiser of a “Lakeside fun run” under the “contact us” button.
Literacy (Level 2 example)
In the numeracy example (on level 3), the respondent has to read a simple graph and identify the period with declining births in the US.
Numeracy (Level 3 example)
III. Key results
A) Large proportions of adults struggle with the most basic skills
As the graph and the table below reveal, in many countries the share of under-performing adults (on level below 1, 1 and 2) is larger than the share of good performers (on level 3, 4 and 5). Both groups are divided by the line in the middle of the graph. In the literacy domain, for example, the problem group is as large as 69.7% of the working age population in Italy, ranging to 27.6% in Japan. The “hard-core” problem group with proficiency level below 1 and 1 ranges from 27.7% to 4.9%. This means in absolute terms that, for example, 27.8m persons in Germany, 104.6m in the US, and 23.4m in France are performing on a level below 3.
Referring to the literacy example “Lakeside fun run” above, these results have to be read in the following way, taking again the German case: 18.3m adults performing on level 2 in Germany can solve this literacy problem with a chance of 67%, only. 9.5m persons with a proficiency level of 1 or below 1 can solve it with a chance of 0.27% or even lower (for persons with below 1 outcomes). This means: around 10m Persons in Germany find it extremely difficult to solve a problem as simple as the “Lakeside fun run”, some 37m for the US – really a worrying result for politicians and society in general!
Shifting to the numeracy example “US birth rates 1957-2007” from above (level 3) and taking again the German case, 16.7m adults (level 2) can respond correctly in only 27% of all cases and 9.9m even with a lower chance of some 6% or lower. Again, an extremely worrying result, given the very simple nature of the graph! To sum up the results for the literacy and numeracy domain: For Germany, around 20m adults find it extremely difficult to even solve very simply problems on level 3, for the US these figures sum up to around 90m adults.
And in the problem solving domain, the group of low performing adults (below level 2) is even larger, ranging from 80.8% in Poland to 55.9% in Sweden.
B) Low mean differences but high within variances
A rather striking result throughout the OECD Survey of Adult Skills is the close range of mean differences in many fields like country or educational differences on the one hand and rather large differences within the area under consideration like countries or educational levels.
The graph below shows country means of numeracy proficiency scores and the distribution between low and high performance levels measured by the 5th and the 95th percentile of the distribution. In the numeracy domain, for example, the highest country-average proficiency level of 288 points in Japan differs from the lowest of 246 points in Spain by some 42 points, only. But the distance between 5th and the 95th percentile is on average 171 points, i. e. four times larger, with a maximum in the US of 188 points and a minimum of 142 points in Japan. What does this mean? An easy interpretation does not be to hand easily.
The figure above also reveals the fact that there is no simple correlation between mean and variance. Finland and Japan, for example, have both high mean scores but in the former country this mean goes together with a higher variance and in the latter country with a lower.
C) Educational Attainment matters
On average, the differences in literacy proficiency scores between persons with below upper-secondary and tertiary education is 51.2 points (297.0 minus 245.8), ranging from 31.8 in Cyprus to 67.4 in the US. So, education matters! But again, the differences within the educational attainment groups are larger than between them (see graph below showing results for Austria and the OECD average). This leads to the rather striking result that, for example, many persons with low educational attainment levels have higher reading proficiencies than persons with academic qualifications.
D) Migration matters, too
In countries like Sweden, Finland and the Netherlands, the differences between persons born in the country or foreign born is in the range of 40 to 50 points, in stark contrast to a county like Australia, where the difference amounts some 13 points. Again, an extremely interesting result.
Given the fact that the test examples are rather easy up to a proficiency level of 3, it is particularly worrying that so many are not able the respond correctly. And given the fact that the skills and competences which are needed for correctly answering the examples are highly relevant for the world of work but the whole society in general, the large sector of poor performing participants should be even more worrying, for employers, employees, politicians, and citizens.
OECD (2013a), OECD Skills Outlook 2013, First Results from the Survey of Adult Skills, Paris/Brussels, 8 October 2013, http://dx.doi.org/10.1787/9789264204027-en .
OECD (2013b), The Survey of Adult Skills. Reader´s Companion, Paris, 8 October 2013