Knowledge exaggeration: Are you sure you really know it?

 I’ve seen the CVs of some Americans that wildly exaggerate their Japanese ability, designating knowledge of a few phrases as “working-level competence”. And then, conversely, the CVs of some Japanese fail to even mention their English language skills unless they have a TOEFL or IELTS score to back it up. If they do include any information, they err on the side of downplaying their ability.

   This is the problem with self-reporting: people self-report differently, which is to some extent a reflection of cultural norms. This skews research data relying on self-reports, and this can particularly be an issue when attempting to make cross-cultural comparisons. I recently read an article published by Hana Vonkova, a specialist on comparative education reporting behavior, along with Ondrej Papajoanu and Jiri Stipek in Volume 49 of the Journal of Cross-Cultural Psychology that investigates these differences, using a devilishly clever method called the overclaiming technique (OCT).

OCT is a means to measure students’ knowledge exaggeration (KE) and knowledge accuracy (KA). In the study done by Vonkova and her colleagues, they examined how 275,904 15-year-olds in 64 countries responded to questions that asked about their familiarity with various mathematical concepts. There was one hitch, though – some of the mathematical concepts were fabricated. The data came from the OECD’s Programme for International Student Assessment (PISA), which measures students’ reading, math, and science knowledge and skills.

485,000 students in 68 countries took one of three versions of the test. Two versions included a section that asked students about their familiarity with 16 mathematical concepts. Excluding those who didn’t answer all or some of the questions, 275,904 in 64 countries responded to this section, using a 5-point scale: 1) never heard of it; 2) heard of it once or twice; 3) heard of it a few times; 4) heard of it often; and 5) know it well, understand the concept. Unbeknownst to them, three of the 16 terms were fictious. Can you spot the fakers?

• exponential function

• divisor

• quadratic function

• proper number

• linear equation

• vectors

• complex number

• rational number

• radicals

• subjunctive scaling

• polygon

• declarative fraction

• congruent figure

• cosine

• arithmetic mean

• probability

Knowledge exaggeration was calculated as the proportion of hits (PH) (the percentage of existing mathematical concepts that a respondent claimed to be familiar with) and the proportion of false alarms (PFA) (the percentage of non-existent concepts they claimed to know). Knowledge accuracy is the difference between PH and PFA. There were four categories of responses: 1) hits: it’s real and they recognize it; 2) false alarms: it’s not real but they recognize it; 3) misses: it’s real but they don’t recognize it; 4) correct rejections: it’s not real and they don’t recognize it.

The three foils were proper number, subjunctive scaling, and declarative fraction. Across the board, 83% of students claimed a higher familiarity with real mathematical concepts than made-up ones, but proper number proved to be a particularly tricky decoy. Although roughly 20% of the students said they had never heard of it, about 40% said they heard of it often or know it well!

Respondents fell into four basic groups: PA+HE: Positive accuracy and higher exaggeration; NA+HE: Negative accuracy and higher exaggeration; PA+LE: Positive accuracy and lower exaggeration; and NA+LE: Negative accuracy and lower exaggeration. There were clear cultural tendencies. Examples of countries with a higher index of accuracy and a lower index of exaggeration than average are Korea, Spain, Japan, Vietnam, and Germany. The following lists show which countries’ respondents demonstrated the highest accuracy and which displayed the lowest exaggeration.

Top 20 highest countries for accuracy

1.    Korea

2.     Shanghai-China

3.     Chinese-Taipei

4.    Japan

5.    Spain

6.     Macao-China

7.     Latvia              

8.     Estonia

9.    Vietnam

10.  Hungary

11.  Hong Kong-China

12. Greece

13. Czech Republic

14. Turkey

15. France

16. Belgium

17. Perm (Russian Federation)

18. Israel

19. Russian Federation

20. Germany

(Adapted from Vonkova, Papajoanu, and Stipek, 2018)

Top 20 lowest countries for exaggeration

64. Sweden

63. Iceland

62. Finland

61. Ireland

60. New Zealand

59. Switzerland

58. Netherlands

57. Austria

56. United Kingdom

55. Spain

54. Argentina

53. Liechtenstein

52. Uruguay

51. Korea

50. Germany

49. Japan

48. Vietnam

47. Austria

46. Slovak Republic

45. Costa Rica

(Adapted from Vonkova, Papajoanu, and Stipek, 2018)

The US was ranked 24 for both measures, making it close to the top third for both higher accuracy and higher exaggeration.

Facility in communicating in a culture often involves an often unconscious adjustment of utterances to ascertain the core message. The rankings obtained by Vonkova, Papajoanu, and Stipek help indicate the direction of the adjustments to be made and are super fascinating to contemplate – and I say this with no exaggeration!