Soft marking and academic fraud

The thing to understand about modelling and econometrics in particular is that the story often gets ahead of the analytics. Some of you may recall doing the FuelWatch debacle that the government was relying on a regression model that was ultimately shown to be incomplete, while last year the government had a very dodgy graph and regression in the Budget papers that got exposed here. So it is with the latest ‘evidence’ of soft marking at universities. The analysis is much more sophisticated than what we’ve seen out of Canberra, but again the story is running ahead of the evidence actually presented.

The paper by Gigi Foster has received massive attention in the Australian and also some publicity at Club Troppo (cross posted at Core Economics) and Andrew Norton.

GIGI Foster knows her disturbing research findings on international students won’t make her many friends. In a university sector grown dependent on international fee revenue, it might not do much to progress her academic career either.

But the audience she wants to reach is not academe but the policy-makers. It’s at this level where change could be driven to address the poor language and cultural skills she says are undermining their performance.

“It is risky for me, but it is my duty to look at this,” says Foster, a Harvard graduate who moved to Australia in 2003.

But she believes her research provides evidence that universities are too often turning a blind eye to the poor written and verbal English skills of many international students.

She says her statistical analysis reveals that international students are being allowed to underperform and this is being camouflaged to an extent by grade inflation.

At the same time, these poor English skills weigh on the results of domestic students in the same tutorials.

“I want Australian policy-makers to see what is actually happening,” Foster says.

But she believes concerns over fee revenue, sensitivities over the potential for appearing “xenophobic” and political correctness are preventing the sector from confronting the issues.

Those comments are not supported by the paper. The research question being asked is

educational equity concerns would also lead us to ask how the infusion of international and NESB students into Australian higher education impacts upon the marks of other students.

I worry when I see questions like this. I can imagine Archie Bunker asking this sort of question. If we really believe that the presence of international students reduces the marks of ‘other students’ what sort of policy conclusions should/would be adopted? Anyway, I digress. The paper takes a ‘have data, will regress’ approach to things in the way economists often do things.

The sample consists of undergraduate students at two Australian universities (UniSA and UTS) in Business faculties during the autumn and spring semesters of 2008 and 2009. There are 74,276 observations. Not that many if you think that there are four semesters and full-time students should be taking four subjects per semester (so 74,276 divided by 16 is a rough estimate of the number of actual students about 4,600 or so). What we don’t know is the break-up of those students. For example, how many are full-time or part-time? We also don’t know what subjects they are doing. As I suggest below, this could be an important factor. I have also some difficulty understanding the grade Foster uses in the analysis. She speaks of a ‘tutorial mark’ – is that a grade earned in tutorials or is it the in-term assessment?

She then slices and dices the data into a number of categories: international v non-international and NESB v Non-NESB. Unsurprisingly she reports that non-international students get an average grade of 61.69% and international students get 57.4% while non-NESB get 62.23 and NESB get 58.34. Those average marks are a bit lower than I expected but the relativities are about right. The ‘lowness’of the marks could be driven by the fact that some students fail and get to repeat subjects. But Foster doesn’t tell us what she has done with those students or how she has handled drop-outs (probably nothing). As Andrew Norton explains this is not evidence of soft-marking. But then Foster goes into a whole bunch of regression analysis. When the paper gets re-written hopefully she will explain her regressions and variable definitions much better. In the meantime, there are some strange results (or at least my understanding of the results could be enhanced with more explanation). For example in table 3 in some regressions it looks like new students (first years?) get higher marks ceteris paribus than everyone else. It also appears that domestic NESB students get lower marks ceteris paribus. That is very counter-intuitive. So that would be the kids of migrants who have come to Australia and speak a foreign language at home but have come up through the Australian education system.

We then get to the crunch. Figure 3 shows the distribution of international students in tutorials.

If I understand the figure correctly, about 15% of tutorials have no international students in them, while some (a very small number) have close to 100% international students. This raises a question about tutorials; are these tutorials compulsory or voluntary and are students assigned to a tutorial class or can they self-select into a tutorial? Table 5 is the controversial table; columns (3) and (4) to be precise.

In Table 5 columns (1) and (2) we see the performance of non-NESB students as a function of domestic NESB and international NESB students enrolled in the course and in the tutorial. An increase in domestic NESB students in tutorials leads to non-NESB students getting better marks on average while an increase in international NESB leads to non-NESB students getting lower marks on average.

In table 5 columns (5) and (6) we see the performance of domestic NESB students as a function of domestic NESB students in the course and in tutorials and international NESB students in the course and in tutorials. Here we see the more domestic NESB students in tutorials the better the students perform on average. This could be interpreted as going to tutorials leads to better marks.

The controversial bits are columns (3) and (4). The performance of international NESB students is better the greater the percentage of international NESB students are enrolled in the course. Some might interpret this as evidence of soft-marking in high-revenue subjects. But that is being polite – this appears to be evidence of corruption; some students are being singled out and given higher marks than they would otherwise receive. I do not believe for one moment that interpretation to be correct. Unfortunately many others do.

We need to know a lot more about the data before coming to any conclusions. For a start, what about self-selection? Are we simply observing the fact that students self-select into particular subjects that have a reputation for being soft? Or conversely self-selecting into subjects that emphasise skills that international students might do better at like maths or lots of memorisation? To my mind that almost certainly explains columns (5) and (6). In the worst case scenario, we may simply be observing a tutor-effect. There are two possible explanations here: first international NESB students could self-select to those tutors who are known to be a soft touch or second international NESB students could self-select to those tutors who better understand their language difficulty (or indeed speak their language) and who are better able to accurately gauge student understanding and performance.

The problem with the Foster analysis is that she assumes a grade to the curve effect and assumes her results show ‘a course-wide downward adjustment in the grading standards applied to these student’ when her analytics do not exclude a whole bunch of other less sinister explanations.
Update: Gigi Forster responds in comments below.

This entry was posted in Uncategorized. Bookmark the permalink.

83 Responses to Soft marking and academic fraud

  1. I guess being American she assumes grading to a curve because that’s the way it’s always done over there?

    I don’t remember curve grading being very common when I was at uni, most assessments were graded numerically. Then again I did real courses (engineering), not sure how they grade the fake visa subjects.

  2. Bh

    It is not just international students. Years ago, I was heavily involved in the deaf community. There was a push from the govt to get the deaf to go to university, so a B.Ed in LOTE was set up. The deaf doing this course were given a year’s credit for “life experience”. After graduation, students were encouraged to go on for their Masters. While I don’t recall how many did this, or whether they even finished, I recently came across a submission on the ‘net written by one of these Masters students. The language and reasoning skills shown in this submission was Grade 2 level.

  3. Peter Patton

    Sinc – there is – as you know – a hell of a lot in this, and much politically combustible. But just for now, are there any academics left who consider their jobs to be teachers/scholars, or has the whole of academia now merely about that fricking idiotic word, equity?

    FUCK equity. Teach me n-dimensional space, the great vowel shift, the efficient markets hypothesis, Real Analysis, the origins of Greek tragedy, the nitrogen cycle, human anatomy, plant physiology…

  4. Fred

    I studied Computer Science a couple of decades ago and am now doing an accounting degree where foreign students make up over 80%.

    One thing I have noticed is the number of ‘easy’ marks that are now given away. Whereas in my previous degree, turning up to a tutorial and doing homework attracted no marks, today you often receive 10-25% of your overall grade from tutorial attendance, participation and homework.

    We also have online tests where there is an unlimited number of attempts, designed to give students 5-10% of their grade.

    I think it is ridiculous, but I’m not complaining because I find it a very easy way to achieve As.

  5. Pingback: Andrew Norton » Blog Archive » The increasing pass rate for international students

  6. Andrew Norton

    Students are very assessment driven. Unless there are marks for attendance they won’t turn up.

    I’m not against multiple attempts for online questions – that’s one advantage of them. Some things students should keep trying until they get it right.

    The single-attempt assignments and exams will still drive overall grades.

  7. Peter Patton

    Fred

    We also have online tests where there is an unlimited number of attempts, designed to give students 5-10% of their grade.

    I recently finished a course, which during the semester had five online multiple choice quizzes of 25 questions each. You could take as long as you liked with each, but combined they only accounted for 10% of the entire semester’s grade. That’s 125 multiple choice questions that I would not have attempted with it being compulsory. I think it was a very strong positive educationally. I actually spent a lot of time on those 125 questions, which turned out to be useful study for the exams.

    As a pedagogical device, I give a tick to these in-semester little quizzes.

  8. daddy dave

    are there any academics left who consider their jobs to be teachers/scholars

    Plenty.
    The diversity push is coming from administrators, not the academics themselves. Most adacemics enjoy teaching and love teaching smart, motivated students.

  9. C.L.

    I have some experience (limited) at two universities when it comes to tutoring and marking. As the newbie, I quickly became amazed at the corruption, which masqueraded as – what to call it? – social work, I guess. We’d have these end-of-sem ‘moderation meetings’ whose key purpose, it quickly became apparent, was to protect the bell curve. Foreign students figured prominently in the individual case discussions.

    ‘I see you’ve given Joe Mohammed Blow a high 3.’ [Say, 40 percent fail – and that was being generous].

    Me: ‘Yes, it was a 1500 word paper. He submitted one and half pages of jibberish that made no sense, as either answer or English.’

    ‘But he showed he understood the question and he attempted to answer it?’

    Me: ‘Um, I guess.’

    Long story short:

    Mo Blow is bumped up to a just-pass.

    There were many Mo Blows and a large percentage of them fails but this could never be conceded or allowed – because it would be bad for the university’s business and the HoD’s reputation for ‘successfully’ ‘teaching’ students.

  10. Infidel Tiger

    I thought all the foreign students came here because a degree from Ausralian Tertiary Factory was worth more than one from Sumatra Tech School? Surely if we hand them out like drivers licences we’re screwing ourselves?

  11. Tillman

    You need Harvey Mansfield and his ironic grading

  12. daddy dave

    Surely if we hand them out like drivers licences we’re screwing ourselves?

    Yes.
    Australian Universities are slowly destroying their reputation. They’re flushing a hard-earned long standing global reputation down the toilet for a few lousy bucks. It’s pathetic.
    Any bets on how many Asian students will be coming here in 20 years time?

  13. I thought all the foreign students came here because a degree from Ausralian Tertiary Factory was worth more than one from Sumatra Tech School?

    No, they come here because after 4 years of continual habitation in Australia they are eligible for permanent residency.

  14. war+hog

    Same experience as C.L. here.
    I was a tudor for two years 15 years ago and even then, we were told to give soft passes to anyone who attempted the work. Failing students affected KPIs and budgets – so it was a no no, unless the twerp handed in nothing at all.
    More recently I have had friends study or work at universities where much of the class could not understand the lectures at all, and had tutorial groups of all foreign students where the tutor would translate the course into their native tongue.
    Group-based assignments are quite regular at business schools too, which allows the one or two NES students to be carried by the English speakers.
    The data in this report cant show that sort of corruption – but it is deliberate and it is systemic.
    Universities in Australia are rapidly losing their credibility.

  15. .

    C.L., et al.,

    I actually worked in a place dominated by accountants, who are notorious for being hard markers and strict with deadlines, so out culture was if you can’t read it, fail them, they are here to get a degree in English. A Win! for that place. A good culture existed.

    I had a low fail rate in high ESL classes because I was fair. They are admitted to the country with “language certificate” they must get out of the claw machine or as a lucky door prize. I had high pass rates in even 2nd year subjects with large cohorts because I gave a shit…

    I also did promise not to fail people if they attended all tutes and did all of their homework. It worked – their assignment and exam marks always put them up in credit to distinction grades alone.

    Ultimately it was up to me what grade a student got. Sometimes if a student made a marked improvement in the final I’d reweight the marks as I saw fit.

    I actually had a lot of power except for the moderation committee or whatever it was called. I told them to get stuffed basically: the bell curve is bullshit, or that cohorts got high grades when they deserved it or low grades when they did sweet FA or were given the green light to further study when they should have been held back – ESL, uni or high school. I never changed the grades I gave out.

  16. Sid Vicious

    Ted Steele felt the very cold temperature of a person frozen out when he was dumped and ostracised by his colleagues for questioning the level of English when marking the papers of overseas students.

    An unhealthy nexus exists between cash strapped universities and cashed up overseas students.

  17. Sinclair Davidson

    Sid – those two instances look very bad. But let me say this, I too would have sacked Steele and Firns and several other acedmics who have appeared in the press as ‘whistleblowers’ over the years.* Not because they highlighted problems, but because they did not follow university procedures and brought their institutions into disrepute.

    Unfortunately too many academics engage is what I call academic vigilantism. So they catch a student cheating or plagiarising or something. Rather than go through the proper university procedures they take it upon themselves to met out some punishment. The student then arrives in the HoS’s office all lawyered up and the university clearly in the wrong and clearly in breach of contract has to back down. HoS is absolutely furious and tells academic that they are in the wrong and their vigilantism has caused the student to get off scott free and then academic who cannot see that they have done anything wrong goes to the press. Big scandal ensues. It is extraordinarily frustrating for university management. We have all these academics crapping on about students not understanding English and cannot follow instructions, when in fact they themselves cannot follow simple instructions like, do not deviate from university procedures when dealing with cheating/plagiarism etc.

    * If these individuals had gone through the proper channels they would have whistleblower protection.

  18. Peter Patton

    An unhealthy nexus exists between cash strapped universities and cashed up overseas students.

    Buttressed by two very thick layers of propagandists who prowl the campus, sniffing its lecture and seminar rooms, the writings and speech of its employees – and even students – for any whiff of questioning that nexus.

    One layer of propagandists – as dd pointed out above – is snuggled within the university bureaucracy. It has its own language, filled with words like “diversity”, “dialogue”, “unhelpful”, “conversation”, “inappropriate”, “inclusive”.

    The second layer is found within academe itself, but its tentacles spread far and wide through the government bureaucracies, research funding bodies, and beyond. This second Equity octopus sucks on not only the same language as the Diversity bureaucrats, but high falutin’ notions such as “Indigenous knowledges”.

    Just watch the Diversity Divas and the Equity Octopus call in the cavalry to shut down this “conversation” about the wisdom of our universities relying so heavily on foreign NESB students.

  19. There’s not just grade inflation for international students, ‘establishment’ students (students who’re well connected thru their parents) are getting passes even tho’ they spend their time getting pissed.

    There seems to be an idea that these people are paying customers and that what they’re buying is a degree, not tuition.

  20. Sinclair Davidson

    There seems to be an idea that these people are paying customers and that what they’re buying is a degree, not tuition.

    I’ve noticed this attitude amongst many staff members but never the administration. What administration, in my experience, do however is insist that academics be more professional – so grade in a timely manner etc.

  21. Peter Patton

    Adrien

    Really? Given all domestic university students occupy the same status – paying customers – what pull could ‘establishment’ parents have? By definition, the number of these kids must be tiny, so how have you managed to discern this practice within such a tiny group?

    I imagine Uncle Dot, the Economagician, would tell Lord Didley-Pomps to Get Stuffed, if his lordship phoned demanding his son and heir – Tarquin Didley Pomps – receive a passing grade, even though he had scored only 38/100.

    And since when has there been a class barrier to students getting pissed!?

  22. .

    Sinclair – I used to make them rewrite the essay and resubmit. It was more effective and made everyone better off.

  23. .

    Patton,

    I’ve had to explain to people they were not entitled to HDs because they wanted one.

    It’s sickening we’ve got a sense of entitlement that extends so far as people thinking they are being shoed into honours courses and top level grad positions for doing sub standard undergrad work (which isn’t that hard really).

  24. Peter Patton

    dot

    I have to say, the number of times when I have ever thought I was unjustly graded you could on one hand, probably several times over. Once, I was having a pre-lecture chat to my lecturer. At the end he said, ‘I’ve just graded your paper, but now having chatted to you, I think I might have misjudged your argument. I am giving the papers back in Thursday’s lecture. Have a look, and if you think I’ve got the wrong end of the stick, I’ll give it to somebody else to remark’. He’d given me 70 or something. I took the remark, which was a Distinction. That’s about as close to acrimony on that point I’ve come. All very civilized.

    A couple of times, I know I was given a fail mark (<10/20) for tutorial participation, because I attended only a couple – 25 students in a Medieval History tutorial was simply a waste of time. I was pissed off at the policy in general – of allocating 20% of final grade to tute participation. But it was my call not to attend, so I can't blame "the system".

  25. Peter Patton

    dot, I’m not really sure how ‘soft marking’ would even work in your part of the university. My experience in the Business/Commerce/Economics faculty has been a very high emphasis on ‘objective’ assessment, especially multiple choice, followed by calculations using theories from the lectures, which are rarely more than the textbook verbatim, with longer responses, such as essays pretty rare.

  26. daddy dave

    There seems to be an idea that these people are paying customers and that what they’re buying is a degree, not tuition.

    An important distinction that is unfortunately blurred by the current system.

  27. .

    That’s right PP, we’d pretty much have to cheat to have it work.

  28. Good old boilerplate from Adrien. The shiny-arsed bosses kids (who are all stupid, like all rich people) are putting pressure on universities to pass them because if they don’t then their parents (who donate $0 to Australia’s publically funded universities) might be upset?

    You’re an idiot Adrien.

  29. .

    That’s harsh but it seems to be fair.

  30. .

    Nothing tops the well worn comedic genius of writing scrawl on the toilets of a non sandstone:

    “Why didn’t you go to a real uni?”

    It’s probably more poignant for the foreign cheaters.

  31. There seems to be an idea that these people are paying customers and that what they’re buying is a degree, not tuition.

    Since many courses contain basically no tuition at all, I think this is a fair expectation.

    The money spent on higher education is largely for the credentials, because you sure as hell don’t learn much unless you are doing hard sciences.

  32. The money spent on higher education is largely for the credentials,

    Unless you major in media in which case your credentials are for shit.

    because you sure as hell don’t learn much unless you are doing hard sciences.

    It’s a grave error of mind to make such a judgment. How would you know? After all you have no respect for what you find uninteresting. You don’t even think history’s worth teaching.

  33. How would you know? After all you have no respect for what you find uninteresting.

    It’s not about what’s interesting, it’s about whether or not they actually teach you anything.

    A lot of degrees don’t actually impart any knowledge at all. They just give you a reading list and then assess you on your ability to remember what you read. That is not tuition, it’s just testing. Testing in order to gain a credential.

    You could learn the same things via google, since there is no understanding needed, only memorisation. But Google won’t give you a piece of paper, so that is, in fact, what you are paying for.

  34. I’ve noticed this attitude amongst many staff members but never the administration. What administration, in my experience, do however is insist that academics be more professional – so grade in a timely manner etc.

    Are you casting aspersions on the quality of your profession sir? 🙂

    Really? Given all domestic university students occupy the same status – paying customers – what pull could ‘establishment’ parents have?

    At a level of sufficient concentration of power amongst privately associating individuals parents always exercise a certain pull. This is one reason why the culture industries in this country are awash with untalented brats of senior figures in it. What authority is exercised by teachers is trumped by a number of other walks of life.

    By definition, the number of these kids must be tiny, so how have you managed to discern this practice within such a tiny group?

    My evidence is anecdotal and observational. And, barring some covert investigation, how could one establish concrete proof?

    One such anecdote: an (undergrad) acquaintance’s (well-connected) flatmate was passed in a subject he should’ve failed because he, quote, ‘pulled strings’. What strings exactly I wouldn’t know. But I grew up in an environment where such string pulling was taken for granted so I don’t doubt it. I only hope the little shit wasn’t studying medicine or something.

    Thing you ain’t supposed to pull certain strings. And when you do it’s a sign of aristocratic decadence. Like in America where the stupid Ivy League stiffos have been getting their snots outta military service since the ’60s. No idea why it’s called a ruling class. They who are ignorant of history are doomed to repeat it.

    As for overseas students I’ve got many anecdotes of them putting inappropriate pressure on academic staff: guilt trips, threats etc. They’re basically passing it on from their parents. Asians, in particular Chinese people, set unrealistic standards for mediocre children. Some of ’em just cannae hack it.

  35. You’re an idiot Adrien.

    Y’know Yobbo it’s been civil. If you want it uncivil I’ll show you exactly what I’ve learned and you haven’t. 🙂

    I can do a lot better than ‘idiot’, ‘fucktard’ n’ all the rest of it. I can base insults on an accurate assessment of your commentary for starters.

    Bring it on I say. It’s been to quiet here since they booted the Bird. 🙂

  36. They just give you a reading list and then assess you on your ability to remember what you read. That is not tuition, it’s just testing. Testing in order to gain a credential.

    I agree to a certain extent. But I think you’re reducing too many disciplines to that level. There’s more to reading then simply reading.

  37. Peter Patton

    OK, Adrien, I have no doubt you are telling the truth. And no doubt strings get pulled, discretions exercised, and so forth, but I doubt your ‘establishment’ conspiracy, especially so as you put in the context of the NESB international student issue, which really does involve a large and identifiable group of people.

    For starters, if you get an ATAR over 80, or so, it is pretty hard to Fail a uni course, unless you just don’t attend. Others here, who are actual academics, can talk to this much better than I can, but the limited experience I have had with grading undergraduate work, shows that any piece of assessment, which is given only 50/100 will be of a shockingly woeful standard.

    When I did my undergrad, a clear pattern existed of private school ‘establishment’ kids spending the first two years of their degrees, barely sober, but in the final year – especially if they did Honours – they swotted crazily, often moving back home, and emerging with 1st Class Honours, or straight Distinctions in their final (Pass) year. This was especially if they were doing a combined degree (such as Arts/Law).

    It is true, that so long as they can see some evidence of a High Credit/Distinction student, prestigious employers are more likely to see it as a plus, that a student spent a few years partying, living on campus, playing sport, drama, politics, whatever. OTOH, if all they see is 4/5 years of Passes and Credits, with no spurt at the end, Macquarie Bank, McKinsey, and Mallesons, will not be at all impressed, regardless of the excuses/reasons why not.

    So, how many of these ‘establishment’ kids even fail a subject, without withdrawing, or just blowing it off, not caring? I’d be interested to hear any academics here who could back your ‘establishment’ conspiracy theory.

  38. Peter – I didn’t advocate any conspiracy theory, you’re thinking of my Elvis is responsible for 9/11 routine. It’s more like cronyism. I agree with your assessment of the majority of undergrad careers.

    The dude in the anecdote did exactly that, he didn’t attend. There’s a lot of that. Anyone perusing Melb Uni campus these days would be forgiven for gaining the impression that it’s some kind of Bacchus Cult Revival Centre.

    The overseas students are swats. But some of them don’t cut it. That’s my impression.

    Elvis is responsible for 9/11 btw. You know it’s true.

  39. The solution is obviously to a. detach names from papers for marking purposes and b. tell o/s students that there’s to be no personal appeals to staff.

    Or maybe we should go back to the Edwardian Halcyonia depxcted in Blackadder Goes Forth: In my day education could go hang as long as a boy could hit a ‘6’, sing the school song very loudly and take a hot crumpet from behind without blubbing.

  40. .

    I believe in oral examination.

    Then again, I’m a complete and utter bastard.

  41. The solution is obviously to a. detach names from papers for marking purposes and b. tell o/s students that there’s to be no personal appeals to staff.

    So academics also suffer from that arrogant presumption that their marking is objective. It should be a matter of course not to have student names on the papers.

  42. 2dogs

    Australia is in the business of selling visas, with the funds being used to subsidise our universities. It is merely the case that this policy is not overt.

    I thought selling visas/an immigration tariff was LDP policy (http://www.ldp.org.au/index.php?option=com_content&view=article&id=1156:immigration&catid=101:policies&Itemid=290), why are the LDP members on this site objecting to it?

  43. .

    Selling a Visa doesn’t mean adding to the credentialism mess by dumbing down uni courses.

  44. I believe in oral examination.

    For some stuff, yeah.

  45. Peter Patton

    Yobbo

    Since many courses contain basically no tuition at all, I think this is a fair expectation.

    Is this true? What are some examples? How do such courses survive? Why would somebody enrol in such courses, coz surely word would get out pretty quickly, and no employer would value it. Maybe, there is a significant time lag before the signal turns negative.

    The money spent on higher education is largely for the credentials, because you sure as hell don’t learn much unless you are doing hard sciences.

    Again, if that were so, then surely those credentials must be worth something to somebody? Either the person paying the money, or the person employing people with said credential.

    On what basis do you say only ‘hard sciences’ teach you anything?

    Having said that, I agree that if your university subject does not see the need for a substantial final exam, then I don’t think it belongs in a university. If the course does not involve imparting to the student a significant body of knowledge and methods, at a level otherwise not obtainable in one/two semesters, then what is the point? There ARE a lot of Arts subjects in Australian universities, where 50% of the assessment is done by Week 6! Then, there is no final exam, merely another ‘Research Essay’ due in Week 10! This is especially obvious and scandalous in the teaching of undergrad Australian history, for example.

    A lot of degrees don’t actually impart any knowledge at all. They just give you a reading list and then assess you on your ability to remember what you read. That is not tuition, it’s just testing. Testing in order to gain a credential.

    If only this were so. In fact, the opposite is the case. Those courses with the long reading lists are the least likely to use tests and exams at all. However, your implicit assumption that a textbook – as opposed to a long reading list – is automatic evidence of tuition, the learning from which equips the graduate with ‘knowledge’, doesn’t really stand up to scrutiny.

    The subjects at university with the most heavily memory-based assessment are those, which rely on languages other than English – and by language I include C++ – with the reliance on memory-based assessment increasing the more that foreign language is Mathematics. More than 70% – often 100% – of Mathematics assessment is time-constrained tests/exams, preparation for which involves going over and over calculations and problems, which are barely indistinguishable, in the hope that when you open the exam paper, you will have seen and solved as many of the exam questions as possible. It really is monkey-see, monkey-do.

    I don’t say this as a criticism. A lot of defensive Arts types DO say this is a criticism, to which I reply that in order to get to that stage where memory provides most of the cognitive sweat; you have to jump through a lot of hoops that rely on variously visuospatial, semantic, procedural, numerical ways of thinking. Sure, when you get into an exam room, to do well, you’ll will have to be able reflect a two-dimensional function into three dimensions, before doing the tango with it, and deliver it home by midnight, without having to use any higher abstract reasoning skills. You will overwhelmingly rely on basic semantic long-term memory, as you do with a lot of other university subjects, which are anything but ‘hard sciences’. A subject like Microeconomics is learnt in almost identically the same way.

    OTOH, I don’t think English should even be taught in universities. If you can’t read and use your mother-tongue by the time you get uni; it’s too late. You see this problem all the time with English majors. They spend 3/4 years repeating the same stuff as the HSC. Meanwhile life and knowledge in 2011 requires basic conversation level in a number of languages, such as Stats, Maths, C++, Accounting, Finance, Biology, blah.

    Unless they are careful – or attend a top US college, with strict breadth requirements – the English major crowds out all these other languages, and so sounds like a doofus when they leave university, and start pronouncing on all these subjects, which use languages the English major never learnt.

    You could learn the same things via google, since there is no understanding needed, only memorisation.

    So long as you have a rigorous HSC training, surely the same can be said for heaps of subjects, including the hard sciences? I haven’t studied Physics, Chemistry, or Biochemistry, etc. at university-level, so maybe you think it is not possible with these subjects? Though, I’ve taught myself uni-level maths-intensive courses – due to other commitments clashing with lectures – largely through the textbook, and precisely Google, because you have access to the best teacher’s in the world, who post all there notes, including stunning visuals, often dynamic. Check out MIT’s Open Source courses, for example.

    OTOH, I was quite shocked when majoring in History, for the Arts part of my undergrad at just how important tuition by a very highly educated practitioner is. History is very much a craft, with a hell of a lot of craftsman’s tools, techniques, and nuance that can only be mastered at the foot of a master. That is why Oxford and Cambridge are so superlative in History, because that extended one-on-one time with a master is vital to ‘getting it’.

    The ‘hard sciences’ are crucial and central to the modern university, but there is a hell of lot more knowledge out there that is equally so.

  46. .

    Oh man HSC history is bad. Say wahtever C W Bean said and ignore what happened in the war. You must blame Russia. You must not blame Austria…deadshits.

  47. Peter Patton

    dot, NSW has bragged for years about how advanced its curriculum is, especially as History is compulsory from Years 7 to 10. Yet, check out the Year 10 School Certificate examination they give. Play ‘spot the history’ among all the blatant neomarxist propaganda.

    http://www.boardofstudies.nsw.edu.au/schoolcertificate/sc2010/2010-SCT-AHGCC.pdf

  48. daddy dave

    There’s nothing wrong with making people memorize lots of facts. It’s an unjustly demonized method of teaching. There should be more of it.

  49. Peter Patton

    dd

    I reckon every History course should have 2 final exams. The first – could even be all multiple choice – tests chronology, dates, personalities, names, events, buildings, geography, and so on. The second, two essays in two hours.

  50. daddy dave

    it’s usually described as “rote learning” “lots of disjointed facts.”
    But the collection of facts in a course are rarely disjointed, and only by getting down into the detail can students really gain deep insights into the overarching patterns.

  51. and only by getting down into the detail can students really gain deep insights into the overarching patterns.

    In the days when I used to study hard I would spend hours just absorbing facts. There I would go for long fast motorcycle rides and let it all bubble back up again.

    There was a study released a few weeks ago which stated all those special learning methods were worse than one simple strategy: recall practice. Stuff revision, after reading about Kandel’s work and gaining a modicum of understanding about memory consolidation, I would set up recall schedules. That method proved so effective that after some months of practice I no longer had to engage my prospective memory for recall intervals, I would spontaneously have the recently learnt information popping up in all manner of strange circumstances, like fast rides through the mountains. Perhaps it was because of the high heart rate and increased cerebral oxygen supply plus the cortisol and nore increases. It was nonetheless strange to be belting through some corner waiting for the peg scrap whilst simultaneously recalling acronym city with things like bax, apaf1, psd95, caspase cascade, etc etc. One great advantages of memorising great chunks of facts before someone comes along and tells you all the inter connections is that you get to think for yourself.

  52. Again, if that were so, then surely those credentials must be worth something to somebody? Either the person paying the money, or the person employing people with said credential.

    Credentials for a lot of degrees are basically useless. The value they show to employees are that the person in question was capable of sticking to a task for 4 years without failing.

    The majority of jobs that require degrees do not assume that the applicant has knowledge of anything taught in that degree. It’s just a signal.

    There are even plenty of jobs that advertise as a requirement a “4 year degree in any discipline”. Which tells you all you need to know. In fact that sort of thing is also a prerequesite for certain kinds of work visas abroad.

    Not all degrees are like that, of course. But a lot are, typically in the humanities, but also to a very large extent in the “business” and “computer science” degrees. Unless you do CPA accreditation a business degree is basically meaningless, which is why anyone who needs real credentials has to do an MBA now.

    And the “business” line of degrees are the ones that are overwhelmingly marketed at overseas students.

  53. And the “business” line of degrees are the ones that are overwhelmingly marketed at overseas students.

    Circa 2 years ago a business studies student at Griffith said to me: the only difference between 1st and 3rd year seems to be that they use different words to describe the same thing. Lots of public servants also do business degrees. If you want to be good at business you don’t need a degree you need to know the business and relevant market backwards and forwards and work really bloody hard.

  54. .

    You virtually need hons to get a job as an economist. A PhD or work exp. for the title.

  55. .

    If you want to be good at business you don’t need a degree you need to know the business and relevant market backwards and forwards and work really bloody hard.

    You need a good methodology and an open mind in my line of work.

  56. Peter Patton

    I agree that undergraduate Business degrees are a scandal, and should have no place in a G8 university. Marketing? FMD.

  57. .

    I agree that undergraduate Business degrees are a scandal, and should have no place in a G8 university. Marketing? FMD.

    Don’t be too harsh. A lot of economists don’t understand marketing – they simply haven’t bothered to learn about it.

    At a high level or done rigourously and with knowledge of economics, financial management, the product etc., it’s quite a worthy discipline.

    Otherwise, it’s a soft management course with some Cliff note stock in trade answers. At least they do stgats 101 and a regression/quant/qual analysis course.

  58. Marketing is vital to good business strategy. A good product with a poor marketing strategy will not sell as well as a poor product with a good marketing strategy.

  59. Just because something is vital to business doesn’t mean you need to do a 4 year degree to learn how to do it.

  60. Sinclair Davidson

    OT discussion has been deleted.

  61. Peter Patton

    Yobbo, I think the real issue we are debating here can probably be reduced to ‘what is the signal value’ of a non-professional university degree, which I’d say there is plenty of data, even if it has not been synthesised comprehensively.

  62. Peter Patton

    My first two jobs out of uni culled for good Honours degrees or Distinction Pass degrees in anything from Engineering to Law to Medicine to English, Maths, Economics, and Classics. They said to us, “we can teach you Business”.

  63. daddy dave

    ‘what is the signal value’ of a non-professional university degree

    The “quality signal” argument fails.

    * It’s a subsidy to recruiters;
    * It’s easily gamed;
    * It’s too expensive for both the taxpayer and the student as a quality gate.

    It should at most be a secondary benefit of university education.

  64. But a lot are, typically in the humanities, but also to a very large extent in the “business” and “computer science” degrees. Unless you do CPA accreditation a business degree is basically meaningless, which is why anyone who needs real credentials has to do an MBA now.

    And an MBA isn’t meaningless? 🙂

    I agree with you to a large extent. A lot of what they teach in business degrees won’t really prepare you for business. Studying English Literature won’t make you a writer etc.

    And I think there’s a lot of admin bullshit that gets created by people whose degrees gift them with the illusion that they know something.

    But I believe your concept of what knowledge is and what is worthy of higher education is too narrow. The study of History and Literature is, imho, vital to higher education. Why?

  65. Peter Patton

    Adrien, while I agree with History, I am even remotely persuaded of the case for studying English at uni.

  66. Peter Patton

    Unless it were part of a broader, much more integrated degree in say, “The Humanities”, which means ALL the humanities. There are all these people being awarded PhDs in Humanities by Australian universities, and the only language they know is English!

  67. Peter Patton

    Whenever, I have had to choose humanities/Arts options, the first thing I look for is the lecturer’s country of origin, and then where they got their PhD. If the answer to both of those questions is Australia, I cross that course off the list immediately.

  68. Peter Patton

    Yobbo

    There are a lot of sub-markets working in MBA education, with graduates from the top ones being paid a bomb, due to the very strong signal that a particular program sends about both its own screening process (saves McKinsey, Morgan Stanley, IBM, GE, Goggle a lot of search costs), and course content.

    Employers know that from day 1, they will have an employee who holds, not only a number of valuable personal attributes, but is fluent in the language and customs of business in general, the methods and processes of a roll-call of analytical skills, which really are demanded: how to value a multi-business multinational corporation, which a client is considering a merger proposal; run diagnostics on a company that is suspected of under performing; sensitivity to subtleties of board-level politics; whether or not some local market is a flabby duopoly/oligopoly/cartel, which a fitter new entrant might be able to clean up in; how a BigPharma might use derivatives to cut the volatility of returns on the their R&D spend; whether a company is facing bankruptcy; how much cash and long a line of credit a start-up will need; confidence and poise in the presence of unfamiliar people and locations, and so on. These things DO constitute “knowledge”.

    OTOH, I doubt the data would show any benefit from spending two years at some Australian provincial university getting an MBA. But I might be wrong.

  69. THR

    Whenever, I have had to choose humanities/Arts options, the first thing I look for is the lecturer’s country of origin, and then where they got their PhD.

    It seems to be the first thing you do in every situation.

  70. Peter – L’histoire c’est la littérature et la littérature c’est histoire!!

  71. Peter Patton

    La littérature n’est pas l’histoire, en soi; la littérature est un artefact historique ou trace – comme le luvvies l’habitude de dire – ou fragment, morceau, bit ‘pièce du puzzle’; personnellement, j’aime shard.

    L’historien doit composer avec les – er, how you say, “éclats” de beaucoup de différents genres artefactuelle: à partir de lois, les dossiers fiscaux, l’archéologie, nusimatics, à la poésie, et même l’historiographie elle-même.

    L’inverse n’est pas le cas, ni même nécessaires; un fait qui devrait impressionné lors de types département d’anglais dès le premier jour.

    J’espère que cette aide de fèves anciennes.

  72. Peter Patton

    dd/Yobbo

    One real recent change has been the huge growth in individual employers – private and government – using their own testing, including psychometric/personality, IQ, and general aptitude. But so far, consistent with Yobbo’s point, I have not noticed much – if any – testing of actually job-specific knowledge presumed to have been learnt at uni. Maybe engineering employers do, and/or architects, I have heard computer programmers do test actual knowledge learnt also.

  73. Peter Patton

    The worrying thing is that the plethora of IQ/aptitude tests suggests that employers are not seeing enough “raw intellect” signal in current degrees.

  74. daddy dave

    I have not noticed much – if any – testing of actually job-specific knowledge presumed to have been learnt at uni.

    I have. There will be more of it. This is the future.

  75. Peter Patton

    dd

    Do you work in the area? I’d be keen to know what patterns you’re seeing, if you can, without divulging any personal stuff you prefer to keep offline.

  76. Gigi Foster

    Let me respond to the original critique posted here by Sinclair Davidson.

    1 – N. The number of students covered in the data set i use for this paper is about 12,800 (as reported correctly in The Australian). Yes, this implies that many students are part-timers. The marks differential between international and domestic students is completely robust to the inclusion of a control for the number of courses completed that semester by a given student. It is not part-timers who are driving this result.

    2 – The outcome. The only outcome variable i use in the paper is final course mark. The key “social effects” regressors include the percentage concentration of international NESB students in the same tutorial with the given student, and the percentage concentration of international NESB students in the same course with the given student, naturally excluding the student himself from all concentration measures.

    3 – Attrition. Yes, i do simply drop students for whom i have an invalid mark. The reason i do this is that it is not absolutely clear from a theoretical standpoint what the correct approach is, so i chose to do something simple and defensible in the working paper version of the paper. If students who end up with invalid marks in a course actually attended most of the classes and took the exam, and then just threw in the towel at the very end of the course, then they should be part of the sample from which i calculate the tutorial and course concentrations. However, if they stopped coming to class midway through the semester, say, then it’s questionable whether they were there to influence other students and therefore perhaps they shouldn’t be included in the tutorial concentration ratios. Those who didn’t sit the final exam almost certainly shouldn’t be included in any measure of coursewide context, including concentration measures, on whose effects is placed an interpretation related to course-wide marking. Regardless of when the attrition occurred, i cannot think of a simple attrition-related story that could explain the social effects i see. If anything, if one believes that the lower performers drop out, then indeed the marks differential between the international and domestic students that i estimate might be smaller than the true differential. However, i have found in previous published work using a smaller data set from one of the insitutions in this study that attrition from a course is essentially unrelated to how well the student is performing in that course.

    4 – Course repeats. Sure, there are almostly certainly some students repeating courses in the data. It’s quite possible that international NESB students are more likely to fail courses and then have to repeat them, although i haven’t run a lot of tests on failure rates. Perhaps i should, but I don’t see how course repetition would invalidate what i find or how i interpret it.

    5 – Tutor effects and student selection. Sure, tutor effects could be going on, which is why i desperately tried to get teacher information from both institutions at the start of this project. Unfortunately, that didn’t happen. But i can and do still control for course-level fixed effects, which should eradicate any “easy course” or “hard course” marking effects that you think might be going on. I also doubt highly that international students (and supposedly not domestic students) are selecting whole courses just because particular people are teaching them who are known to be easy on international students. Most student-course observations in the data are from large courses (300+ students) which are usually required by the business program a student is taking, and student flexibility is quite low. Add to that the fact that many courses do not rotate teachers every semester, and you get a situation in which large-scale teacher selection by students is very dubious. Just for fun i re-ran the regressions including only student-course observations from courses with more than 300 student records, and the coursewide concentration effect is highly robust and even stronger than in the full sample. My conclusion is that this is not an artefact of student selection.

    To me, the Occam’s razor argument points to the interpretations i provide in the paper as being the most likely. If we had more data, i’d love to do more research.

  77. Tony English

    On 18 March, Sinclair Davidson wrote: “I too would have sacked Steele and Firns and several other academics who have appeared in the press as ‘whistleblowers’ over the years. Not because they highlighted problems, but because they did not follow university procedures and brought their institutions into disrepute.”
    Does Sinclair really think Steele and Firns did not follow procedures at first, then give up in disgust and go public for moral and other reasons? I tried for many years to deal in-house with soft assessment in some parts of my university, especially the business school, but got nowhere. In confidence, I gave irrefutable evidence of soft assessment to deans, professors, the vice-chancellor, deputy vice-chancellors, other senior managers, and colleagues at every level. No one told me I was wrong, but no one would do anything about it. In the end, urged by some colleagues and many former students, I decided to go public because I believed it was immoral to continue to deceive local and foreign students, and their backers, into thinking their degree was of the same quality as it would have been only a few years earlier. (See my article in The Australian on 16 March 2011.)

  78. Sinclair Davidson

    Sorry Tony. I’ve heard these allegations for years myself, but have never experienced it, seen it or participated in it as a junior academic all the way through to being a senior administrator.

  79. Tony English

    Sinclair, you are not alone. Over many years it has been normal for me to meet senior academics and administrators who have heard about soft assessment as they have climbed the ladder but have not encountered the problem, let alone investigated the allegations.

Comments are closed.