"In the Next Decade, Digital Scholarship (In Open Journals, Blogs, and Social Media) Will...
"In the next decade, digital scholarship (in open journals, blogs, and social media) will achieve the same status in academic settings as traditional scholarship..."
Follow this discussion to receive e-mail notifications whenever it is updated. Follow
Log in to post a a comment in this discussion.
Can't recall who it was now, but someone tweeted 'but how do we rank [products]'? After a bit of back and forth he said he really meant 'how do we establish quality?' It's a great question and an important one before we can move forward.
I think Andrew Law's keynote the following day arrived at the same place I'd arrived in thinking about this issue: reputation systems.
If you get every person to rank something, and all the ranks have equal weight, you can get some sort of measure of popularity. And in some sense that can be a proxy for quality...
But I'm a bit of a snob in some ways, albeit weird and twisted ways. I like Norwegian black metal and Swedish melodic death metal and weird doom-folk hybrids. I hate FM radio and America's Got Talent and... everything that's popular. Given that, raw popularity is almost an anti-quality indicator!
Now, I do have a lot of online friends with similar tastes to me, and if something is popular with them, there's a good chance I'll like it.
There are various services such as iTunes and Last.fm that do this kind of work.
In an academic context, a reputation web will, as it builds up, mean that not only those who like the same things but those who have 'runs on the board' (sorry, cricket metaphor) in terms of publications, ranks in universities, research grants, books, keynote invitations and so on will be identified, and their recommendations weighted differently.
But that's only a very rough first approximation - I've been to some very dodgy presentations from full Professors at Harvard! There needs to be a process for dynamic reputation development that ends up being a true meritocracy: those who write the best papers, comments and so on will in turn have their views given more weight in ranking the contributions of others.
It's definitely an exciting possibility - and IMO a key complement to open publication.
Posted
Hi David - interesting point about the distinctions in what counts as quality based on preference. Any model of evaluation needs to address the concerns about popularity vs. quality.
I agree with your point about reputation in an academic context. The model should include multiple criteria - formal and informal. And yes - it's an exciting possibility!!
Posted