SICSA PhD Conference 2012 – Keynote: Ursula Martin

Professor Martin from three Mary University of London or Sula has a degree from Cambridge and PhD from Warwick and has been working for many years in Scotland at st. Andrews University she then moved to London where she raced through the Rams up to the level of pro vice-chancellor which I guess some Scottish Irish is called vice principal and now is spending a year in sabbatical in a dobrze center for attention systems and applications also has a lot of experience in assessing the impact of computer science research in the impact of research more general pneus you may not know so much since you’re still young and tender but in fact is a big thing that we are required to demonstrate so I’m sure we can all learn for the rest of our career thank you ok thank you we do what it’s great to be here um because I’ve only just turned up today I don’t mean everyone talking to so we start talking to a bunch of PhD students how many in from universities in Glasgow and how many are from universities in Oh girls go wins and how many are from neither of the above double DD Aberdeen places North who’s from the farthest north we’ve got anyone from the University the highlands and islands here nobody from Psalm or pasta why hasn’t some more thing is this wonderful Gallic College on star I really think they should have six of those great engage me great anyway so what I’m going to do register surfing because this screen is not shown my talk the screens of you what I’m going to do today is to talk about the impact leg research and I know probably quite likely you saw that before time so we’re so if you go and look up in the Oxford English Dictionary what’s impact lives what you think means object coming forcibly in contact with another rock at a marked effect or influence my mum told me never to eat ice cream in the street had much effect or influence at all actually but she thought this was going to all right something about me I know all kinds of research and this is just a little bit about the kind of research i do which is to frame what I’m going to be saying about research and impact of research more generally um I work in all sorts of things i work in maths i work in computing a looking uncertainty for sociologists provide 84 past seven or eight years I’ve had a project with industry engineers which was quite fun because we were taking classical computer science logic corelogic a little bit closer sort of thing that Jay was talking about on yesterday and we realize that you could apply that to control systems until we applied it to control systems and built a system and the engineer’s liked it it doesn’t fit any much model of how research is supposed to work because I had a bunch conversation with a bunch of Engineers in a bar and they I thought my goodness that sounds like logic i watch when did the logic and then we wrote this paper fully complicated logic that I was too embarrassed to even can go back and explain to the engineers in the bar it was put a category theory um but of course it’s not how it’s supposed to work what is supposed to work as i wrote the brilliant paper on category theory and then all the engineers we’re supposed to say wow that’s fantastic but it didn’t actually work like that like they gave me the idea in the first place I’m now doing a different thing I’m used working with sociologists to see how people talk about science on the web and you can’t fail to notice lots of people are talking about science on the web people are doing science of web people having use groups discussion groups blogs all sorts of things when you go and look at that you see a kind of science happening that it’s really quite exciting so we’re looking at that available budget jewels and my qualification for talking about impact is that for the past three years I’ve grown a big project that we married which was all about funding collaboration between academics and users of research so I could have got a bit of money to talk to my engineers these math blogs where people were in companies I’ve got a bit money to talk to them or you could have done um so funds all sorts of collaborations with

users of research people out in the outside world and use our research including 20 PhD internships here and it’s so it’s a whole lot of fun because I’m discovering all sorts of people in Queen Mary do interesting things I never knew about every so often seemed like once a week they’ll be a science press release about germs and hand washing and the latest one I stayed in a hotel last night the latest science press release said the most dangerous thing in a hotel room is the TV remote because it’s covered in germs as one of the projects we were funding under this it was so biologists who have well basically they have they have good techniques but you know the sort of the swabs that you swap over a TV remote to me because they’re so that’s the kind of invested I’m project we’re looking for normal ways to use our news on research well a little bit bit about where I work I woke up to Queen Mary University of London um something building if you don’t know London terribly well um this is the map of London tall enough to reach over this side of motherhood is the posh fit in a bucking and palaces and correct me if I’m wrong anyone i think poking a policy there as a parliament are there um um we wanna go right over here with a 10 is about to be the City of London a sort of about there you go out here with a pretty big red bull bed and you’re in the East End of London a nice end alignment was a traditional immigrant area of London it is where people came to London from why it’s why the immigrant area diamond plaza this thing follow the river from the docks i’m c-czar and so and this has always been being a working-class area of London large amounts of poor housing wait ways of waves of immigrants starting for the human eyes in the 17th century through to later waves of waves of immigration from Central Europe and then from Asia and Africa and whereas if you go to Imperial College which is in the posh bit you look out and see hyde park if you go to our particularly exciting physics building and look out for the window we see Canary Wharf but what you can see out here is an absolutely riveting used car sales room we are really not in a posh bit of London um but there’s lots of things happening in London like the Olympics like um like sukin what we call this week silicon roundabout the rows of high-tech companies in London and so on yes I took this I took this photo actually the poster in the airport in Paris because if you met a thing it says technology is great britain project you this is not a picture of great room this picture is this picture so I’m Queen Mary it was founded in the 19th century astounded as a place giving free helping helping in the language of the day helping young people better themselves and how did they better themselves or they better themselves by studying science and engineering and here are some of the early lads studying Electrical Engineering and messing about the bits of equipment on them and so on we were this is this is for older members of the audience here go on who’s heard of perk workstations yeah very good um is there oh and something feels not here is he used to be one the years and years there was one as a sort of historic antique in front of the offices in Sudan Brisbane in the mid-80s work stations were the high-tech you know work stations were the iPads of the day we had some work stations which came from California we have percs state workstations that came from ICL in Bracknell and in the middle you had a workstation that probably nobody’s ever know but it’s called the Whitechapel and it came from we marry it came from the mile end road um well obviously you kind of you know who won that one one son in California one and the rest is history but UK at one point did have a high-tech won’t be home how to talk why did perk and Whitechapel lose and some wii and of course it’s a whole other talk because it’s not necessarily to do with the technology’s funding it’s to do with them whether they could raise money from venture capitalists it just to do with whether they can keep up the technological advances etc um and now

well we have all sorts of research but are particularly flag because it fits in with what I’m about to talk about we have research in in safe software computer science logic and being applied particularly these a specialism of these guys is reasoning about stacks and heaps reasoning about memory overflows having an abbey road moment here this one is dino mr. Farnum just one of the Roger needham award which is that when computer science prize for best for such contribution in 10 years Jamie who’s sitting in the front row is the first-ever winner progeny reward hmm with a group to work on networks and critical infrastructures they do everything from their very basic mathematics of networks to having a start-up actual experience which is it’s about taking taking all the data that you get from network performance and actually optimizing the actual user experience when you’re when you’re sitting in front of the screen so that the things you want to download fast download fast and so on oh I asked them about the impact of research at one stage because I was rather interested in this idea where get research ideas come from that they come from did they come from scientists did they come from companies and they gave me this picture this of course has to win the car point prize of the decade of amazing piece of picture perhaps it uses a few marks because you can’t actually read it but but the idea is that the the yellow is EPSRC funded grants and the green is industry funded grants and the arrows are what affecting what and the point is that the arrows are just complete spaghetti you know sometimes you have you have a piece of academic work that influenced a piece of industry work for the influence a piece of academic work sometimes you have academic work going both ways sometimes you have a piece of academic work years and years of it before which influence is a piece of academic work right down here sometimes the links are quite quick and so on and so on when you ask them to talk about the impact of their work how they how they work with other researchers how they work with industry you ask them for a nice clear story and they say well if you just don’t like that it’s really complicated um that’s part of the message I have for you these interactions with industry these interactions with users of research are really very complicated but instead of talking about that I thought I just think about a little bit of history which for you know your PhD student you come in and you get your grant and you do your work and you don’t often stop and think about why do we have universities why universities here why is the British government or the image government or quasi-government pudding throw there isn’t an English government yep putting money into science you know why didn’t he do that and that that folks actually looking back at this because there’s a sort of general feeling well it’s a good thing and where did this idea come from that it’s a good thing wine a good thing um and so I’m thinking back and I came across on the web the first royal charter of the royal society of london and that’s often thought of is the beginning of formalized science in the UK and the king we’re getting was getting rather excited about science and you found it the royal society to sponsor and support scientists who have one of the first scientific journals they have time for bikini teams came people came and talked about their experiments and then the papers were written up in a journal and that’s where a lot of that started it’s quite interesting salts are always stuff why is the keep doing this we have resolved to extend not only the boundaries of empire but also to very Arts and Sciences okay so Charles the second plan world domination wasn’t just to go out and call whatever I use whoever who’s nice torrent and the audience who is he at war with exactly in 1662 I’m not sure anyway you know not only it’ll not only to go out and conquer whenever he was trying to conquer but also to conquer the arts and the sciences it was another it was another thing that the conquered from another another thing to show his power therefore we encourage philosophical studies especially those which by actual experiments attempt to shape our two new philosophy or God to protect the old and science science by means of experiment is that was new in order there Paul at such studies which hadn’t been sufficiently brilliant in

any part of the world you know what hasn’t really been very good at this may shine conspicuously among our people we’re going to give it this and at length the whole world of letters may always recognize us and then you know just in case he hasn’t made the point strongly enough he says recognizes not only is the defender of the faith so he’s borrowing is he headed the church in in England but also as a universal lover and patron of every kind of truth you know he’s not he’s not hedging his bets here he wants to be was world world domination doing science is like grabbing territory all over the world and defending science fostering science is just as important as being head of the church and of course being head of church in 1652 pretty important so it was a pretty resounding message from Charles the second and that message is a echoed similarly this is so the foundation of Sir Andrew since I used to work there I have to ban the run with it sir celebrating a 31-year per second yeah very good well mrs. Pope Benedict setting up soon and raised in 14 13 I have an idea that this translation from Pope Benedict’s Latin was done by the Santana’s PR department you know considering the peace and quietness that flourish in the said city it’s basically it’s good pulse its abundant supply of it was an ultimate hospices and other conveniences for students which is known to possess be a letter at the city which that may become the fountain of science and I’m not quite sure here if he’s saying the students will be good customers for the Cubs for the production will make good science happen anyhow you know there was a feeling I think 1413 the word didn’t mean quite the same thing but shut you know the lid on idea it was important to set up universities um Scotland I know from which would be here about three days ago as the first compulsory education system of since ancient Greece education is hugely value in Scotland because in the the purpose of the compulsory education system was to keep teach kids to read the Bible and thereby become better people but during the 18th century the Scottish enlightenment flourished the Royal Society baby row was set up and people were starting to think about science not just as you know for Charles the second it was curiosity it was world domination and both people were starting to think about science also they actually might be useful I said the Royal Society was for the advancement of learning and useful knowledge strathclyde is finding about that time it’s all strathclyde always had the tagline the place of useful learning and people were starting to see that science really could affect prosperity you could grow better food you’ve new means of transport coming along like canals you could have better weapons and I’ve dustrial revolution you have the changes in the in the in the traditional injured industries like um like textiles and so on so that started to be the agenda about science it wasn’t just knowledge for its own sake it’s also knowledge having an influence knowledge being useful oh but the big government investment in science the thing that really triggered governments to start spending money in science was the Second World War it’s Terrans birthday tomorrow cure tomorrow Mr Eames 100th birthday and as a big celebration featuring this year one of the things that aren did that was so influential was he’s working wartime on crack and German codes this was a this is a document I’ve got quite a bit of this document on the screen because i find the term quite surprising because it’s his July 1945 and this is Vannevar Bush as well as a similarly influential scientist in the u.s. in the 1940s he’d been involved in a huge amount of war were people in forever been involved in web science things right now he wrote an extraordinary prescient after what about the web he called it that was it cool at as we may think or something like that and he’s he’s talking about you know he doesn’t know what the technology will be but he’ll talking about web and links between knowledge and so on right but this newspaper here at Roosevelt because the Second World War science / had had contributed to the American victory of the Second World War that the Allied victory in the Second World War when Bush was around a canny book and he could see that now was his moment to get government to invest huge amounts of money in science so he writes this report which is some as in everything else you planning on the web it’s called science the endless frontier and he says very clearly new frontiers of the mind that before us and if they’re pioneered with the same vision bulbous and drive

which risk we avoid this war so is exactly we’re back be back with with with with Charles the second territory science is another frontier conferring Sciences is like it’s just like winning a war Oh government should foster the opening a new frontiers and this is the way to do it advances in science being more jobs higher wages shorter hours more abundant crops etc etc um but Bush is quite kami because what he of course he wants the government to invest lots of money in science but some he doesn’t just want them to spend it on science he wants them to spend it on pure basic fundamental research and so he makes absolutely sure that when the you know when the president puts together they all the funding to science scientific progress results in the free player the intellects working on subjects their own choice in the matter protected by their own curiosity basic research is performed without thought of practical ends and so he’s making the case actually you know give them money get the scientists the money and let them do their own thing and that’s how do you have it funded science before much it’s a pretty audacious request if you think about it you know we’re rather used to that one of course we’re scientists we have to have freedom a fourth but actually do i do basic research has formed without view of sort of practical ends that results in general knowledge which provides needs of answering large number of problems that may not give a complete specific answer so you found all this science but don’t expect you to solve all your problems right away because basic science is different and what rather remarkable about this document is it since I’m about 50 p is it it’s pure polemic there’s actually no evidence in it there’s hardly any numbers except very very basic numbers like how the life expectancy has improved and that’s because of medicine it’s just pure clinic in that vein on and on and on it’s a very fine piece of lobbying but it worked and it worked in the US in the 1940s and it continues to work and so as a result of this in the UK we get I don’t know if you can see those numbers it back but when you analyze how much money is spent on science who provides the money the government the research councils which is also the government of higher education funding council who’s also they’ll basically business private nonprofit which is charities and abroad and they provide money to the government research councils companies etc different people who research well we live yes and I’m sorry and how does the government decide how much money to give to the research council our education funding hands as well every two or three years the UK government has what it’s called to comprehensive spending review on it the big arguments go on in Whitehall about how much should go to science and how much should go to hospitals or schools or defense and some of you may have noticed when the tory government came in who was a notorious ten minutes that we must have cuts cuts cuts and and of course that’s the time when you really have to be if you want science to be supported people have to make the case actually yeah of course we want science to be supported but is it more important than primary schools you know look look your look your look your friends with small children in the eye and say his science is science more important the primary school it so it becomes a it becomes a debate and people are rehashing generally they’re rehashing this kind of argument but in more modern language and with a heck of a lot more data and numbers so and then of course in our column we then from across ways of love you know sorry a rearrangements life it’s this light comes back a minute and I’ll be okay so this huge government huge government investment he science if you look in our column that figure is about seven thousand million and the government is well these three limes at six plus 19 is 25 plus 23 is about is you know so about five million at that seven and a half million to seven seven and a half billion 55 billion of that 7.2 billion comes from the government funding the science in universities and the rest comes from industry from charities particularly for medical research and some of it from overseas from things like EU funded so we are hugely dependent on the government um and of course government to to justify our existence to justify your existence of PHP skills requires us to tell a

story if it’s so important we’ve been immeasurably better explain why what we’re doing is worth all this money and so that’s where impact has started to me not just talking about how are the only qualitative terms well you know this piece of science happens that now we have that the computer networks are faster or we have this kind of medical advance but how are you quantify it how we can measure it um and that really gets that’s how the word impact has become a more complicated word in modern universities because we’ve got to put numbers on these things oh and of course when you think about it researchers measure of in all kinds of ways and um at the individual level you know at some point um you guys are all going to face of PHP committee who’s going to assess your research in some way or other and decide whether you’re going to parse or not well let’s hope let’s hope they decide you’re going to pass but they have some way of assessing your research and if your supervisor hasn’t told you what they’re looking for then you know next time you see your supervisor just remind you know Peter Martin say well tell me what the PHP committee are actually looking for um and of course when you start publishing journals and conferences will assess your research employers who are looking to give you a job well or promote you will in somehow assess your research assess your CV funders will assess your research and then further up the food chain is big decisions that government is taking dominant and hefty and the research councils will assess research well for various reasons so it just like tell secondly what show that UK research is internationally competitive you know we still got we sell them in that dominating world position well sometimes what they want to show is you see that we’re almost competitive but if we spend a little bit more we’d still be back up top and it please you know that’s her all the skills of a lobbyist come in and I know if we’re completely useless at something then government will probably say well sorry we’re not going to bother to spend on that but if you can prove that you’re just almost there then government’s more likely to more likely to invest I so want sure the UK to bet they’d make the case for investment and then of course all these government bodies have got got the money and they want ways of distributing the money in some sort of fair way of transparent is a curious work you know we used to say this ribbit things fairy and then people got a bit uncomfortable about using word fair because it sounded as a sort of moral / terms I think so they now use this work transparently which means well you who’s not actually fair at least we’re going to be very clear about the rules we apply so that we can challenge whether or not William so that you can challenge whether we applied the rooms properly the rules may not be fair but that’s a different story we’re not getting a glimmer so trance they want the government want ways to distribute the investment that are another the transparent and won’t get me to a popular movement maybe too much hot water and we’ll get the results they want so that’s why we’re sort of measuring and assessing research now the simplest way to measure and assess things is of course numbers you know all these politicians love numbers and by was I discovered and so these are the kinds of numbers that people use to assess the value of universities as a whole and this is Vasco caledonian which is where we’re sitting right now who conveniently had lots of numbers up there perhaps the key thing to notice of course the count the case that Glasgow Caledonia if you’re trying to make by putting this on their website is the investment in Falco caledonian is a jolly good thing because it has 100 million million turn over at all the money that comes in every year from all sources but my goodness see the money that it generates for Glasgow City is 220 million almost twice that and that envelops for Scotland is 400 million you know four times that so the other caledonian wants you to know that it’s a good thing and what’s it measuring in one this way to make sure any things like just all the money that tell me everything that happens in the university all the staff that hit employees and so on everything that happens the steel routes all the money that the students spend that wouldn’t otherwise be spent all money that’s spent in pubs and bars everything else the impact of Vasco caledonian on tourism well some of you you see are tourists the pine city of Glasgow so you’re in there somewhere that tongue that dance orange juice gives you you

didn’t get at breakfast his home wearing that one of those figures or may actually maybe are in one of these boxes I’m not sure when you’re in the conference trade box but knowledge transfer commercial training emplacements a university like this doesn’t awful lot of Industry short courses and them and talk about courses for people and local companies health care disparate Glasgow para bien de treinta lots of nurses comparative spend which is just money it puts into the system because it’s building new buildings and so on so um glasgow caledonian makes a case to government but yeah we cost money we pass 211 million from various sources but look at all the all the benefits the country gets out of this now before sort of teaching operations that’s a kind of you know some of these numbers are obviously a joke they can’t really estimate down to the last pound how many cups coffee visitors by but um look you know it sort of you can do a full part figure and it sounds vaguely credible it is of course much much harder when you’re trying to apply that kind of model to research because it’s sort of well are you kind of think you can kind of thing um but when that doesn’t stop the government from trying um so when they’re asked to make a case surfing at the value of money over research they talk about delivering highly skilled people to the labor market that’s you you know who you’re going to go out and get jobs which are on average more highly paid than the regular graduates and you’re going to add value to the places you go work in because of the knowledge you’ve got from your PhD or the technical skills you learn what your while you’re doing your PhD um um the research has improves the performance of existing businesses so there’s some research innovation in a computer with a company like Google Microsoft all of that you know that they’re just based on on all kinds of research whether inside the company or research that’s coming from universities creating new businesses startups um attracting R&D investment from global business so that’s a that’s something in a value add from researcher I Alex Salmond was in California this week trotting around saying investing in grow best in Scotland because of our universities because our science is some is so wonderful I only know this could a Scottish friend of mine posted this on Facebook you know i’m not actually memorably um but um you know so the idea is that universities that research makes it attractive for companies to come and set up in because they’ll be able to get put in good stuff so Oh improving public policy and public services so the research on how to get people to wash their hands or where the most germs are in a hotel room and it’s difficult to put some money value on that you know how do you estimate all the people who didn’t catch a nasty illness because they think wash their hands how do you how do I string value of that research I don’t know but but you can try recourse when you look at them you’re going to be thinking well how long it was rather a lot missing here I like to think of it and some in this sort of way and a wet where is the Higgs boson in this picture and that caption said Higgs finds the boson leads it into captivity you know they think the governments around the world and spent huge amounts of money on particle physicists in in CERN and the US and looking for the Higgs boson and when it going to fit in that while it’s delivering lots of highly skilled people obviously improving the performance of existing businesses not sure creating new businesses well possibly you know the long distant future and maybe some spectacular new version of quantum computing because we found that he’s boson and improving public policy and public services while I’m not sure the old Higgs boson is going to be for that and attracting R&D investment well nebulous in it um and for the Higgs boson you could put eat well there he suppose on it is a spectacular one to put because it’s so expensive to look for the Higgs boson few you can have P equals NP or you know your favorite little problem in semantics there as well all sorts of basic research that just does not fit that model and though and of course that that concerns many

scientists that inner in an age when perhaps government has respect for scientists and it did in the in the postwar days of a battleship Roosevelt them that maybe we’re we’re losing that understanding of government of why the basic science is involved anyhow last for word of it government does spend a huge amount of time and energy trying to assess research um and the ways in which some of you will have come across this are the dreaded words are a concern of the REA you know well if you hadn’t you’re lucky but you probably will fairly soon so mmm as I said the government gives the government gives money to these various organizations and then so the government gives a lump sum or shall we say one point Oh 1900 million pounds to the research councils and then the research councils have to find a way of divvying that up for two universities the next slide is government gives two point four thousand million pounds to the higher education funding councils and they have to find a way of digging it up to universities and then so the money from the higher education funding council that flat green box at the top let’s divvied up by process every seven years or so which is basically a beauty contest it’s called the red or god you’re taping this hunting it’s oh no no but what the government does in this beauty contest or what has he does is they ask universities to send in a list of four papers for each of their researchers or each researcher that they choose to send a paper in for these papers are peer reviewed in some way by panels the panels are so great to them and there’s also an assessment of the impact of the University all of that stuff on the previous slide so the university has to work a little story like that about the impact of its research have lucky people like rod are busy right now kind of write that story for God’s go and look there similar person in sin and rose or a budino or whatever and then the the results are published and all the Vice Chancellors I have to go open the champagne or drown their sorrows it beer and the patterns I didn’t divide a divided up by a formula which is basically the ref result times the number of staff and the rest result is so well it sort of comes out it comes out as a number a kind of average between an average performance at the start grow roughly between five and zero suit you if your ref score is so if your ref score is is is high you’re going to be pleased and get a lot of money if your risk or is low you’re going to be miserable and then all of that feeds into the league tables that that you’ve read in the papers about universities edinboro likes to anywhere has this ember a computer science as this wonderful theme on its web page which shows that Edinburgh computer science was the best that computer science research in the country the last time this was done and furthermore they saw what this it some little sliders so even if you tweet some of the parameters it’s still mess um you know and we might my university likes to go around saying we were 11th in the country and algaas we were depressed we weren’t tenth it gets a bit silly um take that little bits of it city um but of course that’s why universities are now even though you might start thinking all this stuff on the previous slide of well you know you know stop you don’t have to think about it that’s why universities are now having to think about it and people like rod and his opposite number and all the other universities and going around trying to find examples of impact so they can write up a little storage in this huge continent um the other thing I have up here is the research councils they divvy up the money by people send in proposals grant proposals and you know got senior you know the academics in your department writing grant proposals and muttering and so on writing grant proposals grant proposals the money is divvied up because I write a grant save you know please give me five million pounds to look for the Higgs boson and

my peers will kind of review the proposal and say that’s mad and I won’t get money or a maybe on a good day they’ll say okay and they’ll give me the money um but again looking at now with that when you write grant proposals because you’re trying to predict what will happen you have to talk about what the impact might be well they are alive real true flights of fancy put some up to a point and so that’s how this money is divvied up okay so that’s why this is important now what kinds of things that people use to assess research well um of course some of these you see and some of them you haven’t um so much so imagine everybody’s come across peer review you write a paper for a journal or a conference it sent in and you get the referees reports back and that’s something that that’s what the Royal Society we’re doing in the 17th century they were I know you’ll judge by your peers for people you know your fellow experts in the area or if you’re a PhD student in the senior hello experts in your field and peer review is and remains incredibly important of course people sometimes worry about peer review and there’s actually a bit of a debate going on among that peer review in computer science if you want to follow it you can go to this there’s a lot of lot of papers on the web by the president the president for the ACM and the the son of the ACM people who’ve been thinking quite a bit about this because particularly in computer science we have from quite a lot of conferences and people have put in papers for conferences or be involved in conference rectory know that you’re trying to referee a lot of papers very fast and sometimes the referees of course the record quality of a referees report that you’ve done in you know half an hour while you’re watching football is not the same as the quality of a referees report where you spent you know a couple of weeks really thinking about a paper because it’s it’s going into a top ACM journal oh it’s like a bit of discussion about this whether we should be thinking a little bit about the conference culture and there are other ideas for referee coming out in other places our people or some disciplines where people put their papers on the way the head of publication and then lots of people make comments on the web it’s not an anonymous process and period I was quite surprised to find peer review is very different in different disciplines and I talk to people in other disciplines like psychology or medicine but peer review is something we’re kind of all familiar um but peer review is time consuming and therefore expensive and so often when people are judging work instead of going away and reading the paper so look for other sorts of things you know what grants or prizes has this work one you know if maybe if somebody else has already said they think this works good by doing it a big grant I can just take the box that says it’s good and I don’t have any of the papers impact factor of journals is one you may have come across well maybe if the papers in an a you know let’s say one of the top a sea of journals is probably good so I would actually have to read the paper and the impact factor of a journal is the various ways of roughly speaking in see it’s how highly cited papers in general are so if you’ll have a paper if you have a paper in a Jones with a high impact factor all that tells you is it in the past papers in that journal have been highly cited you know my paper in that jail maybe the young hmm you know the one that wasn’t highly cited but it doesn’t matter I’ve got them I’ve got the kudos you see and publishing in that in that film um has another thing people have started using the the Australians for their version of the of the RAF the research assessment produced a ranked list of journals the Australian core journals list and if you google it you can you can find it journals and conferences actually and so some put some institutions some employers have started saying well we’re too lazy to read the papers but at least if it’s in a journal that the Australians ranked a then it’s probably a good paper so they’ve started using that the sky problem with this the Australians themselves then decided that this was a really really bad system and abandoned it in 2011 party because it was really damaging to journals actually if the Australian is this Australian list said your journal was only be been your journal suddenly you know the Germans themselves something complaining about

this and so bleah but the lists are still there being used um of course one of the things you’re all familiar with and I’ll say a little bit more about this in a moment is the use of citations and or other things related to citations like like downloads I’ve got a couple slides about this I’ll come back to that um but of course people are also trying to assess the economic impact and that’s where as I say it gets really hard the wreck impact statements are are going to be a little stories about work that’s had impact but anybody who ever tries to produce a formula for capturing this stuff particulars I say the things like have you put a value on the fact that people wash their hands more optimum that would get sick is hugely complicated plus the pack but when you talk to people who do this kind of policy research people whereas in science we tend to know who did something something over you know when somebody proves b equals NP will remember the name of that person when cheering jerry’s work we remember curious thing an advancing in health care policy or social care you know it may change the way things are done but nobody remembers the name of the person who invented it very often one of the ones i love it i was told by providers of child psychologist is that once upon a time if a child went in the hospital the practice was that the parents were restricted from coming to see the child and that i don’t anyone in the rooms old enough to remember this might my my parents remember this and though and the thinking was that if the parents came to see the child the child cried yeah and it’s upset the child so you didn’t let the parents can’t see each other it sounds extraordinary and then it all changed and now if you have a child in hospital you know them you know they can’t wait up next to the bed and you’ll be with the child oh of course taught us now is finding youngest but that’s a better way but it actually took research this child psychologists called winnicott in 60s who did research on this and say no actually look this is damaging the children it’s much better than how the parents there but nobody now remembers that that was an impact of research they just think of its binding the obvious of having your parents with your child in hospital is better than the charm but i think that’s a very nice example you know how would you know when it cuts to all UCL who’s at UCL UCL complain winnicott except they and then putting together an evidence base for what I’ve just say it gets hugely complicated putting a value on it even knowing that it was it was or is like what chain falls an outcome of research hit very complicated so that’s why this gets deeply murky and so people about all this and this was it’s easy then deposit even murkier if you’re trying to talk about the potential impact of your research well you know we really are you know very damn but that’s what EP SRC you’re doing with grants when you write epsrc grant proposal you do you have to write but a couple of pages on pathways to impact and that’s talking about how it might be helpful but not actually just in the ants are more but also to other researchers okay so I’ve just got a couple of slides about some citation counts um well I had to put this one up for in bro um well for fetal wobbler somebody some of you who personally I’m sure many of you know that citation counts are now becoming quite popular as all sorts of tools on the way to play with citation counts this is don’t do that I leant on the microphone this is so this is Microsoft academic and what Microsoft academic thousand it goes around the way but it proved us up papers and citations and people’s photographs and so on and any it produces this sort of thing so restricted to the University of Edinburgh the person in n broke with the most citations is one Brian charlesworth the person with the second most citations is one my quality is a very eminent mathematician I was amazed to see you in there actually mathematicians always say they never highly cited and there is um humans are naturally he is probably Hot kid many of you in a theoretical computer science area know who he is 10,000 citations neighboring bird Phillip wobbler well of course this was a snapshot couple of days ago you know maybe Gordon’s have a few more citations fine but then you can also look at how many publications they’ve

got oh well Charles worth is clearly where you look Carolyn spot 3 i’m 50 benefits and or you can calculate the h-index the h-index how many people heard of the h-index yeah quite a lot but the 18 X said if you’ve got an H index odd say 54 that means you’ve got 54 papers that have been cited more than 54 times and so in other words if you if you take your papers and sort them by the number of times they incited then you can’t them until the number the number is 100 bigger than the other number where they cross over for Brian charlesworth was 50 49 you see if we sorted by it looks as if if we sorted by a trade exfil wobbler will be second oh I feel what a beat Gordon parking look anyway hours of fun for people with don’t have any work to do there um but also you can see why this is attractive because you can well you can play around with it you can you can get numbers you can put them into spreadsheets and this is for individual people but you can apply the same algorithms to institutions you can type you can work out how many publications in Prescott how many citations the University of Edinburgh Scott what the h-index of the University of Edinburgh is etc I know people complain all kinds of games with with these things um and this makes it fun remember I was talking about government wants to assess how good research is so it can prove that you know it’s doing okay Charles the second problem and this is the kind of thing that government reports produce these spider web diagrams so if you just look down this arm with the spider web this is citations so this is how many citations have paper American papers having well as be a time period but not on slide is it one you know some time period well or a surprise American the US has had more far more citations than there are the red one is the UK the purple one Germany China etc plants citations that’s articles that’s ger d that’s how much money the company spends the country spends on research um usage that’s quite interesting you see because everything’s online these days you’ll go away can’t how many paper times your paper is downloaded well not this something everybody can do now to help their department go out there and check out a robot it’s all system I’m obviously yes you know I’m not so many things are perfect you know is easier than setting up a you know citation them really in a citation cartel highly cited articles patents etc and so you know the UK government or people employed the city Whitehorn produce this sort of thing watching no they don’t they pay our Savior to reduce this sort of thing um patents well okay well it’s all very well but it doesn’t make the UK look terribly good does it I know let’s normalize by population and / researcher oh yeah that’s better look the red one is top on competencies competencies is some too long to explain what it is but it’s the sort of thing you can do with them data mashing on the web I look we’re top of competencies highly find article citations usage articles and look we cheat you see um and old patents we’re not doing so well on patents but japan tour nevermind government likes that you see because that proves to government it doesn’t have to do anything because we’re talking in fact whisper it not that might almost proved to government that we can spend less and we’d still be top so don’t government to see that one but tongue so put it aside more up about citations because um i think it is worth thinking about them because they are so widely used they are very easy but there are certainly minuses about them particularly to do with the accuracy in the bias you’ll know there are wide variations in citation between research areas popkin and wobbler they’re in different areas of you know the looseness into the theoretical computer science there are plenty of terrific researchers in Edinburgh who do things I’ve come visualization or things close to neuroscience or something but those areas have different citation patterns so they think that they have feet up so high also all the people I hope you are highly cited are actually quite own trucks moving around um it was only been accuracy in this stuff you see that the the rock moroseness and the jane

austen’s of the US EULA Martin’s we have you know we have an advantage in this kind of stuff because we have unusual names the John Smith’s and the Mary Joneses and you know don’t only and especially as a lot of this stuff is based on screen scraping my university started getting terribly anxious about citations because the the proper name of my university is Queen Mary comma University of London it turned out that when these things go away and do screen scraping because of the coma they then assign our papers to this mythical entity called the University of London oh that’s dreadful so leave it or not hope she’s still at the camera boy you know me my university we now have a policy document that says when you write a paper you mustn’t put a comma it must be remarried no homo university of london and if you keep doing that then your head of department right in a stiff letter so um which is actually a bit rotten because you know you can reticulan you can slog away and get it right and then the right at the end of the publisher wretched corn back missy but this is how bonkers it’s getting I didn’t say that um but of course when you stop and think about it you know there’s other things we’re all we’re only human which is another way of saying we’re quite lazy know when you’re writing a paper how do you decide what to site well you look at the paper your supervisor wrote about that stuff but may have these five standard paper they always site so you cite them to lansing sort of snowball effect you see so-and-so water and pop Kane of both written sort of seminal text that everybody will you know everybody will just psyched because it we have them oh and so it’s actually quite interesting when we take the UK government when it was trying to do this this red thing that was all about hard to give me up the money they thought well actually you know why don’t we just give the problem to a computer and just use citations to give you up the money they’re not easier than having people sort of refereeing papers all over again and they spend all the money in investigating all this and they decided actually it was too inaccurate to to use for that purpose and they also came up with all sorts of biases I’ve put women there Pam either you’re the investigative there are data sets that show that women tend to be less highly cited than men I don’t know why this is but it may be partly this snowball effect you know um hey man so it was biased against women it was biased against new subfield swords you couldn’t think they thought about you know it’s all sort of fancy mass trying to normalize their whole journals devoted to citations the journalists scientometrics there’s a whole research in so and they’re extra minuses for computer science look not all CS girls and conferences are captured in these databases anyway and so if you compare citation counts in something like Google Scholar which does it by screen scraping and something like Thomson ISI that does it by actually analyzing the journals Gobert they’ll forgive you completely different answers for citation have um I also fact that in computer science you know so what do you cite you cite the journal version the conference version you know it so what that means is that we said one paper that psychic you know 50 times you might have a journal paper that psychic 25 times in the conference paper that psychic 25 times and it sort of makes it a bit you know curate so they’re real concerns about that and I should stop but there’s no clock in here but there’s a clock on my computer so what does all this mean for you well of course the question that you know you don’t need tell any of us the answer the question but it’s always worth asking if you’re a PhD student is what am i doing this 1i doing the research what matters to me about where I’m trying to get to and that may not be something you share with your supervisor or your mate saw it you know maybe it’s beautiful dream in the back of your mind guys I was talking to one of our PhD students from Kenya that some sort of reception for new PhD students and I just sort of said in a wife one who come to Queen Mary and he said well when I finished my PhD said I want to go back and make our hospitals work better that’s a good if they think these gnomes y 3 30 that’s what he’s thinking about who wants to apply the knowledge and skills to make the hospital’s works better from you can see that there was going to be a lot of IT involved in that and for that you know the metrics is well you think about why you’re doing what you want to do and

what your measures of success will be and that may not be any of this stuff it may be making the hospitals in Kenya better or proving / miles theorem or whatever a kids but we live in the real world and so the thing to think about is okay to get where I want to go I don’t have to get my next job I’m going to have to get my papers published how others perceive and measure my work and so it’s worth thinking about well how do everything’s different you can’t read every discipline if it is different but it’s just worth going and looking you know who are they um who are the people were the people in your field who are doing really well maybe not the people who are you know 60 because they long wait for you but the people ten years ahead of you who are doing really well where do they publish you know what sort of journals and conferences do they publish in how many how many papers are they publish do they publish lots over they publish not not so many what are their citations patterns look like oh and make sure you keep records of everything because you know we in this culture where all of this stuff is being recorded them and looked up you know so look at your own record on google scholar and all these things you know not every five minutes that would be a bit sad but occasionally you know look at her own citations look at your downloads is it easy for people to download your papers I mentioned open access you know have you got have you put your papers is it easy for somebody to find your papers online and download them or do they have to go through sort of 15 layers of rubbish on your departmental website well right if the 15 layers of rubbish on your departmental website well maybe go an offer to help the person who set those 15 layers of rubbish up and is it you know just just think things like that think about record kim a record of conference acceptance rates I’ve always looks good on yours I think I’m going to finish with this this is so if you don’t phd comics you need to know PHP comic book said that online every every few days um you know I think this sums up the impact of research for PhD students well I bet if you can read it you know you start off with something very technical and it ends up with your grandma saying I’m wearing his hat to ward off this evil so when it goes through all this other and all this other stuff and on that note on that cartoon and one more thing don’t forget to have fun doing your research he passed off very much for a very informative talk we have time for a couple of questions yes yeah the hyperlink was invented to the hockey a misfit went to each other’s research so why is it that we’ve gone so far since image and make sure that I played until our papers to continue and we can’t wait very well teaching research oh well I don’t know because we’re too busy doing other things I think I think there are there are things that are trying to sort this out you know I know people who haven’t never quite got to grips with things like mendeley myself but there are people who are who are thinking about that but yes it’s a good it’s a good question isn’t it if I has both of you know what makes people do things it can be just a it can be a dynamic has come you know they find they find it compelling think bill or it can be their bosses impose it from on high I don’t know I think some of it is linked to the sort of publishing standards seeing you know so if you look at something like easy chair many conferences now go through an easy chair well it’s easy chair to start enforcing that or making it easy but then it’s also linked to this business of open access and putting your papers online nine our world well most of the world to computer science I know about it’s no big deal to put your paper online but of course there are other disciplines like medicine social sciences where people are really punctilious about obeying the rules at the journal and you know if nature says it and keeping quiet before they’re published and so on and that so that’s not precise answer about had legs but it’s sort of create you know it creates barriers which is sort of your question yes do you feel the interdisciplinary researcher transdisciplinary research chance to sort of news out when measuring impact because it may not clearly i think hit one place and that was that’s fun i think like I kind of stopped because I come to yes that without that is a very important point that I don’t get how to make and I think it’s and that’s another reason I said that you know heskey doing

this resting whether they’ve trying to fit in the money up to they didn’t use citations because of exactly now and it’s what’s all in favor but you know career wise it’s always difficult doing interdisciplinary research because you know you’re doing this great project which is the borderline you know biology and geography and then you’re applying for jobs and you’re not a proper job refer of your life copper by religious there’s one more thing I meant to mention is a book which I colleague lent me only okay give me a week to actually read it but fix time it’s called marketing to scientists how to shine in tough times and I’m also this I think I looked at this hotel honestly you know we can’t get me through business and no really it’s good we should read it this it has a lot of the stuff you need but it’s picking up on a number of these a number of these points right so I’m sure I mean I certainly have other questions that i would like to ask but in the interest of the poster session it’s probably best if we thank Ursula game and ask your questions in private