Lecture 1 | Ubiquitous Computing | Marc Langheinrich | Лекториум

okay so let’s start an ubiquitous computing you’ve probably all seen this not long ago a computer’s looked like this they filled rooms they were produced by companies that had beforehand build power plans and you know railways and all these things and then not so long ago but maybe for some of you definitely before you played with these they were they were able to shrink them down to a desk size and the company who dominated this was previously doing typewriters so a very different kind of company then and later on there was a they fit into the palm of a hand and one of the hot companies was a entertainment company that had a movie studio and created the Walkman and today of course as you know a few days ago Apple became the the most valued IT company even higher value than Microsoft and they are interesting enough now they did away with the keyboard and they are now dominating the the music industry as well as in the publishing industry soon maybe with this creation of them so the computer has changed significantly in the last 3040 years we moved from a concept where we had one computer for many to a brief time where there was one computer for everybody to today where we have this idea that there are many computers for each of us I’m sure each of you carries a computer with them right now in your pocket because these little buggers are probably already more capable than the Space Shuttle was ever in terms of raw energy computing power okay so there’s a clear trend here that the size of these things gets smaller and smaller and the numbers gets higher and higher so the obvious question then is what happens if we take this and move into the future what happens now what is what is the next step you know maybe this is what we’re heading to smart dust where they get as small if you’re just going to sprinkle us selves in a boatload of computer dust something okay so this is the question then and and we can we can look at answering this question with a very brief look at the the metamour forces that computers underwent in this journey from the rooms to the the pocket so first obviously computers were simple calculators they were used to compute ballistic tables for you know getting the cannons aligned and stuff and maybe also the books of company or something later on then it became much more of a data processor there’s an interesting quote from one of the early inventors of computers who said if it should ever turn out that the basic logics of a machine designed for the numerical solution of differential equations coincide with the logics of a machine intended to make bills for a department store and you can hear this contempt in there like I would regard this as those amazing coincidence that I’ve ever encountered so people couldn’t even imagine that you could do anything else with computers but crunch numbers to fire a rocket into space or something like that and here we are today and we’re using our computers to Twitter I mean you know I’m not sure if he would have lived to see that there Howard okay and then of course the computer became a personal tool because it fit on our desks it you know secretaries of course could use it to do work processing and these are early ads where you know the world was good you know the wife is happy in the kitchen and and the husband you know checks checks the takes the investments great these were the days and then later on a computer became an information devise I’m not sure if you had something similar here this is a exam from Germany and from France they actually used the phone system to hook up your TV as an Information Center where you could look up pretty much like the web today train tables and and movie showings and these things so that was the Roaring 80s and and then of course not sure if you visited Google a few days back and also contributor to the time killer of the week pac-man 1:30 so that was a phase where well definitely I grew up That’s not me but could have been me and today the computer is a communication device yeah we have a full Linux system in our in a pocket this is the the n900 this is the the OLPC the One Laptop Per child and they have a little camera here and we use it as a communication device so this is very different from the original ballistics computing computer that of

old and and soon the question is of these computers and work with you it might become an invisible and omnipresent background assistance here’s a cover from from 2007 three years old by now from The Economist where they had this idea of what could happen in the future with a with a computer so you can see here several elements so you could imagine that you know your children and your dogs or pets never get lost anymore because they always know where they are you could imagine that you know your body is always monitored and if there’s some condition your doctor will be alerted your car will never run into trouble again because it will know what’s ahead or your peas you know most importantly you’ll never lose them again because you can find them anywhere where they are you could even imagine then when we move to that smart dust that you know will jump sprinkle the world with them and then you know the ground can say if it’s too dry and and the water can say okay then here it is the war so the computer then moves from this calculator data processor communication device toy into something maybe like an omnipresent guardian angel that’s always with us that we carry with us that’s everywhere around us and that will make sure that all is good this is this is important okay so this is roughly the vision may be behind this ubiquitous computing although maybe not in the full extent then it goes back to this gentleman Mark Weiser who was a research at Xerox PARC who died unfortunately from cancer early and and he wrote this influential paper and in the 90s where he said the most profound technologies are those that disappear they weave themselves into the fabric of everyday life until they are in this dish indistinguishable from it that’s a really hard worker you tried three times in wrong indistinct so remember back in the 90s there was this you know one pc / / secretary whatever vision there it was a very different time from today and he already had this idea that you know these things they disappear from not maybe the physical size but from our idea of interacting with a computer we didn’t turn it on anymore and start it you know booting up we just get out of the pocket and just move the map and look for something we don’t even realize this is a computer it’s just totally normal that that’s a map and if you know 20 years down the road maybe people with a paper map will be looked at it’s like what are you doing there what is what is that what you’re holding so it’s a very different idea of a computer and the motivation that Mark Weiser had was basically that he would wanted to integrate intuitive computer interfaces into these at the time boxes with keyboard and and the monochrome monitor and make the computer a tool for the everyday not only for the everyday work but for the everyday life everywhere so it always have it with you and you could rely on it and last not least to make the computer always a bit smarter than today so that they would be aware of each other so things computers environment altogether let’s see what that means done so a good idea a good way to explain this vision is then contrast it with a very different popular vision and that’s the idea of modeling the world inside the computer it’s called virtual reality and in contrast the vision of the biggest computer is to integrate to put the computer to disperse it into the world so we put all that stuff basically in the form of very tiny computers into the world and then you know good thing will happen the computer will be able to help us with every day course okay so here’s the vision then what I can lead to this is currently very hot topic the smart grid where the idea is that the the typical model of your ears are the power plants and they always have this you know megawatts of power output nuclear or coal and then this is in the grid and then there’s some little part that will you know at the end push it down through the houses that’s really a problem it’s inefficient because if there’s a lot of energy demand then you have to ramp up the the power plans and if there’s suddenly not much energy anymore then you have to you know lower them again because if you have too much energy here it’s bad right and this thing will just kind of go up in smoke which is not good so this it’s a it’s a struggle for the power plant for the power companies and the vision is here that you integrate all these different sources of energy into something called the smart grid and you can interface it directly with your home and allow the two to negotiate what kind of energy do you need when so the idea would be that the freezer waits until the electricity is very cheap if

there’s a over supply maybe there’s strong winds and there’s lots of cheap energy then the freezer starts freezing really cold and then if there’s a maybe no wind then the freezer and stops and and not get doesn’t get much energy because energy is expensive then and the same way the washing machine that could wait a few more hours I started in the evening and maybe at four o’clock in the morning it decides oh now is a good time now things are cheap let’s start washing because at the end I don’t really care when in the night it starts it just kind of negotiates when’s the best time maybe at 3 a.m. my time California has great Sun and then there’s a lot of energy available and all this is done with the help of something called a smart meter that thing is kind of a broker that talks between the house and these mirrors or something and maybe even if it’s so cheap you might as well just kind of a so f energy provider can avoid these spikes and you can even at night to charge your e car because there’s so much energy that you might as well just kind of store it in here for what so this is a vision of the future that could be possible with these smart devices smart homes smart meters that talk to each other now visions are good there are many visions here’s for example another vision from a few more years back this is a vision from the 50s this is a you know fortunately German magazine hobby that that had this vision of the world in in 20 years and they didn’t die cars would be in 20 years and and it said this particularly streamlined model here from the Year 1975 is an elegant two-seater with a nuclear-powered engine in the back didn’t happen and I’m not sure what kind of cars you drive here but we don’t really have nuclear powered engines usually so lots of visions that people can come up with not all of them are really fruitful so why would this vision of a smart meter smart grid actually be any more realistic than a nuclear powered car well there are a number of reasons why the forecasting in this case is easier than in other cases for example here’s a number of forecasts the moon colonies that were the big thing of the of the 60s maybe they were too expensive they’re worried ds4 actually living under the sea you know that boy he was not only expensive but kind of uncomfortable as well people had dreams of flying cars expensive and probably very dangerous if everybody just kind of slides around the city already on the street level here it looks very dangerous to me the car traffic and of course video phones I mean they had video phones in the 50s and thought every year how it’s going to be the year of the video phone never happened maybe Skype finally is the videophone we’ve been waiting for maglev trains you know super high-speed supersonic flight was a time when people thought yes in the future we’ll be you know driving flying with Mack 26 or something in the sky and the Concorde was the only one that ever built and it stopped operating nuclear fusion household robots there’s lots of ideas around that never really get anywhere here’s important predictions for example future that’s also always a fun like this one from Thomas Watson the CEO of IBM at the time I think there’s a world market for five computers that was a big mistake I guess and and here the idea of SMS no one really thought these short text messages would would go anywhere it was just a byproduct and and no one actually advertised for it so anyway forecasting is hard there’s a quote from a brand fair and he’s a chief Disney Imagineer that they’re really bad at predicting how a given technology will be used and for reason okay so why should then this vision of a smart meter that vision of ubiquitous computing be any better you know what what makes this a good vision so here are five technology trends that point us into that direction and which might make this more than just another nuclear-powered car idea the first one you probably all know is called Moore’s Law and that is a is a something that Gordon Moore the founder of Intel stated back in the 60s where he said that the processing speed and storage capacity roughly doubles every one two half years everything gets cheaper smaller and faster basically and it turns out that you know if you ask experts they always say well it’s going to hold maybe another ten years and they’d say this every year so so far if you ask people today well maybe another ten years this will hold it’s been going on like this for the last twenty years so let’s see how far they can get these things down I think at one point when we had a few nanometers then we’ve reached the atomic size of atoms and then I think we all have a barrier there so far that works well in consequence we have very small chips this is for example it’s a calling

radiofrequency ID chip it’s the mood chip body touchy it has a couple of micrometers size here to reseed O’s and and it’s full full chip with actually a radio module on there and stuff very varying stuff this isn’t really exciting anymore you know this is probably a 128 megabytes thick or something which I don’t think anybody produces anymore these days but it’s certainly enough to hold entire library of books there sure if this this is a famous German antique light in seek light whatever you get the idea right thank you thank you so it all fits on you know it used to be shelf space and now it’s on this little stick and these sticks by now have left 64 gigabyte or something you would probably fit the entire Library of Congress on this thing okay so everything gets smaller cheaper faster let’s go that’s one trend the second trend is the progress that we have in communication technology this was you know the past we have this nice little telegraph lines and today we have you know modern telecommunication technology we have 3G we have now 4G everything is Wireless and fast there’s a there’s a practically if you walk through through st. Petersburg today there’s every other meter there’s some free Wi-Fi hotspot or something I can go online you know some phones they just kind of pick up the hotspot and connect me automatically and in fact then my phone can out of my pocket you know download the weather forecast or my email or something without me doing anything so that’s already there in principle we can even go farther and and create wireless communication without any power source this is called contactless chip cards these little bank cards that you have for example they’re similar to that radiofrequency ID technology near-field communication we can talk about these things later this is interesting because they don’t need a power source to communicate wirelessly they get the energy from the power signal of the transmitter so they can actually send data and receive data without having any power source that’s that’s pretty cool there there’s also an interesting concept coming up these days communication by touching it’s called NFC technology where I can take my phone and just touch a certain object and by this act of getting very close in the millimeters suddenly communication happens and I can download for example a ticket or something so you could imagine that in the future there’s a you find in your sports club some racket you just hold your phone next to it and then it will tell you who’s the the owner we have to return it to or who you’re ripping it off from you know whenever whatever you decide to do with it you can also see this in Japan already where they have the the subway system basically is they have these little readers here installed at the entrance they look kind of like this and what you can do here this is the entrance and this is the exit as you can clearly read here and and all you have to do is you basically hold your phone next to it when you enter in when you exit and the system then computes your your fare from it this is this is some communication by touch this an interesting paradigm there very trend there is a wealth of new materials being developed these days this is for example smart paper and flexible oh let’s colors and monochrome monitors that are bendable flexible for example the smart paper is interesting because also called off an electronic ink it offers low power and very nice viewing capabilities because it doesn’t really have backlit it’s it’s basically just lit from the sunlight so you can read it in the Sun it basically consists of your mini mini micro capsules which in turn are filled with these little tiny pigments here black and white and which are charged differently positively with a negative and there’s a little fluid and they can kind of you know kind of jelly fluid to can bounce around there and when I then put some current in here some sorry not current charge then the I can basically control is the white or the black pigments go up or down and then if I stop the charge they basically remain in this position and have then written text on my electronic paper and I can cut the energy and I don’t need to refresh it anymore just like like that and the eye basically then sees the top the black and the white pigment so it’s great because it’s basically no backlit a system so I can read this really nicely and sunlit and of course it doesn’t need any power once the page is set up it has power requirements and this is examples

here the early versions here and today I’m not sure if you know that the Amazon Kindle the e-reader or the Barnes and Noble Nook or nuke actual pronounces right they have at least eep-eep papers they’re okay in the future who knows maybe there are these visions of having a bendable computer where you can just roll out my screen and then you know put together again the biggest problem I guess with these things is that the power source again you probably need this all to be power and a bit of bendable stuff there okay so that was three then we have the fourth big trend out of the five second to last the sensors there’s lots of cool work and sensors going on probably the the thing a princess is pretty much old stuff so biometrics have seen lots of interest there are sensors for temperature humidity acceleration GPS sensors this is a very old picture where these stuff was a was quite big these days they can be integrated into well more phones not but you know even small watches have these days very small GPS receivers in there the trend goes through digital wireless and even a low or no energy and again if I have something that has not much energy requirement I can put it in a lot of stuff so here’s an example well fingerprint sensors yes just me the energy consumption is moved to the collection fish consumption getting much more energy to produce a single life now it’s a good question whether the the overall green carbon footprint is better or worse but important is if I have something that’s low energy or even no energy now show an example in a second then I can put it in a device and I don’t have to change batteries or plug it in like this stupid laptop which dies like every hour or so and that’s a big win so I can I can build stuff that that the lay basic allows me to sense temperature without without needing power I’m here is a well this is it well actually does that does um actually it’s a good question is that actually has I don’t think that needs a battery either because it takes the power from from acceleration so here’s a work from Fraunhofer they instrument the golf ball with a sensor inside that basically measures acceleration so this is called the e grain and and right so it has a built in power source that that last for 200 T offs it can it transmits on 2.4 gigahertz Bluetooth has about 5 meter range and measures once per second and and the idea is that you can you know put that in the golf ball and then just kind of have this personal trainer that tells you exactly how you hit the ball and you can hit that ball 200 times and then let it run out of power obviously if if you’d had no energy a requirement then you could we basically get this thing as as often as you want until you hit it in half or something I guess so here’s an example for something actually does not need power that’s an energy sensor that measures sorry a temperature sensor that measures temperature it does not need any power supply because it gets the energy from the measurement difference so why by getting this exposed to the to the change in temperature that’s where actually I get enough power from to send then this information off wirelessly so there’s a PS electrical materials in there that if a certain temper range has been reached certainly one degree or something that gives me enough power to send up little squirt of wireless information saying one degree higher so you can barely put this in your in your teacup and then it will alert your PC whenever the tea is you know drinkable not not too hot on you and you don’t have to change batteries for this thing you just put it in your tea that’s that’s nice it’s actually well that’s kind of funky with the tea it’s actually in use already in entire pressure sensors so modern cars have sensors in the tires that alert you if you have not enough air in the tire and this way they do the same way it’s a pressure sensor that gets the energy from the the pressure that’s in the in the tire and this way they can communicate it to the the car very short distance but that’s enough and tell it alert it if there’s not enough power sorry not enough pressure in the tire anymore and then you should get some air okay so and last of these five trends Venice localization I’m sure you you know that you know all these phones these days have GPS built-in you have Google Maps on these smart phones so GPS

assisted GPS you have Wi-Fi localization something called something called that I forgot now or uwv stands for its ultra wideband thank you ultra wideband so lots of cool stuff is enabled with location you can find friends find your lost key or your daughter or your dog or find even cool places you know where to hang out there’s lots of good ideas what to do with localization you can find your and keys pets luggage parcels you know tracking shipments find maybe weapons if you’re if you’re police force and you want to make sure that no one takes the wrong person takes them with you rental cars is very popular these days if you have a rental car company they usually have a GPS receiver in so that they invite the cars if they’re stolen there’s also lots of interest in in Road toll so that you can find cars that actually have not paid the road tax and of course you know if your very expensive things you can you can find them again there’s also ideas maybe again to find children there are companies that offer these little funky watches that are in fact a non open the bill so once you put them on your kids you know it’s very important feature but all right yeah not not in the neck here but you know the hand works it it looks like a watch alright so there’s a fun factor here it looks like a watch day they purposely did this this way although maybe you could put them on the leg as well and then you can just kind of you know open that map and it shows you where your child is so that’s good I don’t know I guess if you have there’s also interesting stuff you can obviously you’re your spouse wouldn’t would get the idea if you if you offer that as a present so she’s not that stupid I guess but there are little little things he’s little GPS tracker that’s maybe the size of a USB stick it has a strong magnet and the website says that you can put this underneath the car very nicely to track people that you’d like to know where they are or again and this is in fact the same model but without the fun blue color that’s in fact already use for for prisoners on parole in many countries they get these these little bandits on the foot but you could also imagine maybe you want to track people like me and Fabio who hang out here in this country not to make any any creepy stuff of course for our own safety right so that’s that’s important okay fun ideas you could do that so then all these trends basically can lead to something exciting we have lots of cool technology that are there that are being developed that are coming now into out of development in into production and then where does it lead to well one vision would be that idea of a smart object that smart object that can sense the environment and talk to other objects again remember how we integrate all these computing power and sensors and wireless communication into the individual artifacts maybe not into the dog maybe in the color although there’s actually there are these things that are you can inject directly into your dog into your rubber duck in the car and other things and then they can talk to each other right and now you could you could I mean if you’re really skeptical you could ask what could they ever talk about you know what should like what should these guys be talking about this is they have a lot to talk about actually I’m telling you guys this is not right so here’s a cool thing they could be talking about this is a work from Philips where they had the the toothbrush talk to the mirror and on the mirror there’s a little game with a little parrot or whatever that plays a game with you racing you for brushing your teeth and this way you can get kids excited about you know getting that that teeth healthy and clean maybe you could even you know compared with your your friends parrot and you could see which parrot has the nicer teeth or something so you’re you race each other in the evening so that’s that’s kind of fun stuff so even if you have this kind of weird combination of a mirror and toothbrush they could be talking about exciting stuff okay right so your high school is and everything okay there’s maybe not for kids but for adults there’s been a project in the UK together with BP which is in the news these days without because through other reasons but back then they were not as as preoccupied and they had this interesting project where they would connect the helmet the breathing apparatus maybe your your diplomas of what you’re able what machine you were

able to operate and and maybe other rules like you all we only can enter that area if the two of you are there so you have a backup partner and so forth these things would all talk to each other to make sure that when you enter a specific area that for example you couldn’t go there alone or if you operate the machinery you’d have to have the certificates on you otherwise the thing would just state that you couldn’t operate it here my workplace safety all these things will talk to each other and make sure that you’re not trying to outsmart them okay and then the vision – maybe you could also imagine that we can ourselves become smarter here is an example of something like that smart glasses if you don’t see there in the corner is a little prism and on the side there’s a little laser that can actually project an image directly onto your retina and and you can imagine here’s a story line sometimes it doesn’t look like from from Satya actually how we call himself because no one can actually pronounce his name otherwise so Satya would say you could wear a pair of glasses with a small amount of face recognition built in and then you look at a person and his name would come up in a balloon above his head right so you could kind of uh hi Jamie how is it going you could instantly know who this person is you would look at a tree and it could tell you if it needs water you could you know see your dog and the dog would say if haven’t been out for a while and of course you could look at your spouse and you know be reminded of important facts that you usually forget great vision you you get the idea we can become smarter people with this technology this was back then the early models were kind of heavy on the head today you have a bit more stylish stuff I guess I’m UN to that so glasses I think from another company they’re not sure if that’s really qualifies as stylish but with that green stuff maybe it does again they have a little mirror here prison and then you can see basically if you if you get really close you can try this later it looks like your um you have this this display kind of hovering strangely above you in the field of vision and and that works actually fairly well and in the future could even imagine there’s a project happening right now at the University of Washington Seattle where they’re trying to build the same thing into a contact lens and they’ve been at the stage now where they have built the first prototype with the with the connections on there as you can see they’re not quite yet very small so probably if you put that in the eye you don’t resi anything but they put this already in the eye of I’m sorry mice so little little animals and the animals were happy apparently you know there didn’t die or something or head so that’s good so you can imagine that there’s a in the future you might have all this this the display the processor and and again you can see the battery in here really that’s kind of heavy on the eye so you imagine again the same thing as a sensor the the energy is transmitted wirelessly through an antenna maybe something that sits on your on your hip or something and they’re just kind of wires the power in to interviewer glasses yeah maybe you can even hook it up to some mechanical thing or something lots of ideas but hopefully that they’ll figure something out to get this a bit more nicer looking okay so um lots of visions there and I think that’s already what I wanted to say so we don’t really have to keep you any longer than then you already have been here so the bottom line then of today was just I want to illustrate to you this this vision of ubiquitous computing this potential of the vision of when we think of the computer not as a communication tool but as a guardian angel that can be integrated into lots of things so things places and routines could become computerized digitized and and that has a number of exciting and also exciting consequences there are huge challenges ahead there that I’m very interested in infrastructure so how to build systems that actually scale up to these huge number of nodes usability how do you actually operate a smart lens maybe in your eye with question I mean there’s certainly no keyboard where you can kind of you know drag like a window to the left side and of course and that’s the topic of tomorrow then security privacy issues how does that that digitalization of everything we do everybody we see everybody we talk to how does that affect our lives and how do we have to design these systems so that it doesn’t become the Google ich nightmare something that we always read about so that’s going to

be the topic of two more then look