Jonathan Lipps "Appium: Mobile Automation Made Awesome" SmartDevCon 2013

thank you how is everyone yay there’s less people here today than the last two days I think some monster is devouring smart Devgan attendees one by one I’m afraid so yeah my name is jonathan and i work at a company called sauce labs and we do automated testing in the cloud so i’m gonna talk a little bit about automated testing so if you aren’t aware of that then you’ll learn a little bit today and one of the things that we’re really interested in is automated testing for mobile apps because that’s a new area that not a lot of people have taken advantage of yet so appium i’ll tell you more about it in a minute but it’s an open source tool for mobile automation and I’ve had the privilege of being able to work a lot on this tool this year and build up a really great community so I hope you’ll enjoy the presentation and join the IBM community for testing your own apps so a little bit of an introduction we hear things like mobile is taking over the world a lot and it’s true this conference I think is one of is a testament to that fact this conference is about smart devices not just mobile but I think the trend is true and real that smart devices smart phones are more ubiquitous they’re everywhere and it’s a little different than what we’re used to with the web with desktop computing and so making sure that our apps running on these smart devices work well is also going to be different than the way that we do that for desktop apps or web apps so it’s a really important question how do we make sure that our mobile apps that we’re writing or our smart device apps that we’re writing are at a high level of quality all the time I think everyone agrees that having a high quality app is important if you put a low quality app on to the App Store or if you have some kind of bug then all the users who are using that app will give it one start and you’ll probably lose lots of money and it tends to be much harder to change your application once you’ve published it to a store you can’t just make a fix to the javascript file and upload it to your server like we can with the web to fix a syntax error if your app doesn’t work correctly depending on the platform maybe the worst is Apple it could take weeks before your users can get a bug fix meanwhile you’re getting lots of negative reviews maybe your business fails I don’t know it could be really bad so QA is very important but at the same time it’s painful and the way that that mobile developers tend to do this these days is get as many devices as they can install the app on the devices and and use it manually this is unfortunately very time consuming and it’s easy to miss cases that someone else using your app might run into but you’re not thinking about it’s also not the kind of thing that you want to do every time you change a line of code because it’s so time-consuming but unfortunately the more complex our apps become the higher the risk that we are going to introduce a bug when we don’t realize it the more that our app has different pieces that depend on each other the more likely it is that when we make some small change to one piece it actually has a change across the entire system and if we’re not testing every single bit of functionality every time we make a change it’s possible for bugs to creep in and this is a really bad situation because of all the reasons I just mentioned so the solution to this is something called automated testing how many of you are already familiar with automated testing okay everybody great how many of you actually do automated testing for your apps less than half okay so that’s still pretty good a lot of people haven’t done this at all and in a room full of mobile developers I would have expected that maybe it’s even less so those of you who have done automated testing already know the kind of feeling of confidence and power that it gives you it enables you to have a fast development cycle while maintaining a high level of quality and for the one or two of you who maybe don’t know what automated testing is

it’s the practice of writing code to test your apps rather than testing your apps by hand and there are a lot of different ways to write this code so that you can run a suite of tests any time you make a change and you can know immediately if something is broken in your application so the next thing that you can do once you have a bunch of automated tests written as you can make use of something called continuous integration now you probably also have heard of this yes yeah some nodding heads great so continuous integration is made possible by automated testing and with the addition of automated deployment so this has become a very common place in the so you might have a suite of tests that run when you change some code for your web application and then you have some kind of server that knows when all of the tests have passed and are successful and it can automatically at that point deploy your code to your production environment so that you don’t have to wait days to deploy new code you can deploy code whenever all of your tests have passed or whenever all the tests are green as most people say so this makes for a very fast dev cycle where all you have to worry about is writing code and then your tests take care of making sure that that code works well and the continuous integration server takes care of deploying this automatically to your production environment so this is kind of how I like to visualize the best version of the development cycle I call it the dev cycle of optimal happiness so there’s a couple different colors here they mean different things the light color on the top and right are things that you as a developer are responsible for the darker color on the bottom and the left are things that can happen automatically with a continuous integration workflow so here’s how it works whenever you want to develop a new feature it’s always a good idea to start with the hypothesis you’re making a claim that hey if I build this feature something will happen maybe we’ll sell more copies of the application or maybe some other metric will change but you have to have some way of of making a claim that when you put this feature into the world it will change positively for your company or for your application if you don’t have a hypothesis that can be tested you’ll never know if the work you put in is having any effect so it’s always a good idea to start with some kind of idea of if I build this feature it’s going to impact us positively these one or three or five ways so that when you’re done you can measure and see how well you did compared to that hypothesis once you know what you’re gonna do you can write the code to make this new feature happen or if you’re a test-driven developer you can write the test first and then the code but whichever way you do it you write your code and you write tests and you always want to make sure to write tests for each of the new features in each of the new cases in each of those features that you’re building then you can push that code to your shared repository like github or something at that point everything from there to deployment can be automatic so your continuous integration server can detect that there has been new code pushed to your shared repository it can pull down your new code and your new tests run your new tests as well as well as all of the old tests for your application now assuming all of those tests pass your continuous integration server can then automatically deploy the code to your production environment it can run a series of other tests to make sure that everything actually that you care about in production still works like you didn’t crash the server on accident or something and then you can automatically gather metrics which can be reported back to you so that you can evaluate whether your hypothesis was successful and then you can start the cycle again now of course what often happens is when you push code and your CI server runs tests is you find that tests fail which usually means that you have a bug so then you can go to the middle stage fix your bugs check in the new code with the better the better code or the better tests or whatever and then the cycle continues so I put this picture together with web applications in mind because

that’s that’s where you can do this the most easily unfortunately it hasn’t been as easy to do this in the mobile world because it’s not as easy to deploy changes and it hasn’t been that easy to write and run automated tests for mobile applications so that’s where appium comes in appium is a cross-platform tool to help you automate native and hybrid mobile applications and you can use this automation for testing so it gives you the piece in the bottom right corner where a continuous integration server can run automated tests now it doesn’t give you the rest of it other people are working on those parts right now but appium is one of the pieces that you need to be able to have this really awesome workflow for mobile application development so as I was saying it basically makes automated testing and therefore continuous integration possible for mobile in a way that was very difficult before so that’s a little background introduction of what appium is all about but why why does it exist in the first place what else was out there when we decided to start working on appium why are we feel that we needed this tool specifically a little under a year ago when we started working on appium this was sort of the landscape for mobile automation so there were some tools that already existed for mobile automation but for a variety of reasons we didn’t think that they were suitable for the kind of workflow that we thought would be the best so there were a number of tools for iOS apps and a number of tools for Android apps but for the most part there were not any that considered themselves to be cross-platform so if you had if you had an iOS app and you had an Android app you had to use a completely different language and framework in order to test each of those even if your app was identical across both platforms and so that’s one of the reasons we felt that we needed appium and you can see i’ve put it in the very middle of the slide to say we’re trying to enable you to automate both platforms and since then we’ve expanded to even more platforms including Firefox OS and as you’ll see later I’m hoping to convince some of you to help me port it to even more platforms that we’ve heard about during smart Devcon so out of curiosity has anyone used any of these other frameworks yeah which one will you talk okay same for you yeah cool cool so yeah so robot ian is is a Android only and anyway I’ll get into some more of the specifics in a little bit but appium also has a pretty strict philosophy I guess you could say which we could put in the form of four different rules that we felt that any mobile automation framework needs to obey these four rules in order to be the best possible framework for mobile developers to use so the first rule was that we felt you should test the exact same application that you’re submitting to the marketplace so what this means is that if you have a test framework that forces you to compile in some kind of third-party library in order to run the tests that’s bad because you’re not actually testing the same application that you’re submitting to the store you’re testing a modified version of your application with all of this test harness code inside it as well and we felt that it would be great if you could test a binary that was the same binary as is being shipped to your users that way you have a high degree of confidence that if all of your tests are passing on this binary then it’s exactly what your users are going to experience secondly we felt that it was a bad idea to force people who are writing tests to write in one language or one framework we think that if you like writing JavaScript or Ruby or Java or PHP or Python or go or C you should be able to use that language to write your tests you shouldn’t be forced and locked into using just one language similarly we felt that in automation framework for mobile shouldn’t reinvent the wheel it shouldn’t come up with an

entirely new model for automation there are other successful models for UI automation and we felt it was important to use the one that is already the most standard and I’ll talk about what that is in a moment and lastly we felt that it should be open source it shouldn’t be owned by any company or any individual it should be owned by the community and everything about it should be done completely in the open and this was not the case with some of the other projects that we were looking at so from the very beginning of appium every every commit has gone through a pull request and review process and all of the issues have been handled on the public forums and the github issue tracker and so on and so it’s very important to us that this is a true open-source thing so sauce labs does not own it I do not own it it is it is open-source in every sense of the word so then taking the frameworks that we saw earlier when we evaluated them to see if they met these four rules it looks like they really didn’t and so that’s why we felt that it was a good idea to put some effort into building something that did meet all four of these rules now you know maybe it’s not a bad thing for some of these other frameworks to not meet these rules maybe that’s not what they’re trying to do but we felt it was important to have something that did try and attempt to fit this philosophy so enough about what appium is and why it exists I have a little video of demo I put together and what this demo is is a cross-platform test so it’s a test written in I actually think I wrote it in JavaScript but it’s one series of commands that are run first on an Android device and then on an iOS device with an app that’s been developed for both platforms so it’s the exact same test code and it will run once on Android and then once on iOS so you can see the the proof of a pians cross-platform compatibility so on the bottom of the terminal window is the appium server and so you can see it’s starting up and starting to talk to the android device and the top window I’m just using a test runner to kick off the tests now the application that we’re testing is a photo app called woven that takes your photos from Facebook and Flickr and lets you view them in different ways and so right now the test code has found different UI elements and interacted with them enabled us to log in performed various swipes found a picture to inspect waited for the you to fade away come back in we can we can move left and right and up and down using a variety of touch gestures and we found the settings for my account and that’s where this test ends so now we’re going to run the same code eivol alt tabbed over to the iPhone simulator and it’s doing the exact same steps except this time for the iOS version of the application and you get to see some pictures that I took on a vacation a long time ago I think they’re pretty does anyone been to Guatemala it’s a nice place they have volcanoes there probably no volcanoes in Poland that’s good safe very safe here yeah so and then the iOS version of the app has a logout button so we can click it to logout and and finish this version of the test so that’s what a p.m. can do so that’s what it does and let’s talk a little bit about how it does it first of all this is what appium currently supports we support testing on real devices on simulators and emulators we support testing hybrid apps so apps that have a thin native layer with some kind of webview component to them so we enable you to test both the native portion of the application as well as the the website or web application that’s inside of the webview and we enable you to test mobile web so if you

just want to use Safari or Chrome to test a web application on a mobile device you can do that with appium as well and there’s also a robot support so that you can have physical touch gestures if you want which is obviously very experimental at this point so appium is basically a simple web server that creates and handles webdriver sessions has anyone heard of webdriver one or two people how about selenium is anyone heard of selenium more people so what webdriver is the the new name for the selenium project it’s the the newer version of selenium that has a better API so when you write a p.m. test code you’re actually just writing webdriver test code so if you’ve ever written a selenium test or a webdriver test for the web it’s exactly the same so as I was saying the selenium webdriver is an HTTP protocol so you you send commands to the server using you know typical HTTP verbs and URLs and what that does is makes it possible to automate these applications in any language because every language has some kind of HTTP library that you can use to talk to a server so every language has a client it for selenium-webdriver that lets you write in your native code the commands that you want to send to the device to automate webdriver is also currently the standard for browser automation so if you do if you use open source tools to automate web applications you’re using webdriver these days and more than that it’s actually a w3c specification at this point it’s a draft specification so it’s been proposed by a lot of companies including Mozilla and I think Microsoft is even on board there is a huge working group that is making this a standard for the web so that the browser’s themselves are going to support this HTTP protocol for automation and what we’re doing is we’re extending this protocol for mobile so that no matter what kind of device you’re testing you’re using the same kind of protocol across all platforms and I already made the point that given the client-server architecture you can have a webdriver or an app iam test written in pretty much every language there are a few that don’t have implementations written yet but every language that people use day to day already has an implementation of the webdriver protocol and again I already mentioned that webdriver is a w3c working draft so that’s the interface that’s how you define the types of things that you want to test but how does appium once it receives these commands how does it actually turn them into behaviors on your mobile device well that depends on the platform because at this point every platform has exposed a different method for automating devices for that platform so apple has given us something called instruments which is a little app that will take a bunch of JavaScript and will run it on a device and this JavaScript makes use of something called the UI automation API and that lets us perform different kinds of automation similarly Google has given us tools for automating Android devices their tool is called UI Automator it’s you basically you write java binary and you ship it over to your device and then you can run a test case with that with access to this UI Automator api and that’s for newer Android devices for older Android devices we use the instrumentation framework to get access to UI elements and interact with them in an application and Firefox OS has actually already implemented a web driver based automation protocol in the form of marionette so more than any of the other vendors that IBM supports Firefox OS was the easiest to implement support for in fact I did it in about an hour when I was meeting with some of the people who

made marionette and I just learned yesterday about Ubuntu touches automation support so you know whenever a PM has support for Ubuntu touch that will then go on the list here as the way that automation is made possible for that platform so this isn’t very important to to know if you just want to learn how to use a PM but if you’re interested in how a PM works this is kind of an architectural diagram of how we make use of what apple provides in terms of automation so on the Left we have our webdriver script this is what you are responsible for writing so this is some kind of script in that makes use of the webdriver protocol to send commands to the appium server and the appium server takes those commands turns them into the appropriate UI automation API commands ship some JavaScript over to the iOS device where it’s executed and we can get results and information back so everything inside of that box on the right hand side is what appium takes care of for you so that you don’t have to learn the specifics of Apple’s API you don’t have to set up any kind of infrastructure around managing these tests and so on you just maintain a suite of webdriver tests that hit appium server and that’s how you automate an iOS device and it’s basically the same with Android so it’s a little bit of a nicer picture because the tooling on Android makes it a little more simple to send automation commands back and forth to something running on the device but the point is that on the left hand side your script could be more or less exactly the same for both applicate for both platforms so that’s a p.m. in general one of the great things about appium i think is that the community has already produced some really nice tools to make using appium even easier so there’s actually a graphical user interface for appium called a PM app and there’s there’s also one for Windows if you guys like windows more at a PM app is a simple GUI for for launching a PM it lets you monitor the status of the server and set different preferences and server flags without having to know all of the flags you can pass through the command line so it looks kind of like this so you can pick the app that you want to automate add some different information about it and click Launch and that will start the appium server listening for incoming connections to run tests and we have a preferences window so you can tweak a Pimm’s behavior and all the different ways that are possible and the best part is that it comes with an inspector for probing your application so in and out it enables you to find ways to retrieve UI elements in your app so you can interact with them it enables you to try out different kinds of automation in a point-and-click fashion it lets you record those actions so that you can play them back later and it has a little code exporter so that you can copy and paste your your playback code into a test script in pretty much any language so it looks kind of like this the big window the big main window is basically a hierarchy viewer so you can navigate your applications UI hierarchy that way and you see a little red highlighting on on the screenshot there to tell you what you’re looking at and in the bottom right you get all the kind of metadata about the particular element that you’re introspecting it gives you things like the name or the label the element also a kind of XPath locator which you can use in your tests to find that particular element and interact with it in an automated way and then this is just what it looks like if you want to try out a few automation behaviors and then take the code that that generates and copy it into your own test suite somehow ok so how do we actually write appium tests what does that look like if you’re have your text editor and your you have your app and you want to start writing a test for your app how do you do that well first

we need to understand the model and as I said the appium test model is the same as the webdriver test model it’s the same protocol so if you’ve used webdriver you know that it’s centered around the idea of a session so you first start a session and that will instantiate your application and give you the ability to find elements in your application and interact with them and then introspect your application to make sure that your test conditions are satisfied and then when you’re done you stop the session so that the server knows that you’re done with your test and then you can have another session for another test or however you choose to architect it so this is just a bit of Python code this could be in any language but what we’re doing here basically is creating a string which is the location of the appium server in this case it’s just running on my local computer on its default port and we are creating a desired capabilities object now desire capabilities are a set of things that we want to tell the server so that we automate what we want to automate in this case we’re setting the device desired capability to iPhone simulator which means that we want to run this test on iPhone instead of say Android or Firefox OS and then the only other thing we need for iOS is the absolute path to the application on our computer so whatever we we create a debug build in Xcode and it creates a dot app directory we just pass that straight to a p.m. and a p.m. will take care of launching that in the simulator and running your test so the final line in our setup function is what gives us an instance of the webdriver object in Python and that’s what enables us to then start running automation commands an hour our teardown function which is run after each test just quits the session it cleans everything out so that our next test will run with the fresh session and a fresh instance of the application so the first thing you usually want to do when writing a test is find an element that you want to interact with because you probably want to click a button or enter some text into a field and then click a button and then assert that that text shows up in the right place those are the kinds of things that applications generally do and so that’s what we want to test for so there’s a few different ways to find elements with appium we can find elements by their accessibility label so for iOS and Android anyway the accessibility label is a string which you can attach to pretty much any UI element which is normally invisible but for users who are using your application in an accessibility mode these strings might be spoken aloud to them or something like that if maybe they are heart of sight so the UI automation frameworks give us access to these accessibility labels so if we have labeled our elements well we can find them in one go just by providing the accessibility label that we attached this also enables us to have apps which are easy to automate across multiple platforms because if we set the same accessibility identifiers on our iOS app as on our Android app then we can use the exact same code to find the appropriate element in both applications we can also find elements by their UI type right so we have things like buttons and text fields and list views and different kinds of layouts for Android and so on so we can find elements by that so we can get a whole list of all the buttons in the view and then cycle through them and click each one or whatever it is that we might want to do we can also use a more generalized hierarchical type of query to find elements and so we do this by actually using XPath syntax so in this example we could find the button that has the text of hi on it or we could find the button that has a substring of some other kind of text so this can be really useful if we’re trying to find a UI element that has some kind of user-generated data associated with it though we might not know what it is beforehand the name of the button could be you know somebody’s name in your contact application and we might want to find that and so this is a good way to build up that kind of thing

and finally on Android we can find things by their ID from the string XML files so then once we’ve got elements and they come back to us in our frameworks as an object we can we can call methods on these element objects so we can call things like dot text in Python and that will give me whatever text is on the button this is useful for making assertions so that I can get an element and assert something about one of his properties like it’s text or its value or its position something like that one of the things you always do with the button is click it so it’s really easy to just call click on an element so this is an example of part of the test that I showed you in the video this is the sign-in portion of the test so basically I’m finding the element which has the sign in accessibility label on it which is a button so we click it and then we’re finding the email filled by a hard-coded XPath selector and we’re sending a series of keystrokes to that field which contain my user name and then we’re finding the password field and we’re sending keystrokes to that field to containing my password and then the the new line there is just a kind of shortcut for hitting the the Go button on the keyboard so this triggers a login so you can see that it’s a really clean way of describing the kind of behavior that you want to do so you can think like a user and then write in code the behavior that a user would take in exercising your application and then this is just the same thing in Java just to kind of prove that you can do this and in any language and there’s the actual login logic for Java okay so that’s how you write appium tests now I just want to wrap up with a few more demos about a PM’s other capabilities so as I said we can run it on real devices and while I was sitting over here in between talks earlier I took a video of a test running on Nexus 7 that I had with me so this was the test this is in in node node J s so yet another language and we can see how for Android the desired capabilities look a little different because I have to know the package identifier for my application and the particular activity that I want to start and then I have some helper functions that let me find buttons and texts with certain properties so then this is the actual logic of the test that I’m going to show you basically we’re automating the contacts application or I think it’s called people maybe on Android so we’re doing a little boilerplate and then we’re clicking the create new contact button entering in some details about that new contact saving it adding it to favorites which is the little star button we’re editing the contact we just added or changing the the category of number of for the phone number going back to the home and ultimately we’re deleted in contact again to kind of clear out the state of the application so you can see it’s it’s fairly easy to see what’s going on just by looking at the code it might be a little small I just took this with my phone so we’ve clicked the create button and now we’re going to start entering in details so the name and the hard-coded phone number and whatnot email address and we delete things and clear it out and there we are so IBM works pretty well on real devices so a word about scaling up mobile automated testing infrastructure so what I’ve been showing you so far is what you can do for local test development or running a few tests at a time that kind of thing

but if you take this dev cycle of optimal Happiness to its limit you’re going to end up with hundreds or maybe even thousands of tests for your different applications and you’re going to run into a problem of how you scale your test infrastructure there are there are various limitations one of the biggest limitations is that you can only run one iOS simulator or one iOS device at a time per Mac so if you wanted to run a hundred different iOS tests at a time you’d have to figure out some way of having a hundred different instances of the Mac operating system which is obviously very expensive because of licensing and hardware and and all of their kind of restrictive business decisions that they’ve made with Android it’s a little nicer but you’re eventually going to run into hardware issues so anyway the point I’m trying to make is that if you get into the point where you’re really trying to scale your your test infrastructure you might need a little help and that’s where the company that I work for is is trying to be helpful to the mobile automation community so basically we have a bunch of simulators emulators and very soon we’ll have a bunch of real devices so that you can write your test code and have it run in our cloud rather than having it run locally so that all you have to do is worry about your test logic and maintaining your test suite you don’t have to worry about any of the infrastructure setting up at the end getting all the SDKs working making sure that versions for everything are correct handling upgrades all that kind of stuff so if you want to learn more about that go to sauce labs comm slash appium just think we might have changed this URL to sauce labs comm mobile so it probably redirects but in case it doesn’t that’s where you go and it’s very very easy to run tests on sauce that you already have running locally basically the only difference is that you change the location of the appium server from a URL that probably designates a local host based server to a URL that designates a location in our cloud and you have to give us your username and password so we know whose account is trying to run tests but other than that your test stays exactly the same so it’s very easy to start developing appium locally and then the point at which you need to get a little extra scalability or bandwidth you can then run things on sauce and finally I just wanted to give another little demo of how you can test regular web applications using the mobile web browsers like Safari and Chrome so this is a test that basically runs a very short test on Safari and then on Chrome and again it’s the same test code and it’s just regular selenium webdriver code there’s nothing different about using appium so we’re basically just typing something into a box saving it an asserting that’s there so it works on Safari on iOS and chrome on on Android and those are the different webdriver commands that were executed during that test and then finally I mentioned something about robots earlier about physically automating mobile devices and that’s something that different people in the appium community have been working on and it’s not really useable yet but there’s been some pretty cool demos that people have done including this one which is this thing called a tap store bot which is a delta robot that has a spot for an iPhone and it’s automating the keyboard and sending a tweet so that was done using a p.m. so appium knows where the keys are and it tells the robot software XY coordinates of different locations on the screen and the robot software knows how to turn that into the correct motions of the stylus so that it can actually tap those locations on the screen and as you can see it was fairly accurate it was able to pick out individual keys on the keyboard which are fairly small so if you want to go really crazy with automation or you feel like it’s very important to test the actual physical gestures for your applications that kind of thing is possible – it’s gonna take some more work to be generally usable but it’s coming so lastly I wanted to

just say that the app game community is always looking for help in different ways we have lots of lots of things that we’re working on in a lot of different languages appium itself is written in node J s so if you like that or if you’ve wanted to learn more about that it’s a pretty big and fun project with a lot of interesting bits we have the the gooeys that I showed you before one for Mac and one for Windows and those are written in objective-c and respectively and there’s also bits of Java thrown in there for Android support and you know if you thought the robot was cool the people who run that are always looking for for people to help make that software better and then I’ve been really excited about Smart Def Con and learning about some new operating systems that I didn’t even know about and learning more about ones that I had heard of and I would love to see all of these supported with appium making use of whatever native automation protocols are supported by these different projects and bringing them together into this one mobile automation protocol making it really easy for somebody who maybe has an app on all these different platforms to write a very succinct series of tests and to run them on all the different platforms so I think that would be pretty amazing goal and I’m gonna be checking into some of these different operating systems and seeing how feasible it is to to get support for them in appium in the coming weeks and months and then for those of you who are sticking around for the hackathon tomorrow I’m not gonna be making any mobile apps but I’ll be around working on a p.m. and if anybody wants to take a nap that they’re developing and and see how it works to try and write an automated test for it I can help you get set up so come and see me and burning tomorrow at the hackathon okay so that’s my talk and if you have questions I have a few prizes a couple different kinds of t-shirts and hot sauce things like that so I’m gonna have to come give you the microphone I have a few questions yes maybe first will be the easy one can I run test a peon tests on an emulator that is started in a headless mode which kind of familiar Android emulator for example I didn’t know that you could start Android emulators in a headless well if your continuous integration server is is a machine that doesn’t have any any other interface that then command-line then Android then you know it’s the only option for starting an emulator sorry I thought it would be and easy question well I don’t know I mean I guess it depends on whether the emulator still has the UI layer because this can pass okay I would imagine that it could then it depends because the technology that we use is called UI Automator and that google has designed to actually fit inside of the UI layer itself and so for whatever reason in headless mode this layer isn’t there then it might not work but I don’t see why it wouldn’t right now okay and the second question do you plan to add support or maybe you already have support for some gesture based in the interactions like if you have a gesture like drag and drop mechanism in your application can you test this kind of indirect yes yeah we support most kinds of gestures right now and will support the rest soon we’re limited by what we’re given by say Apple and Google but what we’re given is pretty good we can do several kinds of multi-touch gestures like the drag or no like the pinch and zoom ones we do have tests for drag and drop for Android obviously things like swipe and flick work pretty well too okay thank you yeah what’s your t-shirt size let’s see okay well I’ll give out t-shirts until I run out and then I’ll give people hot sauce nice nice throw all right we’ll see if how many people I hit with t-shirts

today more questions yeah oh oh hi I have a question about using multiple multiple devices multiple drivers in a single test I’m specifically I’m talking about a situation where you have the well some interaction between two devices let’s say two Android emulators and can you open multiple drivers for single okay okay yeah we have so the coolest thing I’ve seen is there’s there’s a company called taxi magic in the states and they’re using appium and they have it they have one test with three drivers one for their website using selenium one for Android device and one for iOS device because they’re testing somebody makes a booking using the website and then the cab driver gets a notification and then they pick them up and then the person pays using their own phone when they leave so they have this whole test flow it takes a little bit of setup because you have to create multiple instances of the server on different ports but if you know about ports it’s easy hopefully you do okay and just one question I think you answered it already but I’m just want to clarify when we used Robottom we had we had a problem with dynamic generated views when you it was an Android it was Android development and when we you’ve got the view which were dynamically generated and they didn’t have any Android IDs well you were pretty unable to do anything with it but I guess that with these with these options to find all these views controls like the Xbox Xbox option well you are able to also cope with this kind of content yes yeah the only the only limitation is that if you make a custom control that doesn’t subclass a standard control then it’s invisible to the UI viewer so it’s hard to automate that you have to resort to like you know you know X&Y locations for taps oh but okay but yeah if it’s a typical UI element like like an element in a list or something like that even if it’s dynamically generated you’ll still be able to find it okay thank you mmm what’s your t-shirt size alright Wow okay yes hello about testing on a real devices yeah that’s Depp I care about you know installing the newest version of the app or you need to do it manually or you need to write in the test some lines to first install then uninstall how it works so for real devices well the nice thing about Android is that real Android devices are not different from emulators so it works the same on both and for Android we can install the application for you we also clean the application state before running each test so we remove all the user data associated with it so that every time you launch your application it’s in a clean State you can also choose to uninstall and reinstall the app for every test to you know make sure that it’s the absolute newest version for iOS real devices it works best if you have the app already on the device because of the way that it works to deploy applications and because of the whole provisioning process for iOS apps it’s the easiest if you just put the app on the device and then you use the bundle ID for the app to to launch the test is there a way to do it automatically there is yeah there’s a tool called fruit strap which tries to install automatically what we found is that that works for some people and for some people it just doesn’t really work that well and that’s that’s just a limitation in apples I don’t know developer workflow decisions I don’t know I don’t know why they chose for it to be so difficult but as soon as we can find a nicer way to make it work we definitely want people to be able to take like an IPA file that’s correctly signed and have it automatically pushed to the application but there’s also no way to like delete applications from a real device easily it’s a bit more complex with iOS so anyway do you want t-shirt what’s what’s your size big all right this is this is the the big one okay next question is the app you can test

against for example games technically yes but in practice it’s not as useful because typically games don’t use the standard UI controls and so it’s very hard to get it’s very hard to find an element in a game because unless it’s a standard UI element that is provided in the SDK you can’t get a handle on it so the best you could do would be click these coordinates and then take a screenshot and an assert that it matches what you expect but for something like a fast-paced 3d moving game I don’t know I don’t think it would be that useful yet but one of the things that I’m trying to add soon is find an area by image so you you pass in like a PNG file or something and it finds the location on the screen that most closely resembles that using some kind of image recognition so if you wanted to find like the star in your game you know you say find by this thing and then it finds it and it gives you the location so you can assert that the star is in the right place for example so that’s the kind of thing that would be more helpful for games but that doesn’t exist quite yet thank you yep so I only have I have small and medium if you want a t-shirt medium okay yeah all right look out guys nice oh hi yeah I have a question I know everyone can imagine automated tests which you can pass or fail but I’m actually more interested in ultimately it’s performance testing do you have any explicit support for this use case and second question doing heaven in data how common disease it’s certainly not that common most people are using a p.m. for like correctness tests not performance tests there is nothing that would limit or prohibit you from doing performance tests because appium is just an automation tool it’s not really a testing tool like we don’t a p.m doesn’t have the concept of assertions or anything you have to put all those in your language that you test with so like what what kind of thing do you have in mind I’m just asking if you have an explicit support so the answer is no you just have you just have the generic generic yeah so it’s it’s a UI automation framework so it takes the perspective that you’re a human being using the application so we we don’t currently have a way to like give back like a performance snapshot of a test run on iOS we do because instruments leaves a trace directory with the log of you know a memory and performance during the test so you could run a test take that and then like evaluate it on your own but we don’t have explicit support to give you like performance statistics or things like that after your test run okay thank you it’s nice suggestions for the guys you are you that you want new hackers on the yeah exactly so I only have small shirts but if you want to come get a hot sauce feel free to come up at the end hey Apple let me introduce bots so they kind of doing some steps into this continuous integration yeah but they are few years behind yeah as always very young but they’re supported right now only a unit testing but I’m seeing that in the future they could support automate risk as well for the you so yeah are you not afraid to lose this this platform well I don’t think we’ll lose the platform I think if they are intentionally providing better functional testing tools I view that as a good thing because it shows they care yeah but but they’re not too open to cooperate yeah they’re not yeah will will always provide whatever support we can with appium I mean the point behind appium is that you shouldn’t be locked into one language or framework and the way that Apple’s gonna do it it’s gonna lock you into using instruments into using UI automation JavaScript and all that so we’re hoping that people who care about cross-platform will choose to use something like appium people who are purely iOS developers and they just live in that world

maybe it maybe doesn’t make sense for them maybe they should just use Apple’s tool I don’t know okay next question basically my colleagues question but most of the of the arrows good could show when you upload a new version of the app so do you have a mechanism that will install previous version of the app do some stuff on this and upgrade the version living that all the user defaults settings and checking if the app doesn’t behave strangely I see not in an automated way right now you would have to install the old one run a test upgrade and then rerun the test okay by hand yes so that’s that’s my suggestion yeah okay thank you interesting case I hadn’t thought about that yeah one question about like the hardware input so what happens or how do you see the use of a PM in case we you want where you need a hard rain boots a accelerometers so portrait landscape switches or if you are doing something with the camera god forbid network that sort of things yeah a lot of those are supported automation behaviors already so like changing the orientation not for real devices of course but for like the simulator you can simulate different kinds of like geolocation you can change orientation you can change I don’t think they’ve given us any way to emulate the accelerometer but hopefully we’ll have that in a future release of the API for physical devices we can’t simulate it that’s where things like the robot could come in handy because you could make the robot like shake the device or turn it or something like that and then you can use appium to assert that the you know the UI has a different orientation or they’re the screen looks different according to what you expect and for example camera so if for example the user input would depend say on what is displayed so there is it yeah well for a real device so you can access the camera by using the UI so if your app has a button that’s like take a picture it clicks it you know it will show you whatever the camera sees whatever the camera sees in reality you can then click the take picture button and do something with that or if you’re using a simulator you can pick a photo from the camera library that was pre taken and then you can exceed them you can know what the app is supposed to do with that ahead of time I only have small shirts left so do you want one all right all right you can couldn’t get them all yeah we’re not gonna go and let you go home just a quick one um also like a bit regarding like the performance testing so the video you showed of the test what actually defines like how fast that test is run because I mean like finding two elements and like filling them in and like submitting and you should be much faster or could be much faster than what we’ve seen so how is like the speed actually defined or like it’s it’s completely determined by the underlying automation so it happens a lot faster on iOS for some reason Android the underlying library takes longer to simulate those events I didn’t add any kind of delay that’s just how life takes make it harder like to test for like whatever load times or let’s say you submit that form and you want to know okay how long does it actually take like until well you can do that I mean you can start a timer when you hit the button and then loop until you until some element is visible but then you would be very different on those well that’s probably determined well that’s determined by the device that’s not determined by the automation I thought you were saying hey it took well long is like two different things yeah one is like the device and the other one it’s like that testing yeah but when you’re not like filling in a field once you’ve hit the button from then until the time when something else shows up the UI automation isn’t altering or affecting the speed at which something loads but you could set your emulator to have a certain network speed and test different network speeds that way and you know did assert that hey even at this network speed I want this next view to show up within five seconds or I consider this a failure you can certainly write that kind of logic okay cool thanks well there is no any other question I could run which is connected to to the last ones I mean how experimental are those robots is becoming a real thing

yeah they’re selling them okay some people are using them for real I I consider it experimental but some people have strategies it for real yeah right because well as we all see it would it would solve the problems with real Hardware right you could if the hardware was there and there was a way for the user to to access it and there would be a way for a robot to access it my other question will be would youwould you see continuous integration with the robot totally stupid thing no no no I would I would see something like you know run your tests on simulators and then like every week run it on real devices and then every month run it on robots or something like that just to like have multiple levels of QA