Tips & Tricks for getting data of any shape and size into Azure Data Lake using – THR3018

>>> GOOD AFTERNOON, EVERYONE. HOW ARE YOU GUYS DOING? HOW MANY YOU CAME TO THE SESSION SPECIFICALLY VERSES HOW MANY JUST HANG OUT HERE TO TAKE A NAP RIGHT AFTER LUNCH? ALL RIGHT. I HOPE MOST OF YOU ARE HER FOR THE SESSION. WE’RE GOING — YOU ARE HERE FOR THE SESSION WE ARE GOING TO TALK ABOUT USING DATA FLOWS AND THE COMMON DATA MODEL MY NAME IS MIGUEL LLOPIS. AND. >> MY NAME IS OLEG WORKING ON CDL >> WE ARE PROGRAM MANAGERS ON THE TEAM. WHICH I ASSUME MOST OF YOU ARE DEVELOPERS ACTUALLY. OKAY. SO ONE OF OUR GOALS IS TO PUT YOU OUT OF A JOB. NO OFFENSE. BUT WE’RE ACTUALLY HERE TO HELP USERS TO DON’T KNOW HOW TO WRITE CODE DO MORE STUFF AND REDUCE THE AMOUNT OF LOAD OR THE — STOP MAKING YOU A BOTTLENECK FOR AS MANY THINGS AS POSSIBLE LET’S TALK ABOUT TRADITIONAL CHALLENGES AROUND DATA PREPARATION AND WE WILL TALK ABOUT HOW DATA FLOWS AND CDM CAN HELP AND WE WILL DO A RECAP AND WE HAVE A SESSION TOMORROW MORNING, A FULL HOUR TO WALK THROUGH THE CONCEPTS IN MORE DETAIL. CHALLENGES SO, A RECAP, TODAY THINKING ABOUT A DEVELOPER LIKE A BUSINESS USER OR DATA ANALYST, IT’S EASY TO FIND AND CONNECT DATA. THE EXPERIENCES THAT THEY HAVE TO CONNECT TO DATA, THEY ARE COMMON ACROSS ANY DIFFERENT TOOL. LIKE EXCEL OR POWER BI OR USE A NON-MICROSOFT, ALL THE SAME EXPERIENCE, ALL WORK FINE. REGARDLESS OF THE SOURCE THEY’RE TRYING TO CONNECT TO, DATA IS IN THE SHED THEY NEED FOR THE SPECIFIC ANALYTIC RESHAPING DATA IN THE CASE THAT IT HAS TO BE DONE WHICH IS RARE, IT’S FAST, QUICK, EASY, NO PERFORMANCE ISSUE, NO ISSUES WITH DATA VOLUMES AND ALSO STRAIGHTFORWARD, YOU DATA CHANGES AT THE SAME PACE REGARDLESS OF THE DATA SOURCE. SO IT’S VERY EASY TO RECONCILE DATA. SO THIS WAS MY TEST. I SAW SOME OF YOU LAUGHING WHICH IS MY TEST TO MAKE SURE YOU GUYS ARE NOT FALLING ASLEEP. IN REALITY, HARD TO FIND AND CONNECT TO DATA FOR A NON-DEVELOPER. THE EXPERIENCES ARE FRAGMENTED, HAVE DIFFERENT EXPERIENCES. DIFFERENT DATA SOURCES REQUIRE DIFFERENT CODING LANGUAGE AND UNDERSTANDING OF DIFFERENT CONCEPTS TO EXTRACT DATA FROM THEM MORE SO, DATA IS NEVER IN THE SHAPE THAT YOU NEED OR THAT THE BUSINESS NEEDS FOR THE ANALYTICS OR THE REPORTING SCENARIOS. AND MORE OFTEN THAN NOT, THEY HAVE TO COMBINE DATA ACROSS DIFFERENT DATA SOURCES WHICH CHANGES AT DIFFERENT PACES. SOURCES THAT CHANGE ONCE A YEAR. SOURCES CHANGE MULTIPLE TIMES PER MINUTE. HOW WE ALLOW THEM TO BRING IT TOGETHER, RECONCILE, CLEAN THE DATA, JOIN IT AND SO THEY CAN REUSE IT MORE EASILY. THAT’S WHAT IT’S ALL ABOUT AND MORE SO, YOU KNOW, THE VOLUME AND THE NUMBER OF DIFFERENT DATA SOURCES AND THE DATA SILOS AND ORGANIZATIONS CONTINUE TO GROW. IT’S VERY COMMON THAT, YOU KNOW, LARGE ORGANIZATIONS WILL HAVE DIFFERENT DATA SOURCES FOR DIFFERENT SCENARIO, DIFFERENT TOOL, THINGS LIKE YOUR SOCIAL, YOUR MARKETPLACE, YOUR ADVERTISING PLATFORM, HAVE DIFFERENT SILOS OF INFORMATION SAME GOES FOR YOUR TRANSACTIONAL SYSTEM OR YOUR IoT DEVICES WITH DATA ON TO SOME UNDERLYING DATA SOURCE. FIRST STEP TO RECONCILE THAT IS HAVING A SINGLE PLACE. AND THIS IS WHERE YOU PROBABLY HEARD WORDS LIKE DATA LAKES, RIGHT? SO WHERE THEY COME INTO PLAY. A SINGLE PLACE WHERE THE BUSINESS USERS CAN FIND THE INFORMATION THAT’S BEEN FROM DIFFERENT TOOLS BUT NOT JUST THE RAW DATA, IT’S BEEN NORMALIZED AND CLEANSED AND PUT TOGETHER IN A WAY THEY CAN MAKE SENSE OUT OF IT. JUST TO EXPAND A LITTLE BIT MORE ON THE CHALLENGES AROUND DATA PREP. A QUOTE FROM GARDNER. AND PARTICULARLY AROUND ANALYTICS. BUT MAKES SENSE IN OTHER DOMAINS. 80 OF THE TIME IT’S ACTUALLY SPENT ON DATA PREP AND GETTING THE DATA TO THE PLACE THEY NEED SO THEY CAN START DOING THEIR WORK WHICH IS DECIDE ON SOMETHING OR ANALYZE THAT DATA. NOW HOW CAN DATAFLOWS HELP? HOW MANY ARE FAMILIAR WITH POWER BI? ALL RIGHT. HOW MANY HEARD ABOUT POWER BI? OKAY, THREE ABOUT IT YOU DON’T GET THE BONUS POINTS FOR THAT. BUT DATAFLOW, A LOCAL EXPERIENCE THAT WE HAVE INTRODUCED INTO POWER BI THAT ALLOWS USERS TO USE THE POWER CHORD EXPERIENCE, GET DATA SPENCER THAT WE EXPOSE IN EXCEL AND POWER BI AND OTHER MICROSOFT PRODUCTS SO ACTUALLY CONNECT TO SOME DATA SOURCES A PLY DATA TRANSFORMATIONS, MAP ALONG THE WAY TO COMMON DATA

MODEL SCHEMAS IF WE REACH AND REACH THE DATA AND LAND IT IN THE AZURE DATA LAKE STORAGE. AND WE SUPPORT ABOUT 45 CONNECTORS TODAY INCLUDING CLOUD AND ON-PREM SOURCES AND WE WILL DO A COUPLE DEMOS OF THAT LET’S TAKE A LOOK AT IT. ALL RIGHT SO, WHAT WE’RE LOOKING AT IS POWER BI. LOOKING AT THE WORK SPACE VIEW AND WITHIN HERE, WE ADDED THE NEW TAB FOR DATAFLOWS. HERE’S WHAT YOU WOULD SEE THE REPORTS AND DATA SETS FOR THE CURRENT WORK SPACE. NOW IF WE GO CREATE, WE CAN CREATE A NEW DATAFLOW. AND THIS IS GOING TO GIVE US A FEW OPTIONS. THE FIRST ONE IS TO DEFINE NEW ENTITY, BRAND NEW ENT ENTITIES FROM AN EXTERNAL DATA SOURCE, SOME DATA FEED OR SOMETHING WHERE WE WANT TO PULL DATA FROM THE OTHER TWO ARE ABOUT REUSING DATA THAT ALREADY EXIST IN THE DATA LAKE. THE FIRST ALLOWS YOU TO LINK TO AN ENTITY THAT SOMEBODY ELSE DEFINED IN A DIFFERENT DATAFLOW AND THE THIRD ONE IS ABOUT ATTACHING A CDM FOLDER WHICH MAY HAVE BEEN CREATED OUTSIDE OF POWER BI. FOR THESE, WE’RE GOING TO ADD BRAND NEW ENTITIES. AND HERE’S WHERE WE GO INTO THE POWER EXPERIENCES. AND THIS IS THE GET DATA. SO HERE WE HAVE ACT STOWS THE 40-PLUS DATA SOURCES WE WERE TALKING ABOUT. THINGS LIKE DIFFERENT TYPES OF FILES, DATABASE SOURCE, A BUNCH OF RELATIONAL DATA, INTI INTEREST ITYS WE CREATED, A BUNCH OF AZURE SERVICES OUT OF THE BOX AND OTHER ONLINE SOURCES LIKE THE COMMON DATA SET FOR SHAREPOINT AND A BUNCH OF OTHERS. AND THE OTHER TAB HAS A FEW MORE THINGS LIKE ACTIVE DIRECTORY, WEB APIs, WEB PAGE, SO YOU CAN DO SCREEN SCRAPE OF HTML TABLES AND OTHER THINGS. SO FOR EXAMPLE, ALL DATA. WE’RE GOING CONNECT TO AN ALL-DATA FEED. MY FAVORITE FEED IN THE WORLD. SO LET’S GO AND ACTUALLY COPY THAT URL. WE WILL PASTE IT IN HERE. AND THIS IS CREATING WHAT I WOULD PICK AN ON-PREM GATEWAY IN THIS CASE, A CLOUD OR INTERNET-FACING SO I DON’T NEED TO PICK ONE. AND CONNECT ANONYMOUS ACCESS. HOW MANY ARE FAMILIAR WITH POWER QUERY? OKAY, A FEW OF. SO THIS PROBABLY FEELS RIGHT AT HOME FOR YOU. THIS IS A POWER QUERY ONLINE EXPERIENCE. THIS IS THE EQUIVALENT TO THE NAVIGATOR DIALOGUE IN THE DESK TOP. AND YOU CAN COME IN AND START CLICKING ON ENTITIES AND YOU CAN ACTUALLY GET PREVIEWS OF DATA. AND SELECT ONE OR MULTIPLE ENTITIES SO. WE’RE GOING TO SELECT THREE CUSTOMERS, ORDERS AND ORDER DETAILS. AND THEN WE’RE GOING TO CLICK NEXT AND THAT’S GOING TO PUT US IN THE EDITOR. THE EDITOR IS THE MAIN SERVICE WHERE YOU WILL APPLY DATA TRANSFORMATIONS. SO WE HAVE THE THREE QUERIES THAT WE SELECTED IN THE CONNECTIONS SEEMS TO BE SLUGGISH HERE. SO BEAR WITH ME ON THAT. ALL RIGHT. IT’S BACK. SO NOW WE HAVE THREE TABLES. WE HAVE ALL THE DETAIL, ORDERS AND CUSTOMERS. AND WHAT WE WOULD LIKE TO DO IS CREATE A SINGLE ENTITY THAT THE TOP 20 CUSTOMERS RIGHT. BY SALES. IN ORDER TO DO THAT, WE NEED TO MASSAGE THE DATA SO THE FIRST THING, THE DETAILS HAS LINES OF ORDER. AND YOU SEE FOR EACH OF THEM, LIKE FOR THIS ORDER, 10, 248, WE HAVE THE PRODUCT ID AND THEN THE UNIT PRICE AND QUANTITY AND ACTUALLY ZOOM IN CAUSE YOU WILL SEE IT BETTER. SO WE HAVE THE COLUMNS THE FIRST THING WE NEED TO DO IS ACTUALLY CALCULATE THE LINE TOTALS SO. WE’RE GOING TO SELECT UNIT PRICE AND QUANTITY. AND WE CAN RIGHT CLICK AND SAY, ACTUALLY NOT RIGHT CLICK, ADD COLUMN AND INSERT THE NEW COLUMN THAT DOES THE MULTIPLICATION OF THE TWO. NOW WE HAVE THE LINE TOTAL HERE. >> AND USING M EXPRESSIONS TO DRIVE THIS. THE M — >> YEAH, ACTUALLY THE USING THE UI TO DEFINE THE MULTIPLICATION HERE. HOW IT GENERATES A EXPRESSION. AND M IS UNDERLYING FORMULA. NOW THAT WE HAVE THE LINE TOTAL, WE CAN GO AHEAD AND DO THE TOTAL FOR EACH ORDER SO WE CAN DO A GROUP BY ORDER ID WE’RE GOING TO CALCULATE THE TOTAL BY DOING A SUM OF THE COLUMN THAT WE JUST CREATED. LINE TOTAL. AND THAT’S GOING TO GIVE US JUST THE OUTPUT. SO NOW WE HAVE FOR EACH ORDER, THE ACTUAL TOTAL. NOW WE CAN GO TO THE ORDER STABLE. AND IN THE ORDER STABLE IS THE ONE THAT BINDS ALL THE IDEAS WITH CUSTOMER IDs. WHEN EVER IT LOADS. SO HERE WE ALSO NEED TO DO ANOTHER JOIN SO THAT WE’RE GOING TO KEEP THESE TWO COLUMNS, REMOVE THE OTHER COLUMNS

AND NOW WE’RE GOING TO DO A JOIN FOR THAT, COMBINE TABLES. WE SUPPORT JOIN AND UNION IN POWER QUERY. IN THIS CASE, A MERGE. AND HERE I CAN SELECT HOW DO I WANT TO JOIN. SO I WANT TO USE FROM THE CURRENT TABLE THE ID COLUMN. AND FROM THE OTHER I’M GOING TO USE THE OTHER ID. WE SUPPORT ALL TYPES OF JOINS. I WILL STICK TO A LEFT JOIN. AND WHEN I CLICK OKAY, WE LAND THE EXTRA COLUMN THAT THAT HAS DETAILS FROM ORDER DETAILS TABLE. AND EXPAND THE SPECIFIC COLUMNS THAT I’M INTERESTED IN AND IN THIS CASE, I ONLY CARED ABOUT THE TOTAL ONE. SO NOW WE HAVE THE TOTAL FOR EACH ORDER. AND NOW WE’RE GOING TO GO AHEAD AND DO GROUP BY CUSTOMER ID. SO THAT WE CAN GET THE ACTUAL TOTAL SALES FOR THAT CUSTOMER. AND THAT’S AGAIN GOING TO BE A SUM OF THE TOTAL COLUMN AND NOW WE END UP WITH THE TWO COLUMN CUSTOMER ID AND TOTAL SALES. AND WE SWITCH TO THE CUSTOMERS TABLE WHICH WAS THE ORIGINAL TABLE FROM HERE, WE CAN PICK A COUPLE COLUMNS THAT WE’RE INTERESTED IN SUCH AS THE CUSTOMER ID AND COMPANY NAME AND THEN WE CAN REMOVE THE OTHERS AND NOW WE’RE GOING TO DO ANOTHER MERGE. SO WE BRING THE TOTAL SALES FOR EACH CUSTOMERS FROM THE ORDERS TABLE. AND WE WILL STICK TO THE LEFT JOIN AGAIN. SO KIND OF A RINSE AND REPEAT FROM THE OTHER JOIN JUST EXPAND TOTAL SALES. AND NOW THAT WE HAVE TOTAL SALES, WE CAN ACTUALLY DO A SORT DESCENDING. SO WE GET THE TOP-PAYING CUSTOMERS ON THE TOP. AND WE’RE GOING THE TOP 20. SO WE GO WITH THE ROWS AND WE WILL DO THE TOP 20 ROWS AND THAT’S HOW WE’RE TOP 20 CUSTOMERS SO. SAY TOP 20 PAYING CUSTOMERS AND. >> THIS IS THE DATA PREPARATION STAGE YOU WERE TALKING ABOUT. >> THAT’S CORRECT. NOW WE CAN DISABLE LOAD FOR THE OTHER TWO ENTITIES BECAUSE WE DON’T REALLY NEED THEM WE JUST WANTED THE FINAL ONE. AND NOW WE CAN SAVE AND CLOSE AND THIS IS SAVING THE DATAFLOW TO FINISH THEM. AND IT’S GOING TO RETURN THAT WITH SAVED JUST THAT ONE, THAT ONE ENTITIES. SO NOW WE CAN SAVE, ACTUALLY LET’S CALL THESE DEMO AT THEATER WE WILL SAVE THE DATAFLOW TO FINISH THEM. AND NOW KICK-OFF REFRESH NOW AND SET UP A SCHEDULED REFRESH FOR THE SAKE OF THE DEMO. JUST DO A REFRESH NOW OPERATION. BUT NOW WE CAN CLOSE THAT DATAFLOW. AND NOW WHAT WE HAVE DONE IS CREATE THESE ENTITIES THAT IS LOADING DATA TO AZURE DATA LAKE AND HAVE OUR USERS REUSE THE ENTITY FROM DIFFERENT POWER BI REPORTS. I HAVE DOWN HERE SO WE SEE POWER BI CONNECT TO DIFFERENT DATA SOURCES DIRECTLY. THE SAME EXPERIENCE. NOW WHAT WE’RE GOING TO DO IN THIS CASE IS CONNECT TO THAT SOMEONE ENTITY CREATED IN DATAFLOWS RATHER THAN HAVING THEM AND REDO THE TRANSFORMATION LOGIC, NOW A DIFFERENT USE AND MORE BASIC USE THAT WANTS TO CONNECT TO THE TOP-PAYING CUSTOMERS. FOR THAT, WE CAN GO TO THE POWER BI TAB HERE AND WE HAVE POWER BI DATAFLOWS. NOW GOING TO TRY TO SIGN WITH THE POWER BI ACCOUNT WHICH IS PROBABLY ALREADY LOCKED IN. SO IT’S GOING TO TAKE ME STRAIGHT TO THE NAVIGATOR DIALOGUE. VERY CONSISTENT EXPERIENCE WITH WHAT YOU SAW ON THE ONLINE VERSION. AND NOW HERE — OH, NOT COMING IN YET ALL RIGHT. SO I’LL USE A DIFFERENT ONE. THE ONE THAT I WAS REFRESHING DIDN’T COMPLETE REFRESHING YET SO LET’S SEE. ONE OF THESE — SOME DATA SO, FOR EXAMPLE, THIS ONE HAS PRODUCT INFORMATION. GOING TO BE BEAT THE SYSTEM WITH THE REFRESH BUT HERE’S WHAT I COULD HAVE SEEN ACTUALLY WHAT IS MY — DATAFLOWS PREMIUM. LET’S SEE IF IT MADE IT IN. OH, YEAH, MADE IT IN JUST NOW BUILD DEMO AT THEATER. THIS IS THE DATAFLOW THAT WE JUST CREATED. AND THERE SHOULD BE ONE ENTITY WITHIN THAT’S THE TOP 20 PAYING CUSTOMERS AND NOW I CAN SELECT THIS FROM HERE AND JUST LOAD AND START BUILDING MY REPORTS AND REUSE THAT DATA IT’S BEEN CREATED AND CURATED BY SOMEBODY ELSE, LOADED INTO THE LAKE AND NOW USERS CAN CONSUME THAT FROM POWER BI. I DO HAVE ONE MORE DEMO TO SHOW YOU THAT CONNECTS WITH ALL XPs ON CDM. YOU SAW HOW THE EXPERIENCE WAS ACTUALLY, YOU KNOW, BOOT STRAPPING NEW DATAFLOW ENTITY, CONNECTING TO SOMETHING I KNEW WAS AN DATA FIT. AND PROVIDING THE TRANSFORMATION STEPS. NOW AS PART OF CREATING A DATAFLOW, WE’RE GOING TO RELEASE WHAT THE NOTION OF A DATAFLOW TEMPLATE

DATAFLOW TEMPLATES ALLOW YOU TO PULL DATA FROM WELL-KNOWN DATA SOURCES SALES FORCE, DYNAMICS 365 SALES AND HAVE PREDEFINED STEPS TO MAP IT TO CDM SCHEMA. THIS IS SOMETHING WILL BE RELEASED IN THE NEXT WEEK OR TWO. SO KEEP AN EYE OUT FOR IT IF I CREATE A NEW DATAFLOW AND SWITCH TO A DIFFERENT ENVIRONMENT HERE, I HAVE THIS FEATURED ENABLED, WITHIN THE DATA, THERE’S GOING TO BE A NEW SECTION FOR TEMPLATES. FROM IT — ALL RIGHT. >> IT’S GETTING THERE. >> GREAT, NOW WE HAVE A NEW SECTION OF THE BOTTOM. THIS WILL BE RELEASED IN THE NEXT COUPLE WEEKS LET’S TAKE ONE AS AN EXAMPLE. JUST PROVIDE MY IDENTITY. SO I CAN SIGN IN WITH HIS CREDENTIALS. AND FROM HERE HOPEFULLY I REMEMBER MY PASSWORD ALL RIGHT. TOO MANY ATTEMPTS. SORRY YOU HAPPEN TO HAVE AN ACCOUNT? >> NO, I’M STAYING WITH ONE SYSTEM >> SORRY ABOUT THAT. >> WE CAN GO TO THE — GO WITH — YES. >> I DON’T HAVE IT AT ALL. I’M SORRY, I DON’T HAVE A BACKUP SCREEN SHOT HERE SOMETHING FUNNY HAPPENED WITH MY ACCOUNT. IT WAS WORKING YESTERDAY TYPICAL EXCUSE, RIGHT? SO WHAT I WANTED TO SHOW YOU IS ACTUALLY THE FACT THAT YOU CAN USE THE TEMPLATE AND PREPOPULATE THE LIST OF QUERIES, LIKE EIGHT DIFFERENT QUERIES THAT HAVE TRANSFORMATION STEPS, RECONCILING COLUMN, COMBINING THEM TOGETHER AND MAPPING TO CDM. AND THEN, YOU KNOW, WITH THAT OUT OF THE BOX TEMPLATE, YOU CAN BRING DATA INTO YOUR DATA LAKE IN CDM COMPLIANCE SHAPE. A RECAP, LET’S TALK ABOUT DATAFLOWS SO DATAFLOW, WHAT ARE THEY? A NEW CAPABILITY TO GET DATA INTO AZURE DATA LAKE STORAGE, LOCAL, NO CODE COMING TO THE PLATFORM SOON. SO USING FLOW, BUT NOT POWER BI, SOON YOU WILL GOING TO BE ABLE TO DEFINE THE DATAFLOWS FOR USE WITHIN YOUR POWER APPS AND FLOWS. THEY USE THE FAMILIAR EXPERIENCE FOR DATA CONNECTIVITY AND PREPARATION. THEY LEVERAGE FOLDERS SO THAT’S THE OUTPUT, A CDM FOLD OREN THE LAKE. AND YOU CAN MAP TO CDM DATA ALONG THE WAY. AND THEY ENABLE ADVANCED CAPABILITIES SUCH AS THE ORCHESTRATION WHICH WE DIDN’T TALK ABOUT TODAY BUT WILL COVER IN-DEPTH TOMORROW. NOT A REPLACEMENT THEY SIT A LEVEL HIGHER SO YOU CAN REUSE. DON’T REPLACE THE CLASSIC DATA WAREHOUSES EITHER. THIS SAY SELF-SERVICE SOLUTION ON TOP OF DATA LAKE. THERE’S A SPACE FOR THOSE THIS IS LOCOED, NO CODE, VERSES DEVELOPER CAPABILITIES. AND THEY WORK ACROSS POWER BI AND PREMIUM AS WELL. AND OTHER CAPABILITIES ON PREMIUM LIKE THE WHOLE LINKED AND CDM FOLDER CAPABILITIES THAT I MENTIONED. THEY’RE NOT ADDITIONAL COST. THEY COME AS PART OF THE LICENSE AND ONE PART THAT DRIVES ME NUTS, WITHOUT A SPACE OR CAPITAL LETTERS >> I GUESS THE DEMO IS MORE IT INTO EXCITING THAN SLIDES SO WE GAVE THE PREMIUM SPACE FOR MIGUEL. DID YOU LIKE WHAT YOU’VE SEEN SO FAR? >> THANK YOU. >> GREAT. NOW YOU HEARD CDM MODEL A FEW TIMES. HOW MANY BROWSE TO THE CDM PAGE AT MICROSOFT SITE? THAT’S IT. OKAY. GO. DATA MODEL. AND CHECK OUT — YOU WILL SEE IN THE SIMILAR SLIDE. SO WHAT IS CDM? IT’S A COMMON DATA MODEL WHICH IS A SHARED DATA MODEL THAT ALLOWS PAPS TO INTERRUPT BETWEEN EACH OTHER USING DATA. IN IT AND IT ENHANCES DATA WITH I-75 I-75 SEMANTIC DATA MATTER. SO WHEN YOU HAVE A DATA SOURCE, THAT SOURCE MAY HAVE ACRONYM, MAY HAVE A SIMILAR AND SORT OF REPEATING NAMES LIKE ONE OR TWO OR THREE. THAT’S NOT EXCITING. AND YOU SPELL LOTS OF TIMES DECIPHERING THAT. SO ONE OF THE GREAT PURPOSES IS TO PUT A SEMANTIC METADATA IN FRONT OF IT SO IT WILL

CAPTURE THE ENTITY, ATTRIBUTE, THE RELATIONSHIPS BETWEEN THEM. SO YOU KNOW THE ADDRESS BE LONGS WITH THE CUSTOMER. OR THAT PHONE NUMBER IS ACTUALLY AN EMERGENCY CONTACT PHONE NUMBER BECAUSE YOU CAN DECORATE IT WITH ACTUAL TRAITS, WE CALL THEM TRAITS. SO VERY ENRICHED METADATA THAT IS GREAT. SO THE CDM IS DISCOVERABLE AT WEBSITE. YOU CAN SEE THE CORE YOU WILL SEE THE — ON THE SITE, YOU WILL SEE THE CORE. WHAT IS THE CORE? THERE’S A LIST OF STANDARD DEFINITIONS THAT MICROSOFT PROVIDES LITERALLY AVAILABLE AT GITHUB AND YOU CAN READ THE DOCUMENTS ON OUR WEBSITE. THIS ISN’T TO SAY THAT YOU ARE FORCED TO USE ONLY THOSE ENTITIES. THESE ARE THE REFERENCE POINTS FOR YOU. THESE ARE THE STANDARD ENTITIES THAT CORRELATE WELL WITHOUT DYNAMICS 356 OFFERING AND USE THEM AS IS AND EXTEND THEM. DON’T MEAN YOU’RE GOING TO JOIN THE LIST. BUT GOING TO BE ENTITIES WHICH ARE YOUR OWN. AND SO THE ECOSYSTEM OF EXTERNAL PARTNER, THEY CAN USE ALL THE STANDARDIZED DEFINITION OF THE ENTITIES SO. IF YOU WRITE AN APP THAT IS AWARE OF CDM OR USE THE POWER BI IN A DATAFLOW, IT’S INPERSONALLY BUILT INTO THE TOOLS SO YOU WILL SEE THE SHAPES AND YOU WILL SEE RELATIONSHIPS WITH THE SHAPES. SO IN OTHER WORDS, IN A TRADITIONAL FEEL, HOW MANY ARE ATL? YEAH. SO YOU NORMALLY THINK, I NEED TO FIRST FIND MY DATA. I NEED TO FIND THE SOURCE OF WHERE THE DATA IS. I GOT ENCOURAGE YOU TO FLIP YOUR THINKING. WHAT AM I TRYING TO ACCOMPLISH? MA WHAT IS MY TARGET DATA? WHAT AM I TRYING TO BREAK? WHAT AM I TRYING TO BRING? YEAH, I SEE SMILES. WHAT IS THE DATA I’M TRYING TO BRING AND DO IT THAT WAY. SO YOU WILL DEFINE YOUR DATA THAT YOU WANT TO BRING IN FIRST IN THE SHAPE OF ENTITIES AND THEN USE THE MACHINERY TO LOAD THAT ONE UP. >> I THINK THEY WANT TO KICK US OUT. >> THE LAST WORDS BEFORE THEY DRAG ME OFF THE STAGE, THE GREAT ADVANTAGE OF USING CDM IS NOT ONLY INNER OP BUT ALSO ABILITY TO DO LOTS OF ANALYSIS ON THE DATA LINK. YOU CAN TURN ON ALL THE AI FEATURES THAT WILL NOT HAVE TO GUESS ANYMORE WHAT THE DATA MEANS. BUT THEY WOULD ACTUALLY KNOW WHAT IT MEANS BECAUSE THEY CAN EXPECT ATTRIBUTES AND TRAITS. AND THEN YOU CAN HAVE REPORTING AND YOU CAN HAVE ALL THE OTHER CONSUMPTIONS HERE. ARE THIS IS THE MONEY SHOT. THIS IS THE POWER OF CDM COMBINED WITH THE DATA LAKE AND UP ON THE TOP, YOU CAN SEE THE SOURCES THAT CAN CONTRIBUTE TO IT AND AT THE BOTTOM, BASICALLY SEE WHAT ARE THE TRANSACTIONAL AND NON-TRANSACTIONAL DATA BECAUSE CDM SPECIFIC. YOU CAN USE IoT DATA TO DEFINE IN TERMS OF CDM. THE CLICK STREAM DATA. CDM TAKE-AWAY, SPOKE TO IT. YOU CAN READ THE SLIDE. BASICALLY STOP THE BREAK-OUT SESSION TOMORROW 8: 30 GO BROWSE THE DATA LAKE BOOTH. FIND THE DATA LAKE, UNDERSTAND HOW IT’S USED. STOP AT POWER BI AND THE CDS, THE COMMON DATA SERVICE. YOU WILL SEE HOW IT WORKS. LOVE IT? >> YEP >> SO JUST — >> GOT TO LOVE CDM >> BREAK OUT 413. THAT’S OUR SESSION TOMORROW MORNING. >> IT’S KIND OF EARLY BUT IT WILL BE FUN. >> FULL HOUR, DATAFLOWS AND CDM. SO REALLY A DEEP DIVE. >> AND IF YOU WANT TO STOP BY OUR BOOTH ON THE OTHER SIDE OF THE EXPO AREA, PICK UP A POSTER OR TOMORROW MORNING AT THE SESSION AS WELL. THANK YOU. [ APPLAUSE ] >> GO CLICK. GO CLICK ON THE SITE