{"id":6693,"date":"2015-03-17T15:02:24","date_gmt":"2015-03-17T18:02:24","guid":{"rendered":"https:\/\/za.mus.br\/?p=6693"},"modified":"2015-07-22T15:47:48","modified_gmt":"2015-07-22T18:47:48","slug":"google-cloud-dataflow","status":"publish","type":"post","link":"https:\/\/za.mus.br\/en\/google-cloud-dataflow\/","title":{"rendered":"Google Cloud Dataflow"},"content":{"rendered":"<p>Com nossa <a title=\"Spin-off!\" href=\"\/?p=6399\">mudan\u00e7a de paradigma<\/a>, come\u00e7amos a lidar com um monte de tecnologias novas e, principalmente, com Big Data. Neste processo, come\u00e7amos a trabalhar com <a href=\"http:\/\/hadoop.apache.org\/\" target=\"_blank\">Hadoop<\/a> e <a href=\"http:\/\/pt.wikipedia.org\/wiki\/MapReduce\" target=\"_blank\">MapReduce<\/a>, na inten\u00e7\u00e3o de poder extrair, analisar e gerar insights para\u00a0o Mercado da M\u00fasica. <a href=\"http:\/\/pt.wikipedia.org\/wiki\/MapReduce\" target=\"_blank\">MapReduce<\/a> foi inventado pelo Google para poder processar a quantidade de informa\u00e7\u00e3o que eles trabalham, mas parece que a quantidade aumentou de uma forma t\u00e3o grande que tiveram que inventar uma nova: <a href=\"https:\/\/cloud.google.com\/dataflow\" target=\"_blank\">Google Cloud Dataflow<\/a>.<\/p>\n<p>Um breve esquema abaixo, sem maiores detalhes, para aqueles que se interessam pela parte mais t\u00e9cnica do assunto.<\/p>\n<p><a href=\"https:\/\/i0.wp.com\/za.mus.br\/wp-content\/uploads\/1218.sdt-google-news.jpg?ssl=1\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-6695\" src=\"https:\/\/i0.wp.com\/za.mus.br\/wp-content\/uploads\/1218.sdt-google-news.jpg?resize=1000%2C563&#038;ssl=1\" alt=\"Google Dataflow Schema\" width=\"1000\" height=\"563\" srcset=\"https:\/\/i0.wp.com\/za.mus.br\/wp-content\/uploads\/1218.sdt-google-news.jpg?w=1000&amp;ssl=1 1000w, https:\/\/i0.wp.com\/za.mus.br\/wp-content\/uploads\/1218.sdt-google-news.jpg?resize=300%2C169&amp;ssl=1 300w, https:\/\/i0.wp.com\/za.mus.br\/wp-content\/uploads\/1218.sdt-google-news.jpg?resize=60%2C34&amp;ssl=1 60w, https:\/\/i0.wp.com\/za.mus.br\/wp-content\/uploads\/1218.sdt-google-news.jpg?resize=672%2C378&amp;ssl=1 672w\" sizes=\"auto, (max-width: 1000px) 100vw, 1000px\" \/><\/a><\/p>\n<p>&nbsp;<\/p>\n<p>No Google I\/O de 2014 o vice presidente de infraestrutura do Google, <a href=\"https:\/\/plus.google.com\/+UrsH\u00f6lzle\" target=\"_blank\">Urs H\u00f6lzle<\/a>, declarou que eles pr\u00f3prios <a href=\"http:\/\/www.datacenterknowledge.com\/archives\/2014\/06\/25\/google-dumps-mapreduce-favor-new-hyper-scale-analytics-system\/\" target=\"_blank\">n\u00e3o utilizam MapReduce<\/a> h\u00e1 anos!<\/p>\n<blockquote><p><em>\u201cWe don\u2019t really use MapReduce anymore.\u201d<\/em><\/p><\/blockquote>\n<p>Todos que trabalham com Big Data j\u00e1 devem saber disso, mas de uma coisa eles ainda n\u00e3o sabem&#8230; n\u00f3s, da Zamus, fomos selecionados para testar o <a href=\"https:\/\/cloud.google.com\/dataflow\" target=\"_blank\">Alpha do Google Dataflow<\/a>!<\/p>\n<blockquote><p><em>\u201cCloud Dataflow is the result of over a decade of experience in analytics. It will run faster and scale better than pretty much any other system out there.\u201d<\/em><\/p><\/blockquote>\n<p>Isso significa que poderemos melhorar ainda mais nossos processos, velocidade, economia e entregar mais qualidade em nossas an\u00e1lises sobre o Mercado da M\u00fasica.<\/p>\n<p>Vamos compartilhar esperi\u00eancias sobre Hadoop, MapReduce e Dataflow? Voc\u00ea utiliza outra solu\u00e7\u00e3o? Qual?<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Com nossa mudan\u00e7a de paradigma, come\u00e7amos a lidar com um monte de tecnologias novas e, principalmente, com Big Data. Neste processo, come\u00e7amos a trabalhar com Hadoop e MapReduce, na inten\u00e7\u00e3o de poder extrair, analisar e gerar insights para\u00a0o Mercado da M\u00fasica. MapReduce foi inventado pelo Google para poder processar a quantidade de informa\u00e7\u00e3o que eles trabalham, mas parece que a [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":6694,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[7],"tags":[1642,1643,34],"class_list":["post-6693","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-noticias","tag-analytics","tag-big-data","tag-google-music"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.9 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Google Cloud Dataflow - Zamus<\/title>\n<meta name=\"description\" content=\"No Universo Analytics temos que lidar com um fluxo de informa\u00e7\u00e3o inimagin\u00e1vel. Mas o Google, nos ajudar\u00e1. Anunciando Google Cloud Dataflow!\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/za.mus.br\/google-cloud-dataflow\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Google Cloud Dataflow - Zamus\" \/>\n<meta property=\"og:description\" content=\"No Universo Analytics temos que lidar com um fluxo de informa\u00e7\u00e3o inimagin\u00e1vel. Mas o Google, nos ajudar\u00e1. Anunciando Google Cloud Dataflow!\" \/>\n<meta property=\"og:url\" content=\"https:\/\/za.mus.br\/google-cloud-dataflow\/\" \/>\n<meta property=\"og:site_name\" content=\"Zamus\" \/>\n<meta property=\"article:publisher\" content=\"http:\/\/facebook.com\/zamus\" \/>\n<meta property=\"article:author\" content=\"http:\/\/www.facebook.com\/100003649494215\" \/>\n<meta property=\"article:published_time\" content=\"2015-03-17T18:02:24+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2015-07-22T18:47:48+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/i0.wp.com\/za.mus.br\/wp-content\/uploads\/cloud-data-flow-600x400.jpg?fit=600%2C400&ssl=1\" \/>\n\t<meta property=\"og:image:width\" content=\"600\" \/>\n\t<meta property=\"og:image:height\" content=\"400\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Zamus\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@zamusbr\" \/>\n<meta name=\"twitter:site\" content=\"@zamusbr\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Zamus\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/za.mus.br\\\/google-cloud-dataflow\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/za.mus.br\\\/google-cloud-dataflow\\\/\"},\"author\":{\"name\":\"Zamus\",\"@id\":\"https:\\\/\\\/za.mus.br\\\/#\\\/schema\\\/person\\\/4f875b70f5af4571b625b6fdfc9455c5\"},\"headline\":\"Google Cloud Dataflow\",\"datePublished\":\"2015-03-17T18:02:24+00:00\",\"dateModified\":\"2015-07-22T18:47:48+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/za.mus.br\\\/google-cloud-dataflow\\\/\"},\"wordCount\":240,\"commentCount\":0,\"image\":{\"@id\":\"https:\\\/\\\/za.mus.br\\\/google-cloud-dataflow\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/i0.wp.com\\\/za.mus.br\\\/wp-content\\\/uploads\\\/cloud-data-flow-600x400.jpg?fit=600%2C400&ssl=1\",\"keywords\":[\"Analytics\",\"Big Data\",\"Google Music\"],\"articleSection\":[\"Not\u00edcias\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/za.mus.br\\\/google-cloud-dataflow\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/za.mus.br\\\/google-cloud-dataflow\\\/\",\"url\":\"https:\\\/\\\/za.mus.br\\\/google-cloud-dataflow\\\/\",\"name\":\"Google Cloud Dataflow - Zamus\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/za.mus.br\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/za.mus.br\\\/google-cloud-dataflow\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/za.mus.br\\\/google-cloud-dataflow\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/i0.wp.com\\\/za.mus.br\\\/wp-content\\\/uploads\\\/cloud-data-flow-600x400.jpg?fit=600%2C400&ssl=1\",\"datePublished\":\"2015-03-17T18:02:24+00:00\",\"dateModified\":\"2015-07-22T18:47:48+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/za.mus.br\\\/#\\\/schema\\\/person\\\/4f875b70f5af4571b625b6fdfc9455c5\"},\"description\":\"No Universo Analytics temos que lidar com um fluxo de informa\u00e7\u00e3o inimagin\u00e1vel. Mas o Google, nos ajudar\u00e1. Anunciando Google Cloud Dataflow!\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/za.mus.br\\\/google-cloud-dataflow\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/za.mus.br\\\/google-cloud-dataflow\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/za.mus.br\\\/google-cloud-dataflow\\\/#primaryimage\",\"url\":\"https:\\\/\\\/i0.wp.com\\\/za.mus.br\\\/wp-content\\\/uploads\\\/cloud-data-flow-600x400.jpg?fit=600%2C400&ssl=1\",\"contentUrl\":\"https:\\\/\\\/i0.wp.com\\\/za.mus.br\\\/wp-content\\\/uploads\\\/cloud-data-flow-600x400.jpg?fit=600%2C400&ssl=1\",\"width\":600,\"height\":400},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/za.mus.br\\\/google-cloud-dataflow\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"In\u00edcio\",\"item\":\"https:\\\/\\\/za.mus.br\\\/en\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Google Cloud Dataflow\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/za.mus.br\\\/#website\",\"url\":\"https:\\\/\\\/za.mus.br\\\/\",\"name\":\"Zamus\",\"description\":\"The New Music Market\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/za.mus.br\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/za.mus.br\\\/#\\\/schema\\\/person\\\/4f875b70f5af4571b625b6fdfc9455c5\",\"name\":\"Zamus\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/za.mus.br\\\/wp-content\\\/uploads\\\/avatars\\\/2\\\/bdaf423d199575f03f26e548d1c12cb9-bpfull.png\",\"url\":\"https:\\\/\\\/za.mus.br\\\/wp-content\\\/uploads\\\/avatars\\\/2\\\/bdaf423d199575f03f26e548d1c12cb9-bpfull.png\",\"contentUrl\":\"https:\\\/\\\/za.mus.br\\\/wp-content\\\/uploads\\\/avatars\\\/2\\\/bdaf423d199575f03f26e548d1c12cb9-bpfull.png\",\"caption\":\"Zamus\"},\"sameAs\":[\"https:\\\/\\\/za.mus.br\",\"http:\\\/\\\/www.facebook.com\\\/100003649494215\"],\"url\":\"https:\\\/\\\/za.mus.br\\\/en\\\/author\\\/zamus\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Google Cloud Dataflow - Zamus","description":"No Universo Analytics temos que lidar com um fluxo de informa\u00e7\u00e3o inimagin\u00e1vel. Mas o Google, nos ajudar\u00e1. Anunciando Google Cloud Dataflow!","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/za.mus.br\/google-cloud-dataflow\/","og_locale":"en_US","og_type":"article","og_title":"Google Cloud Dataflow - Zamus","og_description":"No Universo Analytics temos que lidar com um fluxo de informa\u00e7\u00e3o inimagin\u00e1vel. Mas o Google, nos ajudar\u00e1. Anunciando Google Cloud Dataflow!","og_url":"https:\/\/za.mus.br\/google-cloud-dataflow\/","og_site_name":"Zamus","article_publisher":"http:\/\/facebook.com\/zamus","article_author":"http:\/\/www.facebook.com\/100003649494215","article_published_time":"2015-03-17T18:02:24+00:00","article_modified_time":"2015-07-22T18:47:48+00:00","og_image":[{"width":600,"height":400,"url":"https:\/\/i0.wp.com\/za.mus.br\/wp-content\/uploads\/cloud-data-flow-600x400.jpg?fit=600%2C400&ssl=1","type":"image\/jpeg"}],"author":"Zamus","twitter_card":"summary_large_image","twitter_creator":"@zamusbr","twitter_site":"@zamusbr","twitter_misc":{"Written by":"Zamus","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/za.mus.br\/google-cloud-dataflow\/#article","isPartOf":{"@id":"https:\/\/za.mus.br\/google-cloud-dataflow\/"},"author":{"name":"Zamus","@id":"https:\/\/za.mus.br\/#\/schema\/person\/4f875b70f5af4571b625b6fdfc9455c5"},"headline":"Google Cloud Dataflow","datePublished":"2015-03-17T18:02:24+00:00","dateModified":"2015-07-22T18:47:48+00:00","mainEntityOfPage":{"@id":"https:\/\/za.mus.br\/google-cloud-dataflow\/"},"wordCount":240,"commentCount":0,"image":{"@id":"https:\/\/za.mus.br\/google-cloud-dataflow\/#primaryimage"},"thumbnailUrl":"https:\/\/i0.wp.com\/za.mus.br\/wp-content\/uploads\/cloud-data-flow-600x400.jpg?fit=600%2C400&ssl=1","keywords":["Analytics","Big Data","Google Music"],"articleSection":["Not\u00edcias"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/za.mus.br\/google-cloud-dataflow\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/za.mus.br\/google-cloud-dataflow\/","url":"https:\/\/za.mus.br\/google-cloud-dataflow\/","name":"Google Cloud Dataflow - Zamus","isPartOf":{"@id":"https:\/\/za.mus.br\/#website"},"primaryImageOfPage":{"@id":"https:\/\/za.mus.br\/google-cloud-dataflow\/#primaryimage"},"image":{"@id":"https:\/\/za.mus.br\/google-cloud-dataflow\/#primaryimage"},"thumbnailUrl":"https:\/\/i0.wp.com\/za.mus.br\/wp-content\/uploads\/cloud-data-flow-600x400.jpg?fit=600%2C400&ssl=1","datePublished":"2015-03-17T18:02:24+00:00","dateModified":"2015-07-22T18:47:48+00:00","author":{"@id":"https:\/\/za.mus.br\/#\/schema\/person\/4f875b70f5af4571b625b6fdfc9455c5"},"description":"No Universo Analytics temos que lidar com um fluxo de informa\u00e7\u00e3o inimagin\u00e1vel. Mas o Google, nos ajudar\u00e1. Anunciando Google Cloud Dataflow!","breadcrumb":{"@id":"https:\/\/za.mus.br\/google-cloud-dataflow\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/za.mus.br\/google-cloud-dataflow\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/za.mus.br\/google-cloud-dataflow\/#primaryimage","url":"https:\/\/i0.wp.com\/za.mus.br\/wp-content\/uploads\/cloud-data-flow-600x400.jpg?fit=600%2C400&ssl=1","contentUrl":"https:\/\/i0.wp.com\/za.mus.br\/wp-content\/uploads\/cloud-data-flow-600x400.jpg?fit=600%2C400&ssl=1","width":600,"height":400},{"@type":"BreadcrumbList","@id":"https:\/\/za.mus.br\/google-cloud-dataflow\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"In\u00edcio","item":"https:\/\/za.mus.br\/en\/"},{"@type":"ListItem","position":2,"name":"Google Cloud Dataflow"}]},{"@type":"WebSite","@id":"https:\/\/za.mus.br\/#website","url":"https:\/\/za.mus.br\/","name":"Zamus","description":"The New Music Market","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/za.mus.br\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/za.mus.br\/#\/schema\/person\/4f875b70f5af4571b625b6fdfc9455c5","name":"Zamus","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/za.mus.br\/wp-content\/uploads\/avatars\/2\/bdaf423d199575f03f26e548d1c12cb9-bpfull.png","url":"https:\/\/za.mus.br\/wp-content\/uploads\/avatars\/2\/bdaf423d199575f03f26e548d1c12cb9-bpfull.png","contentUrl":"https:\/\/za.mus.br\/wp-content\/uploads\/avatars\/2\/bdaf423d199575f03f26e548d1c12cb9-bpfull.png","caption":"Zamus"},"sameAs":["https:\/\/za.mus.br","http:\/\/www.facebook.com\/100003649494215"],"url":"https:\/\/za.mus.br\/en\/author\/zamus\/"}]}},"jetpack_featured_media_url":"https:\/\/i0.wp.com\/za.mus.br\/wp-content\/uploads\/cloud-data-flow-600x400.jpg?fit=600%2C400&ssl=1","featured_image_src":"https:\/\/i0.wp.com\/za.mus.br\/wp-content\/uploads\/cloud-data-flow-600x400.jpg?fit=600%2C400&ssl=1","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/za.mus.br\/en\/wp-json\/wp\/v2\/posts\/6693","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/za.mus.br\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/za.mus.br\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/za.mus.br\/en\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/za.mus.br\/en\/wp-json\/wp\/v2\/comments?post=6693"}],"version-history":[{"count":0,"href":"https:\/\/za.mus.br\/en\/wp-json\/wp\/v2\/posts\/6693\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/za.mus.br\/en\/wp-json\/wp\/v2\/media\/6694"}],"wp:attachment":[{"href":"https:\/\/za.mus.br\/en\/wp-json\/wp\/v2\/media?parent=6693"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/za.mus.br\/en\/wp-json\/wp\/v2\/categories?post=6693"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/za.mus.br\/en\/wp-json\/wp\/v2\/tags?post=6693"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}