{"id":2643,"date":"2018-01-30T16:29:44","date_gmt":"2018-01-30T16:29:44","guid":{"rendered":"https:\/\/devbloglavaprotocols.nityo.in\/learning-about-data-processing-on-google-cloud\/"},"modified":"2018-01-30T16:29:44","modified_gmt":"2018-01-30T16:29:44","slug":"learning-about-data-processing-on-google-cloud","status":"publish","type":"post","link":"https:\/\/www.lavaprotocols.com\/the-cloud-blog\/learning-about-data-processing-on-google-cloud\/","title":{"rendered":"Learning about Data Processing on Google Cloud"},"content":{"rendered":"<div style=\"margin-top: 0px; margin-bottom: 0px;\" class=\"sharethis-inline-share-buttons\" ><\/div><p><span style=\"font-weight: 400;\">Everyone\u2019s talking about big data. But do companies actually know how to extract relevant and clean data to derive meaningful insights? <\/span><\/p>\n<p><!--more--><\/p>\n<p><span style=\"font-weight: 400;\">We wanted to equip our developers with the right knowledge about data processing on Google Cloud. So, we sent them for a four-day workshop, \u2018Data Engineering on Google Cloud Platform\u2019.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Held from Jan 30 to Feb 2, the workshop provided participants a hands-on experience on how to:<\/span><\/p>\n<ul>\n<li><span style=\"font-weight: 400;\">design and build data processing systems;<\/span><\/li>\n<li><span style=\"font-weight: 400;\">build end-to-end data pipelines;<\/span><\/li>\n<li><span style=\"font-weight: 400;\">analyse data;<\/span><\/li>\n<li><span style=\"font-weight: 400;\">and carry out machine learning;<\/span><\/li>\n<\/ul>\n<p><span style=\"font-weight: 400;\">on the Google Cloud Platform. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">The workshop also featured presentations and demos and covered structured, unstructured, and streaming data.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The modules covered were as follows:<\/span><\/p>\n<ul>\n<li><span style=\"font-weight: 400;\">Module 1: Google Cloud Dataproc Overview<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Module 2: Running Dataproc Jobs<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Module 3: Integrating Dataproc with Google Cloud Platform<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Module 4: Making Sense of Unstructured Data with Google\u2019s Machine Learning APIs<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Module 5: Serverless data analysis with BigQuery<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Module 6: Serverless, autoscaling data pipelines with Dataflow<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Module 7: Getting started with Machine Learning<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Module 8: Building ML models with Tensorflow<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Module 9: Scaling ML models with CloudML<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Module 10: Feature Engineering<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Module 11: Architecture of streaming analytics pipelines<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Module 12: Ingesting Variable Volumes<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Module 13: Implementing streaming pipelines<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Module 14: Streaming analytics and dashboards<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Module 15: High throughput and low-latency with Bigtable<\/span><\/li>\n<\/ul>\n<div class=\"wp-caption aligncenter\" id=\"attachment_10053\" style=\"width: 827px\">\n<p class=\"wp-caption-text\" id=\"caption-attachment-10053\"><em>Our developers Amiruddin (left) and Dzarief (right)<\/em><\/p>\n<\/div>\n<div class=\"wp-caption aligncenter\" id=\"attachment_10054\" style=\"width: 832px\">\n<p class=\"wp-caption-text\" id=\"caption-attachment-10054\"><em>Workshop in session.<\/em><\/p>\n<\/div>\n<p><i><span style=\"font-weight: 400;\">Lava is an authorised Cloud Partner of Google and is a <\/span><\/i><a href=\"https:\/\/lavaprotocols.com\/google-apps-for-work\/\"><i><span style=\"font-weight: 400;\">reseller of G Suite<\/span><\/i><\/a><i><span style=\"font-weight: 400;\"> (previously known as Google Apps, Google Maps for Work, and Google Cloud Platform) in Malaysia. With more than a decade of experience in the industry, we\u2019re proud to say we\u2019re one of the leading cloud consultants and service providers in the Asia Pacific region.<\/span><\/i><\/p>\n<p><script type=\"text\/javascript\">\npiAId = '418952';\npiCId = '56152';\npiHostname = 'pi.pardot.com';<\/p>\n<p>(function() {\n\tfunction async_load(){\n\t\tvar s = document.createElement('script'); s.type = 'text\/javascript';\n\t\ts.src = ('https:' == document.location.protocol ? 'https:\/\/pi' : 'http:\/\/cdn') + '.pardot.com\/pd.js';\n\t\tvar c = document.getElementsByTagName('script')[0]; c.parentNode.insertBefore(s, c);\n\t}\n\tif(window.attachEvent) { window.attachEvent('onload', async_load); }\n\telse { window.addEventListener('load', async_load, false); }\n})();\n<\/script><\/p>\n<p><span class=\"et_bloom_bottom_trigger\"><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>We wanted to equip our developers with the right knowledge about data processing on Google Cloud. So, we sent them for a four-day workshop.<\/p>\n","protected":false},"author":1,"featured_media":2644,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[53],"class_list":["post-2643","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","tag-events"],"jetpack_featured_media_url":"https:\/\/www.lavaprotocols.com\/the-cloud-blog\/wp-content\/uploads\/2024\/10\/IMG_1998-scaled.jpg","_links":{"self":[{"href":"https:\/\/www.lavaprotocols.com\/the-cloud-blog\/wp-json\/wp\/v2\/posts\/2643","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.lavaprotocols.com\/the-cloud-blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.lavaprotocols.com\/the-cloud-blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.lavaprotocols.com\/the-cloud-blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.lavaprotocols.com\/the-cloud-blog\/wp-json\/wp\/v2\/comments?post=2643"}],"version-history":[{"count":0,"href":"https:\/\/www.lavaprotocols.com\/the-cloud-blog\/wp-json\/wp\/v2\/posts\/2643\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.lavaprotocols.com\/the-cloud-blog\/wp-json\/wp\/v2\/media\/2644"}],"wp:attachment":[{"href":"https:\/\/www.lavaprotocols.com\/the-cloud-blog\/wp-json\/wp\/v2\/media?parent=2643"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.lavaprotocols.com\/the-cloud-blog\/wp-json\/wp\/v2\/categories?post=2643"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.lavaprotocols.com\/the-cloud-blog\/wp-json\/wp\/v2\/tags?post=2643"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}