Our client is looking for an experienced Spark data Engineer to join their team in Singapore.
Cloudera – CCA Spark and Hadoop Developer (CCA175)
Databricks Certified Developer: Apache Spark 2.X
Hortonworks – HDP Certified Apache Spark Developer
· Candidate to have 6 to 9 Years of IT Experience and in Big Data minimum 3+ Years of experience.
· Strong Hadoop , Distributed Computing and Cluster Knowledge, Cloudera Distribution experience is a plus.
· Good knowledge in Scala (with a focus on the functional programming paradigm)
· Hands on experience in spark using Scala, Good hands on experience on DataFrame, RDD logic, looping concepts and Hadoop architecture.
· Data transformation pipeline design/development knowledge is required.
· Need to have Strong problem solving and performance improvement skill.
· Create Scala/Spark jobs for data transformation and aggregation
· Produce unit tests for Spark transformations and helper methods
· Design data processing pipelines