Bromley Business Centre, 27 Hastings Road, Bromley, Kent BR2 8NA, UK

Spark Data Engineer

Our client is looking for an experienced Spark data Engineer to join their team in Singapore.

Cloudera – CCA Spark and Hadoop Developer (CCA175)
Databricks Certified Developer: Apache Spark 2.X
Hortonworks – HDP Certified Apache Spark Developer

· Candidate to have 6 to 9 Years of IT Experience and in Big Data minimum 3+ Years of experience.
· Strong Hadoop , Distributed Computing and Cluster Knowledge, Cloudera Distribution experience is a plus.
· Good knowledge in Scala (with a focus on the functional programming paradigm)
· Hands on experience in spark using Scala, Good hands on experience on DataFrame, RDD logic, looping concepts and Hadoop architecture.
· Data transformation pipeline design/development knowledge is required.
· Need to have Strong problem solving and performance improvement skill.
· Create Scala/Spark jobs for data transformation and aggregation
· Produce unit tests for Spark transformations and helper methods
· Design data processing pipelines