Location: Kochi/Trivandrum/Bengaluru/Remote
Experience: 4+ years

Job Role: Databricks Engineer

Qualification : Bachelors Degree

Mandatory skills: Databricks (Spark/R), ADF, ADLS, Azure Datawarehouse/Synapse, Kafka/Spark

Job Description

  • Strong knowledge of Data Management principles
  • Experience in building ETL/ELT transformation processes
  • Hands on experience in building data pipelines using Azure Data Factory and Databricks
  • Good understanding on Microsoft Azure Big Data Architecture
  • Hands on experience in designing and delivering solutions using the Azure Data Analytics platform
    including Azure Storage, Azure SQL Data Warehouse/Synapse, Azure Data Lake
  • Strong T-SQL skills with experience in Azure SQL DW/Synapse
  • Experience in Data Modeling and Advanced SQL techniques
  • Hands on experience with streaming data (Kafka, Nifi, Spark, etc)
  • Experience with structured and unstructured data
  • Experience in DevOps environment with tools such as Microsoft Visual Studio Team Services.
  • The candidate should work independently with minimal supervision. 4-7 Years of experience

Duties and Responsibilities

  • Work closely with clients/business analysts to translate their business problems into data science
    problems
  • Design and implement highly performant data ingestion pipelines from multiple sources using Azure
    Databricks & ADF
  • Develop scalable and re-usable frameworks for data ingestion
  • Work with event based / streaming technologies to ingest and process data
  • Optimize data processing

Good to have

  • Experience with other Open Source big data products Hadoop (incl. Hive, Pig, Impala)
  • Experience with Open Source non-relational / NoSQL databases (incl. MongoDB, Cassandra,
    etc..)
  • Lakehouse implementation using Databricks

Soft Skills Required

  • Must be a good team player with good communication skills
  • Must have good presentation skills
  • Must be a pro-active problem solver, good at logical reasoning and a leader by self