Data Engineer

Job DescriptionMy client is a leading TechFin company that is revolutionising finance through innovative technology solutions.Role Overview: They are seeking a Data engineer to join our architecture team. You'll be pivotal in designing, developing, and maintaining a cutting-edge data lake on Azure with seamless integration with Databricks Lakehouse.Responsibilities:Develop robust data ingestion pipelines.Implement data transformation processes for analysis.Ensure seamless integration with Databricks Lakehouse.Implement security and compliance best practices.Collaborate on a scalable data lake architecture.Monitor and optimise data pipelines.Maintain thorough documentation.Plan for future scalability.Collaborate with cross-functional teams.Required Skills:Proficiency in Azure services (SQL Database, Data Lake Storage, Databricks, CosmosDB).Understanding of data modelling and governance principles.Experience with Python, Databricks, and ETL tools.Familiarity with Spark, Kafka, and Azure service integration.Skills in Power BI or Azure Data Explorer for visualisation.Knowledge of data security and compliance.Strong analytical and communication skills.Understanding of CI/CD practices.Familiarity with Azure Machine Learning and.NET is a plus.Benefits:Competitive salary & Day rate (looking for one contractor and one permanent position)Excellent benefits package, including pension, holidays, and insurance.Flexible working arrangements.
#J-18808-Ljbffr