Senior Data Engineer

Job Description
Our client within the Education sector are looking for a Senior Data Engineer to join them on a contract basis.
Inside IR35
6 month contract
Hybrid 1 – 2 days per week onsite
Technical Skills:
In depth knowledge of relevant Azure services such as Azure Data Lake Storage, Azure Data Factory, Azure Synapse Analytics, Azure Databricks, Azure SQL Database, Azure Stream Analytics, etc.
Strong Python or Scala with Spark, PySpark experience
Experience with relational databases and NoSQL databases
Significant experience and in-depth knowledge of creating data pipelines and associated design principles, standards, Data modelling concepts, ETL procedures, and all steps of data production process
Experience with unit and integration testing, and data quality frameworks such as Deequ, Great Expectations or Delta Live
Experience working with continuous integration
Good understanding of data modelling (medallion architectures, Kimball architectures), data schemas (Avro Schema, JSON Schema) and data serialisation (Parquet, ORC, Avro)
Azure Services Knowledge:
Azure Data Factory: Advanced knowledge of Azure Data Factory for orchestrating and automating data workflows.
Azure SQL Database: Expertise in SQL and T-SQL for data querying and manipulation.
Azure Synapse Analytics: Ability to design and implement data warehousing solutions using Azure Synapse Analytics.
Azure Databricks: Proficiency in using Azure Databricks for data processing and analytics.
Apache Spark: Deep understanding of Apache Spark for large-scale data processing.
Azure Blob Storage and Azure Data Lake Storage: Expertise in setting up and optimizing Azure Data Lake Storage for storing and managing large volumes of data.
Cloud Experience:
Resource Manager (ARM): Understanding how to deploy, manage, and monitor Azure resources using ARM templates.
Azure Portal and CLI: Proficiency in using the Azure Portal and Command-Line Interface for managing Azure resources.
Azure Security and Compliance: Knowledge of Azure security features and compliance standards.
Monitoring and Logging: Familiarity with Azure Monitor and Azure Log Analytics for monitoring and logging.
Ways of working:
Experience in Data Migration, Data Profiling and Data Quality enhancement.
Experience in Agile (Scrum) way of working
Understanding of CI/CD process & ways of working including Git
Excellent Problem-solving skills
Excellent Communication skills
Strong understanding of software development life cycle
• Knowledge of software engineering best practices for the full software development life cycle, including Agile methodologies, coding standards, code reviews, source control management, build processes, testing, etc
If you believe you have the right skills for this role please submit your CV.