Data Engineer

At The Aventum Group, we’re not one for following the usual rules. We work hard at being different and that very much extends to how we look after our people. Because for us, they’re not just people. They’re part of the family. A family that dreams big and always works together to make insurance better. For everyone.We offer our employees a unique benefits package that includes personal training sessions, well-being provisions, and an additional day off for your birthday! We benefit from a learning culture and are keen to develop our employees to advance their careers at Aventum.Role SummaryYou will work in various settings to build systems that collect, manage, and convert raw data into usable information for data scientists and business analysts to interpret. Your ultimate goal is to make data accessible so that Aventum stakeholders can use it to evaluate and optimise their performance.Role AccountabilitiesSupport designing, developing, implementing, managing and supporting enterprise-level ELT/ELT processes and environmentTechnical and business processes are used to combine data from multiple sources to provide a unified, single view of the data. Be an architect making strategic decisions.Partly responsible for accessing, validating, and querying data from various repositories using available tools. Build and maintain data integration processes using SQL Services and other ETL / ELT processes and scripting tools as well as ongoing requests and projects related to the data warehouse, MI, or fast-moving financial data.Designing the
infrastructure/architecture
of the big data platform.Evaluating, comparing and improving the different approaches including design patterns innovation, data lifecycle design, data ontology alignment, annotated datasets, and elastic search approachesDeveloping, creating and maintaining a reliable data pipeline and schemas that feed other data processes; includes both technical processes and business logic to transform data from disparate sources into cohesive meaningful and valuable data with quality, governance and compliance considerations.Customising and managing integration tools, databases, warehouses, and analytical systems.Identifying and eliminating all non-value-adding activities through automation or outsourcing.Operations:Design and implement the management, monitoring, security, and privacy of data using the full stack of Azure data services to satisfy business needs.Ensuring non-functional system characteristics such as such as security, maintainability, quality, performance, and reliability are captured, prioritized, and incorporated into products.Leverage Agile, CI / CD and DevOps methodologies to deliver high-quality products on time.Architecting, building, testing, and maintaining data platform. Develop and support a wide range of data transformations and migrations for the whole business.Construct custom ETL processes: Design and implement data pipelines, data marts and schemas, access versatile data sources and apply data quality measures.Monitoring the complete process and applying necessary infrastructure changes to speed up the query execution and analyse data more efficiently; this includes Database optimisation techniques (data partitioning, database indexing and denormalisation) & efficient data ingestion (data mining techniques and different data ingestion APIs).Responding to errors and alerts to correct and re-process data. Investigate data mismatches and apply solutions. Data scrubbing and analysis to troubleshoot data integration issues and determine root cause.Any additional duties as assigned.Role RequirementsBachelor’s degree or equivalent in an
engineering/numerate
subject (e.g. Engineering, Stats, Maths, Computer Sciences)Some experience as a developer building data pipelines and schemas, with data WH implementation, with SQL database developmentHands-on experience using Synapse or related tools with cloud-based resources (e.g. Stored Procedures, ADF, NoSQL Databases, JSON/XML data formats)Hands-on experience with Azure Functions, Azure service bus, Azure Data Factory data integration techniquesKnowledge of Data Modelling concepts, monitoring, designs and techniquesKnowledge of Data Warehouse project lifecycle, tools, technologies, and best practicesExperience with Azure SQL Database, Cosmos DB, NoSQL, MongoDBExperience with Agile, DevOps methodologiesAwareness and knowledge of ELT/ETL, DWH, APIs (RESTful), Spark APIs, FTP protocols, SSL, SFTP, PKI (Public Key Infrastructure) and Integration testingSkills and AbilitiesKnowledge of Python, SQL, SSIS, and Spark languages.Demonstrative ability to develop complex SQL queries and Stored ProceduresRelationship-building and stakeholder managementWe are an equal-opportunity employer, and we are proud to share that 93% of our employees say they can be themselves at work. We aim to hire our industry's finest people because the best people drive the best outcomes. And we forever challenge the status quo because we know there are always ways to improve things. Because together, we're limitless.We value applicants from all backgrounds and foster a culture of inclusivity. We understand the need for flexibility, so work in a hybrid model. Please let us know if you require any reasonable adjustments during recruitment.Send me alerts about jobs like this.Please enter your email address to continue setting up an email alert for similar jobs to this one. By entering your email address and clicking apply you will sign up to Jobs4 and agree to our terms and conditions .
#J-18808-Ljbffr