We are looking for a Data Pipeline Engineer for a company dedicated to web development and artificial intelligence.
Duties & Responsibilities
· Leverage Data Quality Central and other software to proactively monitor the data pipeline for performance and data quality issues and degradation.
· Log issues and provide detailed issue tracking and status updates using Jira Support Desk ticketing system.
· Conduct root cause analysis of failed jobs, integration issues, performance issues, and data quality issues.
· Identify and propose scalable solutions to issues, including impact analysis and preventative maintenance proposals.
· Update data pipeline knowledge base in run book with details around client environment nuances, root cause, solution, troubleshooting steps, and ways to determine things are correct.
· Escalate issues and bug fixes to Data development team to ensure that they are aware of all systemic issues and can integrate solutions into code base.
· Train and consult with 24x7 ETL support and Data Services implementation teams to ensure the knowledge base is comprehensive and promotes their ability to independently troubleshoot and implement solutions and understand nuanced situations that require different approaches.
· Provide training on performance concepts to facilitate higher quality builds across the DOS platform.
· Intermediate level in Structured Query Language (SQL).
· Experience working with Azure DevOps.
· Experience working in both SQL Server.
· Experience identifying performance configuration or data quality gaps/issues and writing checks to facilitate validation and monitoring that covers the gap.
· Demonstrated ability and desire to provide excellent and proactive customer service.
· Demonstrated ability and desire to communicate, collaborate, and work within and across teams.
· Ability to quickly understand and explain why something does/does not work and then identify scalable opportunities to improve.
· Demonstrated ability and desire to efficiently learn and effectively leverage new concepts, skills, and technologies; and willingness to ask for help when needed.
· Desire to specialize and become an expert in performance, data quality, and the interaction between the data pipeline and DOS.
· Experience implementing or working with DOS Marts.
· Passionately believes that both timely AND correct data are possible.
· Has a testing mindset and can think of how and why something will or will not work.
· Focuses on scalable and lasting solutions versus custom and one-off stopgaps.
· Health care data experience.
· Experience with SQL Server Integration Services (SSIS)
· Visualization experience using a variety of BI tools.
· Quality improvement - Lean, 6 Sigma, etc.
Education & Relevant Experience
· BS/BA in information technology, math, statistics or business-related field or equivalent relevant work experience
· 5+ years’ experience in technology or technology related field