MDM Data Engineer

Responsibilities

Candidates can look forward to regular client contact, a role in business development and proposal writing/methodology creation, and a

• Act as the proactive and technical architect point person for the consumer data platform end to end (ingestion through activation of data).
• Lead the transformation of existing data and analytic infrastructure to a highly scalable, flexible, and performant big data platform using appropriate SQL, NoSQL, and NewSQL technologies.
• Design and build efficient ETL/ELT process to move data through the data processing pipeline to meet the demands of the business use cases.
• Optimize and automate data ingestion, data processing and distribution data from a variety of sources, including clickstream data, rating data, advertising data, 3rd party sources and sources not yet identified.
• Manage complex data dependencies across datasets and incremental data loading workflows.
• Collaborating with product management and acting as the bridge between product management, engineering teams, and customers to understand requirements and technical solutions
• Help us stay ahead of the curve by working closely with data management team, data engineers, our DevOps team, and analysts to design systems which can scale overnight in ways which make other groups jealous.

Skills/Experience

• First-hand experience – Have a minimum of 5 years of professional experience focusing on building data movement and integration pipelines (especially ETL/ELT) in a large-scale data environment leveraging the latest open source and Java technologies
• Query ninja – Must have a firm understanding of database systems – Data modeling, SQL query Processing and Transactions Know how to scale systems and make them fast with experience debugging and tracing SQL performance issues
• Visionary – Solid understanding of software development from design and architecture to production.
• Have a data toolbox – Familiar with technologies relevant to the data and integration space including Hadoop (EMR, HDInsight or HDP), Redshift or Azure SQL DataWarehouse, Hive, Spark, Python, DI tools (like Streamsets, NiFi), and MDM tools
• Problem solver – Enjoy new and meaningful technology or business challenges which require you to think and respond quickly
• Passion and creativity – Are passionate about data, technology, & creative innovation
• Team player – Enjoy working collaboratively with a talented group of people to tackle challenging business problems so we all succeed (or fail fast) as a team.