Data Engineer II

Data Engineer II - Will be responsible for Ingestion pipelines, Data management, working closely with Product owner to understand the requirements, prioritize, working with suppliers & consumers of data, impact analysis of existing change requests etc.
Key Responsibilities
Data Pipelines process data structured and unstructured data

• Gathers and processes raw, structured, semi-structured, and unstructured data using batch and real-time data processing frameworks.
• Developing, scheduling, and monitoring batch-oriented workflows (Airflow Apache)
• Understands and enforces appropriate data master management techniques.

Data Validation and Quality

• Ensures data quality and implements tools and frameworks for automating the identification of data quality issues.
• Work with internal and external data providers on data validation providing feedback and making customized changes to data feeds and data mappings.

Agile Planning

• Understands the challenges that the data platform organization faces in their day-to-day work and partner with them to design viable data solutions.
• Working closely with Product owner and Engineering Manager on understanding features, capacity planning, prioritization and dependencies.
• Recommends improvements to processes, technology, and interfaces that improve the effectiveness of the team and reduce technical debt.

Application Support
Provides ongoing on-call support, monitoring, and maintenance of deployed data pipelines / data products, working with offshore shore developers/team
Emerging Technology Monitoring
Actively participates in the engineering community, staying up to date on new data technologies and best practices and shares insights with others in the organization.
Essential:

• Working experience in design, development, and implementation of highly scalable data ingestion pipelines
• Working experience with batch and real-time data processing frameworks
• Working experience with data modelling, data access, and data storage techniques
• Working experience working with relational and NoSQL databases
• Working experience with data quality tools
• Working experience with application lifecycle methodologies (e.g., agile, iterative)
• Bachelor's degree in Computer Science or related field plus 2 years working experience or 6 years working experience with no degree or equivalent combination of industry related professional experience and education.
• Scrum Developer Certification or equivalent
• Technical Expertise: SQL, snowflake, data warehousing, ETL/ELT process

Must have skills: Python (API calls, functions/API snowflake, airflow) SQL, AWS Cloud (security, IAM, role & permissions, S3, SQS, SNS, EMR, CloudWatch), performance tuning (pipelines/SQL).
Nice to have: Core Java, Spring boot, Scala & Spark, BigData, Data Lake,
To be employed in this role, you'll need to clear a pre-employment background check and drug test. Cox Automotive does not currently consider a pre-employment drug test for marijuana for this position. However, we are a drug-free workplace, so the possession, use or being under the influence of drugs illegal under federal or state law during work hours, on company property and/or in company vehicles is prohibited.
About Cox Automotive
At Cox Automotive, people of every background are driven by their passion for mobility, innovation and community. We transform the way the world buys, sells, owns and uses cars, accelerating the industry with global powerhouse brands like Autotrader, Kelley Blue Book, Manheim and more. What's more, we do it all with an emphasis on employee growth and happiness. Drive your future forward and join Cox Automotive today!
About Cox
Cox empowers employees to build a better future and has been doing so for over 120 years. With exciting investments and innovations across transportation, communications, cleantech and healthcare, our family of businesses - which includes Cox Automotive and Cox Communications - is forging a better future for us all. Ready to make your mark? Join us today!
Benefits of working at Cox may include health care insurance (medical, dental, vision), retirement planning (401(k)), and paid days off (sick leave, parental leave, flexible vacation/wellness days, and/or PTO). For more details on what benefits you may be offered, visit our benefits page .
Cox is an Equal Employment Opportunity employer - All qualified applicants/employees will receive consideration for employment without regard to that individual's age, race, color, religion or creed, national origin or ancestry, sex (including pregnancy), sexual orientation, gender, gender identity, physical or mental disability, veteran status, genetic information, ethnicity, citizenship, or any other characteristic protected by law. Cox provides reasonable accommodations when requested by a qualified applicant or employee with disability, unless such accommodations would cause an undue hardship.
Statement to ALL Third-Party Agencies and Similar Organizations: Cox accepts resumes only from agencies with which we formally engage their services. Please do not forward resumes to our applicant tracking system, Cox employees, Cox hiring manager, or send to any Cox facility. Cox is not responsible for any fees or charges associated with unsolicited resumes.