Job Categories: Data Engineer

Date Engineer 

As part of the Data Platforms team, the Data Engineer position contributes to the overall BPX Data Platform Strategy. 

• Define data workflow, pipelines, security, and ELT/ETL guidelines, policies, and procedures
• Provide oversight to ensure that project teams are following the approved data workflow, pipelines, security, and ETL/ELT processing guidelines
• Evaluate, compare, and recommend new Data Platform vendor products and/or tools
• Partner with project teams to provide technical direction and new integration patterns, where needed, for complex and/or large data sets

Present modifications to Data Platforms to the Architecture Review Board for approval

Responsibilities: 
 

• Responsible for the development, guidance, and oversight for data ingestions and integrations for BPX cloud data platforms.
• Work with project teams to provide direction on approved patterns to meet the project data requirements for our Data Platforms.
• Strong SQL development knowledge required to provide support to project developers/users for database design, data flow and analysis activities.
• Development and deployment of innovative big data platforms for advanced analytics and data processing.
• Define and build the data rules and pipelines that will enable faster, better, data-informed decision-making within the business.
• Ensure stable solutions, stable infrastructure, and participate in the design and build out of data solutions.

• Be comfortable operating as an individual contributor and using influence and expertise to aid the transformation of an organization.
• The database and data support roles are outsourced to a managed services partner that resides within the Data Platform team, and this role will require the ability to work and partner with those resources.
• Create data storage solutions optimized for performance and maintain data architecture standards across BPX Data Platforms.
• Leads innovation through exploration, benchmarking, making recommendations, and implementing data technologies.
• Partner with the Data Quality Lead to incorporate testing and validation scripts to support the accuracy of data transformations.
• Implement Master Data Management solutions, processes, and governance to support the Data Management team.
• Partner with the DataOps support team to troubleshoot data issues and provide guidance for solving these issues.
• Be knowledgeable of industry trends and best practices, advising senior management on new and improved data engineering cloud strategies that will drive departmental performance leading to improvement in overall improvement in data governance across the business, promoting informed decision-making, and ultimately improving overall business performance.
• Drive excellent, consistent customer service.

 

Teamwork

• Facilitates effective team interaction
• Acknowledges and appreciates each team member's contributions

Works effectively with distributed team members

Education: 
BS in Computer Science, MIS, Mathematics, or equivalent degree

Experience: 
 

• 3-5+ years of Data Engineering experience
• 3-5+ years of relevant work experience in IT/Data and/or Analytics space
• Experience in any cloud data warehouse, Snowflake preferred
• Experience with replication tools, Fivetran preferred
• Experience with transformation tools, dbt preferred
• Experience with programming tools, python preferred
• Experience with REST APIs for data ingestion
• Strong understanding of ETL/ELT processing with large data stores
• Experience designing and delivering large scale, 24-7, mission-critical data pipelines and features using modern big data architectures
• Stream processing services such as Kafka, AWS Kinesis, Apache Storm, Spark Streaming, Azure Event Grid etc
• Demonstrated experience working in large-scale data environments which includes real-time and batch processing
• Strong data modeling skills (relational, dimensional, and flattened)
• Strong analytical and SQL skills, with attention to detail
• Ability to aid in tuning and performance recommendations for poor performing SQL queries and/or python scripts
• Knowledge of Database Administration tasks – Indexing, SQL Tuning/Performance, Backup/Recovery, DR
• Ability to work with multiple external teams and accomplish shared goals by building consensus
• Strong communication (written/verbal) and collaboration skills
• Consulting, negotiation, and relationship skills
• Problem solving skills
• Enthusiastic, high-energy individual, self-motivated, people-oriented, and self-directed
• Must be an intelligent, articulate, and persuasive leader who can serve as an effective member of the team, who can communicate concepts to technical & nontechnical colleagues. 
• Must be able to maintain focus on achieving results, whilst being patient and pragmatic