John Bemis

Benchmark IT - Technology Talent

4.83

Job

Senior Data Engineer

Houston, Texas, United States

DIRECT HIRE

Senior Data Engineer

Core Responsibilities

Design and maintain highly scalable data pipelines and ETL processes using AWS-native services
Build and manage data lake and data warehouse solutions (including Redshift)
Support ingestion, transformation, and modeling of large-scale, market-sensitive datasets
Collaborate with data scientists, quantitative analysts, and trading teams to optimize data access and usability
Ensure high availability, performance, and data integrity across systems
Implement best practices for data versioning, testing, and CI/CD automation

Required Skills & Experience

5+ years of professional experience as a Data Engineer or in a similar role
Expert-level proficiency in Python for building and automating data workflows
Strong experience with the AWS data ecosystem, including:
S3 (storage)
Lambda (serverless compute)
Glue (ETL orchestration)
Redshift (data warehousing)

Experience with Docker for containerized data processing

Hands-on experience with infrastructure-as-code tools (e.g., Terraform or AWS CloudFormation)

Solid understanding of SQL and performance tuning in data warehouses

Experience working with large, complex datasets in production environments Preferred Experience

Background in commodities, energy trading, or broader financial markets
Exposure to data orchestration tools like Apache Airflow
Experience with real-time or streaming data processing (e.g., Kafka, Kinesis)
Familiarity with monitoring and logging for data pipelines (e.g., CloudWatch, Grafana)
Understanding of data governance, lineage, and quality frameworks