Lead Data Engineer Job at Purple Drive, Seattle, WA

bS9VK0NmQ0wwNGpTVG1zaFRENFJkUk9LTVE9PQ==
  • Purple Drive
  • Seattle, WA

Job Description

Role: Lead Data Engineer
Location: Seattle, WA
Duration: 6 months

Responsibilities:

• Design, build, and deploy data extraction, transformation, and loading processes and pipelines from various sources including databases, APIs, and data files.
• Develop and support data pipelines within a Cloud Data Platform, such as Databricks
• Build data models that reflect domain expertise, meet current business needs, and will remain flexible as strategy evolves
• Monitor and optimize Databricks cluster performance, ensuring cost-effective scaling and resource utilization
• Demonstrates ability to communicate technical concepts to non-technical audiences both in written and verbal form.
• Demonstrates strong understanding with coding and programming concepts to build data pipelines (e.g. data transformation, data quality, data integration, etc.).
• Demonstrates strong understanding of database storage concepts (data lake, relational databases, NoSQL, Graph, data warehousing).
• Implement and maintain Delta Lake for optimized data storage, ensuring data reliability, performance, and versioning
• Automate CI/CD pipelines for data workflows using Azure DevOps
• Collaborate with cross-functional teams to support data governance using Databricks Unity Catalog

Qualifications:

• 7+ years of experience in data engineering or a related field
• Expertise with programming languages such as Python/PySpark, SQL, or Scala
• Experience working in a cloud environment (Azure preferred) with strong understanding of cloud data architecture
• Hands-on experience with Databricks Cloud Data Platforms Required
• Experience with workflow orchestration (e.g., Databricks Jobs, or Azure Data Factory pipelines) Required

Job Tags

Flexible hours,

Similar Jobs

Caliber Recruiting Group

Assurance Manager- Top Rated CPA Firm (Regional) Job at Caliber Recruiting Group

 ...you have had it with your "Big 4" stress or just need a change, this could be a great opportunity. Top ranked regional mid-sized CPA firm, is growing and needing an Audit amp; Assurance Manager for performing overall management of audit and assurance assignments.... 

Viking Moving & Storage Services

Tow Truck Driver Job at Viking Moving & Storage Services

 ...Tow Truck Operator / Mover - Non CDL 55 Domino Dr, Concord, MA 01742, USA Full-Time About the Job: Viking On Demand Storage & Moving is looking for a Non CDL driver / mover to operator our medium duty tow-truck and other trucks, assist with household moving... 

Department of Public Health

Program Policy Analyst Job at Department of Public Health

 ...will be the responsibility of the selected candidate. The headquarters office location for this position is identified in the attached duty statement. Special Requirements For experience/education to qualify during the application screening process, and... 

Muffetta Household Staffing Agency, Inc.

Executive Housekeeper Job at Muffetta Household Staffing Agency, Inc.

 ...for homes during travel (plants, mail, security, temperature) Maintain Bentleys routine and transport between homes as needed. Execute travel readiness and return checklists. Errands & Personal Support: Handle errands efficiently and proactively. Manage... 

UPMC Hamot

Respiratory Therapist, CRT Job at UPMC Hamot

Join Our Team as a Respiratory Therapist! Are you a dedicated and compassionate Respiratory Therapist looking to make a meaningful impact in patient care? UPMC Hamot is seeking a skilled professional to join our dynamic team. In this role, youll work with cutting-edge...