Senior Data Engineer
Job Description
Number of Positions: 1 Duration: 12 months initial contract
Must be eligible to work in Canada
Hybrid position, 2-3d/w onsite in Toronto mandatory, 7hr/d billing, no exceptions
The Senior Data Engineer needs to have strong cloud technologies and a minimum of 5 years of hands‑on Databricks experience. As well, this individual needs to have strong ETL background. The resource will be working on a major data platform migration and transformation initiative.
Key Responsibilities
- Design and implement robust batch and streaming ingestion pipelines on Databricks using Apache Spark (PySpark/Scala).
- Utilize advanced Databricks features like Delta Live Tables (DLT) or Workflows, and Auto Loader for scalable file ingestion.
- Model and manage datasets with next‑generation open table formats, preferably Apache Iceberg, registered in Unity Catalog.
- Implement end‑to‑end data governance, lineage, and access controls using Unity Catalog.
- Build robust, observable data pipelines, including implementing monitoring, alerting, and logging mechanisms.
- Take a leadership role in defining Software Engineering best practices for code standardization, branching strategy, optimization, and design patterns.
- Collaborate with Solution and Enterprise Architects to design complex data transformation solutions.
- Optimize cluster performance and cost using appropriate cluster types and tuning Spark jobs.
Must have skills/experience/qualifications
- At least 10 years of strong experience in Data Integration and ETL pipeline development.
- Minimum 5 years of hands‑on experience building data pipelines with Databricks and Apache Spark (PySpark/Scala) in cloud environments.
- Strong understanding of open table formats, with a preference for Apache Iceberg (snapshots, time travel).
- Experience with data governance using Unity Catalog.
- Proficiency in orchestration/scheduling and familiarity with Delta Live Tables or similar frameworks.
- Solid SQL and data modeling skills.
- Experience with streaming ingestion (Spark Structured Streaming) and version control (Git) in a CI/CD environment.
- Familiarity with cloud object storage (AWS S3 / Azure Data Lake / GCS).
Seniority level
Mid‑Senior level
Employment type
Contract
Job function
Information Technology
Industries
IT Services and IT Consulting, Insurance, and Financial Services
Toronto, Ontario, Canada CA$80,000.00-CA$130,000.00
#J-18808-Ljbffr
How to Apply
Ready to start your career as a Senior Data Engineer at Randstad Digital?
- Click the "Apply Now" button below.
- Review the safety warning in the modal.
- You will be redirected to the employer's official portal to complete your application.
- Ensure your resume and cover letter are tailored to the job description using our AI tools.
Frequently Asked Questions
Who is hiring?▼
This role is with Randstad Digital in Toronto.
Is this a remote position?▼
This appears to be an on-site role in Toronto.
What is the hiring process?▼
After you click "Apply Now", you will be redirected to the employer's official site to submit your resume. You can typically expect to hear back within 1-2 weeks if shortlisted.
How can I improve my application?▼
Tailor your resume to the specific job description. You can use our free Resume Analyzer to see how well you match the requirements.
What skills are needed?▼
Refer to the "Job Description" section above for a detailed list of required and preferred qualifications.