Altis Technology
One of our clients is looking to hire a Senior ETL PySpark Developer to develop a scalable platform to deliver public and private reporting products, leveraging a modern technology stack, industry standards, and Agile methodologies. This platform will integrate seamlessly with the client’s public website.
Responsibilities:
The Senior Consultant/Developer will be responsible for developing, operating, and maintaining assigned products. This includes testing, preparing deployment packages, deploying code and script changes, supporting user tests and inquiries, and handling application administration tasks required for product maintenance.
Key Deliverables:
- Gain a deep understanding of the systems and business processes related to assigned products.
- Collaborate with analytical teams to develop, maintain, and support products using Python, R, and Spark.
- Ensure high-quality, clean code and participate in code reviews.
- Analyze software deficiencies, troubleshoot issues, and implement fixes.
- Triage user-reported issues and escalate to relevant administrators as needed.
- Prepare documentation, reports, and presentations related to project activities.
- Share expertise across the organization, providing training and knowledge transfer on advanced analytics, machine learning methods, and data visualization.
- Perform operational tasks such as running scripts, updating configuration files, and supporting scheduled releases.
- Execute maintenance tasks, including fiscal year rollovers and other updates.
- Develop code changes per approved product update requirements.
- Conduct thorough testing of changes and maintain test scenarios.
- Deploy updates to test environments and support QA teams and user testing.
- Provide ongoing user support and resolve inquiries efficiently.
- Communicate effectively with business users and technical teams to ensure smooth product maintenance.
- Handle application administration tasks as required.
- Prepare deployment packages and manage release tickets.
- Coordinate with relevant teams to ensure successful deployment.
- Create and update technical documentation, including ETL mappings and product design specifications, as per client IT team guidance.
Required Skills & Experience:
- Extensive hands-on experience in developing PySpark data pipelines.
- Proficiency with AWS EMR clusters.
- Strong expertise in shell scripting.
- Experience with Apache Airflow for workflow automation.
- Proven ability to manage multiple deliverables while ensuring quality and timeliness.
- Ability to handle a high workload efficiently.
- Flexibility to take on various tasks, including operations, development, and maintenance.
- Strong troubleshooting and problem-solving skills.
- Excellent communication and interpersonal skills, with the ability to convey technical concepts to both business users and technical peers.
- Ability to collaborate effectively with cross-functional teams.
The role can be hybrid in Toronto or Ottawa
