Job Description
At EY, you’ll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we’re counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all.
TechOps - CloudOps – Data Platform & Engineering Support Engineer: Senior
At EY, we’re all in to shape your future with confidence.
We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go.
Join EY and help to build a better working world.
About Global Delivery Services
Global Delivery Services refers to EY's worldwide network of service delivery centers. The GDS team plays an important role in EY’s strategy by ensuring effective support to EY’s growth agenda.
Our journey started in 2002 with approximately 200 people. Today we stand at 80,000+ professionals in ten locations around the world. We operate in Argentina, China, Hungary, India, Philippines, Poland, Sri Lanka, Mexico, Spain and the United Kingdom.
Client service is focused on providing Consulting, Assurance, Tax, Strategy & Transactions, and Knowledge support to our clients around the world. The teams enable account teams worldwide to provide seamless, high-quality, value-added support, helping deliver exceptional client service.
Enablement Services provides cost-effective, high-skilled, and innovative services to support EY’s global and local enablement teams. Markets, BMC, AWS, Finance and Accounting, Risk Management, Procurement, People Shared Services, IT Service Delivery and IT Global Infrastructure services, are among the services offered by Enablement Services.
Our innovation specialists serve the GDS Client Service and Enablement Services teams, along with Service Lines, Core Business Services and Sectors. The team brings the desired environment, technologies and skilled teams together for facilitation, rapid prototyping and innovative thinking. The competencies offered include analytics, digital, user experience, mobile technology, infrastructure, Microsoft technologies and open innovation.
The opportunity
We are looking for a seasoned and proactive Senior Data Platform & Engineering Support Engineer to provide production and operational support for enterprise data platforms and engineering workloads, with a primary focus on Databricks, Python, PySpark, and ETL solutions. The role will combine hands-on troubleshooting of jobs, notebooks, clusters, pipelines, code, SQL, and data issues with Databricks platform administration, performance optimization, release support, and continuous service improvement. The engineer will act as an escalation point for complex incidents and collaborate with data engineering, cloud platform, security, database, DevOps, and business teams to maintain reliable, secure, and cost-effective data services.
Your key responsibilities
- Provide day-to-day production and operational support for Databricks workspaces, jobs, notebooks, workflows, Delta Live Tables or Lakeflow pipelines, clusters, SQL warehouses, libraries, repositories, and related integrations.
- Monitor scheduled and streaming workloads, identify failed, delayed, or degraded jobs, assess business impact, restore services, and coordinate recovery within agreed service levels.
- Troubleshoot Python and PySpark code failures, dependency conflicts, serialization issues, memory errors, data-skew problems, schema changes, malformed records, and source-to-target transformation defects.
- Investigate cluster start-up failures, driver and executor errors, autoscaling issues, pool and policy constraints, runtime incompatibilities, library installation failures, and workspace connectivity problems.
- Support ETL and ELT pipelines across ingestion, transformation, validation, orchestration, and publishing layers, including batch and streaming processing.
- Perform root cause analysis for recurring incidents and implement permanent corrective actions, preventive monitoring, code or configuration improvements, and operational controls.
- Optimize Databricks workload performance through Spark UI analysis, execution-plan review, partitioning, file sizing, caching, broadcast joins, adaptive query execution, data skipping, clustering, and appropriate compute sizing.
- Provide Databricks platform administration support covering workspace configuration, cluster policies, compute governance, runtime management, libraries, groups, permissions, service principals, secrets, tokens, and access troubleshooting.
- Support Unity Catalog administration, including catalogs, schemas, tables, volumes, privileges, external locations, storage credentials, lineage, and controlled access to governed data assets.
- Oversee production deployments and environment promotions for notebooks, jobs, pipelines, configurations, and infrastructure changes using Git and CI/CD practices, with validation and rollback planning.
- Create and maintain runbooks, standard operating procedures, monitoring checks, known-error records, troubleshooting guides, reconciliation controls, and knowledge articles.
- Collaborate with data engineers, platform administrators, DevOps teams, cloud infrastructure teams, database administrators, security teams, and business stakeholders to resolve issues and improve platform reliability.
- Participate in platform health checks, capacity and cost reviews, disaster recovery exercises, release activities, and rotational on-call support.
- Responsible for decision-making, optimizing processes, resource management, and overseeing team management as needed for task execution.
- Accountable for allocating personnel, supervising team members, assigning tasks, ensuring that the team has the necessary tools and support to succeed in their roles and optimizing and evaluating their performance to meet organizational goals.
Skills and attributes for success
- Strong hands-on experience with Databricks jobs, notebooks, workflows, pipelines, clusters, SQL warehouses, Spark UI, Delta Lake, and production troubleshooting.
- Strong Python and PySpark skills for code analysis, debugging, ETL development, data transformation, exception handling, logging, testing, and performance optimization.
- Solid understanding of Spark architecture, drivers, executors, stages, tasks, shuffles, partitions, caching, memory management, adaptive query execution, and common performance bottlenecks.
- Strong ETL and ELT knowledge, including ingestion patterns, orchestration, incremental processing, change data capture, data validation, error handling, restartability, and data reconciliation.
- Good SQL skills for data analysis, source-to-target validation, reconciliation, query tuning, and incident diagnosis.
- Working knowledge of Oracle and Microsoft SQL Server, including schemas, tables, views, stored procedures, indexes, privileges, connectivity, and execution plans.
- Practical knowledge of Databricks platform administration, workspace security, identity and access management, secrets, cluster policies, runtime upgrades, and compute governance.
- Ability to interpret Databricks, Spark, Python, application, database, operating-system, and cloud-service logs to isolate failures across multiple technology layers.
- Strong analytical and problem-solving skills with a disciplined approach to incident management, root cause analysis, and preventive improvement.
- Clear communication and stakeholder-management skills, including the ability to explain technical issues, business impact, recovery status, and corrective actions.
- Ability to mentor junior engineers, coordinate incident resolution, review technical solutions, and promote operational engineering standards.
To qualify for the role, you must have
- Bachelor’s degree in computer science, data engineering, information systems, engineering, or a related discipline, or equivalent professional experience.
- 5–8 years of experience in data engineering, data-platform support, ETL development, application management, or production operations.
- Strong hands-on experience supporting Databricks workloads in production, including jobs, notebooks, clusters, workflows, pipelines, and SQL workloads.
- Strong programming and troubleshooting experience with Python and PySpark.
- Experience supporting ETL or ELT pipelines, including job failures, dependencies, data-quality issues, reconciliation, reruns, and performance problems.
- Experience with Databricks platform administration, including workspace access, compute configuration, cluster policies, libraries, secrets, service principals, and Unity Catalog.
- Practical experience using SQL with Oracle, SQL Server, or comparable relational databases for investigation, validation, and performance analysis.
- Experience leading complex incident resolution, root cause analysis, problem management, change implementation, and production release support.
- Experience working with Git-based development, CI/CD pipelines, environment promotion, and controlled production deployments.
- Experience working with distributed teams and supporting business-critical data services under agreed SLAs.
- Willingness to participate in 24/7 support rotations or cover critical business hours when required.
Technologies and Tools
Must have
- Databricks: workspaces, notebooks, jobs, Workflows, Lakeflow or Delta Live Tables pipelines, all-purpose and job clusters, instance pools, cluster policies, Databricks SQL, SQL warehouses, Spark UI, Delta Lake, and production troubleshooting.
- Databricks platform administration: users, groups, service principals, entitlements, permissions, secrets, tokens, libraries, runtime management, compute governance, workspace configuration, and access troubleshooting.
- Unity Catalog: catalogs, schemas, tables, views, volumes, privileges, external locations, storage credentials, lineage, and governed data access.
- Python and PySpark for ETL development, code troubleshooting, logging, testing, dependency management, exception handling, and performance optimization.
- ETL and ELT engineering covering batch and streaming ingestion, transformations, orchestration, incremental processing, data validation, restartability, and reconciliation.
- Knowledge of agentic AI concepts and AI-assisted operational workflows, including enterprise AI agents for job-failure triage, log analysis, Spark diagnostics, data-quality investigation, reconciliation, runbook guidance, and knowledge retrieval.
- Oracle, Microsoft SQL Server, and strong SQL skills for data validation, query analysis, reconciliation, connectivity troubleshooting, and incident diagnosis.
- Git and CI/CD practices for notebook, code, job, pipeline, and configuration deployment across controlled environments.
Good to have
- Exposure to Databricks Mosaic AI, Agent Framework, Model Serving, MLflow, Microsoft Copilot Studio, Microsoft Foundry/Azure OpenAI, LangChain, LangGraph, or equivalent AI and agent orchestration technologies.
- Awareness of responsible AI practices, including human oversight, secure data access, privacy, evaluation, traceability, auditability, and protection against unsafe automated operational actions.
- Experience with cloud services in Microsoft Azure or AWS, such as storage, networking, identity, key management, monitoring, and private connectivity used by Databricks.
- Experience with infrastructure as code and automation tools such as Terraform, Databricks Asset Bundles, REST APIs, CLI tools, PowerShell, or shell scripting.
- Knowledge of data observability, data quality, metadata, lineage, cost management, FinOps, or enterprise monitoring tools.
- Relevant certifications such as Databricks Certified Data Engineer Associate or Professional, Databricks Certified Associate Developer for Apache Spark, Azure Data Engineer Associate, AWS data engineering certifications, Oracle, or Microsoft SQL Server certifications.
What we look for
- Senior support engineers who combine strong Databricks and data-engineering expertise with disciplined production operations and end-to-end ownership.
- Problem solvers who can analyze Spark, Python, ETL, SQL, database, security, and infrastructure issues methodically and drive sustainable resolution.
- Technically credible team members who can mentor others, coordinate complex incidents, and improve engineering and operational standards.
- Collaborative professionals who communicate clearly with data engineers, platform teams, database administrators, security teams, and business stakeholders.
- Continuous learners who are interested in secure, responsible AI-assisted and agentic approaches that improve data-platform operations.
What we offer
At EY, we’ll develop you with future-focused skills and equip you with world-class experiences. We’ll empower you in a flexible environment, and fuel you and your extraordinary talents in a diverse and inclusive culture of globally connected teams. Learn more.
Are you ready to shape your future with confidence? Apply today.
To help create an equitable and inclusive experience during the recruitment process, please inform us as soon as possible about any disability-related adjustments or accommodation you may need.
Our Commitment: As a commitment, we persistently endeavour to embody our values, fulfil our purpose, and champion inclusiveness. Our dedication is to cultivate EY into an environment where diverse perspectives are celebrated, creating a supportive atmosphere for individuals to authentically be themselves and contribute their utmost.
Professional Development: From entry-level employees to senior leaders, we believe in continuous learning. We offer opportunities to build new skills, take on leadership roles, and connect and grow through mentorship.
People and Culture: In our dynamic workplace, diversity, equity, and inclusiveness are ingrained in our culture. We're united by a commitment to create an environment where every individual's differences are valued, practices are equitable, fostering a sense of belonging. Our shared values include integrity, respect, teaming, inclusiveness, energy, enthusiasm, courage to lead, and building relationships based on doing the right thing.
Benefits: Embark on a transformative career journey with us and indulge in a suite of premium benefits, encompassing exclusive health and wellness packages, enticing rewards, and cutting-edge learning opportunities that empower you to continually grow and excel in your professional and personal development.
How to Apply: If you are passionate to join us and are aligned with our commitment to building a better working world, we invite you to apply by completing our user-friendly form with personal and professional information and by providing your consent to data privacy.
Successful candidates advance to a competency-based interview. If mutual interest persists, a job offer awaits.
“We are an equal opportunity employer and are committed to Diversity, Equity & Inclusion”.
EY | Building a better working world
EY exists to build a better working world, helping to create long-term value for clients, people and society and build trust in the capital markets.
Enabled by data and technology, diverse EY teams in over 150 countries provide trust through assurance and help clients grow, transform and operate.
Working across assurance, consulting, law, strategy, tax and transactions, EY teams ask better questions to find new answers for the complex issues facing our world today.
More Information
Application Details
- Organization DetailsErnst & Young


Recommended Comments
There are no comments to display.
Join the conversation
You are posting as a guest. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.