Data Engineer - Remote (US) - Full Time Only
Role: Data Engineer Location: Remote (Candidate should be comfortable to follow PST Time Zone) Job Type: Full Time Note: Active LinkedIn ID is a must and match with the resume Must Have Skills: Azure Databricks, Apache Spark, Snowflake, Data Governance, Unity Catalog, dBT, Microsoft Purview, Kafka, SQL, Python Experience and Qualifications: Bachelor's or Master's degree in Computer Science, Engineering, or a related field. 10+ years of experience in data engineering / data architecture, with deep hands-on expertise in Microsoft Azure.
- Expert-level experience with Azure Databricks and Apache Spark(PySpark, Spark SQL)
- Delta Lake
- Unity Catalog
- Job orchestration and performance tuning
- Strong experience with Snowflake on Azure, including:
- Schema and data model design
- Performance and cost optimization
- RBAC, Streams, and Tasks
- DBT (Core or Cloud) expertise:
- Project structure and best practices
- Tests, exposures, macros
- Strong fundamentals in SQL and Python
- Proven experience building ETL/ELT pipelines using:
- Azure Data Factory (ADF)
- Azure Synapse / Microsoft Fabric pipelines
- Familiarity with streaming platforms such as Event Hubs and/or Kafka
- Solid understanding of:
- Lakehouse and data warehousing architectures
- Dimensional modeling
- Medallion architecture patterns
- Data quality frameworks
- Strong knowledge of security and governance, including:
- Microsoft Purview (catalog, lineage)
- Data masking and PII handling
- Managed identities
- Private networking
- Excellent communication and documentation skills, with the ability to:
- Create architecture and design documents
- Present to both technical and business stakeholders
- Consulting / agency experience, with the ability to lead multiple concurrent projects
- Willingness to travel as required
Nice to Have:
- Experience with Azure ML, feature stores, or serving ML pipelines from Databricks or Snowflake
- Infrastructure as Code (IaC) using Terraform or Bicep
- Containerization experience with Docker
- Observability and data quality tooling, such as:
- Great Expectations / Delta Expectations
- Databricks Quality Flows
- Monte Carlo
- Datadog
Preferred Certifications:
- Databricks Certified Data Engineer Professional or Databricks Architect
- Microsoft Azure:
- DP-203 Azure Data Engineer
- Azure Solutions Architect Expert
- Azure Security Engineer Associate
- Snowflake:
- SnowPro Core
- SnowPro Advanced
Job Description / Responsibilities:
- Partner with client stakeholders to translate business objectives into scalable Azure data architectures and delivery roadmaps
- Architect and implement lakehouse and data warehouse solutions using:
- Azure Databricks (Delta Lake, Unity Catalog)
- Databricks Bundle
- Snowflake on Azure
- dbt (Core / Cloud)
- ADLS Gen2
- Medallion architecture patterns
- Design and build robust ingestion and transformation pipelines using:
- Azure Data Factory (ADF)
- Microsoft Fabric / Synapse pipelines
- Orchestrate ELT workflows using:
- Databricks Jobs
- Delta Live Tables
- dbt models
- Establish best practices for performance and cost optimization, including:
- Cluster sizing and autoscaling
- Photon and SQL Warehouses
- Snowflake virtual warehouses
- Caching, file layout optimization, Z-ordering
- Drive observability and monitoring using:
- Azure Monitor
- Log Analytics
- Databricks metrics
- Implement robust data governance and security using:
- Azure Authentication
- Unity Catalog
- RBAC / ABAC
- Managed identities
- Private Endpoints and VNet injection
- Azure Key Vault backed secrets
- Lead code and design reviews, setting standards for:
- PySpark, SQL, and dbt
- Unit and integration testing
- Data quality checks (expectations and constraints)
- Implement CI/CD pipelines using Azure DevOps or GitHub Actions
- Guide multi-domain programs (Healthcare, Retail, BFSI, or similar)
- Mentor data engineers and ensure high-quality, production-ready deliverables
- Evangelize solutions through:
- Clear documentation
- Reference architectures
- Executive and stakeholder presentations
Remote About the Company: Saransh Inc Apply tot his job Apply To this Job