Job Description
Principal Engineer -Data Platforms (Enterprise Data Platforms, Lakehouse, Multi-Tenant Architectures) Core Role Expectation This role is a hands-on Principal Engineer responsible for architecting, engineering, and enablement of large-scale, multi-tenant enterprise data platforms. The ideal candidate has deep experience building enterprise data lakes, lakehouses, or data warehouses that support data analytics, data engineering, AI/ML, and regulatory workloads across hundreds or thousands of users. This isnot an application or pipeline-only role. The focus is on platform architecture, scalability, security, and operational excellence for shared enterprise data platforms. Required Technical & Leadership Skillset Enterprise Data Platform Architecture & Engineering Extensive experience designing, engineering, and operating enterprise-scale data platforms, including data lakes, lakehouses, or data warehouses Proven experience leading large, multi-tenant data platforms serving multiple lines of business with strict isolation, governance, and performance controls Deep understanding of data platform reference architectures, including: Lakehouse patterns (compute/storage separation, open table formats) Shared services vs. tenant-owned workloads Platform-as-a-product operating models Demonstrated ownership of end-to-end platform lifecycle: architecture, build, migration, operations, and modernization Multi-Tenancy, Scale & Performance Hands-on experience designing and enforcing multi-tenant isolation. Expertise in capacity planning, workload isolation, quota management, and performance optimization at enterprise scale Experience supporting mixed workloads (batch, interactive SQL, streaming, ML/AI) on shared platforms Data Platform Technologies (Hands-On) Strong hands-on expertise with modern data platform ecosystems, such as: Compute & Processing: Spark (including Spark at scale), distributed processing frameworks Query & Analytics: Trino/Presto or similar distributed SQL engines Table Formats & Storage: Iceberg (or similar), Iceberg Rest Catalogue, object storageand enterprise storage platforms Metadata, Catalog & Governance: DataHub, Apache Atlas, Hive Metastore, or equivalent Experience designing and operating production-grade data services, not just proof-of-concepts Platform Engineering & Automation Strong background in platform engineering principles applied to data platforms: Infrastructure as Code (Terraform or equivalent) Automated environment provisioning and repeatability GitOps or declarative deployment models Experience standardizing and industrializing data platforms to support self-service consumption at scale Security, Governance & Compliance Demonstrated experience building secure-by-design data platforms in regulated environments Hands-on knowledge of: Authentication and authorization models (enterprise IAM integration) Fine-grained access controls and data entitlements Auditability, lineage, and compliance controls Proven ability to partner with Security, Risk, Compliance, and Audit teams to meet regulatory requirements (e.g., SOX, PCI, data privacy) Technical Leadership & Influence Recognized technical leader capable of: Setting data platform strategy and standards across the enterprise Making architecture decisions that balance scalability, cost, risk, and time-to-market Mentoring senior engineers and influencing platform adoption across teams Experience leading complex platform migrations or modernizations (e.g., legacy data platforms to modern lakehouse architectures) Data Platform Components (Platform Enablement) You provide leadership for the platform that runs these technologies, not the pipelines or applications built on them: Compute: Spark on K8s, Kyuubi, JupyterHub Query/Analytics: Trino, Superset Orchestration: Airflow on Kubernetes Catalog/Governance: Gravitino, DataHub, Ranger Storage: Iceberg, S3/NetApp, PostgreSQL Messaging/Search: Kafka, OpenSearch Required Qualifications 7+ years of Engineering experience, or equivalent demonstrated through one or a combination of the following: work experience, training, military experience, education 5+ years of hands-on experience with Kubernetes in production environments (OpenShift Container Platform strongly preferred)