Senior Data Engineering & Analytics Lead

The Role

We are recruiting a Senior Data Engineering & Analytics Lead to architect, build, and govern the Trust’s enterprise data infrastructure, analytics platforms, and data-driven intelligence capability. This is a senior technical leadership and delivery role responsible for the full data engineering lifecycle — from data ingestion and pipeline engineering through to analytical modelling, reporting, and advanced analytics. You will serve as the Trust’s principal technical authority on all matters relating to data architecture, data platform strategy, and analytics delivery. You will lead the design and implementation of scalable, reliable, and governed data pipelines, data warehouses, and analytics environments, while building deep partnerships with executive leadership and business units to align the data capability with organisational strategy. This role demands both hands-on engineering depth and the strategic thinking required to manage a complex, multi-source data environment at organisational scale.

Key Responsibilities

Data Architecture & Platform Strategy

• Define and own the Trust’s enterprise data architecture — covering data ingestion patterns, integration standards, platform topology, and system-of-record designation across all data domains.

• Lead the development of a multi-year data platform roadmap, aligned to the Trust’s strategic objectives, digital transformation programme, and organisational data maturity goals.

• Design and implement the Trust’s data lakehouse or warehouse architecture — selecting appropriate platforms (Snowflake, Databricks, BigQuery, or equivalent) and defining layer strategies (raw, curated, consumption).

• Establish data architecture standards: naming conventions, data modelling guidelines, schema design patterns, storage formats, and partitioning strategies across all governed data assets.

• Evaluate and recommend data platform tooling, services, and vendors — producing evidence-based recommendations with TCO analysis, capability assessment, and implementation roadmaps.

• Design and govern data integration patterns: batch ETL/ELT, real-time streaming, CDC (Change Data Capture), API-based ingestion, and event-driven data flows.

• Conduct enterprise data landscape assessments — mapping source systems, data flows, integration dependencies, and quality risks across the organisation.

Data Engineering & Pipeline Development

• Design, build, and operate production-grade data pipelines: automated ingestion, transformation, validation, and loading workflows — ensuring reliability, observability, and SLA compliance.

• Lead the implementation of ELT/ETL frameworks using dbt, Apache Spark, Apache Airflow, Prefect, or equivalent — with modular, testable, and version-controlled transformation logic.

• Develop and maintain real-time and near-real-time streaming data pipelines using Apache Kafka, AWS Kinesis, Google Pub/Sub, or equivalent — including schema registry management and consumer group governance.

• Implement robust data pipeline monitoring: data freshness alerting, pipeline failure detection, data volume anomaly detection, and automated remediation triggers.

• Manage data pipeline CI/CD: automated testing frameworks for data pipelines, version control strategies, environment promotion workflows, and deployment automation.

• Build and maintain data quality frameworks: automated profiling, completeness and consistency checks, referential integrity validation, and quality score dashboards.

• Design and govern the organisation’s data catalogue, data lineage, and metadata management practices using enterprise tooling (DataHub, OpenMetadata, Alation, or equivalent).

Analytics, Reporting & Business Intelligence

• Architect and govern the enterprise analytics layer — including semantic models, dimensional data models, data marts, and self-service analytics environments.

• Lead the design and delivery of executive dashboards, operational scorecards, and KPI frameworks that provide decision-makers with accurate, timely, and trusted intelligence.

• Design dimensional data models using Kimball or Inmon methodology: star and snowflake schemas, slowly changing dimensions (SCD Types 1–6), and aggregate design patterns.

• Drive adoption of advanced analytics: cohort analysis, predictive modelling, scenario planning, statistical process control, and time-series analysis for operational performance monitoring.

• Define and enforce analytics governance standards: metric definitions, report certification processes, version control of analytical assets, and change management for reporting outputs.

• Establish data literacy programmes and self-service BI enablement strategies that empower business units to produce and consume analytics independently.

• Lead the integration of machine learning outputs and AI-generated insights into the analytics platform — ensuring traceability, explainability, and business consumption readiness.

Data Governance & Compliance

• Develop and own the organisation’s data governance framework: policies, standards, data stewardship roles, accountability structures, and compliance mechanisms.

• Implement and manage enterprise data quality programmes — profiling, cleansing, monitoring, and remediation across all governed data domains.

• Ensure compliance with the Privacy Act 2020, GDPR where applicable, and sector-specific data obligations — advising business units on data handling, classification, and retention requirements.

• Chair or lead the Data Governance Committee, coordinating with data owners, stewards, and IT to resolve data quality issues and enforce governance policies.

• Define information lifecycle management policies: data classification, retention schedules, archival strategies, and secure disposal frameworks.

• Maintain comprehensive data catalogues, data dictionaries, business glossaries, and lineage documentation ensuring all data assets are discoverable, trusted, and well-documented.

• Govern data access controls: role-based access, row-level security, column-level masking, and data sharing agreements with internal and external stakeholders.

Stakeholder Engagement & Technical Leadership

• Serve as the principal data engineering and analytics advisor to the CIO, CEO, and executive leadership — translating complex technical concepts into clear business implications.

• Partner with all business units to understand their data and analytics needs, translate requirements into platform solutions, and ensure ongoing value delivery.

• Lead technical design processes, architecture review boards, and data platform governance forums.

• Mentor and grow data engineering and analytics team members through structured guidance, code review, design feedback, and professional development support.

• Manage vendor relationships with data platform providers, analytics tool vendors, and data consultancies.

• Produce executive-level reporting on data platform health, pipeline reliability, governance compliance, and analytics programme outcomes.

• Lead data engineering discovery and scoping for major initiatives, producing solution architectures executable by the broader team.

Key Performance Indicators

• Data pipeline reliability — SLA compliance across all production pipelines, measured by data freshness, completeness, and on-time delivery.

• Data quality scores across all governed domains — profiled, measured, and trended over time with continuous improvement targets.

• Platform adoption — self-service analytics usage, active data consumers, and reduction in ad hoc data extraction requests.

• Data governance maturity — assessed against DAMA DMBOK or equivalent framework with quarterly improvement milestones.

• Pipeline incident rate and MTTR — tracked quarterly with measurable improvement targets.

• Stakeholder satisfaction with data and analytics services — measured by periodic survey of data consumers.

• IaC and automation coverage — target 100% of data infrastructure managed through version-controlled code.

About You — Technical Expertise

Data Engineering & Pipeline Tools

• Expert-level SQL across multiple platforms (Snowflake, BigQuery, Redshift, PostgreSQL, or equivalent) — complex transformations, query optimisation, execution plan analysis, and stored procedure development.

• Advanced Python for data engineering: PySpark, pandas, data pipeline scripting, custom operator development, and production-quality ETL/ELT pipeline engineering.

• Expert-level dbt (data build tool): modular model design, testing frameworks, documentation, incremental materialisation strategies, and dbt Cloud/Core governance.

• Workflow orchestration platforms: Apache Airflow, Prefect, Dagster, or equivalent — DAG design, dependency management, dynamic task generation, and operational monitoring.

• Stream processing: Apache Kafka, Spark Streaming, Flink, or AWS Kinesis — consumer group management, schema registry (Confluent), exactly-once semantics, and real-time pipeline design.

• Data quality and observability tooling: Great Expectations, Soda, Monte Carlo, or equivalent — automated validation, anomaly detection, and data reliability engineering.

Data Platforms & Cloud

• Cloud data warehouse/lakehouse platforms: Snowflake, Google BigQuery, AWS Redshift, Databricks, or Azure Synapse Analytics — at production depth including performance tuning, cost optimisation, and security configuration.

• Cloud platform experience in AWS, Azure, or GCP — with specific depth in managed data services: S3/GCS/ADLS, Glue/Dataflow/Data Factory, RDS/Cloud SQL, and serverless data processing.

• Infrastructure as Code for data infrastructure: Terraform or Pulumi — managing data platform resources, permissions, network configuration, and environment promotion.

• Containerised data workloads: Docker and Kubernetes — running Airflow, Spark, or dbt in containerised environments with appropriate resource management.

• Data lakehouse architecture: Delta Lake, Apache Iceberg, or Apache Hudi — ACID transactions, time travel, schema evolution, and upsert/merge patterns.

Analytics & BI Tools

• Expert-level proficiency in Power BI, Tableau, Looker, or equivalent — including advanced DAX/LookML, calculated fields, row-level security, and enterprise semantic layer design.

• Dimensional modelling expertise: Kimball methodology, star and snowflake schemas, slowly changing dimensions (SCD Types 1–6), and aggregate design patterns.

• Python or R for statistical analysis, exploratory data analysis, predictive modelling integration, and automated reporting workflows.

• Data catalogue and lineage tooling: DataHub, OpenMetadata, Alation, or equivalent — metadata management, stewardship workflows, and lineage documentation.

Qualifications & Experience

• Bachelor’s degree or higher in Computer Science, Data Engineering, Information Systems, Mathematics, or a closely related technical discipline. Postgraduate qualification strongly preferred at this level.

• 8+ years of progressive experience in data engineering, data architecture, or analytics roles, with a minimum of 4 years at senior or lead level.

• Demonstrated experience designing and delivering enterprise-scale data platforms, data warehouses, or data lakehouse architectures in a production environment.

• Proven track record leading data governance frameworks, data quality programmes, or data strategy initiatives in a complex organisation.

• Professional certifications required: Databricks Certified Data Engineer Professional, dbt Certified Developer, Google Professional Data Engineer, AWS Data Analytics Specialty, or SnowPro Advanced: Data Engineer. CDMP (Certified Data Management Professional) highly regarded.

• Additional certifications advantageous: Databricks Certified Associate Developer for Apache Spark, Google Professional Cloud Architect, CKA, or equivalent.

• Demonstrated experience managing cross-functional data programmes and presenting data strategy and outcomes to executive and Board audiences.

• Experience in regulated or Treaty-based organisations, government agencies, or iwi/Māori entities is highly advantageous.

• 5–10+ years of senior experience in data engineering and analytics. Experience working with Māori organisations, kaupapa Māori data frameworks, or indigenous data sovereignty principles is a significant advantage.

What We Offer

• Opportunity to define and lead the Trust’s data engineering and analytics strategy.

• Annual professional development, certification, and conference budget.

• Access to modern cloud data platform tooling and infrastructure.

• Flexible and hybrid working arrangements.

• A mission-driven, professionally supported working environment.

How to Apply

Please submit your updated CV, cover letter, contact details of two referees (one professional and one personal), and a copy of your New Zealand visa (if applicable) to the designated application email: trusthumanresource@tiakitaongatrust.co.nz