# Job Description – Test Engineer, Cloud & Data Platform (Lakehouse)
Banking Domain
## Purpose of the Role
To design, develop, and execute testing strategies that validate the quality, reliability, security, and compliance of a large-scale, multi-platform cloud data lakehouse on AWS. The Test Engineer ensures correctness across infrastructure-as-code, CI/CD pipelines, data workflows, and integrated data platforms, embedding a shift-left quality culture aligned with SRE principles.
---
## Key Accountabilities
### Infrastructure & IaC Testing
- Design and implement **Terraform unit tests** to validate resource naming conventions, mandatory tagging, encryption controls, and conditional resource configurations.
- Execute infrastructure validation checks (format, syntax, plan review, drift detection) within CI/CD pipelines.
- Validate **CloudFormation templates** for correctness, parameter validation, and multi-account deployment patterns.
- Verify backend state configuration (S3 + DynamoDB locking) across multiple Terraform repositories.
### CI/CD Pipeline Testing
- Test **GitLab CI/CD pipelines** for multi-environment, multi-region matrix deployments including stage sequencing, artifact generation, and concurrency controls.
- Validate configuration conversion pipelines (HOCON → JSON tfvars) for schema correctness and type safety.
- Test environment promotion gates (feature → dev → sit → prod) and manual approval workflows.
### Security & Compliance Testing
- Run **SAST security scans** (Checkov) across all Terraform and CloudFormation code to identify and triage findings against regulatory policy.
- Validate encryption-at-rest (KMS) and in-transit (TLS) controls across all provisioned services.
- Test IAM least-privilege policies, cross-account role assumptions, and fine-grained data access controls (Lake Formation, Immuta).
- Validate regional data residency compliance (EU and India) and mandatory resource tagging.
### Data Pipeline & Quality Testing
- Design end-to-end integration tests across a **three-tier lakehouse architecture** (landing zone, curated zone, analytics zone).
- Implement **DBT test strategies**: schema tests, data assertion tests, freshness tests, and documentation validation.
- Test AWS Glue ETL jobs for transformation correctness, incremental processing, data quality rules, and error handling.
- Validate **Snowflake** multi-tenant configurations, warehouse behaviour, storage integrations, and RBAC entitlements.
- Test event-driven orchestration workflows (EventBridge, Airflow/Astronomer, Step Functions) for correctness and failure handling.
### SRE-Aligned Reliability & Performance Testing
- Define and validate **SLIs, SLOs, and SLAs** for data pipelines (success rates, latency, data freshness).
- Execute **chaos engineering and failure injection tests** to validate auto-remediation, retry logic, and recovery behaviour.
- Perform **load and performance testing** on S3, Glue, Snowflake, Athena, and Databricks workloads.
- Validate CloudWatch monitoring configurations — alarms, dashboards, log queries — for accuracy and alerting correctness.
- Support incident response testing through runbook validation and tabletop exercises.
### Collaboration & Process Improvement
- Partner with platform engineers, SREs, data engineers, and security teams to embed testing at every stage of the development lifecycle.
- Contribute to code and pipeline reviews to ensure testability is a first-class design concern.
- Lead root cause analysis on defects and support rapid resolution in collaboration with engineering teams.
- Promote shift-left quality practices and test-driven IaC development across the programme.
---
## AVP Expectations
- Advise and influence decision-making; contribute to policy development and operational governance.
- Lead or contribute to cross-functional engineering assignments; guide team members through complex testing challenges.
- Consult on complex quality and risk issues; identify mitigations and support control improvements.
- Communicate complex technical information clearly to both technical and non-technical stakeholders.
- Demonstrate the Barclays Values (Respect, Integrity, Service, Excellence, Stewardship) and Mindset (Empower, Challenge, Drive).
---
## Core Technical Skills
| Area | Level |
|------|-------|
| Terraform (IaC testing, unit tests, state management) | Advanced |
| GitLab CI/CD (pipeline testing, matrix jobs, artefacts) | Advanced |
| AWS data services (S3, IAM, Glue, Lake Formation, EventBridge, CloudWatch, Lambda) | Intermediate–Advanced |
| Security scanning tools (Checkov, tfsec) | Intermediate |
| Data platforms (Snowflake, Databricks, DBT, Apache Iceberg) | Intermediate |
| Python (test automation, pipeline utilities) | Intermediate |
| SRE practices (SLIs/SLOs, chaos engineering, performance testing) | Intermediate |
| Compliance testing (encryption, tagging, data residency) | Intermediate |
---