Overview
Get AI-powered advice on this job and more exclusive features.
About Elsewhen
Elsewhen, a London-based consultancy, designs and builds technology solutions for clients like Spotify, Google, Inmarsat, and Zego. Over the past decade, we have built a workplace prioritising impact, drive, and friendliness. We value outcomes over hours and agility over rigid processes.
Work Environment
* Remote-first setup: Fully remote work with the option to use a WeWork membership for those who prefer occasional office access.
* Join our team: https://www.elsewhen.com.
Role
We are building an AI-powered knowledge platform that lets users ask natural language questions and get structured answers from complex data graphs. Your job is to make sure it works, is accurate and can be trusted. You’ll be responsible for validating how we interpret user intent, how we translate that into graph queries, and how we return meaningful, correct results. This is not a "click around and raise bugs" role. You'll be hands-on with data, APIs, automation, and AI behaviour. You'll work closely with engineers, data, AI/LLM specialists, and product management to define what "correct" looks like, then prove it at speed and at scale.
What You’ll Do
* Test natural language-to-graph behaviour
* Verify that a user's question is translated into the correct graph query
* Check that the answer returned is consistent, relevant, and complete
* Challenge ambiguity and edge cases in user intent
* Validate data accuracy
* Confirm that entities, relationships, and metadata in the graph are correct
* Spot broken or missing links in the knowledge graph
* Raise and track data quality issues with engineering / data teams
* Build and maintain automated tests
* Design and implement automated test suites for UI, API, and graph query flows
* Use Python (Pytest) / Selenium or similar to cover high-value user journeys
* Contribute to a shared automation framework and improve test reliability over time
* Test APIs and system integration points
* Confirm inputs, outputs, and error handling across internal services
* Validate permissions, filtering, ranking, and response format
* Support regression testing for new releases
* Performance and stability checks
* Run lightweight performance / responsiveness checks on core user flows
* Flag bottlenecks, flakiness, or instability early
* Contribute to quality strategy
* Help define acceptance criteria for new features before they're built
* Bring a point of view on testability, data requirements, and risk
* Work with engineers to prevent issues instead of only detecting them
What You Bring
Must-Have
* Solid QA automation experience using python-based tooling, for example Pytest, Selenium, Playwright or similar
* Hands-on API testing experience, including request/response validation, schema checks, and error handling
* Comfortable testing data flows: reading from a database, checking relationships, verifying output against source truth
* Ability to design clear, repeatable test cases and document expected behaviour
* Strong debugging mindset, able to isolate whether an issue sits in data, logic, model behaviour, or UI
* Good communication: able to explain quality risk to both engineers and non-engineers
Nice-to-Have
* Experience testing LLM-driven features (prompt → output validation, hallucination detection, relevance scoring)
* Familiarity with graph databases or knowledge graphs
* Exposure to observability / monitoring (logs, traces, dashboards)
* CI/CD experience, for example GitLab/GitHub Actions, running automated tests in pipeline
* Experience in fast-moving / discovery-style product teams
Our Commitment to Diversity
Diverse thoughts, backgrounds, and perspectives create stronger teams and better technology. We welcome everyone, regardless of culture, appearance, or perspective, fostering individuality. We empower our team to challenge norms, grow ideas, and produce their best work.
Seniority level
* Entry level
Employment type
* Contract
Job function
* Engineering and Information Technology
* Industries: Business Consulting and Services
#J-18808-Ljbffr