About the role
Responsibilities
- Design, develop, and execute detailed test plans, test cases, and test scenarios for a life-critical voice AI platform: automated welfare and emergency-adjacent calling where an undetected defect can mean a vulnerable person is not reached. Testing here is a safety function, not a formality.
- Perform manual testing (functional, regression, exploratory, usability) and contribute to automated testing across web dashboards, REST and gRPC APIs, and cloud-hosted services.
- Perform regression, smoke, sanity, and end-to-end testing across releases on both staging and production.
- Validate AI-driven behaviour end to end: speech-to-text transcription accuracy against recorded audio fixtures, LLM agent responses and prompt regression, tool and action invocation correctness, turn-taking and barge-in behaviour, answering-machine detection, text-to-speech output, and call-outcome extraction. This means testing non-deterministic systems: designing tolerance-based assertions and fixture-driven suites rather than exact-match checks, and running against recorded audio and transcripts instead of live vendor calls.
- Conduct API testing across REST and gRPC surfaces using Postman, Bruno, or equivalent, and build contract tests derived from API and schema definitions so that a contract change is caught by a failing test rather than by a customer.
- Own security testing as part of the same role, not as a separate track: write and run test cases against the OWASP API Top 10 (broken object-level authorization, broken authentication, excessive data exposure, mass assignment, missing rate limiting), and triage the automated security gates in CI (dependency vulnerability scanning, container image scanning, secret scanning, license checks).
- Test multi-tenant isolation and access control as a first-class scenario: verify every tenant-scoped path enforces organisation isolation and role and permission gates, and that no endpoint, filter, or export can return another tenant's data.
- Validate data privacy controls: PII redaction in call recordings and transcripts, data subject access and erasure requests, retention windows, and consent, opt-out, and do-not-call handling across voice and SMS.
- Collaborate closely with Developers, Product, and AI Engineers to understand requirements and define acceptance criteria, including an explicit security acceptance criterion for any feature touching tenant data, credentials, or telephony.
- Identify, reproduce, document, and track defects in Jira with evidence a developer can act on without a follow-up call: logs, call identifiers, request and response payloads, screenshots, and audio where relevant.
- Maintain an incident-to-test register so that every production incident is closed only once a test exists that would catch it again.
- Participate in Agile/Scrum ceremonies including sprint planning, stand-ups, and retrospectives.
- Contribute to and help maintain the automated suites (Playwright end-to-end, Vitest component tests, Go service tests) in GitHub Actions, keeping them fast, stable, and trusted.
- Ensure compliance with security, data privacy, and performance standards, and produce the test evidence that supports SOC 2, ISO 27001, and Australian Privacy Act and APP obligations, including verifying that audit logging is complete and tamper-evident.
- Validate performance and resilience: concurrent call capacity, campaign throughput, API latency budgets, and correct degradation when an upstream telephony or model provider is slow or unavailable.
- Create and maintain test documentation, reports, and metrics that measure product quality, testing coverage, and security posture.
- Support release validation and post-deployment monitoring: production smoke tests, watching logs, dashboards, and alerts after a deploy, and driving live issues to root cause with engineering.
Requirements
- 5-6 years of experience in Software Quality Assurance, including hands-on exposure to security testing or a demonstrated appetite to own it with senior support.
- Strong understanding of SDLC, STLC, and QA methodologies.
- Strong knowledge of manual testing methodologies, test case design techniques (boundary value, equivalence partitioning, decision tables, state transition), and the bug lifecycle.
- Working knowledge of automation testing frameworks (Playwright preferred; Cypress, Selenium, or similar acceptable).
- Experience with API testing tools (Postman, Bruno, RestAssured, or similar) and basic scripting (JavaScript/TypeScript, Python, or Go preferred).
- Practical understanding of web and API security fundamentals: authentication versus authorization, token and session handling, the OWASP Top 10 and OWASP API Top 10, secrets handling, and TLS basics.
- SQL competence for test data setup and result verification; comfortable reading a database schema and querying it directly.
- Familiarity with bug tracking and test management tools (Jira, TestRail, Zephyr, ClickUp, or equivalent).
- Familiarity with version control systems such as Git, pull request workflows, and reading a diff to scope regression risk.
- Understanding of Agile methodologies.
- Strong command of written and spoken English.
- Strong analytical, problem-solving, and communication skills, and the ability to work in a fast-paced, collaborative environment.
Nice-to-Haves
- Experience testing AI/ML-integrated systems: validating model predictions, handling non-deterministic outputs, prompt testing and regression for LLMs, and evaluation harnesses.
- Voice or telephony testing experience: SIP, Twilio or similar carriers, IVR and call flows, call state machines, or audio quality assessment.
- Hands-on experience building automation frameworks and scripting (JavaScript/TypeScript, Python, Go, or Java).
- Knowledge of CI/CD pipelines and tools such as GitHub Actions, Jenkins, or similar.
- Performance and load testing experience using k6, JMeter, LoadRunner, or similar.
- Hands-on security testing experience: Burp Suite or OWASP ZAP, dependency and container scanning (Trivy, Snyk, Dependabot), static analysis, and CVE triage and prioritisation.
- Understanding of data testing concepts (ETL validation, database testing with SQL, data integrity checks).
- Familiarity with cloud platforms such as AWS, Azure, or GCP, and with Docker-based local and CI test environments.
- Working awareness of SOC 2, ISO 27001, GDPR, or the Australian Privacy Act 1988 and the Australian Privacy Principles.
- Exposure to test data management in AI/ML environments, including synthetic and de-identified datasets.