Case study

A world-class QA experience

About


Athena is a quality assurance software that enables automated QA of web applications ensuring a world-class QA experience. It provides an extensive test suite and a platform for in-depth test results data analysis and test execution management. For the past two years of cooperating with Samsung, Athena has been servicing samsung.com globally by monitoring domain health and supporting digital production through integration with the production workflow and user acceptance testing audits for major worldwide projects. These include Samsung flagship product launches such as the newest Galaxy S21 5G.

Being a brand with such a vast and ever-evolving digital presence, Samsung requires future-proof solutions that fulfil business requirements and guarantee exceptional user experience every time a customer visits samsung.com.

Understanding our client’s strategic goals and processes, we’ve recognised complex issues, defined areas of improvement, and challenged ourselves to build a platform that now elevates the performance of Samsung’s online marketing and supports the brand’s long-term plans.

Our approach


We’ve analysed the Samsung brand guidelines, understood best authoring and localisation practices, learnt more about SEO, content accessibility, and site performance, reviewed every CMS component available in the client’s library and built an extensive test suite that considers design differences and authoring approaches across all page types under the samsung.com domain.

Then in collaboration with key stakeholders, we’ve built a quality assurance software that is easy to use and does not require heavy integration or complicated onboarding. The whole World Service Centre based in Poland (WSC PL) that carries digital production for Samsung and comprises one hundred QA Analysts, Web Developers, and PMs were early adopters of the new Athena platform.

Using Athena as a secondary step in the total QA for every web production task, WSC PL has reduced time spent on QA and improved content quality thanks to consistent and clear test reports generated by the software. To further accelerate the QA process, we’ve integrated Athena with Jira allowing seamless test execution and delivering results in the ticket with no intervention from a QA specialist.

Shortly after, we’ve started conducting recurring audits on a large number of pages to find out possible global issues and with the help of HQ resolve those on a domain level. This was possible thanks to a test execution scheduling solution we’ve built to enable testing during the night when Athena operates under a low workload. Scheduling functionality is particularly useful during major projects linked to social media campaigns or live events when no time can be wasted.

Recently we’ve built a web application providing an Athena user with a dashboard, easy access to test data, test suite documentation, and test execution manager. Athena app reveals overall site health in every market, highlights business-critical issues, and reports every single problem in detail including live site source code reference.

Outcome


With Athena, we’re able to test 6000 web pages within 24 hours every day of the week, perform automated QA as a part of the digital production workflow with no intervention from a QA specialist, decrease time spent on quality assurance tasks, reduce content quality errors and improve user experience on samsung.com by promptly addressing issues reported by the Athena.

We are sure that tests are performed according to test scenarios and to rely on Athena as a source of credible test results. Athena’s consistency and speed bring much-needed clarity during the realisation of complex digital projects.

We’ve analysed content quality error levels of the three global scale projects – the launch of Samsung Galaxy Note20, Samsung Galaxy S21|S21+, and Samsung Galaxy S21 Ultra. Each of the projects has had identical scope, process, number of resources, and a very tight deadline of two weeks. By joining manual QA and Athena, results of the user acceptance testing have shown that the number of content quality errors is dramatically lower compared to results that would have been achieved by performing manual testing only.

Close Bitnami banner
Bitnami