By Dan LeBeau
Coming from a traditional software testing background, the thought of testing a blockchain solution can be quite daunting. Unlike testing a conventional database or common web application, the blockchain presents engineering teams with unique testing challenges that lie at the core of the technology we are building. Validating the quality of the Symbiont Assembly Platform requires skilled and creative engineering teams. The Quality Assurance (QA) process at Symbiont is a true team effort and does not begin or end with our QA team. The personalities, talents, skill sets and unique backgrounds of each professional are carefully considered when building our team. We select people who we believe will thrive in a culture where the experience and perspectives that each of us brings to the table are highly valued and respected.
Our “secret” to a successful quality program and end product, comes from our people. We value working within a culture that fosters an honest but respectful dialogue where the best ideas and plans are implemented with the consensus and consideration of all involved. With an outstanding team in place, anything is possible, and that is a core principle of how we operate at Symbiont, from the founders of our company to the newest members of our team.
Close collaboration at every step of our Software Development Life Cycle (SDLC) between all stakeholders is fundamental to delivering a quality end-product to our customers. The QA, product, software, infrastructure, operations, and security engineering teams work together towards the efficient and high-quality deployment of our entire Assembly Platform. At Symbiont, the QA team has an embedded partnership with the engineering teams where mutual trust is at the heart of our process. This means we succeed (or fail) together. The entire team shares a commitment to upholding individual ownership and accountability, which is one of the primary factors that ensures our consistent and successful delivery of Symbiont’s software.
HOW WE CONDUCT QUALITY ASSURANCE AT SYMBIONT
To divide and conquer the larger task before us of testing a blockchain platform like Symbiont Assembly™, our teams focus on ensuring that the functionality, core components and properties of our platform are first clearly understood by the test engineers. These components of our platform are validated separately, and also in an integrated environment using automated tests during our continuous integration (CI) build process. We complement our automation with manual tests wherever a human touch is needed. Specific components and functionalities we test include:
Network - The network or consensus layer of a blockchain is critical to a functioning system and requires a targeted set of tests to ensure that the network remains consistent, resilient, secure and behaves predictably under any situation. Without a solid network layer, a blockchain network is not operational, so a large focus of our test effort is ensuring the quality of Symbiont’s Byzantine-fault tolerant (BFT) network implementation. We implement adversarial testing and use tools to simulate many failure scenarios that are regularly run against our network layer.
Transaction Processing - We also focus a great deal of effort on testing the processes our engineering teams develop which validate and act upon the various inputs to our system. The transaction engine of Symbiont Assembly™ is designed for 100 percent network consistency, high performance and a rich feature set that the users of our platform desire. The testing of this layer needs to be exhaustive and could easily require an infinite set of input data for validation. Engineers must consider how to focus testing in order to provide the best possible coverage of the above requirements, while guaranteeing that transaction processing always provides a deterministic result and that the implementation strictly aligns with our design.
Privacy and Security - A novel attribute of the Symbiont Assembly™ platform that differentiates us from our competitors is our respect for a strict privacy model. This model allows parties on our network to conduct confidential transactions where the details of such transactions are encrypted in a truly peer-to-peer scheme and with a full audit trail. Along with our privacy model, we also need to ensure the security of our network to adversarial attacks. Strong cryptography is at the core of our privacy and security models. Extensive adversarial tests that focus on ensuring the confidentiality and security of all transactions on our network must be created and an engineering and test team is devoted to validating this feature of our system.
Infrastructure - Our software can be deployed to various cloud and on-premises infrastructures. AWS, Azure, and on-premise Kubernetes hosted solutions are possible deployment targets. These deployments of Assembly™ need to be fully tested using the current versions of cloud services available to our customers. We spend a significant amount of effort guaranteeing that the processes we build to deploy a node, or to join up nodes to create a network, will work smoothly on all supported infrastructures.
Administration Interface - Assembly™ software is deployed and managed through a command-line interface. This is an essential component of our software stack which also requires extensive testing. Commands to bring up a node or control network membership, to name a few, must be tested for function as well as for the user experience. Automated tests in our CI pipeline can perform a large portion of this testing, but manual testing is always an invaluable skill that we rely upon to verify that users of our platform will have a smooth experience with our Symbiont command line interface (CLI).
Documentation - Software documentation testing is the non-functional validation of the customer facing documentation we create to support users of our platform. QA, product and engineering teams write and validate the customer-facing documentation to be complete, correct and very user-friendly. While it may be easy to overlook thorough testing of documentation, it is a critical step in our overall release process. We treat documentation bugs as seriously as software bugs in our platform. Our goal is to provide first-rate product documentation with each release, and make it available on our customer portal.
BEYOND CORE PLATFORM TESTING
At Symbiont, we do more than just build a best-in-class permissioned enterprise blockchain platform. We provide solutions to real-world problems by leveraging our core platform using smart contracts. We provide value by removing inefficient processes and reducing the unnecessary overhead and expense inherent in many current solutions with smart contracts. With these goals in mind, the scope of our platform testing increases substantially due to the need to test the diverse applications integrated on our blockchain.
Once our platform software is approved for Beta release, the QA, engineering and business teams test the entire platform with the smart contract business logic deployed as an integrated solution. We perform exhaustive regression and end-to-end test coverage by leveraging domain-knowledge experts who test these smart contracts and processes in each of our business lines where “the rubber meets the road”. This phase of testing is where our entire platform is validated from the perspective of an end-user before we release it.
Although there are many details to our testing strategy not covered in this post, it should serve to provide a high-level description of our comprehensive approach to testing the Symbiont Assembly™ platform.
The high-quality of our software is not wholly due to a talented QA team (which we have) or to a group of really smart software engineers (which we also have plenty of), rather it’s a team process. Successful execution of this process is only possible with a dedicated and talented team who can work together in a culture that promotes open collaboration, while always being helpful towards each other at every level of our organization.