Assessing Blockchain Potential in Clinical Research Project Management through the Colony dApp
Northwestern University’s Blockchain Group
Rushmin Khazanchi,
Sung Lee, Tony Luo
Abstract
This report contains an analysis of the Colony decentralized application (dApp) in the context of clinical research. Detailed use case simulations, cost breakdowns, and competitive analyses were used to evaluate Colony’s potential. The findings provide significant insight into the strengths and weaknesses of the Colony platform, as well as the challenges of blockchain implementation in the clinical research space as a whole.
Introduction
Prerequisites
For this article, basic familiarity with blockchain concepts is required.¹ See the terms sheet in the Appendix for reference. It may also be useful to have a high level understanding of the Ethereum blockchain protocol.²
Acknowledgements
We would like to thank our project mentor, Abigail Sirus, for her assistance throughout the project as well as the research expert we interviewed, Tobias Hoppe, for providing subject matter expertise that was integral to our analysis. Special thanks to Jasper Ng for his early contributions to this project.
Purpose
Blockchain technology has long been touted as a revolutionary agent for the global marketplace. However, deriving a true understanding of blockchain’s value requires exploring its capabilities in regards to a specific industry. Our research group identified clinical research as a promising subfield due to (1) the complex set of stakeholders along both the patient and provider journeys and (2) the density and complexity of data as well as privacy regulations (e.g. GDPR, HIPAA). On a more practical note, the knowledge base of our IBM mentorship aligned well with clinical/pharmaceutical applications.
A core means by which blockchain interfaces with industry is through decentralized applications (dApps). dApps are built on blockchains and can leverage the core tenets of the technology in ways that are distinct from the highly publicized financial applications.
Our team limited the scope of our search to dApp’s built on Ethereum due to the significant developer activity on the protocol.³ Our team identified the Colony dApp as a promising candidate for deeper exploration. Colony’s domain-based compartments and reputation subsystems seemed to hold significant potential for streamlining clinical workflows and robustly assessing performance among researchers and employees.
Over the course of our analysis, our team was able to gain a deeper appreciation of the capabilities, challenges, and ecosystems behind blockchain technology. We hope that this article will serve as an educational resource for understanding the potential of a specific dApp and how blockchain technology can be applied in a practical, industry-focused example across the phases of ideation to implementation.
Colony Background
The Colony dApp (“the dApp”) is a series of smart contracts built on top of the Ethereum blockchain protocol. A beta version of the dApp is currently live on the Ethereum network. The dApp allows users to create and contribute to decentralized organizations called colonies. There is one main colony in the Colony Network, the MetaColony, that handles development improvements, reputation calculations, network fees, setting up new colonies, and other general network maintenance that applies for every other colony. The MetaColony is currently administered by the Colony dApp team and operates like an open source project with involvement from the community to run the Colony Network. All other colonies in the Colony Network are sub-colonies of the MetaColony as shown in Figure 1.
A colony contains one or multiple domains, which can represent different teams, projects, departments, etc. The root domain is administratively-focused and allows colonies to set user permissions. The work within a colony is organized into tasks. A manager, worker, and evaluator are associated with every task. A task is made up of a title, a description, a due date, a skill needed to complete the task, and the associated reward (bounty) for completing the task. Bounties can be in the form of Ethereum (ETH), DAI (a less price-volatile cryptocurrency), or the ERC-20 tokens specific to a colony (native tokens). Groups of tasks can be organized into Programs. A Program is a series of tasks that are broken into levels, and there is an established order in which tasks have to be completed. When users complete all the tasks from a level in a program, they will earn a badge and can unlock the next level similar to a video game.
In order to keep track of individual contributions to the colony over time, each user is associated with a reputation, much like in the real world. Whenever users complete a task, their reputations within a particular domain and skill will increase accordingly. In addition, users can provide contributions and ideas from a community-level through Suggestions that can be upvoted by any member in the colony.
The decentralized nature of the dApp allows Colony to be a dynamic, fluid project management structure for organizations that tracks specific contributions from employees/teams. Figure 2 shows the colony structure of a software project. The built-in Colony reputation system, as shown in Figure 3, will aggregate a single user’s reputation scores across all the subdomains they are active in so that the organization can get a holistic understanding of what the employees have contributed and then reward them, or focus on potential improvements, based on their aggregate reputations.
To gain a deeper understanding of the practical applications of the Colony dApp in industry, our team explored a clinical lab management use case.
Use Case Exploration: Clinical Lab Management on Colony
Selecting a Use Case
Our team came into this project looking for a way to integrate Colony into an existing organization to explore how blockchain tools may be adopted in the clinical research space. Our team identified that clinical data management has complex, regulatory-compliant processes that could benefit from traceability of contributions. After learning more about clinical data management, our team broadened the use case to clinical lab management for clinical research organizations because Colony in practice is a project management tool.
Blockchain Potential in Clinical Lab Management
Our team identified the following aspects of clinical lab management that could be improved with the use of blockchain tools.
Internal Factors
Internally, changes in policy, data entry issues, and changes in staffing were cited as key drivers of inefficiency.⁶ Due to these problems, submission of institutional review board approval to begin a clinical study is delayed. Blockchain could potentially improve this issue by maintaining an immutable record of the clinical data. Additionally, our team identified the potential benefit of fluid staffing, enabled by a colony’s task transparency, for reducing internal time delays.⁷
Additional Factors
Other issues are related to the scalability of projects. For example, coordinating work execution at scale across multiple work-sites was cited as a key issue.⁸ Also, it is difficult for researchers to maintain regulatory compliance.⁹ Additionally, financial transparency is important when it comes to allowed versus billed charges.
With regard to all of these factors, blockchain, if designed and deployed appropriately, could provide a measure of trust that could considerably streamline operations.
Colony Implementation
Following our team’s identification of blockchain’s potential in clinical lab management, our hypothesis was that Colony could be a viable tool to supplement current clinical lab management operations and provide additional value.
After exploring the live beta’s features and functionality (conducted in June 2020), our team designed the following Colony implementation for clinical lab management.
For the implementation, a research organization would have one colony with each lab having its own domain. Within these domains, project managers would assign tasks, with corresponding bounties, to employees. Project managers or authorized project section leaders would evaluate the work, and staff would use Suggestions to give their input. To interact with Colony, every clinical staff member would have their own ETH wallet with enough ETH to pay for transactions, like setting up a Colony account or evaluating completed tasks. A research organization would use programs for appropriate staff training and reputation/collected bounties to track individual contributions. Also, the root domain would be used for tasks associated with all clinical research projects.
As seen in the red box of Figure 4, each lab is assigned a domain; this domain, in particular, falls under Lab 2 as indicated in the purple box. Lab managers can post and assign tasks shown in the green box. Organization members with particular skills can complete bounties tied to critical lab-related tasks across domains.
Figure 5 shows a sample Task. The black box contains the employee assigned to the task with the associated bounty, the blue box contains the title and description, and the yellow box shows additional information about the task. On the right side of the figure, there is a log for changes to the task and chat functionality.
Value-Adds and Thought Process
Based on the proposed implementation of Colony, we identified the following potential value-adds, shown in Figure 6, for lab management.
Auditability
Colony allows for an immutable audit trail detailing who defined tasks, who did the work, and who evaluated the work done. This allows organizations to stay organized, keep staff accountable, and give recognition to value contributors in clinical projects.
Our team sought to take advantage of blockchain’s auditability, so this was the first value add we explored.
Performance Metrics
The native tokens for the colony would be used to measure the value that someone contributed to the organization. Individuals collect native tokens by working on tasks, and tasks can have different payouts depending on the task’s importance. The reputation gained for doing tasks gives a time-dependent indicator of the value that individuals provide to the organization. These metrics would be used for performance reviews and for improving organizational efficiency.
Our team struggled to find a way to make use of Colony’s tokenization feature. We did not want the native tokens to have monetary value due to the traditional payroll system and tokens generally being considered a security. We wanted to use these tokens as a utility.
Incentivized Collaboration
The record of contribution and associated performance evaluations could allow individuals to become more open to working on tasks from other projects.
We made the assumption that in the current landscape, individuals are not incentivized to collaborate on tasks outside of their main responsibilities when there is no structure for evaluating their efforts. We believe that without a better guarantee of their efforts being noticed, staff would be hesitant to work on other tasks even if it was to the benefit of the organization.
Flexible Staffing
Employees can view tasks for other work groups and may request or be assigned to work on tasks outside of their typical responsibilities. This open-endedness enables greater collaboration between labs and can reduce time delays by shifting human resources when there are time-sensitive, costly tasks that need to be addressed, leading to the consumption of fewer resources.
Our team hypothesized that organizations would value flexible staffing to improve the time delays that may occur in different stages of clinical research projects.
Trainings/Certificates
Colony programs consist of sequential tasks with different levels of achievement. Once an individual completes a program, they receive an indicator that shows their completion. These programs could be used for training staff or giving certificates. An example is using a Colony program to ensure that new medical coders have basic competency with their software through the use of task modules. With digital identity on the rise, we believe that being able to prove and audit credentials of lab professionals in real time is a significant value-add.
In the following section, our team constructed a specific user scenario to illustrate the identified value-adds.
User Scenario
A pharmaceutical company uses Colony for clinical lab management with each project having its own domain. Recently, a project manager has unexpectedly left the company during a clinical project. Other project managers use Colony to check the status of the project, evaluate work, and assign new tasks to continue the project without significant delay. Staff members from other projects are assigned tasks on the project as needed and Colony is used for project documentation.
A new project manager is hired. They work through a Colony training program that teaches them how to use Colony and enables them to take responsibility for the ongoing project. Once completed, the research labs resume normal operation. Afterwards, the company uses their colony-specific token and the reputation feature to inform performance evaluations. The performance evaluators notice the extra output given by staff that helped the project while in its transition period.
Our team hypothesized that using Colony can allow for a more efficient leadership transition for clinical lab projects. Furthermore, we hypothesized that research organizations would be able to streamline lab management operations through Colony, and these firms may also enjoy greater organizational transparency and increased accountability.
Cost Case
All organizations need to project costs and plan their future for budgetary review. By evaluating the cost case of implementing Colony for clinical lab management, our team aimed to get insight into the scalability of operational expenses. Our team noted that the task estimates used for each clinical research phase are rough estimates as it is difficult to estimate the costs of using Colony to assign the tasks of individual clinical projects.
Our team made the following assumptions for our cost case:
- The research organization has three labs — each within their own domain — that have one project manager and 10 staff members.
- Every staff member uses the Colony dApp.
- All ETH costs from using Colony come from the costs associated around the date of publication using the Colony Beta including the average mining fees at the time.
- Costs associated with storage, broadband, and power will not be analyzed.
- The 3% cryptocurrency token fee for using the Colony Network is not included.
The following items involve spending ETH while using the Colony dApp:
Startup costs:
- Creating a colony and native token
- Individual account setup
- Adding special permissions
Operational costs:
- Creating a task
- Evaluating a task
See Appendix A for the exact ETH costs for each item. Our team considered all costs with ETH prices at $220 USD/ETH and at the maximum price ever recorded of $1.4k USD/ETH.
Our team identified a quick project and a slow project. See Appendix A for information about the timelines associated with different phases of clinical projects and how we estimated the number of tasks for each phase. The timespan of this quick project is aggressive and may be unrealistic in the context of clinical research.
Our team then considered a “small organization” and a “medium-sized organization”, where the small organization has three slow projects and the medium-sized organization has three quick projects. The following tables show the startup and operational cost of implementing Colony for both organizations.
The costs presented show that implementing Colony on a large scale may be costly and fluctuate significantly especially as the number of tasks change. Organizations must have a flexible budget and carefully consider the potential benefits versus potential cost. Our team noted that the costs estimated using the all time high’s of Ethereum likely overestimate costs due to the fluctuation in mining fees.
With the cost in mind, our research group proceeded to compare Colony to existing project management solutions.
Comparison to Status Quo
Overall, Colony’s functionalities and flexibility are rather limited compared with the tools currently used by the industry. We have visualized the drawbacks of Colony compared to its contemporaries in Figure 8. First, as a dApp in its early development phase, Colony cannot be integrated with other tools like Microsoft Project with Microsoft office suite applications and Atlassian with Slack or other in-house softwares. This means that users in Colony have to import/export data manually and cannot attach relevant files to each task easily. Second, Colony is less customizable than other project management tools. For example, program administrators in JIRA can add, remove and reorder fields and tabs in their screen to fit the way the team wants to work while in Colony the task structure and the dashboard cannot be changed.
Third, Colony fails to visualize the work progress and display any analytical data of its user activity. There is no visualization or ranking of users’ reputations in Colony so it is difficult for a lab manager to execute a performance review. Currently, there is only a list of task names in the main dashboard instead of a clear visualization of the content of the tasks. A future implementation of Colony would also benefit from a centralized schedule with a listing of all tasks including task due date and current status. In comparison, JIRA offers real-time key performance indicator reports for internal use and Microsoft Project has a calendar with clear visualization of various tasks and their due dates. Trello also allows users to see a summary of the task and any file attached to the task in the dashboard instantly without having to click into a specific task.
Lastly, unlike other project management software that have fixed subscription costs, Colony runs on Ethereum, a relatively new cryptocurrency that is subjected to wild price fluctuations. Indeed, while setting up the parameters for our user scenario and cost case as seen in the aforementioned pages, our research dictated a multitude of assumptions, including a relatively fixed ETH and gas value. Since research firms and corporations depend on long-term cost forecasts, this highly-variable pricing will dis-incentize many parties from immediately switching to Colony.
Suggested Improvements
Our team, therefore, suggests several improvements to reduce the gap between Colony and other project management software currently used in the industry. To facilitate a more efficient onboarding to Colony, APIs should be used to import tasks from other existing project management software. Colony should build a more customizable task structure so that it can adapt to different types of tasks in a lab setting including data input, experiment, or clinical trial, etc. As the user reputation performance metric is a significant value-add from using Colony, the app should use clear visual metrics to display the ranking of users’ reputations filtered by domains. It should also support queries for all the tasks completed by one user. These improvements will give lab managers significant insights for performance reviews. Lastly, Colony could potentially adopt batched transactions of ETH to simplify the user experience and improve usability of the app.
Findings from Use Case Review with an Industry Expert
To test our hypothesis and Colony implementation, our team interviewed Tobias Hoppe, a former IBM expert consultant in healthcare and life sciences. As indicated by our interview with Mr. Hoppe, the potential value-adds from a tool like Colony are not in line with industry demands. Despite our team’s enthusiasm for blockchain-enabled solutions, the technology itself is not the ‘be-all-end-all’ answer to bottlenecks but only part of the solution. Revisiting the value-adds, our team has noted the following results:
Auditability
While Colony can provide an immutable ledger of who created, worked, and evaluated the tasks, as a reminder from Mr. Hoppe, most research organizations already have a system in place for tracking and auditing contributions. Colony excels at tracking individual contributions to clinical projects with its reputation-based system, but the operational cost involved in implementing Colony does not justify upgrading the system of documentation in research organizations today.
Performance Metric
The reputation-based and token-based framework of Colony appears viable for evaluating individual contributions, but without built-in visualizations (graphs/plots/tables) or outputs that allow executive leaders to review these performance metrics, Colony falls short as a trust-based meritocracy.
Incentivized Collaboration
Mr. Hoppe indicated that clinical projects do not lack collaboration. It is ambiguous whether or not cross-project collaboration is beneficial for research organizations. Also, it is unclear whether collaboration is actually incentivized in the workplace without further testing.
Flexible Staffing
The expert interview with Mr. Hoppe indicated that flexible staffing is one of the least pressing issues in clinical lab management. Moreover, leadership transitions are uncommon. In the event of a leadership change, constant documentation has enabled project leaders to resume work without significant interruption. Flexible staffing does not provide much value to research organizations.
Trainings/Certificates
At the time of the expert interview, our team had not prepared concrete training/certificates that could be implemented and did not receive feedback on this value-add. Interviews with project managers or human resources departments would be needed to propose specific training/certificates that are viable on Colony. The value-add’s importance remains unclear.
Conclusively, our analysis illustrates that the Colony dApp is limited by the cost volatility of Ethereum and the lack of functional improvements relative to status quo project management solutions. With that in mind, there is little incentive for clinical research organizations to implement this tool in their day-to-day workflows.
Conclusion
Colony’s Strengths, Weaknesses, and Potential
Colony seeks to build an organizational model that would replace the traditional management hierarchy with a bottom-up decision-making system to enable a functioning meritocracy. We believe that Colony’s true potential lies in anonymous contributions, open source projects and non-profit organizations due to its decentralized, self-managing nature. This evaluation naturally aligns with projects that value community input and social good, like open source health initiatives like OpenMRS¹⁰ and WorldVistA.¹¹ In both of these examples, individuals with different skill sets could supplement working teams and earn reputation. In addition, funding from non-profit projects could be distributed as bounties and thus provide further incentives for contributions.
However, as a dApp in its early development phase, Colony lacks both the flexibility and the functionality to compete with other project management software tools. For example, Colony cannot be integrated with other platforms and is less customizable than other competing tools. Moreover, Colony fails to visualize work progress and display analytical data related to user activity, and there is no visualization or ranking of users’ reputations in Colony making performance reviews difficult.
Despite its limited features, Colony is making incremental improvements and moving towards a more engaging platform. For example, two new features were introduced this April, Suggestion and Program, which are similar to the backlog in Kaplan in the Atlassian tool suite.¹³ Colony’s upcoming reputation weighted voting organizations could reduce high coordination costs associated with dispute settlement. In the future, Colony could continue its trajectory and adopt more community improvements beyond those outlined in their roadmap.¹⁴ Examples include a centralized schedule with all tasks’ due date and current status, a reputation report for internal use, and a more customizable interface.
Although Colony has price volatility, we believe that increasing features and functionality may place Colony on par with other software tools and incentivize established open source projects and non-profit organizations to incorporate Colony into daily operations.
Realities of Implementing Blockchain Tools in Clinical Research
In building out this use case, our team identified three realities to implementing blockchain tools in clinical research:
- Domain knowledge limits flexibility: In clinical lab settings, most research specialists have a nuanced understanding of the project and research pipeline. Consequently, it is difficult to delegate tasks to third-parties without sacrificing efficiency.
- Accountability and auditability suffer with off-chain tasks: Since most clinical research work involves physical access to resources, developing a consistent data entry and validation mechanism is a significant challenge. In a similar sense, constructing verification processes that are non-digital provides additional layers of challenges for a blockchain implementation.
- Clinical research best-practices evolve slowly, and greater development of a decentralized ecosystem is needed: Considering most health-related data is stored on HIPAA-compliant health record databases, specific mechanisms need to be developed to interface with the overall health ecosystem. Additionally, most project management/oversight tools in the market today interact with a suite of add-ons with varying degrees of functionality. These tools complement workstreams, and provide project managers with immense flexibility. The equivalents to these tools have not translated to decentralized applications, limiting their potential.
Blockchain Tools’ Potential in Clinical Research
Despite the limitations, blockchain tools have the potential to disrupt healthcare in the following ways:
- Auditability could be game-changing: In a world where seminal research developments grant credit to a few individuals despite having many contributors, an immutable record could highlight the contributions of all parties and provide more equitable recognition and compensation.
- Crowd-sourcing capabilities and data could be effective under particular circumstances: Projects that require large amounts of clinical data to be voluntarily contributed by participants could benefit from blockchain’s systems of trust and authenticity. Additionally, research studies that are digital in nature and require skills in data analysis, software development, and media marketing could tokenize tasks and decentralize talent.
Appendix A: Cost Case Calculations
Disclaimer: The costs in the table above all have a gas price of approximately 23 Gwei/gas and no fee was charged by the Colony Network since these costs come from the free beta.
Estimating Tasks:
Our team estimates the number of tasks by factoring the different stages of clinical research¹⁶ and the amount of time associated with each phase.¹⁷
Phase 1 trials are generally small studies with 10–15 volunteers. Traditionally, there are less staff involved at this stage, and our team estimates there being five tasks per workday.
Phase 2 trials are generally studies that focus on whether the treatment is safe with 10–30 volunteers. There would be more staff involved at this point, and our team estimates there being 10 tasks per workday.
Phase 3 trials are generally studies that focus on the effectiveness of the treatment with 25–100 patients. There would be more staff involved during this phase, and our team estimates there being 30 tasks per workday.
Phase 4 trials are generally studies that assess long-term safety and effectiveness of the treatment with more patients, generally from 300–3000+. More staff are involved, so our team estimates there being 60 tasks per workday.
Quick Project
- Total Time: 27 months
- Total Tasks: 20475
- Average Tasks per Year: 1706 tasks
Slow Project
- Total Time: 66 months
- Total Tasks: 63630
- Average Tasks per Year: 11570 tasks
Appendix B: Terms Sheet
- blockchain — a distributed, decentralized ledger
- bounty — a token reward for completing a task on Colony
- CDM — clinical data management
- Colony — an Ethereum community management decentralized application
- Colony Network — the network of colonies associated with Colony
- colony — a decentralized organization within the Colony dApp
- cryptocurrency — a digital currency built with blockchain technology
- DAI — a cryptocurrency that is pegged to assets to maintain a stable value
- dApp — a decentralized application that runs on blockchain technology
- domain — a subset of a colony
- ERC-20 token — tokens created and hosted on the Ethereum blockchain
- ETH — the Ethereum protocol’s cryptocurrency
- Ethereum — a blockchain featuring smart contract functionality
- Gas — a unit for computations on the Ethereum blockchain
- Gas price — the transaction price per computation paid to miners
- GDPR — General Data Protection Regulation
- Gwei — 10E-18 ETH
- HIPAA — Health Insurance Portability and Accountability Act
- native tokens — tokens that are colony-specific
- MetaColony — the colony run by the Colony dApp team that governs the protocol
- Program — a structured set of tasks in a colony with levels and badges upon completion
- smart contract — a computer protocol that runs on blockchain technology
- task — the base unit of work in colonies
- token — a cryptocurrency asset
Note: The definitions in this glossary come from the Northwestern Blockchain Group’s team knowledge.
Appendix C: About Us
Organization:
Northwestern University Blockchain Group
Our vision is to foster a healthy and active ecosystem in Northwestern and Chicago, and solidify our community on the map as a strong hub for blockchain technology. Contact us at blockchaingroup@u.northwestern.edu.
Authors:
Gilberto Guadiana
President
Rushmin Khazanchi
Enterprise Chair
Sung Lee
Marketing Chair
Tony Luo
Research Group Member
Project Mentor:
Abigail Sirus
Global Blockchain Governance Strategy and Design Lead at IBM
Expert:
Tobias Hoppe
Managing Director, Centers of Excellence at YourEncore,
Former Executive Consultant in IBM’s Healthcare and Life Science Center of Competence
Appendix D: References
¹ https://www.investopedia.com/terms/b/blockchain.asp
² https://www.investopedia.com/terms/e/ethereum.asp
³ https://consensys.net/blog/news/ethereum-by-the-numbers-october-2019/
⁴ https://colony.io/whitepaper.pdf
⁵ https://colony.io/whitepaper.pdf
⁶ https://forteresearch.com/news/speeding-study-activation-spot- bottlenecks/
⁸ https://bio-optronics.com/breaking-clinical-trial-bottlenecks/
⁹ https://scholarlycommons.law.wlu.edu/crsj/vol25/iss1/9/
¹⁰ https://wiki.openmrs.org/display/docs/Introduction+to+OpenMRS
¹¹ http://worldvista.org/WorldVistA
¹³ https://www.atlassian.com/agile/kanban/kanplan
¹⁴ https://colony.io/product/app/
¹⁵ https://github.com/JoinColony/colonyNetwork/graphs/contributors
¹⁶ https://www.profil.com/knowledge-center/trial-stages
¹⁷ https://www.antidote.me/blog/how-long-do-clinical-trial-phases-take
