Etheroscope: A Smart Contract Visualiser
Guest post by Imperial College Industry Project Teams with Alice and Microsoft Azure Services.
The team
We are a group of undergraduates, studying Computing at Imperial College London.
● Joshua Cooper - www.linkedin.com/in/josh-cooper
● Finn Johnson - www.linkedin.com/in/finn-johnson
● G Suraj - https://g-suraj.github.io/ - https://www.linkedin.com/in/g-suraj/
● William Woodacre - www.linkedin.com/in/william-woodacre
● Elliot Thomas - https://www.linkedin.com/in/elliot-thomas-29a1b6147/
The Task
Etheroscope was produced in cooperation with Alice, as an open source, general-purpose smart-contract explorer, partly funded by a grant provided by Innovate UK. In the short term, the Alice team will continue to work on Etheroscope, with the aim of adapting it specifically to their needs, particularly within the context of the new impact management platform that they are building to complement the conditional donation application they released last year.
Website: Etheroscope
Github: https://github.com/alice-si/etheroscope
“Donations have long been stagnant in the West” - Alice
Etherscope
Etheroscope is a general purpose, open-source tool to visualise historical data stored by smart contracts on the Ethereum blockchain in partnership with Alice. This is how and why we did it.
Charities & Smart Contracts
Many charitable causes in the world today unfortunately lack the funding needed to achieve their goals. Much of this is due to potential donors being concerned about how their money will be spent. A government survey showed that the UK public’s confidence in charities fell to its lowest ever recorded level in 2016 [1] . A separate study also showed that people were willing to give an extra £665m to charities per year in the UK alone if they understood how their money was used [2] . Clearly, it’s crucial for the charity sector to tackle this issue of transparency.
Alice is a social impact startup aiming to revolutionise the current social funding model by using blockchain technology. For the uninitiated, blockchains are “open, distributed ledgers that can record transactions between two parties efficiently, in a verifiable and permanent way”[3]. Essentially, information on the blockchain can be viewed by anyone, and is also immutable. Based on the Ethereum blockchain, Alice leverages the power of “smart contracts”. A smart contract is a way of encoding the conditions for automatic transfers of value between accounts on the blockchain, while retaining the same level of transparency and immutability.
Alice uses smart contracts to make conditional donations to charities, based on whether they are achieving their goals, and allows everyone to track their progress. As an example, Alice has recently completed its first pilot project with St. Mungos, a charity that aims to help the homeless on the streets of London. By appealing for donations on Alice, St Mungo’s was only paid when they actually achieved their goals, such as helping someone find a new home, or deal with mental health issues. The goals were independently verified by the housing team at the Greater London Authority.
This gives donors the confidence that the charity is using their money towards reaching its goals, as money is only paid out when they have proved that they have done what they set out to do. Furthermore, this gives charities the flexibility and freedom to use the funding as they see best.
At the technical level, Alice encodes the donation process as a smart contract that runs on the Ethereum Virtual Machine (EVM). Donations are made in fiat currency, which are then tokenised and placed in cold storage. Then, a separate contract encodes how the progress of charities is verified: each time a charity achieves one of its goals, it must be validated by an independent third party. When the goals are validated, the contract triggers the payout from cold storage to the charity.
This figure demonstrates how Alice handle donations on its platform
The Problem?
So what exactly is the problem that we are trying to solve with Etheroscope?
How interacting with the Blockchain currently looks like
Alice’s goal is transparency, and because it is built on the Ethereum blockchain, the data from a charity’s project is technically open. The problem, however, is that this does not necessarily mean the data is readily accessible to donors. A potential donor would have to interface with transactions on the blockchain directly, which is confusing at best, even for those familiar with blockchain technology.
This is where Etheroscope comes in.
We developed Etheroscope to be a web application that visualises historical data stored by a smart contract on the Ethereum blockchain. From Alice’s perspective, this means that anyone, anywhere, can use it to quickly assess the funding and progress of the causes that matter most to them. Let’s say that you’re donating to the St. Mungos pilot that has a goal of helping to house 15 homeless people. Before donating, Etheroscope gives you the power to check if the charity has managed to help any homeless so far, or see how many other donations there have been made towards the project.
Although we developed Etheroscope with Alice’s problem in mind, it is a general smart contract viewer and could be used for any Ethereum smart contract. A user could track the progress of a crowd sale token, check how your own smart contract is doing or even just look at the most popular contracts on the network. We see it as a complementary tool to transaction data tracking services like Etherscan.io. We believe that Etheroscope’s possibilities are only limited by the creativity of those developing on the Ethereum blockchain.
Developing The Solution
The first thing we needed to do was get the proof of concept that Alice had provided us with up and running. Arguably the biggest initial problem we faced was the reliance on Parity - an Ethereum blockchain client - and the fact that we needed to wait for Parity to archive the entire blockchain. Waiting for Parity to process the entire blockchain takes considerable time (several days) - and a modest amount of space - a couple hundred gigabytes. For this reason it was important for us to deploy our application somewhere reliable, available and accessible.
Using our Azure sponsorship, we were able to quickly spin up a virtual machine to host our project on. We used this for our deployment server, as well as for testing. Space requirements that grew over time due to the size of the Ethereum blockchain were easy to accommodate given how simple it was to increase Azure’s VM space offering.
One key requirement from Alice was to move their existing database solution, a LevelDB database running on their web server, to a relational database. This was needed to provide more powerful queries over our dataset, and to help improve scalability. After some contemplation and discussion with academics at Imperial College London’s Centre for Cryptocurrency Research and Engineering, we opted to use the Microsoft SQL Server, for performance reasons. Using Azure, we could quickly start up an SQL server external to our VM and start using it straight away. Down the line we had the option both to scale up by simply increasing the databases IOPS and storage, and scale out with multiple web servers connecting to the same database.
User Experience
Etheroscope was designed primarily with the donor in mind. We wanted Etheroscope to be accessible for a user with little experience or knowledge of the blockchain, and potentially even use this project as an opportunity to educate users on how Ethereum works. As such, things like responsiveness and user experience were critical to Etheroscope’s implementation.
The main example of this is the design of our explorer page. Etheroscope tries to guide a potential donor or user throughout the entire process. When you start viewing a contract, it shows the variables by name. While developing Etheroscope, we noticed that lots of contracts had the same variable names. This is because there are some common types of smart contract, for instance, ERC20 Standard tokens. To make it easier for first time users, we added descriptions for these standard variables as well as their units allowing them to get a better understanding of what the data means.
Once a user clicks on a variable, Etheroscope loads a graph that shows the evolution of that variable over time. Etheroscope also gives the user more information on what is being shown, options to filter the data and different ways to display it. If the contract has been viewed before, we cache the data points in our database so they load instantaneously. If not, the back-end has to scan over all the blocks on the Ethereum blockchain to gather the data points.
This is the most computationally demanding part of Etheroscope. While this process is relatively fast, some contracts can take a few minutes to fully load. To keep Etheroscope responsive, the front-end maintains a socket connection to the back-end so that it can load data points immediately and update graphs in real time.
Another important aspect of the user experience was enabling users to search for contracts intuitively. Initially, we just had a text field where users could enter contract hashes. However, this was not very easy to use and tested poorly amongst users. As a result, Etheroscope provides users with a smart search bar. This search bar performs fuzzy searches with contract names as registered on etherscan.io, while still allowing users to search for contracts by hash if needed. Etheroscope also provides advanced querying features for power users, who might want to add more complex constraints to their searches.
Scanning The Blockchain
At the time of writing, the Ethereum blockchain consists of over 4.9 million blocks. Asking Parity for all these points in one go would take a very long time. In order to provide a responsive experience on our WebApp, we break down the blockchain into “chunks”. Our application then finds all the points in one chunk, and sends those on to the front-end. Starting from the most recent blocks on the chain, we then sequentially process the chain chunk-by-chunk, sending the data points to the front-end in real time. The figures below detail the progression of this process.
How interacting with the Blockchain currently looks like
Obviously, as the scan progresses, additional blocks might be added to the Ethereum blockchain. In this case, Etheroscope scans forwards to discover data in the new blocks, before resuming scans of older chunks.
New block in the Ethereum blockchain
Progressing through newer blocks
Generating Live Data
The front-end connects to a socket.io room, which immediately returns all the points found so far. The act of connecting also subscribes the front-end to receive any further data points as they are discovered. As a side note, multiple users looking at the same contract will be subscribed to the same room, so that they will all receive the points generated by the one caching process.
When we were developing Etheroscope we noticed that the web server would become very slow or even unresponsive when a very intensive contract, or many contracts were being cached at the same time. This was because all of the caching work was being done on the same process as the web server and so taking its resources. To improve the scalability of Etheroscope, we opted to make the data point generation into its own independent microservice. Thus we can now run the webserver in complete isolation of the data point generation process, ensuring they are not competing for resources. Using Azure, we were able to scale up instantly and easily using this new model, removing this issue entirely.
The Future
Temporal Tables
One of the suggestions given to us by Alice, and a possible future extension of Etheroscope would be to use a temporal database. We had considered this, but as it turned out, our database did not contain much data of temporal nature - in fact, the data points representing the values of variables at given points in time were all there was. This fact limited the usefulness of using a temporal database. Conveniently, however, we discovered that SQL Server supports “temporal tables”, which we may use in future to provide some more advanced queries over the data points.
Scaling Out
We made sure to structure the project in such a way that enables scalable deployment of our application. The process that discovers the points on the blockchain acts independently from our web server. And indeed, we can simply run many instances of this process, and distribute the work over multiple servers.
Alice
In the long term, Alice aims to create a unified platform that provides transparency to all kinds of social funding, be it donations, grants or impact investing. Etheroscope will help track very granular data about each project on Alice, making it easy for funders to identify the most effective ones, and help scale them until they eradicate the social and environmental problems they’re tackling.
References
https://www.theguardian.com/voluntary-sector-network/2013/mar/14/charities-missing-donations-survey