First Lighthouse testnet, development approaches, and progress updates.
We are proud to announce that Lighthouse now produces a functional testnet!
Like many of the Eth 2.0 teams, we adopted a goal to have a working testnet by the end of March 2019. We have been working hard over several months to reach this goal, which marks a major milestone for the Lighthouse project.
In this Lighthouse update we describe our approach to development so far and identify how we have been focusing our time. We then describe the exact nature of the testnet that we have achieved, with an overview of each component, followed by a quick development update.
Teams behind various Serenity clients have adopted a range of approaches to development. When undertaking such an enormous software project, it is necessary to build a "scaffold" of code within which intended features are built.
Sometimes, gaps in the scaffolding are filled with temporary stopgap solutions, with the intention of replacing them later. The advantage to this approach is a shorter turnaround to achieving a running executable that can be tested in the wild, expediting the release of usable public betas. The disadvantages of this approach is that components must be (re)built more than once, potentially leading to significant code restructures down the line.
An alternative approach is to carefully complete as components within the scaffolding with code that represents the final state as closely as possible. While this approach reduces the amount of code-rewriting, it naturally takes longer to achieve a working prototype.
There is no correct way to approach this, both approaches are perfectly valid and all projects are a blend of these approaches.
Our intention is to lean toward the latter approach as much as possible, developing as many of the internal
components as we can, with the aim of producing most feature complete code possible at the time. On one hand this has
caused us difficulties with the constantly changing spec requiring constant updating, however on the other hand, it
has pushed us to produce a more complete networking stack with our extensions to
My First Testnet
Over the last month, we have been heads-down working on Lighthouse as part of a big push towards having a testnet ready by the end of March. Our hard work has paid off and we are proud to announce that we now have an operational single-client, private, short-lived testnet. (For a description of what this means, please refer to our Update #07 blog, where we set the expectations regarding Serenity testnets in March.)
Below, we provide further details on what each component of Lighthouse does, and its role within the testnet.
Presently, the main binary in the lighthouse project is the
beacon_node. Upon starting, it will use
to listen on TCP port
9000 for incoming P2P connections, and if a bootnode is specified it will attempt to
establish a direct connection with it. Discovery is not yet implemented, so it will not look for additional peers
and instead immediately starts syncing a blockchain via the
Paul has invested a huge effort getting the beacon node ready and has ensured that it is up-to-date with the Eth2.0 spec version: v0.5.0. While we have a patch to assist with the bugs that were fixed in v0.5.1, we are yet to integrate the correct solution as described by the spec.
Blocks discovered from network peers, or signed by the validator client, are processed by the beacon node and its state is updated. Presently, the state transition function must re-compute and re-merkleise the entire state for every slot, which is highly inefficient. Paul is currently working on an implementation of tree hash caching, which will drastically reduce the amount of computation required for each slot, making everything a lot faster.
beacon_node is started with the
--rpc flag, then it will also listen on localhost port
5051 for incoming
connections from Lighthouse validator clients. This internal RPC enables the
validator_client to request
data pertaining to blocks and attestations, sign them, and subsequently publish them.
validator_client binary runs as a stand-alone process, communicating with a running instance of
The validator is responsible for signing blocks and attestations during the appropriate slot; we refer to these actions as
"duties". The validator is informed of its duties for the current epoch by performing calls each slot to the
GetValidatorDuties endpoint on the running beacon node. This ensures the
validator client knows when to sign/produce blocks/attestations at any time,
given the validators it holds a private key for.
Upon startup, the validator begins by establishing a connection to a running instance of
beacon_node, and uses the
Info RPC call to check that it is
connected to a beacon node with an appropriate configuration (i.e. a sensible
The validator will then request its duties for the current epoch, and keep track of the slot in which it must perform each duty. At the appropriate time, the validator will produce a signed block or attestation, as the duty requires.
At present, the keys used by the validator in this first iteration of the testnet are pre-configured and generated deterministically. In future, they will be generated securely by a separate binary and loaded from disk.
Adrian has been working hard expanding upon the existing
rust-libp2p library, implementing the
and our own custom implementation of the phase0 wire protocol.
Our testnet demonstrates the following protocols in
- A p2p network routing protocol, to be used in Eth2.0
- phase0 RPC
- Custom RPC for Eth2.0 node inter-communication.
We have published a video on YouTube, where Adrian demonstrates:
- Starting the
--rpcflag (so that the validator can connect)
libp2plistens on localhost port
9000with a new
- The node subscribes to the
- The RPC listens on localhost port
- Slots are generated
- Starting the
- Connects to
- Obtains duties for the loaded set of validators
- Continues to produces blocks and attestations for the current slot
- Connects to
beacon_nodeis started, but with
- Connects to first
- Receives notification of new block over
- Retrieves block data from first
- Continues to sync blocks from the other
- Connects to first
We are very excited to have the talented Michael Sproul join the Sigma Prime team. His contributions so far have been excellent, and we are confident that his background in formal verification and experience with Rust will help shape Lighthouse into world-class software.
Our development sprint on Lighthouse this month saw our recent recruits Kirk and Michael leap over any learning curve before them, identify bugs, and push code.
Adrian and Paul have led the pack this month, driving everything forward with networking, syncing, spec updates, and validator client development.
Where the previous month has been a big push towards the March testnet milestone, with many late nights and most weekends spent on Lighthouse (props to Adrian and Paul), we will now be returning to a more sustainable Lighthouse development cycle.
We expect Lighthouse development to temporarily slow down in April while we engage with other Eth 2.0 developers and researchers, who will be in Sydney for EDCON. Paul and Adrian will also be presenting on Lighthouse at EDCON, so be sure to catch them on stage if you can make it! The Easter holidays are also in April, so some members of the team will be taking a much needed break.
While development may slow this month, it will not stop. We expect to release the tree hash caching implementation, alongside all kinds of improvements and bug fixes, as we optimise and stabilise our testnet.
The next major milestone for us will be achieving interoperability with other Serenity clients, transforming our single-client testnet into a multi-client testnet.
With all the development going on, it may have been more difficult than usual for external contributors to keep up. Now that we're back to business-as-usual, Lighthouse code should change less frequently, and we will dedicate ourselves to on boarding anyone who would like to contribute. If you would like to to get involved, please find a "good-first-issue" and leave a comment, or drop us a line on the gitter channel.
This is not an April fools joke!