Lighthouse Update #12

Luke Anderson | Estimated read time: 5 min Image credits: Silas Baisch

Updated to v0.6.3 of the specification, progress with networking & interoperability, and more human resources.

Summary

We've made steady progress over the last few weeks with our networking stack and upgrades to the latest specification:

  • Up-to-date with spec v0.6.3.
  • RESTful HTTP API merged into eth2.0-spec.
  • Significantly improved metrics and monitoring.
  • Lots of progress on Discv5.
  • Fuzzing Lighthouse.
  • BLS standardisation and optimisation.
  • More dedicated Lighthouse resources.

Next Steps

  • Merge feature branches.
  • Meeting Eth2 implementers in Toronto and Ithaca.
  • Collaborate on networking interoperability.
  • Replace gRPC RESTful HTTP.

Achievements

We've made steady progress this month with a focus on improving developer experience.

Up-to-date and passing tests

Since our last update, we have updated Lighthouse to specification v0.6.3.

As described in our previous blog post, the v0.6.x specification has introduced a more comprehensive test suite. We are happy to announce that Lighthouse now passes all of these new test vectors (except a few which cannot be passed yet, but will be fixed in v0.7.0).

The test vectors are provided as large YAML files which are not appropriate for storage in a regular git repository. As such, those who wish to build and test Lighthouse are now required to install the git extension: git-lfs.

Defined RESTful HTTP API

Luke has finalised the REST API which will allow the validator client binary to communicate with a running beacon node. The proposal has been merged into the Eth2.0 specification repository, and is also maintained on SwaggerHub for easy viewing.

Metrics and Monitoring

Paul has implemented a range of new metrics across the Lighthouse code base, which helps to monitor the health of and interactions between nodes. These metrics are intended to be a utility for developers to measure how code changes affect operation, and are implemented using the Prometheus monitoring system.

The metrics are provided for developer use only, and as such there is no guarantee that any metric will remain the same between different software versions. In future, a more stable API for fetching operational data is likely to be provided via the REST API.

Discv5

In our previous update Adrian described Discovery v5 (discv5) and it's role in Eth2.0. We are currently testing an initial implementation of discv5 within a libp2p context. This implementation is focused around the discovery aspect of discv5 (PING/PONG and FINDNODE RPC requests) with the topic advertisement and registration development being delayed until the discovery aspect is complete and tested. We will be focusing more time on client interoperability and less on building out all the features on the network protocols over the next stretch and imagine the completion of extended features of discv5 to be delayed.

Fuzzing for bugs

As Mehdi wrote about in a blog post last year, we are implementing fuzzing techniques to detect crashes (a.k.a panics in Rust) and memory allocation errors in Lighthouse. Work had already started on this and has intensified this week, targeting primarily our networking stack and leading to the identification of a few bugs. Some of these bugs impact external crates (dependencies) included in Lighthouse. For example, libFuzzer identified a memory allocation bug in the ASN.1-DER encoder/decoder used by rust-libp2p - see the related GitHub issue for further details.

Expect to see more details and information about fuzzing Lighthouse in the next update.

BLS standardisation and optimisation

The standard for BLS is fast approaching a stable state. We're happy to announce that we now have a fully functional implementation of the standard, using Apache Milagro as the backing library, which can be found here. Included in this library is a small list of JSON test vectors which have been verified against the reference implementation.

Furthermore, Vitalik recently published an optimisation for verifying multiple signatures from a variety of public keys and messages, by reducing the number pairings that need to be done. This optimisation was implemented and benchmarked, where we recorded a reduction of approximately 40% in verification times when verifying 300 messages.

More resources

@michaelsproul was employed by Sigma Prime on a part-time basis in March to work on Lighthouse and has been a tremendous help since then. We are very excited that Michael has recently decided to join us full-time and therefore increase his excellent contributions to the Lighthouse project.

Up next

The next few weeks will see some further code clean-up, testing, and collaboration.

Merging feature branches

While we have been working on separate tasks such as networking, spec. updates, and various other fixes, there is currently a lot of unmerged code. Dubbed by some as "the great merge of 2019", consolidating all of our changes into the master branch is a major task that will happen over the next couple of weeks.

Collaboration for interoperability

In our push towards a multi-client testnet We have been collaborating with other Eth2.0 implementers to achieve interoperability. As such, Paul has been catching up with other implementers in North America at the Scaling Ethereum research workshop and the IC3 Blockchain Boot Camp. We are also in communication with other Eth2.0 teams in an effort to standardise our network communication.

Swapping gRPC for REST

With the publication of the RESTful HTTP API, we will soon be replacing the gRPC interface used for communication between the validator client and beacon node. The new REST interface should make it easier for validator clients, written in any language, to easily interact with Lighthouse. As an added bonus, this change removes Lighthouse's dependency on both gRPC and protobuf.

Updating to v0.7.0

v0.7.0 of the specification was released the same day as this update, and so the next few weeks will also see a push to get Lighthouse up-to-spec with this latest version and passing all the tests.

Give us a hand!

As always, we warmly welcome anyone who is interested to come join us in developing Ethereum 2.0. Find us in the lighthouse Gitter, or on GitHub.